| *** dmellado170420 is now known as dmellado17042 | 05:05 | |
| opendevreview | Jakub Skunda proposed openstack/nova master: Drop Fedora support https://review.opendev.org/c/openstack/nova/+/886596 | 10:11 |
|---|---|---|
| opendevreview | Amit Uniyal proposed openstack/nova-specs master: Adds cleanup to remove dangling volumes https://review.opendev.org/c/openstack/nova-specs/+/878757 | 11:37 |
| noonedeadpunk | hey folks, I've found this wiki page lately, and it looks like this should be supported by nova, as the `migrate_set_capability` is a command that's passed to the qemu monitor https://wiki.qemu.org/Features/Migration-Multiple-fds | 11:47 |
| noonedeadpunk | So I was wondering, if that's smth that should be supported by nova, or I can set it in some other way as well? | 11:48 |
| noonedeadpunk | And also according to https://bugzilla.redhat.com/show_bug.cgi?id=1968540 it seems there's another way around in progress now. But maybe it's worth mentioning in docs as a note about performance hit by using TLS for migrations? | 12:02 |
| *** d34dh0r5- is now known as d34dh0r53 | 12:12 | |
| *** mgoddard- is now known as mgoddard | 12:28 | |
| bauzas | noonedeadpunk: sorry was barely paying attention to the chan | 13:43 |
| bauzas | noonedeadpunk: so, basically you have concerns about migration performance ? | 13:46 |
| noonedeadpunk | well, yes, as they can hardly finish with 1.2gbitp/s | 13:48 |
| noonedeadpunk | It's slightly different on different hardware, but basically it's always single thread now that's handling migration/encryption | 13:48 |
| noonedeadpunk | I've tried to check if gnutls 3.7.3 (default for ubuntu 22.04) helps (as it has this merged) https://gitlab.com/gnutls/gnutls/-/commit/6462916d2f6810409d5da1e13c4a0720f412c166 | 13:50 |
| noonedeadpunk | But seems that it's not enough to have that only in gnutls, apparently qemu should have smth as well | 13:50 |
| noonedeadpunk | But was also wondering if anybody heard anything about x-multiple-fds in qemu capabilities | 13:51 |
| bauzas | I'm not really a SME of this matter | 13:54 |
| bauzas | maybe kashyap may help | 13:54 |
| * kashyap looks | 13:55 | |
| bauzas | kashyap: thanks, tl;dr noonedeadpunk has some concerns by the migration TLS performance due to a QEMU single threading | 13:56 |
| kashyap | noonedeadpunk: Before I say anything further, please note that the "x-" prefix in QMP commands (like "x-multiple-fds") means it's experimental | 13:56 |
| kashyap | That _said_: the multiple-fd work did come out of experimental, IIRC. Lemme check | 13:56 |
| kashyap | bauzas: I see; I missed the earlier context; so would be good to know what kind of performance concerns are these | 13:56 |
| noonedeadpunk | so using TLS for libvirt gives migration speed like 1.2gbitps - 3 gbitps depending on CPU, TCP live migrations are 20gbitps | 13:57 |
| noonedeadpunk | (basically full link) | 13:57 |
| kashyap | noonedeadpunk: Hmm, I mean I'd definitely expect some loss w/ built-in TLS -- as it involves the additional encryption over head -- but I don't have numbers off-hand as to what's "expected" | 14:01 |
| kashyap | noonedeadpunk: Can you come to #virt (on OFTC) and ask there too? More libvirt migration experts hang out there | 14:02 |
| noonedeadpunk | so diffference is quite dramatical | 14:02 |
| noonedeadpunk | yes, sure | 14:02 |
| kashyap | Yeah; it's almost 10 fold diff | 14:02 |
| bauzas | noonedeadpunk: ohoh, good to know | 14:03 |
| noonedeadpunk | enabling auto-convergance kinda solves issue, at least migrations do not fail | 14:07 |
| noonedeadpunk | but being able to utilize more then 1 cpu would be better ofc... | 14:08 |
| kashyap | Ah, that's good to know that auto-convergence did its job at least | 14:08 |
| kashyap | I agree on utilization; let's check w/ the libvirt-QEMU guys | 14:08 |
| bauzas | gtk | 14:09 |
| noonedeadpunk | But I guess if that's performance diff is kinda confirmed, it might be worth mentioning in nova docs at very least | 14:14 |
| kashyap | noonedeadpunk: I think you're using an older QEMU by judging the "x-" prefix; so first would be good to upgrade to a newer one that removes the "x-" prefix (experimental) | 14:16 |
| noonedeadpunk | I've taken that from qemu wiki page :) | 14:17 |
| noonedeadpunk | I guess I was testing on qemu 8.0.0 or smth | 14:17 |
| kashyap | Yeah, the wiki-page tends to not get updated. I'm pretty sure it's now out of experimental; otherwise libvirt won't enable it (as you saw on #virt) | 14:17 |
| kashyap | Also, see this response from danpb: https://listman.redhat.com/archives/libvir-list/2020-January/msg00527.html | 14:17 |
| noonedeadpunk | yup | 14:17 |
| kashyap | (Scroll down a bit) | 14:17 |
| noonedeadpunk | Yeah, fair | 14:22 |
| noonedeadpunk | Soooo.... Should we say that it make sense to add `parallel` option support to nova? | 14:45 |
| noonedeadpunk | I would try to check what needs to be done for that... Should be pretty much trivial to add | 14:54 |
| bauzas | noonedeadpunk: sorry again, but what do you mean ? is it due to a new QEMU release ? | 14:57 |
| noonedeadpunk | bauzas: there's a parallel flag that's for sure already present in 8.0.0: https://libvirt.org/manpages/virsh.html#migrate | 14:58 |
| noonedeadpunk | let me check if it's available even before that | 14:59 |
| noonedeadpunk | It's also available in 6.0.0, and nova min libvirt version is already beyond that | 15:01 |
| noonedeadpunk | but we also should be careful with amount of parallel connections not to affect running workloads | 15:02 |
| bauzas | noonedeadpunk: sounds to me a feature request | 15:04 |
| bauzas | but yeah, turning on this know could be possible | 15:04 |
| kashyap | Yeah, it's a separate request to enable the parallel connections | 15:23 |
| opendevreview | Merged openstack/nova stable/wallaby: Remove mentions of removed scheduler filters https://review.opendev.org/c/openstack/nova/+/858050 | 16:32 |
| melwitt | stephenfin, sean-k-mooney: just fyi I updated https://review.opendev.org/c/openstack/nova/+/877056 to add the bug and release note and return early+unindent code block. and I've also rechecked the tempest test skip for the unrelated CI failures https://review.opendev.org/c/openstack/tempest/+/886496 | 17:33 |
| opendevreview | Jakub Skunda proposed openstack/nova master: Drop Fedora support https://review.opendev.org/c/openstack/nova/+/886596 | 21:59 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!