| opendevreview | Goutham Pacha Ravi proposed openstack/nova master: Support cold migration with virtiofs shares https://review.opendev.org/c/openstack/nova/+/989633 | 06:16 |
|---|---|---|
| opendevreview | Goutham Pacha Ravi proposed openstack/nova master: Allow rebuild for instances with virtiofs shares https://review.opendev.org/c/openstack/nova/+/999048 | 06:52 |
| opendevreview | Goutham Pacha Ravi proposed openstack/nova master: Allow rebuild for instances with virtiofs shares https://review.opendev.org/c/openstack/nova/+/999048 | 06:53 |
| *** zseguin is now known as Guest14350 | 07:21 | |
| opendevreview | Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299 | 09:48 |
| opendevreview | Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840 | 09:48 |
| *** ralonsoh_ is now known as ralonsoh | 11:01 | |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Make functional-py313-threading voting https://review.opendev.org/c/openstack/nova/+/997216 | 13:48 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Stabilize functional test for slow instance.save https://review.opendev.org/c/openstack/nova/+/999121 | 13:48 |
| gibi | OK now that ^^ series feels complete to stabilize the functional-py313-threading tox target | 13:49 |
| opendevreview | Joan Gilabert proposed openstack/nova master: Report OWNER_NOVA trait on resource providers https://review.opendev.org/c/openstack/nova/+/994299 | 13:50 |
| opendevreview | Joan Gilabert proposed openstack/nova master: Add libvirt driver MDEV ARQ support for Cyborg https://review.opendev.org/c/openstack/nova/+/995840 | 13:50 |
| ralonsoh | sean-k-mooney, gibi hi folks, I've seen recently a raise in the live-migrations errors, in particular in the ML2/OVS jobs | 15:05 |
| ralonsoh | I've reported https://bugs.launchpad.net/nova/+bug/2161894 | 15:05 |
| ralonsoh | can you please take a look? Because this is happening randomly. Most of the times is because the port ofport=-1 | 15:06 |
| sean-k-mooney | interesitng | 15:06 |
| sean-k-mooney | yes we can take a look | 15:06 |
| ralonsoh | with os_vif_delegation=True, that means this is created by qemu | 15:06 |
| ralonsoh | (if I'm not wrong) | 15:06 |
| sean-k-mooney | yes and no | 15:06 |
| ralonsoh | I thought initially that could be related to the tap creation flag, but that was for ML2/OVN only (at least now) | 15:07 |
| sean-k-mooney | so ml2/ovs is require to finish bidnign prot with of-port=-1 | 15:07 |
| sean-k-mooney | os_vif_delegation=True menas we use interface type ethernet | 15:07 |
| ralonsoh | yes, but who creates the TAP? | 15:08 |
| sean-k-mooney | libvirt | 15:08 |
| ralonsoh | that's the point, the TAP is created when the VM is unpaused | 15:08 |
| sean-k-mooney | if you want os-vif to create the tap then you need to tell os-vif ot do that with the new fucntioantly | 15:08 |
| ralonsoh | or at least the ofport assigned | 15:08 |
| sean-k-mooney | ralonsoh: yes | 15:09 |
| sean-k-mooney | and ml2 ovs is not allowed to rely on the ofport being set | 15:09 |
| sean-k-mooney | to send network-vif-plugged | 15:09 |
| ralonsoh | yes, OVS agent needs to see ofport different from INVALID (-1) | 15:09 |
| ralonsoh | and then Neutron API will send the vif-plugged event | 15:10 |
| sean-k-mooney | that is a neutron bug | 15:10 |
| sean-k-mooney | ti is not allowed to requrie that | 15:10 |
| sean-k-mooney | because it will break dpdk if it does | 15:10 |
| ralonsoh | this is why in dpdk we disable live_migration_wait_for_vif_plug | 15:10 |
| sean-k-mooney | no | 15:11 |
| ralonsoh | let's focus on the kernel datapath | 15:11 |
| sean-k-mooney | for dpdk we shoudl not reuire that | 15:11 |
| sean-k-mooney | ok well the ovs agent regardless of the kerle or dpdk | 15:11 |
| sean-k-mooney | is not allowed to depen on osprot being set to a non -1 value | 15:12 |
| sean-k-mooney | https://github.com/openstack/neutron/commit/7ebcce0b8dcd4cd482da92672f5ae7c51530a429 | 15:12 |
| sean-k-mooney | is likely a regression | 15:12 |
| sean-k-mooney | well maybe not | 15:12 |
| sean-k-mooney | ``network-vif-plugged`` shoudl be sent when osport is -1 | 15:13 |
| ralonsoh | we have never done this | 15:13 |
| sean-k-mooney | we never depened on the ofport out side of the openflow security group driver | 15:14 |
| sean-k-mooney | so if you use ip tbales ofport was never reuqired | 15:14 |
| sean-k-mooney | and even with the ovs firewall driver we just defered the security gorup flwo setup | 15:15 |
| ralonsoh | but regardless of the FW setup, that is done after the port receives a valid ofport | 15:15 |
| sean-k-mooney | it wasnt in the past | 15:16 |
| ralonsoh | I pushed this https://review.opendev.org/c/openstack/neutron/+/992423 1 month ago | 15:16 |
| sean-k-mooney | the tap is created befor ethe vm is unpased by the way | 15:16 |
| ralonsoh | (related to the previous patch) | 15:16 |
| sean-k-mooney | but not a lot before it | 15:16 |
| ralonsoh | yes, but the TAP is created by libvirt, and that requires the vif-plugged event | 15:17 |
| ralonsoh | I really don't know how this could work now | 15:18 |
| sean-k-mooney | no the vif plugged event does not depend on the tap exsiting in general | 15:18 |
| ralonsoh | without the TAP, the ofport=-1 | 15:18 |
| ralonsoh | right? | 15:18 |
| sean-k-mooney | yes | 15:18 |
| sean-k-mooney | and that is valid | 15:18 |
| ralonsoh | so yes, it depends | 15:18 |
| ralonsoh | until we have a valid ofport, Neutron does not send vif-plugged | 15:19 |
| sean-k-mooney | ml2/ovs need to supprot seting network-vif-plugged with ofport=-1 | 15:19 |
| sean-k-mooney | then that is a regression in neutron | 15:19 |
| ralonsoh | we never did that before | 15:19 |
| sean-k-mooney | yes we di | 15:19 |
| ralonsoh | this could not be a regression | 15:19 |
| sean-k-mooney | for a very very very long time | 15:19 |
| ralonsoh | no no, hold on | 15:20 |
| sean-k-mooney | this is how dpdk has worked for ever | 15:20 |
| ralonsoh | I really need to understand how it could work now | 15:20 |
| ralonsoh | because is working randonly, most of the times | 15:20 |
| sean-k-mooney | https://github.com/openstack/neutron/commit/7fd2725cb14b81f442eb57a38755829270ff2c43 | 15:20 |
| ralonsoh | ok, I need to review this patch too (I remember it) | 15:21 |
| ralonsoh | I think this is working by accident | 15:22 |
| opendevreview | Philipp Dreesens proposed openstack/nova-specs master: Add spec for bidirectional RPC liveness handshake https://review.opendev.org/c/openstack/nova-specs/+/984384 | 15:22 |
| ralonsoh | --> The network-vif-plugged during live migration was sent by the SOURCE agent calling update_device_up (for the existing binding that's still active on the source), NOT by the destination agent. | 15:22 |
| sean-k-mooney | that shoudl not happen becuse we set migrating_to | 15:23 |
| sean-k-mooney | in the port profiel | 15:23 |
| ralonsoh | I know, I changed this code | 15:23 |
| sean-k-mooney | and only the host named there is ment to send it with yes you updated that code | 15:24 |
| ralonsoh | sean-k-mooney, thanks a lot, I need to recheck the recent changes in the OVS agent | 15:25 |
| ralonsoh | (done by me, trying to fix other issues) | 15:26 |
| sean-k-mooney | as an aside "``os_vif_delegation=True`` and ``create_port=True``" will mena the tap is create before we add the ovs port | 15:32 |
| sean-k-mooney | so in that case we wont see -1 for kernel ovs but we still will for dpdk | 15:32 |
| sean-k-mooney | os we shoudl not have divergent code paths if we can aovid it and just alwasy supprot -1 | 15:32 |
| ralonsoh | but isn't supporting ofport=-1 something wrong? | 15:38 |
| ralonsoh | with ``create_port=True``, when the port is created? who creates it? | 15:39 |
| ralonsoh | I need to leave now, I'll check this chat later. I'll also review the current OVS agent code | 15:40 |
| sean-k-mooney | not in general. with create_port=True its not required | 15:42 |
| sean-k-mooney | for kernel ovs | 15:42 |
| sean-k-mooney | this is not the first time we have had to revert skipping prot wot ofprot=-1 https://github.com/openstack/neutron/commit/f1fa9529ee6753e2a3f47e4a6eb0286118520d25 | 15:44 |
| opendevreview | Goutham Pacha Ravi proposed openstack/nova master: Allow rebuild for instances with virtiofs shares https://review.opendev.org/c/openstack/nova/+/999048 | 21:44 |
| opendevreview | Ghanshyam Maan proposed openstack/nova master: Task tracking mechanism for graceful shutdown https://review.opendev.org/c/openstack/nova/+/996299 | 22:12 |
| gmaan | melwitt: thanks for review, ^^ updated the wrapper cache feedback. To make revert_resize better in shutdown, we need more work on some RPC redesign or so. I added the details in gerrit and reported a separate bug for tracking it for graceful shutdown part-3 or something to fix as a separate bug https://bugs.launchpad.net/nova/+bug/2162090 | 22:21 |
| gmaan | we need to make revert_resize more sync between source and dest compute like live migration or other operations which involves two compute | 22:22 |
| melwitt | gmaan: ack thanks | 22:45 |
| opendevreview | Ghanshyam Maan proposed openstack/nova master: Call manager graceful_shutdown while RPC server waits https://review.opendev.org/c/openstack/nova/+/996583 | 22:46 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!