| opendevreview | David Hill proposed openstack/nova master: Fail on parse_dom if vif quota sizes > 32 bit unsigned int https://review.opendev.org/c/openstack/nova/+/829064 | 00:00 |
|---|---|---|
| opendevreview | David Hill proposed openstack/nova master: Fail on parse_dom if vif quota sizes > 32 bit unsigned int https://review.opendev.org/c/openstack/nova/+/829064 | 00:14 |
| opendevreview | David Hill proposed openstack/nova master: Fail on parse_dom if vif quota sizes > 32 bit unsigned int https://review.opendev.org/c/openstack/nova/+/829064 | 00:34 |
| opendevreview | David Hill proposed openstack/nova master: Fail on parse_dom if vif quota sizes > 32 bit unsigned int https://review.opendev.org/c/openstack/nova/+/829064 | 00:35 |
| opendevreview | David Hill proposed openstack/nova master: Validate burst value is equal or smaller than 4194303 https://review.opendev.org/c/openstack/nova/+/829064 | 00:40 |
| opendevreview | Takashi Kajinami proposed openstack/nova stable/xena: Fix the wrong exception used to retry detach API calls https://review.opendev.org/c/openstack/nova/+/829049 | 07:00 |
| *** amoralej|off is now known as amoralej | 07:00 | |
| gibi | sean-k-mooney: hi! when you are up, regarding https://bugs.launchpad.net/neutron/+bug/1942329 I made some progress and added a proposal to the today's neutron meeting https://wiki.openstack.org/wiki/Network/Meetings#On_Demand_Agenda you might be interested | 08:27 |
| opendevreview | alecorps proposed openstack/nova master: VMware: Support volumes backed by VStorageObject https://review.opendev.org/c/openstack/nova/+/808791 | 09:44 |
| *** dasm|off is now known as dasm | 11:00 | |
| sean-k-mooney | gibi: oh ok cool so that is something we can fix entirly on the neutron side | 11:14 |
| sean-k-mooney | * entirly on the nova side | 11:15 |
| gibi | sean-k-mooney: nope we need both nova and neutron change | 11:15 |
| sean-k-mooney | we just need to update the mac addres in the neutron port when we migrate | 11:15 |
| sean-k-mooney | well you are thinking of extending the port bindings? | 11:15 |
| sean-k-mooney | to avoid just doing the mac update? | 11:15 |
| gibi | 1) in single binding case we need to update the mac on a bound port that is rejected today by neutron | 11:16 |
| sean-k-mooney | ah ok | 11:16 |
| sean-k-mooney | i didnt know that would be rejected | 11:16 |
| gibi | 2) in multi binding case we might want to keep the MAC per binding to be able to switch | 11:16 |
| sean-k-mooney | yep | 11:16 |
| sean-k-mooney | the thing is the mac is not part of the port binding | 11:17 |
| sean-k-mooney | which i guess is why we need neutron changes | 11:17 |
| gibi | but then we need neturon to use the MAC from the active binding when returning a port | 11:17 |
| gibi | yes, and that is a neutron change too | 11:17 |
| sean-k-mooney | right but the mac is an atribute or the port extention | 11:17 |
| gibi | yes, | 11:17 |
| sean-k-mooney | rather then port binding extentions so ya if we do that we obviously need to handel this slight differently | 11:18 |
| gibi | so we need to be carefully overlay the mac from the active binding top of the port.mac_address field | 11:18 |
| gibi | if there is a binding | 11:18 |
| sean-k-mooney | what i was orginaily thinking is we would not update the mac until after we had activated teh dest binding | 11:19 |
| sean-k-mooney | but if we cant update the port while its bound | 11:19 |
| sean-k-mooney | then that is not an option | 11:19 |
| sean-k-mooney | by the way technically when we update binding:host_id we are unbinding and rebinding the port so we really should be able to update the mac in that case | 11:20 |
| gibi | we could relax the check in neutron to allow mac update if the same request updates the binding:host_id but it becomes messy when we have multiple binding or if nova needs to update the mac separately from the binding activation | 11:20 |
| sean-k-mooney | ya | 11:20 |
| sean-k-mooney | i can see why you would want to extend the port bidnign extention | 11:20 |
| *** carloss is now known as carloss|afk | 11:20 | |
| sean-k-mooney | that seams reasonable to me | 11:21 |
| sean-k-mooney | and just have neutron do the mac update when we do the activation | 11:21 |
| gibi | the API request that activates the binding cannot carry a mac update today so that path is a lot harder to take | 11:21 |
| sean-k-mooney | in the cold migrate case we will still need to be able to update the mac when setting the host id however | 11:21 |
| gibi | so I would put the mac to the binding instead | 11:21 |
| gibi | sean-k-mooney: you are right we have two cases | 11:22 |
| sean-k-mooney | we are not allow to write to the port binding:details but you can put the "migrate_mac" in the binding:profile | 11:22 |
| gibi | sean-k-mooney: I'm wondering if we can add the mac to the binding in case of cold migrate and let neutron multiple binding logic see that as the single active binding | 11:22 |
| sean-k-mooney | gibi: we could also finally start using multiple port binding for cold migration | 11:23 |
| sean-k-mooney | we already do for cross cell | 11:23 |
| gibi | that is a biiig change | 11:23 |
| gibi | afaik | 11:23 |
| gibi | but yes | 11:23 |
| gibi | that would be nice | 11:23 |
| sean-k-mooney | it is but its on out todo list for a long time. perhaps after the bug is fixed :) | 11:23 |
| sean-k-mooney | unless we just want to do this as a feature next cycel | 11:24 |
| gibi | the keyword is _after_ ;) | 11:24 |
| gibi | this MAC update thing is something I would like to backport to victoria if possible | 11:24 |
| gibi | as we need this downstream | 11:24 |
| sean-k-mooney | so what i woudl propsoe is addign a migrat_mac to the binding profile like the other migrating too metadata we have | 11:24 |
| sean-k-mooney | and then allow updating the mac to that migration_mac when we set the host id | 11:25 |
| sean-k-mooney | and in the multiple case have that automaticaly update when we activate the port binding | 11:25 |
| gibi | yeah that is my plan | 11:25 |
| gibi | and you have a point, if we actually update the mac in the db then we can simplify the logic in neutron. initially I planned to just overlay the binding mac over the port mac, but that is complicated from implementation perspective | 11:26 |
| gibi | and one extra conditional that we only need to do this whole thing to PFs (direct-physical) ports. In the rest of the case the mac flows from neutron to the device, but for PF the mac flows from the device to neturon | 11:28 |
| sean-k-mooney | well if we want to do that we can basicaly have neutron do (if host_id == profile:migrating_to then old_mac=mac; mac=profile:migration_mac profile:old_mac=old_mac; | 11:29 |
| sean-k-mooney | ) | 11:29 |
| sean-k-mooney | we would need to record the old mac for revert althoug we wont have cleared the souce pci device allcoation so we might not need old_mac we can just look that up if we revert on the source | 11:30 |
| gibi | I think we don't even have to save the old mac. either it is in an inactive binding already or the port is being unbound | 11:30 |
| gibi | when neutron unbinds a PF it generates a random mac for it | 11:31 |
| sean-k-mooney | ya we dont for cold migration we have it and for live migration we eitehr activate teh dest binding and never rever or we revert before the mac update | 11:31 |
| sean-k-mooney | gibi: no nueton does not generate random macs for pfs | 11:31 |
| gibi | but if we revert then we revert to the source binding that will have the source mac | 11:31 |
| sean-k-mooney | it generate a mac when you first create the port | 11:31 |
| sean-k-mooney | but that is the same for any port | 11:32 |
| sean-k-mooney | it does not generate random macs after that when you change the bining | 11:32 |
| gibi | sean-k-mooney: a/neutron/plugins/ml2/plugin.py | 11:32 |
| gibi | sean-k-mooney: https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L438 | 11:33 |
| sean-k-mooney | that looks like a bug | 11:33 |
| gibi | called from https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py#L1782 | 11:33 |
| sean-k-mooney | actully it looks like they tried to work around this once before by adding that | 11:33 |
| gibi | today nova writes the port.mac_address for PF ports before the first binding. So if there won't be a reset in neutron then after unbind the port would keep the mac of a real device | 11:34 |
| sean-k-mooney | https://bugs.launchpad.net/neutron/+bug/1830383 | 11:34 |
| sean-k-mooney | ya so that code was an incorrect fix for the current issue your trying to solve | 11:35 |
| sean-k-mooney | gibi: that should be reverted | 11:35 |
| gibi | but then what mac a PF port should have after it is unbound | 11:35 |
| gibi | ? | 11:35 |
| gibi | empty? | 11:35 |
| sean-k-mooney | the same one it had when it was bound | 11:35 |
| sean-k-mooney | nova is still telling the vm to use that mac | 11:35 |
| sean-k-mooney | in the cold migration case | 11:36 |
| gibi | hm | 11:36 |
| sean-k-mooney | i can see why they did this but today since we are not updating the mac on the neutron port in the cold migration case this only help in the event we detach the port | 11:36 |
| gibi | so if I boot a VM with a PF port, then that port gets the MAC of the PF today. Then I delete the VM. The port should forget the MAC of the PF | 11:36 |
| sean-k-mooney | ya in that case the currnt reset hack works | 11:37 |
| gibi | so we have to keep that reset for the unbinding case | 11:37 |
| sean-k-mooney | we can yes | 11:37 |
| gibi | for the rebinding there will be a new MAC from nova | 11:38 |
| sean-k-mooney | however if you have 2 vm and 3 hosts and you move vm a form host a to host c and vm b form host b to host a today | 11:38 |
| sean-k-mooney | vm b will get a dhcp responce for vm a | 11:39 |
| sean-k-mooney | because in neutorn we are not updating the macs and vm a still has a port with the mac for port b | 11:39 |
| sean-k-mooney | *for host a | 11:39 |
| sean-k-mooney | assuming vm a and b are on the same netork | 11:39 |
| sean-k-mooney | so the reset is not happeing today when we unbindi the port unle when we detach the port | 11:40 |
| gibi | I have verified it. When I delete the VM neutron resets the MAC | 11:40 |
| gibi | today | 11:41 |
| sean-k-mooney | well i guess it depend on how you define "unbind" | 11:41 |
| sean-k-mooney | yep | 11:41 |
| sean-k-mooney | so if you update the host-id today the port will breifly go to vif_type=unbound | 11:41 |
| sean-k-mooney | so i consider updating the host-id to be unbinding/rebinding the port | 11:42 |
| gibi | but yes if I migrate the VM (and not delet it) then I can create the mixed MAC case you described | 11:42 |
| sean-k-mooney | updating the device_id/device_onwer to '' i do not consider to be unbinding the port | 11:42 |
| sean-k-mooney | that is detaching the port | 11:42 |
| gibi | ahh I see | 11:43 |
| sean-k-mooney | gibi: sorry if im beign pedantic with the terminology there it just the device_id and device_owner filed are not part of the port_bindign or port_binding_extended api extentions | 11:44 |
| gibi | true | 11:44 |
| gibi | just the host_id | 11:44 |
| gibi | but I think that is OK | 11:45 |
| sean-k-mooney | https://github.com/openstack/neutron-lib/blob/master/neutron_lib/api/definitions/portbindings_extended.py#L75-L105 yes | 11:45 |
| sean-k-mooney | so that is why updating the host-id in the cold migrat case set the vif_type to unbound and then to the correct vif | 11:45 |
| sean-k-mooney | it first unbindign the port in the ml2 driver then binds it again | 11:45 |
| sean-k-mooney | potentially to a different ml2 driver | 11:46 |
| sean-k-mooney | e.g. from linux bridge to ovs or whatever if both dirver supprot the vnic_type and you have different ones on different hosts | 11:46 |
| sean-k-mooney | gibi when it the neutron meeting | 11:47 |
| gibi | 14:00 UTC today | 11:47 |
| gibi | afaik | 11:48 |
| sean-k-mooney | i can try and attend but i think we are more or less on teh same page | 11:48 |
| sean-k-mooney | ok so basicaly just over 2 hours form now | 11:48 |
| gibi | no worries | 11:50 |
| gibi | I agree that we are on the same page | 11:51 |
| sean-k-mooney | is this something you will have time to work on | 11:51 |
| sean-k-mooney | i have not booted the old servers i used to use for sriov work in the better part of a year maybe more so not sure if they are still working. i can review and maybe try and test the fix at some point but likely not until after FF | 11:52 |
| sean-k-mooney | we will see how much progress i make on the health checks this week/next | 11:53 |
| sean-k-mooney | im not sure if they will get done before FF but if not im hoping early next cycle they can land | 11:53 |
| sean-k-mooney | gibi: by the way there is a slim change i can partly test dmitriis off-path code too. | 11:54 |
| sean-k-mooney | i dont have bluefield 2 nics but i have a conectx-6 and if i lie to ovn and nova about it beign remote managed i should be able to test it end to end | 11:55 |
| sean-k-mooney | but again likely wont get to that until FF has passed but im hoping to do it before RC1 | 11:56 |
| gibi | sean-k-mooney: I have priority on this bug now from downstream I will work on it. But it should not effect me reviewing patches before FF | 11:56 |
| gibi | so if you have anything just link me up | 11:56 |
| dmitriis | sean-k-mooney: I am looking into adding more func tests and into doing a multi-node test. Unfortunately, SR-IOV is hidden by the QEMU's VFIO layer when PFs are passed through to a VM so I cannot reuse 2 PFs on the same host to fake 2 computes. | 11:57 |
| dmitriis | brb in ~ 1h | 11:57 |
| sean-k-mooney | dmitriis: yep it is | 11:57 |
| gibi | I cannot test dmitriis patch I have only old SRIOV capable PFs | 11:57 |
| sean-k-mooney | dmitriis: you can pass the PF to the vm but it nologner has the sriov extion | 11:58 |
| dmitriis | https://git.qemu.org/?p=qemu.git;a=commitdiff;h=e37dac06dc4e85a2f46c24261c0dfdf2a30b50e3 | 11:58 |
| dmitriis | Still there in recent versions of QEMU: https://github.com/qemu/qemu/blob/v6.2.0/hw/vfio/pci.c#L2094-L2097 | 11:58 |
| dmitriis | this landed upstream https://lore.kernel.org/linux-pci/20200319184653.6c10638b.cohuck@redhat.com/T/ but the QEMU limitation is still there | 11:58 |
| sean-k-mooney | oh.. | 11:58 |
| opendevreview | Merged openstack/nova stable/xena: libvirt: Add announce-self post live-migration workaround https://review.opendev.org/c/openstack/nova/+/825064 | 11:59 |
| dmitriis | (jumped into a meeting) | 11:59 |
| sean-k-mooney | i wonder what happens if you jsut remove the sriov case there | 11:59 |
| sean-k-mooney | /* Read-only VF BARs confuse OVMF */ implies its a uefi limitation | 12:01 |
| sean-k-mooney | https://github.com/qemu/qemu/commit/e37dac06dc4e85a2f46c24261c0dfdf2a30b50e3 | 12:02 |
| opendevreview | David Hill proposed openstack/nova master: Validate burst value is equal or smaller than 4194303 https://review.opendev.org/c/openstack/nova/+/829064 | 13:03 |
| dmitriis | sean-k-mooney: yeah, the commit message says "OVMF tries to size the VF BARs and comes up with absurd results, ending with an assert". Haven't looked at the OVMF in-depth but I suspect it might be around this code which haven't seen much change besides automatic style fixes in 13 years: | 13:14 |
| dmitriis | https://github.com/tianocore/edk2/blame/c095122d4b5f3152417cd97dabecfe31cc3b6508/MdeModulePkg/Bus/Pci/PciBusDxe/PciEnumeratorSupport.c#L494-L502 | 13:14 |
| dmitriis | https://github.com/tianocore/edk2/blame/c095122d4b5f3152417cd97dabecfe31cc3b6508/MdeModulePkg/Bus/Pci/PciBusDxe/PciEnumeratorSupport.c#L1554-L1578 | 13:14 |
| *** amoralej is now known as amoralej|lunch | 13:20 | |
| *** carloss|afk is now known as carloss | 13:25 | |
| opendevreview | Jonathan Race proposed openstack/nova master: object/notification for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/828369 | 13:35 |
| opendevreview | Jonathan Race proposed openstack/nova master: driver/secheduler/docs for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/822053 | 13:35 |
| opendevreview | Jonathan Race proposed openstack/nova master: zuul-job for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/828372 | 13:35 |
| opendevreview | Jonathan Race proposed openstack/nova master: object/notification for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/828369 | 13:43 |
| opendevreview | Jonathan Race proposed openstack/nova master: driver/secheduler/docs for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/822053 | 13:43 |
| opendevreview | Jonathan Race proposed openstack/nova master: zuul-job for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/828372 | 13:43 |
| dmitriis | sean-k-mooney: One other way of working around this which might work is using two containers for Nova compute services (I've certainly ran Libvirt even in unprivileged containers before https://dshcherb.github.io/2017/12/04/qemu-kvm-virtual-machines-in-unprivileged-lxd.html). PCI device access isn't namespaced: we'd need privileged containers and | 13:46 |
| dmitriis | have passthrough_whitelist to use "VFs of a PF" type of config in two different containers: | 13:46 |
| dmitriis | `passthrough_whitelist = {"vendor_id":"*", "product_id":"15b3", "address": "0000:82:00.0", "physical_network": null, "remote_managed": "true"}` | 13:46 |
| dmitriis | `passthrough_whitelist = {"vendor_id":"*", "product_id":"15b3", "address": "0000:82:00.1", "physical_network": null, "remote_managed": "true"}` | 13:46 |
| dmitriis | where 0000:82:00.0 and 0000:82:00.1 are PF addresses. | 13:46 |
| dmitriis | That should give us two "compute hosts" with different hostnames. We would be able to test the control plane path at least while faking the notion of a different host. In the live migration case a QEMU process would migrate locally into a different container and get a VF allocated from a different PF. | 13:49 |
| sean-k-mooney | dmitriis: ya you can do that | 13:51 |
| sean-k-mooney | but its not quite the same as a real live migration to differnt hosts | 13:51 |
| sean-k-mooney | however it would work | 13:51 |
| sean-k-mooney | that is proably more work then i want to do to do some testing but i have a 2 port connectx-6 so i could consider it | 13:52 |
| sean-k-mooney | you woudl need two copies of libvirt too | 13:53 |
| sean-k-mooney | with different hostnames in each contaienr | 13:53 |
| sean-k-mooney | its doable but a lot of work. hopefully you have access to 2 hosts and can test this properly | 13:53 |
| dmitriis | sean-k-mooney: yeah, a UTS namespace should give me that. IPC is namespaced too so shouldn't run into an issue with unix sockets. | 13:53 |
| dmitriis | sean-k-mooney: agreed, just testing the control path in this case. I have a host with ConnectX5 as well that can be used for the live migration case (from BF2 to ConnectX 5). | 13:54 |
| sean-k-mooney | dmitriis: by the way are you working on updating the melonox sriov ci to test this | 13:54 |
| sean-k-mooney | i think it proably should be extended to test this | 13:54 |
| sean-k-mooney | dmitriis: well in production that would not work | 13:54 |
| sean-k-mooney | live migration form BF2 to connectx 5 | 13:55 |
| sean-k-mooney | they woudl use differnt vnic types form a neutorn point of view | 13:55 |
| sean-k-mooney | so that is not a capablity you will be able to leverage in an openstack env | 13:55 |
| dmitriis | sean-k-mooney: I was hoping to get some clarity on the mellanox CI last week but that was postponed, I'm still pursuing it though | 13:55 |
| sean-k-mooney | even though it could be made work | 13:55 |
| sean-k-mooney | ack | 13:55 |
| dmitriis | sean-k-mooney: I could configure the remote-managed feature to work with connectx as well since it's not mandatory to have networking agents on a different host | 13:56 |
| dmitriis | i.e. I could tell Nova that a VF is remote-managed but run ovs-vswitchd and ovn-controller locally | 13:56 |
| sean-k-mooney | technically however i dont think we shoudl really support that | 13:56 |
| sean-k-mooney | we certenly should not document it in any upstream docs | 13:57 |
| sean-k-mooney | except perhaps a developer docs | 13:57 |
| dmitriis | yes, I don't intend to to avoid a clash with the existing approach | 13:57 |
| sean-k-mooney | well its not so much that it would be a clash its more that its semanticaly incorrect. im also not sure what would add teh representor vf to ovs in that case | 13:58 |
| sean-k-mooney | since os-vif wont do that nor will nova | 13:58 |
| dmitriis | sean-k-mooney: that would be ovn-controller with ovn-vif | 13:58 |
| sean-k-mooney | ovn would have to do that | 13:58 |
| sean-k-mooney | ya | 13:58 |
| dmitriis | yep, much like we do on BF2 | 13:58 |
| sean-k-mooney | so you would have to also deploy that | 13:58 |
| dmitriis | yes | 13:59 |
| sean-k-mooney | so for a dev setup it coudl be used to fake a BF2 | 13:59 |
| dmitriis | yes, exactly. ConnectX devices also have a VPD | 13:59 |
| sean-k-mooney | but we dont want operators doing that really becasue we should not assuem that will always work | 13:59 |
| sean-k-mooney | dmitriis: ya i think there are some broadcom cards that have it too not sure about intel | 14:00 |
| dmitriis | sean-k-mooney: ack, agreed | 14:00 |
| sean-k-mooney | in princiapl vpd is all that is needed | 14:00 |
| sean-k-mooney | well and switch deve prot representer netdevs i guess for ovs-vif | 14:00 |
| sean-k-mooney | *ovn-vif | 14:01 |
| dmitriis | yes | 14:01 |
| sean-k-mooney | hopefully other manufactures will implement both and they can just be enabeld using the work you already did | 14:01 |
| dmitriis | sean-k-mooney: for now the switchdev-capable NIC kernel driver. Plenty of drivers in DPDK support representors so we are looking into that too | 14:02 |
| sean-k-mooney | the dpdk ones are userspace only however | 14:02 |
| sean-k-mooney | we dont have support for them in openstack | 14:02 |
| *** amoralej|lunch is now known as amoralej | 14:02 | |
| sean-k-mooney | so you would have to add supprot for that end to end in a similar way | 14:02 |
| sean-k-mooney | although i assume they just use vhost-user on the qemu side | 14:03 |
| dmitriis | sean-k-mooney: hmm, why would the hypervisor side VF care how a VF representor is handled at the remote side? | 14:03 |
| dmitriis | i.e. it could be ovs + DPDK at the BF2 side while the hypervisor side VF would use anything it wants | 14:03 |
| sean-k-mooney | because dpdk does not require VFs | 14:03 |
| sean-k-mooney | it support userspace only represntors that use intels sio cabliteis too | 14:04 |
| sean-k-mooney | so in generally you shoudl not assuem with dpdk that we woudl have a vf attach to the vm at all | 14:04 |
| sean-k-mooney | it could just be a vhost-user prot that is then interally connect to a subfunction | 14:05 |
| sean-k-mooney | useing ovs-dpdk on the BF2 side is different and likely can be hiddne since you are enabelign a VF based approch | 14:05 |
| dmitriis | yeah | 14:05 |
| dmitriis | https://doc.dpdk.org/guides/prog_guide/switch_representation.html#port-representors | 14:05 |
| sean-k-mooney | dpdk woudl just be used for faster flow programing via dpdk flow instead fo tc_flower | 14:06 |
| dmitriis | sean-k-mooney: yep, that's where I was going | 14:06 |
| sean-k-mooney | ya so that is not the same as dpdk port-represntors feature | 14:06 |
| dmitriis | right, I see | 14:06 |
| dmitriis | so this would be to have a broader coverage at the DPU side (BF2 or not) | 14:06 |
| sean-k-mooney | you can use vfs with it but it is more general | 14:07 |
| sean-k-mooney | you should be able to use dpdk with what you have provided today | 14:08 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Record SRIOV PF MAC in the binding profile https://review.opendev.org/c/openstack/nova/+/829248 | 14:08 |
| sean-k-mooney | since we still have a VF on the host side | 14:08 |
| sean-k-mooney | dmitriis: it would only change if we wanted to supprot subfunciton or ovs-dpdk runnign on the host with hardware offload | 14:08 |
| dmitriis | sean-k-mooney: yes, probably with some minor modifications to os-vif but at the OpenStack side things would remain the same | 14:09 |
| dmitriis | sean-k-mooney: yes, subfunctions need a little bit more thinking since they are more dynamic | 14:09 |
| dmitriis | we currently just enable SR-IOV and expect VF representors to appear the other side | 14:09 |
| sean-k-mooney | ya | 14:10 |
| sean-k-mooney | so i proably would still prefer to track subfunciton in the pci tracker as a new type | 14:10 |
| sean-k-mooney | like vdpa | 14:10 |
| sean-k-mooney | even if they are not a vf and may not have a pci adress | 14:11 |
| sean-k-mooney | at which point we might want to rename the module to be host_dev tracker or something | 14:11 |
| sean-k-mooney | i kind of want it to eventualy track mdevs too | 14:11 |
| sean-k-mooney | so it woudl be nice if we could unify it all in a common module | 14:12 |
| sean-k-mooney | but that is not currently a priority for me | 14:12 |
| sean-k-mooney | we alternitivly can use the generic resouces table to track these liek we do for persictent memroy namespaces | 14:12 |
| sean-k-mooney | it was created to be generic | 14:12 |
| sean-k-mooney | it might be a better fit for subfunciton and mdevs | 14:13 |
| dmitriis | sean-k-mooney: on module renaming: yes, kind of going along the way VFIO is going (it has a common part and bus-specific parts) | 14:13 |
| sean-k-mooney | but i have not really spent time looking, at least not in the last 2 years | 14:13 |
| dmitriis | sean-k-mooney: one other direction which we haven't covered is ways for an operator to choose which device will be selected. E.g. in case of multiple DPUs per host or in the cloud we might want to supply product id-like info. | 14:15 |
| dmitriis | currently InstancePCIRequest gets a generated spec | 14:15 |
| sean-k-mooney | no | 14:16 |
| sean-k-mooney | we do not | 14:16 |
| sean-k-mooney | the operator gets to select which device are allowed | 14:16 |
| sean-k-mooney | and can also associate them with a neutron phsyical network | 14:16 |
| sean-k-mooney | but we explcitly do not want them to be able to select device via the pci aliase or the request by other means | 14:17 |
| sean-k-mooney | what we could support is traits requests | 14:17 |
| sean-k-mooney | or perhaps resouce classes | 14:17 |
| sean-k-mooney | i have covered both of those in my pci in placment spec | 14:18 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova-specs/+/791047 | 14:18 |
| dmitriis | yeah, agreed. I didn't mean we want to specify bus-specific info in a request - just that there's no way to alter device selection | 14:18 |
| dmitriis | ^ yes, that. I need to look at the spec again | 14:19 |
| sean-k-mooney | it depend on what you mean by bus specific but in generally i dont think we shoudl | 14:19 |
| sean-k-mooney | we can do it indrectly | 14:19 |
| sean-k-mooney | vai resouce classes or traits | 14:19 |
| sean-k-mooney | vendor id and product id are some what generic and somewhat specific | 14:20 |
| sean-k-mooney | in that other buss tend to have a similar concept | 14:20 |
| sean-k-mooney | like usb has a similar mapping | 14:20 |
| dmitriis | sean-k-mooney: yeah, an indirect approach as with classes or traits would do it I think. | 14:22 |
| sean-k-mooney | i was propsoing using CUSTOM_<Vendor_id>_<product_id> as the resouce class unless you set one in the pci whitelist | 14:27 |
| sean-k-mooney | and extending the pci_alias to supprot a resouce class too | 14:27 |
| sean-k-mooney | the neutron port could potentally specify it but in generally it woudl be nicer if it specified thigns like Support ipse via a HW_NET_IPSEC trait request instead | 14:28 |
| sean-k-mooney | you would need a new neutron extenion however to tie that all together | 14:29 |
| dmitriis | sean-k-mooney: yes, it would be preferable to use a Neutron extension the way I see it | 14:39 |
| sean-k-mooney | there is a security consern with allow arbitry Resouce classes | 14:40 |
| sean-k-mooney | but traits or vendor id and product id may be allowable | 14:40 |
| sean-k-mooney | we perhasp could allow a RC but not allow you to specify the quantity | 14:40 |
| sean-k-mooney | that is proably safe | 14:41 |
| ralonsoh | sean-k-mooney, is it possible to run nova API without wsgi? | 14:53 |
| sean-k-mooney | yes | 14:54 |
| sean-k-mooney | well | 14:54 |
| sean-k-mooney | without an external wsgi server yes | 14:54 |
| sean-k-mooney | it still use wsgi but the python webserver | 14:54 |
| sean-k-mooney | if you just run nova-api | 14:54 |
| ralonsoh | to be able to run it in debug mode, with pycahrm | 14:54 |
| sean-k-mooney | oh well that is harder you can try but eventlet tends to break things | 14:55 |
| sean-k-mooney | in pycharm you can just have it launch the nova-api console script | 14:55 |
| ralonsoh | sean-k-mooney, at least to know when the port receives the allocation information | 14:55 |
| ralonsoh | ok, that could be an idea | 14:55 |
| sean-k-mooney | you have to enabel the gevent pdb option | 14:56 |
| sean-k-mooney | im not sure if its still considerd experimental or not | 14:56 |
| sean-k-mooney | but it was off by default before in pycharm | 14:56 |
| ralonsoh | it works fine | 14:56 |
| sean-k-mooney | the gevent supprot help with debuging with eventlets | 14:56 |
| sean-k-mooney | ya you could actuly disable eventlets in the nova-api | 14:56 |
| sean-k-mooney | with an env argument | 14:57 |
| sean-k-mooney | we only use it for one thing which is multi cell scater gater | 14:57 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/monkey_patch.py#L98 | 14:57 |
| sean-k-mooney | OS_NOVA_DISABLE_EVENTLET_PATCHING | 14:58 |
| ralonsoh | yes, but the wsgi part? | 14:58 |
| sean-k-mooney | the api is a wsgi applicaiton | 14:58 |
| sean-k-mooney | you cant run it with out wsgi but you do not need mod_wsgi or uwsgi | 14:58 |
| ralonsoh | sean-k-mooney, when the allocation info is added to the port? | 15:00 |
| sean-k-mooney | as in placment allcoations | 15:01 |
| sean-k-mooney | i belive as part of port binding | 15:01 |
| sean-k-mooney | ralonsoh: i doubt that will happen in the api | 15:02 |
| sean-k-mooney | it may | 15:02 |
| sean-k-mooney | but conductor, is more likely if not the comptue or schduler | 15:02 |
| ralonsoh | ok | 15:02 |
| sean-k-mooney | the allcoation will only be know after we select the host so its after the api ahs started the async boot in the conductor | 15:03 |
| bauzas | reminder: nova meeting in 57 mins | 15:03 |
| sean-k-mooney | ralonsoh: https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L1172-L1181 | 15:03 |
| ralonsoh | sean-k-mooney, yes but I have no idea who calls this method | 15:04 |
| ralonsoh | compute | 15:04 |
| sean-k-mooney | likely the compute agent or conductor | 15:04 |
| sean-k-mooney | https://opendev.org/openstack/nova/src/branch/master/nova/compute/manager.py#L1769 | 15:05 |
| gibi | ralonsoh: during normal VM create, that is called from the nova-compute service in build_and_run_instance | 15:07 |
| gibi | but it is async as sean-k-mooney linked | 15:07 |
| sean-k-mooney | ralonsoh: so ya this all happens on the compute node from build_and_run_instance | 15:07 |
| ralonsoh | gibi, is there something extra I need to add for handling the QoS in the port? | 15:07 |
| ralonsoh | for placement testing, I'm implementing the OVN placement support | 15:07 |
| sean-k-mooney | no if the port has the resouce request we shoudl include it | 15:08 |
| ralonsoh | (I have created the RPs when the chassis is created) | 15:08 |
| gibi | ralonsoh: hm I have to look but I think that code has no conditional on OVN backend | 15:08 |
| ralonsoh | perfect! | 15:08 |
| gibi | so as sean-k-mooney said, make sure the port has resource request | 15:08 |
| ralonsoh | yeah | 15:08 |
| gibi | then nova should do the scheduling and allocation accordingly | 15:08 |
| gibi | and then adds the RP uuid to the port during binding | 15:09 |
| sean-k-mooney | ralonsoh: also check in the db. as you know neutron has a habbit fo resusing the request to generate teh respocne so if the extions is disabled in the neutron server it can say its there but not persist it into the db | 15:09 |
| sean-k-mooney | which is what happened to me with port numa requests | 15:09 |
| ralonsoh | sean-k-mooney, I'm retrieving this info from the chassis, not the RP | 15:10 |
| ralonsoh | not the placement | 15:10 |
| sean-k-mooney | osc clearly showed it in the port create respocne but a show afterword did not have it | 15:10 |
| sean-k-mooney | i did not mention the RP | 15:10 |
| sean-k-mooney | but i assume you mean you are retriving the bandwith inventories config options form the chassis | 15:11 |
| ralonsoh | yes | 15:11 |
| sean-k-mooney | to create the RP inventories in placment yes | 15:11 |
| sean-k-mooney | the other half of it si actully requesting allcoations form that via the resouce_requests object in the port | 15:12 |
| sean-k-mooney | gibi: its not a dict correct to account for the fact you can have multple different requst groups for different qos polices | 15:12 |
| sean-k-mooney | *now | 15:12 |
| sean-k-mooney | or do we still just have one request group | 15:13 |
| gibi | sean-k-mooney: it depends on an extension :) | 15:14 |
| sean-k-mooney | ah right | 15:14 |
| sean-k-mooney | we detect if neutron support the new format | 15:14 |
| sean-k-mooney | i forgot that was dynamic | 15:14 |
| gibi | yepp | 15:15 |
| gibi | port-resource-request-groups <-- this is the extension alias | 15:15 |
| gibi | it makes the resource_request handle multiple groups | 15:15 |
| sean-k-mooney | gibi: by the way i read back over the neutron meeting logs in the channel so the disucssion is going to mvoe to the review for the pf bug ya? | 15:26 |
| gibi | sean-k-mooney: yepp. there was no imediate objection | 15:26 |
| gibi | sean-k-mooney: meanwhile I pushed wip patches that work for cold migrate now, I will do more testing | 15:26 |
| sean-k-mooney | ack | 15:27 |
| opendevreview | Jonathan Race proposed openstack/nova master: object/notification for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/828369 | 15:35 |
| opendevreview | Jonathan Race proposed openstack/nova master: driver/secheduler/docs for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/822053 | 15:35 |
| opendevreview | Jonathan Race proposed openstack/nova master: zuul-job for Adds Pick guest CPU architecture based on host arch in libvirt driver support https://review.opendev.org/c/openstack/nova/+/828372 | 15:35 |
| bauzas | last reminder : nova meeting in 9 mins here | 15:51 |
| sean-k-mooney | i might drop off irc for a while to go work on the health check stuff without getting too distracted . | 15:54 |
| sean-k-mooney | is there anything i shoudl be aware of re the meetign today | 15:54 |
| sean-k-mooney | if now i might drop now before it starts | 15:54 |
| sean-k-mooney | ill have my client conencted but was going to minimise the terminal or change to a differnt laptop | 15:55 |
| gibi | sean-k-mooney: If you have some time then a review on the any-traits series would be appreciated | 15:57 |
| sean-k-mooney | ack i try and take a look this evening or tomorrow | 15:58 |
| gibi | sean-k-mooney: thanks, happy hacking on the health check | 15:59 |
| bauzas | #startmeeting nova | 16:00 |
| opendevmeet | Meeting started Tue Feb 15 16:00:00 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'nova' | 16:00 |
| gibi | o/ | 16:00 |
| bauzas | heya | 16:00 |
| gmann | o/ | 16:00 |
| bauzas | #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting | 16:00 |
| Uggla | o/ | 16:00 |
| chateaulav | \o | 16:00 |
| elodilles | o/ | 16:00 |
| bauzas | ok, let's start | 16:01 |
| bauzas | #topic Bugs (stuck/critical) | 16:01 |
| bauzas | #info No Critical bug | 16:01 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 42 new untriaged bugs (+3 since the last meeting) | 16:01 |
| bauzas | #help Nova bug triage help is appreciated https://wiki.openstack.org/wiki/Nova/BugTriage | 16:02 |
| bauzas | #link https://storyboard.openstack.org/#!/project/openstack/placement 27 open stories (+0 since the last meeting) in Storyboard for Placement | 16:02 |
| bauzas | I had no time to look at the new bugs | 16:02 |
| bauzas | any bug in particular to raise ? | 16:02 |
| gibi | - | 16:02 |
| bauzas | ok, moving on | 16:03 |
| gibi | the centos-9 issue has been triaged we need the waiting for ssh tempest series to fix it | 16:03 |
| bauzas | yeah | 16:03 |
| bauzas | let's discuss this in the next topic | 16:03 |
| bauzas | #topic Gate status | 16:03 |
| bauzas | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:03 |
| bauzas | #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status | 16:03 |
| bauzas | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:03 |
| gmann | I did not get chance to check that series yet but will do this week or next for sure. | 16:04 |
| bauzas | so, yeah we need the tempest series to be merged | 16:04 |
| gibi | context https://bugs.launchpad.net/nova/+bug/1960346/ | 16:04 |
| gmann | gibi: should i rebase those and few are still WIP? | 16:04 |
| gibi | and https://review.opendev.org/q/topic:wait_until_sshable_pingable | 16:04 |
| gibi | bauzas: do you happen to know if somebody is taking over the tempest series from lyarwood ? | 16:05 |
| bauzas | context https://review.opendev.org/q/topic:wait_until_sshable_pingable | 16:05 |
| bauzas | damn burned | 16:05 |
| gibi | :) | 16:05 |
| bauzas | gibi: honestly, I dunno | 16:05 |
| bauzas | if I can help, let's do it | 16:05 |
| gmann | bauzas: gibi I can do | 16:05 |
| bauzas | cool then | 16:05 |
| gibi | gmann: cool. I can help | 16:05 |
| gmann | late this week or early next week | 16:05 |
| gmann | gibi: thanks | 16:05 |
| gibi | awesome | 16:05 |
| bauzas | I mean, let's discuss in between us then | 16:06 |
| gmann | sure | 16:06 |
| gibi | ack | 16:06 |
| bauzas | lovely | 16:06 |
| sean-k-mooney | i know we did not have anyone dowstream lined up to take it over | 16:06 |
| sean-k-mooney | so im sure ye can work that out after the meeting | 16:06 |
| gibi | ack | 16:06 |
| bauzas | maybe after next Thursday, if people don't mind ;) | 16:06 |
| bauzas | guess why :) | 16:07 |
| gibi | no clue :) | 16:07 |
| bauzas | damn | 16:07 |
| bauzas | it was a good opportunity for jumping to the next agenda item | 16:07 |
| bauzas | if so, | 16:07 |
| bauzas | #topic Release Planning | 16:07 |
| bauzas | #link https://releases.openstack.org/yoga/schedule.html#y-ff FeatureFreeze in 1.5 weeks | 16:07 |
| bauzas | voilà ! | 16:07 |
| *** amoralej is now known as amoralej|off | 16:08 | |
| bauzas | #link https://etherpad.opendev.org/p/nova-yoga-blueprint-status Etherpad for blueprints tracking | 16:08 |
| gibi | ohh that?! :) | 16:08 |
| gmann | :) | 16:08 |
| bauzas | yeah, I'm quite diverted this week + some internal help I need to do for this week :) | 16:08 |
| bauzas | anyway | 16:08 |
| bauzas | so | 16:08 |
| bauzas | sean-k-mooney had a thought on last Thursday | 16:08 |
| bauzas | and I hadn't chance to talk about it | 16:09 |
| bauzas | yet | 16:09 |
| bauzas | what folks think about some specific implementation review day ? | 16:09 |
| gibi | I'm up for it | 16:10 |
| gibi | prefer not to have it on friday | 16:10 |
| bauzas | this thursday? agreed on leaving friday off the table | 16:11 |
| gibi | works for me | 16:11 |
| bauzas | that would be a short notice | 16:11 |
| bauzas | but later means nothing | 16:11 |
| bauzas | also | 16:11 |
| gmann | sounds good | 16:11 |
| sean-k-mooney | friday is also a redhat internal PTO day | 16:11 |
| bauzas | wait, before the also | 16:11 |
| * sean-k-mooney discoverd this this morning | 16:12 | |
| bauzas | sean-k-mooney: oh shit,you're right | 16:12 |
| gibi | :) | 16:12 |
| * bauzas forgot about it | 16:12 | |
| sean-k-mooney | so thrusday is preferable | 16:12 |
| bauzas | sounds a plan | 16:12 |
| gibi | then let's have it on Friday and gmann and I will merge everything :D | 16:12 |
| sean-k-mooney | :) | 16:12 |
| bauzas | #agreed Implementation review day happening on Thursday Feb 17th, bauzas to communicate after the meeting | 16:13 |
| bauzas | ok, next point | 16:13 |
| bauzas | should we consider prioritizing some blueprints over some others ? | 16:13 |
| bauzas | also, | 16:13 |
| bauzas | should we start planning to look at all the API changes adding a new microversion ? | 16:13 |
| bauzas | and asking the owners to rebase in between them ? | 16:14 |
| gibi | do we have more than one series with microversion? | 16:14 |
| bauzas | at least we have one largez | 16:14 |
| gibi | I know about the tenant_id one | 16:14 |
| gmann | yeah one is tenant-id i know of | 16:14 |
| gibi | then no worries on the microversion bumpo | 16:14 |
| gibi | but I agree to say out loud that bps without patches are now being ignored for the FF | 16:15 |
| gibi | that is some level of prioritization :) | 16:15 |
| bauzas | fine with me | 16:15 |
| bauzas | I started to look at https://review.opendev.org/q/topic:bp%252Fremove-tenant-id | 16:15 |
| bauzas | and I'll prioritize unified-limits, this one plus the policy changes and the any-traits support | 16:16 |
| bauzas | at least for the 4 days | 16:16 |
| gibi | I can be on the hook for chateaulav's series | 16:16 |
| gibi | about emulation support | 16:16 |
| gmann | +1, I have kept tenant-id first patch as -W until whole series is ready. | 16:16 |
| chateaulav | gibi: appreciated | 16:16 |
| gibi | and I'm also available for the unified-limits one | 16:17 |
| bauzas | I already have 4 series on my plate | 16:17 |
| bauzas | hence my question about who could cover the others | 16:17 |
| bauzas | good (or bad) news are, 10 blueprints are showing no effort | 16:18 |
| bauzas | and 2 need serious rework | 16:18 |
| bauzas | so most of the things we can reasonably land are already marked up for reviews + wayward | 16:18 |
| gibi | bauzas: if you seem some patch series that I haven't reviewed yet and not on your plate already then let me know and I will jump on it | 16:18 |
| bauzas | the policy series is hard to review | 16:19 |
| sean-k-mooney | i think some of the things without patche have patches we just have not linked them properly | 16:19 |
| bauzas | so a second core with good knowledge is appreciated | 16:19 |
| gmann | I will continue on tenant-id and more of pushing the policy and 'host info to project' changes by this week max. | 16:19 |
| bauzas | sean-k-mooney: hence the etherpad | 16:19 |
| bauzas | it's open to anyone | 16:19 |
| sean-k-mooney | https://blueprints.launchpad.net/nova/+spec/multiqueue-flavor-extra-spec is an example. i added a new section at the top for things that are feature complete but have followup by the way | 16:20 |
| gibi | bauzas: noted | 16:20 |
| bauzas | sean-k-mooney: thanks, works with me | 16:20 |
| gmann | bauzas: I will catch dansmith for policy one. | 16:20 |
| bauzas | anyway, let's continue to chat on IRC | 16:20 |
| bauzas | and keep the pace | 16:21 |
| bauzas | any additional note to say ? | 16:21 |
| sean-k-mooney | if there are no patches for https://blueprints.launchpad.net/nova/+spec/nova-change-default-overcommit-values i might do that quickly | 16:21 |
| gibi | bauzas: do you still aiming to land https://blueprints.launchpad.net/nova/+spec/boot-vm-with-unaddressed-port ? | 16:22 |
| bauzas | gibi: yes, I need to update based on your comment | 16:22 |
| gibi | ack | 16:23 |
| bauzas | was planning to do it today :) | 16:23 |
| bauzas | as you see, I'm lagging :) | 16:23 |
| bauzas | I have a tempest test for verifying https://blueprints.launchpad.net/nova/+spec/boot-vm-with-unaddressed-port that I need to make progress on | 16:23 |
| gibi | cool | 16:23 |
| bauzas | but given tempest is branchless, I guess people agreed this is not impacted by our milestones, right? | 16:24 |
| bauzas | agree* | 16:24 |
| gibi | right | 16:24 |
| gibi | I was about to suggest to decouple tempest from the nova impl | 16:24 |
| bauzas | ok, I'll try ty upload a WIP change against tempest as a proof of work | 16:24 |
| gmann | feature flag is something you can try for branchless tempest | 16:25 |
| bauzas | gmann: did that :) | 16:25 |
| gmann | ohk | 16:25 |
| bauzas | again, need to upload my WIP | 16:25 |
| gmann | gibi: and this one? you are targeting for yoga ? I can skip re-review if not (for now :)) https://review.opendev.org/c/openstack/nova/+/821423 | 16:25 |
| bauzas | but it's far from being mergeable | 16:25 |
| sean-k-mooney | bauzas: i don tthink we need the tempest test to merge the nova change | 16:26 |
| gibi | gmann: that is a bugfix so we have an extra week for that after FF. | 16:26 |
| gibi | gmann: but yes, I would like to have it | 16:27 |
| gmann | gibi: yeah, ack | 16:27 |
| sean-k-mooney | if you have a WIP patch and plan to work on it i think we can trust you to finish it | 16:27 |
| bauzas | sean-k-mooney: yup indeed | 16:28 |
| bauzas | anyway | 16:29 |
| bauzas | any other blueprint to mention ? | 16:29 |
| gibi | - | 16:29 |
| bauzas | then, | 16:30 |
| bauzas | #topic Review priorities | 16:30 |
| bauzas | #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+label:Review-Priority%252B1 | 16:30 |
| bauzas | we said we could use this flag to tell things we plan to review | 16:30 |
| bauzas | I'll do this then accordingly | 16:31 |
| bauzas | at least for the 4 BPs I mentioned | 16:31 |
| bauzas | any other point to mention ? | 16:31 |
| bauzas | looks not | 16:32 |
| bauzas | #topic Stable Branches | 16:32 |
| bauzas | elodilles: ? | 16:32 |
| elodilles | yes, | 16:33 |
| elodilles | actually the state is more or less the same as last week | 16:33 |
| elodilles | gates are OK, except queens and pike | 16:33 |
| elodilles | I still not proposed a working fix for the docs job for them | 16:34 |
| artom | Oh, wallaby has that same tagged attachment bug, I think | 16:34 |
| elodilles | artom: hmmm | 16:34 |
| elodilles | actually this one progressed a bit: https://review.opendev.org/q/topic:workaround-disable-apic | 16:34 |
| * artom should have come prepared with hard data | 16:35 | |
| elodilles | nova part in wallaby is on the gate, | 16:35 |
| artom | OK, so maybe we won't need https://review.opendev.org/c/openstack/nova/+/828542 | 16:35 |
| elodilles | whenever that lands then we need the devstack part | 16:35 |
| elodilles | artom: i think that is not needed in wallaby | 16:36 |
| elodilles | but have to double check | 16:37 |
| bauzas | ack | 16:37 |
| artom | Anecdotally it seems that we do? But as I said, no hard data | 16:37 |
| gmann | elodilles: is this ready? https://review.opendev.org/c/openstack/devstack/+/805632 | 16:37 |
| gmann | I can check devstack backports | 16:37 |
| elodilles | gmann: i think yes, we can remove the Workflow label *when* the nova part has merged | 16:38 |
| elodilles | gmann: i'll ping you | 16:38 |
| artom | And yeah, if it's fixed by the apic workaround we can abandon | 16:39 |
| gmann | elodilles: ack, thanks | 16:39 |
| elodilles | gmann: thanks too :) | 16:39 |
| bauzas | ok, done with that ? | 16:40 |
| elodilles | sorry, yes | 16:40 |
| elodilles | i don't have any other update | 16:40 |
| * elodilles needs to check the the wallaby issue what artom mentioned | 16:41 | |
| bauzas | cool | 16:41 |
| bauzas | , | 16:41 |
| artom | elodilles, it's most likely related to our volume detachment woes | 16:41 |
| artom | So I'm like 90% sure the apic workaround will make it go away | 16:41 |
| bauzas | #topic Open discussion | 16:42 |
| elodilles | artom: ack | 16:42 |
| bauzas | Z release name should be known in 1 week-ish | 16:42 |
| bauzas | that's it, we exhausted the agenda | 16:42 |
| artom | Err, it's official Zed now? | 16:42 |
| artom | And I have a last minute thing for the open discussion | 16:42 |
| gmann | yes, its final. I will reply in email soon after meeting or so | 16:43 |
| bauzas | oh damn, missed it | 16:43 |
| bauzas | there it goes | 16:43 |
| gmann | I do not think we can change it at this stage | 16:43 |
| bauzas | #info Next release is named "Zed" | 16:43 |
| gibi | Zed, boring, but at least short. | 16:43 |
| artom | Also, name of the, err, chopper dude in Pul Fiction | 16:43 |
| artom | *Pulp Fiction | 16:43 |
| gmann | yeah, i was hoping zombie but not sure why i did not pass trademark checks | 16:43 |
| * artom won't mention what Zed was doing *before* getting his chopper stolen | 16:43 | |
| gibi | we could have a screening of that in Berlin :) | 16:43 |
| gmann | *it did not | 16:44 |
| bauzas | oh man | 16:44 |
| bauzas | indeed, I remember the Zed character | 16:44 |
| bauzas | anyway, will propose a patch for creating the zed directory against nova-specs | 16:45 |
| bauzas | we can rename it if needed but I guess ship has sailed | 16:45 |
| * bauzas just wonders how Zombie got hit by trademarks while Zed did not | 16:45 | |
| bauzas | for Zen, it was obvious | 16:45 |
| bauzas | for Zeta, nothing comes out of my mind | 16:46 |
| artom | Aren't they a violent gang somewhere? | 16:46 |
| bauzas | (fwiw, Zombie was a pun when I proposed it) | 16:46 |
| artom | Yup https://en.wikipedia.org/wiki/Los_Zetas | 16:46 |
| bauzas | because everybody else is saying OpenStack is dead | 16:46 |
| gmann | true | 16:46 |
| bauzas | I just wanted to tell that if OpenStack is dead, it's still alive | 16:47 |
| bauzas | hence the pun | 16:47 |
| bauzas | anyway, we're diverting | 16:47 |
| bauzas | #action to propose a patch against nova-specs for creating the "zed" directory | 16:47 |
| bauzas | artom: you had an item | 16:48 |
| artom | Yeah, so this is last minute, but I want to start socializing this strawman idea | 16:48 |
| artom | The tripleo folks often have bugs that block their CI | 16:49 |
| artom | Sometimes those are Nova folks, or they initially think they're Nova bugs | 16:49 |
| artom | The current process for those is... well, bad | 16:49 |
| bauzas | having a boot failure isn't always due to Nova, y'know :) | 16:49 |
| artom | I know, I Know | 16:50 |
| artom | But like the volume detach issues | 16:50 |
| artom | It blocked us, it blocked them as well | 16:50 |
| artom | What they currently do is file bugs against the *tripleo* component, and get us, via a *downstream internal Red Hat call*, to look at those they think are Nova | 16:50 |
| artom | Would there be willingness here in this community to get bugs filed againt *Nova*, with the gate-blocker tag, and to leverage the existing upstream bug triage process (however light it is)? | 16:51 |
| bauzas | anyone can report a bug against nova :) | 16:52 |
| bauzas | actually this is a good point | 16:52 |
| gibi | artom: I'm OK with that, if they take the time to point to the nova part of the tripleo problem as I'm not familiar with tripelo | 16:52 |
| bauzas | I won't refrain anyone to do this | 16:52 |
| bauzas | the only problem is the triage capacity | 16:52 |
| bauzas | I do it when I have time | 16:52 |
| artom | Their expectation would be, as it's a gate-blocker for them, that we would prioritize triaging those, similar to how Neutron would do it for our gate blocker, for example | 16:52 |
| artom | But yeah, we can definitely ask for high bug report quality | 16:53 |
| bauzas | artom: I'd say it's worth adding nova as an impacted project, I agree | 16:53 |
| gibi | artom: if they need priority I suggest to ping us with the gate bug here on irc | 16:53 |
| bauzas | gibi: exactly | 16:53 |
| bauzas | upstream first | 16:53 |
| bauzas | file a LP bug, go shout the folks on IRC | 16:53 |
| bauzas | and we'll triage it | 16:53 |
| artom | ooo has a very... "flexible" concept of upstream/downstream | 16:54 |
| gibi | :) | 16:54 |
| artom | Since they're all Red Hat folks, essentially | 16:54 |
| bauzas | expect the nova folks to magically triage this LP bug isn't exactly a good recipe for success | 16:54 |
| * gibi needs more magic | 16:54 | |
| * bauzas just needs time or duplicity | 16:54 | |
| bauzas | anyway, | 16:55 |
| artom | OK, so my takeaway would be "yes, do it upstream, but high quality bug reports please" | 16:55 |
| bauzas | artom: feel free to tell them to file a launchpad bug against nova and ask us on IRC to look at it | 16:55 |
| bauzas | artom: yes | 16:55 |
| bauzas | 100% yes | 16:55 |
| artom | Awesome, much thanks | 16:55 |
| gibi | +1 | 16:55 |
| bauzas | artom: appreciated the thought | 16:56 |
| bauzas | any last item to discuss before I call it a wrap ? | 16:56 |
| bauzas | looks not | 16:56 |
| bauzas | thanks all | 16:56 |
| bauzas | #endmeeting | 16:56 |
| opendevmeet | Meeting ended Tue Feb 15 16:56:53 2022 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:56 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2022/nova.2022-02-15-16.00.html | 16:56 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2022/nova.2022-02-15-16.00.txt | 16:56 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2022/nova.2022-02-15-16.00.log.html | 16:56 |
| gibi | bauzas: thanks! | 16:58 |
| elodilles | artom: about the test_tagged volume detach issue: in wallaby there is a different problem as i remember (yes, though, *everything* ends up in volume detach timeouts :)) | 16:59 |
| elodilles | artom: in wallaby the volume detach issue is mostly because some kernel panic in cirros | 17:00 |
| elodilles | artom: that's why there is the possibility to use in wallaby the libvirt_disable_apic workaround | 17:01 |
| artom | elodilles, cool, cool - tbh I didn't look into it in a lot of depth, my hope is that merging the apic workaround will make it go away | 17:03 |
| elodilles | artom: in wallaby the 'volume detach' failures have significant higher numbers than in xena, because the workaround is there in xena for some time already. at least that is what i understood / saw so far | 17:05 |
| elodilles | so i think the apic workaround makes the wallaby gate more stable | 17:06 |
| artom | Oh, wait, I said wallaby, didn't I? | 17:06 |
| elodilles | hopefully we will see that | 17:06 |
| artom | *facepalm* https://review.opendev.org/c/openstack/nova/+/828542 is against Xena | 17:06 |
| artom | So wth's going on? | 17:06 |
| elodilles | actually, xena is less impacted with the volume detach issue as far as i know | 17:08 |
| artom | Yeah, so I'm confused | 17:09 |
| elodilles | https://zuul.opendev.org/t/openstack/builds?job_name=nova-next&project=openstack%2Fnova&branch=stable%2Fxena&skip=0 | 17:09 |
| artom | I mean, that's still 25% failure rate | 17:13 |
| artom | Roughly | 17:13 |
| elodilles | and now imagine that in wallaby it's worse :) | 17:14 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Record SRIOV PF MAC in the binding profile https://review.opendev.org/c/openstack/nova/+/829248 | 17:19 |
| opendevreview | Sylvain Bauza proposed openstack/nova-specs master: Create specs directory for Zed https://review.opendev.org/c/openstack/nova-specs/+/829385 | 17:26 |
| bauzas | gibi: sean-k-mooney: gmann: zed directory created ^ | 17:26 |
| * bauzas bails out | 17:27 | |
| gibi | bauzas: cool, will check | 17:27 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: DNM: trigger nova tests with the neutron fix included https://review.opendev.org/c/openstack/nova/+/829386 | 17:28 |
| gmann | bauzas: thanks, will check | 17:35 |
| gmann | dansmith: rbac changes are ready for review, I am +2 on your patches and pushed other policy change on top of it. I am left with couple of patches including releasenotes etc which I am working in parallel https://review.opendev.org/q/topic:bp%252Fpolicy-defaults-refresh-2 | 18:21 |
| dansmith | gmann: ack, is there any hold up on my patch at the bottom? | 18:23 |
| gmann | dansmith: no, all good except one comment for testing PROJECT_ADMIN for all-tenants APIs which I have pushed as separate patch in that series | 18:24 |
| spatel | My vm stuck in bad state because we were running load-test now its not responding to virsh console and also not letting me reboot that VM. any other way i can force reboot vm without destroy? | 18:50 |
| spatel | I have tried virsh shutdown 6 --mode acpi and signal but still no luck | 18:51 |
| spatel | nevermind virsh destroy works | 19:03 |
| opendevreview | sean mooney proposed openstack/nova master: [WIP] add initial healthcheck support https://review.opendev.org/c/openstack/nova/+/825015 | 19:47 |
| opendevreview | sean mooney proposed openstack/nova master: [WIP] add healthcheck manager to manager base https://review.opendev.org/c/openstack/nova/+/827844 | 19:47 |
| opendevreview | Merged openstack/nova master: Move 'hw:pmu', 'hw_pmu' parsing to nova.virt.hardware https://review.opendev.org/c/openstack/nova/+/792364 | 21:41 |
| opendevreview | melanie witt proposed openstack/nova-specs master: Amend unified limits spec to explain "API limit" enforcement https://review.opendev.org/c/openstack/nova-specs/+/829413 | 21:59 |
| opendevreview | melanie witt proposed openstack/nova-specs master: Amend unified limits spec to explain "API limit" enforcement https://review.opendev.org/c/openstack/nova-specs/+/829413 | 22:27 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!