| *** tetsuro has joined #openstack-nova | 00:07 | |
| openstackgerrit | sean mooney proposed openstack/os-vif master: update tox envs and support pdf docs https://review.opendev.org/728037 | 00:13 |
|---|---|---|
| openstackgerrit | sean mooney proposed openstack/os-vif master: support pyroute2 0.5.13 https://review.opendev.org/744809 | 00:13 |
| *** _erlon_ has quit IRC | 00:14 | |
| openstackgerrit | sean mooney proposed openstack/os-vif master: support pyroute2 0.5.13 https://review.opendev.org/744809 | 00:15 |
| openstackgerrit | sean mooney proposed openstack/os-vif master: update tox envs and support pdf docs https://review.opendev.org/728037 | 00:15 |
| openstackgerrit | sean mooney proposed openstack/os-vif master: [goal] migrate testing to ubuntu focal https://review.opendev.org/738130 | 00:17 |
| *** gyee has quit IRC | 00:30 | |
| *** songwenping__ has joined #openstack-nova | 00:34 | |
| *** songwenping_ has quit IRC | 00:37 | |
| openstackgerrit | sean mooney proposed openstack/os-vif master: deprecate ovs-vsctl driver and make native the default https://review.opendev.org/744816 | 00:49 |
| *** tetsuro_ has joined #openstack-nova | 00:54 | |
| *** tetsuro_ has quit IRC | 00:56 | |
| *** tetsuro_ has joined #openstack-nova | 00:56 | |
| *** mgariepy has quit IRC | 00:57 | |
| *** tetsuro has quit IRC | 00:58 | |
| *** tetsuro_ has quit IRC | 00:59 | |
| *** tetsuro has joined #openstack-nova | 00:59 | |
| *** mgariepy has joined #openstack-nova | 01:12 | |
| *** xiaolin has joined #openstack-nova | 01:23 | |
| *** mgariepy has quit IRC | 01:32 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Fix CI failture: Introduce decorator>=3.4.0 to test-requirements.txt https://review.opendev.org/744385 | 01:32 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Fix issue with decorator in lower-constraints job https://review.opendev.org/744385 | 01:33 |
| *** mgariepy has joined #openstack-nova | 01:33 | |
| *** dave-mccowan has joined #openstack-nova | 01:45 | |
| *** tetsuro_ has joined #openstack-nova | 02:11 | |
| *** dave-mccowan has quit IRC | 02:13 | |
| *** tetsuro has quit IRC | 02:15 | |
| *** aarents has quit IRC | 02:22 | |
| *** aarents has joined #openstack-nova | 02:23 | |
| *** songwenping_ has joined #openstack-nova | 02:25 | |
| openstackgerrit | sean mooney proposed openstack/os-vif master: deprecate ovs-vsctl driver and make native the default https://review.opendev.org/744816 | 02:27 |
| *** songwenping__ has quit IRC | 02:28 | |
| *** mtreinish has joined #openstack-nova | 02:39 | |
| *** brinzhang has joined #openstack-nova | 02:54 | |
| openstackgerrit | Brin Zhang proposed openstack/nova master: Fix issue with decorator in lower-constraints job https://review.opendev.org/744385 | 02:56 |
| *** songwenping__ has joined #openstack-nova | 02:59 | |
| *** Yumeng has joined #openstack-nova | 03:00 | |
| *** songwenping_ has quit IRC | 03:03 | |
| *** JamesBen_ has quit IRC | 03:03 | |
| *** rcernin has joined #openstack-nova | 03:07 | |
| *** markvoelker has joined #openstack-nova | 03:10 | |
| *** tetsuro_ has quit IRC | 03:12 | |
| *** markvoelker has quit IRC | 03:15 | |
| *** mkrai has joined #openstack-nova | 03:32 | |
| *** psachin has joined #openstack-nova | 03:33 | |
| *** JamesBenson has joined #openstack-nova | 03:38 | |
| *** markvoelker has joined #openstack-nova | 03:46 | |
| *** markvoelker has quit IRC | 03:51 | |
| *** tkajinam has quit IRC | 03:51 | |
| *** tkajinam has joined #openstack-nova | 03:52 | |
| *** markvoelker has joined #openstack-nova | 03:53 | |
| *** markvoelker has quit IRC | 04:05 | |
| *** vishalmanchanda has joined #openstack-nova | 04:15 | |
| *** ociuhandu has joined #openstack-nova | 04:18 | |
| *** sapd1_x has joined #openstack-nova | 04:22 | |
| *** ociuhandu has quit IRC | 04:23 | |
| *** songwenping_ has joined #openstack-nova | 04:24 | |
| *** brinzhang_ has joined #openstack-nova | 04:24 | |
| *** brinzhang has quit IRC | 04:27 | |
| *** songwenping__ has quit IRC | 04:27 | |
| *** udesale has joined #openstack-nova | 04:33 | |
| *** evrardjp has quit IRC | 04:33 | |
| *** evrardjp has joined #openstack-nova | 04:33 | |
| *** brinzhang has joined #openstack-nova | 04:46 | |
| *** swp20 has joined #openstack-nova | 04:46 | |
| *** brinzhang_ has quit IRC | 04:48 | |
| *** songwenping_ has quit IRC | 04:49 | |
| *** ratailor has joined #openstack-nova | 04:56 | |
| *** ratailor has quit IRC | 04:59 | |
| *** ratailor has joined #openstack-nova | 04:59 | |
| *** JamesBenson has quit IRC | 05:01 | |
| *** sapd1_x has quit IRC | 05:02 | |
| *** lbragstad_ has joined #openstack-nova | 05:04 | |
| *** lbragstad has quit IRC | 05:07 | |
| *** songwenping_ has joined #openstack-nova | 05:35 | |
| *** swp20 has quit IRC | 05:39 | |
| *** udesale_ has joined #openstack-nova | 05:45 | |
| *** udesale has quit IRC | 05:47 | |
| openstackgerrit | Merged openstack/nova master: Fix lower-constraints conflicts https://review.opendev.org/744506 | 05:56 |
| *** gokhani has joined #openstack-nova | 05:58 | |
| *** Yumeng has quit IRC | 06:00 | |
| *** yaawang has quit IRC | 06:02 | |
| *** yaawang has joined #openstack-nova | 06:02 | |
| *** yaawang has quit IRC | 06:20 | |
| *** yaawang has joined #openstack-nova | 06:21 | |
| *** udesale_ has quit IRC | 06:24 | |
| *** maciejjozefczyk has joined #openstack-nova | 06:28 | |
| *** dklyle has quit IRC | 06:31 | |
| *** mlycka has joined #openstack-nova | 06:43 | |
| *** links has joined #openstack-nova | 06:48 | |
| *** udesale has joined #openstack-nova | 06:52 | |
| *** lchabert has quit IRC | 06:53 | |
| *** mvorwerk has joined #openstack-nova | 06:54 | |
| *** yaawang has quit IRC | 06:57 | |
| *** yaawang has joined #openstack-nova | 06:58 | |
| *** rcernin has quit IRC | 06:58 | |
| *** rcernin_ has joined #openstack-nova | 06:59 | |
| *** mkrai has quit IRC | 07:00 | |
| *** mvorwerk has quit IRC | 07:00 | |
| *** mvorwerk has joined #openstack-nova | 07:00 | |
| *** nightmare_unreal has joined #openstack-nova | 07:00 | |
| *** tesseract has joined #openstack-nova | 07:00 | |
| *** rcernin_ has quit IRC | 07:05 | |
| *** rcernin has joined #openstack-nova | 07:06 | |
| *** slaweq has joined #openstack-nova | 07:07 | |
| gokhani | hi team , I have an OpenStack environment in pike version. we have been using it for 2 years. Our environment consists of 3 controller and 23 compute hosts. but for 1 week , we are getting warnings like "Function 'nova.servicegroup.drivers.db.DbDriver._report_state' run outlasted interval by 24.01 sec" in compute nodes. what can be the reason of this ? do I need to look at mariadb galera cluster or rabbitmq cluster ? logs are in http://paste.opens | 07:12 |
| gokhani | tack.org/show/796586/ | 07:12 |
| gokhani | * http://paste.openstack.org/show/796586/ | 07:28 |
| *** dougsz has joined #openstack-nova | 07:32 | |
| *** tosky has joined #openstack-nova | 07:38 | |
| *** mkrai has joined #openstack-nova | 07:41 | |
| *** songwenping__ has joined #openstack-nova | 07:57 | |
| *** songwenping_ has quit IRC | 07:59 | |
| *** dtantsur|afk is now known as dtantsur | 08:04 | |
| *** xek has joined #openstack-nova | 08:21 | |
| *** tosky has quit IRC | 08:26 | |
| *** tosky has joined #openstack-nova | 08:27 | |
| *** k_mouza has joined #openstack-nova | 08:29 | |
| *** ralonsoh has joined #openstack-nova | 08:30 | |
| *** bjolo has joined #openstack-nova | 08:35 | |
| *** songwenping_ has joined #openstack-nova | 08:36 | |
| *** songwenping_ has quit IRC | 08:37 | |
| *** songwenping_ has joined #openstack-nova | 08:37 | |
| *** songwenping__ has quit IRC | 08:39 | |
| *** songwenping_ has quit IRC | 08:41 | |
| *** songwenping_ has joined #openstack-nova | 08:42 | |
| *** derekh has joined #openstack-nova | 08:51 | |
| *** tetsuro has joined #openstack-nova | 08:54 | |
| *** rcernin has quit IRC | 09:04 | |
| *** rcernin has joined #openstack-nova | 09:04 | |
| *** rcernin has quit IRC | 09:05 | |
| *** rcernin has joined #openstack-nova | 09:05 | |
| *** jangutter has quit IRC | 09:11 | |
| *** tetsuro has quit IRC | 09:11 | |
| *** jangutter has joined #openstack-nova | 09:11 | |
| *** ociuhandu has joined #openstack-nova | 09:23 | |
| *** sapd1_x has joined #openstack-nova | 09:30 | |
| *** yaawang has quit IRC | 09:35 | |
| *** yaawang has joined #openstack-nova | 09:36 | |
| *** mlycka has quit IRC | 09:42 | |
| *** mlycka has joined #openstack-nova | 09:44 | |
| *** jangutter has quit IRC | 09:46 | |
| *** jangutter has joined #openstack-nova | 09:46 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: fakelibvirt: Remove nova-network remnants https://review.opendev.org/737329 | 09:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Add type hints https://review.opendev.org/744869 | 09:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Drop 'vpn' parameter from 'allocate_for_instance' https://review.opendev.org/744870 | 09:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Remove unused 'affect_auto_assigned' parameter https://review.opendev.org/744871 | 09:49 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: network: Remove 'kwargs' from 'get_instance_nw_info' https://review.opendev.org/744872 | 09:49 |
| *** ralonsoh has quit IRC | 09:50 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add helpers for suspend, resume and reboot of server https://review.opendev.org/741285 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Pass context, instance to '_create_guest' https://review.opendev.org/741286 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: api: Reject non-spawn operations for vTPM https://review.opendev.org/741500 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Add emulated TPM support to Nova https://review.opendev.org/631363 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Add docs for vTPM support https://review.opendev.org/739213 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add type hints to 'nova.compute.manager' https://review.opendev.org/742863 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Don't unset Instance.old_flavor, new_flavor until necessary https://review.opendev.org/741995 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: privsep: Add support for recursive chown, move_tree operations https://review.opendev.org/742864 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add type hints to 'nova.virt.libvirt.utils' https://review.opendev.org/742865 | 09:54 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Add support for resize and cold migration of emulated TPM files https://review.opendev.org/639934 | 09:54 |
| lyarwood | RIP zuul | 09:57 |
| kashyap | stephenfin: In the default Nova configuration, VMs are allowed to overcommit and float freely across host CPUs -- still true? | 10:11 |
| stephenfin | lyarwood: blame artom and his damned reviews | 10:11 |
| stephenfin | kashyap: yes | 10:12 |
| stephenfin | only caveat is they can't overcommit against themselves | 10:12 |
| stephenfin | can't boot an 8 core instance on a 6 core host | 10:12 |
| kashyap | stephenfin: Ah, okay; I was about to ask for an example :) | 10:12 |
| stephenfin | (or host with only 6 cores enabled in nova, via 'vcpu_pin_set' (legacy) or combo of 'cpu_dedicated_set' and 'cpu_shared_set') | 10:13 |
| *** tkajinam has quit IRC | 10:15 | |
| kashyap | Thanks | 10:15 |
| openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Functions to merge provider configs to provider tree https://review.opendev.org/676522 | 10:18 |
| openstackgerrit | Tony Su proposed openstack/nova master: Provider Config File: Enable loading and merging of provider configs https://review.opendev.org/693460 | 10:18 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: zuul: Start to migrate nova-live-migration to zuulv3 https://review.opendev.org/711604 | 10:19 |
| tony_su | stephenfin: gibi: the last 3 provider-config-file patches are ready to go. change log: 1) all comments reflected in code 2) add three more test case to cover negative coe logic 3) unified coding style. | 10:21 |
| *** rcernin has quit IRC | 10:23 | |
| *** xek has quit IRC | 10:24 | |
| gibi | tony_su: thanks. I added that to the queue for runway slot | 10:27 |
| tony_su | gibi: thanks gibi. I will continue working on these patches in the next two weeks... | 10:31 |
| *** mvorwerk has quit IRC | 10:31 | |
| *** mvorwerk has joined #openstack-nova | 10:32 | |
| *** jangutter_ has joined #openstack-nova | 10:32 | |
| *** songwenping_ has quit IRC | 10:35 | |
| *** jangutter has quit IRC | 10:35 | |
| *** hemna has quit IRC | 10:38 | |
| *** hemna has joined #openstack-nova | 10:38 | |
| *** mvorwerk_ has joined #openstack-nova | 10:42 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: nova-multinode-evacuate https://review.opendev.org/744883 | 10:43 |
| lyarwood | hmm I thought the grenade issues had been resolved? | 10:44 |
| *** mvorwerk has quit IRC | 10:44 | |
| *** ociuhandu has quit IRC | 10:54 | |
| *** ociuhandu has joined #openstack-nova | 10:55 | |
| *** ociuhandu has quit IRC | 11:00 | |
| brinzhang | stephenfin: hi, do I need fixed your point comments in https://review.opendev.org/#/c/742863/2/nova/compute/manager.py@2615 | 11:01 |
| stephenfin | brinzhang: Not at all. I was saying that your fix was correct and mine was not :) | 11:02 |
| brinzhang | change the LOG.exception(exc.format_message()) to LOG.exception(exc), and use Exception instead of exception.NovaException | 11:02 |
| brinzhang | an I will report a bug for this change | 11:02 |
| brinzhang | stephenfin: oh sorry, your patch are working now, I will just fixed mine ^ | 11:04 |
| openstackgerrit | Brin Zhang proposed openstack/nova master: [Trivial] Remove wrong format_message() conversion https://review.opendev.org/744280 | 11:07 |
| *** ralonsoh has joined #openstack-nova | 11:08 | |
| brinzhang | stephenfin: updated, thanks | 11:09 |
| *** sapd1_x has quit IRC | 11:09 | |
| brinzhang | stephenfin, sean-k-mooney, gibi: the cyborg evacuate support patch was updated, pls review again while you have time, thanks. | 11:10 |
| brinzhang | https://review.opendev.org/#/c/715326/ | 11:10 |
| brinzhang | and I was rebased on the optimize patch, pls see https://review.opendev.org/#/c/726564/6 | 11:10 |
| *** jangutter has joined #openstack-nova | 11:12 | |
| *** jangutter_ has quit IRC | 11:15 | |
| *** markvoelker has joined #openstack-nova | 11:16 | |
| *** raildo has joined #openstack-nova | 11:21 | |
| *** markvoelker has quit IRC | 11:25 | |
| artom | stephenfin, sorry :( it was a drive-by thing as well, I didn't actually look at anything else | 11:27 |
| artom | Hence no vote | 11:28 |
| lyarwood | artom: how dare you provide useful reviews | 11:30 |
| lyarwood | I mean really | 11:30 |
| lyarwood | you're making the rest of us look bad ^_^ | 11:31 |
| lyarwood | or maybe I'm doing that on my own | 11:31 |
| artom | lyarwood, "useful" would be a stretch :P | 11:31 |
| gibi | sean-k-mooney: thanks for the repeated test of the sriov attach. I responded in the review. Unfortunately I cannot reproduce your failures in my env. | 11:33 |
| artom | (Yes, I'm knowingly leaving your last sentence unanswered >;) | 11:33 |
| gibi | brinzhang: ack | 11:33 |
| *** ociuhandu has joined #openstack-nova | 11:36 | |
| openstackgerrit | Merged openstack/python-novaclient master: Remove unused code https://review.opendev.org/744136 | 11:38 |
| *** xek has joined #openstack-nova | 11:40 | |
| *** JamesBenson has joined #openstack-nova | 11:40 | |
| sean-k-mooney | gibi: so in your env removal of direct physical or macvtap devices worked? | 11:41 |
| gibi | sean-k-mooney: yes | 11:41 |
| gibi | the only thing that I can see is the remaining MAC on the VF after macvtap removal | 11:41 |
| gibi | the rest of your failure does not appear to me | 11:41 |
| sean-k-mooney | i used a single vm to do all the testing | 11:42 |
| *** ociuhandu has quit IRC | 11:42 | |
| sean-k-mooney | ill try it again with multiple vms and see it it makes any difference | 11:42 |
| *** ociuhandu has joined #openstack-nova | 11:42 | |
| sean-k-mooney | what os and libvirt version are you using? | 11:42 |
| gibi | ubuntu 18.04, libvirt 6.0.0 qemu 4.2 | 11:43 |
| sean-k-mooney | the libvirt behavior may have changed? i was using centos 8.1 maybe 8.2 i might be using older libvirt | 11:43 |
| sean-k-mooney | i need to boot up the server to check | 11:43 |
| gibi | the silent failure of macvtap and direct physical removel feels like a problem with findind the device that needs to be removed. I can try to add extra LOGs around that logic to trace the matching in your nev | 11:44 |
| gibi | env | 11:44 |
| gibi | two weeks ago I upgraded the libvirt from 4.0.0 to 6.0.0 and qemu from 2.11 to 4.2 due to a different failure in the "simple" direct case | 11:45 |
| sean-k-mooney | if we need libvirt 6.0.0 we could not that as a min verion i guess. | 11:46 |
| sean-k-mooney | but yes | 11:46 |
| sean-k-mooney | both feel like we just did not find it in the xml and remove it | 11:46 |
| *** brinzhang_ has joined #openstack-nova | 11:47 | |
| gibi | regarding the leaking MAC addess after macvtap removal, who should do the removal of the MAC from the VF? is it libvirt? | 11:47 |
| sean-k-mooney | gibi: libvirt should altrhough we also have code in nova to clear it for old libvirts | 11:49 |
| sean-k-mooney | its also not reseting the programed vlan on the vf | 11:49 |
| sean-k-mooney | but that is likely the same issue | 11:49 |
| *** brinzhang has quit IRC | 11:49 | |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L774-L784 | 11:50 |
| sean-k-mooney | we seam not to be calling unplug | 11:50 |
| sean-k-mooney | in the direct case it proably is not clearing the trused vf status | 11:50 |
| sean-k-mooney | i didnt actuly test that | 11:51 |
| sean-k-mooney | we should be calling https://github.com/openstack/nova/blob/master/nova/virt/libvirt/vif.py#L833 as part of detach | 11:51 |
| gibi | sean-k-mooney: thanks I will trace this missing unplug in my env | 11:52 |
| sean-k-mooney | we are apprently https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L2201 | 11:52 |
| gibi | I even have a log in the dmesg about the MAC removal during macvtap | 11:53 |
| gibi | [1911821.990047] ixgbe 0000:81:00.0: removing MAC on VF 2 | 11:53 |
| gibi | but right after it | 11:53 |
| gibi | [1911822.241163] ixgbe 0000:81:00.0: Setting VLAN 100, QOS 0x0 on VF 2 | 11:53 |
| gibi | it seems the vlan is set back to it | 11:53 |
| sean-k-mooney | oh god damit.... | 11:53 |
| sean-k-mooney | the neutron sriov nic agent is proably racing with unplug | 11:54 |
| sean-k-mooney | although i guess it could be libvirt? | 11:54 |
| *** k_mouza has quit IRC | 11:54 | |
| sean-k-mooney | we might want to do unplug after detach? | 11:55 |
| gibi | good points | 11:55 |
| gibi | I will gather logs from libvirt and the neutron agent to see if this is a race | 11:55 |
| *** xek has quit IRC | 11:56 | |
| *** rcernin has joined #openstack-nova | 11:56 | |
| sean-k-mooney | [centos@sriov-1 ~]$ libvirtd --version | 11:57 |
| sean-k-mooney | libvirtd (libvirt) 6.0.0 | 11:57 |
| sean-k-mooney | so same libvirt version | 11:57 |
| sean-k-mooney | im using differnet nics then you you have nicantic 10G nics and im using 1G e1000 nics but that should not matter | 11:57 |
| *** mkrai has quit IRC | 11:58 | |
| sean-k-mooney | @chat:button1[centos@sriov-1 ~]$ /usr/libexec/qemu-kvm --version | 11:58 |
| sean-k-mooney | QEMU emulator version 4.2.0 (qemu-kvm-4.2.0-19.el8) | 11:58 |
| sean-k-mooney | i think that is the same qemu too? | 11:58 |
| sean-k-mooney | yep | 11:58 |
| sean-k-mooney | so ya likely not related to the versions | 11:58 |
| gibi | cool, on set of possible differences is ruled out | 11:59 |
| gibi | one | 11:59 |
| * gibi trying to find the racing component | 12:01 | |
| *** rcernin has quit IRC | 12:02 | |
| sean-k-mooney | its proably libvirt | 12:09 |
| stephenfin | gibi, lyarwood: Could you folks stick https://review.opendev.org/#/c/744021/ on your respective review queues, please? Feel free to chuck something my way too | 12:10 |
| lyarwood | stephenfin: I was looking at that yesterday | 12:12 |
| lyarwood | stephenfin: I *think* I get it, I just wanted to grep around a little more before voting | 12:12 |
| stephenfin | nw. Lots of context needed for it, unfortunately | 12:13 |
| *** gokhani has left #openstack-nova | 12:14 | |
| lyarwood | yup indeed, func tests helped however so thanks for that at least | 12:14 |
| *** rcernin has joined #openstack-nova | 12:17 | |
| *** alex_xu has joined #openstack-nova | 12:20 | |
| *** brinzhang0 has joined #openstack-nova | 12:21 | |
| *** rcernin has quit IRC | 12:22 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Drop support for Xen https://review.opendev.org/743231 | 12:24 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: libvirt: Remove 'hypervisor_version' from 'libvirt_info' https://review.opendev.org/744199 | 12:24 |
| *** brinzhang_ has quit IRC | 12:24 | |
| *** derekh has quit IRC | 12:24 | |
| *** martinkennelly has joined #openstack-nova | 12:28 | |
| gibi | sean-k-mooney: yeah, it is a race between libvirt detaching the device and nova unpluging the vif (and reseting the MAC), If I move the unplug after the detach in the nova code then the VF MAC and VLAN is reset properly after the macvtap port is detached | 12:35 |
| sean-k-mooney | i have been wondering if we should do unplug twice | 12:36 |
| sean-k-mooney | we generally wantto disconnect the device form the network backend before removing it form the vm | 12:36 |
| sean-k-mooney | but just doing it once might be fine too | 12:36 |
| lyarwood | stephenfin: LGTM btw | 12:36 |
| sean-k-mooney | at the end | 12:36 |
| sean-k-mooney | gibi: did you confirm it was libvirt by stoping the sriov nic agent? | 12:37 |
| *** udesale has quit IRC | 12:37 | |
| *** udesale has joined #openstack-nova | 12:38 | |
| sean-k-mooney | gibi: or did you manage to find a log message | 12:38 |
| gibi | sean-k-mooney: stopping the neutron nic agent did not solved the race so I assumed it is libvirt | 12:42 |
| sean-k-mooney | yep makes sense to me | 12:42 |
| sean-k-mooney | i saw in the libvirtd log that it does set the mac a number of time sbut i dont have the devstack logs to corralte the timestamps | 12:43 |
| alex_xu | efried: do you know what is the usecase for this https://review.opendev.org/#/c/693414/3/specs/ussuri/approved/provider-config-file.rst@247 | 12:43 |
| gibi | sean-k-mooney: I will add a separate patch into the series that moves the unplug | 12:45 |
| efried | alex_xu: Yes. | 12:45 |
| efried | The idea there was that you could define a "default" rule to apply to all your compute nodes, but then override it for specific ones. | 12:45 |
| efried | Used for ironic, but also in cases where you want to have your rules centralized and ansibled out to the hosts. | 12:47 |
| alex_xu | efried: I see, thanks | 12:47 |
| *** nweinber has joined #openstack-nova | 12:48 | |
| alex_xu | efried: is there any reason we should ignore the addtional inventories and traits when conflict the virt driver managed ones, instead of error out the conflict? | 12:48 |
| *** JamesBenson has quit IRC | 12:49 | |
| efried | We debated this at design time. I can tell you for sure the answer to your question is "yes". But I can't remember exactly why :P | 12:49 |
| alex_xu | efried: ok, so the result is ignore, not the error out, right? I saw the code is error out, not ignore. | 12:50 |
| efried | Oh, whatever the design says is what we decided on. | 12:51 |
| *** sapd1_x has joined #openstack-nova | 12:51 | |
| alex_xu | efried: ok, thanks :) I'm not going dump our the history | 12:51 |
| *** sapd1 has quit IRC | 12:51 | |
| efried | For this issue, I'm reasonably sure whatever is in the design is going to be there because it's what we decided on, not because we accidentally missed it. | 12:51 |
| alex_xu | efried: ok, cool | 12:52 |
| efried | https://specs.openstack.org/openstack/nova-specs/specs/victoria/approved/provider-config-file.html#provider-config-consumption-from-nova says "ignore" under "Provider Tree Merging". | 12:53 |
| alex_xu | efried: yes, that is what I read also | 12:55 |
| alex_xu | I can't thinking of a reason the different between ignore and error out also | 12:56 |
| efried | alex_xu: here's an explanation of that other thing https://review.opendev.org/#/c/693414/3/specs/ussuri/approved/provider-config-file.rst@137 | 12:56 |
| alex_xu | nice | 12:58 |
| efried | alex_xu: https://review.opendev.org/#/c/612497/12/specs/train/approved/provider-config-file.rst@204 | 13:00 |
| efried | I remember now: | 13:00 |
| efried | The conflicts in question should error on startup, but be ignored thereafter. | 13:00 |
| *** rcernin has joined #openstack-nova | 13:00 | |
| alex_xu | efried: for the case, there are virt driver managed inventory or trait show up later? | 13:03 |
| * alex_xu is reading the comment, found something complex | 13:04 | |
| *** derekh has joined #openstack-nova | 13:09 | |
| *** priteau has joined #openstack-nova | 13:09 | |
| *** iurygregory has quit IRC | 13:12 | |
| *** rcernin has quit IRC | 13:15 | |
| *** xek has joined #openstack-nova | 13:18 | |
| *** iurygregory has joined #openstack-nova | 13:18 | |
| *** udesale has quit IRC | 13:21 | |
| *** udesale has joined #openstack-nova | 13:22 | |
| alex_xu | efried: virt driver's update_provider_tree always overwrited provider tree's inventory. so the conflict will be found at startup, and there won't be any conflict after startup. so the code feel like right | 13:26 |
| *** mlycka has quit IRC | 13:29 | |
| efried | Yeah, I thought there was a theoretical edge case where we could come across an error at runtime. The chances were very small, but we wanted to make sure we didn't crater the driver if it did happen. I think that's what led to the design decision as it stands. | 13:30 |
| *** raildo has quit IRC | 13:35 | |
| sean-k-mooney | can there be a conflict if i make a call to placement directly and modify the inventory | 13:35 |
| sean-k-mooney | i know you are not really ment to do that but users be users and they dont always do what we tell them | 13:36 |
| efried | Heh. "You just voided your warranty. You're on your own." | 13:36 |
| sean-k-mooney | have you started getting support request form customer yet | 13:37 |
| sean-k-mooney | they do that alot... | 13:38 |
| sean-k-mooney | then ask us to fix it anyway | 13:38 |
| *** raildo has joined #openstack-nova | 13:38 | |
| efried | of course. | 13:39 |
| efried | Unless you really break things, any manual change to the placement inventory ought to be scrubbed back out on the next periodic. | 13:40 |
| sean-k-mooney | we had one customer that for example when they wanted to spawn a vm on specifci core on a host, stoped nova-compute, updated the vcpu_pin_set to only have those cores, booted the vm with --avaiablity-zone <zone>:<host> and then complained that if someone did a concurent operation on another vm on the host it could cause issue with pinning ... | 13:40 |
| efried | You could probably add things that neither the compute nor the config care about, and they would stick around. Probably. But that won't break the code, I don't think. | 13:41 |
| sean-k-mooney | efried: ya it should heal on the next run | 13:41 |
| sean-k-mooney | if you create your own RP it defnitly shoudl be ok | 13:41 |
| *** ratailor has quit IRC | 13:42 | |
| sean-k-mooney | if you add inventories to one of nova's RPs via the api well thats not allowed so nova is free to delete it | 13:42 |
| *** raildo has quit IRC | 13:43 | |
| sean-k-mooney | im not sure if we actully will delete the inventory but we are allowed too | 13:43 |
| *** raildo has joined #openstack-nova | 13:43 | |
| *** raildo has quit IRC | 13:44 | |
| efried | It's been a hot minute since I looked at update_from_provider_tree, but I think we would delete it, yes. | 13:44 |
| *** raildo has joined #openstack-nova | 13:44 | |
| sean-k-mooney | alex_xu: by the way i reworked https://review.opendev.org/#/c/739131/ after your comments in version 5, im hoping stephenfin will get back to it to be the second +2 later today but just an fyi incase you want to look at it before then. | 13:46 |
| *** k_mouza has joined #openstack-nova | 13:46 | |
| *** raildo_ has joined #openstack-nova | 13:46 | |
| sean-k-mooney | efried: this is basicaly why we are providing the provider.yaml i.e. to enable a supported way to do this so ya. | 13:46 |
| sean-k-mooney | efried: hows openshift land going? | 13:47 |
| *** kaisers has joined #openstack-nova | 13:47 | |
| efried | correct | 13:49 |
| *** dave-mccowan has joined #openstack-nova | 13:49 | |
| efried | Things are going well. After a couple months of pretty serious culture shock and vertical learning curve, I'm getting my feet under me. | 13:49 |
| *** raildo_ has quit IRC | 13:50 | |
| efried | I've written an operator (in go) and feel pretty comfortable navigating openshift/kube APIs. | 13:50 |
| sean-k-mooney | cool, i have read some go but havent really written any, never had the need. | 13:51 |
| sean-k-mooney | i am vaguly aware of what operators do but never looked under the cover to see how they are implemented | 13:52 |
| efried | With existing tools (operator-sdk) and any kind of sweng background, they're ridiculously easy to write. | 13:52 |
| sean-k-mooney | i belive there are some frameworks to implement them without go too using declaritive yaml files referencing CRDs too right | 13:52 |
| sean-k-mooney | i know you can go the custom contoler route too in go | 13:53 |
| *** swp20 has joined #openstack-nova | 13:53 | |
| sean-k-mooney | but again more or less have just read the docs and then done nothing with that info | 13:53 |
| sean-k-mooney | /read/skimmed/ | 13:54 |
| efried | The CRDs are how the operators extend the kube API. Instances of those (CRs) are how you trigger the operator to do its thing. I'm not aware of a way to "declare" an operator with just yaml. | 13:54 |
| efried | And yes, an operator is effectively a custom controller that follows some rules. | 13:54 |
| * sean-k-mooney read is too strong an implication for what i actully did | 13:54 | |
| *** jangutter_ has joined #openstack-nova | 13:55 | |
| sean-k-mooney | efried: are you invovled with the current experiments to deploy openstack using operators | 13:56 |
| efried | I don't get anywhere near openstack anymore. | 13:56 |
| sean-k-mooney | there is an experiment to run openstack on openshift the repo is public somehere | 13:56 |
| *** jangutter has quit IRC | 13:58 | |
| *** JamesBenson has joined #openstack-nova | 13:59 | |
| *** Liang__ has joined #openstack-nova | 13:59 | |
| *** Liang__ is now known as LiangFang | 14:00 | |
| *** dave-mccowan has quit IRC | 14:00 | |
| sean-k-mooney | efried: https://github.com/openstack-k8s-operators | 14:00 |
| efried | cool | 14:01 |
| sean-k-mooney | efried: mdbooth and other have been working on that for a while | 14:01 |
| *** k_mouza has quit IRC | 14:02 | |
| sean-k-mooney | efried: also this is what i was thining of https://kudo.dev/ | 14:03 |
| *** k_mouza has joined #openstack-nova | 14:04 | |
| *** ociuhandu has quit IRC | 14:04 | |
| *** ociuhandu has joined #openstack-nova | 14:05 | |
| sean-k-mooney | efried: i was considring writing an operator for deploying zuul at one point and i was looking at kudo amoung other things as a way to do it | 14:05 |
| *** xek has quit IRC | 14:06 | |
| sean-k-mooney | well zuul, nodepoll , zookeyper, german and gerrit | 14:06 |
| sean-k-mooney | but i just wrote hte manifest by had instead. | 14:07 |
| *** ociuhandu has quit IRC | 14:10 | |
| openstackgerrit | sean mooney proposed openstack/os-vif master: support pyroute2 0.5.13 https://review.opendev.org/744809 | 14:16 |
| *** links has quit IRC | 14:16 | |
| openstackgerrit | Merged openstack/nova master: Document nova in tree virt drivers https://review.opendev.org/740061 | 14:22 |
| *** JamesBen_ has joined #openstack-nova | 14:27 | |
| *** JamesBenson has quit IRC | 14:30 | |
| *** redrobot has joined #openstack-nova | 14:32 | |
| *** brinzhang_ has joined #openstack-nova | 14:36 | |
| *** dklyle has joined #openstack-nova | 14:37 | |
| *** mlavalle has joined #openstack-nova | 14:37 | |
| *** brinzhang0 has quit IRC | 14:39 | |
| *** jangutter has joined #openstack-nova | 14:46 | |
| *** JamesBenson has joined #openstack-nova | 14:46 | |
| *** JamesBen_ has quit IRC | 14:48 | |
| *** jangutter_ has quit IRC | 14:50 | |
| *** mvorwerk_ has quit IRC | 14:54 | |
| *** xek has joined #openstack-nova | 14:54 | |
| *** mvorwerk has joined #openstack-nova | 14:56 | |
| *** JamesBenson has quit IRC | 14:56 | |
| *** psachin has quit IRC | 14:57 | |
| *** mvorwerk has quit IRC | 15:02 | |
| *** LiangFang has quit IRC | 15:13 | |
| *** mkrai has joined #openstack-nova | 15:15 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: nova-multinode-evacuate https://review.opendev.org/744883 | 15:17 |
| *** JamesBenson has joined #openstack-nova | 15:18 | |
| *** tosky has quit IRC | 15:31 | |
| *** lbragstad_ is now known as lbragstad | 15:42 | |
| *** slaweq_ has joined #openstack-nova | 15:45 | |
| *** slaweq has quit IRC | 15:45 | |
| *** slaweq_ is now known as slaweq | 15:45 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move equality check into LibvirtConfigGuestInterface https://review.opendev.org/744524 | 15:53 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Remove unused vpn param from allocate_for_instance https://review.opendev.org/744933 | 15:53 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: [WIP] Support SRIOV interface attach and detach https://review.opendev.org/740995 | 15:54 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Only unplug vif after the device is detached from libvirt https://review.opendev.org/744934 | 15:54 |
| *** dave-mccowan has joined #openstack-nova | 15:54 | |
| *** xek_ has joined #openstack-nova | 15:57 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM: add logging for device matching https://review.opendev.org/744936 | 15:57 |
| *** maciejjozefczyk has quit IRC | 15:59 | |
| *** xek has quit IRC | 16:00 | |
| openstackgerrit | Harshavardhan Metla proposed openstack/nova stable/rocky: [stable-only] Moved the quoted section https://review.opendev.org/744681 | 16:04 |
| melwitt | lyarwood: I just barely started looking at ceph job failures on stable/queens, saw this error message "libvirtError: internal error: unable to execute QEMU command 'device_add': Property 'virtio-blk-device.drive' can't find value 'drive-virtio-disk1'" https://zuul.opendev.org/t/openstack/build/fa85e5eeb1f84389b5dd259c8a829552/log/controller/logs/screen-n-cpu.txt#32824 | 16:05 |
| melwitt | does that ring any bells to you? | 16:06 |
| *** hamalq has joined #openstack-nova | 16:08 | |
| melwitt | hm nvm maybe https://ask.openstack.org/en/question/95328/ceph-cinder-attach-volume-to-running-instance | 16:09 |
| *** hamalq has quit IRC | 16:10 | |
| *** JamesBenson has quit IRC | 16:10 | |
| *** hamalq has joined #openstack-nova | 16:11 | |
| *** JamesBenson has joined #openstack-nova | 16:15 | |
| openstackgerrit | Harshavardhan Metla proposed openstack/nova stable/rocky: "[stable-only]" Removed the quoted section https://review.opendev.org/744681 | 16:21 |
| *** mkrai has quit IRC | 16:24 | |
| stephenfin | melwitt, bauzas: not sure if you'll know this off the top of your head, but should we be releasing resources before we confirm a resize? | 16:26 |
| *** dave-mccowan has quit IRC | 16:26 | |
| *** markvoelker has joined #openstack-nova | 16:27 | |
| stephenfin | Based on https://review.opendev.org/#/c/641806/21, it seems we do (via the resource tracker's periodic task run) | 16:27 |
| melwitt | stephenfin: releasing as in deleting allocations on the source before confirming? AFAIK no, supposed to hold allocations on the source and dest until confirm/revert | 16:28 |
| *** tosky has joined #openstack-nova | 16:29 | |
| *** dougsz has quit IRC | 16:30 | |
| melwitt | I think that patch's commit message is saying prior to that patch, we were reporting resources held on the source *after* a confirm resize (until the next periodic run) whereas after a confirm, the resources should be shown as released on the source. but only after the confirm, not before | 16:31 |
| *** udesale has quit IRC | 16:32 | |
| stephenfin | ah, that makes sense | 16:32 |
| stephenfin | so now we're freeing those resources immediately | 16:32 |
| melwitt | immediately upon a confirm, yeah | 16:33 |
| melwitt | or rather reporting them immediately after a confirm. the claim has always been dropped at the time of confirm, but apparently the reporting was delayed potentially by one periodic task run | 16:34 |
| stephenfin | okay, cool. There's a bug in it that I'm working on (https://bugs.launchpad.net/nova/+bug/1879878, fwiw); tl;dr: if the periodic runs between the API request and the 'confirm_resize' call in the RT, the ComputeNode.numa_topology gets out of whack | 16:34 |
| openstack | Launchpad bug 1879878 in OpenStack Compute (nova) "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Medium,Confirmed] - Assigned to Stephen Finucane (stephenfinucane) | 16:34 |
| melwitt | ah, gotcha | 16:34 |
| melwitt | oh interesting, so if the periodic task fires while confirm resize is still processing, it turns the instance to ERROR | 16:36 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Move equality check into LibvirtConfigGuestInterface https://review.opendev.org/744524 | 16:37 |
| stephenfin | yeah, the '_update_available_resource' function of the RT seems to regenerate ComputeNode objects from scratch, including the embedded numa_topology | 16:37 |
| *** brinzhang0 has joined #openstack-nova | 16:37 | |
| stephenfin | and it doesn't seem to be accounting from the still-present allocations on the host once the instance has been confirmed | 16:37 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: Only unplug vif after the device is detached from libvirt https://review.opendev.org/744934 | 16:38 |
| *** dtantsur is now known as dtantsur|afk | 16:39 | |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: [WIP] Support SRIOV interface attach and detach https://review.opendev.org/740995 | 16:39 |
| openstackgerrit | Balazs Gibizer proposed openstack/nova master: DNM: add logging for device matching https://review.opendev.org/744936 | 16:39 |
| *** brinzhang_ has quit IRC | 16:40 | |
| melwitt | stephenfin: note that sean-k-mooney's comments are correct though, that if this is in a version older than claims in placement, races are expected (especially since we don't have numa in placement). in your repro, is that with cpu_dedicated_set and on the master branch? | 16:42 |
| *** maciejjozefczyk has joined #openstack-nova | 16:42 | |
| gibi | sean-k-mooney: added a patch top of the sriov_attach series with extra logging that can help you figuring out why the macvtap and direct-physical devices are not detached for you | 16:42 |
| gibi | sean-k-mooney: https://review.opendev.org/744936 | 16:42 |
| gibi | also added a patch to move the vif unplug after libvirt detach | 16:42 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add reproducer for bug #1879878 https://review.opendev.org/744950 | 16:43 |
| openstack | bug 1879878 in OpenStack Compute (nova) "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Medium,Confirmed] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane) | 16:43 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: TODO https://review.opendev.org/744951 | 16:43 |
| stephenfin | melwitt: ^ | 16:43 |
| stephenfin | I can reproduce trivially with new-style configuration | 16:43 |
| melwitt | while using cpu_dedicated_set? /me looks | 16:43 |
| stephenfin | yup | 16:43 |
| melwitt | ok I see | 16:43 |
| stephenfin | I think I'm close to a fix at least. Worst case scenario, we put mriedem's stuff inside a conditional to only run if we still know that it exists | 16:45 |
| stephenfin | i.e. if it's in the list of tracked migrations and instances | 16:45 |
| stephenfin | though I haven't figured out what that will leak yet | 16:46 |
| stephenfin | tbd | 16:46 |
| melwitt | sorry, what's the proposed fix? being that it's intended that resources be held on source and dest until confirm or revert | 16:47 |
| stephenfin | https://github.com/openstack/nova/blob/master/nova/compute/resource_tracker.py#L579-L591 | 16:48 |
| *** gyee has joined #openstack-nova | 16:48 | |
| stephenfin | we only do that if the instance appears in our list of tracked_migrations or tracked_instances | 16:48 |
| *** nweinber has quit IRC | 16:49 | |
| stephenfin | afaict, '_update_usage_from_instances' wipes those and regenerates them. When we confirm the resize, the instance is marked as deleted, which means it's not picked up by '_update_usage_from_instances' | 16:50 |
| stephenfin | so the race is between that (the instance getting marked as deleted and '_update_usage_from_instances' running) and the call to 'drop_move_claim' | 16:51 |
| stephenfin | I _think_ | 16:51 |
| sean-k-mooney | gibi: thanks ill restart with that now and see how it works | 16:58 |
| *** derekh has quit IRC | 17:00 | |
| melwitt | stephenfin: this is interesting bc the commit message of mriedem's change says "This fixes the issue by also updating usage in drop_move_claim when the instance is not in tracked_migrations but is in tracked_instances." (which would do part of what you're suggesting) but it appears not to be what is in the code | 17:00 |
| gibi | sean-k-mooney: thanks. I will end my day about now but I will be back tomorrow morning | 17:00 |
| sean-k-mooney | stephenfin: sound plausible i had not got to the point of diging into how the race happens | 17:00 |
| sean-k-mooney | gibi: no worries | 17:00 |
| melwitt | stephenfin: oh, nvm, I'm misreading this... it's removing usage by removing the instance from tracked_instances and then updating usage | 17:03 |
| sean-k-mooney | melwitt: stephenfin if this race is happening the way you say | 17:08 |
| sean-k-mooney | would this also happen with the old config options? | 17:09 |
| stephenfin | sean-k-mooney: I think so | 17:09 |
| sean-k-mooney | ok its just more obvious now? | 17:09 |
| stephenfin | I think the bug is with that patch from mriedem | 17:09 |
| sean-k-mooney | or they were just unlucky | 17:09 |
| stephenfin | I don't think it's anything to do with the PCPU work | 17:09 |
| sean-k-mooney | ok | 17:09 |
| stephenfin | tbc, it's still a working theory but it would be easy prove out by backporting that fix to e.g. stable/stein where none of that code is present | 17:10 |
| sean-k-mooney | i think the upgrade procedure we proposed for our downstream customer still makes sense | 17:10 |
| *** hamalq has quit IRC | 17:10 | |
| sean-k-mooney | but i think they will need the backport of your two fixes | 17:10 |
| sean-k-mooney | the one for this race | 17:10 |
| sean-k-mooney | and the one for the isolate on smt host with new config options | 17:10 |
| sean-k-mooney | right? | 17:10 |
| *** hamalq has joined #openstack-nova | 17:11 | |
| stephenfin | the isolate on SMT fix is mostly unrelated and should be backported regardless | 17:11 |
| stephenfin | still unsure about this race | 17:12 |
| sean-k-mooney | ok | 17:12 |
| *** hamalq has quit IRC | 17:18 | |
| *** ociuhandu has joined #openstack-nova | 17:18 | |
| *** nightmare_unreal has quit IRC | 17:20 | |
| *** ociuhandu has quit IRC | 17:22 | |
| *** jdillaman has quit IRC | 17:23 | |
| *** brinzhang_ has joined #openstack-nova | 17:26 | |
| *** brinzhang0 has quit IRC | 17:28 | |
| *** janno has quit IRC | 17:31 | |
| *** janno has joined #openstack-nova | 17:33 | |
| *** brinzhang0 has joined #openstack-nova | 17:35 | |
| *** brinzhang has joined #openstack-nova | 17:37 | |
| *** priteau has quit IRC | 17:38 | |
| *** brinzhang_ has quit IRC | 17:39 | |
| *** brinzhang0 has quit IRC | 17:39 | |
| *** nweinber has joined #openstack-nova | 17:40 | |
| *** k_mouza has quit IRC | 17:43 | |
| *** hamalq has joined #openstack-nova | 17:44 | |
| *** hamalq has quit IRC | 17:45 | |
| *** hamalq has joined #openstack-nova | 17:46 | |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: tests: Add reproducer for bug #1879878 https://review.opendev.org/744950 | 17:51 |
| openstack | bug 1879878 in OpenStack Compute (nova) "VM become Error after confirming resize with Error info CPUUnpinningInvalid on source node " [Medium,Confirmed] https://launchpad.net/bugs/1879878 - Assigned to Stephen Finucane (stephenfinucane) | 17:51 |
| openstackgerrit | Stephen Finucane proposed openstack/nova master: Don't unset Instance.old_flavor, new_flavor until necessary https://review.opendev.org/744958 | 17:51 |
| stephenfin | melwitt: I think that's the fix. Could you sanity check at some point? | 17:51 |
| melwitt | stephenfin: yeah I'll take a look | 17:51 |
| stephenfin | dansmith: You should probably look too ^ It's a variant of a patch I have for vTPM and as I think I've mentioned before, your name is all over the code I'm touching | 17:52 |
| *** bbowen has quit IRC | 17:59 | |
| *** raildo has quit IRC | 18:03 | |
| *** raildo has joined #openstack-nova | 18:08 | |
| *** tesseract has quit IRC | 18:09 | |
| *** raildo_ has joined #openstack-nova | 18:11 | |
| *** ralonsoh has quit IRC | 18:12 | |
| *** raildo_ has quit IRC | 18:13 | |
| *** k_mouza has joined #openstack-nova | 18:13 | |
| *** k_mouza has quit IRC | 18:16 | |
| dansmith | okay I'm kinda heads-down on something else right now, but melwitt I guess let me know if you're unsure | 18:36 |
| melwitt | I haven't looked yet but I think I can already say I'm unsure :) | 18:36 |
| *** brinzhang_ has joined #openstack-nova | 18:37 | |
| *** brinzhang has quit IRC | 18:40 | |
| *** raildo has quit IRC | 18:46 | |
| *** raildo has joined #openstack-nova | 18:48 | |
| *** xek_ has quit IRC | 19:06 | |
| *** bbowen has joined #openstack-nova | 19:21 | |
| *** nweinber has quit IRC | 19:35 | |
| *** brinzhang0 has joined #openstack-nova | 19:36 | |
| *** brinzhang_ has quit IRC | 19:39 | |
| *** tosky has quit IRC | 19:39 | |
| lyarwood | stephenfin: https://review.opendev.org/#/c/699291/10 - finally got back to this btw if you can take a look | 19:43 |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: Handle Neutron errors in _post_live_migration() https://review.opendev.org/729763 | 20:11 |
| *** vishalmanchanda has quit IRC | 20:25 | |
| openstackgerrit | Artom Lifshitz proposed openstack/nova master: WIP: Centralize wait_for_unversioned_notification https://review.opendev.org/744985 | 20:29 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: nova-evacuate https://review.opendev.org/744883 | 20:33 |
| *** gyee has quit IRC | 20:36 | |
| *** gyee has joined #openstack-nova | 20:38 | |
| *** raildo has quit IRC | 20:40 | |
| *** raildo has joined #openstack-nova | 20:40 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: nova-evacuate https://review.opendev.org/744883 | 20:41 |
| *** smcginni1 has joined #openstack-nova | 20:47 | |
| *** smcginnis has quit IRC | 20:50 | |
| *** smcginni1 is now known as smcginnis | 20:50 | |
| *** ociuhandu has joined #openstack-nova | 20:51 | |
| *** ociuhandu has quit IRC | 20:56 | |
| *** raildo has quit IRC | 20:57 | |
| lyarwood | melwitt: https://review.opendev.org/#/c/743319/ - would you mind +W'ing that again as CI is finally green again | 20:58 |
| * lyarwood wonders if it's okay for him to just +W it in this case if it has already been +W'd in the past? | 20:58 | |
| melwitt | lyarwood: yeah, I will | 20:59 |
| *** maciejjozefczyk has quit IRC | 21:00 | |
| lyarwood | thanks :) | 21:01 |
| melwitt | np | 21:03 |
| openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP zuul: nova-evacuate https://review.opendev.org/744883 | 21:08 |
| openstackgerrit | Sean McGinnis proposed openstack/nova master: Add lsscsi to bindep https://review.opendev.org/744992 | 21:17 |
| *** slaweq has quit IRC | 21:18 | |
| smcginnis | Needed for an os-brick change in the latest release ^ | 21:19 |
| *** xek has joined #openstack-nova | 21:19 | |
| *** gyee has quit IRC | 21:24 | |
| sean-k-mooney | smcginnis: should that not be listed in os-bricks bindep | 21:24 |
| sean-k-mooney | not novas | 21:25 |
| *** tosky has joined #openstack-nova | 21:25 | |
| *** gyee has joined #openstack-nova | 21:25 | |
| sean-k-mooney | smcginnis: nova's unit tests should be mocking any calls to os-brick | 21:25 |
| sean-k-mooney | smcginnis: os-brick is a deliverable of cinder not nova so if it was to be added to any project for devstack would it not be better to add it to cinders bindep | 21:27 |
| smcginnis | smcginnis: It is in os-brick's bindep, but it turns out when we install libs from their released version, that doesn't do us any good. | 21:28 |
| smcginnis | sean-k-mooney: Hah, oops. Talking to myself. :) | 21:28 |
| sean-k-mooney | it happen i am told its only a problem if you are surpised by the answer | 21:29 |
| smcginnis | sean-k-mooney: I was able to get a change in devstack so it will use bindep when installing from source (it didn't before) but still nothing to address this case. | 21:29 |
| smcginnis | ;) | 21:29 |
| sean-k-mooney | well im wondier if you shoudl be adding this do cinder | 21:29 |
| openstackgerrit | Merged openstack/nova stable/train: Silence amqp heartbeat warning https://review.opendev.org/728057 | 21:30 |
| sean-k-mooney | i mean at a minium it probaly should be lsscsi [cinder] | 21:30 |
| smcginnis | That one is in https://review.opendev.org/#/c/743291/ | 21:30 |
| smcginnis | Ah, didn't see there was a profile for that. | 21:30 |
| sean-k-mooney | well there proably isnt | 21:30 |
| sean-k-mooney | im suggesing adding one | 21:30 |
| sean-k-mooney | its not a dep of nova when not using cinder right | 21:31 |
| smcginnis | So then zuul playbooks would also need to be updated to use that profile. | 21:31 |
| sean-k-mooney | just if you are using os-brick ? | 21:31 |
| smcginnis | Yeah, so probably only applicable to compute nodes. | 21:31 |
| sean-k-mooney | ya only compute nodes and if cinder is deployed | 21:31 |
| sean-k-mooney | what you propose is probaly ok but just said i would ask since its not really a dep of nova | 21:32 |
| smcginnis | Yeah, makes sense. | 21:32 |
| smcginnis | I don't have time now, but I may be able to follow up later to make it better. | 21:33 |
| sean-k-mooney | we are not really strick about listing the min dpes in bindeps | 21:33 |
| smcginnis | It is a small package, so at least it's not pulling down the world for this. | 21:33 |
| sean-k-mooney | ya | 21:33 |
| sean-k-mooney | i would kind of prefer if we use profiles more | 21:33 |
| sean-k-mooney | e.g. add a mysql and postgress profile | 21:33 |
| smcginnis | That could speed things up overall if we did. | 21:33 |
| smcginnis | I wonder how many places we would need to update playbooks now though. :/ | 21:34 |
| sean-k-mooney | we would need a way to pass info from the job in a declaritive way | 21:34 |
| sean-k-mooney | if we used it more optimally where we have a profile for each of the configurable backend a project used and then listed the profiles that correstpond to the deployment we ar testing that would be nice but also a lot of work :) | 21:35 |
| *** rcernin has joined #openstack-nova | 21:36 | |
| smcginnis | Yeah. Good idea though. | 21:36 |
| *** rcernin has quit IRC | 21:36 | |
| *** brinzhang_ has joined #openstack-nova | 21:36 | |
| *** rcernin has joined #openstack-nova | 21:36 | |
| *** brinzhang0 has quit IRC | 21:39 | |
| openstackgerrit | Merged openstack/nova master: Removed the host FQDN from the exception message https://review.opendev.org/743950 | 21:42 |
| *** xek has quit IRC | 21:47 | |
| *** markmcclain has quit IRC | 21:53 | |
| *** rcernin has quit IRC | 22:03 | |
| *** rcernin has joined #openstack-nova | 22:17 | |
| *** martinkennelly has quit IRC | 22:26 | |
| *** jmlowe has quit IRC | 22:27 | |
| *** jmlowe has joined #openstack-nova | 22:31 | |
| *** rcernin has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-nova | 22:40 | |
| *** rcernin has quit IRC | 22:40 | |
| *** rcernin has joined #openstack-nova | 22:44 | |
| *** artom has quit IRC | 22:45 | |
| *** artom has joined #openstack-nova | 22:46 | |
| openstackgerrit | Merged openstack/nova master: compute: Don't delete the original attachment during pre LM rollback https://review.opendev.org/743319 | 22:53 |
| openstackgerrit | Merged openstack/nova master: func: Add CinderFixture to _IntegratedTestBase https://review.opendev.org/743535 | 22:54 |
| *** tkajinam has joined #openstack-nova | 23:00 | |
| *** mlavalle has quit IRC | 23:09 | |
| *** ociuhandu has joined #openstack-nova | 23:10 | |
| *** markmcclain has joined #openstack-nova | 23:12 | |
| *** tosky has quit IRC | 23:14 | |
| *** ociuhandu has quit IRC | 23:16 | |
| *** irclogbot_1 has quit IRC | 23:34 | |
| *** irclogbot_3 has joined #openstack-nova | 23:38 | |
| *** yoctozepto3 has joined #openstack-nova | 23:44 | |
| *** yoctozepto has quit IRC | 23:45 | |
| *** yoctozepto3 is now known as yoctozepto | 23:45 | |
| openstackgerrit | Lee Yarwood proposed openstack/nova stable/ussuri: compute: Don't delete the original attachment during pre LM rollback https://review.opendev.org/744162 | 23:56 |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!