*** macz has quit IRC | 00:02 | |
*** dave-mccowan has quit IRC | 00:10 | |
*** slaweq has joined #openstack-nova | 00:11 | |
*** dave-mccowan has joined #openstack-nova | 00:15 | |
*** slaweq has quit IRC | 00:16 | |
*** KeithMnemonic has joined #openstack-nova | 00:16 | |
*** KeithMnemonic has quit IRC | 00:17 | |
*** KeithMnemonic has joined #openstack-nova | 00:17 | |
*** KeithMnemonic has quit IRC | 00:23 | |
*** igordc has quit IRC | 00:24 | |
*** igordc has joined #openstack-nova | 00:25 | |
*** slaweq has joined #openstack-nova | 00:28 | |
*** igordc has quit IRC | 00:32 | |
*** slaweq has quit IRC | 00:33 | |
*** ccstone has quit IRC | 00:33 | |
*** mlavalle has quit IRC | 00:40 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Expose instance action event details out of the API https://review.opendev.org/694430 | 00:43 |
---|---|---|
*** mriedem has quit IRC | 00:45 | |
*** slaweq has joined #openstack-nova | 00:50 | |
*** TxGirlGeek has quit IRC | 00:52 | |
*** slaweq has quit IRC | 00:55 | |
*** nanzha has joined #openstack-nova | 00:59 | |
*** ileixe has joined #openstack-nova | 01:00 | |
*** dave-mccowan has quit IRC | 01:01 | |
*** ociuhandu has joined #openstack-nova | 01:01 | |
*** TxGirlGeek has joined #openstack-nova | 01:05 | |
*** slaweq has joined #openstack-nova | 01:06 | |
*** rpittau|afk has quit IRC | 01:11 | |
*** mnasiadka has quit IRC | 01:11 | |
*** rpittau|afk has joined #openstack-nova | 01:11 | |
*** TxGirlGeek has quit IRC | 01:11 | |
*** guilhermesp has quit IRC | 01:11 | |
*** Li_Liu has quit IRC | 01:12 | |
*** lamt has quit IRC | 01:12 | |
*** TheJulia has quit IRC | 01:12 | |
*** icey has quit IRC | 01:12 | |
*** mnasiadka has joined #openstack-nova | 01:12 | |
*** icey has joined #openstack-nova | 01:13 | |
*** Li_Liu has joined #openstack-nova | 01:13 | |
*** TheJulia has joined #openstack-nova | 01:13 | |
*** guilhermesp has joined #openstack-nova | 01:13 | |
*** jkulik has quit IRC | 01:14 | |
*** slaweq has quit IRC | 01:15 | |
*** jkulik has joined #openstack-nova | 01:17 | |
*** artom has joined #openstack-nova | 01:17 | |
*** slaweq has joined #openstack-nova | 01:18 | |
*** Liang__ has joined #openstack-nova | 01:18 | |
*** Liang__ has quit IRC | 01:20 | |
*** ociuhandu has quit IRC | 01:23 | |
*** slaweq has quit IRC | 01:23 | |
*** zhanglong has quit IRC | 01:36 | |
*** slaweq has joined #openstack-nova | 01:37 | |
*** ociuhandu has joined #openstack-nova | 01:38 | |
*** zhanglong has joined #openstack-nova | 01:38 | |
*** ociuhandu has quit IRC | 01:42 | |
*** slaweq has quit IRC | 01:45 | |
*** slaweq has joined #openstack-nova | 01:52 | |
*** slaweq has quit IRC | 01:56 | |
*** ociuhandu has joined #openstack-nova | 02:02 | |
*** openstackstatus has joined #openstack-nova | 02:03 | |
*** ChanServ sets mode: +v openstackstatus | 02:03 | |
*** ociuhandu has quit IRC | 02:06 | |
*** ociuhandu has joined #openstack-nova | 02:07 | |
*** prince_nana has joined #openstack-nova | 02:10 | |
*** slaweq has joined #openstack-nova | 02:11 | |
*** prince_nana has quit IRC | 02:11 | |
*** ociuhandu has quit IRC | 02:12 | |
*** chenhaw has joined #openstack-nova | 02:12 | |
*** slaweq has quit IRC | 02:15 | |
*** chenhaw has quit IRC | 02:17 | |
openstackgerrit | Merged openstack/nova master: Move compute_node_to_inventory_dict to test-only code https://review.opendev.org/693438 | 02:20 |
openstackgerrit | Merged openstack/nova master: Remove get_minimum_version mocks from test_resource_tracker https://review.opendev.org/693439 | 02:20 |
openstackgerrit | Merged openstack/nova stable/queens: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/694357 | 02:20 |
*** ociuhandu has joined #openstack-nova | 02:26 | |
*** jbernard has joined #openstack-nova | 02:31 | |
*** ociuhandu has quit IRC | 02:36 | |
*** jbernard has quit IRC | 02:50 | |
*** jbernard_ has joined #openstack-nova | 02:50 | |
*** jbernard_ has quit IRC | 02:51 | |
*** igordc has joined #openstack-nova | 02:53 | |
*** artom has quit IRC | 02:54 | |
*** artom has joined #openstack-nova | 02:54 | |
*** artom has quit IRC | 02:55 | |
*** gyee has quit IRC | 02:55 | |
*** zzzeek has quit IRC | 02:56 | |
*** zzzeek has joined #openstack-nova | 02:57 | |
*** zzzeek has quit IRC | 02:58 | |
*** zzzeek has joined #openstack-nova | 02:59 | |
openstackgerrit | Merged openstack/nova master: Stop using NoAuthMiddleware in tests https://review.opendev.org/687416 | 02:59 |
*** tbachman has quit IRC | 03:00 | |
*** ociuhandu has joined #openstack-nova | 03:07 | |
openstackgerrit | Merged openstack/nova master: Remove TODO from ComputeTaskManager._live_migrate https://review.opendev.org/693696 | 03:10 |
*** ociuhandu has quit IRC | 03:12 | |
*** ociuhandu has joined #openstack-nova | 03:13 | |
*** chenhaw has joined #openstack-nova | 03:14 | |
*** francoisp has quit IRC | 03:19 | |
*** abaindur has quit IRC | 03:22 | |
*** zhubx has joined #openstack-nova | 03:35 | |
*** ociuhandu has quit IRC | 03:36 | |
*** jbernard has joined #openstack-nova | 03:36 | |
*** boxiang has quit IRC | 03:39 | |
*** bhagyashris has joined #openstack-nova | 03:42 | |
*** JamesBenson has joined #openstack-nova | 03:52 | |
*** chenhaw has quit IRC | 03:54 | |
*** mkrai has joined #openstack-nova | 03:56 | |
*** ociuhandu has joined #openstack-nova | 04:01 | |
*** ociuhandu has quit IRC | 04:06 | |
*** psachin has joined #openstack-nova | 04:16 | |
openstackgerrit | Archit Modi proposed openstack/nova stable/pike: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/694450 | 04:31 |
*** ociuhandu has joined #openstack-nova | 04:40 | |
*** ratailor has joined #openstack-nova | 04:42 | |
*** ociuhandu has quit IRC | 04:45 | |
*** ileixe has quit IRC | 04:46 | |
*** JamesBenson has quit IRC | 04:58 | |
*** ileixe has joined #openstack-nova | 04:58 | |
*** ociuhandu has joined #openstack-nova | 05:00 | |
*** ratailor has quit IRC | 05:04 | |
*** ratailor has joined #openstack-nova | 05:04 | |
*** ileixe has quit IRC | 05:19 | |
*** udesale has joined #openstack-nova | 05:22 | |
*** awalende has joined #openstack-nova | 05:28 | |
*** ociuhandu has quit IRC | 05:28 | |
*** links has joined #openstack-nova | 05:29 | |
*** ociuhandu has joined #openstack-nova | 05:30 | |
*** awalende has quit IRC | 05:33 | |
*** ociuhandu has quit IRC | 05:36 | |
openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Add resources metadata of instance https://review.opendev.org/663563 | 05:41 |
*** zhanglong has quit IRC | 05:43 | |
*** Luzi has joined #openstack-nova | 06:04 | |
*** jbernard has quit IRC | 06:07 | |
*** jbernard has joined #openstack-nova | 06:14 | |
*** ociuhandu has joined #openstack-nova | 06:15 | |
*** jbernard has quit IRC | 06:19 | |
*** aloga_ has joined #openstack-nova | 06:32 | |
*** jbernard has joined #openstack-nova | 06:32 | |
*** gouthamr_ has joined #openstack-nova | 06:35 | |
*** Jeffrey4l_ has joined #openstack-nova | 06:36 | |
*** ociuhandu has quit IRC | 06:37 | |
*** links has quit IRC | 06:37 | |
*** rcernin has quit IRC | 06:37 | |
*** trident has quit IRC | 06:37 | |
*** huaqiang has quit IRC | 06:37 | |
*** tetsuro has quit IRC | 06:37 | |
*** yaawang has quit IRC | 06:39 | |
*** jmlowe has quit IRC | 06:39 | |
*** mgoddard has quit IRC | 06:39 | |
*** johnthetubaguy has quit IRC | 06:39 | |
*** hamzy has quit IRC | 06:39 | |
*** mtreinish has quit IRC | 06:39 | |
*** hoonetorg has quit IRC | 06:39 | |
*** gouthamr has quit IRC | 06:39 | |
*** antonym has quit IRC | 06:39 | |
*** amorin has quit IRC | 06:39 | |
*** brtknr has quit IRC | 06:39 | |
*** aloga has quit IRC | 06:39 | |
*** elod_off has quit IRC | 06:39 | |
*** gryf has quit IRC | 06:39 | |
*** yankcrime has quit IRC | 06:39 | |
*** alex_xu has quit IRC | 06:39 | |
*** Jeffrey4l has quit IRC | 06:39 | |
*** evrardjp has quit IRC | 06:39 | |
*** ryn_eq has quit IRC | 06:39 | |
*** kashyap has quit IRC | 06:39 | |
*** spotz has quit IRC | 06:39 | |
*** ileixe has joined #openstack-nova | 06:39 | |
*** openstackstatus has quit IRC | 06:40 | |
*** kashyap has joined #openstack-nova | 06:42 | |
*** spotz has joined #openstack-nova | 06:42 | |
*** ileixe has quit IRC | 06:45 | |
openstackgerrit | Brin Zhang proposed openstack/nova-specs master: Add resources metadata of instance https://review.opendev.org/663563 | 06:45 |
*** links has joined #openstack-nova | 06:48 | |
*** rcernin has joined #openstack-nova | 06:48 | |
*** trident has joined #openstack-nova | 06:48 | |
*** huaqiang has joined #openstack-nova | 06:48 | |
*** tetsuro has joined #openstack-nova | 06:48 | |
*** yaawang has joined #openstack-nova | 06:48 | |
*** jmlowe has joined #openstack-nova | 06:48 | |
*** mgoddard has joined #openstack-nova | 06:48 | |
*** hamzy has joined #openstack-nova | 06:48 | |
*** mtreinish has joined #openstack-nova | 06:48 | |
*** hoonetorg has joined #openstack-nova | 06:48 | |
*** antonym has joined #openstack-nova | 06:48 | |
*** amorin has joined #openstack-nova | 06:48 | |
*** brtknr has joined #openstack-nova | 06:48 | |
*** elod_off has joined #openstack-nova | 06:48 | |
*** gryf has joined #openstack-nova | 06:48 | |
*** alex_xu has joined #openstack-nova | 06:48 | |
*** evrardjp has joined #openstack-nova | 06:48 | |
*** ryn_eq has joined #openstack-nova | 06:48 | |
*** nanzha has quit IRC | 06:52 | |
*** dpawlik has joined #openstack-nova | 06:59 | |
*** abaindur has joined #openstack-nova | 07:00 | |
*** zhanglong has joined #openstack-nova | 07:00 | |
*** nanzha has joined #openstack-nova | 07:02 | |
*** dpawlik has quit IRC | 07:15 | |
*** links has quit IRC | 07:15 | |
*** rcernin has quit IRC | 07:15 | |
*** trident has quit IRC | 07:15 | |
*** huaqiang has quit IRC | 07:15 | |
*** tetsuro has quit IRC | 07:15 | |
*** yaawang has quit IRC | 07:15 | |
*** jmlowe has quit IRC | 07:15 | |
*** mgoddard has quit IRC | 07:15 | |
*** hamzy has quit IRC | 07:15 | |
*** mtreinish has quit IRC | 07:15 | |
*** hoonetorg has quit IRC | 07:15 | |
*** antonym has quit IRC | 07:15 | |
*** amorin has quit IRC | 07:15 | |
*** brtknr has quit IRC | 07:15 | |
*** elod_off has quit IRC | 07:15 | |
*** gryf has quit IRC | 07:15 | |
*** alex_xu has quit IRC | 07:15 | |
*** evrardjp has quit IRC | 07:15 | |
*** ryn_eq has quit IRC | 07:15 | |
*** links has joined #openstack-nova | 07:16 | |
*** rcernin has joined #openstack-nova | 07:16 | |
*** trident has joined #openstack-nova | 07:16 | |
*** huaqiang has joined #openstack-nova | 07:16 | |
*** tetsuro has joined #openstack-nova | 07:16 | |
*** yaawang has joined #openstack-nova | 07:16 | |
*** jmlowe has joined #openstack-nova | 07:16 | |
*** mgoddard has joined #openstack-nova | 07:16 | |
*** hamzy has joined #openstack-nova | 07:16 | |
*** mtreinish has joined #openstack-nova | 07:16 | |
*** hoonetorg has joined #openstack-nova | 07:16 | |
*** antonym has joined #openstack-nova | 07:16 | |
*** amorin has joined #openstack-nova | 07:16 | |
*** brtknr has joined #openstack-nova | 07:16 | |
*** elod_off has joined #openstack-nova | 07:16 | |
*** gryf has joined #openstack-nova | 07:16 | |
*** alex_xu has joined #openstack-nova | 07:16 | |
*** evrardjp has joined #openstack-nova | 07:16 | |
*** ryn_eq has joined #openstack-nova | 07:16 | |
*** dpawlik has joined #openstack-nova | 07:17 | |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Ignore root_gb if instance is booted from volume https://review.opendev.org/612626 | 07:18 |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Handle new is_volume_backend join column query https://review.opendev.org/694462 | 07:18 |
openstackgerrit | Shilpa Devharakar proposed openstack/nova master: Instance object changes for the new 'is_volume_backed' expected_attr https://review.opendev.org/694463 | 07:18 |
*** chenhaw has joined #openstack-nova | 07:24 | |
*** ociuhandu has joined #openstack-nova | 07:27 | |
*** KeithMnemonic has joined #openstack-nova | 07:29 | |
*** ociuhandu has quit IRC | 07:29 | |
*** ociuhandu has joined #openstack-nova | 07:30 | |
*** KeithMnemonic has quit IRC | 07:34 | |
*** ociuhandu has quit IRC | 07:41 | |
*** ociuhandu has joined #openstack-nova | 07:42 | |
*** ociuhandu has quit IRC | 07:44 | |
*** rcernin has quit IRC | 07:44 | |
*** ociuhandu has joined #openstack-nova | 07:45 | |
*** tbachman has joined #openstack-nova | 07:46 | |
*** slaweq has joined #openstack-nova | 07:46 | |
*** nanzha has quit IRC | 07:47 | |
*** nanzha has joined #openstack-nova | 07:48 | |
*** trident has quit IRC | 07:49 | |
*** slaweq_ has joined #openstack-nova | 07:53 | |
*** ociuhandu has quit IRC | 07:55 | |
*** dpawlik has quit IRC | 07:56 | |
*** slaweq has quit IRC | 07:56 | |
*** rpittau|afk is now known as rpittau | 07:56 | |
*** trident has joined #openstack-nova | 07:58 | |
*** tkajinam has quit IRC | 08:00 | |
*** sridharg has joined #openstack-nova | 08:01 | |
*** ileixe has joined #openstack-nova | 08:02 | |
*** ileixe has quit IRC | 08:02 | |
*** bhagyashris has quit IRC | 08:02 | |
*** slaweq_ is now known as slaweq | 08:03 | |
*** tbachman has quit IRC | 08:03 | |
*** dpawlik has joined #openstack-nova | 08:04 | |
*** ileixe has joined #openstack-nova | 08:04 | |
*** dpawlik has quit IRC | 08:08 | |
bauzas | good morning Nova | 08:08 |
*** dpawlik has joined #openstack-nova | 08:10 | |
*** tesseract has joined #openstack-nova | 08:15 | |
*** bhagyashris has joined #openstack-nova | 08:17 | |
*** damien_r has joined #openstack-nova | 08:17 | |
*** gibi_off has quit IRC | 08:19 | |
*** slaweq_ has joined #openstack-nova | 08:19 | |
*** gibi has joined #openstack-nova | 08:19 | |
*** slaweq has quit IRC | 08:19 | |
*** awalende has joined #openstack-nova | 08:19 | |
*** ralonsoh has joined #openstack-nova | 08:32 | |
*** nanzha has quit IRC | 08:32 | |
*** ociuhandu has joined #openstack-nova | 08:36 | |
*** nanzha has joined #openstack-nova | 08:39 | |
awalende | Hello, I have trouble passing through an nvidia t4 card to my guests on Rocky. nova-compute log reports the final resource as: pci_stats=[PciDevicePool(count=1,numa_node=1,product_id='1eb8',tags={dev_type='type-PF'},vendor_id='10de')] | 08:39 |
awalende | However, when I set the alias as alias={"name":"T4","vendor_id":"10de","product_id":"1eb8"} on the controller and alias={"name":"T4","vendor_id":"10de", "product_id":"1eb8"} on compute node does not work | 08:40 |
awalende | Also I wonder, why it is declared as a "PF". I clearly inserted an PCIe card | 08:41 |
stephenfin | awalende: How have you configured the PCI whitelist in nova.conf? | 08:42 |
awalende | passthrough_whitelist={"vendor_id":"10de"} | 08:42 |
stephenfin | awalende: So this is interesting. The Tesla T4 supports SR-IOV https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-t4/t4-tensor-core-product-brief.pdf | 08:45 |
*** slaweq_ is now known as slaweq | 08:45 | |
*** ivve has joined #openstack-nova | 08:46 | |
stephenfin | and because we're detecting that, we're flagging it as a PF https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L6562-L6565 | 08:46 |
awalende | So there is probably a difference passing them through as other regular gpus? | 08:46 |
awalende | Can we pass them even at all? | 08:47 |
stephenfin | Possibly. I could be wrong, but I do recall that type-PF devices were disabled by default because you typically wouldn't want to use them when you talk about NICs | 08:47 |
stephenfin | but it can be enabled | 08:47 |
stephenfin | do you want to pass through the PF or the VF? | 08:48 |
awalende | the whole hardware | 08:48 |
awalende | I guess probably the PF....so the usecase is, we want to pass the whole card to the guest. so that they have to install nvidia drivers by themselfs on the guest machine | 08:48 |
awalende | like we did with our older Tesla V100 cards | 08:49 |
stephenfin | Seems like you could get better resource utilization if you passed through the VFs though (since apparently that would give you up to 16 devices) | 08:50 |
stephenfin | That doesn't matter for now though | 08:50 |
stephenfin | If you want to use type-PF devices, I _think_ you need to either (a) set the 'device_type' flag in the 'alias' config option or (b) make sure the VFs are not whitelisted | 08:51 |
awalende | ye we already tried to set 'device_type' but it didn't make a difference :( | 08:51 |
stephenfin | have you tried (b)? | 08:52 |
stephenfin | to do that, you'll need to also set the 'product_id' in the 'passthrough_whitelist' | 08:52 |
stephenfin | to whatever the product_id of the PF is, assuming the PF and VF have different product_id's (they should) | 08:52 |
awalende | my colleague means he "maybe" tried it as well. We try it again and let you know | 08:52 |
stephenfin | If they don't, you're going to need to use the 'address' field, I'm afraid | 08:53 |
stephenfin | okay, do please | 08:53 |
stephenfin | sean-k-mooney might be able to help too but they're definitely fast asleep atm :) | 08:53 |
awalende | How do you make sure that "VFs are not whitelisted"? | 08:54 |
bauzas | the T4 ? | 08:55 |
bauzas | awalende: which grid driver are you using ? | 08:55 |
bauzas | GRID8 ? | 08:55 |
awalende | no driver at all, we plugged the T4 into our hypervisor nodes and want to pass them through to guests | 08:55 |
stephenfin | awalende: If SR-IOV is enabled on the device, you will have additional PCI devices appearing at e.g. 'lspci' | 08:56 |
*** jangutter has joined #openstack-nova | 08:56 | |
awalende | one moment | 08:56 |
stephenfin | 'lspci | grep -i nvidia' should flag them, I'd imagine | 08:57 |
bauzas | oh right, then that's normal | 08:57 |
bauzas | Nvidia plans to deliver virtual GPUs as VFs later | 08:57 |
bauzas | I guess they already started to expose the GPU as a PF | 08:57 |
awalende | stephenfin, it only shows me one card as af:00.0 3D controller: NVIDIA Corporation TU104GL [Tesla T4] (rev a1) | 08:57 |
stephenfin | I'm not sure how you'd turn them on/off. Probably by echoing stuff to sysfs like you do for SR-IOV NICs, I guess | 08:57 |
bauzas | but I thought the Tesla models wouldn't get it | 08:57 |
bauzas | just the Volta ones | 08:57 |
bauzas | I'm not a nVidia salesperson, (un?)fornutately :p | 08:58 |
stephenfin | okay, it's probably disabled so | 08:58 |
*** nanzha has quit IRC | 08:58 | |
bauzas | awalende: just do something like 'lspci -vv | grep 10de | 08:59 |
stephenfin | awalende: So there is definitely _something_ specific you need to do in order to use type-PF devices. I'm just trying to figure out what it is | 08:59 |
awalende | bauzas, returns empty | 08:59 |
bauzas | mmm | 08:59 |
stephenfin | awalende: This is what I'm thinking of https://github.com/openstack/nova/blob/master/nova/pci/stats.py#L289-L297 | 09:00 |
bauzas | awalende: are you OK with pasting your whole lspci output ? | 09:00 |
stephenfin | so if I'm reading that correctly, you *must* set 'dev_type': 'type-PF' in the alias | 09:00 |
awalende | which lspci command to you want pasted? 'lspci -vv'? | 09:01 |
bauzas | I agree | 09:01 |
bauzas | awalende: yes please | 09:01 |
bauzas | stephenfin: looks like you need to expose your type-PF this way | 09:02 |
stephenfin | wdym? | 09:02 |
bauzas | but I'm amazed that nvidia provides a PF and not a straight PCI ID | 09:02 |
bauzas | stephenfin: you have to expose the PCI devices in nova.conf in order to allow Nova to use them, right? | 09:03 |
stephenfin | yeah, awalende has done that mostly correctly I think | 09:03 |
awalende | https://paste.ubuntu.com/p/xXkRfFMRYh/ | 09:03 |
* bauzas clicks | 09:04 | |
stephenfin | the '[pci] passthrough_whitelist' on the compute node is correct, but the '[pci] alias' on the controller and compute node doesn't appear to be (it needs 'dev_type') | 09:04 |
awalende | stephenfin, we will try soon! | 09:04 |
awalende | does dev_type needs to be specified on controller and compute? | 09:05 |
stephenfin | sorry, 'device_type' | 09:05 |
stephenfin | in the 'alias' option on both | 09:06 |
awalende | ok, we'll try it | 09:06 |
stephenfin | alias={"name":"T4","vendor_id":"10de","product_id":"1eb8","device_type":"type-PF"} | 09:06 |
stephenfin | or something like that | 09:06 |
bauzas | very interesting lspci | 09:07 |
bauzas | https://paste.ubuntu.com/p/xXkRfFMRYh/ L3241 | 09:08 |
bauzas | it shows a SR-IOV capability | 09:08 |
bauzas | but no VGA capability (!) | 09:08 |
bauzas | that's beyond my comfort zone | 09:08 |
stephenfin | yeah, it's an accelerator, not a GPU | 09:09 |
bauzas | my bad then | 09:09 |
bauzas | I'm not familiar with the product line | 09:09 |
bauzas | so we're just talking of PCI passthrugh, not VGA passthru, my bad | 09:09 |
bauzas | anyway, it doesn't expose a standard PCI address | 09:10 |
*** yan0s has joined #openstack-nova | 09:11 | |
bauzas | I just feel that stephenfin is absolutely right, nova punts the address because of the internal conditional he said | 09:12 |
*** dave-mccowan has joined #openstack-nova | 09:14 | |
awalende | someone rebooted the server =.= one more minute | 09:15 |
awalende | ok, its spawning :D fingers crossed | 09:17 |
awalende | Ok, the device shows up in the guest | 09:20 |
awalende | hooray | 09:20 |
awalende | But why do we explicitly have to specify device_type. In all of our other devices it wasn't really needed.....is it because pci is default? | 09:21 |
stephenfin | awalende: https://github.com/openstack/nova/blob/master/nova/pci/stats.py#L289-L297 | 09:21 |
stephenfin | If you have a device that supports SR-IOV, the assumption we've made is that you'll generally want to use that capability because of the scalability it offers | 09:22 |
awalende | Uhhh just a minor complaint....this isn't documented :D :D. But nice, the T4 is now there in the guest. Thanks all for helping us! | 09:23 |
stephenfin | So by default we don't allow you to use the parent/root/PF device because if you do, you won't be able to use the child/VF device(s) | 09:23 |
stephenfin | Agreed. I'm working on a patch to do that now. I'll stick you on the review once it's up, if you like | 09:24 |
awalende | sure thing :D | 09:24 |
*** nanzha has joined #openstack-nova | 09:24 | |
*** gibi has quit IRC | 09:26 | |
awalende | irc name is launchpad name | 09:27 |
huaqiang | sean-k-mooney: about the per-vm-pci-NUMA-policy spec, I want to confirm that it defines a policy applied to all PCI devices for whole VM, right? | 09:33 |
*** ociuhandu has quit IRC | 09:35 | |
*** huaqiang has quit IRC | 09:36 | |
*** huaqiang has joined #openstack-nova | 09:36 | |
*** psachin has quit IRC | 09:39 | |
*** abaindur has quit IRC | 09:41 | |
huaqiang | sean-k-mooney: seems I have sent my questions we have drawn a conclusion agian by mistake, pls ignore it. | 09:43 |
*** xek__ has joined #openstack-nova | 09:54 | |
*** bhagyashris has quit IRC | 09:55 | |
*** yankcrime has joined #openstack-nova | 09:59 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: WIP block_device: Use original volume_type when creating volumes from snapshots https://review.opendev.org/694497 | 10:01 |
*** luksky has joined #openstack-nova | 10:07 | |
*** udesale has quit IRC | 10:14 | |
*** udesale has joined #openstack-nova | 10:15 | |
*** bhagyashris has joined #openstack-nova | 10:16 | |
openstackgerrit | Liang Fang proposed openstack/nova-specs master: Support volume local cache https://review.opendev.org/689070 | 10:21 |
*** gibi has joined #openstack-nova | 10:21 | |
*** lpetrut has joined #openstack-nova | 10:26 | |
*** zhanglong has quit IRC | 10:29 | |
*** shilpasd has joined #openstack-nova | 10:30 | |
*** ociuhandu has joined #openstack-nova | 10:31 | |
*** slaweq has quit IRC | 10:34 | |
*** chenhaw has quit IRC | 10:37 | |
*** chenhaw has joined #openstack-nova | 10:37 | |
*** ociuhandu has quit IRC | 10:37 | |
*** elod_off is now known as elod | 10:38 | |
*** ociuhandu has joined #openstack-nova | 10:38 | |
*** chenhaw has quit IRC | 10:42 | |
*** ociuhandu has quit IRC | 10:46 | |
*** zhubx has quit IRC | 10:51 | |
*** zhubx has joined #openstack-nova | 10:51 | |
*** tesseract has quit IRC | 10:52 | |
*** tesseract has joined #openstack-nova | 10:52 | |
*** mkrai has quit IRC | 10:53 | |
*** mkrai_ has joined #openstack-nova | 10:53 | |
*** mkrai__ has joined #openstack-nova | 10:55 | |
*** mkrai_ has quit IRC | 10:58 | |
*** JamesBenson has joined #openstack-nova | 10:59 | |
*** bhagyashris has quit IRC | 11:01 | |
openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: Virtual instance rescue with stable disk devices https://review.opendev.org/693849 | 11:01 |
openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: Boot from volume instance rescue https://review.opendev.org/694063 | 11:01 |
*** mkrai__ has quit IRC | 11:02 | |
*** JamesBenson has quit IRC | 11:03 | |
*** ratailor has quit IRC | 11:06 | |
*** ratailor has joined #openstack-nova | 11:06 | |
*** bhagyashris has joined #openstack-nova | 11:18 | |
*** jawad_axd has joined #openstack-nova | 11:32 | |
*** rha has left #openstack-nova | 11:39 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Change order of PCI configuration steps https://review.opendev.org/694521 | 11:46 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Clarify configuration steps for PF devices https://review.opendev.org/694522 | 11:46 |
stephenfin | awalende: ^ | 11:46 |
stephenfin | Also sean-k-mooney, adrianc: ^ | 11:46 |
stephenfin | I'd like to backport that as far as we can go | 11:46 |
* stephenfin -> 🏋️ | 11:46 | |
*** dpawlik has quit IRC | 11:47 | |
openstackgerrit | Merged openstack/nova master: Reset vm_state to original value if rebuild claim fails https://review.opendev.org/692185 | 11:48 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Clarify configuration steps for PF devices https://review.opendev.org/694522 | 11:48 |
*** dpawlik has joined #openstack-nova | 11:53 | |
openstackgerrit | Merged openstack/nova master: FUP for Ib62ac0b692eb92a2ed364ec9f486ded05def39ad https://review.opendev.org/693556 | 11:53 |
*** slaweq has joined #openstack-nova | 11:55 | |
*** ratailor has quit IRC | 12:00 | |
*** artom has joined #openstack-nova | 12:01 | |
*** gshippey has joined #openstack-nova | 12:02 | |
sean-k-mooney | stephenfin: im not sure if that is correct | 12:19 |
sean-k-mooney | if you dont set the device_type in the alias at all i think it will work with type-PF | 12:19 |
sean-k-mooney | the device_type is not required in the alias as far as i remember | 12:19 |
sean-k-mooney | if it is present it definetly needs to match | 12:19 |
sean-k-mooney | i need to test some sriov stuff later today so ill try this if i have time otherwise ill do it on monday | 12:20 |
shilpasd | gibi: hi, can you please elaborate on point 'add compute nodes to the aggregates of shared RP aggregates' regarding Shared NFS, so that will think on designing the same | 12:21 |
sean-k-mooney | the product id will be differente for the PF vs VF so if you have the correct product id and no VF are allocated i think it just works but again that is just going off memory i have not tried this in a year or two | 12:22 |
shilpasd | gibi: at driver level, how ww will get aggragtes associated to shared RP and will map those to compute node? And if compute node already has aggregates (not the same as of shared RP) are we really bother of them? | 12:25 |
*** mmethot has quit IRC | 12:26 | |
*** dpawlik has quit IRC | 12:28 | |
*** artom has quit IRC | 12:28 | |
*** mkrai has joined #openstack-nova | 12:29 | |
*** ociuhandu has joined #openstack-nova | 12:32 | |
*** dpawlik has joined #openstack-nova | 12:33 | |
*** jangutter has quit IRC | 12:33 | |
*** ociuhandu has quit IRC | 12:37 | |
openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Extract rescue from reboot https://review.opendev.org/694529 | 12:37 |
*** ociuhandu has joined #openstack-nova | 12:38 | |
*** ociuhandu has quit IRC | 12:39 | |
gibi | shilpasd: there are two types of aggregates: there are placement aggregates https://docs.openstack.org/api-ref/placement/#resource-provider-aggregates and there are nova host aggregates https://docs.openstack.org/api-ref/compute/#host-aggregates-os-aggregates | 12:39 |
*** ociuhandu has joined #openstack-nova | 12:39 | |
gibi | the disk sharing RP needs to be in the same placement aggregate as the compute RP it shares disk with | 12:39 |
gibi | we agreed that a new nova config option will tell the nova compute the uuid of the placement aggregate | 12:40 |
gibi | in which the admin added the sharing disk RP | 12:40 |
openstackgerrit | Lee Yarwood proposed openstack/nova master: docs: Extract rescue from reboot https://review.opendev.org/694529 | 12:40 |
gibi | so when the compute start up it can change if its own RP (the compute PR) is already part of the placement aggregate configured in it's nova.conf | 12:41 |
gibi | and if not then call PUT | 12:41 |
gibi | /resource_providers/{uuid}/aggregatesPUT | 12:41 |
gibi | /resource_providers/{uuid}/aggregates | 12:41 |
gibi | to put it's compute PR into the that aggregate | 12:42 |
gibi | shilpasd: an PR can be in any number or placement aggregates | 12:43 |
gibi | so if the compute RP is already in other aggregates that does not really matter to us | 12:43 |
*** ociuhandu has quit IRC | 12:45 | |
openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: Virtual instance rescue with stable disk devices https://review.opendev.org/693849 | 12:45 |
openstackgerrit | Lee Yarwood proposed openstack/nova-specs master: Boot from volume instance rescue https://review.opendev.org/694063 | 12:45 |
shilpasd | gibi: new nova config option which holds uuid of the placement aggregate or uuid of the shared RP? | 12:46 |
shilpasd | and it will at compute level or libvirt level? | 12:47 |
gibi | shilpasd: uuid of the placement aggregate | 12:49 |
gibi | shilpasd: and I think the logic can be in the compute manager level | 12:49 |
*** ricolin has joined #openstack-nova | 12:50 | |
gibi | shilpasd: have you seen this mailthread ? http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010624.html | 12:51 |
shilpasd | gibi: no, will go through, thnaks for sharing | 12:52 |
gibi | shilpasd: I think that summarized most of what we talked above | 12:52 |
shilpasd | gibi: yes, that's clear | 12:53 |
shilpasd | gibi: thank you | 12:53 |
*** bhagyashris has quit IRC | 12:54 | |
*** zhubx has quit IRC | 12:54 | |
*** zhubx has joined #openstack-nova | 12:54 | |
gibi | shilpasd: you're welcome | 12:54 |
openstackgerrit | Merged openstack/nova master: Add functional recreate test for bug 1852610 https://review.opendev.org/694351 | 12:56 |
openstack | bug 1852610 in OpenStack Compute (nova) "API allows source compute service/node deletion while instances are pending a resize confirm/revert" [Medium,In progress] https://launchpad.net/bugs/1852610 - Assigned to Matt Riedemann (mriedem) | 12:56 |
*** zhubx has quit IRC | 12:56 | |
*** zhubx has joined #openstack-nova | 12:56 | |
*** awalende has quit IRC | 12:59 | |
*** awalende has joined #openstack-nova | 12:59 | |
*** Luzi has quit IRC | 13:05 | |
*** dpawlik has quit IRC | 13:06 | |
*** awalende has quit IRC | 13:14 | |
*** awalende has joined #openstack-nova | 13:14 | |
*** awalende has quit IRC | 13:15 | |
*** mmethot has joined #openstack-nova | 13:15 | |
*** awalende has joined #openstack-nova | 13:15 | |
*** ociuhandu has joined #openstack-nova | 13:16 | |
*** Luzi has joined #openstack-nova | 13:20 | |
*** jangutter has joined #openstack-nova | 13:20 | |
*** mriedem has joined #openstack-nova | 13:23 | |
*** dpawlik has joined #openstack-nova | 13:45 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Remove duplicate ServerMovingTests._resize_and_check_allocations https://review.opendev.org/694538 | 13:45 |
*** mkrai has quit IRC | 13:47 | |
*** dlbewley has quit IRC | 13:49 | |
*** sapd1_x has joined #openstack-nova | 13:55 | |
*** mlavalle has joined #openstack-nova | 13:56 | |
*** ociuhandu has quit IRC | 13:58 | |
*** dpawlik has quit IRC | 14:03 | |
*** dpawlik has joined #openstack-nova | 14:05 | |
*** ociuhandu has joined #openstack-nova | 14:05 | |
*** shilpasd has quit IRC | 14:06 | |
*** nweinber has joined #openstack-nova | 14:15 | |
*** Luzi has quit IRC | 14:16 | |
openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Add functional recreate test for bug 1852610 https://review.opendev.org/694544 | 14:18 |
openstack | bug 1852610 in OpenStack Compute (nova) "API allows source compute service/node deletion while instances are pending a resize confirm/revert" [Medium,In progress] https://launchpad.net/bugs/1852610 - Assigned to Matt Riedemann (mriedem) | 14:18 |
openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Add functional recreate revert resize test for bug 1852610 https://review.opendev.org/694545 | 14:18 |
openstackgerrit | Matt Riedemann proposed openstack/nova stable/train: Block deleting compute services with in-progress migrations https://review.opendev.org/694546 | 14:18 |
*** tbachman has joined #openstack-nova | 14:19 | |
*** artom has joined #openstack-nova | 14:20 | |
*** tbachman_ has joined #openstack-nova | 14:21 | |
artom | "Before posting a comment to any patch, a third party testing system must contact the project they wish to test and get approval to post comments on their patches. This can be done by attending the project’s meeting." | 14:22 |
artom | From https://docs.openstack.org/infra/system-config/third_party.html | 14:22 |
artom | Is that an actual thing for Nova? | 14:22 |
*** tbachman has quit IRC | 14:23 | |
*** tbachman_ is now known as tbachman | 14:23 | |
dansmith | it's for every project I think | 14:23 |
dansmith | we used to have problems with people setting up their own CI and it going haywire and spraying comments everywhere | 14:23 |
dansmith | (in nova) | 14:23 |
artom | dansmith, so context is http://post-office.corp.redhat.com/archives/rh-openstack-dev/2019-October/msg00060.html (sorry for the internal-only link) | 14:23 |
artom | dansmith, and http://post-office.corp.redhat.com/archives/rh-openstack-dev/2019-October/msg00209.html that's sort of a summary of where we ended up after initial discussions | 14:25 |
dansmith | you could probably just say "I want to set up a CI system" | 14:26 |
artom | I want to set up a CI system | 14:27 |
artom | :D | 14:27 |
dansmith | but anyway, yes, there are hoops to jump through | 14:27 |
artom | Yeah | 14:28 |
artom | Internally as well as here, *sigh* | 14:28 |
artom | Hopefully it'll be worth it | 14:28 |
artom | I also noticed Mellanox has their own SRIOV CI | 14:28 |
artom | But... presumably that only tests their hardware | 14:28 |
artom | And I have no idea what the status is | 14:28 |
artom | adrianc, ^^ ? | 14:29 |
artom | And the point of RHEx (Red Hat Exotic hardware CI) would be more than just SRIOV, SRIOV is just the initial MVP scope | 14:29 |
artom | GPUs come to mind | 14:29 |
dansmith | artom: are you looking for help setting it up? If so, I'm sure the infra people are who you want to talk to | 14:29 |
artom | dansmith, yeah, that conversation is already happening - migi as well | 14:30 |
dansmith | ack | 14:30 |
mriedem | i wish the dell emc people would have asked b/c their's comments and always fails | 14:30 |
bauzas | gibi: I'm about to provide a new revision for the audit command, would it be possible for you to check it with the bandwidth-aware instances ? | 14:32 |
bauzas | (at least once I'm done with reno) | 14:32 |
*** ociuhandu has quit IRC | 14:32 | |
*** awalende has quit IRC | 14:33 | |
*** awalende has joined #openstack-nova | 14:33 | |
*** jawad_axd has quit IRC | 14:37 | |
*** tbachman has quit IRC | 14:37 | |
*** awalende_ has joined #openstack-nova | 14:37 | |
*** awalende has quit IRC | 14:38 | |
*** jawad_axd has joined #openstack-nova | 14:38 | |
*** awalende_ has quit IRC | 14:39 | |
*** jawad_ax_ has joined #openstack-nova | 14:40 | |
gibi | bauzas: sure | 14:40 |
gibi | bauzas: If I time out on it today then I will do it next week | 14:41 |
bauzas | gibi: thanks | 14:41 |
bauzas | hopefully, I'll push it in 20 mins | 14:41 |
*** KeithMnemonic has joined #openstack-nova | 14:41 | |
*** luksky has quit IRC | 14:42 | |
*** jawad_axd has quit IRC | 14:43 | |
*** jawad_ax_ has quit IRC | 14:45 | |
*** sridharg has quit IRC | 14:45 | |
openstackgerrit | Merged openstack/nova master: Add functional recreate revert resize test for bug 1852610 https://review.opendev.org/694364 | 14:47 |
openstack | bug 1852610 in OpenStack Compute (nova) "API allows source compute service/node deletion while instances are pending a resize confirm/revert" [Medium,In progress] https://launchpad.net/bugs/1852610 - Assigned to Matt Riedemann (mriedem) | 14:47 |
*** usr2033 has quit IRC | 14:54 | |
*** tbachman has joined #openstack-nova | 14:57 | |
*** links has quit IRC | 14:59 | |
*** jangutter has quit IRC | 15:00 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a placement audit command https://review.opendev.org/670112 | 15:03 |
*** eharney has joined #openstack-nova | 15:04 | |
*** awalende has joined #openstack-nova | 15:05 | |
*** awalende has quit IRC | 15:10 | |
*** dpawlik has quit IRC | 15:10 | |
*** sridharg has joined #openstack-nova | 15:10 | |
gibi | eandersson: fyi, I reported a bug about the false error log from the compute you found https://bugs.launchpad.net/nova/+bug/1852759 | 15:12 |
openstack | Launchpad bug 1852759 in OpenStack Compute (nova) rocky "false error log at compute restart during error out stuck instances" [Low,Triaged] - Assigned to Balazs Gibizer (balazs-gibizer) | 15:12 |
*** jaosorior has joined #openstack-nova | 15:13 | |
gibi | bauzas: building a devstack for your audit patch... | 15:14 |
*** ricolin has quit IRC | 15:18 | |
*** ivve has quit IRC | 15:24 | |
slaweq | mriedem: hi | 15:29 |
slaweq | mriedem: recently I noticed in neutron job error like https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_300/678438/22/check/neutron-tempest-dvr-ha-multinode-full/3008cc3/testr_results.html.gz | 15:29 |
slaweq | and I see in nova logs (src host) error while connecting to libvirt on dst node: https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_300/678438/22/check/neutron-tempest-dvr-ha-multinode-full/3008cc3/compute2/logs/screen-n-cpu.txt.gz | 15:30 |
slaweq | do You know about such issue or maybe I should create new LP for this? | 15:30 |
*** JamesBenson has joined #openstack-nova | 15:32 | |
artom | slaweq, mriedem, doesn't look like a Nova bug: https://zuul.opendev.org/t/openstack/build/3008cc3eeaea44369a2fa3db3a29ae67/log/compute2/logs/screen-n-cpu.txt.gz#3392 | 15:32 |
*** SonPham has joined #openstack-nova | 15:32 | |
artom | Just unable to connect to the dest libvirt | 15:32 |
artom | Not sure why tho | 15:32 |
mriedem | you mean this libvirt.libvirtError: unable to connect to server at 'ubuntu-bionic-rax-dfw-0012676801:49152': Connection refused | 15:33 |
slaweq | artom: mriedem exactly | 15:33 |
mriedem | usually need to look at the guest log | 15:33 |
mriedem | but no this isn't a nova bug | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Add TODOs for remaining nova-network functional tests https://review.opendev.org/684345 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove 'os-security-group-default-rules' REST API https://review.opendev.org/686807 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove unused '*_default_rules' security group DB APIs https://review.opendev.org/686808 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove (most) '/os-networks' REST APIs https://review.opendev.org/686809 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove '/os-tenant-networks' REST API https://review.opendev.org/686810 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove 'USE_NEUTRON' from functional tests https://review.opendev.org/686811 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove 'networks' quota https://review.opendev.org/686812 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: Remove nova-manage network, floating commands https://review.opendev.org/686813 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove associate, disassociate network APIs https://review.opendev.org/686814 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove 'nova-dhcpbridge' binary https://review.opendev.org/686815 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: nova-net: Remove 'nova-network' binary https://review.opendev.org/686816 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Blast most references to nova-network https://review.opendev.org/686817 | 15:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: WIP https://review.opendev.org/686818 | 15:34 |
slaweq | mriedem: artom ok, thx for looking into that | 15:34 |
slaweq | I will than just leave it alone for now :) | 15:34 |
artom | slaweq, uh, I think it's trying to live migrate to itself o_O | 15:35 |
artom | https://zuul.opendev.org/t/openstack/build/3008cc3eeaea44369a2fa3db3a29ae67/log/zuul-info/host-info.compute2.yaml#403 | 15:36 |
*** JamesBenson has quit IRC | 15:36 | |
artom | Oh no, that's the controller | 15:36 |
mriedem | ubuntu-bionic-rax-dfw-0012676801 != ubuntu-bionic-rax-dfw-0012676804 | 15:36 |
artom | mriedem, yeah sorry, got confused | 15:36 |
artom | But anyways, something to look into - is the controller running the full nova-compute stack? | 15:37 |
*** ociuhandu has joined #openstack-nova | 15:37 | |
mriedem | yes, this is a 3 node job | 15:37 |
mriedem | dvr-ha-multinode-full makes the lights dim when it runs | 15:37 |
artom | It dimmed my lights :( | 15:38 |
mriedem | you scamp | 15:38 |
*** JamesBenson has joined #openstack-nova | 15:38 | |
*** JamesBenson has quit IRC | 15:38 | |
*** JamesBenson has joined #openstack-nova | 15:39 | |
*** KeithMnemonic1 has joined #openstack-nova | 15:41 | |
mriedem | cpu usage was pretty high on the controller when it failed | 15:41 |
mriedem | load spiked up around then too | 15:42 |
*** ociuhandu has quit IRC | 15:42 | |
openstackgerrit | Merged openstack/nova-specs master: Virtual instance rescue with stable disk devices https://review.opendev.org/693849 | 15:42 |
artom | slaweq, there's your failure: https://zuul.opendev.org/t/openstack/build/3008cc3eeaea44369a2fa3db3a29ae67/log/controller/logs/libvirt/qemu/instance-00000011_log.txt.gz#4 | 15:42 |
artom | mriedem too if you care ^^ | 15:43 |
mriedem | artom: that's not a faliure | 15:43 |
mriedem | that shows up in like every guest log in thegate | 15:43 |
artom | Oh | 15:43 |
* artom gives up and goes back to his corner | 15:43 | |
mriedem | e.g. random guest https://storage.gra1.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_300/678438/22/check/neutron-tempest-dvr-ha-multinode-full/3008cc3/controller/logs/libvirt/qemu/instance-00000023_log.txt.gz | 15:43 |
mriedem | my guess is the failure is due to, like many gate failures these days, overloaded nodes crapping out | 15:44 |
mriedem | though the rax nodes usually aren't one of them | 15:44 |
openstackgerrit | Balazs Gibizer proposed openstack/nova stable/rocky: Fix false ERROR message at compute restart https://review.opendev.org/694581 | 15:46 |
gibi | eandersson: ^^ | 15:50 |
*** ociuhandu has joined #openstack-nova | 15:55 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: docs: Extract rescue from reboot https://review.opendev.org/694529 | 15:56 |
lyarwood | thanks for that mriedem | 16:00 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: doc: mention that rescuing a volume-backed server is not supported https://review.opendev.org/694584 | 16:02 |
mriedem | np | 16:02 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: doc: mention that rescuing a volume-backed server is not supported https://review.opendev.org/694584 | 16:03 |
*** nanzha has quit IRC | 16:04 | |
*** SonPham has quit IRC | 16:04 | |
*** mmethot has quit IRC | 16:04 | |
gibi | bauzas: I hit a bug in https://review.opendev.org/#/c/670112/8/nova/cmd/manage.py@2867 | 16:07 |
mriedem | gibi: you were +2 on this before it was rebased a bit in earlier changes https://review.opendev.org/#/c/642591/ - can you hit that again? dansmith - that's also the one where you suggested re-doing the logic so it should be simple and familiar | 16:08 |
bauzas | gibi: graaah ok, thanks ! | 16:08 |
gibi | mriedem: on it. | 16:08 |
bauzas | gibi: just respinning, sec | 16:08 |
gibi | bauzas: sure, I will retry with the new ps | 16:09 |
mriedem | bauzas: why wouldn't your functional tests hit that? | 16:09 |
bauzas | mriedem: the CI is still in the weeds | 16:09 |
bauzas | no result yet | 16:09 |
bauzas | and I was lazy | 16:09 |
mriedem | bauzas: i meant locally... | 16:09 |
mriedem | yeah | 16:09 |
bauzas | but that's a good call, lemme check | 16:10 |
mriedem | this is why i said you don't really need unit testing on these types of commands, it should mostly, if not all, be functional test driven | 16:10 |
* gibi notes to run functional test on bauzas patches before pulling them into devstack | 16:10 | |
bauzas | that's what happens when you do three things at same time... | 16:10 |
bauzas | meeting, patching, discussing | 16:10 |
bauzas | gibi: I just run the functest now | 16:11 |
bauzas | hopefully it will get caught | 16:11 |
bauzas | oh, and FWIW, I was rushing on updating a new revision, I haven't used yet the COMPUTE_NODE trait or did the split | 16:12 |
openstackgerrit | Merged openstack/nova stable/train: Use admin neutron client to query ports for binding https://review.opendev.org/694013 | 16:13 |
*** dlbewley has joined #openstack-nova | 16:13 | |
bauzas | mriedem: I'm not opposed to have most of the logic being checked by functional tests, I just added a few unit tests for sanity :) | 16:13 |
*** sridharg has quit IRC | 16:13 | |
gibi | bauzas: nova.tests.functional.test_nova_manage.TestNovaManagePlacementAudit.test_audit_orphaned_allocations_from_deleted_compute_evacuate failed for me locally now but with a different error than what I saw in the devstack | 16:14 |
mriedem | that's a slippery slope and bad habit to get into with commands like this | 16:14 |
mriedem | so just, don't do it | 16:14 |
gibi | bauzas: File "nova/cmd/manage.py", line 2927, in audit | 16:14 |
gibi | ctxt, placement, output, provider, delete) | 16:14 |
gibi | File "nova/cmd/manage.py", line 2781, in _check_orphaned_allocations_for_provider | 16:14 |
gibi | inst_uuids, mig_uuids = result | 16:14 |
bauzas | gibi: yeah, I just saw it | 16:14 |
gibi | TypeError: 'bool' object is not iterable | 16:14 |
bauzas | I'm on it | 16:15 |
gibi | bauzas: cool | 16:15 |
gibi | mriedem: do you mean sanity is slippery slope with nova manage cli ? :) | 16:15 |
bauzas | gibi: but for *some* reason, the functests should have caught the argument issue | 16:15 |
bauzas | (for the microversion) | 16:16 |
bauzas | I mean, the other issue is unrelateed | 16:16 |
bauzas | or a side effect | 16:16 |
bauzas | so, mriedem's point is legit | 16:16 |
bauzas | I'm leaking coverage here | 16:16 |
mriedem | gibi: heh, no, i just don't really trust unit tests in nova anymore for anything that involves more than one service or even more than a couple of methods interacting within the same service | 16:17 |
mriedem | it's too easy to mock things that become false positives | 16:17 |
*** jawad_axd has joined #openstack-nova | 16:18 | |
*** tbarron has quit IRC | 16:20 | |
bauzas | so the bug is fixed, but the argument issue isn't shown | 16:20 |
*** jawad_axd has quit IRC | 16:22 | |
*** dpawlik has joined #openstack-nova | 16:24 | |
gibi | mriedem: I share your view regarding the unit tests in nova | 16:26 |
bauzas | 2019-11-15 17:26:12,996 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers" status: 200 len: 840 microversion: 1.0 | 16:27 |
bauzas | magnifico ^ | 16:27 |
bauzas | the report client is blindly accepting the 'microversion' keyword | 16:27 |
bauzas | but just says "meh" to it | 16:27 |
bauzas | gibi: mriedem: ^ | 16:27 |
bauzas | I don't see how to catch this unless to mock and assert the call | 16:28 |
bauzas | which is nonsense | 16:28 |
gibi | bauzas: then something is wrong with the placement fixture we use in the functional test | 16:29 |
bauzas | and, with using the 'version' keyword : | 16:29 |
bauzas | 2019-11-15 17:29:16,087 INFO [placement.requestlog] 127.0.0.1 "GET /placement/resource_providers" status: 200 len: 1684 microversion: 1.14 | 16:29 |
bauzas | we probably kwargs | 16:30 |
mriedem | def get(self, url, **kwargs): is your problem | 16:30 |
bauzas | gibi: ^ | 16:30 |
bauzas | heh, this | 16:30 |
mriedem | like gibi said it's a bug in the fixture | 16:30 |
mriedem | SchedulerReportClient.get won't let you pass microversion= | 16:30 |
mriedem | def get(self, url, version=None, global_request_id=None): | 16:30 |
gibi | then it is a good time to fix the fixture | 16:30 |
gibi | :) | 16:30 |
bauzas | aaaaand I love to see a bug about some code with a few TODO(sbauza) around it... | 16:31 |
* bauzas hides | 16:31 | |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Change order of two classes https://review.opendev.org/689178 | 16:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Rework '_delete_server' https://review.opendev.org/689179 | 16:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Make '_wait_for_state_change' behave consistently https://review.opendev.org/689180 | 16:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Unify '_wait_until_deleted' implementations https://review.opendev.org/689181 | 16:33 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: functional: Make 'ServerTestBase' subclass 'InstanceHelperMixin' https://review.opendev.org/689182 | 16:33 |
* stephenfin crosses fingers and hopes https://review.opendev.org/#/c/692374/ passes | 16:33 | |
mriedem | should just remove those todos in the placement fixture about passing a token, that's never going to happen if it hasn't happened by now | 16:33 |
mriedem | we should like, put a 3 year timer on todos in the code | 16:33 |
mriedem | "TODO(vishy): do this in grizzly" | 16:34 |
gibi | :) | 16:34 |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a placement audit command https://review.opendev.org/670112 | 16:36 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Remove 'adv-config', 'system-admin' subdocs https://review.opendev.org/684402 | 16:38 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Replacing underscores with dashes https://review.opendev.org/685929 | 16:38 |
openstackgerrit | Stephen Finucane proposed openstack/nova master: docs: Strip '.rst' suffix https://review.opendev.org/687264 | 16:38 |
*** yan0s has quit IRC | 16:42 | |
stephenfin | melwitt: If you have time before the end of the day, think you could hit this quotas doc patch? https://review.opendev.org/#/c/667165/ | 16:42 |
stephenfin | It's been around for a few months. Would be nice to close it out | 16:42 |
*** macz has joined #openstack-nova | 16:42 | |
KeithMnemonic1 | hello all any chance to get some more reviews and a possible WF+1 on this one ? https://review.opendev.org/#/c/683008/ | 16:43 |
*** TxGirlGeek has joined #openstack-nova | 16:43 | |
melwitt | stephenfin: that's a biggun. I'll try to get to it | 16:43 |
stephenfin | melwitt: Sure thing. tbh, I'd just forget whatever was there previously and read it like a new doc. If nothing stands out as totally wrong, it's gonna be better than what we have | 16:44 |
stephenfin | IMO, obv | 16:44 |
sean-k-mooney | KeithMnemonic1: i think lyarwood should proably review that | 16:44 |
melwitt | stephenfin: ok, I'll approach it with that in mind | 16:45 |
stephenfin | ta | 16:47 |
* stephenfin -> 🏃 | 16:48 | |
stephenfin | Have a good weekend, all o/ | 16:48 |
melwitt | happy weekend | 16:49 |
*** damien_r has quit IRC | 16:50 | |
gibi | bauzas: another bug in https://review.opendev.org/#/c/670112/9/nova/cmd/manage.py@2695 this is also not caught by any functional test as that is green for me on PS9 | 16:51 |
*** rpittau is now known as rpittau|afk | 16:52 | |
bauzas | gibi: thanks | 16:52 |
*** bnemec is now known as beekneemech | 16:52 | |
gibi | bauzas: don't be confused by the line numbers in my stack trace. I had to cherry-pick your patch top of the nova master in my devstack | 16:52 |
melwitt | gibi: I dunno if you saw this but I attempted to do a TODO you mentioned in the NeutronFixture for fun https://review.opendev.org/693453 | 16:52 |
bauzas | gibi: ack no worries | 16:52 |
gibi | melwitt: yeah I think I saw it during the PTG but then I got distracted. Looking at it now. Thanks for picking that TODO up | 16:53 |
*** udesale has quit IRC | 16:53 | |
melwitt | gibi: don't thank me yet haha, I hope I didn't misunderstand | 16:53 |
*** eharney has quit IRC | 16:55 | |
*** luksky has joined #openstack-nova | 16:56 | |
bauzas | gibi: and I get why you catched the error and not my tests | 16:56 |
bauzas | gibi: that's because when I use the cache, I forget to use the cell mapping | 16:56 |
*** sapd1_x has quit IRC | 16:57 | |
bauzas | so, unless you have more than one instance for the same compute, you don't have the problem | 16:57 |
bauzas | (and I only verify one instance... :) ) | 16:57 |
KeithMnemonic1 | thanks sean-k-mooney, lyarwood would you have time in the next few business days to look at it? | 16:59 |
KeithMnemonic1 | can someone please continue the reviews on mriedem patches for the "openstack list marker hang" https://review.opendev.org/#/c/690721/4 | 17:00 |
*** gyee has joined #openstack-nova | 17:01 | |
sean-k-mooney | KeithMnemonic1: lyarwood for what its worth im +0.5 on it. i dont know htat part of the code well enought to tell if all of the change form queens make sense but the conflcits are called out in the commit and it looks more or less correct | 17:01 |
gibi | bauzas: I have one instance in shutoff state on the compute | 17:03 |
*** dpawlik has quit IRC | 17:03 | |
bauzas | no worries, I'm just fixing the bug | 17:03 |
gibi | bauzas: sure | 17:03 |
bauzas | and thanks, I should actually test it | 17:04 |
bauzas | it's 6.04pm tho, so I'll just fix the bug and add a TODO in the commit msg for saying I need to add a new func test for it :) | 17:04 |
bauzas | ie. two instances :-) | 17:04 |
bauzas | s/add/modify | 17:04 |
gibi | bauzas: yeah, I will time out soon as well | 17:05 |
bauzas | actually, you know what ? I'm just gonna amend the test now | 17:05 |
bauzas | and see whether it's seen | 17:05 |
*** tbarron has joined #openstack-nova | 17:06 | |
*** lpetrut has quit IRC | 17:10 | |
*** jaosorior has quit IRC | 17:11 | |
openstackgerrit | Merged openstack/nova stable/queens: Revert "openstack server create" to "nova boot" in nova docs https://review.opendev.org/693239 | 17:17 |
*** mriedem is now known as mriedem_afk | 17:19 | |
*** dpawlik has joined #openstack-nova | 17:19 | |
*** dpawlik has quit IRC | 17:25 | |
openstackgerrit | Sylvain Bauza proposed openstack/nova master: Add a placement audit command https://review.opendev.org/670112 | 17:26 |
gibi | melwitt: two small cleanup suggestion in https://review.opendev.org/#/c/693453/3 then I'm +A | 17:30 |
melwitt | gibi: thanks, will update | 17:31 |
gibi | melwitt: I thank you :) | 17:31 |
* gibi steps away from the keyboard | 17:33 | |
*** dpawlik has joined #openstack-nova | 17:35 | |
*** mmethot has joined #openstack-nova | 17:35 | |
melwitt | gibi: thank you for review :) | 17:35 |
*** jaosorior has joined #openstack-nova | 17:39 | |
*** dpawlik has quit IRC | 17:39 | |
openstackgerrit | Archit Modi proposed openstack/nova stable/pike: doc: fix and clarify --block-device usage in user docs https://review.opendev.org/694450 | 17:42 |
*** TxGirlGeek has quit IRC | 17:59 | |
*** TxGirlGeek has joined #openstack-nova | 17:59 | |
*** TxGirlGeek has quit IRC | 18:01 | |
*** TxGirlGe_ has joined #openstack-nova | 18:01 | |
openstackgerrit | melanie witt proposed openstack/nova master: Use wrapper class for NeutronFixture get_client https://review.opendev.org/693453 | 18:02 |
melwitt | gibi: updated ^ | 18:04 |
openstackgerrit | Dustin Cowles proposed openstack/nova-specs master: Update provider config spec for identification conflicts https://review.opendev.org/693414 | 18:18 |
eandersson | gibi thanks a lot for fixing that bug | 18:18 |
*** tbachman has quit IRC | 18:18 | |
eandersson | I completely forgot to open a bug for it. | 18:19 |
*** KeithMnemonic1 has quit IRC | 18:19 | |
*** tbachman has joined #openstack-nova | 18:20 | |
*** ralonsoh has quit IRC | 18:20 | |
*** ivve has joined #openstack-nova | 18:21 | |
*** jaosorior has quit IRC | 18:26 | |
gibi | melwitt: +2 thank you! | 18:26 |
gibi | eandersson: no worries | 18:26 |
*** eharney has joined #openstack-nova | 18:26 | |
melwitt | thanks! | 18:26 |
*** jaosorior has joined #openstack-nova | 18:40 | |
*** artom has quit IRC | 18:42 | |
*** ociuhandu has quit IRC | 18:47 | |
*** jaosorior has quit IRC | 18:50 | |
*** jawad_axd has joined #openstack-nova | 18:55 | |
*** pcaruana has quit IRC | 18:56 | |
*** ociuhandu has joined #openstack-nova | 19:00 | |
*** KeithMnemonic1 has joined #openstack-nova | 19:00 | |
*** tesseract has quit IRC | 19:04 | |
*** ociuhandu has quit IRC | 19:10 | |
*** jmlowe has quit IRC | 19:12 | |
*** trident has quit IRC | 19:20 | |
openstackgerrit | Matt Riedemann proposed openstack/nova master: doc: mention that rescuing a volume-backed server is not supported https://review.opendev.org/694584 | 19:21 |
*** mriedem_afk is now known as mriedem | 19:23 | |
*** trident has joined #openstack-nova | 19:29 | |
*** jmlowe has joined #openstack-nova | 19:34 | |
*** artom has joined #openstack-nova | 19:36 | |
*** mriedem has quit IRC | 19:37 | |
*** mriedem has joined #openstack-nova | 19:42 | |
*** jawad_axd has quit IRC | 19:43 | |
*** TxGirlGe_ has quit IRC | 19:53 | |
*** JamesBen_ has joined #openstack-nova | 19:59 | |
*** lennyb has quit IRC | 20:00 | |
*** JamesBenson has quit IRC | 20:02 | |
*** JamesBenson has joined #openstack-nova | 20:03 | |
*** JamesBenson has quit IRC | 20:03 | |
*** JamesBenson has joined #openstack-nova | 20:03 | |
*** JamesBen_ has quit IRC | 20:03 | |
efried | melwitt: aren't you doing something with host status UNKNOWN? http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010887.html | 20:29 |
efried | ah https://blueprints.launchpad.net/nova/+spec/policy-rule-for-host-status-unknown | 20:29 |
*** TxGirlGeek has joined #openstack-nova | 20:30 | |
*** awalende has joined #openstack-nova | 20:34 | |
*** awalende has quit IRC | 20:38 | |
*** slaweq has quit IRC | 20:50 | |
*** spatel has joined #openstack-nova | 20:55 | |
*** nweinber has quit IRC | 20:56 | |
*** spatel has quit IRC | 20:59 | |
*** JamesBenson has quit IRC | 21:11 | |
*** slaweq has joined #openstack-nova | 21:11 | |
*** gshippey has quit IRC | 21:12 | |
*** slaweq has quit IRC | 21:16 | |
mriedem | s/doing/done/! | 21:27 |
* mriedem proxies the melwitt mic drop | 21:28 | |
*** JamesBenson has joined #openstack-nova | 21:37 | |
*** zhubx has quit IRC | 21:37 | |
*** zhubx has joined #openstack-nova | 21:37 | |
*** TxGirlGeek has quit IRC | 21:58 | |
*** TxGirlGeek has joined #openstack-nova | 21:59 | |
*** alex_xu has quit IRC | 22:01 | |
efried | yeah, once I saw I had even approved it, it all came back to me. | 22:07 |
*** kaisers has joined #openstack-nova | 22:30 | |
*** kaisers1 has quit IRC | 22:32 | |
*** JamesBenson has quit IRC | 22:41 | |
*** JamesBenson has joined #openstack-nova | 22:42 | |
*** TxGirlGeek has quit IRC | 22:43 | |
*** TxGirlGeek has joined #openstack-nova | 22:46 | |
*** JamesBenson has quit IRC | 22:46 | |
*** KeithMnemonic1 has quit IRC | 22:50 | |
openstackgerrit | Merged openstack/nova master: Always trait the compute node RP with COMPUTE_NODE https://review.opendev.org/688979 | 23:01 |
openstackgerrit | Merged openstack/nova master: docs: Extract rescue from reboot https://review.opendev.org/694529 | 23:01 |
*** KeithMnemonic has quit IRC | 23:08 | |
openstackgerrit | Merged openstack/nova master: Remove fixed sqlalchemy-migrate deprecation warning filters https://review.opendev.org/690704 | 23:11 |
*** slaweq has joined #openstack-nova | 23:11 | |
efried | mriedem: quick ack on https://review.opendev.org/#/c/689823/ if you can please | 23:13 |
*** slaweq has quit IRC | 23:16 | |
*** rcernin has joined #openstack-nova | 23:17 | |
mriedem | i thought you thought set commands should be a full overwrite? | 23:17 |
efried | mriedem: Didn't we agree earlier that adding --amend to a `set` would mean "add on"? | 23:22 |
efried | And there's no way to do partial removals with --amend, you have to implement an `unset`? | 23:22 |
efried | I don't love it, but it's simple(r than the alternatives) | 23:23 |
mriedem | i left some comments | 23:24 |
mriedem | we have some docs to update | 23:24 |
*** mmethot has quit IRC | 23:26 | |
efried | Thanks mriedem. I actually tried `set` with no ``-trait`` before I posted my comments, it does indeed wipe out all the traits on the provider. | 23:40 |
efried | With that, I'm out o/ | 23:41 |
mriedem | \o | 23:41 |
mriedem | just played the clean version of big poppa by notorious big to my kid and had to explain what "macking" means | 23:48 |
mriedem | harder than you'd think | 23:48 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add RevertResizeTask https://review.opendev.org/638046 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add revert_snapshot_based_resize conductor RPC method https://review.opendev.org/638047 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Revert cross-cell resize from the API https://review.opendev.org/638048 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Confirm cross-cell resize while deleting a server https://review.opendev.org/638268 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add archive_deleted_rows wrinkle to cross-cell functional test https://review.opendev.org/651650 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add CrossCellWeigher https://review.opendev.org/614353 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add functional test for anti-affinity cross-cell migration https://review.opendev.org/661859 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Support cross-cell moves in external_instance_event https://review.opendev.org/658478 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: libvirt: flatten rbd image during cross-cell move spawn at dest https://review.opendev.org/691991 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: Add cross-cell resize policy rule and enable in API https://review.opendev.org/638269 | 23:49 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Enable cross-cell resize in the nova-multi-cell job https://review.opendev.org/656656 | 23:50 |
openstackgerrit | Matt Riedemann proposed openstack/nova master: WIP: Add negative test to delete server during cross-cell resize claim https://review.opendev.org/688832 | 23:50 |
*** mriedem has quit IRC | 23:50 | |
*** mdbooth has quit IRC | 23:53 | |
*** mdbooth has joined #openstack-nova | 23:55 | |
*** mmethot has joined #openstack-nova | 23:55 | |
*** macz has quit IRC | 23:56 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!