Wednesday, 2025-03-26

opendevreviewmelanie witt proposed openstack/nova master: DNM try minimalistic imagebackend refactor  https://review.opendev.org/c/openstack/nova/+/92563502:05
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Move cache filename generation to imagebackend  https://review.opendev.org/c/openstack/nova/+/93096102:05
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Add base image format tracking with file extensions  https://review.opendev.org/c/openstack/nova/+/93096202:05
opendevreviewmelanie witt proposed openstack/nova master: db: Add image_type to block_device_mapping table  https://review.opendev.org/c/openstack/nova/+/93096402:05
opendevreviewmelanie witt proposed openstack/nova master: objects: Add image_type to BlockDeviceMapping object  https://review.opendev.org/c/openstack/nova/+/93096502:05
opendevreviewmelanie witt proposed openstack/nova master: virt: Add image_type to relevant DriverBlockDevice  https://review.opendev.org/c/openstack/nova/+/93096602:05
opendevreviewmelanie witt proposed openstack/nova master: WIP libvirt: Add image multi-backend  https://review.opendev.org/c/openstack/nova/+/93096702:05
opendevreviewmelanie witt proposed openstack/nova master: WIP trait, filter, request spec  https://review.opendev.org/c/openstack/nova/+/93215302:05
opendevreviewmelanie witt proposed openstack/nova master: api: Add microversion 2.97 to add image_type to block_device_mapping_v2  https://review.opendev.org/c/openstack/nova/+/93096802:05
opendevreviewmelanie witt proposed openstack/nova master: api: Add microversion 2.101 to add image_type to block_device_mapping_v2  https://review.opendev.org/c/openstack/nova/+/93096802:06
opendevreviewmelanie witt proposed openstack/nova master: libvirt: Add base image format tracking with file extensions  https://review.opendev.org/c/openstack/nova/+/93096202:25
opendevreviewmelanie witt proposed openstack/nova master: db: Add image_type to block_device_mapping table  https://review.opendev.org/c/openstack/nova/+/93096402:25
opendevreviewmelanie witt proposed openstack/nova master: objects: Add image_type to BlockDeviceMapping object  https://review.opendev.org/c/openstack/nova/+/93096502:25
opendevreviewmelanie witt proposed openstack/nova master: virt: Add image_type to relevant DriverBlockDevice  https://review.opendev.org/c/openstack/nova/+/93096602:25
opendevreviewmelanie witt proposed openstack/nova master: WIP libvirt: Add image multi-backend  https://review.opendev.org/c/openstack/nova/+/93096702:25
opendevreviewmelanie witt proposed openstack/nova master: WIP trait, filter, request spec  https://review.opendev.org/c/openstack/nova/+/93215302:25
opendevreviewmelanie witt proposed openstack/nova master: api: Add microversion 2.101 to add image_type to block_device_mapping_v2  https://review.opendev.org/c/openstack/nova/+/93096802:25
opendevreviewmelanie witt proposed openstack/nova master: db: Add image_type to block_device_mapping table  https://review.opendev.org/c/openstack/nova/+/93096403:09
opendevreviewmelanie witt proposed openstack/nova master: objects: Add image_type to BlockDeviceMapping object  https://review.opendev.org/c/openstack/nova/+/93096503:09
opendevreviewmelanie witt proposed openstack/nova master: virt: Add image_type to relevant DriverBlockDevice  https://review.opendev.org/c/openstack/nova/+/93096603:09
opendevreviewmelanie witt proposed openstack/nova master: WIP libvirt: Add image multi-backend  https://review.opendev.org/c/openstack/nova/+/93096703:09
opendevreviewmelanie witt proposed openstack/nova master: WIP trait, filter, request spec  https://review.opendev.org/c/openstack/nova/+/93215303:09
opendevreviewmelanie witt proposed openstack/nova master: api: Add microversion 2.101 to add image_type to block_device_mapping_v2  https://review.opendev.org/c/openstack/nova/+/93096803:09
elodillessimondodsley: hi, sorry, didn't ping you, but i wrote this: https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2025-03-25.log.html#t2025-03-25T13:04:5008:32
opendevreviewribaudr proposed openstack/nova master: FUP Remove unnecessary PCI check  https://review.opendev.org/c/openstack/nova/+/94410509:15
opendevreviewribaudr proposed openstack/nova master: FUP improve comment accuracy and variable naming for tag removal  https://review.opendev.org/c/openstack/nova/+/94312409:15
opendevreviewribaudr proposed openstack/nova master: FUP Add a warning to make non-explicit live migration request debugging easier  https://review.opendev.org/c/openstack/nova/+/94413309:15
opendevreviewribaudr proposed openstack/nova master: FUP Update pci-passthrough and virtual-gpu documentation  https://review.opendev.org/c/openstack/nova/+/94415309:15
opendevreviewribaudr proposed openstack/nova master: FUP: Improve libvirt fixture for hostdevs  https://review.opendev.org/c/openstack/nova/+/94523209:15
opendevreviewribaudr proposed openstack/nova master: FUP improve and add integration tests for PCI SR-IOV servers  https://review.opendev.org/c/openstack/nova/+/94410609:15
opendevreviewhuanhongda proposed openstack/nova master: [WIP]Only cleanup local deleted instance  https://review.opendev.org/c/openstack/nova/+/94554609:26
opendevreviewStefan K proposed openstack/nova-specs master: Add Cloud Hypervisor support spec  https://review.opendev.org/c/openstack/nova-specs/+/94554910:16
opendevreviewStefan K proposed openstack/nova-specs master: Add Cloud Hypervisor support spec  https://review.opendev.org/c/openstack/nova-specs/+/94554910:28
opendevreviewStefan K proposed openstack/nova-specs master: Add Cloud Hypervisor support spec  https://review.opendev.org/c/openstack/nova-specs/+/94554910:40
opendevreviewStefan K proposed openstack/nova-specs master: Add Cloud Hypervisor support spec  https://review.opendev.org/c/openstack/nova-specs/+/94554911:41
opendevreviewmelanie witt proposed openstack/nova master: Call volume detach rollback API if detach fails  https://review.opendev.org/c/openstack/nova/+/88039912:37
simondodsleyelodilles thank you for the response. I read this that I can create the backports for the 3 patches for Caracal and then the Nova team can decide... Correct me if I am wrong13:00
elodillessimondodsley: that sounds good to me13:21
opendevreviewSimon Dodsley proposed openstack/nova stable/2024.1: Fix device_type=lun with boot_index  https://review.opendev.org/c/openstack/nova/+/94559414:07
opendevreviewSimon Dodsley proposed openstack/nova stable/2024.1: Avoid setting serial on raw LUN devices  https://review.opendev.org/c/openstack/nova/+/94559514:08
opendevreviewSimon Dodsley proposed openstack/nova stable/2024.1: Enable virtio-scsi in nova-next  https://review.opendev.org/c/openstack/nova/+/94559614:09
zigoWhen attempting to launch tempest (to validate Epoxy), I'm getting this: https://paste.openstack.org/show/bAvPPkNaAQ5Ftw46O6xj/15:19
zigoDoes  anyone have a clue of what's going on?15:19
dansmiththe only clue I have is that it looks entirely unrelated to nova15:20
dansmithheat, tempest, keystone, maybe.. but not nova :)15:20
opendevreviewRajesh Tailor proposed openstack/nova master: Fix disable memballoon device  https://review.opendev.org/c/openstack/nova/+/94562115:21
zigoLooks like an issue in heat-tempest-plugin.15:25
zigoSorry ... :)15:25
zigoWill ask over there.15:25
zigoRemoving the heat plugin fixed it.15:25
opendevreviewDan Smith proposed openstack/nova master: Extend invalidate_rp to only invalidate cache  https://review.opendev.org/c/openstack/nova/+/94414815:43
opendevreviewDan Smith proposed openstack/nova master: Invalidate PCI-in-placement cached RPs during claim  https://review.opendev.org/c/openstack/nova/+/94414915:43
opendevreviewDan Smith proposed openstack/nova master: Support "one-time-use" PCI devices  https://review.opendev.org/c/openstack/nova/+/94381615:43
opendevreviewDan Smith proposed openstack/nova master: Add one-time-use devices docs and reno  https://review.opendev.org/c/openstack/nova/+/94426215:43
opendevreviewDan Smith proposed openstack/nova master: DNM: Test nova with placement alloc fix  https://review.opendev.org/c/openstack/nova/+/94562615:43
opendevreviewRajesh Tailor proposed openstack/nova master: Add support for showing instance-action finish_time  https://review.opendev.org/c/openstack/nova/+/92893316:59
*** iurygregory_ is now known as iurygregory17:26
dansmithsean-k-mooney: I guess the thing limiting us from testing pci passthrough upstream is the host iommu stuff, right?18:34
dansmithI guess I'm not sure now if nested kvm is actually required or not18:35
sean-k-mooneywe can enable viommu to enable that18:36
sean-k-mooneyor rather18:36
sean-k-mooneyto be able to allocate device to second level vm the first level vm need a viommu18:37
sean-k-mooneythe l0 host does not need an iommu configured unless you are doing doubel passthough18:37
dansmithso you mean the host (i.e. the cloud provider) would need to enable that for our instances, not that we could do that in our own images with grub18:37
sean-k-mooneyright we can enable it in the iamges we upload to them via nodepool18:38
dansmithwhy are we not already doing that?18:38
sean-k-mooneyso we can turn on teh viommu in the ci image and enabel igb18:38
sean-k-mooneyit kills performance18:38
dansmithah18:38
sean-k-mooneywell if you actully use the iommu emulation18:38
sean-k-mooneyif its on and you do nothing im not sure what the overhead is18:39
dansmithfor regular jobs that don't use it at all would it matter?18:39
dansmithokay18:39
dansmithalso, I've switched to using the virtio rng and it works very well and doesn't crash my first level guest each time.. any reason not to just use that for our test device?18:39
sean-k-mooneynone we just need a device that would not hard fault18:40
sean-k-mooneyif that work then we could do it today with our exisitng proviers18:40
dansmithack, the QXL kernel driver crashes in the first-level guest when it gets returned to it, but the rng seems to work fine18:40
sean-k-mooneywelll not teh xen rax nodes18:40
sean-k-mooney but most of the others18:40
dansmithokay maybe I'll see if that's doable18:41
sean-k-mooneywell i suspect the virtio-gpu might far better i used ot use virtio-blk device or virtio-net in the past18:41
sean-k-mooneyso what we need to so is add a new image and lable in node pool18:41
sean-k-mooneythen defien a new zuul nodeset that uses that and swawp say nova next to use that18:41
dansmithwhy better than the rng? I just figure either of the other types are more likely to be complicated and more failure-prone18:42
sean-k-mooneyoh it was more i could add several neturon prots or cinder volumes if i needed more18:42
sean-k-mooneybut ya no reason not to use the rng18:42
sean-k-mooneyits mainlly needed to avoid slow boot up when there is low entropy18:43
sean-k-mooneybut once the first level vm is up the kernel hsould have enough to conitnue without it18:43
dansmithoh I see, that would give us some extra devices I guess okay.. either way, if we get it all working it probably doesn't matter which we use and/or we could add aliases for all of them so jobs can configure their flavors18:44
sean-k-mooneydansmith: so this is how you add image metadata vai node pool https://github.com/openstack/project-config/blob/master/nodepool/nb07.opendev.org.yaml#L26-L2918:44
dansmithack18:45
sean-k-mooneyi belive our stadnar images are all defiend here https://github.com/openstack/project-config/blob/master/nodepool/nodepool.yaml#L191-L46118:47
sean-k-mooneythe other wones were arm sepcific versions18:47
sean-k-mooneyi dont think we need to modify the image build jsut the upload to each cloud18:48
sean-k-mooneyor a cloud i guess to start with18:48
sean-k-mooneydansmith: im sure you at least partly famiarly with this but the nodeset used for the jobs are typeiclly defined in devstack although we can define those in nova18:50
sean-k-mooneywe jsut need to refence the lable provided by node pool (the label is basicly and image an flavor mapped to each provider)18:50
sean-k-mooneyso you would basicly copy paste https://github.com/openstack/devstack/blob/master/.zuul.yaml#L154-L18218:51
sean-k-mooneyand change  label: ubuntu-noble ro  label: ubuntu-noble-with-iommu unless we just update  ubuntu-noble to turn it on18:51
sean-k-mooneyin which case you coudl skip that stpe but i woudl impact everyone18:52
dansmithvaguely, I don't think I've ever had to mess with it directly, but thanks for the pointers18:57
dansmithI wonder if we could ask the virt team if there's any harm in just enabling viommu if it's not used.. if so that would make it easy18:57
sean-k-mooneyso this is the chart i remember seeing before https://kvm-forum.qemu.org/2021/vIOMMU%20KVM%20Forum%202021%20-%20v4.pdf#page=1619:06
sean-k-mooneynot that really talkign about the affect of enabling the viommu on a pci device that is passhth though to the guest19:06
sean-k-mooneybe it an ssd or a nic19:06
sean-k-mooneyand we can see the overhead is a 60-80% performance loss19:07
sean-k-mooneybut i dont know if we will see any performance hit for virual device like virtio-net or virito-blk19:07
sean-k-mooneyas such in ci it might not matter19:08
sean-k-mooneythat also with kernel 5.12, 6.14 just came out so this has proably impvoed over the last 4 years 19:09
sean-k-mooneythe qemu they were using to test does not evne meet our current min requirements for nova and we are not exactly agressive with moving that forward19:10
sean-k-mooneyi think it would be wroth just trying it locally with some basic test with iperf and fio19:11
sean-k-mooneyand if you dont see any horible impact we could talk to the infra folks about trying it on one image in say the vexhost cloud or ovh adn see how it goes19:12
dansmithack, I should be able to do that.. maybe just tweaking the pool we run nova-next on or something and seeing if it looks to be performing the same for a week would be enough19:19
sean-k-mooneyyep19:21
dansmithsean-k-mooney: so, I thought the iommu changes I was making in my first-level guest were to share/expose the actual iommu to the guest,19:28
sean-k-mooneyno19:29
dansmithbut reading danpb's post I'm thinking what I've been using is actually just the viommu stuff that nova would be enabling19:29
sean-k-mooneyyes19:29
dansmithdon't mind me, first day on the job people!19:29
sean-k-mooneynova and qemu does not care if the host has an iommu or even if its enabeld19:29
dansmithack19:29
dansmithso, also from reading danpb's post it seems like the non-passthrough devices should be unchanged with it enabled, but I'll do some testing to make sure19:30
sean-k-mooneyack, that what i was expecting/hoping but i was unsure19:30
sean-k-mooneythe iommu can levage some host hardware accalration if its supproted but like vtpm its a purly vritual impelmetion by default19:31
sean-k-mooneyif i rememebr correctly we do enable caching by defualt https://libvirt.org/formatdomain.html#iommu-devices19:32
sean-k-mooneyya we do https://github.com/openstack/nova/blob/master/nova/virt/libvirt/driver.py#L7987-L801919:34
sean-k-mooneyso if the host support intel vt-d it can use it19:35

Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!