| gokhan | hello folks, I am trying to enable gpu virtualization on nova. I am trying to test on nvidia l40s gpu card. My os is ubuntu 22.04. I have only one gpu. I installed nvidia vgpu manager 18.1 on ubuntu and then running "ls /sys/class/mdev_bus/*/mdev_supported_types" command returned no such file. then I run "/usr/lib/nvidia/sriov-manage -e all" command and new pcis are created. "lspci -nn | grep -i nvidia" command returned newly created 32 gpus. it | 07:17 |
|---|---|---|
| gokhan | seems it is about sriov. I want to be sure this is expected behaviour.these are the gpu information on my system > https://paste.openstack.org/show/bpDV7Ie9xshYowaWgAOK/ The other issue is I can only create gpus with l40s-1a:2a, l40s-1q:2q and l40s-1b:2b but can not create l40s-12q. what can be reason for this problem ? can you help please? | 07:17 |
| opendevreview | Sylvain Bauza proposed openstack/nova master: [doc] Adding vGPUs max_instances caveat fix for virtual-gpu https://review.opendev.org/c/openstack/nova/+/949579 | 09:15 |
| bauzas | sean-k-mooney: gibi: we forgot to add upstram docs for the SRIOV GPU capping fix, could you please quickly review https://review.opendev.org/c/openstack/nova/+/949579 ? | 09:16 |
| sean-k-mooney | oh by capping fix you mena the max_instanaces config option sure | 10:34 |
| pas-ha[m] | gokhan: the NVIDIA drivers generally create more PCI devices than the actual max possible number of vGPUs they can be split to (even for the smallest size). You can use mdevctl or sysfs interface to check the actual 'available instances' on each mdev type on each PCI dev. After you created those mdev you can - check again, you will see that now everything would be 0, you've reached max capacity for your card, even if there are PCI | 10:36 |
| pas-ha[m] | devices left where you did not create any mdevs. | 10:36 |
| pas-ha[m] | Consult your card docs for what is possible with it, what profiles in which mode (single vGPU vs heterogeneous vGPU) and in which quantity are possible with it. | 10:36 |
| sean-k-mooney | we didnt entirly forget we added the help text as part of the config option but ya the improvment looks reasonable | 10:37 |
| sean-k-mooney | pas-ha[m]: ritght it basically create enouch VF for the smallest vgpu type it can support | 10:39 |
| sean-k-mooney | wich i get why they do that but its also not a great ux | 10:40 |
| hertrste | Hi. I would like to ask to get some further feedback on https://review.opendev.org/c/openstack/nova-specs/+/945549. Tank you! | 10:45 |
| gokhan | pas-ha[m], yes it is correct. L40S created 32 vf. I select nvidia-1153 as l40s-12q and nova/placement created 4 resource providers beacuse max instances are 4 in l40s-12. If I change from l40s-12q to l40s-1q, nova/placement creates 32 new resource providers. It maps every pci slot to one vm. It seem it is working as expected. | 10:52 |
| gokhan | https://paste.openstack.org/show/bdL9DHdT9pnIbGEFrbEO/ | 10:53 |
| gokhan | but sometimes when I try to change types, nova is deleting previous resource providers and not create new ones | 10:54 |
| gokhan | pas-ha[m], host vgpu mode is SR-IOV https://paste.openstack.org/show/bg5WeH9FWvpWhenTJA70/ | 10:58 |
| gokhan | I wonder can we create different vgpu types per VFs? | 11:00 |
| pas-ha[m] | gokhan: the problem is that very rarely you can really create that many resources, which leads to overreporting in placement. Nova does not know that of those 32 resource providers that each can handle 1 vgpu resource, only say 12 can really be occupied (because you set Nova to create that vgpu type that can fit only 12), mdev initially still reports that every of those 32 pci devs can hold 1 vgpu of your type... for that recent nova | 11:00 |
| pas-ha[m] | versions have 'max instances' option in the [mdev_...] section, to limit the amount of resource providers created. | 11:00 |
| pas-ha[m] | https://docs.nvidia.com/vgpu/17.0/grid-vgpu-user-guide/index.html#virtual-gpu-types-grid-reference - here is a comprehensive list of all the vgpu types _and their combinations_ (for the heterogeneous vGPU mode which your card supports, and which I guess is enabled when you put the card into the SR-IOV mode). | 11:00 |
| pas-ha[m] | again, check mdevctl types, it will show you 'available instances' for each mdev type on each PCI device - and mind that this number will change (everywhere, on all the PCI VFs) as you create new mdevs. | 11:03 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Adjust validation helpers for a single-method future https://review.opendev.org/c/openstack/nova/+/936365 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Stop using wsgi.Controller.api_version to switch between API versions https://review.opendev.org/c/openstack/nova/+/936366 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Only check minimum API version https://review.opendev.org/c/openstack/nova/+/940325 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add new, simpler api_version decorator https://review.opendev.org/c/openstack/nova/+/936367 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Only run format checks on strings https://review.opendev.org/c/openstack/nova/+/936368 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Simplify parameter types https://review.opendev.org/c/openstack/nova/+/936369 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: tests: Ensure all APIs have a response body schema https://review.opendev.org/c/openstack/nova/+/924598 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: doc: Add missing API samples https://review.opendev.org/c/openstack/nova/+/937012 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for hosts APIs https://review.opendev.org/c/openstack/nova/+/937047 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for instance actions https://review.opendev.org/c/openstack/nova/+/937048 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for hypervisors APIs (1/3) https://review.opendev.org/c/openstack/nova/+/937245 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for hypervisors APIs (2/3) https://review.opendev.org/c/openstack/nova/+/937246 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for hypervisors APIs (3/3) https://review.opendev.org/c/openstack/nova/+/937247 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server IPs APIs https://review.opendev.org/c/openstack/nova/+/937608 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for remote consoles https://review.opendev.org/c/openstack/nova/+/940114 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for keypairs APIs https://review.opendev.org/c/openstack/nova/+/940115 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for image metadata APIs https://review.opendev.org/c/openstack/nova/+/940299 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server topology API https://review.opendev.org/c/openstack/nova/+/940300 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server diagnostics API https://review.opendev.org/c/openstack/nova/+/940506 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server external events API https://review.opendev.org/c/openstack/nova/+/940508 | 11:05 |
| opendevreview | Stephen Finucane proposed openstack/nova master: api: Add response body schemas for server password APIs https://review.opendev.org/c/openstack/nova/+/945736 | 11:05 |
| gokhan | thanks for great explanation pas-ha[m] I got it :) | 11:12 |
| opendevreview | Stephen Finucane proposed openstack/nova stable/2024.1: add pyproject.toml to support pip 23.1 https://review.opendev.org/c/openstack/nova/+/949617 | 11:52 |
| gibi | bauzas: is it just me or nova-next is failing with mdev stuff https://zuul.opendev.org/t/openstack/builds?job_name=nova-next&skip=0 | 12:07 |
| gibi | 3237 Async compile_mdev_samples:45784]: finished compile_mdev_samples with result 1 in 37 seconds | 12:07 |
| gibi | https://zuul.opendev.org/t/openstack/build/b8c32faa528e4bddbfb027f43f1f184c/log/job-output.txt#21834 | 12:08 |
| sean-k-mooney | gibi: perhaps ocationally but it seam sto be passign for the most part unless it just started today? | 12:08 |
| gibi | filed bug https://bugs.launchpad.net/nova/+bug/2110545 | 12:10 |
| gibi | it started today | 12:10 |
| sean-k-mooney | https://zuul.opendev.org/t/openstack/build/f28f307aa79646f6a9a54965ae3592e6/log/controller/logs/devstacklog.txt#19105 | 12:11 |
| sean-k-mooney | so it looks like the kernel we downloaded is gettign a 404 | 12:11 |
| gibi | yeah it seems | 12:12 |
| sean-k-mooney | we may need to check how we are selectign that package | 12:12 |
| sean-k-mooney | this is either an infra issue where the package should be avioable | 12:12 |
| sean-k-mooney | or perhaps we hard coded it instead fo gettign the one for the currently installed kernel | 12:13 |
| sean-k-mooney | https://github.com/openstack/nova/blob/master/devstack/lib/mdev_samples#L40 | 12:13 |
| sean-k-mooney | sorry https://github.com/openstack/nova/blob/master/devstack/lib/mdev_samples#L30-L32 | 12:13 |
| sean-k-mooney | or perhaps https://github.com/openstack/nova/blob/master/devstack/lib/mdev_samples#L36 is what actully failing | 12:14 |
| sean-k-mooney | in any case we are using uname to get the runnign kernel | 12:14 |
| sean-k-mooney | so this is proably an issue with our mirros or an issue with the ubuntu repos | 12:15 |
| sean-k-mooney | we could temproally disabele the compile and reenable it in bauzas's serise where he is actully usign the compiled module | 12:15 |
| bauzas | mmmm, this is just for the moment compiling the kernel indeed | 12:16 |
| sean-k-mooney | right now we are building the mdev support but not actully useing it to test with mdevs becuase that patch is not merged yet | 12:16 |
| bauzas | yeah exacylu | 12:16 |
| sean-k-mooney | ok so the tar is gone form the ubuntu archive... | 12:17 |
| gibi | I vote for disabling this compliation and then search for a permanent solution | 12:18 |
| sean-k-mooney | +1 | 12:18 |
| bauzas | fine by me | 12:18 |
| bauzas | I need to look how to get the kernel examples then | 12:19 |
| sean-k-mooney | it looks like the drop all the kernle tars from that archive | 12:19 |
| gibi | I can push a patch... | 12:19 |
| sean-k-mooney | so they might have moved them | 12:19 |
| bauzas | gibi: please do | 12:19 |
| gibi | OK | 12:19 |
| gibi | on it | 12:19 |
| sean-k-mooney | their is nolonger any "linux_*" packages | 12:20 |
| sean-k-mooney | actuly there are no tars there anymroe so maybe that location now only has the debs and the source is at a new location | 12:20 |
| sean-k-mooney | so we might need to update the repos we nebale | 12:20 |
| bauzas | that's weird | 12:20 |
| sean-k-mooney | we are enableing the soruce repos https://github.com/openstack/nova/blob/master/devstack/lib/mdev_samples#L21-L23 | 12:21 |
| sean-k-mooney | so that shoudl be all that is required but perhaps we can reach out to some of the cannocal folks to understand what to do | 12:21 |
| sean-k-mooney | perhaps we should be using https://launchpad.net/ubuntu/noble/+package/linux-source | 12:22 |
| sean-k-mooney | instead of linux-image-unsigned-$kver | 12:22 |
| sean-k-mooney | hum https://canonical-kteam-docs.readthedocs-hosted.com/en/public/how-to/build-kernel.html#get-kernel-source-for-version-installed-on-build-machine at least suggest we are doing it properly | 12:24 |
| opendevreview | Balazs Gibizer proposed openstack/nova master: Disable NOVA_COMPILE_MDEV_SAMPLES to unblock CI https://review.opendev.org/c/openstack/nova/+/949622 | 12:24 |
| gibi | fix ^^ | 12:24 |
| *** tosky_ is now known as tosky | 12:27 | |
| jlejeune | 13:00 | |
| opendevreview | Elod Illes proposed openstack/nova master: [tool] Fix backport validator for non-SLURP https://review.opendev.org/c/openstack/nova/+/949628 | 13:15 |
| opendevreview | Antony Messerli proposed openstack/nova master: docs: Corrects a typo on a placement audit command https://review.opendev.org/c/openstack/nova/+/922850 | 13:25 |
| opendevreview | Takashi Natsume proposed openstack/nova master: Update contributor guide for 2025.2 Flamingo https://review.opendev.org/c/openstack/nova/+/944603 | 14:23 |
| dansmith | gibi: argh, that's frustrating | 14:35 |
| dansmith | if the kernel sources for the kernel we're running are ever not present, that should mean our image is (way) too old | 14:36 |
| dansmith | I wonder if they moved/hid them for the same reason RHEL did a while back.. that would really suck | 14:36 |
| gibi | dansmith: I did not dig but I think Sean did more | 14:45 |
| opendevreview | Konrad Gube proposed openstack/nova master: Use Cinder's os-extend_volume_completion volume action. https://review.opendev.org/c/openstack/nova/+/873560 | 14:46 |
| Uggla | Nova meeting in ~50mn | 15:10 |
| Uggla | Meeting in 5mn | 15:56 |
| Uggla | #startmeeting nova | 16:01 |
| opendevmeet | Meeting started Tue May 13 16:01:07 2025 UTC and is due to finish in 60 minutes. The chair is Uggla. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:01 |
| opendevmeet | The meeting name has been set to 'nova' | 16:01 |
| elodilles | o/ | 16:01 |
| fwiesel | o/ | 16:01 |
| Uggla | Hello everyone | 16:01 |
| masahito | o/ | 16:01 |
| gibi | o/ | 16:01 |
| Uggla | awaiting a moment for people to join. | 16:01 |
| bauzas | o/ | 16:02 |
| Uggla | #topic Bugs (stuck/critical) | 16:02 |
| Uggla | #info https://bugs.launchpad.net/nova/+bug/2110545 nova next is bloqued. | 16:02 |
| *** ralonsoh is now known as ralonsoh_out | 16:03 | |
| Uggla | # info https://review.opendev.org/c/openstack/nova/+/949622 proposal to skip this until the bug is fixed. | 16:03 |
| gibi | yepp I needed to recheck the disablement https://review.opendev.org/c/openstack/nova/+/949622 as it failed with a kernel panic :/ | 16:03 |
| Uggla | This issue was discussed before on the channel. | 16:03 |
| Uggla | I understood that kernel src vanished or are incorrect and so we can't build the tool. | 16:04 |
| bauzas | I'll try to find a solution with my mtty series | 16:04 |
| Uggla | the tool is for testing mdev, if I'm not wrong. | 16:05 |
| Uggla | gibi, bauzas anything you want to add ? | 16:05 |
| gibi | nope | 16:05 |
| Uggla | #topic Gate status | 16:06 |
| Uggla | #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs | 16:06 |
| Uggla | So the bug mentioned before is available ^ | 16:06 |
| Uggla | #link https://etherpad.opendev.org/p/nova-ci-failures-minimal | 16:07 |
| Uggla | #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status | 16:07 |
| Uggla | Btw I have checked these periodics they are all ok, despite some of them are a bit flaky. | 16:07 |
| Uggla | #info Please look at the gate failures and file a bug report with the gate-failure tag. | 16:08 |
| Uggla | #info Please try to provide meaningful comment when you recheck | 16:08 |
| Uggla | #topic tempest-with-latest-microversion job status | 16:08 |
| Uggla | #link https://zuul.opendev.org/t/openstack/builds?job_name=tempest-with-latest-microversion&skip=0 | 16:08 |
| gmaan | not much update this week except I made keypair test also passing now | 16:10 |
| gmaan | #link https://review.opendev.org/q/topic:%22latest-microversion-testing%22 | 16:10 |
| Uggla | Hi gmaan, I thought you were not in the call, thx for the update. | 16:10 |
| Uggla | gmaan, something else you want to add ? | 16:10 |
| gmaan | nothing else from me | 16:11 |
| Uggla | thx, so moving on | 16:11 |
| sean-k-mooney | o/ | 16:11 |
| Uggla | #topic Release Planning | 16:11 |
| Uggla | #link https://releases.openstack.org/flamingo/schedule.html | 16:11 |
| Uggla | #link https://releases.openstack.org/flamingo/schedule.html | 16:11 |
| Uggla | #info We reach Flamingo-1 milestone, only 3 weeks before Nova Spec Soft Freeze, do not forget to submit your specs. | 16:11 |
| bauzas | milestone 1 if I'm not wrong | 16:12 |
| bauzas | heh | 16:12 |
| Uggla | ? | 16:12 |
| bauzas | just saying where we are | 16:12 |
| bauzas | but jinxed | 16:13 |
| Uggla | yes Flamingo-1 milestone ^ | 16:13 |
| masahito | quick question for the spec freeze. is it deadline to merge a spec or deadline to submit spec? | 16:13 |
| elodilles | yepp, this week is Flamingo-1 :] | 16:13 |
| sean-k-mooney | milestone 1 is thursday | 16:13 |
| sean-k-mooney | so we shoudl be creatign the first release of our libs like os-vif | 16:14 |
| sean-k-mooney | if they have not had one already | 16:14 |
| elodilles | sean-k-mooney: release patches are already generated where were (functional) code changes | 16:14 |
| sean-k-mooney | ack | 16:14 |
| Uggla | masahito, soft spec is deadline to submit spec | 16:14 |
| Uggla | spec freeze is deadline to merge the spec | 16:15 |
| elodilles | sean-k-mooney: with a quick check i don't see any nova related flamingo-1 release patch ( https://review.opendev.org/q/topic:flamingo-milestone-1 ) | 16:15 |
| masahito | got it. i understand i need to submit the spec and poc code quickly. | 16:16 |
| sean-k-mooney | elodilles: we definally have had some functional changes. althogu i know for os-trait for one we already had at least 1 release | 16:16 |
| Uggla | masahito, mainly the code. | 16:17 |
| Uggla | oops | 16:17 |
| Uggla | sorry mainly the spec | 16:17 |
| sean-k-mooney | masahito: you tecnially can submit a spec after the soft freeze but the later you do the more risk peole will run out of time to review | 16:17 |
| elodilles | sean-k-mooney: that is 'cycle independent' project, so that one is not closely followed by release team | 16:18 |
| masahito | thanks | 16:18 |
| sean-k-mooney | elodilles: ack, we can likely move on to the next topic? | 16:18 |
| elodilles | sean-k-mooney: ++ | 16:18 |
| Uggla | sean-k-mooney, elodilles question should I propose a release patch for os-trait ? | 16:20 |
| bauzas | that's an independent | 16:20 |
| bauzas | no need | 16:20 |
| elodilles | Uggla: there is no deadline for os-traits to Flamingo-1, but it can be proposed any time you want | 16:20 |
| Uggla | ok thx I just wanted to be sure. | 16:21 |
| bauzas | i mean independent release cadence | 16:21 |
| Uggla | #topic Review priorities | 16:21 |
| Uggla | #link https://etherpad.opendev.org/p/nova-2025.2-status | 16:21 |
| elodilles | on the other hand it's good to release when there are merged patches, so that it won't be forgotten during the cycle o:) | 16:22 |
| bauzas | Uggla: I could show you up where to find the release models for each of the projects we have in OpenStack | 16:22 |
| sean-k-mooney | i know mstill would like to do one for the vdi stuff but we can just do that out of band | 16:22 |
| Uggla | Just to let you know the the title of the channel was changed to point to the correct document | 16:23 |
| sean-k-mooney | i.e. we have a reqeust to do a release but we dont need to do one proceudrlly on any deadline we just should to unblock them | 16:23 |
| sean-k-mooney | Uggla: yes i want to add an opendicssion topic related to that btu we can talk about that when we get there | 16:23 |
| Uggla | ok | 16:24 |
| Uggla | #topic Stable Branches | 16:24 |
| Uggla | time for you elodilles | 16:24 |
| elodilles | o7 | 16:24 |
| elodilles | #info stable branches (stable/2025.1 and stable/2024.*) seem to be in OK state | 16:25 |
| elodilles | #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci | 16:25 |
| elodilles | that's all from me | 16:25 |
| Uggla | cool thx | 16:25 |
| Uggla | #topic vmwareapi 3rd-party CI efforts Highlights | 16:25 |
| Uggla | fwiesel, anything special ? | 16:25 |
| fwiesel | No updates from my side. | 16:25 |
| Uggla | thx | 16:26 |
| Uggla | #topic Gibi's news about eventlet removal. | 16:26 |
| Uggla | #link Series: https://gibizer.github.io/categories/eventlet/ | 16:26 |
| gibi | latest blog post is https://gibizer.github.io/posts/Eventlet-Removal-The-First-Threading-Bug/ | 16:26 |
| Uggla | #link reminder patches to review: https://review.opendev.org/q/topic:%22eventlet-removal%22+project:openstack/nova | 16:26 |
| gibi | working on cleaning up the nova-scheduler series | 16:27 |
| gibi | review is appreciated on https://review.opendev.org/c/openstack/nova/+/948437?usp=search | 16:27 |
| gibi | and on https://review.opendev.org/c/openstack/nova/+/948437?usp=search | 16:27 |
| gibi | I mean https://review.opendev.org/c/openstack/nova/+/948437?usp=search | 16:27 |
| gibi | somebody break my copy paste buffer :) | 16:28 |
| bauzas | I so would like to help reviewing | 16:28 |
| bauzas | I'll ty to find some time | 16:28 |
| gibi | https://review.opendev.org/c/openstack/nova/+/947265 | 16:28 |
| gibi | https://review.opendev.org/c/openstack/nova/+/948437 | 16:28 |
| gibi | the two patches to land | 16:28 |
| gibi | the long series is not ready but working on it | 16:28 |
| sean-k-mooney | 37 is pretty short so i can proably look at that after the meeting | 16:28 |
| gibi | that is all from me | 16:28 |
| sean-k-mooney | and the hacking one i gues | 16:28 |
| gibi | yepp | 16:29 |
| Uggla | #topic Open discussion | 16:30 |
| fwiesel | That would be my topic, I suppose. | 16:30 |
| Uggla | fwiesel, can you go ahead with your topic | 16:30 |
| fwiesel | Support for libvirt VIR_MIGRATE_PARAM_PARALLEL_CONNECTIONS | 16:30 |
| fwiesel | So, since QEMU 2.11 and libvirt 5.2.0 there is support for using multiple threads in the precopy operation. | 16:31 |
| sean-k-mooney | oh :) that could be useful | 16:31 |
| fwiesel | I was wondering, if I can just to a specless blueprint for that? | 16:31 |
| sean-k-mooney | i think that would be supported by our min qemu/libvirt versions | 16:31 |
| sean-k-mooney | so it would really just be a config option to allow enabling it | 16:32 |
| sean-k-mooney | and perhaps dicussing if it should be on or off by defualt | 16:32 |
| gibi | would it be a compute config param to enable? Does it only need to be enabled on the source node? | 16:32 |
| Uggla | I think because it require a config option a small spec will be worth it. | 16:32 |
| sean-k-mooney | this is really only useful if you have more bandwith aviable on the migration link then a singel core can saturate. | 16:32 |
| fwiesel | Ah, so a spec. I can do that | 16:33 |
| fwiesel | Yes, we need roughly 12 threads to saturate our 200GigE links. | 16:33 |
| sean-k-mooney | well it could be specless maybe but yes i blive this need to be set on the source | 16:33 |
| Uggla | yes so we can bikeshed on the config name. :) | 16:33 |
| sean-k-mooney | like the other migration parmaters i dont think it can be changed once the migration is started and that is done form the source node | 16:33 |
| Uggla | sean-k-mooney, this is a parameter to pass to migrateToURI3, if I'm not wrong. | 16:34 |
| sean-k-mooney | fwiesel: based on that math if you have a 25G link or grather you would benifit form this | 16:34 |
| sean-k-mooney | Uggla: yes i belive it is | 16:34 |
| gibi | Thanks. I'm OK with a specless but fine with a spec too | 16:35 |
| Uggla | sean-k-mooney, I looked quickly before the call. | 16:35 |
| sean-k-mooney | i belvie we were asked about this either in the ptg or on irc shourtly before it too | 16:35 |
| Uggla | I think the spec is cool to discuss names and defaults | 16:35 |
| Uggla | sean-k-mooney, more or less this is from masahito but this is another settigns | 16:36 |
| sean-k-mooney | i think it makes sense to do so sepless or small spec is ok with me | 16:36 |
| sean-k-mooney | *specless or a small spec | 16:36 |
| sean-k-mooney | we can debate the name in the gerrit review | 16:36 |
| fwiesel | The consensus seems then a small spec? | 16:36 |
| sean-k-mooney | presumable its a new option in the libvirt section | 16:37 |
| bauzas | I'd say that depends on the default value | 16:37 |
| sean-k-mooney | like https://docs.openstack.org/nova/latest/configuration/config.html#libvirt.live_migration_inbound_addr | 16:37 |
| bauzas | meaning whether we would have upgrade concerns | 16:37 |
| fwiesel | I'd gone with a default that keeps the old settings. | 16:37 |
| fwiesel | So, it is opt-in | 16:37 |
| sean-k-mooney | if we default to off. i dont think there would be an upgrade concern | 16:38 |
| sean-k-mooney | and we can consider changing the default in 2026.2+ | 16:38 |
| bauzas | yup so then I specless works for me | 16:38 |
| sean-k-mooney | we proably shoudl test it in nova-next | 16:38 |
| gibi | opt-in works for me | 16:38 |
| Uggla | ok so go for spec less | 16:38 |
| bauzas | that's an easy enablement knob | 16:38 |
| fwiesel | Perfect, saves me some work. | 16:38 |
| sean-k-mooney | fwiesel: can you create a bluepirnt for this to tack it and detail that it will be off by default | 16:39 |
| sean-k-mooney | unless you already have one? | 16:39 |
| fwiesel | sean-k-mooney: will do. I haven't yet. | 16:39 |
| Uggla | fwiesel, and ping me, so I would add it to the tracking doc. | 16:39 |
| fwiesel | Uggla: Will do | 16:39 |
| Uggla | thx | 16:40 |
| Uggla | anything else to discuss, if not we can triage some bugs. | 16:40 |
| sean-k-mooney | i have one quick item | 16:40 |
| Uggla | shoot | 16:41 |
| sean-k-mooney | so last week i asked frickler i think to update the channel topic if Uggla aggreed | 16:41 |
| sean-k-mooney | which has now been done but they pointed out that we coudl also add ops to the channle https://opendev.org/openstack/project-config/src/commit/50197dce1589385b1256150bcd082093cf7e8898/accessbot/channels.yaml#L239 | 16:41 |
| sean-k-mooney | nova currently does nto have any | 16:41 |
| gibi | lets add some :) | 16:41 |
| sean-k-mooney | do we want to do that so we can update the topic ectra our selves | 16:41 |
| bauzas | I was thinking Dan was +o | 16:41 |
| sean-k-mooney | dan might have it form something else | 16:42 |
| sean-k-mooney | but we dont ahve anyone lised in the file today | 16:42 |
| Uggla | I looked at it only gmaan if I recall well. | 16:42 |
| bauzas | fair | 16:42 |
| sean-k-mooney | shall i create a patch to do it? and if so who should we add | 16:42 |
| dansmith | I was pretty sure I had ops at one point | 16:42 |
| gmaan | I can do | 16:42 |
| gmaan | I think I still have ops | 16:43 |
| sean-k-mooney | Uggla: im also ok to delgate doign that to someone else | 16:43 |
| bauzas | I could add french jokes to the chan title | 16:43 |
| bauzas | but hey the PTL should have rights | 16:43 |
| gmaan | I have from global ops, ++ to add nova specific also | 16:44 |
| Uggla | sean-k-mooney, I can create this patch no worries. | 16:44 |
| sean-k-mooney | Uggla: cool that all then | 16:44 |
| sean-k-mooney | i just wanted to make sure we could udate the topic when needed | 16:44 |
| Uggla | who wants to be ops ? | 16:44 |
| bauzas | you should be | 16:45 |
| gmaan | do we also need to update topic now or all good for now? | 16:45 |
| Uggla | gmaan, topic is ok atm. | 16:45 |
| gmaan | k | 16:45 |
| sean-k-mooney | gmaan: it should be good now | 16:45 |
| sean-k-mooney | the only thing that need to change was the status url | 16:45 |
| gmaan | ++ | 16:46 |
| sean-k-mooney | it was pointing to https://etherpad.opendev.org/p/nova-2025.1-status until last week | 16:46 |
| sean-k-mooney | Uggla: that shoudl also proably get added to the nova PTL guide | 16:46 |
| sean-k-mooney | as a thing to do at the start of each cycle | 16:46 |
| Uggla | sean-k-mooney, ++ | 16:46 |
| bauzas | yup | 16:46 |
| bauzas | good idea | 16:47 |
| Uggla | #action Update ptl guide to mention topic update | 16:47 |
| sean-k-mooney | we have about 12 minutes left. want to traige a bug or two | 16:47 |
| Uggla | yep | 16:47 |
| Uggla | #topic Bug scrubbing | 16:47 |
| Uggla | #link: https://etherpad.opendev.org/p/nova-bug-selection-for-triaging#L4 | 16:47 |
| Uggla | First one: https://bugs.launchpad.net/nova/+bug/2106500 | 16:48 |
| Uggla | I'm not sure we are concerns by this one. | 16:48 |
| Uggla | That looks more a packaging pb. | 16:49 |
| gibi | graceful shutdown issue maybe | 16:50 |
| sean-k-mooney | im not sure that we shold be providing log rotate config | 16:51 |
| sean-k-mooney | actully | 16:51 |
| sean-k-mooney | https://github.com/openstack/nova/tree/master/etc/nova | 16:51 |
| sean-k-mooney | i dont think we ar | 16:51 |
| sean-k-mooney | *are | 16:52 |
| sean-k-mooney | so this may be a packaging issue yes | 16:52 |
| sean-k-mooney | os it proably shoudl be marked invalid in nova but left open for the nova ubuntu package | 16:52 |
| Uggla | I propose to answer in that way and set the bug to incomplete. Sounds good ? | 16:53 |
| sean-k-mooney | so i would put it to invalid personally | 16:53 |
| sean-k-mooney | we do not provide a logroate script in our repo | 16:54 |
| sean-k-mooney | i think this is part of the debian package | 16:54 |
| fwiesel | Uggla: Here is the blueprint https://blueprints.launchpad.net/nova/+spec/libvirt-migrate-parallel | 16:54 |
| Uggla | fwiesel, thx | 16:54 |
| Uggla | sean-k-mooney, ok that works for me. | 16:54 |
| Uggla | next one: | 16:54 |
| Uggla | https://bugs.launchpad.net/nova/+bug/2103413 | 16:54 |
| gibi | do we officially support 3.13? | 16:56 |
| Uggla | We are not supporting 3.13 yet. | 16:56 |
| sean-k-mooney | no yet | 16:56 |
| sean-k-mooney | its part of the experimantal runtime for this release | 16:57 |
| sean-k-mooney | and offically should be supported in 2026.1 | 16:57 |
| gmaan | not yet but we should start testing it as it can be mandatory for next cycle | 16:57 |
| gmaan | yeah | 16:57 |
| sean-k-mooney | however ubuntu 25.04 is already using it | 16:57 |
| sean-k-mooney | and its an issue for debian 13 also i think | 16:57 |
| gibi | it is probably yet another eventlet issue we might or might not able to patch | 16:57 |
| sean-k-mooney | eventlet is curently broken on it in general | 16:57 |
| Uggla | and also it might be linked to gibi's work on eventlet. | 16:57 |
| sean-k-mooney | only by name | 16:58 |
| gibi | the eventlet work will not be finished in F so if in G we need to support 3.13 then we are in the pickle | 16:58 |
| sean-k-mooney | so if someone knwo how to make it work on 3.13 and can propose a fix | 16:58 |
| sean-k-mooney | then we shoudl review it and conside rbackporting to enable debian based distros to supprot epoxy | 16:59 |
| sean-k-mooney | but its not offically supproted there so its best effort | 16:59 |
| sean-k-mooney | gibi: well i know that eventlet on 3.13 has issues with the threadid | 16:59 |
| sean-k-mooney | as in its ment to patch the thread id so that it returns the greenthread id instead | 17:00 |
| gibi | this seems to be a GC + eventlet issue | 17:00 |
| sean-k-mooney | and thats not workign properly | 17:00 |
| gibi | I would not bet on an easy fix | 17:00 |
| gibi | not in F. | 17:00 |
| gibi | in G we might be able to get rid of eventlet :) | 17:00 |
| sean-k-mooney | that one way to fix the issue :) | 17:00 |
| gibi | that is the only way I want to bet on :) | 17:01 |
| sean-k-mooney | https://github.com/eventlet/eventlet/issues/1030 | 17:01 |
| sean-k-mooney | so this was the issue i was aware of that was the blocker for nova on 3.13 | 17:01 |
| sean-k-mooney | you dont actully need nova to trigger it | 17:01 |
| sean-k-mooney | but it happens for nova too | 17:02 |
| sean-k-mooney | ah and https://github.com/eventlet/eventlet/issues/1032 | 17:02 |
| sean-k-mooney | is the issue we are talking about | 17:02 |
| sean-k-mooney | so we might want to mark it incomplete or invalid and refnrence that https://github.com/eventlet/eventlet/issues/1032 | 17:03 |
| sean-k-mooney | basicaly eventlet needs to supprot 3.13 before we can | 17:03 |
| Uggla | yep I can explain the situation in the ticket. | 17:03 |
| gibi | OK | 17:03 |
| JayF | Those issues getting fixed were blocked by CI + some administrivia in that repo which I think are resolved now. | 17:03 |
| masahito | sorry for the slow reply for the 1st bug. the attach volume API makes 2 RPC from nova-api to nova-compute, 1st one is call RPC and 2nd one is cast RPC. if the nova-api gets failed between the 2 rpc, garbage block device information remaining. | 17:04 |
| masahito | (I think it's okay to talk the details later.) | 17:04 |
| Uggla | there is also a status deferred for the ticket | 17:05 |
| Uggla | Are you ok to do one more bug ? | 17:06 |
| sean-k-mooney | sure | 17:06 |
| Uggla | https://bugs.launchpad.net/nova/+bug/2103413 | 17:06 |
| sean-k-mooney | that seams to be mostly the same thing? | 17:07 |
| sean-k-mooney | wiat its exaclty the same bug | 17:07 |
| sean-k-mooney | Uggla: did you mean to post a differnt url? | 17:08 |
| gibi | then we are done :) | 17:08 |
| Uggla | https://bugs.launchpad.net/nova/+bug/2103513 | 17:08 |
| Uggla | sorry wrong copy pasta | 17:08 |
| gibi | I have no clue about TLS spice | 17:09 |
| sean-k-mooney | so i dont know if nova actully supprot this yet | 17:09 |
| sean-k-mooney | i remember talking about this in the recent spci enabling | 17:10 |
| Uggla | what it is on dalmatian | 17:10 |
| sean-k-mooney | right but im not sure if that is expected to work even on master | 17:10 |
| Uggla | oh | 17:10 |
| sean-k-mooney | it might | 17:10 |
| sean-k-mooney | but i dont belilve we have tempest testing of it currently | 17:10 |
| sean-k-mooney | we only recnetly readded spice testing last cycle | 17:11 |
| sean-k-mooney | after not having any tempest testing with it for years | 17:11 |
| sean-k-mooney | lets loop back to this again next week | 17:11 |
| sean-k-mooney | and maybe ask mikal about it | 17:11 |
| Uggla | ok that sounds good. | 17:12 |
| Uggla | At least I can right a comment about it. | 17:12 |
| Uggla | that's all for today. | 17:12 |
| sean-k-mooney | https://review.opendev.org/c/openstack/nova/+/922544 | 17:13 |
| sean-k-mooney | ok in theory that was added in dalmation | 17:13 |
| sean-k-mooney | but perhaps only when usign kerbside | 17:13 |
| sean-k-mooney | in anycase we can wrap for today | 17:14 |
| Uggla | Thanks all and thanks for the extended time for bug triage. | 17:14 |
| Uggla | #endmeeting | 17:14 |
| opendevmeet | Meeting ended Tue May 13 17:14:33 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 17:14 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/nova/2025/nova.2025-05-13-16.01.html | 17:14 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/nova/2025/nova.2025-05-13-16.01.txt | 17:14 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/nova/2025/nova.2025-05-13-16.01.log.html | 17:14 |
| gibi | Uggla: thanks | 17:14 |
| masahito | thanks | 17:14 |
| Uggla | you are welcome | 17:15 |
| Uggla | Fyi I will fill the tickets tomorrow morning. I need to afk now to feed the family. | 17:16 |
| opendevreview | Merged openstack/nova master: Replace eventlet semaphores with threading one https://review.opendev.org/c/openstack/nova/+/948437 | 18:39 |
| opendevreview | Merged openstack/nova master: [hacking] N373 do not use eventlet primitives https://review.opendev.org/c/openstack/nova/+/947265 | 18:40 |
| opendevreview | Dan Smith proposed openstack/nova master: Example NVMe cleaning script for one-time-use https://review.opendev.org/c/openstack/nova/+/948446 | 20:02 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!