16:00:40 <bauzas> #startmeeting nova 16:00:40 <opendevmeet> Meeting started Tue Feb 25 16:00:40 2025 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:40 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:40 <opendevmeet> The meeting name has been set to 'nova' 16:01:40 <gibi> sean-k-mooney: https://paste.opendev.org/show/bEWhDvmH5NPNozp8V4pU/ confirmed the pci_stats also cleaned 16:02:12 <sean-k-mooney> gibi: cool i expected it to be. its just it should not be there at all. but we can adress that in a trivial followup 16:02:20 <sean-k-mooney> or dicsuss it more after the meeting 16:02:36 <bauzas> #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 16:02:42 <elodilles> o/ 16:02:44 <bauzas> howdy folks, sorry was late 16:02:54 * bauzas is trampled by many things at same time 16:03:02 <sean-k-mooney> o/ 16:03:11 <bauzas> let's start 16:03:23 <bauzas> #topic Bugs (stuck/critical) 16:03:24 <masahito> o/ 16:03:32 <bauzas> #info No Critical bug 16:03:39 <bauzas> #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster 16:04:09 <bauzas> any bugs to talk ? 16:04:56 <bauzas> looks not 16:05:02 <bauzas> #topic Gate status 16:05:12 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:05:17 <bauzas> #link https://etherpad.opendev.org/p/nova-ci-failures-minimal 16:05:27 <bauzas> (I'll leave the next PTL to curate that long list) 16:05:34 <bauzas> #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status 16:05:40 <bauzas> #info Please look at the gate failures and file a bug report with the gate-failure tag. 16:05:46 <bauzas> #info Please try to provide meaningful comment when you recheck 16:05:56 <bauzas> for me, the gate is correct 16:06:30 <bauzas> yet a few rechecks because of the usual suspects (guest panic and ssh timeouts) but nothing new and putting epoxy-3 at risk 16:06:39 <sean-k-mooney> bauzas: there is perhaps one 16:06:49 <Uggla> o/ 16:06:58 <sean-k-mooney> one of the funtional test for vgpu multi type is failing randomly 16:07:08 <bauzas> hmmm 16:07:15 <sean-k-mooney> bauzas: i dont have the link but i pingged it in the channel yesterday 16:07:16 <bauzas> that may be related to the leaky greenlet 16:07:26 <sean-k-mooney> it triggering the greenlet leak detachtion 16:07:32 <sean-k-mooney> right it is 16:07:37 <bauzas> yeah, but that's not new 16:07:40 <sean-k-mooney> so that a bug in the test 16:07:40 <fwiesel> o/ 16:07:53 <sean-k-mooney> well that somethign that shoudl be fixed as its causing extra rechecks 16:07:59 <bauzas> so maybe some side effect happened that would increase the failure rate 16:08:03 <sean-k-mooney> its not critical but we should not ignore it 16:08:08 <bauzas> I don't disagree 16:08:18 <sean-k-mooney> bauzas: if there is an excption in the leaked greenthread 16:08:27 <sean-k-mooney> it cn cause other test to fail becuase of that 16:08:45 <bauzas> I can try to take a look on it, particularly given I have another bugfix patch I'd want to be merged before RC1 16:08:47 <sean-k-mooney> so it cause hard to diagnose inter test interaction which is why we added the leak detection 16:09:00 <sean-k-mooney> ack its more an fyi 16:09:05 <bauzas> cool 16:09:20 <bauzas> #topic Release Planning 16:09:32 <bauzas> #link https://releases.openstack.org/epoxy/schedule.html 16:09:56 <bauzas> #info Nova deadlines are set in the above schedule 16:10:07 <bauzas> #info 2 days before Feature Freeze 16:10:07 <elodilles> note that there are client library release patches open: 16:10:15 <elodilles> https://review.opendev.org/c/openstack/releases/+/942534 16:10:21 <elodilles> https://review.opendev.org/c/openstack/releases/+/942551 16:10:39 <elodilles> (osc-placement and python-novaclient) 16:11:05 <elodilles> just an fyi as we have Epoxy-3 now o:) 16:11:06 <sean-k-mooney> so those have the same freeze deadline as nova right 16:11:08 <bauzas> noted, I'll review them 16:11:10 <sean-k-mooney> i.e. thrusday 16:11:18 <elodilles> yepp 16:11:30 <sean-k-mooney> but we will want to do a release at m3 to test with them for RC1 16:11:41 <elodilles> +1 16:12:11 <sean-k-mooney> we shoudl not haver anything pending for osc-palcement or novaclient that im aware of 16:12:18 <sean-k-mooney> so we can probly do those effectivly now 16:12:50 <sean-k-mooney> on clients 16:13:08 <sean-k-mooney> we have some new api versions this cycle and im not sure we will have osc suprot for those in time 16:13:20 <sean-k-mooney> so they may get enabled next cycle 16:13:54 <sean-k-mooney> but there isnt really anything we can do there at this point if the sdk supprot is not already merged 16:14:59 <sean-k-mooney> elodilles: am i correct in saying the sdk is subject to the non clinet code freeze so we should not expect to merge supprot for spice direct or image property shod api microversion between now and FF 16:15:32 <elodilles> sean-k-mooney: if i'm not mistaken sdk already merged it's final release patch recently 16:15:48 <sean-k-mooney> ack that what i was expecting 16:15:59 <elodilles> well, 16:16:05 <elodilles> it's still open, 16:16:32 <elodilles> waiting for a 2nd +2: https://review.opendev.org/c/openstack/releases/+/941859 16:16:38 <elodilles> from release team 16:17:06 <sean-k-mooney> i think we are missing a finaly release for os-traits and perhaps os-resouce classes too by the way 16:17:45 <elodilles> those are 'independent' projects, hence not picked up by release tools 16:17:46 <sean-k-mooney> we have not merged any content since the last release we did for either 16:17:53 <sean-k-mooney> ah ok 16:17:57 <elodilles> but wasn't there os-traits release recently? 16:18:30 <sean-k-mooney> yes but we had traits we wanted to add and have anouther release 3 days after that happend:) 16:18:49 <sean-k-mooney> the os-traits patch is still pending a second +2 16:18:56 <elodilles> sean-k-mooney: this is the content of SDK release (if it doesn't change anymore): https://zuul.opendev.org/t/openstack/build/24cc79c0e8e84d7d853e679e4e928585/log/tox/list-changes/list-changes-results.log#918-947 16:19:13 <opendevreview> Jay Faulkner proposed openstack/nova master: PoC: Allow unversioned-notify specific transport url https://review.opendev.org/c/openstack/nova/+/942710 16:19:26 <sean-k-mooney> ok we can likely move on and revisit in open dicuss if there is more to bring up 16:19:49 <elodilles> +1 16:21:19 <bauzas> can we move on ? 16:21:41 <elodilles> yepp 16:23:02 <bauzas> cool 16:23:13 <bauzas> #topic Review priorities 16:23:20 <bauzas> #link https://etherpad.opendev.org/p/nova-2025.1-status 16:25:30 <bauzas> nothing to tell about, I'm doing hard reviews 16:25:39 <bauzas> anything else ? 16:26:08 <bauzas> looks not 16:26:15 <opendevreview> Doug Goldstein proposed openstack/nova master: ironic: fix logging of validation errors https://review.opendev.org/c/openstack/nova/+/942019 16:26:17 <bauzas> #topic PTG planning 16:26:23 <bauzas> #info Next PTG will be held on Apr 7-11 16:26:38 <bauzas> usual reminder, please fill the agenda ! 16:26:42 <bauzas> #link https://etherpad.opendev.org/p/nova-2025.2-ptg 16:27:59 <bauzas> #topic Stable Branches 16:28:04 <bauzas> elodilles: your turn 16:28:17 <elodilles> ACK 16:28:23 <elodilles> actually, same as last week: 16:28:31 <elodilles> #info stable gates seem to be healthy 16:28:39 <elodilles> #info stable/2023.2 release patch: https://review.opendev.org/c/openstack/releases/+/941420 (stable/2023.2 (bobcat) is going to transition to EOL in ~2 months) 16:28:47 <elodilles> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci 16:28:57 <elodilles> that's all 16:31:22 <bauzas> cool 16:31:34 <bauzas> #topic vmwareapi 3rd-party CI efforts Highlights 16:31:37 <bauzas> fwiesel: around ? 16:31:48 <fwiesel> No updates from my side. I am a bit under the weather. 16:33:08 <bauzas> ditto here 16:33:10 <bauzas> I understand you 16:35:38 <bauzas> #topic Open discussion 16:35:45 <bauzas> I have one item 16:36:01 <bauzas> kudos to Uggla for being our next PTL \o/ 16:36:10 <Uggla> thx 16:36:25 <gibi> congrats 16:36:40 <gibi> and thanks bauzas for the long service as PTL 16:36:45 <fwiesel> Congrats 16:36:53 <elodilles> yepp ++ \o/ 16:37:27 * Uggla hoping bauzas will still be around to help me jumping in that role. ;) 16:38:26 <bauzas> I will ! 16:38:29 <masahito> congrats and thank you 16:38:39 <bauzas> anyway, moving to the questions 16:39:09 <bauzas> (hertrste) Nova Cloud Hypervisor support 16:39:23 <hertrste> Hi o/ 16:39:55 <hertrste> We would like to add support for the Cloud Hypervisor VMM to openstack Nova 16:40:12 <hertrste> As we have not contributed anything yet, we would like some guidance how to approach that 16:40:29 <sean-k-mooney> this yes https://github.com/cloud-hypervisor/cloud-hypervisor 16:40:31 <hertrste> As you can see in the meeting notes, we have a proof of concept 16:40:40 <hertrste> exactly 16:41:08 <sean-k-mooney> i breifly looked at that a few months ago out of interest when i was playing with rust 16:41:13 <hertrste> We also have a proof of concept adding some basic support for that VMM: https://github.com/openstack/nova/commit/617ac2f655492d1b84b2d924412ac7063d231110 16:41:14 <sean-k-mooney> it has an interesting feature set 16:41:29 <gibi> I think the next step there is to have PTG topic proposed for this 16:42:12 <sean-k-mooney> hertrste our main concerns would be likely testing, and building out a minimal supportrable feature set 16:42:36 <bauzas> yeah, looks to me a good PTG session 16:42:39 <sean-k-mooney> that would invovle defining a spec (if the propsal to add a new driver was green lit) 16:42:42 <bauzas> hertrste: can you attend it ? 16:42:44 <dansmith> I'd also be interested in knowing how widely it is used/deployed 16:42:44 <sean-k-mooney> to descibe the expecations 16:43:05 <dansmith> and also, why it's not just a libvirt driver :) 16:43:50 <sean-k-mooney> well i dont neesislay think we shoudl say all linux hyperviors shoudl be behind libvirt 16:43:55 <hertrste> I can very much attend a PTG. What would I need to prepare for that? When would that meeting take place? 16:43:57 <sean-k-mooney> it has its pros and cons 16:44:43 <dansmith> sean-k-mooney: no, doesn't have to be, but there's a lot of stuff to re-implement in nova if it's not, and so I'd expect there to be a good reason for it 16:44:50 <gibi> hertrste: https://openinfra.org/ptg/ 16:44:54 <bauzas> hertrste: see the above discussion about PTG 16:45:32 <Uggla> hertrste, Apr 7-11 16:45:38 <sean-k-mooney> hertrste: i think the main things to prepare is an elevator pitch explaining why an operator may want ot use this 16:45:53 <sean-k-mooney> wwhat usecases and target workloads woudl it enable 16:46:02 <tpressure> libvirt has a poor hypervisor abstraction. If we would go the libvirt route, we would effectively have to maintain two components instead of only one. You cannot simply map qemu calls to cloud hypervisor 16:46:05 <dansmith> sean-k-mooney: especially since it appearently already exist: https://libvirt.org/drvch.html 16:46:19 <sean-k-mooney> and beyond that how you plan to test and develop supproted for it on an ongoing basis 16:46:58 <sean-k-mooney> tpressure: well we dont generate qemu calls today 16:47:10 <sean-k-mooney> nova generate a semi hypervior indepent xml 16:47:31 <sean-k-mooney> we "supprot" libvirt with lxc, qemu and i think openvz 16:47:47 <sean-k-mooney> althoug everything excpt for qemu is basiclly not tested or maintaiend anymore 16:48:19 <sean-k-mooney> tpressure: hertrste based on https://libvirt.org/drvch.html provied by dan there si already some supprot for cloud-hypervior in libvirt 16:48:31 <hertrste> yeah we know of that support 16:48:36 <sean-k-mooney> so the question woudl be what does a full virt driver provide that cant be done vai that 16:49:04 <dansmith> and more than that, why would it be worth maintaining that *and* a separate full nova driver 16:49:29 <dansmith> like, what will never be possible via libvirt that would be if we have a whole separate implementation in nova 16:49:31 <hertrste> we decided not to use it because we felt the overhead of dealing with the existing 14k LoC for the libvirt backend in Nova was not worth. at least for a proof of concept 16:49:53 <dansmith> because libvirt brings all kinds of things like network and storage abstraction that would have to be written from scratch in a nova driver 16:50:03 <hertrste> Further, libvirt only has very basic support for cloud hypervisor. so we basically would need to add features in 2 complex code bases 16:50:31 <sean-k-mooney> the main concern i would have iwth the current poc 16:50:41 <sean-k-mooney> is its directly invoking a command line client correct 16:50:52 <dansmith> hertrste: that's only true if you're talking about a feature that isn't supported at all by libvirt, which is honestly going to be a hard sell in nova anyway 16:51:01 <sean-k-mooney> which means you are commiting to providign a forward compatiable stable cli fro nova to use 16:51:40 <bauzas> folks, because we have another item to discuss, can we stop here ? 16:51:46 <dansmith> sean-k-mooney: and hopefully there is some way to not kill those instances when nova restarts.. I hope it doesn't make nova responsible for being the init system for those binaries... 16:51:59 <sean-k-mooney> sure we can move on 16:52:05 <dansmith> bauzas: yes, I have to run anyway.. but a preview of my concerns for PTG :) 16:52:06 <sean-k-mooney> dansmith: ack 16:52:19 <hertrste> thanks for the feedback 16:52:30 <bauzas> hertrste: ping me if you have any questions about the PTG, I'll try to answer your concerns 16:52:40 <bauzas> moving on then 16:52:50 <bauzas> (mzhang741) Add Burst Length Support to Cinder QoS 16:52:58 <bauzas> is mzhang741 here ? 16:53:05 <MengyangZhang[m]> yes 16:53:08 <bauzas> https://review.opendev.org/c/openstack/nova-specs/+/932653 16:53:42 <bauzas> seems to me a design question 16:53:53 <bauzas> could we just do this in the gerrit change review ? 16:54:11 <bauzas> do we need a quorum consensus ? 16:55:58 <MengyangZhang[m]> besides that min version check question, also need review and approval of the spec to move forward 16:56:29 <bauzas> MengyangZhang[m]: that's our usual review cycle 16:56:44 <bauzas> but for the moment, even 2025.2 work hasn't started 16:56:59 <bauzas> so please understand that people have other priorities for now 16:57:34 <bauzas> MengyangZhang[m]: the best if you want is to have a cross-project discussion with the cinder folks at the PTG 16:57:46 <sean-k-mooney> bauzas: context for this 16:57:50 <bauzas> that would I guess answer most of the design questions, and it would be quick 16:58:01 <MengyangZhang[m]> understood, meanwhile can I have some clarity on that design question? I may shift my focus to other projects in my company and won't be able to follow this until q2 16:58:06 <sean-k-mooney> i was orginally asking that we have a min version ceck to know that the compute node can supprot the new frontend qos option 16:58:24 <sean-k-mooney> it turns out that cinder allows you to update the qos on a volume type 16:58:28 <bauzas> sean-k-mooney: that's the usual upgrade pattern yeah 16:58:37 <sean-k-mooney> and it will take effect to all volumes of that type 16:58:42 <sean-k-mooney> on the next move op 16:58:50 <sean-k-mooney> or when we next update the attachment 16:59:06 <bauzas> so even old computes would see it ? 16:59:07 <sean-k-mooney> so cinder is not caching it liek we do with the falvor/image properties when the volume is created 16:59:10 <sean-k-mooney> yep 16:59:21 <sean-k-mooney> so this is a pre exisitng upgrade problem 16:59:24 <bauzas> that's an harder upgrade pattern to solve 16:59:30 <bauzas> yeah, sounds 16:59:43 <sean-k-mooney> so i dont think we can actully really protect agaisnt this in nova 16:59:43 <bauzas> like, someone has to do a pre-upgrade flight 17:00:04 <bauzas> we could say 'unsupported until all your computes are upgraded' 17:00:06 <sean-k-mooney> anyway we shoudl dicuss more with the cinder folks 17:00:32 <sean-k-mooney> but we cant just enforce "not supported until all upgraded" in our code just in docs 17:00:33 <bauzas> is there any API impact? tbh I haven't reviewed the spec 17:00:39 <bauzas> hah 17:01:00 <bauzas> sounds to me then a perfect cinder-nova PTG discussion 17:01:13 <bauzas> MengyangZhang[m]: could you join the PTG ? 17:01:29 <bauzas> anyway, we're on time 17:01:35 <bauzas> thanks all 17:01:39 <bauzas> #endmeeting