16:00:40 <bauzas> #startmeeting nova
16:00:40 <opendevmeet> Meeting started Tue Feb 25 16:00:40 2025 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:40 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:40 <opendevmeet> The meeting name has been set to 'nova'
16:01:40 <gibi> sean-k-mooney: https://paste.opendev.org/show/bEWhDvmH5NPNozp8V4pU/ confirmed the pci_stats also cleaned
16:02:12 <sean-k-mooney> gibi: cool i expected it to be. its just it should not be there at all. but we can adress that in a trivial followup
16:02:20 <sean-k-mooney> or dicsuss it more after the meeting
16:02:36 <bauzas> #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
16:02:42 <elodilles> o/
16:02:44 <bauzas> howdy folks, sorry was late
16:02:54 * bauzas is trampled by many things at same time
16:03:02 <sean-k-mooney> o/
16:03:11 <bauzas> let's start
16:03:23 <bauzas> #topic Bugs (stuck/critical)
16:03:24 <masahito> o/
16:03:32 <bauzas> #info No Critical bug
16:03:39 <bauzas> #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster
16:04:09 <bauzas> any bugs to talk ?
16:04:56 <bauzas> looks not
16:05:02 <bauzas> #topic Gate status
16:05:12 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:05:17 <bauzas> #link https://etherpad.opendev.org/p/nova-ci-failures-minimal
16:05:27 <bauzas> (I'll leave the next PTL to curate that long list)
16:05:34 <bauzas> #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status
16:05:40 <bauzas> #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:05:46 <bauzas> #info Please try to provide meaningful comment when you recheck
16:05:56 <bauzas> for me, the gate is correct
16:06:30 <bauzas> yet a few rechecks because of the usual suspects (guest panic and ssh timeouts) but nothing new and putting epoxy-3 at risk
16:06:39 <sean-k-mooney> bauzas: there is perhaps one
16:06:49 <Uggla> o/
16:06:58 <sean-k-mooney> one of the funtional test for vgpu multi type  is failing randomly
16:07:08 <bauzas> hmmm
16:07:15 <sean-k-mooney> bauzas: i dont have the link but i pingged it in the channel yesterday
16:07:16 <bauzas> that may be related to the leaky greenlet
16:07:26 <sean-k-mooney> it triggering the greenlet leak detachtion
16:07:32 <sean-k-mooney> right it is
16:07:37 <bauzas> yeah, but that's not new
16:07:40 <sean-k-mooney> so that a bug in the test
16:07:40 <fwiesel> o/
16:07:53 <sean-k-mooney> well that somethign that shoudl be fixed as its causing extra rechecks
16:07:59 <bauzas> so maybe some side effect happened that would increase the failure rate
16:08:03 <sean-k-mooney> its not critical but we should not ignore it
16:08:08 <bauzas> I don't disagree
16:08:18 <sean-k-mooney> bauzas: if there is an excption in the leaked greenthread
16:08:27 <sean-k-mooney> it cn cause other test to fail becuase of that
16:08:45 <bauzas> I can try to take a look on it, particularly given I have another bugfix patch I'd want to be merged before RC1
16:08:47 <sean-k-mooney> so it cause hard to diagnose inter test interaction which is why we added the leak detection
16:09:00 <sean-k-mooney> ack its more an fyi
16:09:05 <bauzas> cool
16:09:20 <bauzas> #topic Release Planning
16:09:32 <bauzas> #link https://releases.openstack.org/epoxy/schedule.html
16:09:56 <bauzas> #info Nova deadlines are set in the above schedule
16:10:07 <bauzas> #info 2 days before Feature Freeze
16:10:07 <elodilles> note that there are client library release patches open:
16:10:15 <elodilles> https://review.opendev.org/c/openstack/releases/+/942534
16:10:21 <elodilles> https://review.opendev.org/c/openstack/releases/+/942551
16:10:39 <elodilles> (osc-placement and python-novaclient)
16:11:05 <elodilles> just an fyi as we have Epoxy-3 now o:)
16:11:06 <sean-k-mooney> so those have the same freeze deadline as nova right
16:11:08 <bauzas> noted, I'll review them
16:11:10 <sean-k-mooney> i.e. thrusday
16:11:18 <elodilles> yepp
16:11:30 <sean-k-mooney> but we will want to do a release at m3 to test with them for RC1
16:11:41 <elodilles> +1
16:12:11 <sean-k-mooney> we shoudl not haver anything pending for osc-palcement or novaclient that im aware of
16:12:18 <sean-k-mooney> so we can probly do those effectivly now
16:12:50 <sean-k-mooney> on clients
16:13:08 <sean-k-mooney> we have some new api versions this cycle and im not sure we will have osc suprot for those in time
16:13:20 <sean-k-mooney> so they may get enabled next cycle
16:13:54 <sean-k-mooney> but there isnt really anything we can do there at this point if the sdk supprot is not already merged
16:14:59 <sean-k-mooney> elodilles: am i correct in saying the sdk is subject to the non clinet code freeze so we should not expect to merge supprot for spice direct or image property shod api microversion between now and FF
16:15:32 <elodilles> sean-k-mooney: if i'm not mistaken sdk already merged it's final release patch recently
16:15:48 <sean-k-mooney> ack that what i was expecting
16:15:59 <elodilles> well,
16:16:05 <elodilles> it's still open,
16:16:32 <elodilles> waiting for a 2nd +2: https://review.opendev.org/c/openstack/releases/+/941859
16:16:38 <elodilles> from release team
16:17:06 <sean-k-mooney> i think we are missing a finaly release for os-traits and perhaps os-resouce classes too by the way
16:17:45 <elodilles> those are 'independent' projects, hence not picked up by release tools
16:17:46 <sean-k-mooney> we have not merged any content since the last release we did for either
16:17:53 <sean-k-mooney> ah ok
16:17:57 <elodilles> but wasn't there os-traits release recently?
16:18:30 <sean-k-mooney> yes but we had traits we wanted to add and have anouther release 3 days after that happend:)
16:18:49 <sean-k-mooney> the os-traits patch is still pending a second +2
16:18:56 <elodilles> sean-k-mooney: this is the content of SDK release (if it doesn't change anymore): https://zuul.opendev.org/t/openstack/build/24cc79c0e8e84d7d853e679e4e928585/log/tox/list-changes/list-changes-results.log#918-947
16:19:13 <opendevreview> Jay Faulkner proposed openstack/nova master: PoC: Allow unversioned-notify specific transport url  https://review.opendev.org/c/openstack/nova/+/942710
16:19:26 <sean-k-mooney> ok we can likely move on and revisit in open dicuss if there is more to bring up
16:19:49 <elodilles> +1
16:21:19 <bauzas> can we move on ?
16:21:41 <elodilles> yepp
16:23:02 <bauzas> cool
16:23:13 <bauzas> #topic Review priorities
16:23:20 <bauzas> #link https://etherpad.opendev.org/p/nova-2025.1-status
16:25:30 <bauzas> nothing to tell about, I'm doing hard reviews
16:25:39 <bauzas> anything else ?
16:26:08 <bauzas> looks not
16:26:15 <opendevreview> Doug Goldstein proposed openstack/nova master: ironic: fix logging of validation errors  https://review.opendev.org/c/openstack/nova/+/942019
16:26:17 <bauzas> #topic PTG planning
16:26:23 <bauzas> #info Next PTG will be held on Apr 7-11
16:26:38 <bauzas> usual reminder, please fill the agenda !
16:26:42 <bauzas> #link https://etherpad.opendev.org/p/nova-2025.2-ptg
16:27:59 <bauzas> #topic Stable Branches
16:28:04 <bauzas> elodilles: your turn
16:28:17 <elodilles> ACK
16:28:23 <elodilles> actually, same as last week:
16:28:31 <elodilles> #info stable gates seem to be healthy
16:28:39 <elodilles> #info stable/2023.2 release patch: https://review.opendev.org/c/openstack/releases/+/941420 (stable/2023.2 (bobcat) is going to transition to EOL in ~2 months)
16:28:47 <elodilles> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci
16:28:57 <elodilles> that's all
16:31:22 <bauzas> cool
16:31:34 <bauzas> #topic vmwareapi 3rd-party CI efforts Highlights
16:31:37 <bauzas> fwiesel: around ?
16:31:48 <fwiesel> No updates from my side. I am a bit under the weather.
16:33:08 <bauzas> ditto here
16:33:10 <bauzas> I understand you
16:35:38 <bauzas> #topic Open discussion
16:35:45 <bauzas> I have one item
16:36:01 <bauzas> kudos to Uggla for being our next PTL \o/
16:36:10 <Uggla> thx
16:36:25 <gibi> congrats
16:36:40 <gibi> and thanks bauzas for the long service as PTL
16:36:45 <fwiesel> Congrats
16:36:53 <elodilles> yepp ++ \o/
16:37:27 * Uggla hoping bauzas will still be around to help me jumping in that role. ;)
16:38:26 <bauzas> I will !
16:38:29 <masahito> congrats and thank you
16:38:39 <bauzas> anyway, moving to the questions
16:39:09 <bauzas> (hertrste) Nova Cloud Hypervisor support
16:39:23 <hertrste> Hi o/
16:39:55 <hertrste> We would like to add support for the Cloud Hypervisor VMM to openstack Nova
16:40:12 <hertrste> As we have not contributed anything yet, we would like some guidance how to approach that
16:40:29 <sean-k-mooney> this yes https://github.com/cloud-hypervisor/cloud-hypervisor
16:40:31 <hertrste> As you can see in the meeting notes, we have a proof of concept
16:40:40 <hertrste> exactly
16:41:08 <sean-k-mooney> i breifly looked at that a few months ago out of interest when i was playing with rust
16:41:13 <hertrste> We also have a proof of concept adding some basic support for that VMM: https://github.com/openstack/nova/commit/617ac2f655492d1b84b2d924412ac7063d231110
16:41:14 <sean-k-mooney> it has an interesting feature set
16:41:29 <gibi> I think the next step there is to have PTG topic proposed for this
16:42:12 <sean-k-mooney> hertrste  our main concerns would be likely testing, and building out a minimal supportrable feature set
16:42:36 <bauzas> yeah, looks to me a good PTG session
16:42:39 <sean-k-mooney> that would invovle defining a spec (if the propsal to add a new driver was green lit)
16:42:42 <bauzas> hertrste: can you attend it ?
16:42:44 <dansmith> I'd also be interested in knowing how widely it is used/deployed
16:42:44 <sean-k-mooney> to descibe the expecations
16:43:05 <dansmith> and also, why it's not just a libvirt driver :)
16:43:50 <sean-k-mooney> well i dont neesislay think we shoudl say all linux hyperviors shoudl be behind libvirt
16:43:55 <hertrste> I can very much attend a PTG. What would I need to prepare for that? When would that meeting take place?
16:43:57 <sean-k-mooney> it has its pros and cons
16:44:43 <dansmith> sean-k-mooney: no, doesn't have to be, but there's a lot of stuff to re-implement in nova if it's not, and so I'd expect there to be a good reason for it
16:44:50 <gibi> hertrste: https://openinfra.org/ptg/
16:44:54 <bauzas> hertrste: see the above discussion about PTG
16:45:32 <Uggla> hertrste,  Apr 7-11
16:45:38 <sean-k-mooney> hertrste: i think the main things to prepare is an elevator pitch explaining why an operator may want ot use this
16:45:53 <sean-k-mooney> wwhat usecases and target workloads woudl it enable
16:46:02 <tpressure> libvirt has a poor hypervisor abstraction. If we would go the libvirt route, we would effectively have to maintain two components instead of only one. You cannot simply map qemu calls to cloud hypervisor
16:46:05 <dansmith> sean-k-mooney: especially since it appearently already exist: https://libvirt.org/drvch.html
16:46:19 <sean-k-mooney> and beyond that how you plan to test and develop supproted for it on an ongoing basis
16:46:58 <sean-k-mooney> tpressure: well we dont generate qemu calls today
16:47:10 <sean-k-mooney> nova generate a semi hypervior indepent xml
16:47:31 <sean-k-mooney> we "supprot" libvirt with lxc, qemu and i think openvz
16:47:47 <sean-k-mooney> althoug everything excpt for qemu is basiclly not tested or maintaiend anymore
16:48:19 <sean-k-mooney> tpressure: hertrste  based on https://libvirt.org/drvch.html provied by dan there si already some supprot for cloud-hypervior in libvirt
16:48:31 <hertrste> yeah we know of that support
16:48:36 <sean-k-mooney> so the question woudl be what does a full virt driver provide that cant be done vai that
16:49:04 <dansmith> and more than that, why would it be worth maintaining that *and* a separate full nova driver
16:49:29 <dansmith> like, what will never be possible via libvirt that would be if we have a whole separate implementation in nova
16:49:31 <hertrste> we decided not to use it because we felt the overhead of dealing with the existing 14k LoC for the libvirt backend in Nova was not worth. at least for a proof of concept
16:49:53 <dansmith> because libvirt brings all kinds of things like network and storage abstraction that would have to be written from scratch in a nova driver
16:50:03 <hertrste> Further, libvirt only has very basic support for cloud hypervisor. so we basically would need to add features in 2 complex code bases
16:50:31 <sean-k-mooney> the main concern i would have iwth the current poc
16:50:41 <sean-k-mooney> is its directly invoking a command line client correct
16:50:52 <dansmith> hertrste: that's only true if you're talking about a feature that isn't supported at all by libvirt, which is honestly going to be a hard sell in nova anyway
16:51:01 <sean-k-mooney> which means you are commiting to providign a forward compatiable stable cli fro nova to use
16:51:40 <bauzas> folks, because we have another item to discuss, can we stop here ?
16:51:46 <dansmith> sean-k-mooney: and hopefully there is some way to not kill those instances when nova restarts.. I hope it doesn't make nova responsible for being the init system for those binaries...
16:51:59 <sean-k-mooney> sure we can move on
16:52:05 <dansmith> bauzas: yes, I have to run anyway.. but a preview of my concerns for PTG :)
16:52:06 <sean-k-mooney> dansmith: ack
16:52:19 <hertrste> thanks for the feedback
16:52:30 <bauzas> hertrste: ping me if you have any questions about the PTG, I'll try to answer your concerns
16:52:40 <bauzas> moving on then
16:52:50 <bauzas> (mzhang741) Add Burst Length Support to Cinder QoS
16:52:58 <bauzas> is mzhang741 here ?
16:53:05 <MengyangZhang[m]> yes
16:53:08 <bauzas> https://review.opendev.org/c/openstack/nova-specs/+/932653
16:53:42 <bauzas> seems to me a design question
16:53:53 <bauzas> could we just do this in the gerrit change review ?
16:54:11 <bauzas> do we need a quorum consensus ?
16:55:58 <MengyangZhang[m]> besides that min version check question, also need review and approval of the spec to move forward
16:56:29 <bauzas> MengyangZhang[m]: that's our usual review cycle
16:56:44 <bauzas> but for the moment, even 2025.2 work hasn't started
16:56:59 <bauzas> so please understand that people have other priorities for now
16:57:34 <bauzas> MengyangZhang[m]: the best if you want is to have a cross-project discussion with the cinder folks at the PTG
16:57:46 <sean-k-mooney> bauzas: context for this
16:57:50 <bauzas> that would I guess answer most of the design questions, and it would be quick
16:58:01 <MengyangZhang[m]> understood, meanwhile can I have some clarity on that design question? I may shift my focus to other projects in my company and won't be able to follow this until q2
16:58:06 <sean-k-mooney> i was orginally asking that we have  a min version ceck to know that the compute node can supprot the new frontend qos option
16:58:24 <sean-k-mooney> it turns out that cinder allows you to update the qos on a volume type
16:58:28 <bauzas> sean-k-mooney: that's the usual upgrade pattern yeah
16:58:37 <sean-k-mooney> and it will take effect to  all volumes of that type
16:58:42 <sean-k-mooney> on the next move op
16:58:50 <sean-k-mooney> or when we next update the attachment
16:59:06 <bauzas> so even old computes would see it ?
16:59:07 <sean-k-mooney> so cinder is not caching it liek we do with the falvor/image properties when the volume is created
16:59:10 <sean-k-mooney> yep
16:59:21 <sean-k-mooney> so this is a pre exisitng upgrade problem
16:59:24 <bauzas> that's an harder upgrade pattern to solve
16:59:30 <bauzas> yeah, sounds
16:59:43 <sean-k-mooney> so i dont think we can actully really protect agaisnt this in nova
16:59:43 <bauzas> like, someone has to do a pre-upgrade flight
17:00:04 <bauzas> we could say 'unsupported until all your computes are upgraded'
17:00:06 <sean-k-mooney> anyway we shoudl dicuss more with the cinder folks
17:00:32 <sean-k-mooney> but we cant just enforce "not supported until all upgraded" in our code just in docs
17:00:33 <bauzas> is there any API impact? tbh I haven't reviewed the spec
17:00:39 <bauzas> hah
17:01:00 <bauzas> sounds to me then a perfect cinder-nova PTG discussion
17:01:13 <bauzas> MengyangZhang[m]: could you join the PTG ?
17:01:29 <bauzas> anyway, we're on time
17:01:35 <bauzas> thanks all
17:01:39 <bauzas> #endmeeting