16:00:20 <bauzas> #startmeeting nova 16:00:20 <opendevmeet> Meeting started Tue Sep 24 16:00:20 2024 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:20 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:20 <opendevmeet> The meeting name has been set to 'nova' 16:00:30 <bauzas> #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 16:00:33 <bauzas> hey folks 16:00:42 <elodilles> o/ 16:00:57 <tkajinam> o/ 16:01:00 <Uggla> o/ 16:01:10 <gibi> o/ 16:01:41 <bauzas> I guess we can softly start 16:01:54 <auniyal> o/ 16:01:56 <bauzas> #topic Bugs (stuck/critical) 16:02:01 <bauzas> #info No Critical bug 16:02:05 <bauzas> (huzzah) 16:02:11 <bauzas> #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster 16:02:21 <bauzas> any critical bug you'd like to raise ? 16:02:38 <bauzas> we're btw. on RC stage, which means that any regression is important 16:02:50 <bauzas> if you found one, please raise 16:03:59 <bauzas> apparently not 16:04:02 <bauzas> moving on then 16:04:10 <bauzas> #topic Gate status 16:04:17 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:04:22 <bauzas> #link https://etherpad.opendev.org/p/nova-ci-failures-minimal 16:04:29 <bauzas> #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status 16:04:41 <bauzas> the nova-emulation job failed on master 16:04:58 <bauzas> #link https://zuul.openstack.org/build/2f16f1f9ff5d4f0c9bae021f54cd5d27 16:05:14 <gibi> that is failing pretty frequently as far as I remember 16:05:31 <bauzas> the nova-emulation job failing ? yeah 16:05:40 <bauzas> but in general that's on the stable branches 16:05:54 <bauzas> this time, looks like the periodic run got the kernel panic problem 16:06:11 <bauzas> oh wait no 16:06:17 <bauzas> they never got the DHCP lease 16:06:59 <gibi> the emulaton job is shaky on master too https://zuul.openstack.org/builds?job_name=nova-emulation&project=openstack/nova 16:07:04 <bauzas> hopefully a race condition, but that kind of error was there a while ago 16:07:09 <bauzas> https://220ad47fc85766504430-165a16385c4a60267b84fdc053ee074d.ssl.cf5.rackcdn.com/periodic-weekly/opendev.org/openstack/nova/master/nova-emulation/2f16f1f/testr_results.html 16:07:19 <gibi> https://zuul.openstack.org/builds?job_name=nova-emulation&project=openstack%2Fnova&branch=master&skip=0 16:07:31 <bauzas> the ssh connection failed because the instance never got the DHCP lease in time 16:08:17 <bauzas> anyway, I propose we defer the discussion to next week if the periodic fails again on master 16:08:25 <gibi> anyhow I don't see a repeating pattern in the failed jobs 16:08:41 <gibi> so I guess it is general instability not a regression 16:08:44 <bauzas> yup, that's why I feel the job acting more like a canary 16:09:47 <bauzas> so, can we move on or is anyone wanting to say more ? 16:09:54 <opendevreview> Gorka Eguileor proposed openstack/nova master: Support os-brick specific lock_path https://review.opendev.org/c/openstack/nova/+/849328 16:10:40 <bauzas> looks so 16:10:47 <bauzas> #info Please look at the gate failures and file a bug report with the gate-failure tag. 16:10:52 <bauzas> #info Please try to provide meaningful comment when you recheck 16:10:58 <tkajinam> in case you haven't seen it... osc was bumped again and now 7.1.2 is supposed to be used in gate 16:10:59 <tkajinam> https://review.opendev.org/c/openstack/requirements/+/929938 16:11:11 <bauzas> yup, I saw the lift 16:11:20 <tkajinam> in case you see that causes any problems then report it asap 16:11:38 <bauzas> ack 16:11:39 <tkajinam> I'm not aware of any, so far 16:12:02 <tkajinam> that's all from me :-) 16:12:04 <bauzas> ditto here, I haven't checked whether we merged stuff, but the CI jobs sound correct 16:12:21 <bauzas> thanks tkajinam for the note, btw. 16:12:27 <bauzas> #topic Release Planning 16:12:34 <bauzas> #link https://releases.openstack.org/dalmatian/schedule.html 16:12:41 <bauzas> #info Dalmatian RC1 was tagged on Monday 16:12:47 <bauzas> (huzzah again) 16:13:07 <bauzas> the fun part is that the deadline for a new RC is on this Thursday :) 16:13:13 <bauzas> so we have two days 16:13:42 <bauzas> but that means, we can pivot ! 16:13:57 <bauzas> (a 'friends' reference if you may not got it) 16:14:02 <bauzas> #link https://releases.openstack.org/epoxy/schedule.html 16:14:11 <gibi> do you track anything that is needed before RC2? 16:14:14 <bauzas> our master branch is now Epoxy 16:14:29 <bauzas> gibi: I haven't got any occurrence yet 16:14:35 <bauzas> but we still have the tracking etherpad 16:14:53 <bauzas> https://etherpad.opendev.org/p/nova-dalmatian-rc-potential 16:15:17 <bauzas> the only follow-up item is https://review.opendev.org/c/openstack/nova/+/928462 but will be merged on master 16:15:47 <gibi> yeah the oslo dep is not merged yet 16:15:55 <bauzas> so, now our master branch is Epoxy, spec up, folks ! 16:16:50 * tkajinam is merging the pending oslo.utils patches 16:16:50 <bauzas> #topic Review priorities 16:17:00 <bauzas> #link https://etherpad.opendev.org/p/nova-2025.1-status 16:17:09 <bauzas> please add your review requests in that doc 16:17:22 <bauzas> I still have the AI to update our docs to mention it 16:17:31 <bauzas> (and I don't forget it) 16:17:38 <auniyal> I added 2 bugs-patches here in doc 16:17:43 <bauzas> cool 16:17:58 <bauzas> I'll do a bit of grooming and check the states 16:18:08 <bauzas> and add some notes for the other cores 16:18:16 <bauzas> #topic PTG planning 16:18:22 <bauzas> #info as a reminder, we'll meet (virtually) at the PTG on Oct 21-25 2024 16:18:29 <bauzas> #info Please register yourself to the virtual PTG 16:18:38 <bauzas> #link https://etherpad.opendev.org/p/nova-2025.1-ptg 16:18:51 <bauzas> as you may have seen, I've already booked the rooms 16:19:13 <gibi> who will provide the virtual beers? ;) 16:19:34 <bauzas> this will be as usual 4 days of 4 hours, 1300-1700UTC 16:20:11 <bauzas> gibi: I can refill my very physical ones, but that's it unless folks travel to Grenoble 16:20:29 * bauzas misses the physical PTGs 16:20:58 <bauzas> we may have one extra session on an asian-friendly timeslot 16:21:16 <bauzas> based on my trip to Suwon, KR, this may not be useless 16:21:34 <bauzas> and we may have an item at least : translating our docs 16:21:51 <auniyal> ++ from Asia 16:21:57 <bauzas> but I need to discuss that with ianwchoi and sungsoo 16:22:15 <gibi> I can take an early EU slot if that helps meeting the folks 16:22:26 <bauzas> I'll first ensure we have audience :-) 16:22:31 <gibi> :) 16:22:38 <tkajinam> yeah it may be important to reach out to some folks who may be interested to get their attendance 16:23:04 <tkajinam> I myself don't care about much about late meetings as I can settle the other things during my day but it may not be a common thing :-P 16:23:06 <bauzas> that's why I think the agenda is more important than the slots 16:23:11 <tkajinam> yup 16:23:52 <bauzas> I'll probably first reach out to ian and sungsoo and see whether they're available 16:24:28 <bauzas> tkajinam: I honestly share your pain about late meetings :-) 16:24:58 <bauzas> that's why it's important to ensure that those meetings are productive 16:25:21 <bauzas> (corrolar with very early meetings, per say) 16:25:37 <bauzas> anyway, moving on 16:25:53 <bauzas> #topic Stable Branches 16:25:58 <bauzas> elodilles: I hear ya 16:26:08 <elodilles> o/ 16:26:11 <elodilles> to be quick: 16:26:15 <elodilles> #info stable/202*.* gates seem to be OK 16:26:25 <elodilles> #info stable/2024.2 branches are present on all nova repos 16:26:30 <bauzas> I pretty much like the apparences 16:26:32 <elodilles> and that's all from me 16:26:44 <bauzas> elodilles: bug me with review requests 16:26:57 <elodilles> bauzas: ACK, will do 16:27:02 <bauzas> tbh, I'm currently diverted from upstream, so I need someone to harass me 16:27:40 <bauzas> particularly with release requests (although I see them in my mailbox) 16:28:29 <bauzas> #topic vmwareapi 3rd-party CI efforts Highlights 16:28:35 <bauzas> fwiesel: anything to mention ? 16:28:56 <fwiesel> Not much: https://review.opendev.org/c/openstack/nova/+/910627?tab=comments is green, waiting for a reply from dansmith 16:29:23 <fwiesel> The CI is not 100% reliable, sometimes there are random failures. I'll have to find out why. 16:29:34 <bauzas> good luck with that 16:29:42 <fwiesel> That's from my side. 16:30:27 <bauzas> investigating a CI failure is sometimes harder than playing piano on a rope above one thousand feet 16:30:42 <bauzas> fwiesel: thanks 16:31:04 <bauzas> #topic Open discussion 16:31:12 <bauzas> (gibi): Seeking approval for: Support hw_vif_model igb 16:31:18 <bauzas> gibi: your turn 16:31:19 <gibi> o/ 16:31:43 <gibi> so new enough libvirt and qemu supports to emulate a new nic type igb (old intel nic) 16:31:57 <gibi> the big benefit of it is for testing 16:32:08 <bauzas> yuuup 16:32:10 <gibi> as igb support SRIOV with 7 VFs per PF 16:32:35 <gibi> nova support hw_vif_model image property to ask for a nic type 16:32:44 <gibi> I'd like to extend the supported values there with ig 16:32:45 <gibi> ig 16:32:47 <gibi> igb 16:32:59 <gibi> https://blueprints.launchpad.net/nova/+spec/igb-vif-model 16:33:10 <bauzas> two questions 16:33:11 <gibi> https://review.opendev.org/q/topic:%22bp/igb-vif-model%22 16:33:19 <bauzas> 1/ does it require a new os-vif release ? 16:33:22 <gibi> no 16:33:44 <bauzas> 2/ I guess you have a plan for rolling-upgrades ? 16:34:28 <gibi> yes. there is a new trait for the new igb vif model, libvirt driver will expose it only if both libvirt and qemu is new enough to support it 16:34:46 <gibi> and the image properies pre-filter adds the trait to the placement query 16:34:54 <bauzas> and then we'll schedule the instance with pre-filtering with that trait 16:35:02 <gibi> yepp 16:35:06 <bauzas> okay, sounds the right pattern to me 16:35:15 <tkajinam> +1 16:35:22 <gibi> the code is up with documentation included 16:35:35 <bauzas> I don't see any other design concern 16:35:37 <gibi> it impacts os-traits and nova, + glance doc 16:35:46 <bauzas> yup, saw the series 16:36:11 <gibi> it is unfortunately non backportable due to ovo change 16:36:17 <bauzas> I honestly like the approach of a specless blueprint if we can avoid a spec just for the trait thing 16:36:39 <bauzas> there is no upgrade impact, since all the support will be made additive 16:37:02 <gibi> yepp 16:37:05 <bauzas> and there are no API changes, despite an image property additon 16:37:14 <gibi> yepp 16:37:42 <gibi> frickler was super nice and proposed cirros update to have the igb kernel module in cirros available 16:37:56 <bauzas> pardon my ignorance but do we have flavor extraspecs with hw:vif_model ? 16:38:13 <gibi> there is no flavor extra spec for that 16:38:20 <bauzas> perfect 16:38:43 <gibi> so I will be able to add a tempest test that actully proves that VFs can be created in the guest 16:38:57 <bauzas> okay, I don't see any reason to not accept that blueprint as specless, but is anyone having concerns with the design ? 16:39:50 <tkajinam> no 16:39:57 <bauzas> let's be it. 16:40:05 <gibi> \o/ 16:40:13 <bauzas> #agreed https://blueprints.launchpad.net/nova/+spec/igb-vif-model approved as specless 16:40:22 <bauzas> I'll do the paperwork right after the meeting 16:40:26 <gibi> I will ask for review on the os-traits patch first as I need a release there to be able to bump it in nova 16:40:31 <bauzas> ++ 16:40:39 <tkajinam> the feature sounds quite cool. 16:40:41 <gibi> https://review.opendev.org/c/openstack/os-traits/+/928582 16:40:46 <tkajinam> I'm wondering if any of the node provider in CI infra and provide nodes with the flavor so that we can test sriov features in gate :-) 16:40:52 <bauzas> I honestly prefer we keep such features as specless as possible 16:40:59 <gibi> tkajinam: that is the long term goal 16:41:20 <gibi> tkajinam: if the node providers have this patch we can freely ask for SRIOV without using hardware 16:41:35 <bauzas> gibi: I assume noble is recent enough to ship the right libvirt versions? 16:41:52 <gibi> throughput is bad as it is emulated, but we can cover our SRIOV logic 16:41:55 <clarkb> johnsom has been discussing this with us and the main issue is the lack of support from nova aiui 16:42:10 <bauzas> clarkb: this will be a solved problem soon 16:42:17 <gibi> bauzas: I need to double check as I use debian sid 16:42:22 <clarkb> bauzas: not really because then we have to wait a few years for the clouds to upgrade 16:42:22 <tkajinam> I assume that is for sriov amphora. 16:42:22 <bauzas> cool 16:42:32 <gibi> tkajinam: yes 16:42:43 <clarkb> but yes the first step is fixing nova then seeing what we can do about upgrading nova (but that is unlikely to happen quickly) 16:42:53 <gibi> clarkb: understood. 16:43:03 * bauzas would love to see the same kind of emulated SRIOV device with a vfio-pci variant driver :) 16:43:15 <bauzas> clarkb: noted 16:43:24 <bauzas> and thanks, didn't thought about it 16:43:27 <gibi> btw without the nova fix one could use igb in local devstack to test SRIOV 16:43:35 <tkajinam> libvirt 10.0.0 and qemu 8.2.2 16:43:44 <tkajinam> these are what package search for noble tells me 16:43:45 <bauzas> sounds noble 16:43:55 <bauzas> iirc 16:43:58 <gibi> tkajinam: good we need 9.3 and 8.0 16:44:04 <clarkb> when discussing it with johnsom I think we decided that centos 9 stream is also new enough 16:44:11 <clarkb> for libvirt and qemu 16:44:16 <bauzas> yeah 16:44:31 <bauzas> c9s ships libvirt-10 IIRC 16:44:32 <tkajinam> yeah centos has been updating libvirt/qemu quite regularly 16:44:52 <bauzas> okay, I think we're done 16:45:10 <bauzas> thanks all, let me give you 15 mins back of your time 16:45:14 <tkajinam> can I ask one quick question about a different topic ? 16:45:19 <bauzas> shoot then, 16:45:20 <johnsom> Glad to hear there is progress on this. I opened a bug for the nova/igb issue in launchpad a while ago 16:45:38 <bauzas> I can drink my beer later 16:45:49 <tkajinam> I've re-proposed spec for SEV-ES support to E. The patches are up but haven't get enough review really https://review.opendev.org/c/openstack/nova-specs/+/928817 16:46:02 <gibi> johnsom: I picket it up couple of weeks ago as I also need this for testing 16:46:10 <johnsom> +1 16:46:11 <bauzas> tkajinam: I'll add it to the status etherpad 16:46:19 <johnsom> https://bugs.launchpad.net/nova/+bug/2077195 16:46:32 <tkajinam> I wonder if it can be approved quickly or do I need any steps before that ? I can bring it to vPTG but I'm not aware of actual topics we can/should discuss 16:46:36 <bauzas> tkajinam: we still need to agree on some spec review days at the PTG 16:46:53 <bauzas> it was approved in D, right ? 16:47:03 <tkajinam> yes it was. 16:47:30 <bauzas> okay, there is a specific fast-approval process for pre-approved specs that are just straighly forwarded 16:47:39 <bauzas> I can surely help then that situation 16:48:07 <bauzas> particularly given you have patches up 16:48:20 <tkajinam> thanks. because this may still need some rounds of reviews if it can be approved early then that'd be helpful 16:48:41 <tkajinam> (so that it can be in the review list 16:48:45 <bauzas> if the spec is just reproposed without changes, this will be quick 16:49:27 <tkajinam> yeah. the reporposed spec is exactly same ... but I noticed I have to update the history section maybe. 16:49:38 <bauzas> oh you'll nee 16:49:40 <bauzas> need 16:49:42 <tkajinam> I'll fix it soon and then ping you. 16:50:24 <bauzas> I left a comment 16:50:35 <bauzas> tkajinam: sure, don't hesitate 16:50:47 <tkajinam> bauzas, thanks ! 16:50:53 <bauzas> as I already said, I'm a bit on and off those days, so please help me doing my work by pinging me 16:51:12 <tkajinam> ack. sure 16:51:40 <bauzas> if that's all, then let's close the meetign 16:51:44 <bauzas> anything else ? 16:51:45 <tkajinam> that's all from me 16:51:52 <bauzas> going once ? 16:51:56 <bauzas> going twice ? 16:52:03 <bauzas> okay, thanks all 16:52:06 <bauzas> #endmeeting