16:01:51 <bauzas> #startmeeting nova 16:01:51 <opendevmeet> Meeting started Tue Mar 18 16:01:51 2025 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:01:51 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:01:51 <opendevmeet> The meeting name has been set to 'nova' 16:01:54 <bauzas> haha 16:02:03 * bauzas was wondering why it wasn't running 16:02:23 <gibi> o/ 16:02:30 <bauzas> #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 16:02:48 * bauzas will add our next PTL for this meeting as a chair :) 16:02:52 <bauzas> #chair Uggla 16:02:52 <opendevmeet> Current chairs: Uggla bauzas 16:02:56 <elodilles> o/ 16:03:00 <Uggla> o/ 16:04:09 <bauzas> awaiting a few before starting 16:05:29 <bauzas> #topic Bugs (stuck/critical) 16:05:38 <bauzas> #info No Critical bug 16:05:43 <bauzas> #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster 16:05:47 <bauzas> anything about bugs ? 16:06:25 <masahito> hello o/ 16:07:11 <bauzas> #topic Gate status 16:07:16 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:07:22 <bauzas> #link https://etherpad.opendev.org/p/nova-ci-failures-minimal 16:07:29 <bauzas> #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&branch=stable%2F*&branch=master&pipeline=periodic-weekly&skip=0 Nova&Placement periodic jobs status 16:07:35 <bauzas> #info Please look at the gate failures and file a bug report with the gate-failure tag. 16:07:39 <bauzas> #info Please try to provide meaningful comment when you recheck 16:08:44 <bauzas> anything about the gate ? 16:10:10 <bauzas> if not, moving on 16:10:20 <bauzas> #topic Release Planning 16:10:25 <bauzas> #link https://releases.openstack.org/epoxy/schedule.html 16:10:31 <bauzas> #link https://etherpad.opendev.org/p/nova-epoxy-rc-potential 16:10:41 <bauzas> as you can see, all of the open changes are now merged so... 16:10:45 <bauzas> #info RC1 tag https://review.opendev.org/c/openstack/releases/+/943941 16:10:52 <bauzas> should be merged today 16:11:48 <bauzas> which means that as soon as RC1 is tagged, epoxy branch is created and master becomes Flamingo :) 16:11:51 <elodilles> it's already on the gate ;) 16:12:09 <Uggla> \o/ 16:12:15 <bauzas> which means we're already mostly in the Flamingo cycle now 16:12:20 <bauzas> #link https://releases.openstack.org/flamingo/schedule.html 16:12:25 <bauzas> #info Nova Flamingo deadlines will be discussed at the PTG. 16:14:00 <bauzas> we don't know yet the next deadlines for feature freeze and spec approvals, those will be discussed there 16:14:22 <bauzas> now, I'm passing the chair baton to Uggla :) 16:14:37 <Uggla> bauzas, thx 16:14:50 <Uggla> #topic Review priorities 16:15:07 <Uggla> #info Flamingo priorities will be discussed at the PTG. 16:15:31 <Uggla> #topic PTG planning 16:15:45 <Uggla> #info Next PTG will be held on Apr 7-11 16:15:58 <Uggla> #link https://etherpad.opendev.org/p/nova-2025.2-ptg 16:17:17 <Uggla> Thanks to fill the doc as soon as possible it will help to organize the PTG. 16:18:08 <Uggla> #topic Stable Branches 16:18:17 <Uggla> #info stable gates seem to be healthy 16:18:29 <Uggla> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci 16:18:47 <Uggla> @elodilles, if you want to say something the floor is yours. 16:18:47 <bauzas> Uggla: you usually can let elodilles speak :) 16:19:00 <elodilles> thanks bauzas Uggla :) 16:19:07 <elodilles> nothing to say ;) 16:19:20 <elodilles> stable gates were quiet this week :) 16:19:35 <elodilles> i guess due to RC1 rush 16:19:43 <sean-k-mooney> we probaly will want to do some stable release in the next week or two 16:19:58 <sean-k-mooney> but obvioulsy finishing 2025.1 is the higher priority 16:20:04 <elodilles> sean-k-mooney: sure, we can do that 16:20:15 <elodilles> indeed, 2025.1 is the top prio right now 16:20:21 <elodilles> and for the coming 2 weeks ;) 16:20:26 <sean-k-mooney> once that done its good to look at the stable branch and see if any have patches that should be put in a release 16:20:49 <elodilles> especially as stable/2023.2 (bobcat) will move to End of Life 16:20:56 <sean-k-mooney> yep 16:21:03 <sean-k-mooney> do we have a deadlien for that? 16:21:06 <elodilles> and will be deleted ~1 month after 2025.1 Epoxy release 16:21:16 <elodilles> sean-k-mooney: ^^^ that 16:21:19 <sean-k-mooney> :) 16:21:41 <elodilles> and we should not leave the release at the deadline 16:22:00 <elodilles> in case we deliver some regression we cannot fix that anymore 16:22:04 <sean-k-mooney> right we should proably try to do a stable review spritn in like 3 weeks 16:22:16 <elodilles> which is unlikely, but let's not leave a chance for that ;) 16:22:30 <elodilles> sean-k-mooney: +1 16:23:02 <elodilles> so yeah, that's all about stable i think 16:23:49 <Uggla> thx @elodilles, so moving on. 16:23:49 * gibi has a lot of pending backports :) 16:24:12 <elodilles> and some of them are waiting for a 2nd +2 if i'm not mistaken o:) 16:25:01 <gibi> :) 16:25:29 <Uggla> probably some FUP from my side too... 16:26:08 <Uggla> ok next topic 16:26:10 <Uggla> #topic vmwareapi 3rd-party CI efforts Highlights 16:26:20 <Uggla> #info Investigating another regression on the network side 16:26:39 <fwiesel> Hi, no update from my side. I am currently on sick leave, so no probably no updates next week either. 16:26:45 <Uggla> fwiesel, if you want to say something. that's your turn. 16:26:59 <sean-k-mooney> fwiesel: no worreis, you proably shoudl go rest so instead of being on irc 16:27:10 <sean-k-mooney> fwiesel: dont feel pressuered to be here if your unwell 16:27:27 <opendevreview> OpenStack Release Bot proposed openstack/nova stable/2025.1: Update .gitreview for stable/2025.1 https://review.opendev.org/c/openstack/nova/+/944888 16:27:31 <opendevreview> OpenStack Release Bot proposed openstack/nova stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1 https://review.opendev.org/c/openstack/nova/+/944889 16:27:35 <opendevreview> OpenStack Release Bot proposed openstack/nova master: Update master for stable/2025.1 https://review.opendev.org/c/openstack/nova/+/944890 16:27:44 <fwiesel> No, no problem. 16:28:05 <fwiesel> I take it slowly. So, that's from my side. 16:28:13 <fwiesel> Back to you Uggla. 16:29:07 <Uggla> fwiesel, take it easy, I wish you a prompt recovery. 16:29:46 <Uggla> #topic Open discussion 16:30:45 <ishanwar[m]> #topic Bug #2027156 Low hanging fruit 16:30:45 <ishanwar[m]> https://bugs.launchpad.net/nova/+bug/2027156 16:30:46 <Uggla> I know @bauzas wants to discuss about being release liaison. I think this is to make the next release point easier. 16:31:08 <bauzas> oh, it was just because I miss my PTL-Approval gerrit label :D 16:31:30 <bauzas> and I want to know if anyone has concerns with me continuing to do release liaison ? 16:31:50 <gibi> having redundancy in our staffing to push a release make sense to me 16:32:09 <Uggla> no pb on my side. 16:32:10 <sean-k-mooney> not really, we are allowed to chnage that at any time during the release os we can just propsoe the patch 16:32:14 <elodilles> +2 from me too 16:32:15 <sean-k-mooney> and dicuss it ther 16:32:32 <sean-k-mooney> ishanwar[m]: we will loop back ot you next 16:32:38 <bauzas> cool cool, then I'll propose myself :) 16:32:50 <bauzas> that was short and quick :) 16:32:56 <ishanwar[m]> sean-k-mooney: sure 16:33:30 <Uggla> so I think we can discuss ishanwar[m] topic. 16:33:48 <Uggla> ishanwar[m], please go ahead. 16:34:29 <ishanwar[m]> I wanted to have a discussion on https://bugs.launchpad.net/nova/+bug/2027156 16:34:35 <dansmith> that doesn't look like low-hanging fruit to me :) 16:34:47 <dansmith> is there a review up? 16:35:05 <ishanwar[m]> no I am still working on it 16:35:38 <dansmith> okay, if the quota api is not something that we can guarantee is available, then we have to have all the code we have today in nova, plus new code to use the api if present, right? 16:35:40 <ishanwar[m]> wanted to discuss if I understood it correctly,... (full message at <https://matrix.org/oftc/media/v1/media/download/AZZiNbMs1GZbZLUOwiWvG2I_Dg7_qJ2sHecbqrafIK6K_9GcmEgBnqAHwcu-vfVKIV_6lx6PmdvT59UIS_UEVuBCeV8jVjWgAG1hdHJpeC5vcmcvekZxaUt3b3hkYnpmanJ2dkZqQ2lESFJY>) 16:36:30 <ishanwar[m]> the bug mentions https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L2654 16:36:30 <ishanwar[m]> as the candidate for change but i am not sure, if we are counting in ports in that function. 16:36:39 <ishanwar[m]> am i right ? 16:37:15 <sean-k-mooney> so part of the problem is the orginal bug report does not use permalinks 16:37:28 <dansmith> ishanwar[m]: that bug was filed by a neutron person so I suspect we won't be able to answer your question authoritatively without really digging in 16:37:28 <sean-k-mooney> so they dont nessiarly link to the correct code any more 16:38:08 <sean-k-mooney> ishanwar[m]: you are correct that validate_networks does use the quota api already https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L2714 16:38:50 <sean-k-mooney> i guess the real quetion is do we use that in all places we do quota checks 16:39:05 <sean-k-mooney> this was orginaly reported downstream for trian 16:39:13 <sean-k-mooney> so its very possibel that htis was fixed already in master 16:39:25 <ishanwar[m]> Yes correct, but it's still listing all ports to count them, rather than using show_quota_details, right ? 16:40:07 <sean-k-mooney> we do a port list in get_requested_resource_for_instance 16:40:43 <sean-k-mooney> but that not listing all ports for the porject 16:40:49 <ishanwar[m]> sean-k-mooney: not sure if this is fixed elsewhere, I was only able to find one usage of len(ports) 16:40:52 <sean-k-mooney> that is geting port for the current instnace wihch is fine 16:41:03 <ishanwar[m]> sean-k-mooney: yes 16:41:16 <dansmith> we fetch the quota, but we do our own usage calculation it looks like 16:41:31 <sean-k-mooney> oh its this second call https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L2721 16:41:42 <dansmith> we don't ask if the quota is exceeded, we ask what the quota is and then sum the current+new and see if it's over 16:41:45 <sean-k-mooney> that ^ is the line that shoudl be updated 16:41:49 <ishanwar[m]> dansmith: yes in https://github.com/openstack/nova/blob/master/nova/network/neutron.py#L2714 16:41:49 <ishanwar[m]> we are still counting all the ports 16:42:06 <sean-k-mooney> ishanwar[m]: so the bug is this 16:42:23 <sean-k-mooney> there is another extention that gives use both the quota and the uses cont in one call 16:42:34 <ishanwar[m]> yes correct, i think we can use show_quota_details here right ? 16:42:42 <ishanwar[m]> sean-k-mooney: which one ? 16:42:48 <sean-k-mooney> so instead of getting all the ports for the curent project just to count them we can use show_quota_details 16:43:02 <sean-k-mooney> ishanwar[m]: so the performace bug is still there 16:43:04 <dansmith> ...if available 16:43:07 <sean-k-mooney> on this line https://github.com/openstack/nova/blob/6042300453411133652dd8d3b789f5057a084075/nova/network/neutron.py#L2721 16:43:24 <sean-k-mooney> but ya as dan impleid we have to fall back if the extension is not there 16:43:45 <dansmith> it's also racy either way, which is unfortunate 16:43:55 <ishanwar[m]> sean-k-mooney: correct I agree 16:43:55 <ishanwar[m]> so we can use a try catch mech to check if the extension is available or not 16:44:06 <ishanwar[m]> dansmith: oh, how is that ? 16:44:27 <dansmith> ishanwar[m]: if we calculate the quota, allow to proceed and other things in parallel use more quota, we'll be wrong and run out of quota while actually creating the ports 16:45:06 <dansmith> since we already need to handle that, why check quota at all? Just try to create the ports and roll back if you run out.. I guess maybe to provide a quick answer via API, but it will never be guaranteed 16:45:25 <sean-k-mooney> well we do prot creation on the comptues 16:45:36 <sean-k-mooney> so we do the api check to fail fast if we are already at quota 16:45:40 <dansmith> this can say "definitely not enough quota" but it can't say "definitely enough" 16:45:50 <dansmith> right 16:46:30 <ishanwar[m]> Not sure I understood 😓 16:46:45 <sean-k-mooney> ishanwar[m]: so to answer your question, the bug is still there adn we can still optimize this call to neturon 16:46:45 <dansmith> there's a long window between checking quota and consuming it 16:46:59 <sean-k-mooney> but we need to fallback if the extion is not there to the current logic. 16:47:58 <ishanwar[m]> I have another question, i am running the openstack nova dev env on devstack. is there a way how I can test my changes for this bug, sorry I am new to this ? 16:48:53 <dansmith> with devstack yes, make your changes and test manually, but this will also need unit/functional tests in the patch 16:49:37 <sean-k-mooney> ishanwar[m]: to do ^ you just modify the code and do "systemctl restart devstack@n-*" 16:49:50 <sean-k-mooney> i.e. modify the code in /opt/stack/nova 16:50:53 <sean-k-mooney> i generally do it slightly idffently by have a copy in another location and pip install that into the devstack virutal env but you can basiclly just modify the code and restart the affected services. in thsi case devtack@n-api is the nova api 16:50:53 <Uggla> ishanwar[m], also finding something similar to the bug in gerrit can be a good example to know what the patch should have in terms of unit/fn tests. 16:51:33 <ishanwar[m]> this restarts all nova services right ? Shouldn't we check just one, in general what process do you follow ? Do you add breakpoints etc for testing things out ? 16:51:40 <sean-k-mooney> you can look at the function above https://github.com/openstack/nova/blob/6042300453411133652dd8d3b789f5057a084075/nova/network/neutron.py#L2672-L2693 16:51:48 <sean-k-mooney> it does an if/else on if the extenion is supproted 16:52:07 <sean-k-mooney> you can look at howe has_extended_resource_request_extension is implemnted to test for the quta details one 16:52:10 <ishanwar[m]> sean-k-mooney: yes understood, makes sense . 16:52:41 <sean-k-mooney> ishanwar[m]: so you cant really run nova under a debugger 16:52:46 <sean-k-mooney> its posssible but hard 16:52:52 <sean-k-mooney> so generally we add debug logs 16:52:53 <ishanwar[m]> sean-k-mooney: yeah I can implement ifelse for this 16:53:13 <sean-k-mooney> that useful for production as well as in production we will generally only have logs aviable 16:53:43 <ishanwar[m]> sean-k-mooney: okay I see. 16:54:44 <ishanwar[m]> sean-k-mooney: hmm i see. So from what I understood, add logs, for testing, restart the n-api service. Use openstack command line for testing, correct ? 16:55:06 <sean-k-mooney> initally but then you will need to add unit test for each branch 16:55:20 <sean-k-mooney> so mocking the has_extenion_* function for true and false 16:55:37 <ishanwar[m]> Can you share an example ? 16:55:40 <sean-k-mooney> and assert list_port or quota_detials is called approreatly 16:56:00 <sean-k-mooney> yep we can proably do that perhaps after the meeting 16:56:05 <sean-k-mooney> incase there are any other topics 16:56:35 <ishanwar[m]> sure, thanks sean-k-mooney dansmith Uggla 16:56:35 <ishanwar[m]> i think we can close this discussion 16:56:40 <ishanwar[m]> back to you Uggla 16:56:49 <Uggla> thx @ishanwar[m] 16:57:15 <Uggla> any other topic ? 16:58:25 <Uggla> looks we can close the meeting 16:58:34 <Uggla> 3 16:58:51 <bauzas> shoot :) 16:58:53 <Uggla> 2 16:58:55 <dansmith> blastoff 16:59:05 <Uggla> thanks all 16:59:10 <Uggla> #endmeeting