16:00:12 #startmeeting nova 16:00:12 Meeting started Tue Jan 10 16:00:12 2023 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:12 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:12 The meeting name has been set to 'nova' 16:00:21 #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 16:00:45 good UTC afternoon everyone 16:00:48 who's around ? 16:00:49 Oj 16:00:51 o/ 16:00:53 o/ 16:00:58 o/ 16:01:05 o/ 16:01:17 o/ 16:01:18 o/ 16:01:26 o/ 16:01:29 welcome everyone 16:01:48 #topic Bugs (stuck/critical) 16:01:53 #info No Critical bug 16:01:57 #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 27 new untriaged bugs (+2 since the last meeting) 16:02:02 I have to do my duty 16:02:27 if someone has time for triaging upstream bugs, that'd be lovely but I can do it 16:02:35 and then we'll go back to the roster 16:02:41 #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster 16:03:22 bauzas, bug triage, I can do it is it helps you. 16:03:27 +1 for the roster 16:03:51 Uggla: I'll take the baton for this week, but if you want, we can both look at bugs 16:04:07 #info bug baton is being passed to bauzas 16:04:29 any particular urgent bug to discuss ? 16:04:40 I guess we'll discuss the tox saga in the next chapter 16:04:59 if no, let's move on and go to the gate topic 16:05:06 #topic Gate status 16:05:12 #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:05:16 #link https://zuul.openstack.org/builds?project=openstack%2Fnova&project=openstack%2Fplacement&pipeline=periodic-weekly Nova&Placement periodic jobs status 16:05:21 #link https://review.opendev.org/c/openstack/tempest/+/866049 proposal to add more delay for centos9s-fips job 16:05:26 #info Please look at the gate failures and file a bug report with the gate-failure tag. 16:05:30 placement and nova gate is unblocked as far as I know. we merged the two tox4 patches 16:05:31 #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures 16:05:51 I was about to tell it once I was done with the weekly items 16:05:52 so, 16:06:12 I see nova patches progressing in the gate queue 16:06:17 so I think we are OK 16:06:33 I haven't chacked osc-placement and python-novaclient status 16:06:40 os-vif should be unblocked too 16:06:46 their master gate is good, I checked 16:06:54 but for stable branch gate, python-novaclient patches need to merge #link https://review.opendev.org/q/I442568a5f5900e593feb2b5527109e0aa79e5aa7+status:open 16:07:12 nice 16:07:15 and osc-placement is failing <= stable/yoga https://review.opendev.org/c/openstack/osc-placement/+/869451 16:07:17 #info we had a funky week due to tox4 but now gate is better https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031709.html 16:07:41 it is failing on placement functionla job where we use master placement but stable branch constraints so mismatch 16:07:42 gmann: cool, I'll look at the novaclient patch 16:08:11 I cannot remember why we do not use master constraints in that placement functional job? 16:08:12 oh, I did it already 16:08:13 the there are follow up request for nova and placement too: i) remove the install_command usage ii) check if pdf doc build works or not 16:08:33 gmann: we should I believe 16:08:43 this one https://github.com/openstack/osc-placement/blob/master/tox.ini#L30 16:09:26 we want it t o run against master i belvie 16:09:28 gmann: hm, do we have placement in the upper-constraints? 16:09:43 perhaps although we shoudl not 16:09:54 only lib projects should eb in uc 16:09:57 gibi: no because we use it from master source 16:10:12 sean-k-mooney: this is for testing placement itself 16:10:22 so we need the latest 16:10:22 this is testing osc-placement I believe 16:10:32 correct 16:10:36 yeah testing osc-placement with master placement 16:10:42 its the fucntional test for placement openstackclient plugin 16:10:45 and constarints are inherited from https://github.com/openstack/osc-placement/blob/fdf10423bf29b7cfd5e88d9d06e1313ee881ab80/tox.ini#L17 16:10:48 we're discussing of testing osc-placement 16:10:59 using placement master branch 16:11:07 yes and how the functional job forces the master branch of placment 16:11:14 yup 16:11:18 i think this was because fo the placment fixture 16:11:18 gibi: it use stable/yoga for yoga testing #link https://github.com/openstack/osc-placement/blob/stable/yoga/tox.ini#L18 16:11:23 for master it is all good 16:11:28 master gate 16:11:33 agreed with gmann 16:11:34 https://github.com/openstack/osc-placement/commit/da8cd4d68b06399c607776db2a704b4578146996 16:11:45 I don't see the issue sorry 16:11:54 for stable/zed somehow constraints matches so we do not see failure 16:11:55 if I have a osc client patch, I don't wanna hold my check on a placement release 16:11:55 so the tox.ini is correct 16:12:25 sean-k-mooney: stable/yoga constraints will not work with latest placement 16:12:39 anyway, tox4 upgrade was fun 16:12:45 that is why it is failing on os-traits version mismatch 16:13:02 for stable branchs we likely need to pin yes 16:13:05 * bauzas wonders whether pyproject.toml files were having the same issues than tox.ini 16:13:26 yeah, either pin placement or use master constraints for this latest placement testing 16:13:27 bauzas: likely not because they handel constratits differnrlty 16:13:42 anyways we can discuss it after meeting as it seems need more discussions 16:13:43 gmann: so on stable/yoga this is an issue https://github.com/openstack/osc-placement/blob/stable/yoga/tox.ini#L31 16:13:57 as it install master placement on stable yoga 16:14:09 gibi: yes. 16:14:11 ya so we just need to pin the branch in that line which is posible 16:14:14 OK I got it 16:14:21 as i remember constraints from master also caused issues on stable branches (in placement) 16:14:30 sean-k-mooney: yeah, my point is that if the tox developers don't verify the behaviours of tox.ini and rather say they just verify the toml ones, then I'm a bit afraid 16:14:47 bauzas: its not related to that 16:15:14 bauzas: openstack is quite differnt form ontehr python project in how we use setuptools/pbr and the idea of a constratis file 16:15:23 all other stable branches gate are good. I tested them after pin 16:15:32 bauzas: tox4 was a rewrite, and they are not intending to keep all the tox3 behavior (I learned yesterday) 16:15:43 sean-k-mooney: sure I'm not saying we should create a pyproject.toml filze 16:15:50 gmann: thanks! 16:16:06 sean-k-mooney: but I was wondering why nobody was finding the problem until we told them 16:16:08 bauzas: well that has been discussed and we might want to eventually but we should lop back to one topic 16:16:25 bauzas: because constrait files is baicaly an openstack thing 16:16:44 sean-k-mooney: we didn't had only problems with constraints 16:16:45 sean-k-mooney: openstack is the primary user in the wild probably but it is a normal pip functionality 16:16:53 we are the comunity that drove that in pip 16:17:10 but yes its is a pip capablity 16:17:11 gibi: ack, good to know 16:17:14 bauzas: fwiw I'm helping to move zuul and opendev projects to nox 16:17:29 * bauzas notes it 16:17:33 isnit that a pti issue 16:17:41 dont we specify tox explictly 16:17:43 (because there are lots of little tox changes and when I file issues upstream the resposne is often "why would you do that?" and the answer is well because v3 supported it so ya...) 16:17:51 sean-k-mooney: yes for openstack. I personally think openstack should change too 16:17:56 but it is work 16:18:03 perhaps a good ptg topic 16:18:10 +1 16:18:12 I'm not super keen on the nox thing 16:18:13 how to evovle pti 16:18:14 not only for our project 16:18:21 so I'd like to have a real discussion about it 16:18:33 should be a cross-project discussion honestly 16:18:34 (I'm also not super keen on tox destroying the world for fun and profit) 16:18:37 right i was thinking comuity/cross project topic 16:18:45 dansmith: me too 16:18:50 right 16:19:11 honestly, I'd have preferred to continue using tox3 until we test all the modifications they did 16:19:11 ok so looping back. we need to pin placment on stable branches fo osc-placment 16:19:19 but, this is done 16:19:30 and we have unbolcked the other "nova" gates 16:19:42 with followups for the install_commands changes correct? 16:19:54 sean-k-mooney: gibi: ok for pin but I was thinking we have to test latest placement there? I do not know reason just want to confirm 16:20:21 gmann: git+https://opendev.org/openstack/placement.git#egg=openstack-placement line on master 16:20:27 is so we get the latest placement fixture 16:20:32 and the fact that tox developers just rewrite their tox.ini by what they want without continuing to support the existing (and without deprecating them at least) let me think about any new major tox versions we'd have 16:20:38 so we need to test latest placement in master gate only 16:20:38 gmann: I don't know the reason either 16:20:38 so we can test when new feture are addded 16:21:17 sean-k-mooney: right but that we do not need on stable osc-placement version right? 16:21:41 gmann: correct it can and should use the stable version 16:21:52 I was hesitating to pin placement there as I do not know the original reason ot use latest placement there 16:21:54 they do need to be kept in sync however 16:22:06 its caputed in teh comme messange here https://github.com/openstack/osc-placement/commit/da8cd4d68b06399c607776db2a704b4578146996 16:22:18 ack 16:22:48 thanks 16:24:33 ok i think we have covered those topics and can move on 16:24:42 I was about to ask 16:24:49 ok, let's move on then 16:24:54 yeah 16:25:00 #topic Release Planning 16:25:04 #link https://releases.openstack.org/antelope/schedule.html 16:25:07 #info Antelope-3 is in 5 weeks 16:25:16 (and featurefreeze... :) ) 16:25:21 #info As a reminder, Nova SpecFreeze deadline is this Thursday https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031636.html 16:26:08 I'll modify launchpad's https://blueprints.launchpad.net/nova/antelope once we're done with reviewing specs 16:26:58 I have then a question 16:27:07 #link https://lists.openstack.org/pipermail/openstack-discuss/2023-January/031645.html I guess we'll want to have a virtual PTG againĀ ? 16:27:39 for the Bobcat planning :) 16:27:55 (man, haven't but should have voted this time...) 16:28:02 can we create teh 2023.2 specs repo after spec freeze 16:28:19 i dont recall seeing a vote email go out 16:28:24 there was one 16:28:28 i tought this was selected by the foundation 16:28:51 nope, sent during your holidaus 16:28:56 ok well the foundation has never consicidently sent me vote urls anyway 16:28:58 hence why you haven't probably seen it 16:29:01 format will be 1 virtual PTG + 1 in-person PTG per year. in-person PTG will be aligned to summit+release 16:29:36 gmann: any chance to have those physical attendances (I don't like the PTG naming) to be at the start of the release time ? 16:29:47 for Vancouver, ship is sailed 16:30:25 sean-k-mooney: I didn't get one either 16:30:50 bauzas: yeah, that is what foundation staff updated that they will try to aligned it with summit and openstack relaese but let's see as it can be challenging too 16:30:52 dansmith: sean-k-mooney: https://lists.openstack.org/pipermail/openstack-discuss/2022-December/031551.html 16:31:10 oh, to the list 16:31:19 70% of the list was probably on PTO 16:31:24 but that vote url is a personal one 16:31:28 not one everyone can use right 16:31:33 for vancouver, yes as event is in between of cycle it needs to be that time 16:31:44 you are ment to get one to your personal email for all votes 16:31:56 sean-k-mooney: good point, again, I haven't voted so I haven't tested it 16:32:31 sean-k-mooney: I think it is public vote so anyone can vote with that url but one per IP or so 16:32:37 anyway, this is done anyway 16:32:42 if I am remembering public vote things correctly 16:32:46 gmann: fun fact is, voting isn't closed AFAICS 16:33:02 perhaps in the tool but the name has been released 16:33:14 yup, another ship sailed 16:33:18 in anycase i can create a 2023.2 folder in the spec repo 16:33:20 ohk :) honestly saying I have not given mush attention to release naming :) 16:33:28 sean-k-mooney: +1 16:33:33 sean-k-mooney: wait for the spec freeze 16:33:41 but bauzas if you want to do that after spec freeze we can wait till after thrsday 16:33:41 Merged openstack/nova stable/wallaby: Adapt websocketproxy tests for SimpleHTTPServer fix https://review.opendev.org/c/openstack/nova/+/866194 16:33:42 sean-k-mooney: or it would perhaps confuse people 16:33:53 sean-k-mooney: well, I'm not opposed to delegate 16:34:07 cool well i ahve done it in the past but lest do it next week 16:34:44 anyway, I guess we vote for a virtual PTG on end of March ? (that was the original question) 16:35:20 and we don't say "meh, no, just let's do a physical one at right the middle of the release, and go f*** our deadlines !" 16:35:22 :D 16:36:16 I hear crickets, so I decide so 16:36:19 well, that is the usual PTG time, so i think it is not something to vote for :) 16:36:23 not sure physical one can be utilized same as our normal PTG but can plan for some operaotr oriented discussions 16:36:41 #info we'll have a virtual Nova PTG on March 27-31 16:36:45 virtual one in March will be the normal vPTG we have for cycle 16:36:48 I'll register our name :) 16:37:08 gmann: honestly, I don't expect much more from what we currently have at the Forum 16:37:13 gmann: but let's be creative 16:37:17 bauzas: yeah. 16:37:42 if we really need to s/Forum/PTG to have more people coming, I'm not opposed to a rebranding 16:37:53 some teams do mid-cycle PTG-s, so the in-person one can be considered such, i guess 16:37:56 we asked same PTG fomat questions on vancouver PTG but that is not answer yet 16:38:23 elodilles: given the Summit will be around Spec Freeze, that's gonna be fun 16:38:27 I think in openstack-discuss ML but anyways let's see 16:38:50 elodilles: but we could do 'review sprints' or 'implementation sessions' 16:39:03 if that helps people wanting to come :) 16:39:10 +1. good idea 16:39:16 I guess we're done on that topic 16:39:31 moving on 16:39:36 #topic Review priorities 16:39:44 #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+(label:Review-Priority%252B1+OR+label:Review-Priority%252B2) 16:39:50 #info As a reminder, cores eager to review changes can +1 to indicate their interest, +2 for committing to the review 16:40:13 I'll slowly but progressively use this label for knowing what to review 16:40:17 so you know about it 16:40:25 next topic (we only have 20 mins) 16:40:29 #topic Stable Branches 16:40:34 elodilles: happy to see you back 16:40:41 :) 16:40:49 we mostly discussed, but 16:41:10 #info stable branches seem to be unblocked / OK -- except placement, osc-placement, ... 16:41:33 thanks gmann for fixing the tox4 issues on stable branches! 16:41:45 #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci 16:41:54 and maybe one more thing: 16:41:55 for pin not fix :) 16:42:13 gmann: yepp, a good workaround for now ;) 16:42:53 so maybe we should do nova stable releases as there are already some merged content on zed, yoga and xena 16:43:29 and a small side track: i would want some stable core opinions about my comment here: https://review.opendev.org/c/openstack/nova/+/867912 16:43:46 as the zed release might depend on that 16:43:53 elodilles: sure, want me to create the releases ? 16:44:01 stable releases, I mean 16:44:04 but that can be done after the meeting 16:44:08 we're in a good timing 16:44:45 elodilles this is your main question right https://review.opendev.org/c/openstack/nova/+/867912/5/nova/conf/workarounds.py 16:44:49 bauzas: yes, if you have time :) otherwise i can also propose them 16:44:57 sean-k-mooney: yes 16:45:06 so i considerd that when reviewing 16:45:25 i tought the value of the fix in behaivor outwaied the possibel performance impact 16:45:44 but you are correct that fliping the default would maintain the old behaivor 16:46:01 fliping the default woudl also result in a workaorunds option being enabeld by default 16:46:07 which we generally avoid 16:46:20 so not changing the default felt more consitent to me 16:46:41 im not sure what other think ^ 16:46:47 hmmm 16:46:52 loading the context 16:47:20 the context is mostly captured in teh help text :) 16:47:21 so it was a opt-out config 16:47:36 and when backporting it suddently became opt-in 16:47:44 right? 16:47:49 no we did not change it when back porting 16:47:55 elodilles: question is should we 16:48:14 so on master we default to the new more desireable beahvior 16:48:15 no. to put it simple: we change the default behavior on stable branches 16:48:30 and elodilles is asking shoudl we keep the old beahivor in the backports 16:48:36 https://review.opendev.org/c/openstack/nova/+/864773/3/nova/conf/workarounds.py default on master is False 16:48:40 so that's opt-in 16:49:00 which is the same for the backports 16:49:02 i mean, given an installed environment, they upgrade, and the behaviour is changed with the upgraded nova 16:49:14 elodilles: correct 16:49:18 elodilles: I don't see a problem here with keeping it opt-in 16:49:41 bauzas: well it will required all operator to update there config to adress the bug 16:49:57 bauzas: it is opt-out :) you have to change the configuration to have the existing behaviour 16:50:10 sean-k-mooney which is the current behaviour on master, right? 16:50:32 on master we now set the reseved value on instnace boot 16:50:51 before this patch we set reserved wehn deleteing 16:51:12 because the option defaults to False, right? 16:51:28 if you set to True, you keep the original behaviou 16:51:29 the workaroudn option is a skip to opt out yes and it default to false 16:51:31 behaviour 16:51:35 correct 16:51:43 ok 16:51:48 so, say we backport this 16:51:54 as it is 16:52:22 to understand, that means we'll change the behaviour by upgrading to the latest (obviously .y) release 16:52:26 as is then the stable branchs get the new behaivor and you need to modify the config to restore the old behaiovr 16:52:44 but this is an internal implemation detail that is not part of any public contract 16:52:47 I think we have semver for communicating the behavioural change 16:52:49 so we felt this was better 16:53:29 can we discuss this out of the meeting ? 16:53:33 we have 5 mins left 16:53:35 sure 16:53:42 #topic Open discussion 16:53:46 (bauzas) As a reminder, Summit proposals deadline is today 11:59pm PT 16:53:57 so if we release now zed, then the operators who uses zed, need to update their configs to have NOT to change how nova works 16:53:59 you have the very last minute to add your topics 16:54:26 fwiw, I proposed two forum sessions about nova onboarding and operator meet&greet 16:54:41 if someone wants to co-host the onboarding forum session, let me know 16:55:19 gmann: correct me but forum proposal deadline is on april ? 16:55:37 I'm confused by the fact they added the forum proposals in the tool 16:55:42 bauzas: I hope so. I asked Erin in email but no reply yet. But from form it shows april 16:55:54 yup, hence my confusion 16:56:31 I just hope we won't get like Berlin some Forum sessions that very look like Summit sessions with slides and one-sided discussions 16:56:31 Oh, come on...after all the jobs passed, one "tempest-integrated-compute" job times out :( https://zuul.opendev.org/t/openstack/build/8b07fa0b71c94fb6abb2b350c5c84c74 16:56:44 kashyap: on our meeting atm 16:56:45 Oops, sorry for that random chime in. 16:56:52 bauzas: Yeah, only realized it just now (sorry) 16:56:56 np 16:57:00 (gmann) updates on switching the RBAC(scope and new defaults) defaults in nova 16:57:08 bauzas: if you need a co-host for the onboarding then you can write me up 16:57:09 you have 3 mins 16:57:36 gibi: noted. I'm used to do some onboarding and I have ideas but having a co-speaker is somehow better 16:57:44 sure 16:58:07 looks like gmann is gone and our meeting will end 16:58:14 patch is ready for review #link https://review.opendev.org/c/openstack/nova/+/866218/12 16:58:19 cool thanks 16:58:40 gmann: I've seen your ping, I just didn't had time to review it yet 16:58:44 please check and it need placement test fixture change too which is depends-on on nova change #link https://review.opendev.org/c/openstack/placement/+/869525/3 16:59:05 do not want to delay the switching at the end of cycle 16:59:32 I've just said review-prio +2 16:59:44 Tempest already run integration job with new RBAC on Nova, glance, neutron and cinder and it is passing finw 16:59:48 thanks 16:59:51 \o/ 17:00:03 Pavlo Shchelokovskyy proposed openstack/nova master: Process all nodes when host aggregate changes https://review.opendev.org/c/openstack/nova/+/869687 17:00:15 ok, we're on time 17:00:16 thanks all 17:00:20 #endmeeting