16:00:11 #startmeeting nova 16:00:11 Meeting started Tue Sep 6 16:00:11 2022 UTC and is due to finish in 60 minutes. The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:11 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:11 The meeting name has been set to 'nova' 16:00:16 hello everyone 16:00:18 o/ 16:00:26 #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 16:00:46 o/ 16:01:31 let's start while people arrive 16:01:38 we have a few things to discuss 16:02:04 #topic Bugs (stuck/critical) 16:02:10 o/ 16:02:12 o/ 16:02:19 #info Three Critical bugs 16:02:24 that's a bummer 16:02:36 but we have a story against each 16:02:41 #link https://bugs.launchpad.net/nova/+bug/1988311 See https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030325.html 16:02:49 seems we have a way forward 16:03:04 with oslo.concurrency patches 16:03:23 yeah there is a new oslo.concurrency release 16:03:30 yup 16:03:44 the next step is to get the global version bump 16:03:52 correct 16:04:00 I will abandon the nova specific patches once we have the bump merged 16:04:28 we have to make sure though that stable/yoga gets the new oslo version too 16:04:37 yeah 16:04:43 i also have patches to fix fastener and update oslo in flight 16:04:48 I don't see any u-c change yet 16:04:49 thos will be for A 16:04:51 a this point 16:05:07 o/ 16:05:23 but I assume this patch will be done sooner than later 16:06:07 yes based on the mail thread Herve will propose the bump 16:06:26 https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030352.html 16:06:36 ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (db) https://review.opendev.org/c/openstack/nova/+/831193 16:06:37 ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (objects) https://review.opendev.org/c/openstack/nova/+/839401 16:06:37 ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (manila abstraction) https://review.opendev.org/c/openstack/nova/+/831194 16:06:38 ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (drivers and compute manager part) https://review.opendev.org/c/openstack/nova/+/833090 16:06:38 ribaudr proposed openstack/nova master: Attach Manila shares via virtiofs (api) https://review.opendev.org/c/openstack/nova/+/836830 16:06:39 ribaudr proposed openstack/nova master: Bump compute version and check shares support https://review.opendev.org/c/openstack/nova/+/850499 16:06:39 ribaudr proposed openstack/nova master: Add metadata for shares https://review.opendev.org/c/openstack/nova/+/850500 16:06:40 ribaudr proposed openstack/nova master: Add instance.share_attach notification https://review.opendev.org/c/openstack/nova/+/850501 16:06:40 ribaudr proposed openstack/nova master: Add instance.share_detach notification https://review.opendev.org/c/openstack/nova/+/851028 16:06:42 ribaudr proposed openstack/nova master: Add shares to InstancePayload https://review.opendev.org/c/openstack/nova/+/851029 16:06:42 ribaudr proposed openstack/nova master: Add instance.power_on_error notification https://review.opendev.org/c/openstack/nova/+/852084 16:06:44 ribaudr proposed openstack/nova master: Add instance.power_off_error notification https://review.opendev.org/c/openstack/nova/+/852278 16:06:44 ribaudr proposed openstack/nova master: Add helper methods to attach/detach shares https://review.opendev.org/c/openstack/nova/+/852085 16:06:46 ribaudr proposed openstack/nova master: Add libvirt test to ensure metadata are working. https://review.opendev.org/c/openstack/nova/+/852086 16:06:46 ribaudr proposed openstack/nova master: Add virt/libvirt error test cases https://review.opendev.org/c/openstack/nova/+/852087 16:06:48 ribaudr proposed openstack/nova master: Add share_info parameter to reboot method for each driver (driver part) https://review.opendev.org/c/openstack/nova/+/854823 16:06:48 ribaudr proposed openstack/nova master: Support rebooting an instance with shares (compute and API part) https://review.opendev.org/c/openstack/nova/+/854824 16:06:50 ribaudr proposed openstack/nova master: Change microversion to 2.94 https://review.opendev.org/c/openstack/nova/+/852088 16:06:59 oops sorry for that ^ 16:07:24 gibi: would you propose a patch to nova's requirements.txt or do we automatically pull the latest ? 16:07:38 https://review.opendev.org/c/openstack/requirements/+/856044 16:07:39 we are not constrained in noca 16:07:43 nova 16:07:50 i think that is the requiremtns bump 16:08:05 gibi: ok, so we'll benefit from it 16:08:17 yep, automatically 16:08:20 we shoudl once that merges yep 16:08:27 https://github.com/openstack/nova/blob/master/requirements.txt#L34 16:08:33 yup, we're not constrained 16:08:46 ok, let's wait for the req patch to be merged then 16:09:11 for stable/yoga, seems we don't cap too 16:09:12 https://github.com/openstack/nova/blob/stable/yoga/requirements.txt#L30 16:09:26 correct 16:09:31 coo 16:09:31 l 16:09:41 we can either pull in the latest oslo release there or cap fasterners 16:09:42 ok, we're on the good way 16:09:55 moving on to the next bug then 16:10:09 #link https://bugs.launchpad.net/nova/+bug/1988316 Will be set to High now that the gate is skipping the test 16:10:17 people agree with the proposal ? 16:10:23 sean-k-mooney: I would bump this https://github.com/openstack/requirements/blob/stable/yoga/upper-constraints.txt#L26 16:10:24 => High 16:10:45 gibi: oh you're right about yoga's u-c 16:11:32 on the unshelve stuff I think we need a nova fix to return a better error message, but I'm OK to drop it from Critical to High 16:11:40 yeah 16:11:45 it is not blocking the gate at the moment 16:11:47 not saying there will be no bug to fix 16:11:51 ya its a latent bug 16:12:06 but this isn't longer critical as the gate is now working back 16:12:07 unshelve thing meaning the to-host cell one? 16:12:13 yes 16:12:15 yup 16:12:29 saying we have to restrict to the same cell 16:12:35 yeah, that's not new just recently exposed so doesn't seem like we should be too critical in the bug status there 16:12:55 im saying its latent as its the same error we would have got with unshelve to AZ we just dont have test coverage for that in tempest that was also corss cell 16:13:23 right, and I'm saying being latent it's not a regression, so really shouldn't be >high :) 16:13:31 +1 16:13:45 honestly, I'm in favor of saying "sorry but we don't support *yet* cross-cell unshelve to host" that's it 16:13:59 so the fix could be just docs 16:14:06 and a better exception handling 16:14:09 yep we can do both 16:14:18 anyway, set to High, there is 16:15:13 well, the test needs fixing too 16:15:24 it's just being skipped there right now right? 16:15:29 yup 16:15:34 the test is not broken 16:15:40 it just should not be run in a cross cell env 16:15:41 yeah, the test shouldn't be assuming we could 16:15:57 no this is a job config issue 16:15:59 if we would want a test, this would have to be negative 16:16:05 you cant tell if its cross cell form the api 16:16:09 like "I know this can't work" 16:16:12 so tempest cannot detech that 16:16:20 so this has to be done via job config 16:16:24 ahah true 16:16:47 anyway, bug report is still open, feel free to comment it out for resolution 16:16:50 moving on 16:16:57 Rajat Dhasmana proposed openstack/nova-specs master: Clarify client changes in rebuild spec https://review.opendev.org/c/openstack/nova-specs/+/856164 16:17:00 #link https://bugs.launchpad.net/nova/+bug/1988482 Proposed patch on the fly https://review.opendev.org/c/openstack/nova/+/855658 16:17:13 as said, reviews are welcome ^ 16:17:23 sean-k-mooney: shoudn't the test be using an az? 16:17:25 I exceptionally relaxed my review needs 16:17:32 anyway, we can discuss elsewhere 16:17:42 so I gave +2 for a patch without testing 16:17:43 dansmith: i dont think so be we can follow up after ya 16:18:21 tl;dr: the problem is with PrettyTable having a changed behavior 16:18:43 yes so they have now fixed this by reverting the behaivor 16:18:49 gibi: sean-k-mooney: wants to address this issue now ? 16:18:56 i tested with the previous release broken release and new release with the revert 16:19:19 i woudl prefer to merge and backprot https://review.opendev.org/c/openstack/nova/+/855658 16:19:22 so https://review.opendev.org/c/openstack/nova/+/855658 wouldn't be needed ? 16:19:25 ack 16:19:27 to yoga so we never need to thnk about htis again 16:19:51 its not needed unless the decided to reinstate the feature or change the default 16:20:01 or we can hardcode the alignmend and not worry 16:20:18 I agree we sean-k-mooney 16:20:31 gibi: didn't know this verb 16:20:43 I sean-k-mooney, you sean-k-mooney 16:20:49 :) 16:20:53 :) 16:20:57 but as an extra: in general we don't have py39 lines in the global requirements so this can hit us with different packages 16:21:00 bauzas: :) 16:21:17 I think there was a discussion in relmgmt about adding those lines 16:21:40 in yoga we should have been upper constrained 16:21:44 it could hit distos if the distro had 3.4.0 16:21:45 (rather on requirements, but yes) 16:21:52 yeah 16:22:10 elodilles: then there :) same folks :) 16:22:12 anyway i think we can just review this normally 16:22:14 not critical 16:22:18 agree 16:22:23 it was just annoying as it was blocking some backports 16:22:25 its not now 16:23:07 ok, so chaning the prio to High ? 16:23:16 because of the requirements patch ? 16:24:04 I agree that this is not critical now 16:24:20 ok, punting the prio then 16:24:35 sean-k-mooney: please just update the bug report with your thoughts please 16:24:41 ack 16:24:43 will do 16:24:47 moving on, we still have lots of things to discuss 16:25:12 #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 9 new untriaged bugs (+2 since the last meeting) 16:25:18 #link https://storyboard.openstack.org/#!/project/openstack/placement 26 open stories (-1 since the last meeting) in Storyboard for Placement 16:25:24 #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster 16:25:29 #info bug baton is being passed to Uggla 16:25:32 that's it for bugs 16:25:47 yep I will not forget that time. 16:25:48 (yeah Uggla kindly offered to keep the baton for this week, thanks to him) 16:26:02 Uggla: no pain here 16:26:10 this is stretch goal 16:26:15 anyway, moving on 16:26:24 #topic Gate status 16:26:31 #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs 16:26:35 #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status 16:26:40 #link https://zuul.openstack.org/builds?job_name=tempest-integrated-compute-centos-9-stream&project=openstack%2Fnova&pipeline=periodic-weekly Centos 9 Stream periodic job status 16:26:45 #link https://zuul.opendev.org/t/openstack/builds?job_name=nova-emulation&pipeline=periodic-weekly&skip=0 Emulation periodic job runs 16:26:50 #info Please look at the gate failures and file a bug report with the gate-failure tag. 16:27:01 I checked all the periodic runs and all are green 16:27:08 #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures 16:27:23 I think we discussed about the current gate bugs, we can move on 16:27:33 #topic Release Planning 16:27:39 #link https://releases.openstack.org/zed/schedule.html 16:27:45 #info Zed-3 was last Thursday 16:27:50 hereby, I declare 16:27:55 #info FeatureFreeze officially declared today 16:28:02 \o/ 16:28:05 #action bauzas to -2 open changes that were having an accepted spec 16:28:11 #action bauzas to run the script to move specs with merged implementation to 'implemented' 16:28:21 #link Zed tracking etherpad: https://etherpad.opendev.org/p/nova-zed-blueprint-status 16:28:40 we still have some client changes that require reviews 16:29:25 in theory, we are in client libs freeze too 16:29:27 https://releases.openstack.org/zed/schedule.html#z-final-clientlib 16:29:48 elodilles: what do you think of that ? 16:30:13 Merged openstack/nova master: Follow up for the PCI in placement series https://review.opendev.org/c/openstack/nova/+/855185 16:30:13 just wanted to warn that we are in freeze period 16:30:16 for that as well 16:30:18 shall we stop merging novaclient and OSC patches until next cycle or shall we pretend this never existed and be pragmatic ? 16:30:21 Merged openstack/nova master: Doc follow up for PCI in placement https://review.opendev.org/c/openstack/nova/+/855186 16:30:47 ok, so releasing wouldn't be acceptable 16:31:01 but I guess merging patches doesn't create issues 16:31:09 we can review but proably hold +w for a week 16:31:10 well, if patches merges, and new release is needed, then RFE is needed 16:31:17 the branches will reopen at RC1 16:31:31 so non critical things probably should be postponed to Antelope 16:32:18 I don't disagree with this, I was just wondering about the difference between holding a merge, and merging without releasing 16:32:26 we can alwasy do an early release of novaclient or osc in a few weeks 16:32:49 bauzas: rc bug fixes is really the only issue 16:32:52 merging without releasing can be confusing for zed as the branch have it but the release does not 16:32:54 until we have the branch 16:33:02 ok, so let's agree to hold our commits 16:33:03 we cant merge feature incase there is a bug fix needed 16:33:11 branches will be create at rc1 16:33:35 bauzas: ++ 16:33:40 #agreed Client patches are on Client libs freeze, please don't accept to merge new changes until RC1 happens 16:33:52 voila 16:33:53 did the rebuild bfv change merge? 16:34:02 for the client I mean 16:34:02 the novaclient one, not 16:34:17 we were reviewing it today, hence my question 16:34:40 we have novaclient support until 2.92 16:34:42 hmm, seems bad to not have that present in the client if we've got it as a server feature 16:34:49 and OSC support until 2.91 IIRC 16:35:31 dansmith: fwiw, we have a difference between what the server supports and what the client can negociate 16:35:31 dansmith: that is why i was suggesting we do a release in a few weeks 16:35:48 yoga didn't had this issue, no microversion patch was merged 16:35:56 so, this is my first time :) 16:36:21 okay, so you're saying we wait for rc1 then do a client release soonly so people will be able to actually use it? 16:36:28 its not the first time we have not had parity between api and cli 16:36:31 dansmith: yep 16:36:37 okay 16:36:38 dansmith: at leat that is what im suggesting 16:36:48 dansmith: I'm not saying anything, I'm asking for help 16:37:15 if we all agree on the discrepancy, we should also agree on the fact this isn't great 16:37:29 as long as there's a short timeline to getting the client released then I guess that's okay 16:37:34 that client libs freeze doesn't really help, fwiw 16:37:43 it does 16:37:46 normally 16:37:59 ideally, we should have a one week difference I think between the server freeze and the client freeze 16:38:09 bauzas: do we not? 16:38:21 I guess I'm confused, seems like this is the week to get the client things finalized 16:38:23 dansmith: not in the zed schedule 16:38:37 hence my call for clarification 16:38:42 hrm okay 16:38:54 that 16:38:57 is a bit of a bummer 16:39:11 can't disagree 16:39:25 I dunno what distros do for picking client releases, but they'd need to know that they need the one right after the zed server release to have this 16:39:28 but whatever 16:39:35 feedback i guess for tc/cross project ptg session 16:39:41 agrred 16:39:46 I think that's just for the release team 16:39:57 the antelope schedule is already accepted but I guess we can debate it at the PTG 16:40:00 ya perhaps 16:40:03 we approve the schedule but don't really do much other than that I think 16:40:30 well the schdule can be updated if we think it need changes 16:40:32 just as a hint for next cycle https://releases.openstack.org/antelope/schedule.html 16:40:34 its just anohter review 16:40:51 anyway, moving on 16:41:19 #link https://review.opendev.org/c/openstack/releases/+/855974 Nova Zed cycle highlights 16:41:29 I'd appreciate if people could review my prose 16:41:52 even if we didn't had a lot of blueprints, this was a productive cycle 16:42:07 and I guess the marketing folks will love all the buzz words I used 16:42:36 kudos to the team for the hard work you made, very well appreciated 16:42:44 I will re-review shortly 16:42:58 gibi: YAMLs don't help with fancy formatting, unfortunatelty 16:43:19 next topic then 16:43:20 that makes me sad 16:43:25 #topic PTG planning 16:43:33 thanks to gibi, 16:43:36 #link https://etherpad.opendev.org/p/nova-antelope-ptg Antelope PTG etherpad 16:43:50 I needed it as I had a topic :) 16:44:00 feel free to add your items you'd like to discuss 16:44:07 the earlier be the better 16:44:32 since we didn't had an agenda yet, I made a proposal : 16:44:35 also we should add a retro topic about prorities and more frequent deadlines 16:44:38 #info bauzas has asked for 4 hours per day for Tuesday to Friday (Bexar room) 16:44:52 gibi: feel free to add anything 16:45:04 anything? :) no you don't want that :D 16:45:22 well, this is an open text file 16:45:31 just joking :) 16:45:37 but yeah, we'll have a retro, of course 16:45:48 * bauzas has to add the usual topics we discuss 16:45:56 so, about the proposed schedule 16:46:03 I booked 4x4 hours 16:46:12 with the bexar room 16:46:17 maybe we would need less 16:46:25 I can unbook the room if so 16:46:38 so, the plan is to not feel constrained by the schedule 16:47:08 but obviously, if we keep with 4x4hours, we'll manage decent time offs :) 16:47:24 #link https://ptg.opendev.org/ptg.html PTG schedule 16:47:40 anyone having serious concerns with the proposed timeslots ? 16:47:47 anyone wanting an asian-friendly timeslot ? 16:48:14 I don't hear anything 16:48:15 sold. 16:48:34 last point, and I'm rushing because time flies 16:48:37 #link https://lists.openstack.org/pipermail/openstack-discuss/2022-September/030301.html operator friendly timeslots, which ones for Nova ? 16:49:07 as you probably already read, TC proposes us to reserve some slots for operators-friendly discussions 16:49:14 I'm more than happy with the idea 16:49:39 so, shall we reserve one hour per day about this ? 16:49:53 or just, for example, one hour on the first two days ? 16:49:58 do we need 4 hours total? 16:50:01 sorry 16:50:03 or two hours back-to-back ? 16:50:11 or no operator hours at all ? 16:50:23 personnally, I feel this can be more than interesting 16:50:26 the idea is just one of those slots for nova 16:50:39 and we should have at least 2 hours at the beginning of our talks 16:50:46 so you can have more slots than that, but the idea is that they're not overlapping so operators have only one place to be, if they have multiple interests 16:50:56 in order to discuss about what they told us during the other days 16:51:21 dansmith: yean, we could reserve other slots than the 4x4 hours we already have 16:51:44 but honestly, I just hope we can find propre timeslots ? 16:51:50 within the existing ones 16:51:51 so, 16:52:08 any proposal or should I just shoot a strawman ? 16:52:53 OK, you leave me no choice 16:53:16 #info bauzas to propose Tuesday 13-15UTC for operator-friendly slots 16:53:40 works for me :) 16:54:11 #info bauzas to propose a tentative slot on Wed 16-17UTC for operators who aren't able to join on Tuesday 16:54:37 that would leave 3 hours for operations-friendly talks 16:54:56 this is a bit huge but I expect those discussions to be productive 16:55:17 that's it for me 16:55:22 next topic 16:55:29 #topic Review priorities 16:55:35 let's skip them, we're overtimle 16:55:42 #topic Stable Branches 16:55:45 elodilles: shoot 16:55:52 #info stable/yoga is blocked due to py39 job is failing with prettytable 3.4.0; prettytable 3.4.1 fixes the issue so as soon as constraints are updated in requirements repository the gate will be OK 16:56:00 (the same we discussed above) 16:56:01 we discussed this 16:56:03 cool 16:56:07 #info stable/stein (and older) are blocked: grenade and other devstack based jobs fail with the same timeout issue as stable/train was previously 16:56:15 #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci 16:56:18 the infection increases. 16:56:23 these were the main points :X 16:57:33 well, I'd say stable/train can be a concern to me 16:57:40 for others, well, ExtendedMaintenance 16:58:23 so, the ones who care should just raise a hand 16:58:39 but we're approaching the top of the hour 16:59:03 #topic Open discussion 16:59:09 anything anyone for last min ? 16:59:37 guess not 16:59:42 thanks all 16:59:44 #endmeeting