15:59:36 <gibi> #startmeeting nova
15:59:36 <opendevmeet> Meeting started Tue Jun 21 15:59:36 2022 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:59:36 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:59:36 <opendevmeet> The meeting name has been set to 'nova'
15:59:41 <gibi> chair gibi bauzas
15:59:44 <gibi> #chair gibi bauzas
15:59:44 <opendevmeet> Current chairs: bauzas gibi
16:00:01 <gibi> lets wait a bit before we start
16:02:06 <elodilles> o/
16:02:36 <gibi> OK, lets get started
16:02:48 <gibi> bauzas asked me to run this as he might be late a bit
16:02:51 <gibi> #topic Bugs (stuck/critical)
16:03:01 <gibi> #info One critical bug
16:03:07 <gibi> #link https://bugs.launchpad.net/nova/+bug/1979047 Centos 9 Stream bug failure
16:03:22 <gibi> the centos 9 steam job is made non-voting on Friday to unblock the gate
16:04:52 <gibi> there is some info from tripleooo about the same issue, they pinned libvirt version
16:04:59 <gibi> should we try to do the same?
16:06:48 <elodilles> good question o:) is there any other option? :-o
16:06:57 <gibi> keep it non voting forever?
16:07:04 <gibi> :)
16:07:04 <elodilles> :S
16:07:40 <gibi> we would need somebody who care about this job to propose a fix
16:08:00 <gibi> I don't see a long line of volunteers
16:08:01 <gibi> so moving on
16:08:12 <gibi> #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 12 new untriaged bugs (-2 since the last meeting)
16:08:13 <elodilles> i mean, does it make any difference to pin libvirt compared to keep the job non-voting ?
16:08:33 <artom> Looks like a libvirt fix is being released, based on https://bugzilla.redhat.com/show_bug.cgi?id=2092856
16:08:39 <gibi> elodilles: if we keep non-voting and the libvirt issue is fixed the job will start being green again
16:08:47 <artom> So maybe just wait for that to land in CS9, and make the job voting again?
16:08:56 <gibi> artom: yepp, we can do that
16:09:09 <Uggla> o/
16:09:20 <elodilles> ack, then that is the best option for now
16:09:28 <gibi> ack, seems like we do that as that is easy
16:09:50 <gibi> so untriaged bug backlog looks healthy thanks melwitt for the triages
16:10:14 <gibi> #info If you are interested add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster
16:10:22 <gibi> next the baton goes to bauzas
16:10:29 <gibi> I will ping him when he is back
16:10:34 <gibi> let's assume he took it :)
16:10:41 <gibi> #info Next bug baton is passed to bauzas
16:10:50 <gibi> any other bug you would like to discuss?
16:11:16 <artom> Not me? Cool then
16:11:42 <gibi> I think bauzas rescheduled himself as he missed having the baton during the summit
16:12:00 <gibi> (I'm following bauzas agend from the wiki :)
16:12:07 <artom> For once it's not my job to figure out the "overwatch" rotation :P
16:12:15 <artom> (Sorry for leaking downstream)
16:12:32 <gibi> anyhow I don't see any bugs raised so moving on
16:12:35 <gibi> #topic Gate status
16:12:40 <gibi> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:12:46 <gibi> #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status
16:12:57 <gibi> looks green
16:13:07 <gibi> #link https://zuul.opendev.org/t/openstack/builds?job_name=nova-emulation&pipeline=periodic-weekly&skip=0 Emulation periodic job runs
16:13:18 <gibi> green too
16:13:25 <gibi> #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:13:29 <gibi> #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures
16:13:35 <gibi> we had two gate blockers on Friday
16:13:42 <gibi> the one with the libvirt issue from above
16:13:56 <gibi> and another with a tempest test case failure
16:14:19 <gibi> the tempest failure was fixed by https://review.opendev.org/c/openstack/tempest/+/846345/3
16:14:27 <gibi> so the gate should be OK now
16:14:36 <gibi> but honestly I haven't checked the results today
16:15:29 <gibi> I quickly checked the master gate is not blocked
16:15:42 <gibi> any gate issue we need to discuss here?
16:16:51 <gibi> #topic Release Planning
16:16:57 <gibi> #link https://releases.openstack.org/zed/schedule.html
16:17:02 <gibi> #info Zed-2 is in 3.5 weeks
16:17:07 <gibi> #startvote Spec review day on July 5th ? (yes/no)
16:17:07 <opendevmeet> Begin voting on: Spec review day on July 5th ? Valid vote options are , yes, no, .
16:17:07 <opendevmeet> Vote using '#vote OPTION'. Only your last vote counts.
16:17:26 <gibi> #vote yes
16:17:59 <Uggla> #vote yes
16:18:48 <gibi> others?
16:19:13 <elodilles> #vote yes
16:19:50 <elodilles> (though my vote really does not count i guess o:))
16:20:02 <sean-k-mooney> #vote yes
16:20:25 <sean-k-mooney> i think i can do the 5th but folks form the us may be off
16:20:44 <gibi> hangover from 4th of July?
16:20:53 <sean-k-mooney> perhaps :)
16:21:21 <sean-k-mooney> i think if there are no objections however the 5th shoudl be fine
16:21:21 <elodilles> :)
16:21:51 <gibi> #endvote
16:21:51 <opendevmeet> Voted on "Spec review day on July 5th ?" Results are
16:21:51 <opendevmeet> yes (4): Uggla, sean-k-mooney, gibi, elodilles
16:22:18 <gibi> #action bauzas to send out a note to the ML about the spec review day on 5th of July
16:22:35 <gibi> #topic Review priorities
16:22:42 <gibi> #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+label:Review-Priority%252B1
16:22:59 <gibi> #link https://review.opendev.org/c/openstack/project-config/+/837595 Gerrit policy for Review-prio contributors flag. New proposal there, please vote.
16:23:03 <gibi> #link https://docs.openstack.org/nova/latest/contributor/process.html#what-the-review-priority-label-in-gerrit-are-use-for Documentation we already have
16:23:26 <gibi> anything to raise about review priority?
16:24:19 <gibi> #topic Stable Branches
16:24:24 <gibi> elodilles:
16:24:26 <gibi> your turn
16:24:44 <elodilles> #info gates are mostly not blocked
16:24:56 <elodilles> #info stable/train is blocked - melwitt's fix: https://review.opendev.org/c/openstack/nova/+/844530/
16:25:04 <elodilles> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci
16:25:13 <elodilles> release patches proposed (yoga, xena, wallaby): https://review.opendev.org/q/project:openstack/releases+is:open+intopic:nova
16:25:27 <elodilles> the release patches are on the way now
16:25:34 <gibi> \o/
16:25:58 <elodilles> (sorry i've re-used last week's lines as there are not that much news o:))
16:26:36 <gibi> elodilles: thanks
16:26:41 <elodilles> np
16:26:43 <gibi> anything else about stable?
16:26:51 <elodilles> nothing from me
16:28:03 <gibi> #topic Open discussion
16:28:10 <gibi> (artom) Can we revisit stable func test backport policy? Specific patch stack: https://review.opendev.org/c/openstack/nova/+/791480/1 Previously we didn't want to backport func test infrastructure because it just offloads the backport debt onto whoever is doing backport for older than train releases. Some time has passed now, are there still operators running < stable/train and needing backports?
16:28:16 <gibi> https://etherpad.opendev.org/p/r.ea2e9bd003ed5aed5e25cd8393cf9362 indicates a majority of "train or older", but how many are on the "older" half of that?
16:29:02 <artom> Bringing this up again because at this point there are 3 separate bugfixes depending on those test refactor patches
16:29:08 <artom> But basically $topic :)
16:29:28 <gibi> personally I'm OK to bring back any test refactors to stable branches
16:30:17 <artom> IIRC last time we talked about this, elodilles was worried that anyone backporting to rocky and older would get the "refactor" debt unloaded onto them
16:30:35 <elodilles> IF there are enough reviewers then maybe it could be OK, though it's best to keep things on the safe side and backport less risky things
16:31:11 <artom> Are func test refactors really risky though?
16:31:14 <sean-k-mooney> well test code is less risky in general since it does not affect the runing code
16:31:27 <elodilles> artom: yepp, if we backport mass amount of functional test refactors, then it makes the backport harder for older branches
16:31:27 <gibi> it does not risk the production code, it risk the CI stability
16:31:36 <sean-k-mooney> and in some cases are not actully installed with the production code
16:32:11 <artom> elodilles, yep, agreed on that. So in practice, bauzas and gibi were at summit, is anyone still doing backports for < stable/train?
16:32:18 * elodilles agrees with gibi
16:32:34 <artom> As in, Red Hat will have to care about stable/train for a long time
16:32:46 <sean-k-mooney> :(
16:32:53 <gibi> I tend to propose backports to stable/pike while I were in E///
16:32:54 <sean-k-mooney> its true but :(
16:32:55 <artom> Yeah, sad face indeed
16:33:16 <gibi> I assume E/// still uses stable/pike
16:33:24 <gibi> but I don't think we will see much backports there
16:33:48 <elodilles> i see that there are less and less backports pushed toward old branches, though if we make it harder for developers, then it will not help the situation as well
16:33:56 <artom> gibi, elodilles, so I can buy the gate stability argument for integration tests, but when was the last time we had an issue with func tests that wasn't about versions of things like tox?
16:34:20 <sean-k-mooney> elodilles: well right now its hard to backprot to train because once you get past about victoria you are missing the helpers
16:34:25 <gibi> we have still open a bug where nova funct test leaks notifications between tests :)
16:34:26 <elodilles> gibi: unfortunately my pike patches are hanging there without reviews, so.... o:)
16:34:34 <gibi> elodilles: I know :)
16:34:56 <gibi> artom: so func test could be problematic
16:35:11 <gibi> as they run eventlets
16:35:26 <gibi> and sometimes depends on extrenal things like sysfs :)
16:35:44 <gibi> still I think we should backport func test infra
16:35:53 * sean-k-mooney notes we have backported some of the helpers downstream already
16:36:27 <sean-k-mooney> its the integrated_helpers that are most useful
16:37:05 <artom> gibi, I feel like sysfs should be poisoned in func tests...
16:37:15 <gibi> artom: I have a patch
16:37:26 <gibi> https://review.opendev.org/c/openstack/nova/+/844627
16:37:40 <sean-k-mooney> artom: for the most part its mocked already modulo bugs
16:37:56 <sean-k-mooney> but yes the poison is also good to do
16:38:31 <gibi> does anyone here strongly disagree to backport func test infra?
16:38:36 <elodilles> also note, i'm not completely against backporting func test refactors, but i still think it is best to keep it in a low level and we should not backport massive refactors :/
16:39:12 <gibi> elodilles: it is a tradeoff, either you take the risk by backporting the refactor or take the risk when you backport a fix that needs to be changed due to the missing refactor
16:39:13 * artom would not consider https://review.opendev.org/c/openstack/nova/+/791480/1 massive
16:39:33 <gibi> artom: 3 lines! come on! :)
16:39:33 <sean-k-mooney> well or we drop the functest on backport
16:39:43 <elodilles> if a refactor breaks something then we don't have the bandwidth to keep it maintained i think. stable should be stable :(
16:39:45 <artom> The one on top is a bit bigger ^_^
16:39:46 <gibi> I strongly against droping the func test on backport
16:39:56 <artom> elodilles, so that was the crux of my argument
16:40:09 <artom> Red Hat *will* maintain stable/train for literally years, we have no choice
16:40:28 <sean-k-mooney> well at least 2.5 more
16:40:29 <artom> But we don't want to inflict pain on anyone maintaining older than stable/train
16:40:38 <artom> So: do those folks... well, exist? :)
16:41:01 <artom> At Summit, what release did operators say they were on?
16:41:05 <sean-k-mooney> technially we still maintain queens downstream too for a while more
16:41:42 <sean-k-mooney> but i would prefer to have the backports of the func infra as that makes backporting simpelr in the long run
16:41:45 <gibi> artom: there were no specifics other than what is in the etherpad
16:42:01 <artom> So only "train or older" with no info if it's train... or older :(
16:42:43 <elodilles> should have been added 'train' + 'stein and older' :D
16:43:24 <artom> Yeah :S
16:43:35 * artom checks stable/rocky backports: https://review.opendev.org/q/project:openstack/nova+branch:stable/rocky
16:44:34 <artom> So compared to stable/train, there are 4 patches last updated this year, compared to train's ~50
16:44:53 <elodilles> why not https://review.opendev.org/q/project:openstack/nova+branch:stable/stein ?
16:45:04 <artom> Because I suck at alphabet :P
16:45:09 <elodilles> o:)
16:45:36 <artom> Seems to be mostly Felix and Vlad Gusev...
16:45:50 <artom> But similar level of involvement drop-off
16:46:21 <elodilles> a bit more patch but without reviews, yes :/
16:46:24 <gibi> so what if we say, func infra backport are OK to stable/train as there are maintainers there but not further backwards
16:46:44 <gibi> due to lack of maintainers
16:47:04 <elodilles> gibi: that is good for RH but not really helps to encourage backporting for older branches
16:47:26 <sean-k-mooney> well even train is in em now right
16:47:34 <artom> I think elodilles's point is that if we rewrite the fixes to not need func test refactors *before* train, it helps maintainers of older branches, such as they are
16:47:48 <sean-k-mooney> given the peopel we have its hard to keep maintianing older branches
16:47:57 <keerthi> can some one help on review this https://blueprints.launchpad.net/nova/+spec/define-max-volume-limit-at-flavor ?
16:47:58 <sean-k-mooney> train is 5 releases old currently
16:48:03 <artom> If we backport func test refactors, we're offloading that rewriting work onto whoever is still working with rocky and older
16:48:14 <gibi> keerthi: we are in a meeting right now
16:49:04 <artom> OTOH, why would it be wrong of facilitating the work of the majority?
16:49:10 <gibi> so we say no func test backport as it there is no maintainers but also we say if we dont backport func infra then we dont have maintaniers, this is contradiction now
16:49:12 <sean-k-mooney> keerthi: we can proably discuss it after the meeting or when we are done with this topic
16:49:31 <keerthi> sure Sean, I will wait for it
16:50:23 <elodilles> anyway, i have said my preference, and i'm only one of the stable maintainers o:)
16:50:52 <gibi> either we don't have stein maintainers and then I don't feel back about not helping them, or we have maintainers and the I would ask them to backport the func refactor from train to stein
16:51:01 <gibi> s/back/bad/
16:51:45 <gibi> and I would help them by backporting the refactor up until train
16:51:56 <gibi> that would be a win-win
16:52:00 <artom> gibi, hard to argue with that
16:52:37 <gibi> elodilles: would you be -1 on a func infra backport to stable/train?
16:52:45 <gibi> (or even -2?)
16:53:01 <elodilles> 'func infra'?
16:53:12 <gibi> functional test infrastructure backport
16:53:30 <artom> As opposed to infrastructure that's really grooooovey :D
16:53:31 <gibi> like the one artom linked above
16:53:54 <elodilles> so you mean zuul jobs change?
16:54:06 <elodilles> (i've lost in the links :))
16:54:11 <artom> No, stuff like https://review.opendev.org/c/openstack/nova/+/791481/1
16:54:14 <artom> For example
16:54:38 <gibi> elodilles: changes that are not adding a bugfix but refactoring the functional test infrastrucutre
16:54:49 <gibi> like backporting helpers from newer branches
16:54:51 <artom> I mean, if https://review.opendev.org/c/openstack/nova/+/751364 is allowed...
16:54:53 * bauzas waves super super late
16:54:59 <bauzas> sorry, it took longer than expected :(
16:55:10 <artom> And https://review.opendev.org/c/openstack/nova/+/751363/3
16:55:28 <artom> Honestly, with those last 2, we've set the precedent that func test refactors are fair game
16:55:46 <sean-k-mooney> artom: wasnt the main issue with the backport you did that you skipped ussui by the way
16:56:01 <artom> sean-k-mooney, that's a different series, and I fixed that
16:56:07 <sean-k-mooney> oh ok
16:56:12 <sean-k-mooney> nevermind then
16:56:24 <gibi> we are running out of time and still had on point on the agenda
16:56:39 <elodilles> artom: we did backport simple&small func test refactors in the past
16:57:33 <elodilles> gibi: if we are waiting for my vote: as i said, i'd prefer not, but not against :)
16:57:50 <elodilles> so no -2 from me
16:57:57 <gibi> OK, so if you are not blocking such backport then we are done
16:58:06 <gibi> artom: anything else?
16:58:13 <bauzas> so, MHO is that we should be picky if needed
16:58:33 <elodilles> bauzas: +2
16:58:34 <bauzas> anyway, time flies
16:58:53 <gibi> sure still proper review on those patches but not blanket -2
16:59:02 <bauzas> my other point can be deferred to next week
16:59:02 <artom> gibi, nope, I'm coming away from this with the understanding that func test refactors are fair game if they help make the fixes cleaner and easier to backport (and increase confidence in the reproducer test)
16:59:10 <gibi> ack
16:59:21 <gibi> moving on then
16:59:22 <gibi> (bauzas) Opportunities for low-hanging-fruits, anyone ? (only if we have time left)
16:59:31 <gibi> 40 secs :)
16:59:32 <bauzas> no time, let's punt to next week
16:59:36 <gibi> ack
16:59:46 <gibi> any last minute issue to raise today?
16:59:50 <bauzas> tl;dr: think about how to help on-off contributors like interns
17:00:10 <gibi> #endmeeting