16:00:00 <bauzas> #startmeeting nova
16:00:00 <opendevmeet> Meeting started Tue Jun 14 16:00:00 2022 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:00 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:00 <opendevmeet> The meeting name has been set to 'nova'
16:00:06 <bauzas> howdy back
16:00:19 <dansmith> o/
16:00:26 <bauzas> and welcome on our first June nova meeting
16:00:51 <gibi> o/
16:01:04 <melwitt> o/
16:01:36 * bauzas just hopes we'll have more people joining
16:01:43 <bauzas> but we can start
16:01:50 <Uggla> o/
16:02:03 <bauzas> #topic Bugs (stuck/critical)
16:02:08 <bauzas> #info No Critical bug
16:02:13 <elodilles> o/
16:02:13 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 14 new untriaged bugs (+0 since the last meeting)
16:02:18 <bauzas> #link https://storyboard.openstack.org/#!/project/openstack/placement 26 open stories (0 since the last meeting) in Storyboard for Placement
16:02:22 <bauzas> #info Add yourself in the team bug roster if you want to help https://etherpad.opendev.org/p/nova-bug-triage-roster
16:02:31 <bauzas> I have to admit publicly I feel ashamed
16:02:47 <bauzas> I forgot about the baton when it was mine
16:02:52 <bauzas> throw me tomatoes
16:03:01 <melwitt> 🍅
16:03:04 <bauzas> but alas I triaged one bug :)
16:03:32 <bauzas> apparently, preparing the Summit trip and doing bug triage doesn't mix on my side
16:03:52 <gibi> I took the baton from bauzas in Berlin in person \o/
16:03:59 <bauzas> literally
16:04:14 <gibi> there was a normal amount of bug inflow
16:04:16 <gibi> https://etherpad.opendev.org/p/nova-bug-triage-20220607
16:04:19 <bauzas> it could have been an olympic torch
16:04:47 <gibi> does that count as carry on baggage?
16:05:03 <melwitt> it could be your personal item
16:05:08 <gibi> :)
16:05:08 <bauzas> depends on the size I guess
16:05:19 <bauzas> or it could be seen as a sport gear
16:05:23 <bauzas> anyway
16:05:24 <gibi> I saw two interesting bugs
16:05:28 <bauzas> thanks gibi
16:05:30 <gibi> https://bugs.launchpad.net/nova/+bug/1978372 numa_fit_instance_to_host() algorithm is highly ineffective on higher number of NUMA nodes
16:06:06 <gibi> sean-k-mooney updated me that this is a known ineffciency of our algo
16:06:19 <bauzas> what kind of hardware has this large amount of NUMA nodes ?
16:06:49 * bauzas is always unpleasantly surprised by all the new things that are created around him
16:07:27 <gibi> I'm not sure
16:07:35 <gibi> but I accept that it is possible
16:07:38 <bauzas> 8 NUMA nodes seems large to me, but I'm not a tech savvy
16:07:38 <sean-k-mooney> bauzas: most recnet amd servers
16:07:49 <sean-k-mooney> 16 numa nodes is not uncommon now
16:08:02 <bauzas> my brain hurts.
16:08:08 <sean-k-mooney> you can get 16 numa nodes in a singel socket now
16:08:18 <sean-k-mooney> and i have see systems with 64
16:09:09 * bauzas is network-bound by his gigabit switches at home while he can download at 10
16:09:27 <sean-k-mooney> our current packing default falls apart after about 4-8 numa nodes
16:09:34 <gibi> so right now we are slow by default, but if numa spread is enabled instead of the default pack then it is much better a sean-k-mooney discovered
16:09:37 <bauzas> anyway, sounds an opportunity for optimization then
16:10:04 <sean-k-mooney> i have a patch in flight to change the default
16:10:13 <bauzas> the whole packing strategy is hidden within the code
16:10:13 <sean-k-mooney> ill work on the release note and push it later
16:10:21 <sean-k-mooney> we can continue the discussion there if you like
16:10:27 <bauzas> sure
16:10:36 <sean-k-mooney> bauzas: yep its also not part of the api contract and never was
16:11:10 * bauzas shrugs
16:11:18 <gibi> so the other bug I would like to mention
16:11:21 <gibi> https://bugs.launchpad.net/nova/+bug/1978549 Placement resource_providers table has dangling column "can_host"
16:11:33 <bauzas> anyway, I understand people wondering why our packing stragegy should struggle only after 16 nodes to iterate
16:11:33 <gibi> I marked as wontfix with a small note in the placement documentation
16:11:48 <gibi> https://review.opendev.org/c/openstack/placement/+/845730
16:12:24 <gibi> this was a mistake back in stable/stein
16:12:34 <gibi> and I don't want to go back and touch DB migrations there
16:13:39 <gibi> that is all I had for bug triage this week
16:13:45 <bauzas> gibi: can_host is not part of the DB contract ?
16:13:53 <bauzas> I mean the model
16:14:01 <gibi> it was removed from the DB model since stein
16:14:13 <gibi> but we never added a DB migration to drop the coulmn from the schema
16:14:33 <gibi> but when Placement was split out of nova a new initial DB schema was defined but now without can_host
16:14:37 <gibi> hence the inconsistency
16:14:41 <gibi> on the schema level
16:14:45 <gibi> but nothing is uses can_host
16:14:52 <bauzas> oh I understand
16:16:05 <bauzas> but, if this is post-Stein, the table is removed anyway, no ?
16:16:31 <bauzas> as said in 'Finalize the upgrade'
16:17:05 <sean-k-mooney> i think the issue here is they are doing a postgres to mariadb migration
16:17:22 <sean-k-mooney> so they were using placement manage to create the new db schma
16:17:27 <sean-k-mooney> then trying to do a data migration
16:17:37 <sean-k-mooney> and there orginal db had the column
16:17:41 <sean-k-mooney> btu the target does not
16:18:16 <sean-k-mooney> so if they drop the colum on the souce db then do the data migration it would be fine
16:18:17 <bauzas> ok, I didn't want to enter into the details, let's move on, I think it's safe bet what gibi did
16:18:26 <gibi> ack
16:19:15 <bauzas> melwitt: are you OK with bug triaging this week or do you want me to do it due to my negligence last week as a punition ?
16:19:27 <bauzas> the latter is fine to me
16:19:53 <melwitt> bauzas: sure, maybe better bc I am out on pto next week
16:20:10 <bauzas> melwitt: cool, then I'll steal it from gibi
16:20:16 <melwitt> cool thanks
16:20:23 <bauzas> #info Next bug baton is passed to bauzas
16:20:48 <bauzas> if you don't mind, I'll pass you the baton next week
16:21:01 <bauzas> or we could give it to anyone else
16:21:01 <melwitt> yes that is ok
16:21:27 <bauzas> moving on
16:21:33 <bauzas> #topic Gate status
16:21:35 * gibi feels sudden emptiness in his life
16:21:38 <bauzas> #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure Nova gate bugs
16:21:44 <bauzas> #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly Placement periodic job status
16:21:48 <bauzas> #link https://zuul.opendev.org/t/openstack/builds?job_name=nova-emulation&pipeline=periodic-weekly&skip=0 Emulation periodic job runs
16:21:48 <melwitt> 😂 gibi
16:21:51 <bauzas> #info Please look at the gate failures and file a bug report with the gate-failure tag.
16:21:55 <bauzas> #info STOP DOING BLIND RECHECKS aka. 'recheck' https://docs.openstack.org/project-team-guide/testing.html#how-to-handle-test-failures
16:22:02 <bauzas> voilà
16:22:17 <bauzas> hadn't anything to tell gate-wise
16:22:35 <bauzas> anything anyone ?
16:22:53 <bauzas> gibi: I don't feel exactly empowered with the baton y'know
16:23:42 <bauzas> ok, next topic then
16:23:47 <bauzas> #topic Release Planning
16:23:54 <bauzas> #link https://releases.openstack.org/zed/schedule.html
16:24:01 <bauzas> #info Zed-2 is in 4 weeks, mind your specs
16:24:33 <bauzas> as a reminder, we'll have a SpecApprovalFreeze on Zed-2
16:25:08 <bauzas> fwiw, here is the current list of accepted blueprints, including specless ones : https://blueprints.launchpad.net/nova/zed
16:25:20 <bauzas> (I eventually updated it one hour before...)
16:25:34 <artom> Oh snap, that's in 2 weeks
16:25:40 <bauzas> no
16:25:43 <bauzas> July 14
16:25:49 <artom> wtf brain
16:25:54 <bauzas> unless I'm counting wrong
16:25:56 <artom> Sorry, ignore me, carry on
16:26:19 <bauzas> that being said, as the clock ticks, next week, we'll discuss of a spec review day
16:26:39 <bauzas> just sayin
16:26:56 <bauzas> next topic
16:27:00 <bauzas> #topic OpenInfra Summit
16:27:13 <bauzas> lemme just do a quick wrap-up
16:27:16 <bauzas> #info bauzas, gibi and stephenfin attended the summit
16:27:21 <bauzas> #info Nova meet-and-greet Operators feedback session on Wednesday,  June 8, 2:50pm - 3:20pm got positive feedback
16:27:23 <gibi> (there was beer)
16:27:27 <bauzas> we had a large audience
16:27:36 <sean-k-mooney> gibi: was it tasty
16:27:42 <chateaulav> fun times
16:27:44 <gibi> good beer
16:27:46 <bauzas> gibi: not during sessions tho
16:27:52 <gibi> yeah, bad timing
16:27:57 <sean-k-mooney> bauzas: thats good to hear was the nova session well attended
16:28:03 <bauzas> let's claim this was a productive session
16:28:10 <bauzas> sean-k-mooney: packed room
16:28:18 <sean-k-mooney> excelent
16:28:22 <artom> Full glass, full room, nice
16:28:43 <bauzas> I think this was well deserved, most of the operators thought we disconnected a bit for too long
16:28:53 <bauzas> and the PTG thing doesn't help
16:29:38 <bauzas> one outcome is at least a strong need for a nova recap at every possible gathering
16:30:04 <bauzas> at least they were expecting one at the Summit, but I have to admit I didn't made it
16:30:12 <bauzas> and the OpenInfra Live thing was on April
16:30:26 <bauzas> I guess only a few of them saw it
16:30:34 <sean-k-mooney> the project updates went live on youtube 2 weeks ago
16:30:39 <bauzas> I know
16:30:41 <sean-k-mooney> but i woudl assume many did not see it
16:30:50 <bauzas> that's the OpenInfra Live thing I mentioned
16:31:11 <bauzas> apparently, people pay more attention to cycle highlights when it's in-person
16:31:17 <sean-k-mooney> right but that may have been in april but the videos only got published on youtube in june
16:31:47 <bauzas> anyway, something easily solvable
16:32:01 <bauzas> one other thing, communication
16:32:10 <bauzas> not a surprise, our ML isn't read
16:32:37 <bauzas> and given they lag a lot, they don't think this is a valuable time to chime in
16:32:59 <artom> Wait, so ops show up to Summit, but don't read the ML? How do they know when Summit is? ;)
16:33:04 <bauzas> (they lag by the number of releases)
16:33:14 <bauzas> artom: easy answer : Twitter
16:33:27 <sean-k-mooney> and infra foundation marketing
16:33:29 <gibi> yes, we was asked to tweet more
16:33:36 <bauzas> someone very seriously explained to me they'd prefer nova tweets
16:33:43 <artom> We as in the developers? o_O
16:33:45 <gmann> yeah summit info is communicated in many other ML and places not only openstack-discuss
16:33:50 <gibi> artom: yes, please :)
16:33:51 <melwitt> huh.
16:33:55 <sean-k-mooney> could we auto treat the release notes some how
16:34:02 <dansmith> that's crazy, IMHO
16:34:18 <bauzas> sean-k-mooney: they know about our prelude
16:34:27 <bauzas> but again, they laaaag
16:34:30 * sean-k-mooney rememebrs the april fools twitter as a message bus spec
16:34:39 <artom> If they lag releases, what's the point of tweeting, presumably about stuff we're working on *now*?
16:35:11 <chateaulav> build interest and involvement
16:35:16 <bauzas> sean-k-mooney: I'm half considering to register a Twitter handle like @Yo_the_Openstack_Nova_gang
16:35:26 <gmann> chateaulav: +1
16:35:48 <artom> But what if I'm an anti-social curmudgeon?
16:35:58 <bauzas> artom: heh
16:36:12 <bauzas> anyway, that one wasn't an easy problem to solve
16:36:22 <gmann> operator involvement with developers is one of the key and open issue in board meeting too. and TC also raised it to them.
16:36:29 <bauzas> fwiw, I proposed them to only register to 'ops' and 'nova' ML tags
16:36:41 <gmann> some idea is to combine the events ops meetup and developers one but let's see
16:36:41 <bauzas> both in conjunction
16:36:47 <chateaulav> i am too, dont have to be a genius. just start with little things. if it goes to the ML and you think is worthwhile then tweet it and reference the ML archive and irc chat
16:36:48 <bauzas> gmann: please
16:37:01 <sean-k-mooney> gmann: the best way to adress that woudl proably be to converge the events and bring back the devs summit
16:37:03 <bauzas> gmann: I feel the community more fragmented now we're spît
16:37:05 <bauzas> spliut
16:37:12 <gmann> sean-k-mooney: +1
16:37:17 <gmann> yeah, true
16:37:32 <bauzas> chateaulav: as I said, I begged them to correctly use the ML tags
16:37:35 <gibi> sean-k-mooney: +1
16:37:47 <gmann> we got separated when we combined the things :)
16:38:09 <bauzas> and I ask people to *not* make use of [nova][ops] for communicating unless here we agree on the need to engage
16:39:09 <bauzas> #action nova team will only exceptionnally make use of [nova][ops] for important communication to ops. If you're an Ops, feel free to register to both tags in the ML
16:39:19 <artom> Tbf, the ML lately seems to be openstack-support, anecdotally
16:39:43 <sean-k-mooney> not entirly
16:39:50 <dansmith> mostly
16:39:51 <bauzas> artom: alas, we merged openstack@ openstack-dev@ and openstack-ops@
16:39:58 <sean-k-mooney> we do discuss gate issues and some dev issues
16:40:03 <artom> So maybe there is room for dev -> ops announcement-type stuff.
16:40:11 <bauzas> openstack@ was the place for troubleshooting
16:40:13 <artom> sean-k-mooney, right, I'm missing a "mostly" in there
16:40:39 <artom> @Nova_PTL twitter account? :D
16:40:51 <bauzas> anyway, there are things the nova team can solve and there are other things that are way out of our team scope :)
16:40:53 <sean-k-mooney> so we spilt the events and merged the lists. if only we did the reverse :)
16:41:04 <sean-k-mooney> im not sure there is much we can do right now to adress this topic
16:41:16 <bauzas> sean-k-mooney: correct and I want to move on
16:41:30 <bauzas> this wasn't an ask to find a solution, just a feedback
16:41:53 <artom> You can't tell us "plz tweet moar" and not expect the convo to derail :P
16:41:56 <bauzas> #link https://etherpad.opendev.org/p/r.ea2e9bd003ed5aed5e25cd8393cf9362 readonly etherpad of the meet-and-greet session
16:42:20 <bauzas> artom: I personnally stopped twitting unless exceptional opportunities, not the one to blame
16:42:40 <bauzas> now, back to productive things
16:42:44 * artom never tweets, but always twit
16:42:48 <bauzas> you'll see a long list of complains
16:43:14 <sean-k-mooney> some of which have been adresed in newer releases
16:43:19 <bauzas> I encourage any of you to go read the etherpad and amend it (with the write URL of course)
16:43:23 <gmann> bauzas: any feedback on RBAC scope things if that is discussed in nova sessions also other than ops meetup ?
16:43:46 <bauzas> sean-k-mooney: yeah, I've seen you munging a lot of them, thanks
16:44:04 <bauzas> gmann: I asked about it, this was too way advanced for them
16:44:19 <bauzas> but I pointed them the links to the new rules and personans
16:44:22 <bauzas> personas*
16:44:46 <bauzas> also, this was a 30-min session,
16:45:00 <gmann> bauzas: ok
16:45:01 <bauzas> so, please understand we were basically only able to scratch the surface
16:45:10 <gibi> gmann: we had another session around service roles
16:45:13 <sean-k-mooney> some of the pain points are on our backlog
16:45:33 <sean-k-mooney> so its good that operaters have vlaidated that they still care about them
16:45:39 <gmann> bauzas: I understand, just checking in case any specific feedback we got from nova sessions
16:45:49 <bauzas> for the pain points, I'll diligently try to make sure all of them are adressed
16:45:51 <sean-k-mooney> im thinkign of iothread and virtio-multiqueue
16:46:02 <bauzas> gmann: honestly this was frustrating
16:46:20 <gmann> gibi: ack. do you have any link of that, I will combine those to discuss in RBAC meeting next week
16:46:26 <gibi> gmann: my sort summary on the serivce roles https://meetings.opendev.org/irclogs/%23openstack-nova/%23openstack-nova.2022-06-13.log.html#t2022-06-13T06:43:52
16:46:28 <bauzas> give me 30 mins more and I could have made operators to sign off for sending herds of contributors to the nova project
16:46:40 <gmann> gibi: thanks
16:47:08 <bauzas> don't be surprised if I'm pinging some of you
16:47:17 <bauzas> I want the etherpad to be curated
16:47:18 <gibi> gmann: and this is the session etherpad but it is a bit of a mess https://etherpad.opendev.org/p/deprivilization-of-service-accounts
16:47:37 <bauzas> gibi: this wasn't a mess
16:47:47 <bauzas> this was rather a prank, I guess
16:48:04 <gibi> then I was pranked :)
16:48:13 <bauzas> exactly my point
16:48:19 <bauzas> step 1 : propose a forum session
16:48:28 <bauzas> step 2: let gibi see it
16:48:38 <bauzas> step 3 : make sure gibi will attend it
16:48:41 <sean-k-mooney> in princiapl we shoudl be able to lable all endpoing that are used for inter service comunicatoin as needing the service role
16:48:52 <bauzas> step 4: don't attend your own session and let gibi lead it instead
16:48:59 <bauzas> step 5 : profit.
16:49:15 <gibi> (just background: when I entered the room for that session I was cornered that there is nobody who can lead the session)
16:49:16 <sean-k-mooney> the service role really shoudl not be able to acces any api other then the inter service apis
16:50:01 <sean-k-mooney> that woudl allow use to entirly drop our use fo the admin role eventually
16:50:02 <gmann> sean-k-mooney: yes, that is direction we are going and has to be a careful audit to verify this.
16:50:10 <bauzas> can we stop on the forum discussions ?
16:50:17 <sean-k-mooney> yes we can move on
16:50:20 <bauzas> anyone having a last question or remark ?
16:50:35 <bauzas> (just timeboxing, sorry)
16:50:43 <sean-k-mooney> no worries
16:50:46 <bauzas> #topic Review priorities
16:50:51 <bauzas> #link https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement+OR+project:openstack/os-traits+OR+project:openstack/os-resource-classes+OR+project:openstack/os-vif+OR+project:openstack/python-novaclient+OR+project:openstack/osc-placement)+label:Review-Priority%252B1
16:50:55 <bauzas> #link https://review.opendev.org/c/openstack/project-config/+/837595 Gerrit policy for Review-prio contributors flag. Naming bikeshed in there.
16:51:08 <bauzas> #action bauzas to propose a revision of https://review.opendev.org/c/openstack/project-config/+/837595
16:51:15 <bauzas> #link https://docs.openstack.org/nova/latest/contributor/process.html#what-the-review-priority-label-in-gerrit-are-use-for Documentation we already have
16:51:20 <bauzas> that's it on my side
16:51:59 <bauzas> I encourage cores to make use of the flag if they wish
16:52:06 <bauzas> #topic Stable Branches
16:52:12 <bauzas> elodilles: your time
16:52:15 <elodilles> #info stable/train is blocked - melwitt's fix: https://review.opendev.org/c/openstack/nova/+/844530/
16:52:24 <elodilles> #info stable branch status / gate failures tracking etherpad: https://etherpad.opendev.org/p/nova-stable-branch-ci
16:52:37 <elodilles> release patches proposed (yoga, xena, wallaby): https://review.opendev.org/q/project:openstack/releases+is:open+intopic:nova
16:52:50 <sean-k-mooney> yep i was going to proceed with merging https://review.opendev.org/c/openstack/nova/+/844530 but wanted to ask if there were any objections
16:52:57 <sean-k-mooney> i have also commended on the release patches
16:53:10 <sean-k-mooney> most of the patches i wanted to land are now landed over night
16:53:25 <bauzas> cool, I'll do a bit of reviews then
16:53:52 <bauzas> any other point to raise about stable ?
16:54:01 <elodilles> nothing else i think
16:54:17 <bauzas> cool
16:54:21 <elodilles> sean-k-mooney bauzas : thanks for looking at the release patches
16:54:21 <bauzas> last point then
16:54:32 <sean-k-mooney> elodilles: happy too
16:54:34 <bauzas> elodilles: I have to do it, sean-k-mooney told me already :)
16:54:40 <elodilles> :]
16:54:52 <bauzas> #topic Open discussion
16:54:58 <bauzas> there were nothing on the agenda
16:55:10 <bauzas> for the sake of those last 5 mins, any item to raise ?
16:55:18 <melwitt> bauzas: I realized it would be better if I did bugs this week bc if I'm out next week, that's even less time 😆
16:55:37 <melwitt> I won't be at the next meeting but I can put my bug etherpad link on the agenda for yall
16:55:41 <bauzas> melwitt: I'm both flexible and ashamed
16:55:48 <bauzas> melwitt: pick anytime you want
16:55:54 <bauzas> and I'll do the overlap
16:55:57 <sean-k-mooney> the only item i was gong to raise was releases. we had a request to do stable release last week but that is proceeding anyway
16:55:57 <melwitt> bauzas: ok, I will do this week. sorry for the confusion
16:56:06 <bauzas> melwitt: np
16:56:32 <bauzas> I guess not a lot of people are reading our weekly meeting and even less of them do bug triage
16:56:47 <bauzas> weekly minutes*
16:57:12 <bauzas> but, not a reason for anarchy with no meetings and agenda ! :D
16:57:27 <bauzas> (and proper highlights)
16:57:33 <gibi> we should try to have our meeting on twitter ;)
16:57:41 <bauzas> OK, I guess we can call the wrap
16:58:08 <bauzas> gibi: I was surprised noone debated on the tool itself
16:58:23 * sean-k-mooney looks side eyed at gibi
16:58:26 <bauzas> I could instagram nice pictures of me coding
16:58:42 <bauzas> like, me outside coding
16:58:48 <bauzas> me inside in my office room
16:58:49 <melwitt> start a twitch channel
16:58:53 <sean-k-mooney> totally we should all just stream our coding on twitch :)
16:59:03 <bauzas> I'm feeling too old
16:59:32 <bauzas> but at least I'm happy to hear the TC be young-minded with Tik-Tok releases
16:59:40 <gibi> :D
16:59:51 <bauzas> on that last word,
16:59:59 <bauzas> #endmeeting