16:01:25 <gibi> #startmeeting nova
16:01:25 <opendevmeet> Meeting started Tue Aug 31 16:01:25 2021 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:25 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:25 <opendevmeet> The meeting name has been set to 'nova'
16:01:53 <gibi> I was just reading the scrollback about the nics
16:01:55 <gibi> :)
16:01:56 <gibi> o/
16:01:59 <gmann> o/
16:02:05 <elodilles> o/
16:02:15 <dansmith> o/
16:02:45 <gibi> #topic Bugs (stuck/critical)
16:02:55 <gibi> we have one bug marked critical
16:02:59 <gibi> https://bugs.launchpad.net/nova/+bug/1940555 around SQLAlchemy URL handling and the fix is going through the gate: https://review.opendev.org/c/openstack/nova/+/805663
16:03:15 <bauzas> \o
16:03:24 <gibi> it is bounced since I wrote the agenda but I requeued it
16:03:42 <sean-k-mooney> ack
16:03:53 <gibi> #link 17 new untriaged bugs (+6 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New
16:04:09 <gibi> I had no time to check the untriaged bugs in the last couple of days
16:04:22 <gibi> we have 0 bugs marked with xena-rc-potential tag #link https://bugs.launchpad.net/nova/+bugs?field.tag=xena-rc-potential
16:04:27 <gibi> please start marking release critical bugs with xena-rc-potential tag
16:04:53 <bauzas> tbc, those need to be regression bugs
16:04:59 <gibi> yepp
16:05:12 <bauzas> ie. something that worked before Xena but now has issues with this release
16:05:37 <bauzas> we only have 2 weeks before RC1 in order to merge bugfixes that *aren't* regressions
16:05:49 <bauzas> or those would be fixed in the next Yoga release
16:05:59 <gibi> and backported :)
16:06:09 <sean-k-mooney> i dont think we currenlty have any critical bugs that would need the rc flag
16:06:13 <bauzas> so, pings are appreciated besides classic feature review requests
16:06:30 <bauzas> we have an interesting race condition issue with placement
16:06:41 <gibi> yepp I saved that for the gate issue topic
16:06:51 <gibi> lets go there
16:06:52 <bauzas> this isn't only impacting the gate, right?
16:07:20 <gibi> it is a new race so so far we only have infor from the gate about it
16:07:23 <bauzas> but yeah, let's discuss this bug on the gate section
16:07:28 <gibi> #topic Gate status
16:07:32 <gibi> Nova gate bugs #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure
16:07:41 <gibi> We had the allocation deletion conflict bug #link https://bugs.launchpad.net/nova/+bug/1836754 that hopefully resolved when we landed #link https://review.opendev.org/c/openstack/nova/+/688802 yesterday
16:08:05 <gibi> We have a project creation race in allocation update bug as well #link https://storyboard.openstack.org/#!/story/2009159 that is affecting the those gate jobs that are booting more than on VM at the same time. Octavia is heavily affected. I've pushed a reproducer #link https://review.opendev.org/c/openstack/placement/+/806730 but I don't know how to fix the actual bug.
16:08:40 <gibi> this is the one bauzas mentioned above ^^
16:09:30 <bauzas> if the race isn't occurring a lot of times, we should maybe just call the transaction again
16:09:37 <gibi> in short if two VM for the same project is scheduled at the same time then the PUT /allocations call could cause a race in the project creation in placement
16:10:16 <gibi> bauzas: that was my first idea, but looking the code I'm not sure about the amount of surgery needed for this
16:10:33 <dansmith> it must be the *first* two vms right?
16:10:34 <gibi> if somebody has deeper sqlAlchemy knowledge than me then help is appreciated
16:10:45 <bauzas> dansmith: it *should* be yeah
16:10:47 <gibi> dansmith: first two VMs for a project yes
16:10:55 <sean-k-mooney> gibi: sorry im a bit confusted by the term project createion in a placment context
16:10:59 <dansmith> that definitely seems uncool
16:11:15 <bauzas> I guess we could hit the problem often with multicreate
16:11:30 <gibi> sean-k-mooney: placement stores the project_id and user_id of each consumer so that usages can be aggregated per project
16:11:30 <bauzas> ie. server create --max 2
16:11:38 <sean-k-mooney> why does placement need to be project wawre beyond storigng the project/user id in the allocation?
16:11:53 <bauzas> sean-k-mooney: because we use it from an API query
16:12:01 <dansmith> sean-k-mooney: it's relational so it has a table for it I think
16:12:07 <sean-k-mooney> yes im aware it there for usage
16:12:11 <bauzas> you can filter per project
16:12:17 <gibi> https://docs.openstack.org/api-ref/placement/?expanded=list-usages-detail#list-usages
16:12:19 <sean-k-mooney> but im not sure what resouce in the db we would need to create that would race
16:12:21 <bauzas> this was my question earlier this EU morning time :)
16:12:34 <gibi> sean-k-mooney: it is the row in the projects table
16:12:35 <dansmith> sean-k-mooney: the first time that entry goes into the table for the project?
16:12:43 <gibi> yes ^^
16:12:48 <bauzas> correct
16:12:54 <sean-k-mooney> right im wondering why we need a proejct tbale at all
16:13:09 <dansmith> ...because it's relational :)
16:13:11 <gibi> sean-k-mooney: for the /usages?project_id= query
16:13:12 <sean-k-mooney> i assuem its just for forine key constriants
16:13:28 <sean-k-mooney> dansmith: right but we dont actuly do that in other dbs in openstack
16:13:41 <dansmith> we could create the project first in a separate transaction, which is easy to retry, but that's definitely surgery
16:13:47 <dansmith> sean-k-mooney: not because it's a good idea :)
16:14:10 <gibi> dansmith: actually I think that is what happened before consumer_types
16:14:47 <gibi> dansmith: project_id user_id and consumer creation was each in a separate transaction, then a single transaction created / updated the allocation
16:15:04 <dansmith> gibi: ah, so this is more fallout from that work?
16:15:09 <gibi> it seems so
16:15:16 <dansmith> okay
16:15:22 <gibi> dansmith: this changed, now the whole PUT /allocations is a single DB transaction
16:15:40 <dansmith> for something like the project record, it makes sense to do that in a separate transaction I think
16:15:44 <gibi> and when that transaction founds a duplicate project it dies even if we catch the exception from sqla
16:15:48 <sean-k-mooney> i wonder if we shoudl drop those tables in the future and just store the uuids direcly in the allcoation table
16:15:51 <dansmith> gibi: further explains the large-ness of the transaction being a problem I guess
16:16:00 <bauzas> dansmith: that's why I nearly consider this as a regression
16:16:13 <dansmith> bauzas: definitely sounds like a regression
16:16:19 <bauzas> we stepped into this mess in Xena
16:16:39 <opendevreview> Merged openstack/nova master: Parse extended resource request from the port data  https://review.opendev.org/c/openstack/nova/+/800085
16:16:39 <dansmith> sean-k-mooney: only if we want it to be slower and more bloated I think :)
16:17:15 <gibi> the reason this become a single transaction is that consumer_type can be updated during PUT /allocations and if that happens in a separate transaction then we leak the update
16:17:17 <sean-k-mooney> well right now we are going to have to do a select by uuid to get the id in the project/user tabel then join based on that to the allcoation
16:17:39 <bauzas> I have no idea because I haven't looked at code yet, but is it crazy to switch back to the 2-step transactional model we had before ?
16:17:50 <dansmith> gibi: but we're just recording projects for relational purposes, so even if the allocation put doesn't succeed, it's fine that we've created the project record right?
16:17:51 <sean-k-mooney> so im not sure it would be slow to just to a select againt the allcoation table directly with an index on the user_id/project_id columns
16:18:00 <bauzas> ie. get the projet or create it, first, then do the allocation update
16:18:10 <gibi> dansmith: yes, for project and user it is fine, for consumer_type it is not fine
16:18:14 <bauzas> into two transactions
16:18:16 <dansmith> gibi: right
16:19:03 <gibi> so we can move back to many transactions but then we need to figure out how to move the consumer_type update to the last big transaction
16:19:19 <dansmith> yeah, which is surgery for sure
16:19:36 <dansmith> was melwitt the one that wrote the consumer type patches?
16:19:40 <dansmith> maybe we should get her consult
16:19:51 <gibi> it was melwitt who updated it after cdent left
16:20:03 <dansmith> well, right
16:20:06 <gibi> so she knows more
16:20:16 <dansmith> $current_owner = 'melwitt' is what I meant :)
16:20:20 <gibi> yeah
16:20:44 <gibi> OK, I guess that is it for now about this bug I will ping melwitt about it and record this discussion inthe bug
16:20:48 <gibi> moving on
16:21:03 <gibi> is there any other nova bug (gate or not) we need to discuss?
16:21:20 <bauzas> did we put the hot potato into someone's hand who isn't there ? excellent outcome of this discussion :p
16:21:33 <opendevreview> Merged openstack/nova master: Transfer RequestLevelParams from ports to scheduling  https://review.opendev.org/c/openstack/nova/+/791506
16:21:39 <gibi> bauzas: I can still feel the responsibility until I can talk to melwitt
16:21:40 <dansmith> wasn't it in her hands to begin with? I think it's okay :P
16:21:53 * bauzas is happy to be eventually back, then :p
16:21:58 <gibi> :D
16:22:10 <gibi> I see there is no other bug to talk about great :D
16:22:12 <bauzas> moving on
16:22:21 <gibi> Placement periodic job status #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly
16:22:25 <gibi> they are greeen
16:22:44 <gibi> and last but not least be prepared that the gate will be slow this week due to Milestone 3
16:23:03 <gibi> which brings us to
16:23:03 <gibi> #topic Release Planning
16:23:08 <gibi> Release tracking etherpad #link https://etherpad.opendev.org/p/nova-xena-rc-potential
16:23:16 <gibi> Feature freeze is 3rd of Sept which is end of this week.
16:23:20 <gibi> Also we have Milestone 3 end of this week that marks the last release of our client libs. Release patches:
16:23:24 <gibi> novaclient https://review.opendev.org/c/openstack/releases/+/806592
16:23:27 <gibi> osc-placement https://review.opendev.org/c/openstack/releases/+/806580
16:23:45 <gibi> I'm not tracking any open novaclient patches
16:24:06 <gibi> but for osc-placement the     https://review.opendev.org/c/openstack/osc-placement/+/804458 consumer_types support client patch would be nice
16:24:42 <gibi> the client release should happen this week so we still have a bit of time
16:25:04 <bauzas> I can review it
16:25:04 <gibi> We are tracking the release TODOs in #link https://etherpad.opendev.org/p/nova-xena-rc-potential
16:25:34 <gibi> bauzas: thanks, I'm also plannig to get to it
16:25:51 <gibi> there are two open todos for the release
16:25:56 <gibi> We need to produce the cycle highlights
16:26:00 <gibi> review is there: #link https://review.opendev.org/c/openstack/releases/+/800755
16:26:04 <gibi> I will do a bp status cleanup tomorrow and along that I will update the highlights patch. But you can help me with the highlight by bringing up features in the comments.
16:26:13 <opendevreview> Stephen Finucane proposed openstack/nova master: api: Add support for 'hostname' parameter  https://review.opendev.org/c/openstack/nova/+/778550
16:26:13 <opendevreview> Stephen Finucane proposed openstack/nova master: policy: Deprecate field from 'os-extended-server-attributes' policy  https://review.opendev.org/c/openstack/nova/+/806131
16:26:14 <opendevreview> Stephen Finucane proposed openstack/nova master: tests: Speed up 'servers' API tests  https://review.opendev.org/c/openstack/nova/+/778732
16:26:14 <opendevreview> Stephen Finucane proposed openstack/nova master: tests: Address nits for configurable-instance-hostnames series  https://review.opendev.org/c/openstack/nova/+/806735
16:26:44 <gibi> the other todo is
16:26:45 <gibi> We need a volunteer to write the release notes prelude (deadline for the prelude is RC1)
16:27:04 <dansmith> gosh I wonder who will do that
16:27:19 * gibi turning the general direction of France :)
16:27:28 * gibi turning to the general direction of France :)
16:27:44 <bauzas> hmmm
16:27:56 * bauzas wonders who gibi is looking at
16:28:01 <gibi> :D
16:28:08 <bauzas> but ok, I can volunteer for it
16:28:12 <gibi> thanks
16:28:55 <gibi> anything else about the release before we jump to open bps?
16:29:41 <gibi> #topic Review priorities
16:30:06 <gibi> this is a new topic as the gerrit patch enabling the new Review-Priority label has been landed
16:30:47 <gibi> you can query the label with something like https://review.opendev.org/q/project:openstack/nova+label:Review-Priority%252B1
16:31:15 <gibi> the doc about the label is here https://review.opendev.org/c/openstack/nova/+/792357
16:31:53 <gibi> anyhow as we are close the FF sean-k-mooney collected couple of bps from our list that we quickl agree on to mark as priority
16:32:03 <gibi> instance host name https://review.opendev.org/q/topic:%22bp%252Fconfigurable-instance-hostnames%22+(status:open%20OR%20status:merged)
16:32:14 <gibi> this is basically approved
16:32:17 <gibi> as far as I see
16:32:32 <bauzas> let me ask a stupid question
16:32:41 <bauzas> do we have a formal process on how to use this flag ?
16:32:47 <gmann> or we can run this to filter already +W one #link  https://review.opendev.org/q/project:openstack/nova+Review-Priority:1+label:Verified%253D1++NOT+label:Workflow%253C%253D-1
16:32:53 <bauzas> I remember some change I reviewed a while ago
16:32:56 <gibi> bauzas: this is where we discussing that https://review.opendev.org/c/openstack/nova/+/792357
16:33:01 <bauzas> but did we get into some direction ?
16:33:08 <sean-k-mooney> bauzas: yes
16:33:12 <gibi> I see you have negative comments there :)
16:33:31 <sean-k-mooney> well it was +w'd so i assuems we had agreed
16:33:38 <bauzas> I can see some consensus
16:33:46 <bauzas> so I can rereview this
16:34:09 <gibi> sure I have to back to that too
16:34:43 <gibi> lets then just quickly check some of the open bps that close to lend and ignore the new label until we agree on the doc
16:34:53 <gibi> s/lend/land
16:35:15 <gibi> so instance hostname is a basically done we can move on
16:35:20 <gibi> vhost multi queue flavor extra spec https://review.opendev.org/q/topic:%22bp%252Fmultiqueue-flavor-extra-spec%22+(status:open%20OR%20status:merged)
16:35:26 <sean-k-mooney> well i would prefer to use the new lable and adjust our useage if we need too in a follow up t be hosest but sure lets proced
16:36:01 <gibi> sean-k-mooney: I don't want to hold up the meeting agreeing on the usage as we have FF this week
16:36:19 <sean-k-mooney> gibi sure we can move on
16:36:19 <gibi> so the multique series is a simple one but still lack reviews
16:36:58 <sean-k-mooney> ya tl;dr that is supporting a mutlique via the flaovr in addtion to the existing image property
16:37:18 <bauzas> gibi: I was looking into the multiqueue series before the meeting
16:37:22 <sean-k-mooney> its pretty straigt forward but need reviews
16:37:32 <bauzas> so I could mark it with Review-Prio flag :p
16:37:38 <gibi> bauzas: you can :P
16:38:16 <gibi> if nobody else shows up I can read that too
16:38:21 <gibi> moving on
16:38:26 <gibi> generic mdevs https://review.opendev.org/q/topic:%22bp%252Fgeneric-mdevs%22+(status:open%20OR%20status:merged)
16:38:31 <gibi> as far as I see the code is approved
16:38:36 <gibi> the doc patch is open
16:38:45 <bauzas> thanks all
16:38:58 <gibi> we can land doc only patches after FF so this is in good shape
16:38:59 <bauzas> I can work on the doc change btw.
16:39:12 <bauzas> but my priority for this weeks is reviews.
16:39:21 <gibi> that sounds good to me
16:39:25 <gibi> moving on
16:39:30 <gibi> pps qos https://review.opendev.org/q/topic:%2522bp/qos-minimum-guaranteed-packet-rate%2522+(status:open+OR+status:merged)+project:openstack/nova
16:39:37 <gibi> this is easy as we don't have to rush
16:39:57 <gibi> the neturon API extension enabling the new logic probably wont land in Xena
16:40:06 <gibi> (but already works on in my env)
16:40:39 <sean-k-mooney> ah ok so this is moving to yoga then?
16:40:46 <gibi> the neutron impact seems so
16:41:05 <gibi> the nova impact is good to land now but it wont be triggered if neutron does not have the new api extension
16:41:10 <gibi> so it is dead code
16:41:47 <sean-k-mooney> i mean if its dead code its also going to have little or no impact on people deploying xena but sure
16:41:54 <gibi> yes
16:42:08 <gibi> moving on
16:42:22 <gibi> melwitt updated the unified limits https://review.opendev.org/q/topic:%22bp%252Funified-limits-nova%22+(status:open%20OR%20status:merged) sries
16:42:50 <gibi> dansmith: you had comments before. Will you have time this week to look back to this?
16:42:50 <dansmith> I saw her note about not reaching into limit internals but haven't gone back to look
16:43:14 <dansmith> I will, but are we really expecting to land this? if we are, then I'll try to prioritize, but if not, I'd focus on other things that are time-critical
16:43:30 <dansmith> seems like there are a lot of patches yet to land for thursday,
16:43:45 <dansmith> and I know some have had some review, but I don't think all (but could be wrong)
16:43:53 <bauzas> agreed with dansmith
16:44:06 <bauzas> I'd love this series to merge, but this is very late in time
16:45:33 <gibi> dansmith: it was open for a long time so I guess it is not that time critical
16:46:27 <dansmith> lots of odd gate fail on the bottom few, which I'm sure is not related to those patches, but just in terms of likelihood for merging all of that by thursday seems unlikely
16:46:33 <gibi> I don't see that much thing to close to land in nova but I guess you have commitments outside of nova too
16:46:53 <dansmith> yeah I meant other stuff
16:47:04 <dansmith> we should recheck the bottom three that are +W at least I guess
16:47:21 <gibi> yepp
16:47:40 <dansmith> the API ones at the top of the set are some I want to look closer at, and I haven't looked at the tempest tests that are available yet
16:48:08 <gibi> ack. if you not get to it the it will be moved to yoga, no hard feelings :)
16:48:21 <gibi> moving on
16:48:29 <gibi> here are the rest of the open bps https://launchpad.net/nova/+milestone/xena-3
16:48:45 <gibi> (some of the should be closed which is will do tomorrow)_
16:49:02 <gibi> is there any of those somebody want to get attention?
16:49:31 <gmann> remove tenant-id is still not in shape for Xena, we are still need all code up before we start merging first patch
16:49:46 <gmann> may be for Yoga
16:49:56 <gibi> gmann: thanks for the info. that is also a long time open series.
16:50:05 <gmann> yeah.
16:50:30 <gmann> we are reviewing it on time but just all the code should be up before we start merging it
16:50:54 <gibi> yeah, that is a big change in a single microversion.
16:51:01 <gibi> any other bp?
16:51:32 <gibi> the moving on
16:51:35 <gibi> #topic PTG Planning
16:51:39 <gibi> every info is in the PTG etherpad #link https://etherpad.opendev.org/p/nova-yoga-ptg
16:51:45 <gibi> please add topics ^^
16:51:46 <gibi> If you see a need for a specific cross project section then please let me know
16:51:58 <gibi> any question about the PTG?
16:52:22 <gibi> #topic Stable Branches
16:52:25 <gibi> periodic-stable jobs are not failing
16:52:31 <gibi> every stable gate seem to be working and unblocked
16:52:41 <gibi> EOM( elodilles )
16:52:49 <gibi> anything else about stable?
16:53:15 <gibi> #topic Sub/related team Highlights
16:53:16 <elodilles> nothing exciting compared to FF :X
16:53:20 <gibi> Libvirt (bauzas)
16:53:24 <gibi> elodilles: :)
16:53:51 <bauzas> well, I pledge guilty, I was off for 3 weeks
16:53:58 <gibi> good for you :)
16:54:14 <gibi> #topic Open discussion
16:54:19 <gibi> (gibi) PTL election: as you probably saw from Yoga we will have a new PTL, bauzas. I'm not going anywhere and I will help to make the transition smooth.
16:54:35 <gibi> welcome bauzas :)
16:54:51 <bauzas> thanks, just hoping I won't get a revolution, as in my country
16:55:14 <bauzas> don't cut my head, please.
16:55:28 <gibi> dont put your head near to sharp object :D
16:55:29 <sean-k-mooney> hehe. all hail the new hearder of cat :P
16:55:55 * gibi need more cats
16:56:07 <gibi> anything else for today?
16:56:08 <bauzas> fun fact, I have no cats but a dog
16:56:43 <bauzas> (and 2 kids, but apparently we can't count them as animals)
16:57:55 <gmann> bauzas: :)
16:57:55 <gibi> would you like to? :)
16:58:31 <gibi> if nothing else then lets close this meeting
16:58:32 <bauzas> well, they bark, always ask for food and sleep
16:58:36 <gibi> :D
16:58:45 <dansmith> are we done here? :)
16:58:50 <bauzas> I guess
16:58:50 <gibi> #endmeeting