16:00:51 <bauzas> #startmeeting nova
16:00:51 <opendevmeet> Meeting started Tue Nov  2 16:00:51 2021 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:51 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:51 <opendevmeet> The meeting name has been set to 'nova'
16:01:07 <bauzas> #link https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
16:01:16 <bauzas> good day, everyone
16:01:19 <dansmith> o/
16:01:27 <sean-k-mooney[m]> o/
16:01:59 <gibi> \o
16:02:02 <bauzas> as discussed before, I will exceptionnally be only able to chair this meeting for 30 mins
16:02:12 <bauzas> so I'll let gibi co-chair
16:02:15 <bauzas> #chair gibi
16:02:15 <opendevmeet> Current chairs: bauzas gibi
16:02:29 * gibi accepts the challenge
16:02:39 <bauzas> let's start while people join
16:02:46 <bauzas> #topic Bugs (stuck/critical)
16:02:50 <bauzas> No Critical bug
16:03:13 <bauzas> #link 20 new untriaged bugs (+0 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New
16:03:31 <bauzas> thanks to anybody who triaged a few of them
16:04:11 <bauzas> #help any help is appreciated with bug triage and we have a how-to https://wiki.openstack.org/wiki/Nova/BugTriage#Tags
16:04:35 <bauzas> 32 open stories (+0 since the last meeting) in Storyboard for Placement #link https://storyboard.openstack.org/#!/project/openstack/placement
16:05:20 <bauzas> so, maybe we closed one or more stories in Storyboard, but I don't think so
16:06:00 <bauzas> yeah, last story was written on Oct 21st
16:06:19 <bauzas> any bug to discuss in particular ?
16:07:00 <bauzas> ok, I guess no, moving on
16:07:05 <bauzas> #topic Gate status
16:07:11 <bauzas> Nova gate bugs #link https://bugs.launchpad.net/nova/+bugs?field.tag=gate-failure
16:07:25 <bauzas> nothing new
16:07:33 <bauzas> Placement periodic job status #link https://zuul.openstack.org/builds?project=openstack%2Fplacement&pipeline=periodic-weekly
16:07:45 <bauzas> no issues so far ^
16:08:01 <bauzas> just the usual reminder,
16:08:03 <bauzas> Please look at the gate failures, file a bug, and add an elastic-recheck  signature in the opendev/elastic-recheck repo (example: #link https://review.opendev.org/#/c/759967)
16:08:13 <bauzas> that's it for gate status
16:08:15 <gibi> this is a gate fix that needs a second core https://review.opendev.org/c/openstack/nova/+/814036
16:08:20 <bauzas> oh right
16:08:44 <bauzas> gibi: I'll look at it while we speak
16:09:12 <gibi> thanks
16:09:26 <bauzas> (already looked at it, but need one last glance)
16:09:56 <bauzas> moving on or any gate failure to mention besides the above one ?
16:10:54 <bauzas> #topic Release Planning
16:10:58 <bauzas> Yoga-1 is due Nova 18th #link https://releases.openstack.org/yoga/schedule.html#y-1
16:11:10 <bauzas> (3 weeks from now)
16:11:20 <bauzas> err, 2 weeks
16:11:24 <bauzas> +2d
16:11:37 <bauzas> which means, typey typey your specs
16:12:25 <bauzas> https://review.opendev.org/q/project:openstack/nova-specs+is:open is not that large
16:12:48 <bauzas> what makes me tell :
16:12:56 <bauzas> #startvote Spec review day proposal on Tuesday Nova 16th ? (yes, no)
16:13:22 <gibi> yes
16:13:26 <gibi> #yes
16:13:30 <gibi> (how to vote?)
16:13:30 <bauzas> dang, the meetbot doesn't tell what to say
16:13:39 <bauzas> #vote yes
16:13:43 <dansmith> you have a space in front
16:13:43 <gibi> #vote yes
16:13:47 <dansmith> of the startvote
16:13:48 <bauzas> #startvote Spec review day proposal on Tuesday Nova 16th ? (yes, no)
16:13:48 <opendevmeet> Begin voting on: Spec review day proposal on Tuesday Nova 16th ? Valid vote options are , yes, no, .
16:13:48 <opendevmeet> Vote using '#vote OPTION'. Only your last vote counts.
16:13:54 <bauzas> dansmith: huzzah
16:13:54 <gibi> #vote yes
16:13:54 <opendevmeet> gibi: yes  is not a valid option. Valid options are , yes, no, .
16:14:02 <sean-k-mooney[m]> yes
16:14:04 <gibi> #vote yes, !
16:14:04 <opendevmeet> gibi: yes, !  is not a valid option. Valid options are , yes, no, .
16:14:06 <gibi> #vote yes,
16:14:06 <opendevmeet> gibi: yes, is not a valid option. Valid options are , yes, no, .
16:14:09 <dansmith> lol
16:14:13 <bauzas> oh man
16:14:21 <bauzas> this is an absolutule fail.
16:14:22 <dansmith> #vote yes,
16:14:22 <opendevmeet> dansmith: yes, is not a valid option. Valid options are , yes, no, .
16:14:31 <dansmith> #vote yes, no
16:14:31 <opendevmeet> dansmith: yes, no is not a valid option. Valid options are , yes, no, .
16:14:33 <gibi> #vote ,
16:14:33 <opendevmeet> gibi: , is not a valid option. Valid options are , yes, no, .
16:14:38 <dansmith> #vote  yes, no
16:14:38 <opendevmeet> dansmith: yes, no is not a valid option. Valid options are , yes, no, .
16:14:41 <sean-k-mooney[m]> lets just assume we are ok with the 16th and move on
16:14:43 * bauzas facepalms
16:14:44 <dansmith> #vote  yes, no,
16:14:44 <opendevmeet> dansmith: yes, no, is not a valid option. Valid options are , yes, no, .
16:14:45 <yuriys> this is bot abuse!
16:14:53 <bauzas> #endvote
16:14:53 <opendevmeet> Voted on "Spec review day proposal on Tuesday Nova 16th ?" Results are
16:14:53 <dansmith> #vote meh
16:14:55 <gibi> thet is fun
16:15:23 <bauzas> ok I guess this was epic but we should leave this bot quiet back for 5 years
16:15:38 <bauzas> anyway,
16:15:51 <bauzas> #agreed Spec review day happening on Nov 16th
16:15:56 <bauzas> voilà
16:15:59 <bauzas> moving on
16:16:03 <bauzas> #topic Review priorities
16:16:15 <bauzas> #link  https://review.opendev.org/q/status:open+(project:openstack/nova+OR+project:openstack/placement)+label:Review-Priority%252B1
16:16:16 <dansmith> also leading space
16:16:45 <bauzas> dansmith: good catch, the copy/paste makes me mad
16:16:50 <bauzas> #topic Review priorities
16:16:57 <bauzas> #undo
16:16:57 <opendevmeet> Removing item from minutes: #topic Review priorities
16:17:25 <bauzas> fun, the meetbot isn't telling new topics
16:17:33 <bauzas> anyway, next point
16:17:35 <dansmith> it doesn't on oftc I think
16:17:42 <bauzas> #action bauzas to propose a documentation change by this week as agreed on the PTG
16:17:47 <dansmith> but if you don't do it #properly it won't record them either
16:18:13 <bauzas> for adding a gerrit ACL to let contributors +1ing
16:18:31 <bauzas> didn't had time to formalize it yet
16:18:49 <bauzas> #topic Stable Branches
16:18:59 <bauzas> elodilles: floor is yours
16:20:02 <bauzas> I guess he's not around
16:20:05 <bauzas> so I'll paste
16:20:14 <bauzas> stein and older stable branches are blocked, needs the setuptools pinning patch to unblock: https://review.opendev.org/q/I26b2a14e0b91c0ab77299c3e4fbed5f7916fe8cf
16:20:37 <bauzas> we need a second stable core especially on https://review.opendev.org/c/openstack/nova/+/813451
16:21:00 <bauzas> Ussuri Extended Maintenance transition is scheduled to next week (Nov 12)
16:21:07 <bauzas> the list of open and unreleased patches: https://etherpad.opendev.org/p/nova-stable-ussuri-em
16:22:18 <bauzas> I guess we need to make a few efforts before ussuri becomes EM
16:22:31 <bauzas> elodilles: again, I offer my help if you ping me
16:22:44 <bauzas> patches that need one +2 on ussuri: https://review.opendev.org/q/project:openstack/nova+branch:stable/ussuri+is:open+label:Code-Review%253E%253D%252B2
16:23:08 <bauzas> (I'll skim this list)
16:23:22 <elodilles> oh, sorry, DST :S
16:23:24 <bauzas> last but not least: https://review.opendev.org/806629  patch (stable/train) needed 14 rechecks, I was pinged with the question  whether testing should be reduced in train to avoid this amount of  rechecks (mostly volume detach issue)
16:23:54 <bauzas> elodilles: hah, I warned about it in the channel :p
16:24:06 <sean-k-mooney[m]> are the detach issue due to the qemu version we have in bionic
16:24:28 <sean-k-mooney[m]> i assume train is not on focal?
16:24:36 <elodilles> yes, train is on bionic
16:24:50 <elodilles> (just like ussuri)
16:26:34 <bauzas> hmmm, technically, Train is EM
16:26:53 <sean-k-mooney[m]> im somewhat tempeted to same maybe move it to centos 8 or focal but we could disable the volume tests
16:26:59 <sean-k-mooney[m]> yes it is
16:27:11 <bauzas> I'd rather prefer us fixing the gate issues rather than reducing the test coverage, but this depends on any actions we can take
16:27:24 <bauzas> so, let's be pragmatic
16:28:17 <sean-k-mooney[m]> well the first question would be does train have gibis event based witing patch or is it still using the retry loop
16:28:39 <bauzas> gibi's patch isn't merged yet, right?
16:28:44 <bauzas> could it help ?
16:28:57 <sean-k-mooney[m]> the only options reallly to fi this are change the qemu verions or backport gibis patch
16:29:10 <bauzas> (I'll have to leave gibi chair in the next 2 mins but dansmith has a point I'm interested in)
16:29:37 <dansmith> I also have to go sooner
16:29:43 <bauzas> sean-k-mooney[m]: we can try to backport gibi's patch and see whether that helps
16:29:45 <dansmith> maybe we could swap open and libvirt?
16:29:52 <bauzas> dansmith: I'll
16:29:59 <gibi> I don't think there is anything in the libvirt topic
16:30:02 <gibi> lyarwood is out now
16:30:09 <dansmith> okay
16:30:21 <bauzas> okay, elodilles I'll propose to wait for gibi's patch to land in master and then be backported
16:30:32 <gibi> bauzas: it is backported til wallaby
16:30:33 <elodilles> bauzas: ack
16:30:34 <bauzas> and punt the decision to reduce the test coverage once we get better ideas
16:30:41 <gibi> if we are talking about https://review.opendev.org/q/topic:bug/1882521
16:31:09 <bauzas> gibi: then we need to backport it down to train
16:31:13 <gibi> I don't think it will be a piece of cake to bring that back train
16:31:20 <gibi> *to train
16:31:32 <bauzas> gibi: (apologies I confused with the vnic types waiting patch)
16:31:59 <bauzas> I have to leave, but can we hold this one discussion and go straight to dansmith's point
16:32:01 <bauzas> ?
16:32:09 <gibi> anyhow we can take that outside when lyarwood is back
16:32:09 <bauzas> so I and dansmith can leave
16:32:21 <gibi> lets go to that
16:32:25 <bauzas> #topic Sub/related team Highlights
16:32:27 <bauzas> nothing to tell
16:32:32 <bauzas> #topic Open discussion
16:32:37 <bauzas> Bring default overcommit ratios into sanity (dansmith / yuriys)
16:32:39 <dansmith> So, I think we all know the default 16x cpu overcommit default is insane
16:32:41 <yuriys> exciting
16:32:54 <dansmith> we've got reports that some operators are USING those defaults because they think we're recommending them
16:32:59 <sean-k-mooney[m]> yes it is
16:33:04 <yuriys> I am so used to Slack and Discord for drop a paragraph level of communication, so pardon all the incoming spam! I prewrote stuff.
16:33:04 <bauzas> hah
16:33:08 <dansmith> yuriys is here to offer guidance and help work on this,
16:33:21 <dansmith> but I think we need to move those defaults to something sane, both in code and update the docs
16:33:28 <bauzas> I guess this can be workload dependent, right?
16:33:29 <sean-k-mooney[m]> it basically should not be set over 4x
16:33:39 <yuriys> 4:1 for cpu , 1:1 for mem.
16:33:44 <sean-k-mooney[m]> yep
16:33:48 <bauzas> I think we started to document things based on workloads
16:33:56 <yuriys> yes
16:33:58 <dansmith> bauzas: it's completely workload dependent, but we should not be recommending really anything, and thus I think the default needs to be closer to 1:1 with docs saying why you may increase it (or not)
16:33:58 <bauzas> but we never achieved this
16:34:01 <yuriys> it's VERY use case specific
16:34:17 <sean-k-mooney[m]> it is
16:34:26 <bauzas> I'm referring to https://docs.openstack.org/nova/latest/user/feature-classification.html
16:34:28 <yuriys> Ideally the documentation is restructured on how to scale up these over commits to match the desired use case, density and performance and start at sane values/defaults. Engineers can then template out the necessary config values after they've gotten to know system capabilities. Instead of working backwards from chaos and mayhem, giving future admins the opportunity to reach desired state through scaling up from a stable syste
16:34:28 <yuriys> hould be the goal of arch design.
16:35:07 <dansmith> so can we agree that we'll change the defaults to something that seems reasonable, and modify the docs that just say "these are the defaults" to have big flashing warnings that defaults will never be universal in this case?
16:35:09 <sean-k-mooney[m]> the 16:1 number originally was assuming webhosting as the main usecase or similar workloads
16:35:13 <dansmith> specifically: https://docs.openstack.org/arch-design/design-compute/design-compute-overcommit.html
16:35:19 <sean-k-mooney[m]> that does not fit with how openstack is typicaly used
16:35:28 <bauzas> dansmith: this sounds a reasonable change
16:35:45 <dansmith> cool, specless bp or bug?
16:36:00 <bauzas> my only worries would go on how this is wired at the object level but we can take the opportunity to lift this
16:36:12 <sean-k-mooney[m]> 4:1 for cpu is the highist ratio i would general consider usable in production
16:36:32 <bauzas> dansmith: there are a few upgrade concerns with placement as IIRC this is set by the model itself
16:36:32 <dansmith> I think we already moved towards something better when we moved the defaults to placement right? but we need to do something
16:36:54 <sean-k-mooney[m]> well we have the inital allcoation ratios now
16:36:56 <yuriys> i said 4 to be reasonable haha, i think i've set about 3:1 with nova-scheduler and placement randomization elements.
16:36:59 <dansmith> bauzas: yeah I think placement now has explicit defaults right?
16:37:02 <bauzas> I'm pretty sure we have some default value in the placement DB that says "16"
16:37:03 <sean-k-mooney[m]> but it still default to 16
16:37:07 <dansmith> sean-k-mooney[m]: right
16:37:17 <sean-k-mooney[m]> i think we can decrease inial to 4 for cpu an 1 for memory
16:37:22 <dansmith> so I think we can just move those and reno that operators who took those defualts years ago should change them likely
16:37:39 <sean-k-mooney[m]> +1
16:37:39 <bauzas> I don't wanna go procedural
16:37:51 <bauzas> so a specless BP could work for me but,
16:37:57 <bauzas> we need renos
16:38:01 <dansmith> sean-k-mooney[m]: yeah I think 4:1 CPU and 1:1 memory is fine for a default, we might need up to up for devstack I guess but that's where insane defaults should be :)
16:38:03 <sean-k-mooney[m]> yep i was thinking the same
16:38:10 <bauzas> + we need to ensure we consider the DB impact before
16:38:15 <dansmith> cool, specless bp and renos.. sounds good
16:38:37 <bauzas> if that becomes debatable in the reviews, we could go drafting more
16:38:42 <sean-k-mooney[m]> bauzas:  i dont think there will be any
16:38:59 <bauzas> but here, we're talking of changing defaults, not changing existing deployments
16:39:00 <sean-k-mooney[m]> if we are just changing the initial values it wont affect existing RPs
16:39:02 <dansmith> yeah I think it'll be straightforward, but we can always revise the plan if needed
16:39:07 <dansmith> right
16:39:18 <bauzas> OK, looks to me we have a plan
16:39:24 <dansmith> #micdrop
16:39:40 <bauzas> #agreed changing overcommit CPU ratio to <16.0 can be a specless BP
16:39:49 <bauzas> yuriys: typey typey
16:40:16 <bauzas> and ping me on IRC once you have the Launchpad BP up so I can approve it
16:40:23 * bauzas needs to drop
16:40:29 <gibi> OK
16:40:37 <gibi> is there anything else for today?
16:40:43 <yuriys> no idea what that means, but sounds good?
16:41:03 <yuriys> ill just coordinate through dan i suppose
16:41:16 <gibi> yuriys: you need a file a blueprint here https://blueprints.launchpad.net/nova/
16:41:33 <gibi> so we can track the work
16:41:41 <dansmith> yuriys: with just an overview of what we said, no big deal
16:41:46 <gibi> yepp
16:42:08 <yuriys> Ah sounds good. Dang, I thought I was going to have like do a whole speech and everything
16:42:14 <yuriys> just to win over votes
16:42:15 <yuriys> haha
16:42:18 <gibi> :)
16:42:26 <dansmith> yuriys: I told you it wouldn't be a big deal :)
16:42:44 <gibi> it is not a bid deal if dansmith is on your side ;)
16:42:45 <yuriys> yeah, i think the expandability still needs to be part of that doc btw
16:42:50 <yuriys> for dollar reasons
16:43:00 <yuriys> but ill throw up a BP and we'll go from there
16:43:11 <gibi> cool
16:43:28 <gibi> is there anything else for today? I don't see other topics on the agenda
16:44:12 <gibi> it seems not
16:44:19 <gibi> so then I have the noble job to close the meeting:)
16:44:29 <gibi> thank you all for joining today
16:44:40 <gibi> #endmeeting