16:00:05 <gibi> #startmeeting nova
16:00:06 <openstack> Meeting started Thu Feb 11 16:00:05 2021 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:09 <openstack> The meeting name has been set to 'nova'
16:00:34 <gibi> o/
16:00:40 <gmann> o/
16:00:52 <artom> ~o~
16:01:13 <lyarwood> \o
16:01:55 <elod> o/
16:02:28 <gibi> let's get started
16:02:41 <gibi> #topic Bugs (stuck/critical)
16:02:44 <gibi> no critical bugs
16:02:49 <gibi> #link 13 new untriaged bugs (+1 since the last meeting): #link https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New
16:03:03 <stephenfin> o/ (in another meeting though)
16:03:21 <gibi> is there any specific bug we need to talk about?
16:04:11 <gibi> #topic Gate status
16:04:18 <gibi> gate feels OK to me
16:04:31 <gibi> I know gmann fixed on of the outstanding gate failure recently
16:04:36 <gibi> *one
16:04:59 <gmann> gibi: do we have e-r query for that? if not i can add
16:05:06 <gmann> to see if there is any more tests need fixes
16:05:08 <gibi> gmann: I have to check
16:05:15 <gmann> ok i can check.
16:05:18 <gibi> thanks
16:05:32 <gibi> btw, there is an ongoing POST_FAILURE issue in Zuul that is being investigated by infra
16:05:49 * bauzas waves late, forgot the meeting
16:05:49 <gibi> hm, it was fixed apparently
16:06:26 <gibi> any specific gate failre you want to mention?
16:07:16 <gmann> also if you see tempest regex error
16:07:18 <gmann> #link https://review.opendev.org/c/openstack/tempest/+/774766
16:07:25 <gmann> ^^ that is fixed so you can recheck
16:07:34 <gmann> --exlude typo
16:08:00 <gibi> thanks gmann
16:08:33 <gibi> #topic Runway status
16:08:39 <gibi> #link https://etherpad.opendev.org/p/nova-runways-wallaby
16:08:45 <gibi> #link https://blueprints.launchpad.net/nova/+spec/nova-support-webvnc-with-password-anthentication : has negative feedback to work with
16:08:51 <gibi> #link https://blueprints.launchpad.net/nova/+spec/compact-db-migrations-wallaby : the nova db patches approved the nova_api db patches needs review
16:09:00 <gibi> #link https://blueprints.launchpad.net/nova/+spec/modernize-os-hypervisors-api : the api code landed, the python-novaclient patch and the policy patch needs some work
16:09:25 <gibi> #link https://blueprints.launchpad.net/nova/+spec/support-interface-attach-with-qos-ports the last necessary patch got feedback and now updated, ready to review
16:09:39 * kashyap waves
16:09:41 <gibi> any specific feature we need to talk about
16:09:43 <gibi> ?
16:09:56 <lyarwood> #link https://blueprints.launchpad.net/nova/+spec/libvirt-default-machine-type - I've just added this to the queue btw
16:11:32 <gibi> lyarwood: ack
16:11:47 <gibi> is there a prime candidate who has time to review that?
16:12:19 <stephenfin> I can
16:12:43 <lyarwood> stephenfin: thanks, just a reminder that I'm AFK on Friday's but I'll address any feedback first thing Monday
16:12:43 <gibi> cool, I will try to get there as a second reviewer somewhere mid next week, but early next week I will be busy with internal conference preparation
16:12:53 * kashyap can also look at it from some of the libvirt-related PoV ...
16:13:24 <lyarwood> thanks both
16:13:56 <kashyap> lyarwood: Thank _you_ for taking on the work ... I was supposed to do some of it, and couldn't
16:14:38 <gibi> any other feature that needs attention?
16:15:03 <stephenfin> just the DB stuff
16:15:10 <artom> I guess here or in open discussion but... there's a bunch of pre-requisite os-traits patches up
16:15:22 <stephenfin> the API DB ones are ready for review now
16:15:31 <artom> And I'm told we can't just Depends-on: for those, so we need to land them and release
16:15:39 <stephenfin> I already pinged Dan about them
16:15:49 <gibi> stephenfin: ack, it is on my radar
16:15:55 <stephenfin> artom: Yeah, I reviewed most/all of those last night
16:15:59 <stephenfin> just need another +"
16:16:00 <stephenfin> *2
16:16:03 <artom> stephenfin, yep, so needs the +A
16:16:06 <gibi> artom: which feature depends on the os-traits release?
16:16:18 <artom> gibi, yes ;)
16:16:36 <artom> So there's secure boot, VDPA, ephemeral encryption...
16:16:40 <gibi> I see
16:16:42 <artom> My socket policy thing
16:16:52 <gibi> thanks
16:17:05 <gibi> I will try to hit it this week
16:17:33 <gibi> anything else?
16:18:23 <gibi> #topic Release Planning
16:18:27 <gibi> Feature Freeze is at 11th of March, in 4 weeks from now
16:18:41 <gibi> let's hurry up landing features :)
16:18:58 <gibi> anything else about the coming release?
16:19:52 <gibi> #topic Stable Branches
16:19:58 <gibi> Rocky (and might be older branches too) is blocked by issue (tempest-slow job): https://bugs.launchpad.net/neutron/+bug/1914037
16:20:00 <openstack> Launchpad bug 1914037 in tempest "scenario tests tempest.scenario.test_network_v6.TestGettingAddress fails" [Medium,Triaged] - Assigned to Hemanth Nakkina (hemanth-n)
16:20:04 <gibi> newer branches seem OK
16:20:38 <gibi> EOM
16:20:43 <gibi> thank elod
16:20:59 <gibi> any other news from stable?
16:21:04 <lyarwood> Nope
16:21:11 <elod> np, I'll try to review the fix
16:21:13 <gmann> yeah we are trying to fix that in https://review.opendev.org/c/openstack/tempest/+/774764
16:21:24 <gmann> and testing nova patch #link https://review.opendev.org/c/openstack/nova/+/775003
16:21:36 <gmann> it did not tested due to how zuul pick the job definition
16:21:44 <gmann> which is fixed now and should work.
16:22:20 <gibi> gmann: thanks
16:22:21 <gibi> Now the nova-stable-maint is part of placement-stable-maint group so all our stable love can spread to placement too
16:22:22 <elod> gmann: thanks, looks promising \o/
16:23:01 <gibi> moving on
16:23:01 <lyarwood> gibi: ah that reminds me
16:23:07 <gibi> lyarwood: yes
16:23:21 <lyarwood> gibi: sorry, quick note on placement, stable/victoria was blocked but I didn't have time to look into why
16:23:49 <lyarwood> gibi: I was trying to land the .gitreview changes to actually open up the branch
16:24:04 <lyarwood> https://review.opendev.org/c/openstack/placement/+/754671 for example
16:24:14 <elod> lyarwood: I'll have a look
16:24:18 <lyarwood> thanks
16:24:20 <gmann> lyarwood: pyflakes rror
16:24:21 <gmann> error
16:24:37 <lyarwood> yeah I assumed it would be something lc related
16:24:38 <gmann> pyflakes version conflict
16:24:45 <lyarwood> I've just not had time
16:25:05 <lyarwood> if you and elod could look that would be great, I'll help with reviews once I'm back on Monday
16:25:08 <gibi> hacking needs to be bumped I guess
16:25:11 <gmann> sure
16:25:14 <lyarwood> thanks
16:25:24 <gibi> lyarwood: thanks
16:25:37 <gibi> #topic Sub/related team Highlights
16:25:40 <elod> I'll try to take care of the conflicts there :)
16:25:49 <gibi> elod: cool, thaks
16:25:52 <gibi> thanks even
16:26:05 <gibi> Libvirt (bauzas)
16:26:17 <bauzas> .
16:26:23 <bauzas> (that's it ;) )
16:26:26 <gibi> OK
16:26:30 <gibi> #topic Open discussion
16:26:38 <gibi> we have couple of topics on the agenda
16:26:45 <gibi> (kashyap; 05-FEB-2021) Late blueprint-approval request: https://blueprints.launchpad.net/nova/+spec/allow-disabling-cpu-flags
16:26:51 <gibi> I realize this is late in the cycle, but this really helps alleviate potential live migration problems on some Intel hardware during upgrades and updates. This is technically a simple feature; but can also be considered a "bug fix" that unblocks live migration in some scenarios.
16:26:51 <kashyap> Yep...
16:26:56 <gibi> Main Benefit: The ability to selectively disable CPU features for a guest means: newly launched VMs on a compute node can now disable offending guest CPU flags that block live migration. This facilitates live migration to a host with TSX=off.
16:27:00 <gibi> Example: Today, a VM running on a compute node with Intel TSX=on (which is the default on all Linux kernels until v.5.11) cannot be migrated to a node with Intel TSX=off. But, the ability to selectively disable CPU flags alleviates this (and similar problems) — you can now keep TSX enabled on a host, and yet block it for the guest via `cpu_model_extra_flags`. This unblocks live-migrating the
16:27:06 <gibi> said guest to a host with TSX=off.
16:27:09 <gibi> EOM
16:27:11 <gibi> Notes: On relevant Intel processors, TSX is suggested to be disabled as it can be a potential security problem. TSX is disabled by default upstream Linux v.5.11 (Oct-2019)
16:27:32 <kashyap> gibi: So ... as a cherry on top; the last two hours I've done some tests
16:27:49 <gibi> So I said before that if nobody objects and the implementation patch is ready then I'm willing to approve the bp late and right at the moment +2 the impl patch as well
16:28:00 <ganso> gibi: o/ I have a topic for later when all others are done. Sorry I didn't put it in the agenda
16:28:11 <gibi> ganso: ack, I will ping you
16:28:26 <gibi> kashyap: does the test promising?
16:28:32 <kashyap> Yes!
16:28:38 <gibi> awesome
16:28:38 <kashyap> Let me get the evidence quickly :)
16:28:52 <kashyap> gibi: Here (for later): https://kashyapc.fedorapeople.org/CPU_flags_Nova_tests.txt
16:29:11 <kashyap> I've done thre tests w/ three different Nova [libvirt]cpu_* configurations
16:29:28 <kashyap> And all three yield expected results.  I'm just checking some more; and I'll post the evidence in the review for the record
16:29:54 <gibi> kashyap: thanks
16:29:59 <gibi> it is convincing
16:30:01 <kashyap> gibi: So, if you have a quick look there; the enabled CPU flag shows up in the guest; and the disabled ones don't.
16:30:19 <kashyap> I also want to test on a different Intel host (the problematic ones), and then summarize the results.
16:30:54 <gibi> btw, this is the implementation patch #link https://review.opendev.org/c/openstack/nova/+/774240
16:31:20 <kashyap> Yeah
16:31:39 <kashyap> gibi: A small observation, though, on the XML bits:
16:33:11 <kashyap> As expected, the disabled flags don't show up for the guest.  But I'm wondering if it should also show up as "disable" in the guest XML, e.g.
16:33:14 <kashyap> <feature policy='disable' name='flag1'/>
16:33:17 <kashyap> <feature policy='disable' name='flag2'/>
16:34:07 <kashyap> gibi: As of now, the functionality is as expected: if you tell Nova to disable a flag; it will not give it to the guest.  But if you tell it to explicitly enable, or give neither '+' nor '-', it enables it.
16:34:11 <kashyap> All expected behaviour.
16:34:41 <gibi> kashyap: let's take this in the review
16:34:46 <gibi> so others in this room, is there any objection to late approve the above bp and then quickly rewiew the small implementation patch?
16:34:48 <kashyap> Anyway, I don't want to ramble on about the feature here.
16:34:49 <kashyap> Yep, sorry
16:35:44 * kashyap thinks he made others zone out :D
16:36:44 * gibi fetches his PTL whip
16:36:51 <kashyap> Hehe
16:37:22 * bauzas turns around and doesn't look
16:37:23 <kashyap> stephenfin: lyarwood: or anyone else --^  Any objections, rotten tomatoes, snide remarks?
16:37:46 <stephenfin> nope
16:37:58 <lyarwood> none from me at the moment, but there's time ;)
16:38:05 * lyarwood will review on Monday
16:38:11 <kashyap> No problem
16:38:26 <gibi> OK, I consider this as sold. I will late approve the bp and we will to a proper review on the impl patch
16:38:32 <gibi> moving on
16:38:42 <gibi> (stephenfin) https://review.opendev.org/c/openstack/nova/+/772271 is stuck
16:38:45 <kashyap> gibi: Yeap, thank you!
16:38:47 <gibi> elod is concerned about the backportability of this, as it has user-facing impacts. As noted by lyarwood though, the previous behavior was wrong
16:39:04 <gibi> EOM
16:39:22 <gibi> elod: how strongly you object :)
16:39:25 <gibi> ?
16:40:04 <elod> well, like my last comment there :D
16:40:14 <dansmith> I tend to agree with elod, but haven't fully digested the change
16:40:52 <dansmith> broken forever behavior right? if it's not a regression, then it's less clear that it _needs_ to go back, and since it's a fairly substantial change in behavior, I'd generally rather not
16:41:20 <lyarwood> yeah broken forever
16:42:14 <stephenfin> yeah, broken forever, but use of designate means you'll likely hit it sooner rather than later
16:42:43 <dansmith> I'll review it more in a bit and vote, but probably -1
16:43:28 <gibi> with possible two -1s I consider it as not approved for inclusion
16:43:51 <lyarwood> well -1's from stable cores but yeah I agree
16:44:03 <lyarwood> lets wait for dansmith to review it in full and we can go from there
16:44:10 <gibi> lyarwood: OK
16:44:11 <stephenfin> yeah, seems reasonable
16:44:18 <gibi> moving on
16:44:20 <gibi> (stephenfin) Outreachy projects?
16:44:26 <gibi> I'm already helping mentor some NDSU students over in OSC/SDK land w/ diablo_rojo and gtema and could probably work with someone else. Do we have any nice, self-contained items that we'd like to do but just haven't had time for though?
16:44:33 * gibi thinks of things like PCI in placement (would need hardware though)
16:44:39 <gibi> no me stephenfin :D
16:44:45 <gibi> http://lists.openstack.org/pipermail/openstack-discuss/2021-February/020288.html
16:44:48 <gibi> EOM
16:45:31 <stephenfin> so per $summary
16:45:51 <stephenfin> ideally it should be something useful enough that it will be reviewed, but not so important that it'll be an issue if it isn't done
16:46:27 <lyarwood> shared storage in placement?
16:46:40 <lyarwood> that's huge actually, ignore me
16:46:57 <gibi> PCI also not well understood at least not for me without digging up notes
16:47:09 <stephenfin> fair point
16:47:49 <gibi> mypy could be something that is small, but we don't have consensus on the usefullness of it
16:48:03 <dansmith> yeah, don't put them in the middle of THAT :)
16:48:08 <gibi> :)
16:48:39 <gibi> somebody should fix the gerrit -> launchpad integration, that would be very usefull for me
16:48:46 <dansmith> ++ :)
16:49:03 <gibi> that would not be release critical so can be done slowly
16:49:12 <gibi> I just don't know if somebody already started it
16:49:18 <gibi> and it not nova specific
16:49:58 <artom> And the job results display
16:50:10 <gibi> stephenfin: do you need ideas or do you need a volunteering mentor?
16:50:17 <bauzas> we can open an ethercan of worms
16:50:22 <artom> The current greasemonkey script is buggy
16:50:22 <stephenfin> I can check with infra
16:50:33 <stephenfin> gibi: Ideas. I'm okay with mentoring
16:50:38 <kashyap> gibi: stephenfin: Yeah, for Outreachy ... FWIW, anything hardware-specific would be too much for a novice student
16:51:02 <kashyap> Something that can be done in VMs / et al, with bite-sized-tasks would be nice
16:51:04 <stephenfin> We don't need to figure them out now. Mostly raising it to the front of peoples' minds
16:51:06 * kashyap stops giving unsolicited advice
16:51:29 <kashyap> stephenfin: Is OpenStack accepted to Outreachy this cycle?
16:51:30 <stephenfin> If anyone does have additional ideas, lemme know and I'll chat with Kendall about them
16:51:43 <gibi> stephenfin: thanks
16:51:45 <stephenfin> kashyap: yup, it seems so (see the ML link)
16:51:51 <kashyap> Nice
16:52:06 <gibi> moving on
16:52:09 <gibi> ganso: your turn
16:52:14 <ganso> gibi: thanks!
16:52:28 <ganso> hi everyone! I'd like to revisit this https://bugs.launchpad.net/nova/+bug/1821755
16:52:30 <openstack> Launchpad bug 1821755 in OpenStack Compute (nova) "live migration break the anti-affinity policy of server group simultaneously" [Medium,In progress] - Assigned to Boxiang Zhu (bxzhu-5355)
16:52:55 <ganso> 2 approaches have been suggested, 1 long term ideal solution using placement
16:53:07 <ganso> and 1 short term approach, which seems to be https://review.opendev.org/c/openstack/nova/+/651969/
16:53:21 <ganso> that short term approach seems like it just mitigates the problem and it is still racy
16:53:48 <ganso> so I'm looking at the long term approach. Considering the complexity of integrating all the moving parts, I'd assume it will require a spec
16:54:13 <ganso> because it sounds like it will involve deprecating the affinity and anti-affinity filters, in favor of having this functionality in placement
16:54:55 <ganso> I'd like to know with everyone agrees that this is the correct direction, if this is something worth working into (as it will require review effort from you folks)
16:55:04 <ganso> s/know with/know if
16:55:24 <gibi> ganso: modelling affinity in placement definetly needs a spec
16:55:46 <gibi> ganso: I don't recall if we had any stab at it previously
16:56:19 <gibi> instances are in placement as consumers so locality can be checked
16:57:10 <ganso> I was thinking about having a placement property or something to map the affinity and anti-affinity to the instances, like if it was a resource
16:57:48 <ganso> anyway, those are details that can be sorted out in the spec and in future meetings
16:58:14 <gibi> ganso: the problem is that allocation candidate query does not have a way to say I don't want to be next to a consumer
16:58:35 <gibi> so this probably needs placement api work
16:59:19 <gibi> anyhow I suggest you to look at existing placement pre filters in nova
16:59:23 <gibi> as a starting point
16:59:30 <ganso> gibi: thanks!
16:59:42 <gibi> is there anything else for today?
16:59:44 <ganso> I will start playing aroudn with this and working on the spec
16:59:54 <gibi> ganso:cool, ping me if you have questions
17:00:02 <ganso> gibi: will do =)
17:00:06 <gibi> we are out of time
17:00:09 <gibi> thanks for the meeting
17:00:11 <gibi> we covered a lot
17:00:16 <gibi> #endmeeting