14:00:02 <gibi> #startmeeting nova
14:00:02 <openstack> Meeting started Thu Mar 21 14:00:02 2019 UTC and is due to finish in 60 minutes.  The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:04 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:06 <gibi> o/
14:00:07 <openstack> The meeting name has been set to 'nova'
14:00:14 <takashin> o/
14:00:20 <mriedem> o/
14:00:20 <efried> gibi: you want to run it?
14:00:30 <Kevin_Zheng> o/
14:00:34 <gibi> efried: I have to jump to a call at half past so you can take this
14:00:46 <gibi> #chair efried
14:00:46 <openstack> Current chairs: efried gibi
14:00:53 <efried> I done prepared and everything
14:01:02 <efried> Greetings all.
14:01:06 <gibi> efried: thanks
14:01:11 <efried> #link agenda https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting
14:01:25 <efried> updated like two minutes ago, refresh if needed
14:01:42 <efried> #topic Release News
14:01:54 <efried> #link Stein release schedule: https://wiki.openstack.org/wiki/Nova/Stein_Release_Schedule
14:01:54 <efried> #info Stein RC1 is TODAY, March 21
14:01:54 <efried> #link Stein RC potential changes tracking: https://etherpad.openstack.org/p/nova-stein-rc-potential
14:02:09 <efried> couple of things I wanted to bring up from the etherpad
14:02:13 <efried> Docs for bug 1821015
14:02:14 <openstack> bug 1821015 in OpenStack Compute (nova) "Attaching virtual GPU devices to guests in nova - libvirt reshaping" [Medium,Triaged] https://launchpad.net/bugs/1821015 - Assigned to Sylvain Bauza (sylvain-bauza)
14:02:24 * edleafe wanders in late
14:02:44 <efried> bauzas didn't sign in, was going to ask him about that ^ because it requires (in order to be "authentic") GPU hardware.
14:02:55 <mriedem> he said he's working on it
14:03:02 <efried> okay, cool, thanks.
14:03:10 <efried> #link Cycle highlights https://review.openstack.org/#/c/644697/
14:03:14 <bauzas> oops
14:03:20 <bauzas> I'm there
14:03:46 <efried> Please go have a look at that. I guess somebody on the releases team will decide when to merge it.
14:04:13 <efried> #link reno prelude https://review.openstack.org/#/c/644412/
14:05:06 <efried> This has several +2s. I think we want to +W it today. There's a complex web of deps, but I think the only one that's not yet merged is
14:05:06 <efried> #link network doc side of bw QoS https://review.openstack.org/#/c/640390
14:05:12 <mriedem> need to get https://review.openstack.org/#/c/640390 merged
14:05:22 <mriedem> yeah i asked haleyb to just merge that last night
14:05:26 <mriedem> can also ping mlavalle today
14:06:13 <haleyb> mriedem: sorry, didn't see your ping about that last night, but seeme bence was going to update it?
14:06:26 <mriedem> haleyb: https://review.openstack.org/#/c/645104/
14:06:27 <gibi> haleyb: bence put up a followup patch
14:06:28 <mriedem> he already followed up
14:06:50 <haleyb> mriedem: sorry, missed the reference, let me merge
14:07:07 <mriedem> thanks
14:07:13 <mriedem> efried: i've approved the nova prelude change
14:07:15 <efried> Miguel just did it.
14:07:21 <efried> Cool.
14:07:40 <efried> Any other stein-y stuff to bring up?
14:08:37 <efried> moving on
14:08:41 <mdbooth> efried: Yep
14:08:48 <efried> mdbooth: your mic
14:08:59 <efried> or were you agreeing to move on?
14:09:03 <mdbooth> I just posted to the ML about my monkey patching thing
14:09:29 <mdbooth> I realise how late in the day it is for Stein, but I think it could be worthy of inclusion
14:09:30 <efried> does this seem like kind of a dangerous thing to be trying to land right now?
14:09:54 <mdbooth> Right, but it's a regression which makes it impossible to deploy Nova Stein on RHEL 8
14:10:21 <sean-k-mooney> mdbooth: secifcally with tripleo
14:10:24 <mdbooth> Now downstream that's not a huge issue. As long as we land it upstream Real Soon we'll backport it
14:10:29 <mdbooth> sean-k-mooney: It has nothing to do with tripleo
14:10:43 <mdbooth> But it is a regression from Rocky.
14:11:11 <efried> #link monkey patch early https://review.openstack.org/#/c/626952/
14:11:27 <efried> I only vaguely understand what's going on there (monkey patch early and only once and skip certain monkey patches in certain environments and don't do it for sphinx).
14:12:01 <efried> anyone want to weigh in on relative risks/merits of landing right now?
14:12:06 <mdbooth> It's horrible, but without it nova compute doesn't even start on RHEL 8.
14:12:23 <mdbooth> And remember it's only horrible because it's changing something horrible.
14:12:50 <efried> that seems like it would warrant inclusion. dansmith, mriedem, opinions here?
14:12:55 <mriedem> it's a problem for rhel 8 b/c of py3.6 yeah?
14:13:03 <mriedem> and what is the regression from rocky?
14:13:05 <sean-k-mooney> mriedem: yes
14:13:10 <mdbooth> mriedem: py36 and some combination of libraries
14:13:20 <mriedem> https://review.openstack.org/#/c/592285/ i guess
14:13:25 <mdbooth> I outlined the regression in the ML post
14:13:38 <mriedem> i've avoided that giant thread
14:13:46 <mdbooth> The regression is actually the cross-cells thing
14:14:10 <sean-k-mooney> mriedem: the parallel multi cell list change we did intoduced a eventlet depency in the api
14:14:24 <mdbooth> The wsgi fix to the cross cells thing broke n-cpu without fully fixing wsgi (if you've got the unlucky libraries)
14:14:30 <sean-k-mooney> previously we did not need to monkeypatch when the api ran under wsgi
14:15:38 <mdbooth> So melwitt asked me about this a couple of weeks ago and I didn't think it was a regression at the time.
14:15:46 <mriedem> i wonder why this isn't a problem in our gate which is running bionic nodes with py36
14:16:28 <mdbooth> mriedem: Me, too. However, as I pointed out in the fix itself you're basically just lucky.
14:17:13 <mdbooth> That combination of libraries happens to stumble past all the bugs.
14:17:29 <mriedem> efried: so i'd mark it as stein-rc-potential as a candidate for an RC2
14:17:37 <mriedem> dansmith is in a meeting,
14:17:41 <mriedem> and i need to jump in one soon too
14:18:00 <mdbooth> That would be cool. As I say from a downstream POV if this didn't land in Stein it wouldn't be the end of the world as long as it landed real soon after.
14:18:12 <mdbooth> We don't like backporting things which haven't at least landed on master.
14:18:13 <mriedem> and i've been conveniently avoiding this because i don't trust myself with eventlet
14:18:22 <efried> mdbooth: Would you please do the bug tagging and add to the etherpad?
14:18:31 <mdbooth> efried: Ack. Thanks.
14:18:36 <efried> thanks
14:18:56 <efried> okay, last call for stein topics
14:19:16 <efried> #topic Bugs
14:19:30 <efried> #link Critical bugs (0): https://bugs.launchpad.net/nova/+bugs?search=Search&field.importance=Critical&field.status=New&field.status=Incomplete&field.status=Confirmed&field.status=Triaged&field.status=In+Progress
14:19:43 <efried> #link 64 new untriaged bugs (down 10 since the last meeting): https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New
14:19:50 <efried> #link 6 untagged untriaged bugs (down 7 since the last meeting): https://bugs.launchpad.net/nova/+bugs?field.tag=-*&field.status%3Alist=NEW
14:19:55 <efried> That's goodness :)
14:20:09 <efried> Anyone want to bring up specific bugs?
14:20:32 <efried> #topic Gate status
14:20:41 <efried> #link check queue gate status http://status.openstack.org/elastic-recheck/index.html
14:20:46 <efried> we had a snafu last night:
14:20:52 <efried> #link placement reusing conf https://bugs.launchpad.net/nova/+bug/1821092
14:20:53 <openstack> Launchpad bug 1821092 in OpenStack Compute (nova) "test_local_delete_removes_allocations_after_compute_restart failing since https://review.openstack.org/#/c/644591/" [Critical,Fix released] - Assigned to Chris Dent (cdent)
14:20:53 <efried> #link short-term fix (merged) https://review.openstack.org/#/c/645014/
14:20:53 <efried> #link long-term fix (merged) https://review.openstack.org/#/c/645031/
14:20:53 <efried> #link stein backport of ^ (open) https://review.openstack.org/#/c/645105/
14:21:45 <efried> thanks to cdent for identifying and resolving the problem quickly when he should have been sleeping.
14:22:18 <efried> otherwise, gate seems... okay?
14:22:33 <efried> #topic 3rd party CI
14:22:40 <efried> #link 3rd party CI status http://ciwatch.mmedvede.net/project?project=nova&time=7+days
14:23:11 <efried> zVM is choppy, but succeeding sometimes. Not sure who's the point of contact these days, haven't seen jichenjc in a while. Anyone know?
14:24:21 <mriedem> nope
14:24:25 <mriedem> i forgot that driver was merged
14:25:01 <efried> I'll try to dig something up.
14:25:09 <efried> #action efried to find zVM contact for CI
14:25:15 <efried> PowerVM CI is dead. That team is in "transition" currently (edmondsw is on a different project, outside of OpenStack), so I say give them a couple months to get their feet under them before we start breaking kneecaps.
14:25:45 <efried> ...unless edmondsw happens to have an update?
14:26:13 <sean-k-mooney> efried: looking at https://wiki.openstack.org/wiki/ThirdPartySystems/IBM_z/VM_CI we should contact zvmosci@us.ibm.com but there is no irc nic listed. not sure it up todate however
14:26:24 <efried> sean-k-mooney: Thanks
14:26:42 <efried> I'll cc jichen; if he isn't it, he ought to know who is.
14:26:58 <efried> #action efried to follow up with edmondsw re PowerVM CI
14:27:09 <efried> any other CI business?
14:27:28 <efried> #topic Reminders
14:27:43 <efried> uhhh, I feel like a lot is happening. Does anyone need to be reminded of any of it?
14:27:58 <efried> or have things to remind the rest of us about?
14:28:06 <edmondsw> we are working on the CI
14:28:25 <efried> edmondsw: ack, thank you.
14:28:54 <efried> #topic Stable branch status
14:29:09 <efried> #link stable/rocky: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/rocky,n,z
14:29:14 <mriedem> trying to burn down since pike is going into extended maintenance
14:29:20 <mriedem> need more reviews on stable/queens
14:29:24 <mriedem> we should have a rocky release soonish
14:29:29 <mriedem> just waiting for 2 changes to land
14:29:38 <mriedem> there is also a functional test that keeps failing on stable
14:29:45 <mriedem> http://status.openstack.org/elastic-recheck/index.html#1820337
14:29:49 <mriedem> if someone wants to investigate that
14:30:15 <mriedem> interestingly that is only on stable
14:30:50 * mriedem jumps on other call
14:30:57 * gibi jumps as well
14:31:06 <mriedem> sshhh it's secret
14:31:48 <efried> thanks for the summary mriedem.
14:31:53 <efried> Any other stable issues?
14:32:02 <mriedem> nope
14:33:20 <efried> #topic Subteam Highlights
14:33:26 <efried> scheduler (me)
14:34:01 <efried> First thing: we've proposed to stop doing a separate scheduler meeting, and instead do a placement meeting in the same slot.
14:34:18 <efried> #link n-sch => placement team meeting http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004059.html
14:34:54 <efried> The metadata patch already merged, so it's, like, official, but if you have objections, comments, suggestions, whatever, please respond to ^ or hit people up in IRC or whatever.
14:35:38 <sean-k-mooney> one question will we still cover it in this subteam section going forward
14:35:46 <efried> on that note, does anyone feel the need to have a ...
14:35:52 <efried> I think that's what I was going to ask
14:36:03 <efried> does anyone feel the need to have a "placement team update" in the nova meeting?
14:36:20 <efried> I'm going to go out on a limb and say no, because if you care, go read the logs of that meeting.
14:36:29 <cdent> agree
14:36:36 <efried> That said, I would still expect cdent or whoever to chime in if there are nova-impacting issues coming out of that meeting.
14:36:40 <cdent> especially given the pupdate exists
14:36:48 <cdent> and yes, i'll be in both places
14:36:59 <sean-k-mooney> we may have non placement sechduler topics to discsuss but i have no strong feeling either way
14:37:15 <edleafe> sean-k-mooney: those can be part of this meeting, no?
14:37:24 <sean-k-mooney> correct they can
14:37:36 <efried> sean-k-mooney: Yeah, sorry, that's the intent, for any such topics to be folded directly into this meeting.
14:38:01 <sean-k-mooney> sounds fine to me
14:38:14 <efried> we had subteam because that had turned into a long section, which eventually became almost exclusively about placement. Now that we have placement, it can be its own thing, and any sched-specific stuff can just be done here again
14:38:21 <efried> if it stops working for whatever reason, we can re-evaluate.
14:38:37 <efried> on that note, here are the couple of nova-ish things that were brought up in Monday's n-sch meeting:
14:38:42 <efried> #link inter-provider affinity brainstorm etherpad https://etherpad.openstack.org/p/placement-inter-provider-affinity
14:39:26 <efried> we've been knocking around this idea of "how do we do generic NUMA affinity" in the past several PTGs and a spec or two. Given we're starting to try to model NUMA in placement at all, I think we should bring this to the forefront.
14:39:43 <efried> I've got that ^ on ptg etherpads. If you have ideas etc., please chime in.
14:39:57 <efried> and
14:39:57 <efried> #link Nova scheduler using openstacksdk instead of direct ksa to talk to placement https://review.openstack.org/643664
14:40:25 <efried> Been working with mordred and dustinc on this
14:40:37 <mordred> HECK YEAH BABY
14:40:47 <efried> some people are excited about it
14:40:56 <mordred> HECK YEAH BABY!!!!
14:42:05 <efried> This and its successor
14:42:05 <efried> #link use openstacksdk instead of python-ironicclient https://review.openstack.org/#/c/642899/
14:42:05 <efried> are a couple steps on the road toward getting rid of nova's use of python-*clientZ and bringing us in line with the modern world of openstacksdk
14:42:23 <efried> if you have interest and/or objections, please let me or mordred know.
14:42:50 <efried> moving on
14:42:56 <efried> API (gmann) ?
14:42:59 <sean-k-mooney> efried: just one question. the patch i quickly hacked up to test with ironic client do you want me to abandon that
14:43:31 <efried> sean-k-mooney: I've been leaving it around as a reference for "how to do stuff", but it can serve in that capacity abandoned too.
14:43:54 <efried> sean-k-mooney: but yeah, it's no longer needed, since I decided to go whole hog and bypass python-ironicclient completely.
14:44:10 <efried> thanks for working that up for me. It was educational in any case.
14:44:33 <sean-k-mooney> efried: ok i was debating if i shoudl convert it into a real change but ill abandon instead
14:44:53 <efried> Usually people mention something about gmann having sent an API subteam update. Where does that go?
14:45:13 <efried> For now I guess we'll move on.
14:45:17 <efried> #topic Stuck Reviews
14:45:20 <efried> any?
14:45:45 <efried> #topic Open Discussion
14:46:09 <efried> anyone?
14:46:43 <edleafe> I like turtles
14:46:58 <efried> "turtle guard" is an oxymoron
14:47:15 <efried> Okay, thanks all.
14:47:15 <efried> o/
14:47:15 <efried> #endmeeting