14:00:02 #startmeeting nova 14:00:02 Meeting started Thu Mar 21 14:00:02 2019 UTC and is due to finish in 60 minutes. The chair is gibi. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:06 o/ 14:00:07 The meeting name has been set to 'nova' 14:00:14 o/ 14:00:20 o/ 14:00:20 gibi: you want to run it? 14:00:30 o/ 14:00:34 efried: I have to jump to a call at half past so you can take this 14:00:46 #chair efried 14:00:46 Current chairs: efried gibi 14:00:53 I done prepared and everything 14:01:02 Greetings all. 14:01:06 efried: thanks 14:01:11 #link agenda https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 14:01:25 updated like two minutes ago, refresh if needed 14:01:42 #topic Release News 14:01:54 #link Stein release schedule: https://wiki.openstack.org/wiki/Nova/Stein_Release_Schedule 14:01:54 #info Stein RC1 is TODAY, March 21 14:01:54 #link Stein RC potential changes tracking: https://etherpad.openstack.org/p/nova-stein-rc-potential 14:02:09 couple of things I wanted to bring up from the etherpad 14:02:13 Docs for bug 1821015 14:02:14 bug 1821015 in OpenStack Compute (nova) "Attaching virtual GPU devices to guests in nova - libvirt reshaping" [Medium,Triaged] https://launchpad.net/bugs/1821015 - Assigned to Sylvain Bauza (sylvain-bauza) 14:02:24 * edleafe wanders in late 14:02:44 bauzas didn't sign in, was going to ask him about that ^ because it requires (in order to be "authentic") GPU hardware. 14:02:55 he said he's working on it 14:03:02 okay, cool, thanks. 14:03:10 #link Cycle highlights https://review.openstack.org/#/c/644697/ 14:03:14 oops 14:03:20 I'm there 14:03:46 Please go have a look at that. I guess somebody on the releases team will decide when to merge it. 14:04:13 #link reno prelude https://review.openstack.org/#/c/644412/ 14:05:06 This has several +2s. I think we want to +W it today. There's a complex web of deps, but I think the only one that's not yet merged is 14:05:06 #link network doc side of bw QoS https://review.openstack.org/#/c/640390 14:05:12 need to get https://review.openstack.org/#/c/640390 merged 14:05:22 yeah i asked haleyb to just merge that last night 14:05:26 can also ping mlavalle today 14:06:13 mriedem: sorry, didn't see your ping about that last night, but seeme bence was going to update it? 14:06:26 haleyb: https://review.openstack.org/#/c/645104/ 14:06:27 haleyb: bence put up a followup patch 14:06:28 he already followed up 14:06:50 mriedem: sorry, missed the reference, let me merge 14:07:07 thanks 14:07:13 efried: i've approved the nova prelude change 14:07:15 Miguel just did it. 14:07:21 Cool. 14:07:40 Any other stein-y stuff to bring up? 14:08:37 moving on 14:08:41 efried: Yep 14:08:48 mdbooth: your mic 14:08:59 or were you agreeing to move on? 14:09:03 I just posted to the ML about my monkey patching thing 14:09:29 I realise how late in the day it is for Stein, but I think it could be worthy of inclusion 14:09:30 does this seem like kind of a dangerous thing to be trying to land right now? 14:09:54 Right, but it's a regression which makes it impossible to deploy Nova Stein on RHEL 8 14:10:21 mdbooth: secifcally with tripleo 14:10:24 Now downstream that's not a huge issue. As long as we land it upstream Real Soon we'll backport it 14:10:29 sean-k-mooney: It has nothing to do with tripleo 14:10:43 But it is a regression from Rocky. 14:11:11 #link monkey patch early https://review.openstack.org/#/c/626952/ 14:11:27 I only vaguely understand what's going on there (monkey patch early and only once and skip certain monkey patches in certain environments and don't do it for sphinx). 14:12:01 anyone want to weigh in on relative risks/merits of landing right now? 14:12:06 It's horrible, but without it nova compute doesn't even start on RHEL 8. 14:12:23 And remember it's only horrible because it's changing something horrible. 14:12:50 that seems like it would warrant inclusion. dansmith, mriedem, opinions here? 14:12:55 it's a problem for rhel 8 b/c of py3.6 yeah? 14:13:03 and what is the regression from rocky? 14:13:05 mriedem: yes 14:13:10 mriedem: py36 and some combination of libraries 14:13:20 https://review.openstack.org/#/c/592285/ i guess 14:13:25 I outlined the regression in the ML post 14:13:38 i've avoided that giant thread 14:13:46 The regression is actually the cross-cells thing 14:14:10 mriedem: the parallel multi cell list change we did intoduced a eventlet depency in the api 14:14:24 The wsgi fix to the cross cells thing broke n-cpu without fully fixing wsgi (if you've got the unlucky libraries) 14:14:30 previously we did not need to monkeypatch when the api ran under wsgi 14:15:38 So melwitt asked me about this a couple of weeks ago and I didn't think it was a regression at the time. 14:15:46 i wonder why this isn't a problem in our gate which is running bionic nodes with py36 14:16:28 mriedem: Me, too. However, as I pointed out in the fix itself you're basically just lucky. 14:17:13 That combination of libraries happens to stumble past all the bugs. 14:17:29 efried: so i'd mark it as stein-rc-potential as a candidate for an RC2 14:17:37 dansmith is in a meeting, 14:17:41 and i need to jump in one soon too 14:18:00 That would be cool. As I say from a downstream POV if this didn't land in Stein it wouldn't be the end of the world as long as it landed real soon after. 14:18:12 We don't like backporting things which haven't at least landed on master. 14:18:13 and i've been conveniently avoiding this because i don't trust myself with eventlet 14:18:22 mdbooth: Would you please do the bug tagging and add to the etherpad? 14:18:31 efried: Ack. Thanks. 14:18:36 thanks 14:18:56 okay, last call for stein topics 14:19:16 #topic Bugs 14:19:30 #link Critical bugs (0): https://bugs.launchpad.net/nova/+bugs?search=Search&field.importance=Critical&field.status=New&field.status=Incomplete&field.status=Confirmed&field.status=Triaged&field.status=In+Progress 14:19:43 #link 64 new untriaged bugs (down 10 since the last meeting): https://bugs.launchpad.net/nova/+bugs?search=Search&field.status=New 14:19:50 #link 6 untagged untriaged bugs (down 7 since the last meeting): https://bugs.launchpad.net/nova/+bugs?field.tag=-*&field.status%3Alist=NEW 14:19:55 That's goodness :) 14:20:09 Anyone want to bring up specific bugs? 14:20:32 #topic Gate status 14:20:41 #link check queue gate status http://status.openstack.org/elastic-recheck/index.html 14:20:46 we had a snafu last night: 14:20:52 #link placement reusing conf https://bugs.launchpad.net/nova/+bug/1821092 14:20:53 Launchpad bug 1821092 in OpenStack Compute (nova) "test_local_delete_removes_allocations_after_compute_restart failing since https://review.openstack.org/#/c/644591/" [Critical,Fix released] - Assigned to Chris Dent (cdent) 14:20:53 #link short-term fix (merged) https://review.openstack.org/#/c/645014/ 14:20:53 #link long-term fix (merged) https://review.openstack.org/#/c/645031/ 14:20:53 #link stein backport of ^ (open) https://review.openstack.org/#/c/645105/ 14:21:45 thanks to cdent for identifying and resolving the problem quickly when he should have been sleeping. 14:22:18 otherwise, gate seems... okay? 14:22:33 #topic 3rd party CI 14:22:40 #link 3rd party CI status http://ciwatch.mmedvede.net/project?project=nova&time=7+days 14:23:11 zVM is choppy, but succeeding sometimes. Not sure who's the point of contact these days, haven't seen jichenjc in a while. Anyone know? 14:24:21 nope 14:24:25 i forgot that driver was merged 14:25:01 I'll try to dig something up. 14:25:09 #action efried to find zVM contact for CI 14:25:15 PowerVM CI is dead. That team is in "transition" currently (edmondsw is on a different project, outside of OpenStack), so I say give them a couple months to get their feet under them before we start breaking kneecaps. 14:25:45 ...unless edmondsw happens to have an update? 14:26:13 efried: looking at https://wiki.openstack.org/wiki/ThirdPartySystems/IBM_z/VM_CI we should contact zvmosci@us.ibm.com but there is no irc nic listed. not sure it up todate however 14:26:24 sean-k-mooney: Thanks 14:26:42 I'll cc jichen; if he isn't it, he ought to know who is. 14:26:58 #action efried to follow up with edmondsw re PowerVM CI 14:27:09 any other CI business? 14:27:28 #topic Reminders 14:27:43 uhhh, I feel like a lot is happening. Does anyone need to be reminded of any of it? 14:27:58 or have things to remind the rest of us about? 14:28:06 we are working on the CI 14:28:25 edmondsw: ack, thank you. 14:28:54 #topic Stable branch status 14:29:09 #link stable/rocky: https://review.openstack.org/#/q/status:open+project:openstack/nova+branch:stable/rocky,n,z 14:29:14 trying to burn down since pike is going into extended maintenance 14:29:20 need more reviews on stable/queens 14:29:24 we should have a rocky release soonish 14:29:29 just waiting for 2 changes to land 14:29:38 there is also a functional test that keeps failing on stable 14:29:45 http://status.openstack.org/elastic-recheck/index.html#1820337 14:29:49 if someone wants to investigate that 14:30:15 interestingly that is only on stable 14:30:50 * mriedem jumps on other call 14:30:57 * gibi jumps as well 14:31:06 sshhh it's secret 14:31:48 thanks for the summary mriedem. 14:31:53 Any other stable issues? 14:32:02 nope 14:33:20 #topic Subteam Highlights 14:33:26 scheduler (me) 14:34:01 First thing: we've proposed to stop doing a separate scheduler meeting, and instead do a placement meeting in the same slot. 14:34:18 #link n-sch => placement team meeting http://lists.openstack.org/pipermail/openstack-discuss/2019-March/004059.html 14:34:54 The metadata patch already merged, so it's, like, official, but if you have objections, comments, suggestions, whatever, please respond to ^ or hit people up in IRC or whatever. 14:35:38 one question will we still cover it in this subteam section going forward 14:35:46 on that note, does anyone feel the need to have a ... 14:35:52 I think that's what I was going to ask 14:36:03 does anyone feel the need to have a "placement team update" in the nova meeting? 14:36:20 I'm going to go out on a limb and say no, because if you care, go read the logs of that meeting. 14:36:29 agree 14:36:36 That said, I would still expect cdent or whoever to chime in if there are nova-impacting issues coming out of that meeting. 14:36:40 especially given the pupdate exists 14:36:48 and yes, i'll be in both places 14:36:59 we may have non placement sechduler topics to discsuss but i have no strong feeling either way 14:37:15 sean-k-mooney: those can be part of this meeting, no? 14:37:24 correct they can 14:37:36 sean-k-mooney: Yeah, sorry, that's the intent, for any such topics to be folded directly into this meeting. 14:38:01 sounds fine to me 14:38:14 we had subteam because that had turned into a long section, which eventually became almost exclusively about placement. Now that we have placement, it can be its own thing, and any sched-specific stuff can just be done here again 14:38:21 if it stops working for whatever reason, we can re-evaluate. 14:38:37 on that note, here are the couple of nova-ish things that were brought up in Monday's n-sch meeting: 14:38:42 #link inter-provider affinity brainstorm etherpad https://etherpad.openstack.org/p/placement-inter-provider-affinity 14:39:26 we've been knocking around this idea of "how do we do generic NUMA affinity" in the past several PTGs and a spec or two. Given we're starting to try to model NUMA in placement at all, I think we should bring this to the forefront. 14:39:43 I've got that ^ on ptg etherpads. If you have ideas etc., please chime in. 14:39:57 and 14:39:57 #link Nova scheduler using openstacksdk instead of direct ksa to talk to placement https://review.openstack.org/643664 14:40:25 Been working with mordred and dustinc on this 14:40:37 HECK YEAH BABY 14:40:47 some people are excited about it 14:40:56 HECK YEAH BABY!!!! 14:42:05 This and its successor 14:42:05 #link use openstacksdk instead of python-ironicclient https://review.openstack.org/#/c/642899/ 14:42:05 are a couple steps on the road toward getting rid of nova's use of python-*clientZ and bringing us in line with the modern world of openstacksdk 14:42:23 if you have interest and/or objections, please let me or mordred know. 14:42:50 moving on 14:42:56 API (gmann) ? 14:42:59 efried: just one question. the patch i quickly hacked up to test with ironic client do you want me to abandon that 14:43:31 sean-k-mooney: I've been leaving it around as a reference for "how to do stuff", but it can serve in that capacity abandoned too. 14:43:54 sean-k-mooney: but yeah, it's no longer needed, since I decided to go whole hog and bypass python-ironicclient completely. 14:44:10 thanks for working that up for me. It was educational in any case. 14:44:33 efried: ok i was debating if i shoudl convert it into a real change but ill abandon instead 14:44:53 Usually people mention something about gmann having sent an API subteam update. Where does that go? 14:45:13 For now I guess we'll move on. 14:45:17 #topic Stuck Reviews 14:45:20 any? 14:45:45 #topic Open Discussion 14:46:09 anyone? 14:46:43 I like turtles 14:46:58 "turtle guard" is an oxymoron 14:47:15 Okay, thanks all. 14:47:15 o/ 14:47:15 #endmeeting