14:06:23 <adreznec> #startmeeting PowerVM Driver Meeting 14:06:23 <openstack> Meeting started Tue Nov 15 14:06:23 2016 UTC and is due to finish in 60 minutes. The chair is adreznec. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:06:24 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:06:27 <openstack> The meeting name has been set to 'powervm_driver_meeting' 14:06:34 <adreznec> Helps if I use the right command 14:06:52 <adreznec> #topic Current status 14:07:19 <adreznec> All right, so let's start with the upstream driver related topic 14:07:44 <adreznec> Now that the blueprint is approved/spec merged, efried1 has taken the WIP off all of this change sets 14:08:10 <thorst> wooo 14:08:29 <adreznec> Our global requirements patch for pypowervm has a +1 from Jenkins, but needs responses to the requirements team questions 14:08:30 <thorst> and our very sensible version number is going splendidly 14:08:36 <adreznec> Yes 14:08:44 <adreznec> We'll call it semver++ 14:08:50 <adreznec> One + for each additional digit 14:09:00 <efried1> For the record, I tried to produce the upper-constraints patch using the process outlined in the README. Never got it to work. 14:09:03 * adreznec grumbles off in the background 14:09:09 <efried1> Hence typo 14:09:31 <efried1> Didn't figure it was rocket science. And it's not. I'm just a doofus. 14:09:49 <efried1> I'll take care of that questionnaire this morning. 14:10:00 <adreznec> Was an easy fix anyway, and it means our very excellent versioning scheme can stay intact 14:10:01 <efried1> #action efried to update requirements change set per comments. 14:10:19 <efried1> #action efried1 to update requirements change set per comments. 14:10:30 <efried1> (I guess I'm efried1 since the last time my laptop crashed) 14:10:38 <thorst> I can't wait for efried2 14:10:48 <efried1> Me neither. 14:10:49 <efried1> What's next. 14:10:59 <adreznec> Do we need action on any of the other patches yet? 14:11:00 <thorst> CI stability? 14:11:12 <efried1> oh, hold on, 14:11:21 <efried1> let's talk a little bit about Tony's comment. 14:11:38 <efried1> Do we respond to it in the change set (and attempt to explain the IBM legal reasons we don't use semver)? 14:12:12 <adreznec> I'm... not sure it matters either way? 14:12:18 <efried1> Or maybe something flippant like, "yeah, we don't like it either, but there's a good (if boring) reason for it" 14:12:21 <efried1> or just ignore it. 14:12:29 <adreznec> I mean it's definitely awful 14:12:43 <adreznec> But I'm not sure it's actually a big deal to anyone at the end of the day 14:12:55 <adreznec> Besides me 14:13:00 <efried1> I'm just talking about for the sake of responding to Tony's comment. 14:13:51 <efried1> <silence> 14:13:54 <adreznec> I don't think it really matters as long as we can answer all the other questions 14:13:59 <efried1> Roger that. 14:14:05 <adreznec> From the questionnaire 14:14:06 <thorst> yeah, I'm not sure its a big deal. I'm OK saying 'we don't really like it either'...but I'm not sure we need to make it a big deal 14:14:10 <adreznec> Right 14:14:11 <thorst> I think its a bigger deal to us than them 14:14:35 <efried1> The other thing on current patch sets: do we want to preserve the nova-powervm versioning history on the live migrate data object, or start it at 1.0 in nova proper? 14:14:42 <adreznec> We can just say Watson designated the version 14:14:44 <adreznec> And leave it at that 14:14:47 <adreznec> :P 14:14:48 <efried1> (see comment: https://review.openstack.org/#/c/391284/3/nova/objects/migrate_data.py@301 14:14:49 <efried1> ) 14:15:35 <thorst> efried1: I can respond there if you'd like 14:15:46 <thorst> I'm sure they don't know the history of us being out of tree 14:15:47 <thorst> but that's why 14:16:05 <efried1> So we do want to preserve our out-of-tree version history. 14:16:41 <thorst> it simplifies the transition 14:16:53 <efried1> Right, for people currently using the out-of-tree driver. 14:16:54 <thorst> for operators 14:16:55 <adreznec> To an extent, yeah 14:17:10 <adreznec> I guess it depends on if we ever want to support anyone migrating from the out-of-tree driver to the in-tree driver 14:17:26 <efried1> Heck, maybe we should even bump it up to 1.2 14:17:32 <efried1> or 1.1.0.0.9 14:17:45 <adreznec> +2 14:18:38 <adreznec> I guess we can respond with a comment saying we'd like to preserve it for ops 14:18:52 <adreznec> And if they push back we can talk about the implications of that 14:19:28 <adreznec> Seem reasonable thorst efried1? 14:19:46 <thorst> yep 14:19:55 <efried1> So 14:19:55 <efried1> #action thorst to respond to https://review.openstack.org/#/c/391284/3/nova/objects/migrate_data.py@301 14:19:55 <efried1> ? 14:20:03 <adreznec> So it would seem 14:20:22 <adreznec> All right 14:20:25 <efried1> Okay. The last thing would be socializing the driver skeleton change set. 14:20:37 <efried1> I'm still not clear how that's supposed to happen normally. 14:21:04 <efried1> Guess we're not in a super hurry to get that merged until we have CI stable. Next #topic? 14:21:39 <adreznec> So I think we need to talk with the Nova cores about whether we should be on that subteam priority reviews list now 14:21:57 <thorst> sorry - afk for a bit. 14:22:07 <efried1> adreznec, I don't know what that is. 14:22:14 <adreznec> and list our 'top 5' reviews there like the other driver teams are 14:22:20 <adreznec> Let me dig that up quick 14:23:05 <adreznec> Like this: https://etherpad.openstack.org/p/ocata-nova-priorities-tracking 14:24:08 <efried1> Oh, for sure. 14:24:10 <adreznec> Basically whether or not we should have a PowerVM Subteam section on there where we start listing these reviews 14:24:29 <adreznec> We should bring this up in the next Nova meeting then 14:24:39 <efried1> Sounds like a good plan. 14:25:01 <adreznec> #action adreznec thorst and efried1 to bring up PowerVM subteam/reviews in the next Nova meeting 14:25:12 <adreznec> Okay 14:25:26 <adreznec> #topic Priority reviews 14:25:44 <adreznec> Do we have anything open that needs code review priority? 14:26:01 <efried1> There's nothing on my radar in the community. 14:26:10 <adreznec> FYI for the team - until https://review.openstack.org/#/c/396934/ merges, multi-architecture deploys are broken 14:26:20 <adreznec> But that already has +2/+1s this morning 14:26:25 <adreznec> So just awareness 14:26:41 <adreznec> All right 14:26:50 <adreznec> Sounds like nothing high priority 14:27:05 <adreznec> #topic Priority bugs/defects 14:27:24 <adreznec> Same thing here - any key issues people want to direct attention to? 14:27:48 <efried1> Other than the CI, nothing I'm aware of. thorst may have something. 14:28:24 <efried1> Can we move on to CI? That could be a substantial discussion. 14:28:27 <adreznec> Yep 14:28:31 <esberglu> Ok 14:28:33 <adreznec> #topic CI discussion 14:28:43 <adreznec> esberglu: efried1, it's all yours 14:28:55 <efried1> esberglu, what's the latest? 14:29:09 <esberglu> So amartey thinks that upgrading novalink *may* fix the LUs hanging around after the runs 14:29:24 <esberglu> #action esberglu update novalink in CI env 14:29:31 <esberglu> As far as the actual runs 14:29:44 <adreznec> Upgrading it to what? 14:29:54 <adreznec> Latest 1.0.0.4? Latest dev? 14:30:25 <wangqwsh> however, i found 183 can not retrieve vm info 14:30:41 <esberglu> Sorry I got disconnected. Latest 1.0.0.4 14:30:52 <adreznec> Ok 14:31:16 <adreznec> Well I guess we can give it a shot 14:31:16 <esberglu> I put the latest versions of the patches from efried on yesterday afternoon 14:31:29 <esberglu> But still not working 14:31:43 <esberglu> I let one run through last night 14:31:47 <esberglu> here are the logs 14:31:48 <esberglu> http://184.172.12.213/15/328315/19/check/nova-powervm-pvm-dsvm-tempest-full/9564270/ 14:32:00 <esberglu> Still not finding any valid hosts for many of the tests 14:32:10 <esberglu> I haven't dug into the logs yet from that run 14:33:50 <efried1> #action efried1 esberglu to analyze http://184.172.12.213/15/328315/19/check/nova-powervm-pvm-dsvm-tempest-full/9564270/ 14:34:10 <efried1> We may be at a point where we should put all hands on deck until the CI is stabilized. 14:34:56 <efried1> Throwing more people at it doesn't help the runs themselves finish faster. But having all eyes on the results right away... 14:35:45 <adreznec> Yeah, I know people are getting pulled in a number of directions right now 14:35:52 <adreznec> Maybe even just a one day sprint 14:36:05 <adreznec> Focused on CI improvements could help 14:36:45 <adreznec> Unfortunately the holiday season is starting to creep up on us 14:36:54 <adreznec> So finding availability could be tricky 14:37:38 <esberglu> Yeah I'm out starting friday through all of thanksgiving week. No internet Friday to Wednesday 14:38:33 <adreznec> All right. I guess we'll have to table that discussion for now and discuss it outside the meeting 14:38:53 <adreznec> So what else do we have on CI? 14:39:11 <efried1> Seems service nova-compute on host powervm-ci-powervm-devstacked-25844 is down. Last heartbeat was 2016-11-15 01:54:39. Elapsed time is 126.14957 14:39:19 <efried1> Looks like the compute service died at some point in there. 14:39:49 <thorst> btw - I 100% agree that we need an all hands on deck for CI 14:39:51 <efried1> Compute log has no interrupt or anything - it just stops. 14:39:52 <adreznec> Well that would certainly explain issues finding a host 14:40:02 <thorst> does it just stop because we hit 5 hours? 14:40:05 <thorst> and zuul just shut it down 14:40:13 <esberglu> No the run went through in like 40 minutes 14:40:21 <esberglu> It went super fast because it couldn't find any hosts 14:40:36 <thorst> o 14:40:37 <thorst> huh 14:40:38 <efried1> 2016-11-14 19:54:43.560 is the last time stamp in the compute log. 14:41:40 <esberglu> The 5 hour runs happen when it times out waiting for hosts to become active (which we are past at this point I believe) 14:42:26 <esberglu> So should I let another run through? Maybe the host dying was just a one-off thing? 14:42:58 <thorst> esberglu: the 5 hour thing happens when we wait for the marker LU 14:43:04 <thorst> which we *may* be past, hopefully 14:43:12 <thorst> just needed an uber clean of the env 14:43:26 <esberglu> Yeah and efrieds latest patch 14:43:43 <thorst> right. 14:43:50 <thorst> we also need the novalinks updated 14:43:55 <thorst> did you see that from apearson? 14:44:00 <esberglu> Yep 14:44:05 <esberglu> already did an action for it 14:44:08 <thorst> so are those the immediate next steps? 14:44:17 <adreznec> Yep 14:44:24 <adreznec> Already had the novalink upgrade as an action 14:44:50 <thorst> then all hands on deck for the next run to debug 14:45:15 <adreznec> #action esberglu to try another CI run with latest patch and report results 14:45:37 <adreznec> Ok 14:45:46 <adreznec> Anything else on the CI front? 14:45:58 <esberglu> I think wangqwsh was hitting some OSA CI stuff? 14:46:04 <wangqwsh> yes, 14:46:32 <wangqwsh> the memory is extended to 12g, galera can work. 14:46:44 <wangqwsh> but it's failed to boot a vm. 14:46:57 <wangqwsh> i am trying to understand drew's mail 14:48:04 <wangqwsh> do not know how to set the network on neo14... 14:48:30 <adreznec> Ah, this was the OVS network issues email 14:49:42 <wangqwsh> yes. 14:50:08 <thorst> I kind of want to wait on the OVS bits until we have CI stabilized 14:50:15 <thorst> we need our core CI working first... 14:50:17 <adreznec> thorst probably has a more detailed description than this, but basically where we're at now (IIRC) is that we need to create a PHYP vswitch for each tempest VM running OSA, then put the OVS trunk on that 14:50:18 <adreznec> Right 14:50:29 <thorst> adreznec is right. 14:51:04 <adreznec> thorst: yeah, I agree with you 14:51:15 <wangqwsh> ok 14:51:28 <adreznec> we should probably put the OSA CI work on temporary hold while we all debug the main CI issues 14:52:18 <adreznec> thorst: I will note that on the patch Kyle put up to fix the OSA multi-arch stuff that got broken, Jesse did comment about investigating a multi-arch CI test 14:52:33 <adreznec> Nothing we need to do now 14:52:40 <adreznec> But something for the back burner 14:53:28 <adreznec> So, CI stability is going to take priority 14:53:48 <adreznec> Do we have anything else to discuss here? Or are we just on deck for the next set of results? 14:54:37 <adreznec> Ok, well if there's nothing else I'm going to call it 14:54:43 <adreznec> Thanks for joining everyone 14:54:58 <adreznec> #endmeeting