20:59:15 #startmeeting nova 20:59:15 Meeting started Thu Jan 15 20:59:15 2015 UTC and is due to finish in 60 minutes. The chair is mikal. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:59:17 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:59:19 The meeting name has been set to 'nova' 20:59:21 Who is here for the nova meeting? 20:59:24 o/ 20:59:24 o/ 20:59:24 o/ 20:59:26 o/ 20:59:27 o/ 20:59:29 hi 20:59:31 o/ (your 45 seconds early) 20:59:36 o/ 20:59:45 pings for: k mikal tjones cburgess jgrimm adrian_otto funzo mjturek jcook ekhugen irina_pov krtaylor danpb alexpilotti flip214 raildo jaypipes gilliard garyk edleafe dims moshele anteaya 20:59:50 o/ 20:59:50 Heh, cut and paste fail 20:59:58 \o 21:00:02 mikal: you pinged yourself? 21:00:04 hi! 21:00:06 \o/ 21:00:10 eh, 1 min 21:00:32 flashgordon: its how I roll 21:00:37 Sorry if this week is a bit disorganized 21:00:41 I'm at a conf and doing this on my laptop 21:00:52 Which means I can't have the agenda up at the same time as the IRC session 21:00:56 So... That said... 21:01:05 #topic Kilo specs 21:01:08 https://wiki.openstack.org/wiki/Meetings/Nova 21:01:16 So, we've announced the spec exception process 21:01:19 #link http://lists.openstack.org/pipermail/openstack-dev/2015-January/054039.html 21:01:23 And we've had 18 specs already request an exception 21:01:33 Which is a lot 21:02:05 So, I'd suggest if you want an exception and haven't emailed yet, do that nowish 21:02:14 * alaski slinks in late 21:02:16 I also think we should give exception priority to priority work 21:02:31 Noting that I can't actually compell reviewers to review in any particular manner 21:02:36 Any other thoughts on the exception process? 21:02:40 anyway, it has to be accepted by 2 weeks before K-2 right, so in 1 week ? 21:02:51 1/22 21:02:57 yes 21:02:58 Yeah, the deadline is 22 Jan 21:03:02 Which is just before the mid-cycle 21:03:03 feature freeze is 2/5 21:03:15 If I just want to get into Kilo release, but not necessarily kilo-2, is it enough to find a few reviewers for the spec and the code? 21:03:16 mikal: so indeed, people have to rush if they want exceptions 21:03:33 bauzas: rush to request yes 21:03:37 Is there a plan for core spec reviewers to go over the requests? 21:03:39 I'd like to discuss this at the mid-cycle for future releases, I think there could be improvements in the process 21:03:42 There is a little time for reviewers to do their thing though 21:03:57 mikal: yeah was unclear, rush is for proposers 21:04:00 n0ano: please put it on the agenda etherpad? 21:04:01 feature freeze means any new code patch to be submitted before 2/5? 21:04:06 mikal, will do 21:04:12 n0ano: https://etherpad.openstack.org/p/kilo-nova-midcycle 21:04:14 jichen: yes, any new _features_ 21:04:20 jichen: bug fixes can be later than that 21:04:35 mikal: ok, got it ,thanks 21:04:37 edleafe: there was a proposal for another specs review day 21:04:44 #link https://wiki.openstack.org/wiki/Kilo_Release_Schedule 21:04:44 edleafe: but the date has falled out of my brain right now 21:04:47 It was next week IIRC 21:04:49 interesting 21:04:57 mikal: thanks 21:05:21 So, anything else on spec freeze before we move on? 21:05:42 mikal: so, for inclusion in kilo, code before feb 5th? 21:05:57 code should be up now for anything 21:05:58 flip214: if that code is for a feature yes 21:06:08 including for the 1/22 date 21:06:08 flip214: assuming that feature has a spec approved, or is exempt from specs 21:06:31 Oh yeah, fair point 21:06:38 The 5 Feb date is the merge deadline 21:06:43 mikal: I submitted the spec start of Dec, but it was not approved yet. Sadly, zz_johnthetubagu admitted to have forgotten it. 21:06:45 So you need to leave time for people to review as well 21:06:45 yes before FF 21:07:06 flip214: review number for the spec? 21:07:24 mikal: see http://lists.openstack.org/pipermail/openstack-dev/2015-January/054225.html for the exemption mail 21:07:36 Ok 21:07:36 spec at https://review.openstack.org/#/c/134153/https://review.openstack.org/#/c/134153/ 21:07:39 https://review.openstack.org/#/c/134153/ 21:07:44 So if an exception has already been requested then its in process 21:07:53 all right, thank you. 21:08:10 late but here 21:08:41 Moving on? 21:08:43 Is there a list of specs requesting an exception? Link? 21:08:52 edleafe: its email on openstack-dev 21:08:56 edleafe: I started making a list this morning 21:08:57 ah 21:09:04 edleafe: I will publish after this meeting 21:09:11 I gotta stop reading that at the end of the day :) 21:10:11 ["135387", "136490", "105404", "127283", "130732", "146914", "115716", "138808", "130732", "131729", "127863", "138270", "139910", "133534", "130715", "128825", "134153", "127609"] are the ones I am aware of at this time 21:10:14 is it me or the bot is on vacation now ? 21:10:23 Meeting bot? 21:10:26 yup 21:10:28 good point 21:10:28 It talked at us at the begining 21:10:33 * anteaya moves to infra 21:10:34 Anyways, let's move on 21:10:42 #topic Kilo priorities 21:10:53 So, this is our regular checkup on priority tasks 21:11:16 #link https://etherpad.openstack.org/p/kilo-nova-priorities-tracking 21:11:25 Is our list out currently outstanding priority reviews 21:11:31 I'm hoping people have kept that relatively updated 21:11:46 But I will admit I've been on vacation and haven't looked at it in the last couple of weeks 21:11:49 I know objects and cells have been kept up to date 21:11:55 Cool 21:11:56 it seemed to help in the beginning, 21:12:03 but I don't think that people are actively using it as reference anymore 21:12:07 We should add that etherpad to our agenda for the mid-cycle 21:12:14 scheduler is quite up to date too 21:12:15 As in, did that experiment work? 21:12:18 measured only by the lack of attention I think those reviews are getting 21:12:20 ++ 21:12:29 So, do we think a reminder email to openstack-dev would help? 21:12:32 mikal: I liked that approach 21:12:32 * Vek pulls up the etherpad and makes a mental note to check out some of the outstanding reviews 21:12:46 I'd really like to use a hot poker at this point, but we can start with an email :) 21:12:47 Certainly we should be mentioning it here every week as a reminder 21:12:51 Heh 21:13:13 Ok, I will send a reminder email after this meeting 21:13:16 * anteaya updates her section 21:13:24 a periodic summary to the email list would probably also be handy 21:13:57 Ok, are there any priority things which need a more explicit callout than the etherpad? 21:14:02 How are we blocking people right now? 21:14:08 (Specs are probably the most obvious thing) 21:14:58 Nothing? 21:15:12 Its good to hear we're perfect 21:15:15 I find that reassuring 21:15:16 mikal: eh 21:15:16 I think the holidays got me out of a good habbit 21:15:19 * Vek snorts 21:15:32 flashgordon: I feel the same 21:15:36 But you've been reminded now 21:15:40 So reviewy reviewy 21:15:53 yeh, that's pretty normal, it happened last year as well. 21:15:56 * flashgordon gets coffee ... linus finally shut up 21:16:01 Heh 21:16:09 Let's move on while Joe isn't looking 21:16:12 #topic Gate status 21:16:22 I've been on vacation, blissfully unaware of the gate 21:16:26 But something about eventlet? 21:16:33 I assume it was fun and someone wants to tell the tale? 21:16:44 http://lists.openstack.org/pipermail/openstack-dev/2015-January/054529.html 21:16:46 that's the tale 21:16:49 apparently, 0.16.0 was released which dropped a util.py 21:17:00 that blew up the gate, and people started working. 21:17:03 mriedem: is there any further action required? Or fixed now? 21:17:08 then 0.16.1 was released which really dropped util.py 21:17:10 mikal: there were several terrible things busted in the last few days, so it's a mess 21:17:13 mikal: fixed 21:17:15 my understanding is that it's fixed now 21:17:20 This is good 21:17:29 Who do I need to thank for the fixings? 21:17:30 however, ec2 https://bugs.launchpad.net/nova/+bug/1410622 21:17:34 yeah, eventlet wasn't the only one... 21:17:38 latest ec2 is still capped 21:17:46 Ahh yeah, I put the ec2 thing on the etherpad for the mid-cycle in case we haven't resolved it by then 21:17:53 2 tests fail, keystone complains about not getting a signature, i'm trying to debug it 21:18:24 mriedem: do you need help? 21:18:40 mikal: dims is helping 21:18:48 keystone people would be helpful? 21:18:49 Cool 21:18:51 or people that know ec2 21:18:51 Cool even 21:19:12 Cloud scaling had a bunch of ec2 people IIRC 21:19:15 Not sure where they went though 21:19:16 follow along here if yo ucare https://review.openstack.org/#/c/147601/ 21:19:21 they went to stackforge 21:19:35 We could also ping dolphm or someone like that about the keystone stuff? 21:19:50 should we dig into the ec2 reaping discussion here? 21:20:00 reaping? 21:20:01 mikal: mriedem: i popped in to keystone once, let's gather some more evidence and ping them 21:20:02 dansmith: maybe open discussion? 21:20:11 mriedem: sure 21:20:14 Ok 21:20:15 * dansmith readies his flame thrower 21:20:21 Move on then? 21:20:23 ha, like kurt russell in the thing 21:20:32 yes move on 21:20:34 #topic Mid-cycle meetup 21:20:39 Ok, so lots of people registered 21:20:52 I need to remind people that its a dev meetup and that we wont be very interesting to watch as performance art 21:21:06 I feel that the agenda etherpad needs a lot more work 21:21:12 Its been pretty quiet there 21:21:14 #link https://etherpad.openstack.org/p/kilo-nova-midcycle 21:21:16 i've been tryinhg 21:21:19 Which I guess is because of the holidays 21:21:35 my fear of advertising that is it's going to become a spec exception billboard 21:21:38 which will be annoying 21:21:45 yeah 21:22:00 So, hopefully my spec exception summary thread will distract people 21:22:00 For those of us who did not register and thus won't be there, will their be remote participation of any type? 21:22:02 When I post it 21:22:13 there probably should be a separate etherpad for spec exceptions, then 21:22:17 cburgess: nothing formal 21:22:31 cburgess: vmware is chasing their av people because the glance guys had a plan 21:22:35 I ask because of one of mriedem topics is one I agreed to tackle. 21:22:36 But nothing has come of it yet that I've seen 21:22:39 mriedem: it happens after FEE conclusion, right 21:22:59 OK... I'll wait patiently. then. 21:23:09 cburgess: or could you brief someone to cover for you? 21:23:14 yeah i don't think our IT folks will get anything - sorry 21:23:16 cburgess: or we could try and google hangout you in for that bit? 21:23:28 Yeah I can work with mriedem on it. Its the whole livesnapshot thing. 21:23:34 Ok, cool 21:23:38 mriedem won't be there right? 21:23:41 nope 21:23:42 Or not... 21:23:48 i'll be on irc 21:23:52 as normal 21:23:53 out next week 21:24:00 unfortunately, there are a lot of people not coming 21:24:07 Well I could come if there was space 21:24:10 and I think having a hangout running the whole time is probably something we should try to do 21:24:17 But I was slow on the uptake and didn't register. 21:24:22 * mriedem would have to get a working headset :) 21:24:24 Yeah I can be in a hang out for sure. 21:24:31 cburgess: I can sneak you in if you'd prefer that 21:24:33 dansmith: +1 21:24:41 hangout would be good though 21:24:41 cburgess: that is acutally possible for a very small number of cases 21:24:42 mikal: Lets take it offline. 21:24:44 dansmith, we tried that in Portland and it didn't work out that well but I don't know an alternative 21:24:56 n0ano: it worked well in utah 21:25:08 n0ano: in portland it was kinda off the cuff and wasn't as successful, agreed 21:25:14 personally, I'd rather not even try, 21:25:14 Ok, so let's take the specifics of that out of this meeting too 21:25:16 but ... 21:25:47 alternative 21:25:50 I don't feel like we'll solve that in a single IRC meeting 21:25:50 #link https://wiki.openstack.org/wiki/Infrastructure/Conferencing 21:26:06 Sorry for the distraction. tabled. 21:26:20 cburgess: it was a fair question, we just need to grind on it a bit more 21:26:41 Move on from this topic? 21:26:51 Just please do things on the etherpad 21:27:10 #topic Bugs 21:27:27 Next topic is bugs we should be worried about at the moment 21:27:49 nothing exploding the gate atm 21:27:57 Herm 21:27:58 ec2 is capped so we can work that at our leisure 21:28:04 There are two vmware bugs marked as critical 21:28:10 I thought we agreed that wasn't a thing? 21:28:11 yeh, the ec2 bug is notable though 21:28:24 i know, i'm leisurely joking 21:28:45 again https://launchpad.net/bugs/1410622 21:29:18 https://bugs.launchpad.net/nova/+bug/1374000 is the 2 year old SSL vuln bug 21:29:23 mriedem: that's the one you and dims are chasing, yes? 21:29:28 mikal: yes 21:29:53 sdague: also single driver, so not critical by definition, yes? 21:30:02 sdague: there's a review up for 1374000, am working it 21:30:10 mikal: yes, true 21:30:13 well, it was the security bug that was critical, we split it into 5 bugs critical 21:30:28 mikal: yes 21:30:29 sdague: ahh, ok 21:30:34 we can downgrade it because it's now a single driver 21:30:38 Looks like there is a fix proposed for oslo.vmware now? 21:30:39 sdague: +1 21:30:53 mikal: yes, it depends on another review and MineSweeper ok 21:30:56 but just for history. IIRC the xen version of the bug is unfixable 21:31:03 yep 21:31:19 pretty much. unless python version is at least 2.7 I think 21:31:34 and they have 2.4 21:31:39 Yeah 21:31:54 So it looks to me like people are working on the critical bugs 21:31:59 Have I missed any? 21:32:10 melwitt: 2.6 is fixable if you use requests, it's just that you can't install that in the xen env 21:32:41 sdague: yes, true. I was about to correct myself. I think you can go back farther too if you install m2crypto but that's not something that's wanted 21:33:01 #topic Stuck reviews 21:33:13 I'm not actually tracking any off the rails reviews at the moment 21:33:25 Is there anything we need to discuss or do we move on here? 21:33:28 erm, what counts as a "stuck" review (sorry for noobishness)? 21:33:31 well, https://review.openstack.org/#/c/135700/ 21:33:38 ^ is for a thing we want for priorities 21:33:40 mriedem: oh now you didnuh 21:33:41 needs more core eyes 21:33:47 anish: reviews where the reviewers can't agree without a higher bandwidth discussion 21:33:49 s/now/no/ 21:33:55 mikal: gotcha, thanks 21:34:06 dansmith is going into the 'deep bush' for a week so we should talk about whether or not it's safe to merge in his absence 21:34:22 true story 21:34:22 i need to do another pass on it, but was happy with the replies to my previous scrub 21:34:29 mriedem: well, if we trick dansmith into reviewing before he goes we'd be ok, right? 21:34:31 and melwitt found some goodies 21:34:33 ndipanov had concerns, but didn't comment them on the review 21:34:57 dansmith: when do you actually leave? 21:34:57 Oh, dan is the author 21:35:00 So can't review 21:35:15 but given the weight of this, I am (a) thinking we need more than a couple cores, and (b) need to be sure we leave time for things to 'splode before I go I think 21:35:18 wondering if ndipanov could review in time for replies 21:35:21 mriedem: I'm here monday, but that's it 21:35:31 Do we want to have dansmith WIP this until he gets back if he's still waiting for people? 21:35:32 oh so the weekend 21:35:44 could.. 21:35:51 mikal: I have a sticky -2 on it so it doesn't merge without my permission 21:35:55 Worst case we work through it at the mid-cycle, right? 21:36:01 yep 21:36:16 dansmith: is the -2 earlier in the chain? 21:36:19 dansmith: I don't see it? 21:36:20 yes 21:36:25 it's on the point of no return patch 21:36:27 Ahhhh, cool 21:36:49 it's big and fugly, 21:36:57 dansmith: https://review.openstack.org/#/c/135700/ is legitimately failing unit tests, right? 21:36:58 but see my comments on the patch about it 21:37:15 and I've discussed with a few people about how it's almost impossible to make it smaller without complicaitng things 21:37:24 sdague: is it? I rebased this morning and have been heads down on something else 21:37:37 http://logs.openstack.org/00/135700/45/check/gate-nova-python27/695fb8d/console.html#_2015-01-15_15_08_30_561 21:37:40 yeh, there's a bunch of fails that seem legit 21:37:42 oh fuckity 21:37:42 yeah, everything in the chain is jenkins-1 21:37:49 Heh 21:37:56 could be random, i had a random test failure yesterday 21:37:57 this is why I want to either merge it or kill it 21:37:59 sdague: that's new with the latest patch set 21:38:04 because it's super invasive and a rebase nightmare 21:38:10 the rpc fixture thing might be causing issues 21:38:11 db datasets is also -1 21:38:14 these are probably just new tests that need converting 21:38:24 * anteaya shakes a finger at dansmith 21:38:25 prior to that it was passing fine 21:38:25 Vek: before the rebase this morning, everything was green I think 21:38:35 yes 21:38:36 mriedem: that would only impact things talking between services, this is straight up db 21:38:50 nvm 21:38:51 File "nova/objects/instance.py", line 444, in _maybe_migrate_flavor 21:38:55 dansmith's fault :) 21:39:03 _maybe_blow_up 21:39:04 yep, new tests I'm sure 21:39:04 The system works! 21:39:18 I'll try to get those fixed up this afternoon 21:39:19 but still, 21:39:24 the contentious part is not the test results 21:39:34 so reviews are still doable :) 21:40:08 dansmith: it sounds like the usual suspects are aware of the patch, just need to get down and review it 21:40:18 dansmith: so there's no mileage in pinging people for reviews, yeah? 21:40:19 yeah, there has been lots of talk 21:40:19 well, they are now 21:40:22 not so much with the follow through :/ 21:40:28 Stupid humans 21:40:42 I've been busy enough with other things that I haven't been actively harassing people 21:40:51 Ok, so we're low on time and I think we should move on 21:40:52 if only we could automate them, eh mikal? :) 21:40:52 if it takes until the midcycle then that's fine 21:40:56 Any final words on this one? 21:41:03 but then we need to poo or get off the pooper 21:41:12 dansmith: ummmm, ok 21:41:13 heh 21:41:32 #topic nova-net to Neutron migration 21:41:38 anteaya: its your time to shine 21:41:47 * anteaya tries 21:41:51 so people are talking 21:42:08 gus tells me he's working on some tweaks to the neutron spec 21:42:12 there is a meeting: https://wiki.openstack.org/wiki/Meetings/Nova-nettoNeutronMigration 21:42:15 talkin' 'bout people? 21:42:15 But they're not ready yet 21:42:39 I am harassing gus for his promised patchset appearing here soon https://review.openstack.org/#/c/142456/ 21:42:39 The impact for us sounds like it might be a small migration command line tool which talks to conductor 21:42:53 yeah, we blessed that in paris, right? 21:42:56 Yeah 21:42:56 dansmith: this isn't a bonnie raitt song 21:42:56 there was an lca meeting: https://etherpad.openstack.org/p/nova-network-migration-lca2015 21:43:06 They're still nailing down the details though 21:43:11 But I don't think its a big problem for us 21:43:21 mriedem: +1 for catching that :) 21:43:41 trying to make it not a big problem for nova, or at leaset not a big surprise 21:43:42 So, its on the agenda for the mid-cycle 21:43:45 it is 21:43:50 And it sounds like we'll have a neutron spec we should read before then 21:43:53 To make sure we like it 21:43:58 yes 21:44:19 isn't that https://review.openstack.org/#/c/142456/ ? 21:44:29 if I have to I'll up harassing gus today since he leaves for a week of vacy 21:44:40 I've skimmed the spec 21:44:47 mriedem: yes, but with updates in flight 21:44:49 the whole thing kinda makes me nauseous 21:44:57 gus is re-writting it 21:45:09 but as long as they just (ab)use our RPC stuff and don't require us to have external APIs and such, then ... meh 21:45:14 and told me it would be up fro review yesterday but it didnt' happen 21:45:29 dansmith: well, it might be a command line tool instead of public APIs or REST 21:45:34 right 21:45:38 dansmith: which I don't think we have a particular problem with 21:45:41 right 21:45:45 So yeah 21:45:49 and it doesn't even need to be in our tree, AFAIK 21:45:51 anteaya will continue chasing gus 21:45:55 yes 21:45:58 And we shall continue being all helpful over here in the corner 21:46:04 sorry I wanted to have the revision done today 21:46:16 dansmith: I think we might want it in our tree because it talks to conductor, but we can review that on its merits at the time 21:46:19 sorry to waste your time 21:46:31 anteaya: no, its good for us to know people are still working on it 21:46:36 anteaya: and that it hasn't been forgotten 21:46:43 it hasn't been forgotten 21:46:50 Let's move on though, cause 12 minutes left 21:46:54 mikal: well, it's going to be a hack, and it's just going to slow them down if it's in our tree 21:46:55 #topic Open Discussion 21:46:59 so I don't think it matters, but whatever 21:47:06 Please enjoy your 12 minutes of open discussion 21:47:32 ec2? http://lists.openstack.org/pipermail/openstack-dev/2015-January/054117.html 21:47:53 kill it 21:48:00 as in it's busted and people are actively hacking on it in stackforge 21:48:05 I feel like we never find volunteers for it 21:48:07 but not bringing things back to nova from what i can tell 21:48:13 * Vek remembers a meeting format he once participated in which had a recurring "Sucks!" report... 21:48:16 But if we kill it there will be a lot of blow back 21:48:16 it was forked last june 21:48:34 yeah, there would need to be a migration path 21:48:37 Has anyone looked at the stackforge one recently? 21:48:41 Is it a viable alternative? 21:48:43 i looked at the commits 21:48:53 just saw it was a billion times more active than the nova ec2 api tree 21:49:15 not sure what the diffs are right now 21:49:29