21:00:51 #startmeeting nova 21:00:52 Meeting started Thu Dec 18 21:00:51 2014 UTC and is due to finish in 60 minutes. The chair is mriedem. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:53 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:56 The meeting name has been set to 'nova' 21:00:56 o/ 21:00:58 o/ 21:00:59 o/ 21:00:59 o/ 21:01:02 o/ 21:01:02 raise em up 21:01:03 o/ 21:01:04 o/ 21:01:04 o/ 21:01:14 agenda: https://wiki.openstack.org/wiki/Meetings/Nova#Agenda_for_next_meeting 21:01:18 :w 21:01:23 arrgghhh 21:01:28 #topic kilo-specs 21:01:43 well today is the day for k-1 deadline on spec approvals 21:01:52 o/ 21:01:54 i believe ttx is cutting k-1 today sometime 21:02:01 Non-priority Feature Freeze is kilo-2 (February 5th) 21:02:25 anything that didn't make the approval deadline (today) has to get an exception, and that process is TBD until people get back from vacation 21:02:48 sounds like there will be a meeting the first week of january to iron those detials out, and then i'm assuming ML for the process 21:02:50 mriedem, so we don't even know the process for another 2 weeks 21:02:50 o/ 21:03:03 o/ 21:03:12 n0ano: right 21:03:18 o/ 21:03:32 o/ 21:03:33 n0ano: well, i'd think there is some exception request on the ML, priority will be important, and probably people signed up to review them 21:03:35 but i'm guessing 21:03:49 it just leaves us in limbo for a long time 21:04:12 well, there are things people can do in the interim 21:04:21 like help review the k1 approved specs, 21:04:30 get code in shape for anything proposed as a priority exception for k2, etc 21:04:47 review this https://review.openstack.org/#/c/135700/ :) 21:05:00 * dansmith slips mriedem a $20 21:05:03 re-approve blueprint fast-track - anything needing re-approval 21:05:04 ? 21:05:16 i honestly haven't checked 21:05:36 re-approval is just for things early in the cycle, 21:05:39 i had this in open discussion but the agenda talks about spec free bp's, there was one here https://blueprints.launchpad.net/nova/+spec/network-template-routes-injection 21:05:41 so we can probably start asking about this I think 21:05:47 right, I think we can take that off the agenda now 21:05:49 \o 21:05:52 jogo: ok 21:06:03 regarding https://blueprints.launchpad.net/nova/+spec/network-template-routes-injection - i don't think it needs a spec given the guidelines 21:06:14 mriedem: I thought there was already a spec for that 21:06:17 i can't pretend to understand it really, not really my area, but it's up there 21:06:24 jogo: no, it was reported as a 'nova should do this' bug 21:06:31 and a patch, which i said open a bp 21:06:45 this is for nova-network right? 21:06:50 https://review.openstack.org/85673 21:06:51 dansmith: yeah, i added vishy to the review 21:07:05 ohh nova-network ahh 21:07:15 https://review.openstack.org/#/c/115409/ 21:07:38 anyway, moving on from that - take a peek if you care, i don't think spec is required 21:07:52 can we just restate that priority reviews can be set there https://etherpad.openstack.org/p/kilo-nova-priorities-tracking ? 21:08:00 just about to do that 21:08:03 #topic priorities 21:08:05 #link https://etherpad.openstack.org/p/kilo-nova-priorities 21:08:06 mriedem: agreed sounds like a good candidate for no spec 21:08:10 #link https://etherpad.openstack.org/p/kilo-nova-priorities-tracking 21:08:34 top three are still cells, objects and scheduler 21:08:50 mriedem dansmith: should I approve https://blueprints.launchpad.net/nova/+spec/network-template-routes-injection ? 21:08:54 i don't know if there is much to report here? alaski dansmith jaypipes ^ ? 21:08:58 jogo: up to you 21:09:04 * dansmith doesn't care much 21:09:18 * jogo approves 21:09:20 alaski has cells specs and devref stuff up for review, so we could get some eyes on them 21:09:23 I can proxy jaypipes's voice 21:09:24 mriedem: the objects status has pretty much been the same for weeks, but I'll do some updates there 21:09:25 did the cells specs get approved yet? 21:09:30 in re: scheduler - most important thing is review for https://review.openstack.org/#/c/127609/ 21:09:31 mriedem: some 21:09:31 i don't think they did 21:09:35 two cells specs are approved 21:09:45 dansmith: alaski: ok, i'm assuming anything that didn't would be exception fodder anyway 21:09:51 two others are up for review 21:09:59 * alaski needs to add it to the etherpad 21:10:07 #action need people to review https://review.openstack.org/#/c/127609/ for the scheduler bp priority 21:10:18 mriedem: I think we need to discuss when everyone is back from the holidays 21:10:18 #action alaski to update priorities etherpad for cells specs 21:10:30 alaski: yeah - i'm just drunk with power here 21:10:40 mriedem: as I said, most important reviews for the scheudler are in the etherpad 21:10:53 bauzas: n0ano: ok, thanks 21:10:55 https://review.openstack.org/#/c/127609/ will need an exception 21:11:10 re: the flavor blob one, I don't think there is any way to get serious review on it before the break, so I presume we'll just hold the -2 on that one until January 21:11:11 if we consider k1 as the spec freeze 21:11:43 dansmith: yeah i don't think so either, i guess you could address what i pointed out today and then it will sit for a bit 21:11:50 yep, will 21:12:19 i'm technically out for the rest of the year but with the inlaws staying with us next week i'm sure i'll be online poking around 21:12:28 moving on 21:12:31 #topic gate 21:12:36 the gate is good http://status.openstack.org/elastic-recheck/gate.html 21:12:46 90% classification of bugs 21:12:50 which is as high as i can remember 21:12:50 \o/ 21:13:10 if it craps over the break i guess put out the jogo signal :) 21:13:23 #topic bugs 21:13:27 I have a bug thing 21:13:28 there was nothing on the agenda about bugs 21:13:29 ok 21:13:31 https://bugs.launchpad.net/nova/+bug/1402813 21:13:34 Launchpad bug 1402813 in nova "https://review.openstack.org/#/c/91722 breaks icehouse->juno live migration" [Undecided,New] 21:13:35 apparently, in juno, 21:13:42 we intentionally broke live migrations across icehouse->juno 21:13:52 which breaks the way a lot of people upgrade without dropping workloads 21:14:10 I think this needs to be classified highly, and we need to fix master and backport to juno 21:14:12 thoughts? 21:14:14 is that only for people using shared storage? 21:14:17 no 21:14:23 it should be only for shared storage on RBD, 21:14:29 but it was just broken across the board 21:14:31 wow 21:14:31 for no reason 21:14:34 I know. 21:14:41 i remember that patch, and there was some hate 21:14:52 it didn't do this in the early versions 21:15:06 but the one that merged calls it out in the commit message clearly 21:15:08 so anyway, 21:15:20 I just wanted to see if others felt it was a critical issue like I do 21:15:29 i marked it critical 21:15:36 and I'll work on either helping the author make the change, or just do it (when I get back) 21:15:38 okay 21:15:45 i think for the people i've been talking to about upgrades to kilo, they are going to be doing cold migrates anyway b/c of distro upgrades 21:15:54 well, 21:16:00 but that's not everyone 21:16:00 wow the upgrade impact discussed the issue 21:16:11 we do live migrates to allow people to do distro upgrades without dropping workloads 21:16:21 ++ to critical 21:16:22 like, live migrate from RHEL6 to RHEL7, icehouse to juno 21:16:24 works well 21:16:31 except when we intentionally break it :( 21:16:43 yeah, +1 to critical. this is a pretty big break 21:16:50 dansmith: yeah, we're talking 6.5 to 7 for juno/kilo here 21:16:55 right 21:17:05 so cern will hit this too... 21:17:11 b/c they are upgrading distros 21:17:14 I think lots of people are going to hit it 21:17:17 yeah 21:17:18 ok 21:17:21 marked as critical and for k2 21:17:21 probably right after the break when they sit down to do upgrades :) 21:17:30 cool, thanks 21:17:45 #action fix https://bugs.launchpad.net/nova/+bug/1402813 if you want happy users 21:17:46 Launchpad bug 1402813 in nova "https://review.openstack.org/#/c/91722 breaks icehouse->juno live migration" [Undecided,New] 21:17:59 any other critical bugs? 21:18:12 dansmith: maybe make ndipanov fix it :) 21:18:18 heh 21:18:21 he has enough to do 21:18:25 blargh 21:18:37 #topic stuck-reviews 21:18:46 i think this pacemaker one is old 21:18:48 no link 21:19:04 beagles has https://review.openstack.org/#/q/status:open+project:openstack/nova-specs+branch:master+topic:bp/nova-neutron-refactor,n,z on here for the neutron v2 api refactor 21:19:15 there are 3 specs there, which missed the deadline obviously 21:19:32 mriedem: about Pacemaker, you mean https://review.openstack.org/#/c/139991/ ? 21:19:47 i think we can stagger those in kilo still and are technial debt priority, so at least one or two might get exceptions if done right 21:19:53 since this is in support of neutron usability, and non-damaging to the mainline while developing, I think an exception for this work to start/continue is cool, but we can talk about that in Jan 21:19:55 bauzas: looks right 21:20:04 dansmith: agreed 21:20:16 can I bring up a review I'd just like to get core eyes on? https://review.openstack.org/#/c/141188/ this is to improve efficiency with the ironic driver using the ironic client 21:20:17 looks like https://review.openstack.org/#/c/139991/ isn't stuck... 21:20:31 mriedem: that is dead on arrival IMHO 21:20:35 mriedem: agreed 21:20:38 clif_h: that doesn't look 'stuck' to me 21:20:42 mriedem: it seems there is a consensus 21:20:55 mriedem: sorry, is there a better time to bring this up? 21:21:05 clif_h: open discussion 21:21:06 clif_h: at the end i guess 21:21:08 right 21:21:14 alright, I will, thanks 21:21:30 another was host health monitoring: https://review.openstack.org/#/c/137768/ 21:21:41 doesn't look stuck, 21:21:46 and doesn't look like a priority for kilo 21:21:58 is this another spec about auto-evacuate/migrate or sometihng? 21:22:22 not quite, but along those lines 21:22:25 yup, auto-evacuate 21:22:30 hw failure on your intel box :) 21:22:42 we have a few SR-IOV related specs for cores' attention. Wonder if they still have a chance 21:22:56 mriedem: I don't think it's stuck, there's been discussion on it recently 21:22:57 baoli_: let's leave that for open discussion 21:23:01 alaski: agreed 21:23:06 mriedem: https://review.openstack.org/#/c/137768/ isn't stuck too 21:23:09 mriedem, sure 21:23:24 moving on to https://review.openstack.org/#/c/136487/ 21:23:28 remove server group members 21:23:37 maybe it would be good to reexplain what a stuck review means... 21:23:42 dansmith is -1 b/c he wants remove/add done together, 21:23:50 hell yes I do 21:23:51 i was hoping that johnthetubaguy would review https://review.openstack.org/#/c/136487/ and throw his input in 21:24:15 * johnthetubaguy noticed he is mentioned, looks at review 21:24:16 it was discussed at the summit so was hoping that others involved in that discussion could chim in 21:24:21 *chime in 21:24:28 personally i don't have a dog in that fight so don't care either way 21:24:37 mriedem: have you looked at the add spec yet? 21:24:39 i haven't reviewed the add spec either 21:24:40 nope 21:24:41 I previously had add/remove together, but as discussed at summit split these into two specs to stage remove in for kilo and add later. 21:24:56 dansmith: did you look at the add spec? 21:24:58 assumes yes 21:24:59 mriedem: yes 21:25:04 I've been avoiding commenting 21:25:04 problems? 21:25:16 it adds a ton of new DB traffic to try to avoid the hard problems 21:25:17 * mriedem assumes jmulsow has thick skin 21:25:46 I'm not sure it covers all the bases, although it covers a bunch, but the cost for it is high, IMHO 21:25:52 ok, so the concern is we do remove w/o add and if we can't decide on a design for add, we're stuck w/o it 21:25:58 BTW My driver (mikal) says "hi" 21:26:04 not having read the specs yet, I agree with add/remove being in the same spec. 21:26:12 and then we're stuck with weird can-remove-but-not-add API semantis 21:26:15 er, semantics 21:26:25 right now it's symmetric and easy: 21:26:26 and it's not weird to not be able to remove at all? 21:26:32 you get into a group on birth and leave on death, period. 21:26:37 honestly we have enough other stuff going on I this doesn't seem worth dealing with now 21:26:54 dansmith: like a hard core gang? 21:26:55 jogo: feel free to weigh in on the spec then, you were at the summit session too 21:26:56 jogo: a number of folks hate server_groups as it is, so yeah, that 21:27:00 jogo: ha 21:27:02 jogo: lol 21:27:09 don't the HP guys like server groups? 21:27:17 * jogo pulls out the -2 hammer 21:27:19 PhilD and such? 21:27:24 I dunno 21:27:25 To me it seems less confusing to have just remove than it is to have to figure out you need to delete the VM to get it out of the group 21:27:37 * gilliard not sure can check with PhilD tomorrow. 21:27:44 mriedem, that's my understanding, yes PhilD likes server groups 21:27:46 gilliard: thanks, i added him too just in case 21:27:57 PhilD must be busy externally lately... 21:28:07 anyway, we can move on - anyone interested or has input, please get it in the review 21:28:27 Comments in the add review would be apreciated, too, so they can be addressed 21:28:35 jmulsow: link? 21:28:56 https://review.openstack.org/#/c/139272/ 21:28:59 #topic open discussion 21:29:01 * jogo -2ed 21:29:02 there was one item 21:29:03 Libvirt meetings. https://wiki.openstack.org/wiki/Meetings/Libvirt - Do they happen any more? Someone else was in that meeting room at the time mentioned there, and no minutes for months. 21:29:13 i don't know if danpb is doing those still 21:29:20 I went along and it was just a bunch of people talking about lbaas 21:29:25 gilliard: ha 21:29:29 I don't think there was this last time at least 21:29:29 almost the same 21:29:40 ok, i guess that's a better ML question 21:29:41 I saw people asking about it 21:29:53 #action post the libvirt meeting question to the ML 21:30:00 I'll do that 21:30:04 gilliard: thanks 21:30:08 ok, other open discussion? 21:30:22 no meetings until January? 21:30:28 ++ 21:30:34 tonyb: can you ask mikal ^? 21:30:38 i won't be attending 21:30:41 I am 21:30:54 mriedem: can we discuss the sr-iov specs now? 21:30:57 I'd like to get core eyes on this review if possible: https://review.openstack.org/#/c/141188/ this is to improve efficiency with the ironic driver using the ironic client 21:31:27 start again for the first thursday (2nd)? 21:31:35 tonyb: sounds good 21:31:46 That is an odd week, right? 21:31:53 clif_h: ok, it's not that old though, be patient :) and it's a wishlist bug 21:31:58 the second is friday 21:32:04 Thursday is the 1st 21:32:06 so it will be the eight 21:32:07 h 21:32:09 gilliard: Yeah I guess so, unless we start at 0 21:32:13 i have the nuke-xml reviews lined up 21:32:24 and the ext3->ext4 as well 21:32:25 1/1 won't happen for a meeting i don't think, at least in the US 21:32:33 baoli_: go ahead 21:32:36 dims__: sweet 21:32:37 8th Jan meeting is at 1400UTC 21:32:44 mriedem: sorry :) Is there any expected timeframe to get reviews reviewed? 21:32:46 just to be clear 21:32:50 baoli_: if they are for NFV, i don't think that's on the priority list though.. 21:32:56 okay start then 21:33:01 clif_h: no, we just have a lot 21:33:07 clif_h: feel free to help review some others 21:33:24 mriedem: the bug clif_h is working on is a wishlist bug in the tracker; but is impacting any Ironic installation of size (1 call to identity and 1 call to Ironic for every Ironic instance) 21:33:42 So there's a strong desire in Ironic to get those fixes in. 21:33:57 sure, i'm not telling people to *not* review it 21:33:58 mriedem: we discussed these items during the summit 21:33:59 i'm just saying 21:34:04 baoli_: links? 21:34:15 i've got the network json spec i'd love to get in before the deadline, needs 1 more +2, previously had +2 from dansmith and johnthetubaguy, and hasn't changed significantly since then. i've got the nova code proposed already, and we've got configdrive support ready. https://review.openstack.org/#/c/85673/ 21:34:37 dansmith: johnthetubaguy: ^? 21:34:44 https://blueprints.launchpad.net/nova/+spec/sriov-live-migration https://blueprints.launchpad.net/neutron/+spec/api-specify-vnic-type https://blueprints.launchpad.net/nova/+spec/sriov-interface-attach-detach 21:34:45 * dansmith blinks 21:34:46 might be easy pickins before an exception requirement 21:34:59 hopefully :) 21:35:15 Cinder iSCSI multipath & failover specs ( https://review.openstack.org/#/c/134299/ and https://review.openstack.org/#/c/137468/ ) was -1'ed just because of waiting for cinder-spec approval. Now cinder-specs are approved, so I'd like cores to recheck these. 21:35:16 one more: https://blueprints.launchpad.net/nova/+spec/sriov-sched-with-stateless-offloads 21:35:23 mriedem: how are we doing with the kilo-1 goals we set at the summit? 21:35:34 jogo: we had goals 21:35:34 ? 21:35:35 JoshNang: I was proxying a +2 back on there, I think my last vote was -1 :) 21:35:49 jogo: mriedem: :) 21:35:54 jogo: like CI for NFV? 21:36:01 baoli_: is there CI for SRI-OV yet? 21:36:01 JoshNang: I can take a look, I do like the idea 21:36:03 dansmith: ah, right you are 21:36:07 * jogo digs up the link 21:36:07 baoli_: is there CI running somewhere in the world for NFV yet? 21:36:07 johnthetubaguy: thanks! 21:36:18 so, I can say some words on that 21:36:20 NFV CI 21:36:34 there are still a bunch of things that will require real hardware, of course, 21:36:36 mriedem: as far as I know, folks are working on them 21:36:40 baoli_: so that's like what 4 bps/specs there, and i don't think NFV is a project priority for kilo, the scheduler stuff trumps that as i recall 21:36:46 johnthetubaguy: dansmith: FWIW we're running a downstreamed version of that JSON spec/code downstream, and also have code waiting for cloud-init to support that JSON format, so really we're just waiting on the spec to get done :( 21:36:50 but I think we have nailed down some very real ways to test NUMA on existing CI systems without much trouble 21:36:53 baoli_: they were also supposed to be working on them in juno 21:36:56 Wanting to make sure we have reached a consensus for add/remove server group members. Does everyone agree that going forward in proposing these for the L release, these two specs need to be merged back together? 21:37:19 jmulsow: that's the way i'd lean 21:37:35 mriedem, NFV was never a priority for Nova, it might be for Neutron 21:37:52 dansmith: ok, i have no idea how related that is as a dependency for the 4 bp's that baoli_ linked above 21:37:58 ahhhttps://etherpad.openstack.org/p/kilo-nova-milestone-targets 21:38:02 #link https://etherpad.openstack.org/p/kilo-nova-milestone-targets 21:38:12 baoli_: is there a way to prioritize / stagger these specs? 21:38:23 mriedem: it's not at all, but you brought up NFV CI and numa was one of the things we identified as untested stuff that needed testing, and was previously hung on requiring hardware 21:38:39 mriedem, well, these specs are worked on by various folks. 21:39:07 without CI for SRIOV I don't like the idea of adding more features to it 21:39:11 But we need cores' attention so that we can move forward 21:39:35 baoli_: yeah, core attention is on many things, and nfv isn't a priority for kilo given the list 21:39:38 jogo: we also found a couple critical issues with it right after we released juno 21:39:43 baoli_: and there are many bp's 21:39:46 dansmith: exactly 21:39:50 metion a bug about network https://review.openstack.org/116578 , it break network after migration, even l2 pop doesn't work afte migration. l2 pop is importart feature for neutron also. The fix alread there for sometime 21:40:05 so based on https://etherpad.openstack.org/p/kilo-nova-milestone-targets 21:40:09 baoli_: i think from the nfv session we were hoping for more CI before more bp's, since it was the other way around in juno 21:40:11 we are waayyy off 21:40:25 jogo: we've hit some of the k1 things 21:40:41 who was helping the CI happen? 21:40:44 mriedem: want to go down the list 21:40:46 tonyb: good questoin 21:40:53 alex_xu: yeah, this just needs another +2 https://review.openstack.org/#/c/116578/ 21:40:53 ian from HP 21:41:02 Ian from cisco said he had hw 21:41:03 jogo: did you ever talk to ian wells 21:41:12 oh, thought he was HP 21:41:13 lots of people said they had lots of stuff 21:41:15 jogo: this prioriryt milestone list maybe deserves an update 21:41:29 mriedem, the current CI is testing API only. But more stuff is being added 21:41:36 jogo: because for example some blueprints are not fully merged, but some patches in the series laned 21:41:37 mriedem, yes, thanks for your +2, hope can merge, i think it really break some basic function 21:41:40 *landed 21:42:11 johnthetubaguy: thanks for the review :) 21:42:11 mriedem: no not sure what happened to him, I thought I said they have to initiate 21:42:30 baoli_: so unless others disagree, i think it might be helpful to take something to the nova mailing list and recap where we are with NFV, what specs are up and which are the most important (candidates for exceptions) and what is going on with the state of that CI 21:42:42 mriedem: ++ 21:42:58 mriedem: jogo, I know our team member working on PCI CI. 21:43:01 mriedem: can w e go down the list of 8 or so kilo 1 targets 21:43:06 mriedem: ++ from me 21:43:10 jogo: yeah, sec 21:43:14 mriedem: no idea about mikal 21:43:24 mriedem, ok, sounds good. 21:43:31 ok, going down the k1 milestone targets list 21:43:35 1. cells 21:43:35 cells 21:43:43 you go for it 21:43:45 alaski is working through the testing gaps, there is an etherpad 21:43:52 testing: down to around 40 failures from 153 to start 21:43:54 actually the wiki 21:43:57 #link https://wiki.openstack.org/wiki/Nova-Cells-v2 21:43:57 so.. progress 21:43:59 I think we're doing or have done exactly what the k1 bit says 21:44:07 +1 21:44:08 awesome, progress is good 21:44:09 yeah, 2 specs are merged, 2 are in flight 21:44:17 getting scheduler requirements, specs in progress some merged, testing is going 21:44:17 there are well attended weekly meetings 21:44:20 we are off to a good start :) 21:44:23 yup 21:44:29 2. glance client library 21:44:31 i have no idea on that 21:44:33 yes, things are going well re: cells 21:44:34 flaper87: ^? 21:44:35 I think we merged something on that 21:44:39 the spec 21:44:45 dansmith: yeah we dod 21:44:49 did 21:44:50 so that's good 21:44:55 yeah, +1, saw that go in 21:45:12 was mean to be implemented, but its a step forward 21:45:18 so maybe a bit behind but making strong progress on that as well 21:45:19 http://specs.openstack.org/openstack/nova-specs/specs/kilo/approved/remove-glanceclient-wrapper.html ? 21:45:27 that one 21:45:29 ok 21:45:42 3. neutron - grenade testing with neutron is non-voting 21:45:50 mestery: mtreinish: ^? 21:45:50 I haven't heard anything about this 21:45:58 mestery: ^ 21:45:59 jogo: know anything about that? 21:46:03 guess not :) 21:46:06 oh hmm 21:46:12 I did see something let me check 21:46:14 #action mriedem to the ML on neutron grenade testing 21:46:24 mriedem: there has been no progress on a sideways job yet that I've seen 21:46:26 Yes, this is due to the services split, we're close. 21:46:28 armax: ^^^ 21:46:38 mestery: neutron -> neutron is voting 21:46:43 sdague: Ah 21:46:51 ok, so people are working it i assume? 21:46:52 sdague, mestery: that’s my understanding too 21:46:57 but $dependencies 21:46:59 there is no nova -> neutron effort as far as I know 21:47:05 sdague: Ah, ok, that makes sense. 21:47:09 https://review.openstack.org/#/c/142741/ 21:47:12 its voting and working 21:47:16 sdague: you mean the sideways job? 21:47:22 sdague: is this supposed to be testing n-net to neutron? 21:47:31