21:00:29 #startmeeting nova 21:00:29 Meeting started Thu Nov 5 21:00:29 2015 UTC and is due to finish in 60 minutes. The chair is mriedem. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:30 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:00:33 The meeting name has been set to 'nova' 21:00:35 i'm not pinging people 21:00:41 \o 21:00:41 o/ 21:00:41 Well fine... 21:00:43 * Vek checks in anyway 21:00:45 o/ 21:00:46 . 21:00:47 o/ 21:00:52 o/ 21:00:53 o/ 21:00:55 * mriedem calls everyone's mom to tell them to wake up 21:00:59 o/ 21:01:02 o/ 21:01:07 #link agenda https://wiki.openstack.org/wiki/Meetings/Nova 21:01:08 o/ 21:01:09 \.....o 21:01:10 Hi 21:01:12 o/ 21:01:13 o/ 21:01:26 #topic release-status 21:01:34 spec review day on 11/19 i guess? 21:02:02 mitaka-1 is 12/1; This is also non-priority spec and blueprints freeze 21:02:16 i'm assuming that's proposal freeze 21:02:32 mriedem: correct 21:02:33 #link https://wiki.openstack.org/wiki/Nova/Mitaka_Release_Schedule 21:02:40 http://lists.openstack.org/pipermail/openstack-dev/2015-November/078558.html 21:02:43 so get your specs up before 11/19 to get them reviewed 21:02:50 and before 12/1 for sure 21:03:01 after that we only review specs that were proposed before 12/1 21:03:22 ML thread is what bauzas' linked 21:03:27 questions? 21:03:45 nova design summit etherpads are here https://wiki.openstack.org/wiki/Design_Summit/Mitaka/Etherpads#Nova 21:04:12 on to process changes 21:04:23 note the [release] tag - what's that mean? 21:04:38 anyone? beuler? 21:05:01 there are a few changes in the release process 21:05:03 release-related emails sent to the list should be tagged with that. Think that's largly to get PTL attention 21:05:09 ah ok 21:05:31 that's been around for awhile with the #openstack-release channel 21:05:40 that's where one goes to beg for releases of things, talk about schedules, etc etc 21:05:50 re: to that, I wrote something for reno 21:06:00 reno: http://lists.openstack.org/pipermail/openstack-dev/2015-November/078301.html 21:06:13 yeah, bauzas has a change up for in-tree release notes 21:06:16 link? 21:06:20 https://review.openstack.org/#/q/status:open+project:openstack/nova+topic:add-reno,n,z 21:06:48 releasing from our own stable branches: http://lists.openstack.org/pipermail/openstack-dev/2015-November/078281.html 21:06:53 in the meantime I feel we should stick with the wikipage 21:06:54 * mriedem has not read most of these in detail 21:07:01 bauzas: for now yeah 21:07:13 i updated the liberty release notes yesterday for a thing we missed, a deprecated option 21:07:33 i guess i'll read up on the stable branch release process since that's my job 21:07:57 looser milestone synchronization? http://lists.openstack.org/pipermail/openstack-dev/2015-November/078280.html 21:08:04 man the release ppl have been busy with process 21:08:22 i'm guessing that just means ttx won't herd cats across all projects for cutting m-1 21:08:23 but idk 21:08:37 release request process change: http://lists.openstack.org/pipermail/openstack-dev/2015-October/077034.html 21:08:49 the big thing there is you have to push 2 chagnes for a release on a thing, like novaclient 21:08:55 one to the releases repo for the version bump, 21:08:57 s/ttx/Doug Hellmann/ I believe 21:09:01 and one to the requirements repo for the upper-constraints bump 21:09:07 they are the same person i think 21:09:18 never see them in the same room 21:09:34 mriedem : you obviously didn't watch our conference talk ;-) 21:09:37 lol 21:09:39 smoke and mirrors 21:09:47 haha 21:09:47 finally, release liaisons http://docs.openstack.org/project-team-guide/release-management.html#release-liaisons 21:09:52 john is looking for one 21:10:17 that's doable 21:10:18 looks like that is mostly tracking status and doing admin paperwork in LP 21:10:34 any questions on any of this? 21:10:44 * dhellmann makes a note to update that section of the guide to remove the LP bits 21:11:08 ok 21:11:12 #topic blueprints and specs 21:11:15 https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking 21:11:27 that's the etherpad with the bp's and specs by category 21:11:37 Oh, cool 21:11:41 i've noticed some specs and specless bp's not on there, which isn't the end of the world, 21:11:46 but it helps the core team 21:12:17 "time to revive the code that got blocked by feature freeze" 21:12:27 so, restore changes that you still want, 21:12:31 ping people about removing -2s 21:12:31 etc 21:12:39 re-propose specs before 12/1 21:12:39 etc 21:12:45 questions? 21:13:07 specless blueprints are in the etherpad https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking 21:13:13 mriedem We should be putting approved specs on that etherpad then? 21:13:16 if you have one, post it in the section in there and we'll take a look 21:13:29 cburgess_: it's more been about unapproved 21:13:37 for teh review team, we just strike them out when done 21:14:00 Ahh ok just checking to see if I needed to do paperwork for that RBD spec that just merged or not. 21:14:01 i think it's helpful to categorize the things 21:14:06 nope 21:14:32 questoins on specless blueprints? 21:14:43 moving on 21:14:46 #topic reminders 21:15:02 mitaka review priorities, quick hit bugs and sub-team reviews are in https://etherpad.openstack.org/p/mitaka-nova-priorities-tracking 21:15:27 i haven't checked the trivial bug list in awhile, how is that going? 21:15:52 also, do we increment those merge counters? i do when i remove something from the list, but i don't know if everyone does 21:15:56 so far so good 21:16:04 +1 21:16:18 not sure what that +1 means :) 21:16:26 or you're wondering the same 21:16:28 like I do also the increment 21:16:28 to incrementing the counter 21:16:37 ok 21:16:53 but it has been a while that I haven't looked at the list 21:17:00 yeah, summit will do that for people 21:17:01 lxsli was having some patches 21:17:06 i'm reviewing more specs than code this week 21:17:23 moving on 21:17:24 #topic bugs 21:17:27 gate status: http://status.openstack.org/elastic-recheck/index.html 21:17:31 the only big nova gate issue is the cells job 21:17:34 right 21:17:38 http://status.openstack.org/elastic-recheck/index.html#1489581 21:17:47 which we have several changes up for, mostly debugging 21:18:11 we haven't hit a recreate on the unique constraint change after like 9 rechecks, nor on the change that updates the bdm object logic 21:18:11 I'm thinking of a possible change 21:18:25 mriedem: the UC is just to catch it, 21:18:28 bauzas: did you want to maybe push that so we could recheck it a few times? 21:18:31 dansmith: yeah 21:18:32 but the other one could be actually fixing it right? 21:18:40 could be 21:18:49 mriedem: I'll work on it tomorrow EU morning 21:18:51 i had a question on that one, but we cna do that in -nova 21:18:58 mriedem: and that patch isn't obviously wrong or worse, 21:19:05 so why not just clean it up and merge and get the real data on it? 21:19:11 we could always revert it if it doesn't fix it 21:19:15 +1 21:19:18 well, does BDM(device_name='foo').save() work the same for changed fields? 21:19:21 fair point 21:19:24 you could squash mine into yours 21:19:42 that was the only question on i had on that one 21:19:45 but a unit test would tell me too 21:19:49 I don't know what you mean 21:19:50 okay 21:19:58 * bauzas finding changes to leave people understanding the convo 21:19:58 i can squash and start writing tests, cleaning it up 21:20:10 cool 21:20:10 https://review.openstack.org/#/c/241742/ 21:20:14 ^ is what we're talking about 21:20:19 don't forget my CAB or I'll kill you in your sleep 21:20:24 CAB? 21:20:29 Co-Authored-By 21:20:34 oh on the squash 21:20:38 lol 21:20:41 :P 21:20:45 heh :) 21:20:47 Solely-Authored-By: Matt Riedemann 21:20:51 hehe 21:21:03 moving on 21:21:05 3rd party ci status 21:21:11 intel CI has seemed to be busted 21:21:15 intel pci is in the toilet 21:21:18 http://ci-watch.tintri.com/project?project=nova&time=7+days 21:21:23 i was going to say toilet but restrainted 21:21:25 *restrained 21:21:28 I pinged heyongli earlier, but no response 21:21:31 Do we have someone chasing that? 21:21:35 probably not 21:21:35 restrainted is cool too 21:21:36 again? sigh, I'll find out what happened 21:21:38 I'd be willing to poke some people over that 21:21:50 i'm willing to poke people w/o a reasson 21:21:53 ML can do that too 21:21:54 mikal: at the summit a coup[le of people said they were "on it" 21:21:54 honestly, I think we need to have a bigger talk 21:22:06 it's broken so often that it's just noise at this point 21:22:14 I feel like we need to have some sort of change of management on it 21:22:15 dansmith: yeah, I'm thinking its time for some sort of ops team on the hook for that thing 21:22:16 it's at least non-voting, but yeah 21:22:22 or get rid of it, but that leaves a lot of things uncovered 21:22:27 mikal: something 21:22:33 we'll need it for this sr-iov attach/detach spec 21:22:34 Ok, let me take that to some intel contacts and see what happens 21:22:46 I will report back in the next nova meeting in this timeslot 21:23:02 i usually just use 3rd party ci as a means to not approve things that are covered by those jobs 21:23:11 which is a form of motivation i guess 21:23:12 well, we need PCI CI for more than just a BP :p 21:23:19 yeah i know 21:23:39 freezing PCI changes ? heh 21:24:10 well, like i'm holding up some virtuozzo bp changes b/c of missing CI 21:24:27 anyway 21:24:39 there was one critical bug on the meeting agenda pointing to https://review.openstack.org/#/c/235303/ but that's merged as of 10/21 21:24:44 so i don't know what's up with that 21:24:47 n0ano: I'll cc you on my email 21:25:00 mikal, tnx, I'll poke internally also 21:25:02 any other critical bugs anyone knows about? 21:25:23 mriedem: I think that needs/wants a liberty backport 21:25:32 mriedem: which is in gerrit 21:25:43 ah i see it 21:25:46 i can hit that one after the meeting 21:25:58 thanks tonyb 21:26:10 #topic bug reminders 21:26:12 Triaging: Top 3 subteams with most "New" bugs: volumes: 10, libvirt: 9, vmware: 5 21:26:26 those numbers might have changed by now 21:26:39 i was looking at volume related bugs a couple of weeks ago and we could really use some focus htere 21:26:58 hard part is many of them are elaborate vendor setups that we don't have CI testing for in nova 21:27:02 at least IMO 21:27:16 mriedem: the cinder/nova session the cinder guys said they'd help triage baugs tagged with volumes 21:27:20 anyway, i usually ping cinder people on some nova volume bugs 21:27:29 lots of multipath bugs 21:27:32 from what i remmeber 21:27:43 could be fixed with os-brick in liberty and the bugs were just pre-liberty 21:27:44 not new news 21:27:46 anyway, moving on 21:27:46 yeah 21:28:01 stable branches - juno eol is happening soonish 21:28:17 apevec was asking about that the other day, so we're going to be scrubbing stable/juno changes for the final cut 21:28:25 i think that anything tha'ts wedged by requirements hell is just going to miss out 21:28:31 which puts tonyb out of a job 21:28:38 mriedem: noooooo 21:28:47 * tonyb writes an email 21:29:00 yeeesssssss 21:29:04 #topic stuck reviews 21:29:07 * dansmith plans to delete a mail 21:29:08 there is nothing on the agenda 21:29:24 I was promised a 10 minute meeting 21:29:26 so let's move on 21:29:28 i guess one i was thinking of was markuz's config option centralization spec - i'd like to see people +1/+2 that 21:29:31 haha :) 21:29:36 dude, 30 min standard 21:29:48 https://review.openstack.org/#/c/227948/ 21:29:49 I'll try to hit that config one tomorrow 21:29:53 if people cared about that, please ack ^ 21:30:20 anything else on stuck reviews? 21:30:22 Ahh it's been updated post summit ... 21:30:30 Oh, I need to re-review then 21:30:34 tonyb: yeah, plus i feel like i just keep trolling it 21:30:59 i don't care too much either way, but it impacts all dev so i feel like many people should be acking it 21:31:11 moving on 21:31:15 #topic open discussion 21:31:18 mdorman: you're up https://blueprints.launchpad.net/nova/+spec/cells-v1-interface-events 21:31:26 how to move forward on cells v1 attach/detach vifs 21:31:51 there is code up for review to add that support for cells v1, 21:32:03 but as it's a neutron only API thing, and we don't have a cells v1 + neutron CI job, that's one issue 21:32:10 another is it's adding more stuff to cells v1 21:32:15 which distracts from cells v2 21:32:38 yeah, and I'm -1 on both fronts 21:32:39 * mriedem opens up the floor for debate 21:32:44 testing is definitely a big concern 21:32:49 because I spent the last 24 hours trying to convince someone not to make me support cellsv1 21:32:52 i have a change up for testing cells v1 + neutron 21:33:02 hey sorry 21:33:02 mriedem: how close is it? 21:33:08 dansmith: i haven't looked inawhile 21:33:11 since before summit 21:33:19 if it's anything like nova-net was in terms of effort to get it passing, I'm not interested 21:33:19 internet connection issues 21:33:24 if it already works, then maybe 21:33:26 i think cells v1 + neutron ci is valuable regardless 21:33:33 but I'm also not sure I think it's useful to spend CI resources on it 21:33:35 i don't think it'll be the same effort 21:33:36 heh 21:33:43 see, here's the thing: 21:33:56 I think that people running cells today either have massively modified neutron (rax) or they're using nova-net (cern) 21:34:17 we fit in to that first category 21:34:19 well, at some point we want to test cells v2 + neutron right? 21:34:27 cv1 + neutron modified 21:34:32 we don't run a modified neutron nor nova-net. 21:34:33 mdorman: right 21:34:48 mgagne: are you nova-net or both? 21:34:59 mriedem: stock neutron only 21:35:02 ok 21:35:04 mgagne: which driver? 21:35:11 dansmith: ml2+linuxbridge 21:35:16 okay 21:35:30 mriedem: we did say that nova-net should support cells v2 21:35:30 the job i had been testing was ovs since that's devstack default still (at least i think it is) 21:35:42 but the norm definitely seems to be neutron now 21:35:47 for current cells users 21:35:48 alaski: neutron is the way of the future 21:35:56 there is no fighting it 21:36:02 so are we saying g that attach/detch patch won’t be merged w/o a test, and there is not support for putting effort into adding the test? 21:36:20 I don't think we should merge it without test coverage 21:36:22 i'm in favor of getting cells + neutron ci going 21:36:37 yeah, i think cells attach/detach depends on ci testing 21:36:44 mriedem: I am as well as that represents the biggest class of users we know about 21:37:04 keeping in mind that cells + neutron ci is not a priority 21:37:16 but if it only takes some work to get that going, it'd be worth it imo 21:37:19 alaski: and drop the n-net job or run both? 21:37:23 run both 21:37:26 dansmith: preferably both 21:37:33 or cells + neutron on experimental queue to start 21:37:38 that's the usual progression 21:37:40 I just don't think it's worth that, given we're about to lose a bunch of CI capacity 21:37:44 but if resources are constrained I would go with neutron 21:37:52 experimental is on demand 21:38:03 yeah, to start for sure 21:38:06 but prone to bit rot because it is 21:38:25 my other concern is our ability to debug cells issues on neutron once it's gating us 21:38:41 dude, i'm pretty sure that's easy peasy 21:38:49 what could go wrong 21:38:51 shhhhht 21:39:02 so to summarize, 21:39:16 i think we are -1 on mdorman's attach bp w/o cells + neutron CI 21:39:23 Well we want to run cells V2 when its out and we will be using neutron 21:39:30 yeah 21:39:43 so we should put some effort into a ci job for cells + neutron 21:39:50 which i'm half assedly started 21:39:53 let's hire jogo back 21:39:58 so is there any realistic path to getting it +2’d? is it worth putting more effort into it? 21:40:11 mdorman: get the job working 21:40:20 mdorman: yeah, we need to put effort into the ci job 21:40:29 i would probably not work on the code change until that point 21:40:31 that one you propose mriedem ? 21:40:34 *proposed 21:40:36 yeah 21:40:39 let me find it 21:40:46 does the job exist already? or would it need to be created? 21:40:47 https://review.openstack.org/#/c/235485/ 21:40:50 no 21:40:54 it's just a localrc really 21:40:56 i think you linked it on that review 21:41:05 have we seen any runs of it yet? 21:41:06 ok, found it 21:41:09 that's a lot of red 21:41:20 http://logs.openstack.org/85/235485/4/check/gate-tempest-dsvm-neutron-full/9fec667/console.html.gz#_2015-10-27_22_22_00_087 21:41:22 39 failures 21:41:28 however, 21:41:43 as i've seen with the lxc job testing, those are usually cumulative failures b/c you're hitting some test that isn't supported in that config 21:41:53 once you start pruning the list, things settle down 21:42:01 is the job you're adding even in there? I'm confused 21:42:05 no 21:42:15 it's a d-g change 21:42:19 oh, I see you're just hacking 21:42:22 to stub out the project config 21:42:23 yeah 21:42:41 i think ssh is causing issues 21:42:53 so what's the requirement? do we let them skip tests or make them all have to work? 21:43:01 ssh meaning "networking at all" right? :) 21:43:06 we start by skipping tests for things that we know don't work 21:43:19 which we do today in other ci jobs 21:43:31 we do a lot of that in the nova-net + cells job 21:43:34 I will want to know the difference between the current skip list and a neutron skip list 21:43:35 oh, this is not already skipping the ones we skip for normal cells? 21:43:39 alaski: yeah 21:43:41 it is 21:43:49 okay 21:43:50 but since it's neutron, it flips a switch in tempest to do otther things 21:43:53 that i haven't sifted through yet 21:43:56 okay 21:44:06 so, it just takes some digging 21:44:17 i got the lxc job down to 3 failures 21:44:19 mriedem: do you have time for said digging? 21:44:21 so, really, if we're going to do this, 21:44:28 tonyb: maybe 21:44:30 why aren't we just letting people submit any features to cellsv1? 21:44:58 dansmith: feature? you mean "feature parity" ? 21:45:00 imo the attach/detach change is pretty small 21:45:13 tbh, that’s what we’d like to be doing. many larger deployers are carrying a bunch of local patches for v1 stuff. many of them are common among many of us, so we'd like to get them in upstream. 21:45:14 mgagne: yeah, we specifically said we weren't going to do this, even for parity 21:45:26 mdorman: we know 21:45:30 dansmith: I think this is mostly about getting in changes that operators are sharing amongst themselves 21:45:36 alaski: I understand that 21:45:36 i'm of the opinion that if ops are running with these things already and we can get them in with little impact and tested, then i don't think there is a big problem with it 21:45:41 kk 21:46:01 so are we opening up to anything that is parity and bugs >1 operator? 21:46:16 and + IC 21:46:20 *CI 21:46:32 I think we evaluate based on code impact, similar to stable branches 21:46:37 right 21:46:48 that seems fair to me 21:46:51 i went through the ops cells v1 patches etherpad at one point, there are some things in there we just wouldn't do 21:46:52 for v1 21:47:20 so to take an actual example, we won't see aggregate/flavor update support in cells v1 21:47:24 anyway, let's see if we can get the neutron + cells thing to a sane level, and go from there 21:47:31 okay, well, I'm completely -1 on this plan, but I'm clearly in the minority 21:47:49 if we don't get the job going it's kind of a moot point 21:48:03 yep 21:48:17 anything else on this? 21:48:33 any other open discussion? 21:48:41 i'm good. thanks for the discussion and direction, this is helpful 21:49:04 cool - i'd like a lot more comms between ops and nova really 21:49:06 this is part of that 21:49:10 more so than nova + vendors 21:49:28 if nothing else, let's end early 21:49:39 \o/ 21:49:40 +1 21:49:44 #endmeeting