21:01:23 <mikal> #startmeeting nova
21:01:24 <openstack> Meeting started Thu Jul 17 21:01:23 2014 UTC and is due to finish in 60 minutes.  The chair is mikal. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:01:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:28 <openstack> The meeting name has been set to 'nova'
21:01:28 <dansmith> o/
21:01:31 <thingee> o/
21:01:31 <mriedem> hi
21:01:32 <leifz> o/
21:01:40 <mikal> Heya
21:01:43 <alaski> hi
21:01:47 <mikal> So, let's get this show started
21:01:52 <mikal> #topic Juno-2
21:01:57 <russellb> o/
21:02:01 <mikal> #info j-2 is 24th July
21:02:02 <jaypipes> o/
21:02:04 <mikal> Which is kind of soon
21:02:09 <hemna> howdy
21:02:21 <mtanino> o/
21:02:26 <mikal> So, there's a few j-2 things we should quickly cover
21:02:34 <mikal> #info Reviewers, please focus on j-2 targeted things if possible
21:02:40 * russellb is ttx for juno-2 next week, fyi
21:02:56 <mikal> Yes, and a love ttx you are too
21:03:02 <russellb> heh <3
21:03:05 <mikal> The timeline for the next release is at:
21:03:13 <mikal> #link http://lists.openstack.org/pipermail/openstack-dev/2014-June/038475.html
21:03:31 <mikal> We also need to talk about specs who have requested a spec freeze exception
21:03:40 <hemna> :)
21:03:46 * mikal waits for the page to load
21:03:57 <dansmith> damned NBN
21:04:06 <mikal> dansmith: don't go there
21:04:18 <dansmith> I don't even know if you're on the NBN or waiting for it
21:04:25 <mikal> No, 2mbit DSL for me
21:04:28 <mikal> But anyways...
21:04:41 <mikal> So, we had a list of priority specs based on the summit back in the day
21:04:44 <mikal> #link https://etherpad.openstack.org/p/nova-juno-spec-priorities
21:04:53 <mikal> We should use that as a guide for spec freeze exceptions
21:05:08 <mikal> But nova-drivers can work through that in the spec repo
21:05:10 <dims> o/
21:05:28 <mikal> Its the first time we've done specs, so I imagine the spec freeze process will involve a little discussion about the process as well as the specs themselves
21:05:37 <hemna> multi-attach isn't in that list unfortunately
21:05:45 <russellb> shall we cover requests now?
21:05:55 <russellb> the -2 comment on specs said we would take requests in the meeting
21:05:56 <mikal> So, I'm only aware of two
21:06:00 <mikal> But more might have come up overnight
21:06:03 <mikal> I just woke up
21:06:05 <mikal> russellb: sure, go for it
21:06:08 <russellb> mikal: see "Juno Spec Freeze" section of that etherpad
21:06:20 <russellb> lots listed as potential, some with sponsors
21:06:30 <russellb> earlier today here, i was proposing that we require core review sponsors to get something approved
21:06:32 <mikal> Ugh, that's quite a list
21:06:54 <russellb> so, if something has -core that cares enough to promise review effort on it, that should be strongly considered for an exception
21:07:00 <hemna> I was hoping to add multi-attach to the list if possible ?
21:07:03 <russellb> unless we think it's just way too disruptive to merge this late no matter what
21:07:15 <russellb> that's how i'd think about it, i think
21:07:58 <mikal> russellb: I think that's wise. Two cores as sponsors would be even better because then we can point the finger at them for reviews
21:08:05 <russellb> mikal: indeed
21:08:17 <mikal> hemna: have you sent an email to openstack-dev requesting one?
21:08:20 <russellb> so, require at least 2 sponsors, and no major vetos saying it's too disruptive to do late in the cycle?
21:08:23 <russellb> as the criteria?
21:08:32 <mikal> hemna: I think we should ask people to make a (brief) case on the mailng list, and then we can take it from there
21:08:34 <hemna> mikal, not currently.   I can do that though.
21:08:35 <russellb> and if so, maybe do a pass on the ones that have at least 2 already listed now
21:08:41 <mikal> russellb: that sounds reasonable to me
21:08:44 <russellb> cool
21:08:47 <mikal> hemna: please do, ASAP please
21:08:51 <hemna> will do
21:08:55 <russellb> so, first one with at least 2 sponsors is
21:08:57 <russellb> #link https://review.openstack.org/#/c/95025/9/specs/juno/deprecate-baremetal-driver.rst
21:09:03 <mikal> I am sure we'll tweak the plan as we learn how to make it work, but that's a good start
21:09:06 <russellb> john, dansmith, and alaski on that one
21:09:08 <mtanino> mikal:  I would like to have a exception too
21:09:28 <mikal> mtanino: email to openstack-dev please?
21:09:32 <russellb> thoughts on deprecate-baremetal-driver?
21:09:36 <mtanino> mikal: not yet. So I will send a mail too.
21:09:43 <mikal> We need ironic first for that one
21:09:44 <russellb> actually
21:09:52 <russellb> devananda is going to try to join us to talk about that
21:09:57 <devananda> mikal: so the conversation earlier today was that some folks dont want to acept ironic w/o the deprecation
21:09:59 <dansmith> yeah, no.. I think this should have been part of the ironic spec,
21:10:01 <devananda> hi!
21:10:01 <russellb> devananda: o/
21:10:04 <dansmith> so I think it needs to be tied to the other one
21:10:17 <russellb> agree with dansmith
21:10:26 <dansmith> but, I also think that it's in reasonable shape so I'm not too worried
21:10:28 <mikal> devananda: I thought deprecation was part of the plan from the summit, is that not your recollection?
21:10:29 <russellb> i think being able to deprecate baremetal needs to be tied to merging the ironic driver
21:10:32 <dansmith> just don't want to drop it because it's after the deadline
21:10:45 <alaski> agreed
21:10:54 <russellb> i haven't reviewed the spec, but i view it as ironic drive + this, or neither
21:10:56 <devananda> mikal: yes, both important, and both should happen
21:11:00 <mikal> devananda: I also believe that upgrade is progressed, but I haven't heard much about it?
21:11:10 <devananda> there is code proposed for the upgrade
21:11:12 <devananda> but not yet any testing of it
21:11:20 <mikal> In your repo or outs?
21:11:22 <mikal> ours even
21:11:22 <devananda> yours
21:11:33 <mikal> Cool
21:11:48 <devananda> https://review.openstack.org/#/c/101920/ and https://review.openstack.org/#/c/102563/
21:11:49 <mikal> I think this is complicated enough that perhaps we should ask for an email describing its status
21:11:51 <russellb> anyone *not* think this should get an exception?
21:11:57 <mikal> What's implemented, where its proposed, next steps, etc etc
21:12:01 <russellb> mikal: want to defer approvals to ML?
21:12:07 <russellb> or try to sort some now?
21:12:09 <mikal> russellb: this is a big list to get through
21:12:20 <russellb> well, was thinking we'd just start with ones that have sponsors listed already
21:12:21 <devananda> i'm happy to give an email update, but i'd like to sort one question real quick
21:12:25 <mikal> Let's cherry pick out the most important / obvious ones now and do the rest on the ML
21:12:26 <russellb> up to you
21:12:29 <russellb> k
21:12:36 <devananda> do we (and if so ,how) need grenade testing of the upgrade before we can *land* the driver?
21:12:46 <dansmith> I think we do
21:12:57 <devananda> that seems like another chiken and egg problem which took us a few days at the ATL summit to detangle
21:13:04 <mikal> How do we do that if baremetal never supported devstack?
21:13:08 <devananda> right
21:13:20 <devananda> there is *no* testing of baremetal right now
21:13:23 <devananda> so how do we test upgrades from it?
21:13:30 <russellb> hrm, fair point ...
21:13:37 * dansmith headdesks
21:13:39 <mikal> Adding baremetal devstack is a pretty big tangent
21:13:40 <degwea> Hi, can you please consider https://review.openstack.org/#/c/99615/ too? It was submitted in June and looked fine but was not approved.
21:13:46 <mikal> And probably not something juno has time for
21:13:55 <russellb> maybe for now we agree this spec should be approved, but let's sort out testing details on ML ?
21:13:57 <dansmith> so, the things we need to test are pretty small, right?
21:14:11 <mikal> Yeah, let's be brutal here
21:14:17 <mikal> We're inclined to grant an exception
21:14:28 <mikal> But we want to see a status email on -dev, and to have a quick discussion about upgrade testing first
21:14:33 <devananda> dansmith: can baremeteal spin up some instances is the first criteria before we can even do a grenade test of an upgrade
21:14:40 <mikal> We will then talk about that exception in that thread
21:14:41 <devananda> dansmith: i have no confidence in the first criteria today
21:14:42 <russellb> #agreed exception granted for deprecate-baremetal-driver, pending agreement on test requirements
21:14:49 <mriedem> degwea: request an exception on the -dev mailing list
21:15:13 <mikal> devananda: tripleo uses baremetal though right?
21:15:15 <angdraug> degwea: see above, the process is 1) get email to openstack-dev; 2) add to https://etherpad.openstack.org/p/nova-juno-spec-priorities; 3) get core reviewers to sponsor
21:15:18 <mikal> devananda: so we know it works at least a little
21:15:22 <devananda> now the train starts moving ... i'm gonna drop on and offline a bunch
21:15:39 <russellb> devananda: can you start a thread about test requirements?
21:15:43 <mikal> #info mikal will send an email about how to request an exception to -dev after this meeting
21:15:44 <degwea> OK,if I do it tomorrow will it still be consider? Thx
21:15:52 <devananda> mikal: yes, in tripleo. but it's not tested in devstack
21:15:52 <devananda> russe	ack. feel free to #action me
21:16:02 <mikal> degwea: that will be fine, but it will need to be tomorrow
21:16:08 <raildo> hi, this BP can be considered? https://review.openstack.org/#/c/102201/ I think sajeesh can explain it better
21:16:09 <russellb> #action devananda to start a thread on openstack-dev to nail down test expectations for baremetal -> ironic upgrade
21:16:20 <sajeesh> yes
21:16:20 <russellb> so, the next that has 2 or more sponsors is https://review.openstack.org/#/c/84887/16/specs/juno/use-glance-v2-api.rst
21:16:24 <russellb> i think this is in the obvious category
21:16:25 <mriedem> raildo: see above
21:16:30 <mikal> Yeah, approved
21:16:34 <degwea> Sure it will be tomorrow Thanks
21:16:38 <raildo> mriedem: ok
21:16:38 <mikal> Track that in the etherpad?
21:16:39 <russellb> #agreed glance v2 support exception approved
21:16:45 <mikal> Or that!
21:16:46 <arnaud__> great thanks russellb
21:17:02 <mikal> Similarly, where did we end up with the cinder API spec?
21:17:08 <mriedem> mikal: it's under review
21:17:10 <mriedem> the code that is
21:17:13 <mriedem> the spec was approved
21:17:14 <russellb> thought that one was approved already
21:17:14 <mriedem> awhile ago
21:17:15 <mikal> Oh, good
21:17:20 <mriedem> thingee: ^
21:17:21 <russellb> ah right
21:17:34 <russellb> https://review.openstack.org/#/c/91486/6/specs/juno/rbd-clone-image-handler.rst
21:17:34 <mikal> I will admit to not having all 60 something approved specs in my head
21:17:37 <russellb> is the next one with sponsors
21:17:41 <russellb> has 4 listed, overkill, heh
21:17:57 <mikal> I guess the other thing we're asking...
21:17:59 <thingee> cinder api code is just about done. there is one test still failing
21:18:01 <mriedem> i think we have to dig into the rbd test failures a bit
21:18:02 <russellb> at least 2 people have told me they've already looked over the code and think it's reasonable to land very soon
21:18:02 <mikal> Do we think that spec could be merged today?
21:18:04 <thingee> cinder v2 api support*
21:18:06 <mikal> How close is the spec itself?
21:18:36 <angdraug> mriedem: jaypipes already has a fix in testing for that
21:18:37 <russellb> mriedem: well the good news is that there weren't *that* many failures
21:18:38 <mriedem> mikal: the rbd specs says 'testing? nah"
21:18:45 <mriedem> yeah
21:18:46 <mikal> mriedem: le sigh
21:18:47 <russellb> mriedem: yeah but that's in progress now
21:18:49 <thingee> there is an inconsistency between v1 and v2 in schema type failing. I'll have to update the tempest test to know about both unfortunately.
21:18:51 <mikal> So...
21:18:53 <russellb> spec out of date on the test front
21:18:58 <mikal> If we approve an exception, it has to be timeboxed, right?
21:19:00 <russellb> and i'm tracking debugging / fixing test failures
21:19:07 <mikal> i.e. We will keep reviewing that spec for say a week or something
21:19:15 <mikal> If we can't agree within a week, its still out?
21:19:22 <russellb> mikal: agreed
21:19:23 <mriedem> i think the spec's testing section just needs updating
21:19:28 <mikal> What do we think is a reasonable timeout period?
21:19:33 <russellb> mikal: was assuming we generally were ready to approve the spec if we give an exception
21:19:38 <russellb> a week is generous IMO
21:19:40 <mriedem> then the spec is ok, but we can't land the code w/o the test issues resolved
21:20:04 <mriedem> resolved = bugs with possible skips if using rbd maybe
21:20:05 <mikal> So in this case, we'd be giving a week for someone to come up with a _plan_ for testing that we like
21:20:10 <mikal> (Not nessesarily an implementation)
21:20:17 <russellb> mikal: i'm all over it
21:20:21 <mriedem> mikal: russellb has a devstack patch running with rbd
21:20:24 <russellb> status: https://etherpad.openstack.org/p/LJj1gHesRG
21:20:26 <mriedem> 14 failures, might not all be rbd
21:20:30 <mikal> russellb: sure, but I'm trying to set precident for future exceptions
21:20:34 <mriedem> some are definitely rbd issues
21:20:49 * russellb nods
21:20:52 <mriedem> i'm still not sure how we're going to test glance and cinder v2
21:21:01 <mriedem> or keystone v3 for that matter
21:21:07 <mikal> #info a spec exception means that you have a week from the exception being granted to reach a concensus and get the spec merged. If that doesn't happen, then exception expires.
21:21:20 <russellb> #action russellb will update rbd spec's testing section to reflect progress on ceph testing in the gate
21:21:28 <cyeoh> can we also look at https://review.openstack.org/#/c/84695/14/specs/juno/v2-on-v3-api.rst (v2.1 on v3) - only has one core sponsor, but I'm pretty sure oomchi would have sponsored too if he knew about the request for core sponsors
21:21:34 <mikal> Ok, so it sounds like rbd is a spec exception winner
21:21:37 <russellb> agree on an exception for that one, once the test section is updated?
21:21:39 <russellb> ok cool
21:21:49 <arnaud__> would you guys mind removing the -2 on https://review.openstack.org/#/c/84887/ ?
21:21:58 <russellb> #agree exception approved for rbd spec, pending update of the test section to reflect gate test progress
21:22:10 <mriedem> arnaud__: i think we already did talk about it, else ML
21:22:22 <russellb> mikal: that's all that had 2 people confirmed already
21:22:22 <mikal> Let's go backwards and talk v3 API for a second
21:22:34 <russellb> yeah, that's what i was going to mention next
21:22:36 <mikal> I just sneakily added my name there to get it over the line
21:22:38 <arnaud__> mriedem, we did, but that doesn't encourage people to review if there is a -2
21:22:43 <russellb> ah there it is heh
21:22:55 <mikal> I would like to see us have a really good try at reaching a concensus on that one in the next week
21:22:55 <russellb> #link https://review.openstack.org/#/c/84695/14/specs/juno/v2-on-v3-api.rst
21:23:02 <mikal> I think we're painfully close
21:23:02 <russellb> seems reasonable to me
21:23:05 <mikal> cyeoh: ^--
21:23:06 <mriedem> arnaud__: i'd send something to ML so it's not lost
21:23:10 <mikal> oomichi: ^--
21:23:18 <arnaud__> ok sure mriedem will do
21:23:24 <mikal> So, I think that means its an exception winner too
21:23:30 <russellb> i don't think i should commit to review, but happy to support the exception
21:23:36 <mikal> With a note that we're really close to not having enough time to actually implement it all
21:23:40 <russellb> #agreed v2-on-v3-api gets an exception
21:23:43 <cyeoh> the v2.1 on v3 has all the microversions details merged into it now too so its not dependent on anything else
21:23:55 <mikal> I think the people I need for reviews are dansmith and alaski based on previous review concerns
21:24:06 <mikal> sdague too if he has the time
21:24:14 <raildo> mriedem: from what I understand, no spec will be approved in Juno, only K version, ok?
21:24:19 <alaski> I'm willing to review it
21:24:33 <mikal> Thanks man
21:24:35 <dansmith> mikal: yeah, I feel like several of us have said that we're not comfortable trying to get it done by juno, but I'll take another trip through that spec and see if I can convince myself
21:24:43 <mriedem> raildo: exceptions for juno have to go through the openstack-dev mailing list right now
21:24:57 <russellb> it does seem scary to crunch in
21:25:14 <mikal> I think it depends where we draw the line
21:25:15 <mikal> But yes
21:25:18 <russellb> but i'm not against at least progress in juno if there can be consensus on the spec
21:25:22 <raildo> mriedem: ok, thanks
21:25:31 <mikal> I am worried specifically about the proxying code that isn't written yet (cinder, neutron that is)
21:25:31 <alaski> I agree with that.  But progress would be nice so there's more to discuss at a summit
21:25:57 <cyeoh> we had the spec initially submitted in April, so we've been ready to go, just waiting on spec approval really (and the v2.1 on v3 spec itself hasn't changed much)
21:26:27 <mikal> For better or worse this is where we are
21:26:35 <mikal> But I would like to get a start on it at the least
21:26:47 <mikal> But yeah, let's let the exception process do its work and see where we end up
21:26:52 <cyeoh> yep, agreed
21:27:14 <mikal> I think we should move on from specs to be honest. We only have 30ish minutes to go
21:27:25 <mikal> Any other exception requests to the mailing list please
21:27:40 <mikal> Noting that we already have a really large number of specs approved...
21:27:42 <hemna> mikal, sent a short quick one for mine.
21:27:47 <mikal> I am already worried about code review bandwidth
21:27:59 <russellb> yes.
21:28:01 <dansmith> yep
21:28:07 <dansmith> I'm quite concerned
21:28:12 <mikal> Well, I'm always worried about it
21:28:17 <mikal> But I think its getting worse
21:28:23 <mikal> Anyways
21:28:30 <mikal> #topic Mid-cycle meetup
21:28:38 <mriedem> did we find chairs?
21:28:42 <mikal> #info We ran out of seats, mikal has a plan
21:28:44 <mriedem> or will there be lap sitting?
21:28:50 <tjones> ewwwww
21:28:54 <mikal> So... I've asked Intel to re-confirm the size of the room
21:28:54 <xarses> hot seat
21:29:04 <mikal> Its apparently a training room so we might be able to drag more sets in
21:29:07 <mriedem> it's going to smell like a locker room in there...
21:29:14 <n0ano> I'm sure we're good for 35, >40 might be a problem.
21:29:15 <mikal> Either way, I'm going to try and sneak in everyone from that mail thread
21:29:25 <mikal> Some of the containers people ahve moved to their meetup, which helps
21:29:35 <mriedem> ok, was going to ask about that
21:29:45 <mikal> We will just need to be organized about sending a runner to their room when we start ranting about them
21:29:54 <mikal> Ditto for the ironic folks
21:30:01 <mikal> But yeah, we can make it work
21:30:06 <n0ano> the rooms are withint about 30' of each other
21:30:19 <mikal> n0ano: we're metric for K now. So 10 meters
21:30:21 <alaski> tin cans and string?
21:30:24 <mikal> Heh
21:30:32 <tjones> do we have an agenda?  or did i miss it?
21:30:43 <mikal> So, I had a call with n0ano and the other two meetup organizers about logistics the other day
21:30:46 <mikal> So I think we're set there
21:31:00 <mikal> There will be an email with details (emergency phone numbers etc) next week
21:31:09 <mikal> That will just go to registered attendees
21:31:15 <russellb> tjones:  there's an etherpad with proposed topics
21:31:19 <mikal> Because I don't want to give out the Intel admin's phone number ot the entire world
21:31:25 <mikal> Yes, next is the agenda
21:31:28 <n0ano> I do need a final list of attendees names & email aaddresses by this coming Mon. (have to arrange wifi access)
21:31:45 <mikal> n0ano: ok cool. Can you email me that so I remember?
21:31:55 <n0ano> mikal, already did
21:32:01 <mikal> So there's an etherpad collecting topics
21:32:03 <mikal> n0ano: cool
21:32:11 <mikal> There's also the hot topics from the meeting page
21:32:25 <mikal> So, let's spend until say Wednesday nailing that down
21:32:33 <mikal> And then we can try and turn it into a rough agenda
21:32:58 <mikal> I'm ok with that bit beign a little rubbery -- if somehting needs more time than we think it does, we can keep going on it as long as nothing important misses out
21:33:05 <mikal> Any questions about the midcycle?
21:33:22 <tjones> do we know which day we are going to talk about bugs?  I want to be there for that but can't be there the whole time
21:33:37 <mikal> tjones: what day works best for you?
21:33:56 <mikal> tjones: how about you pick a day and put that on the topics etherpad, we can just work with that
21:34:03 <tjones> monday would be awesone
21:34:08 <tjones> or awesome too
21:34:13 <dansmith> I refuse to talk about bugs on monday
21:34:15 <mikal> Heh
21:34:20 <tjones> dansmith: i figured
21:34:22 <mikal> dansmith: mutter
21:34:28 <dansmith> tjones: just trying to be difficult.. monday is fine
21:34:33 <tjones> dansmith:  :-D
21:34:40 <mikal> Someone bring a blow dart of sedatives for dansmith
21:34:43 <dansmith> tjones: devananda told me I'm always difficult, so I'm trying to live up to that
21:34:45 <mikal> I think you can't get them at walmart
21:34:58 <russellb> no, you can.
21:35:01 <mikal> Oh, good
21:35:06 <mriedem> you can get anything at wal mart
21:35:12 <russellb> sedatives at least, you may have to bring your own blow darts.
21:35:13 <mikal> Including freedom?
21:35:17 <mikal> Anyways
21:35:22 <mriedem> oh wait except music with swearing
21:35:25 <mikal> So, nothing _serious_ about the midcycle?
21:35:53 * mikal times out
21:36:01 <mikal> #topic Bugs
21:36:04 <russellb> no, that's just your DSL
21:36:05 <mikal> Apparently we have bugs
21:36:12 <mikal> russellb: sigh
21:36:16 <mikal> tjones: talk at us
21:36:25 <tjones> ok so most of you have seen http://54.201.139.117/nova-bugs.html
21:36:32 <mikal> Yes, its really good
21:36:38 <tjones> which helps me an others get a better handle on our situation
21:36:53 <mikal> The bug count seems to have gone down a fair bit since that page appeared
21:37:02 <mikal> Last I looked we were at 1,000 instead of 1,400 now
21:37:06 <tjones> jogo:  has been very helpful with geting the abandoned and merged bug list down
21:37:09 <mriedem> jogo has been doing things
21:37:09 <mriedem> yeah
21:37:23 <tjones> yeah things are moving in the right direction!
21:37:40 <tjones> only 2 criical issues today and both are under control
21:37:55 <mikal> Even the SSH timeout one?
21:37:56 <mikal> Wow
21:38:01 <tjones> i'd like to point out that there are 183 bugs ready for review
21:38:13 <russellb> ready for code review?
21:38:20 <tjones> yeah - ssh needs a backport, but last comment was it's ok on master
21:38:27 <tjones> russellb: yes ready for review
21:38:27 <mikal> tjones: so... a web page listing bugs for review where the bug is older than 7 days would excite me
21:38:30 <russellb> tjones: wow
21:38:39 <dansmith> tjones: is that a patch other than the two I did last week to fix that?
21:38:40 <mikal> I think excluding super new bugs would help
21:38:43 <leifz> It may be worth removing owners for long un-updated bugs.
21:38:47 <tjones> mikal: the updated column shows that
21:38:48 <mikal> Because lots of people file a bug and then immediately fix it
21:38:49 <dansmith> tjones: because I've got all my fixed backported
21:39:13 <mikal> tjones: oh, I more meant a page listing reviews I should do
21:39:21 <mikal> tjones: to encourage, well, review of those thigns
21:39:23 <tjones> mikal: sure i can add that
21:39:36 <mikal> tjones: perhaps a separate page or somethign?
21:39:39 <mriedem> dansmith: i'm not aware of any others for the ssh thing
21:39:44 <mikal> tjones: but thanks for keeping hacking on this
21:39:50 <dansmith> mriedem: okay, mine are all backported and committed
21:39:51 <mriedem> dansmith: unless those need to go to stable/havana for some reason
21:39:59 <dansmith> mriedem: yeah, don't think so
21:40:01 <tjones> check jogo comment on https://bugs.launchpad.net/nova/+bug/1298472
21:40:04 <mriedem> dansmith: logstash can tell us, i'll check
21:40:04 <uvirtbot> Launchpad bug 1298472 in nova "SSHTimeout in tempest trying to verify that computes are actually functioning" [Critical,Confirmed]
21:40:14 <mriedem> http://status.openstack.org/elastic-recheck/
21:40:20 <mriedem> we aren't kiling the gate right now
21:40:23 <mriedem> others are
21:40:28 <tjones> mikal: sure it's been fun actually
21:40:33 <mikal> On the bug front we should probably be paying attention to the mysql deadlock problems neutron is seeing, because I think we're seeing them too
21:40:50 <mikal> The fix seems to be lost in pypi land, but we should keep half an eye on it
21:41:04 <dansmith> mriedem: yeah, my last one merged right after jogo's last comment on the bug I think
21:41:09 <dansmith> tjones: so I think that one is good
21:41:16 <tjones> dansmith: great
21:41:21 <mikal> dansmith: as in the SSHTimeout bug can be resolved?
21:41:44 <mikal> Resolved^WMarked fix committed?
21:41:52 <dansmith> mikal: we still hit the problem occasionally, but for a different reason
21:41:57 <dansmith> mikal: this bug is so over-used though,
21:42:09 <mikal> Can we close it and make a new one for the smaller case then?
21:42:09 <mriedem> yeah the query is very generic
21:42:12 <dansmith> mikal: that I think we should mark it done and open a new one to cover the sporadic other ones
21:42:14 <mriedem> so any ssh timeout will hit this query
21:42:16 <dansmith> mikal: yeah, we should do that
21:42:18 <russellb> dansmith: +1
21:42:18 <mikal> dansmith: snap!
21:42:21 <mriedem> i can remove the e-r query
21:42:27 <mikal> Excellent
21:42:38 <mikal> dansmith: please go and expense yourself a bottle of champagne
21:42:39 <dansmith> that was a big nasty bug,
21:42:42 <mikal> dansmith: the cheap stuff please
21:42:52 <dansmith> which was not even gate-related and could be a real problem in production
21:42:56 <dansmith> so glad to get that resolved
21:43:02 <mikal> dansmith: it could be passionfruit flavoured
21:43:07 <dansmith> mikal: will-do
21:43:20 <mriedem> dansmith: can you get the patch links into the bug report?
21:43:23 <mriedem> not sure they are all there
21:43:29 <mriedem> then we can close it
21:43:38 <dansmith> mriedem: I'll look, but I thought they were there, because I've been referencing it
21:43:47 <mriedem> lp linker doesn't work
21:43:48 <mriedem> always
21:44:13 <mikal> Ok, we should probably move on again
21:44:14 <dansmith> mriedem: apparently none of them are, so yeah
21:44:17 <mikal> Anything else on bugs?
21:44:27 <tjones> not from me
21:44:36 <mikal> tjones: thanks for doing a great job
21:44:42 <mikal> dansmith: thanks again for fixing that bug
21:44:43 <tjones> mikal: thanks!
21:44:46 <dansmith> yar
21:44:46 <mriedem> if anyone knows anything about requests connection pools,
21:44:49 <mriedem> bug 1341777
21:44:52 <uvirtbot> Launchpad bug 1341777 in python-glanceclient "glanceclient is not handling http connection pools properly with python-requests" [High,In progress] https://launchpad.net/bugs/1341777
21:44:56 <mriedem> glanceclient is spamming the n-cpu logs badly
21:45:07 <mriedem> i've tried patches but not fixing it
21:45:28 <mikal> mriedem: could you get someone on the glance side to dig into it?
21:45:36 <mriedem> mikal: flaper87|afk helped with one
21:45:39 <mikal> mriedem: do other projects see it?
21:45:43 <mriedem> yes, c-vol
21:45:52 <mriedem> and g-api is spammed in turn by swiftclient b/c of requests
21:45:56 <mriedem> same issue i think
21:45:59 <mikal> Sigh
21:46:15 <mriedem> i tried ML
21:46:42 <mriedem> anyway, just fyi
21:46:47 <mikal> yeah, thanks
21:46:49 <mriedem> help needed by clienty people
21:47:01 <mikal> Or someone looking for a good adventure
21:47:09 <mriedem> 3 days is enough for me
21:47:35 <mikal> So, let's move on
21:47:43 <mikal> #topic Gate Status
21:47:49 <mikal> The gate seems pretty good at the moment?
21:47:59 <mikal> Very quiet right now in fact
21:48:12 <dansmith> seems very good this week, yeah
21:48:23 <mikal> So, j-2 is going to change that, so I think that means we have an incentive to approve as much j-2 stuff as soon as we can
21:48:23 <dansmith> things merge whilst making coffee, unexpectedly
21:48:31 <mikal> Obviously without lowering our standards
21:48:43 <mikal> But if you have a spare moment, do a code review or two
21:49:08 <mikal> Ok, moving on again
21:49:14 <mikal> #topic Review status
21:49:22 <mikal> #info Please focus on j-2 reviews
21:49:29 <mikal> What else do we need to say about reviews?
21:49:40 <mikal> I'm sure there are people who want their reviews rescued (I have some like that myself)
21:49:48 <mikal> But let's stick with j-2 as much as is sane
21:49:50 <russellb> http://russellbryant.net/openstack-stats/nova-openreviews.html
21:49:56 <russellb> is my report on review queue status
21:50:04 <russellb> turnaround times have improved in the last couple weeks actually
21:50:13 <russellb> a few people have been hitting old stuff
21:50:26 <russellb> there's some more interesting stats at the bottom of http://russellbryant.net/openstack-stats/nova-reviewers-30.txt
21:50:27 <mikal> The waiting on submitter number makes me a bit happier
21:50:37 <russellb> Queue growth in the last 30 days: 135 (4.5/day)
21:50:38 <mikal> Those numbers are probably skewed by the lack of abandons as well
21:50:40 <russellb> is the scary one
21:50:48 <mikal> So, cores...
21:50:58 <mikal> Most of us didn't realize that we now need to hand abandon it seems
21:51:02 <mikal> The bot went off for a rest
21:51:13 <russellb> danpb did some nice analysis that showed that the core team is doing a fairly consistent amount over the last year
21:51:16 <mikal> So, if you see a review thats weeks and weeks old and obviously idle, please manually abandon it
21:51:17 <russellb> input to the queue is just going up
21:51:26 <leifz> russellb: I think some of that is my fault, I hit up folks to refresh to verify no merge conflicts.
21:51:53 <russellb> leifz: stats would still treat it as old unless jenkins did a -1 after that
21:52:06 <leifz> russellb: which happened in a number of them :-(
21:52:06 <russellb> at least the one that does Longest waiting reviews (based on oldest rev without -1 or -2):
21:52:13 <mikal> Looking at the time...
21:52:25 <russellb> Changes merged in the last 30 days: 318 (10.6/day)
21:52:26 <mikal> Is there anything else urgent about reviews, or shall we do the fastest sub team report round ever?
21:52:27 <russellb> \o/
21:52:37 * russellb stops spouting off stats
21:52:40 <mikal> Heh
21:52:49 <mikal> #topic Subteam reports
21:52:56 <mikal> Who has something urgent in sub team land?
21:53:02 * n0ano gantt
21:53:07 <mikal> Go
21:53:12 <russellb> i just wanted to note that i'm trying to push on getting ceph testing in the gate going
21:53:18 <russellb> since we've talked about it in here before a little bit
21:53:28 <russellb> hm, i think i mentioned that earlier, sorry.
21:53:34 <mikal> Heh, NP
21:53:34 <n0ano> looking for use cases to justify spliting out gantt, please look at the ether pad https://etherpad.openstack.org/p/SchedulerUseCases
21:53:54 <mikal> n0ano: do I misunderstand, or is that backwards?
21:54:05 <n0ano> mikal, ?
21:54:07 <mikal> n0ano: shouldn't we look for scheduler use cases and then decide what to do?
21:54:15 <dansmith> lol
21:54:15 <mikal> n0ano: not decide what to do, and then find reasons to justify it?
21:54:23 <mikal> n0ano: or perhaps I've totally failed to parse your sentence
21:54:39 <russellb> mikal: stop thinking rationally sir
21:54:48 <n0ano> are there use cases that would require a separate scheduler is the question
21:55:00 <mikal> n0ano: ahh, ok. Cool.
21:55:01 <russellb> but yes, lately the whole split idea has been questioned
21:55:18 <mikal> I am looking forward to getting Jay in a room and talking through that
21:55:18 * russellb hasn't been able to catch up on all the related ML traffic though ...
21:55:24 <mikal> It seems he has thoughts to share
21:55:27 <jaypipes> uh oh.
21:55:28 <melwitt> mikal: I read it the same way, sounds backward
21:55:30 <mikal> LOL
21:55:30 <n0ano> I think there are other, non-technical reasons for the split but having technical justification would help
21:55:39 <russellb> jaypipes: that sounded threatening, didn't it?
21:55:47 <russellb> jaypipes: watch out for the boomerang
21:55:50 <mikal> A bunch of the refactoring proposed makes sense to me regardless of split or not
21:56:04 <jaypipes> angdraug: fyi, *still* waiting for integrated api samples tests to complete... guh, they take around 5 seconds for each test method.
21:56:10 <mikal> Like making the scheduler more advisory and not a part of the boot flow
21:56:19 <russellb> agreed
21:56:31 <mikal> I think this will be a largish topic at the meetup
21:56:36 <mikal> So, any other subteams?
21:56:57 <mikal> #topic Open Discussion
21:57:06 <mikal> You have two minutes for open discussion, use it wisely
21:57:12 <mikal> I like pancakes
21:57:25 * russellb likes pancakes, but prefers a good belgian waffle
21:57:28 <mikal> Oh, and did another python-novaclient release
21:57:41 <mikal> Plus, have creted an ongoing LP burden for future nova PTLs because it amuses me
21:57:47 <mikal> created even
21:57:55 <russellb> using milestones?
21:57:56 <russellb> that's a good idea
21:57:58 <mikal> Yeah
21:57:58 <russellb> should have all along
21:58:06 <mikal> 2.18.0 looks _big_ in LP!
21:58:09 <russellb> ha
21:58:10 <mikal> Its actually pretty easy
21:58:14 <russellb> mikal: that script you're using handle it?
21:58:16 <mikal> I stole dolphm's script to do it
21:58:19 <russellb> cool
21:58:25 <mikal> Its documented on the wiki
21:58:32 <russellb> ttx has some handy scripts that help with this kind of stuff too
21:58:43 <russellb> like, auto deferring bugs not fixed
21:58:47 <mikal> One day we should document all of those
21:58:55 <russellb> pfft.
21:59:03 <mikal> Well, with the client I've been creating the milestone 30 seconds before releasing
21:59:07 <mikal> So deferral isn't a thing
21:59:17 <russellb> create, then target everything fixed?
21:59:19 <russellb> meh, works :)
21:59:22 <sean-k-mooney> can i ask how excetions should be requested via the ml?
21:59:25 <mikal> Yep, that's the flow
21:59:34 <mikal> sean-k-mooney: spec exceptions, yes?
21:59:40 <sean-k-mooney> yes
21:59:45 <mikal> sean-k-mooney: as in... yes, spec exceptions to the ML please
21:59:47 <russellb> ttx's process_bugs.py can do the auto-target bit i think
22:00:01 <dansmith> aaaaand, time.
22:00:06 <mikal> Yep
22:00:10 <sean-k-mooney> ok i was just wondering if it was documented
22:00:13 <mikal> Thanks again for coming peoples
22:00:19 <mikal> You're all lovely people with nice hair styles
22:00:23 <russellb> thanks
22:00:24 <mikal> #endmeeting