21:00:22 <mriedem> #startmeeting stable
21:00:22 <openstack> Meeting started Mon Feb 29 21:00:22 2016 UTC and is due to finish in 60 minutes.  The chair is mriedem. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:26 <openstack> The meeting name has been set to 'stable'
21:00:37 <mriedem> #link agenda https://wiki.openstack.org/wiki/Meetings/StableTeam
21:00:41 <mriedem> who's all around?
21:01:14 <mriedem> mtreinish: tonyb: ttx: mrunge: ping
21:01:31 <tonyb> o/
21:01:44 <mriedem> flaper87: jokke_: ping
21:01:44 <tonyb> Gah I was in meeting-3 bad IRC client!
21:01:54 <mriedem> moves week to week
21:01:56 <mriedem> which is confusing
21:02:12 <mriedem> let's get started
21:02:14 <mriedem> #topic status
21:02:17 <flaper87> o/
21:02:32 <mriedem> i've been watching the periodic-stable ML
21:02:40 <mriedem> fixed some issues in glance last week
21:02:49 <mriedem> ironic and some other projects had fallout from an oslo.config release too
21:02:52 <mriedem> ironic is fixed
21:03:06 <jokke_> mriedem: pong
21:03:10 <mriedem> murano and ceilometer also had issues
21:03:19 * mtreinish is about to step away for a very late lunch
21:03:32 <mriedem> the trakcing bug was https://bugs.launchpad.net/oslo.config/+bug/1547612
21:03:32 <openstack> Launchpad bug 1547612 in Ceilometer "AttributeError: 'NoneType' object has no attribute '_config_dirs'" [Medium,Triaged]
21:03:39 <mriedem> looks like murano and ceilometer are on there now
21:04:20 <mriedem> #link stable issues tracker https://etherpad.openstack.org/p/stable-tracker
21:04:25 <tonyb> mriedem: Thanks.  I've been keeping the failure logs for looking at but ....
21:04:40 <mriedem> sahara pep8 fixes are finally in
21:04:47 <mriedem> ksm g-r sync for kilo is also in
21:04:54 <tonyb> \o/
21:05:04 <tonyb> that was much harder than it shoudl have ben
21:05:07 <mriedem> that took some manual updating by bknudson, looks like the g-r sync bot isn't applying changes to *requirements-py3.txt on stable
21:05:21 <tonyb> mriedem: can we fix that?
21:05:22 <mriedem> i'm assuming b/c i thought the -py3 files were removed in master
21:05:26 <mriedem> tonyb: probably?
21:05:29 <mriedem> i never reported a bug
21:05:43 <mriedem> this was the ksm review: https://review.openstack.org/#/c/277022/
21:06:21 <tonyb> mriedem: Thanks
21:06:22 <tonyb> brb
21:06:24 <mriedem> anyway, it's probably worth digging through git log on the requirements repo to see if the py3 thing was intentionally dropped
21:06:29 <mriedem> but it's lower priority
21:06:51 <tonyb> .
21:07:07 <mriedem> lifeless might also know off the top of his head
21:07:07 <tonyb> mriedem: yeah
21:07:23 <mriedem> #topic action items from previous meeting
21:07:30 <mriedem> tonyb to talk to lifeless about fixtures bug https://bugs.launchpad.net/python-fixtures/+bug/1542984
21:07:30 <openstack> Launchpad bug 1542984 in Python Fixtures "fixtures 1.2.0 isn't compatible with testtools 2.0.0" [Undecided,New]
21:07:35 <mriedem> i didn't see any updates in the bug
21:07:50 <mriedem> last i looked the neutron kilo patch was still failing in weird ways
21:07:57 <mriedem> even though ihar capped testtools in stable/kilo g-r
21:08:02 <mriedem> 2.0.0 was getting used
21:08:32 <tonyb> mriedem: Yeah and an unpatched job passes, so there is somethign moer to it that I haven't been able to find.
21:08:49 <tonyb> I'ev asked lifless to either do ther release or close the bug
21:09:05 <mriedem> oh i see https://review.openstack.org/#/c/282146/
21:09:10 <tonyb> I feel like we can't progress alternate approaches until we have that decision made
21:09:50 <tonyb> Yeah and I have a WiP patch for oslo.concurency to do the capping there too
21:09:54 <bknudson> I hope stable/kilo goes away soon.
21:10:05 <mriedem> i can ping lifeless on the bug again
21:10:20 <tonyb> mriedem: https://review.openstack.org/#/c/282123/ is the one that passes the -api tests without any kind of cap
21:10:24 <mriedem> #action mriedem to ping lifeless about fixtures bug https://bugs.launchpad.net/python-fixtures/+bug/1542984
21:10:25 <openstack> Launchpad bug 1542984 in Python Fixtures "fixtures 1.2.0 isn't compatible with testtools 2.0.0" [Undecided,New]
21:10:28 <tonyb> mriedem: which just doesn't make sense to me
21:10:37 <tonyb> mriedem: that'd be cool
21:10:45 <tonyb> mriedem: I just did it in #openstack-infra
21:11:15 <mriedem> ok, i was trying to unwind some of the neutron job setup b/c there could be something weird in there
21:11:21 <mriedem> i know you patched devstack on kilo to workaround it
21:11:30 <rockyg> o/
21:11:32 <tonyb> mriedem: right that works for devstack
21:11:33 <mriedem> i'm guessing maybe something like that is needed in neutron stable/kilo functional script or something
21:12:00 <tonyb> but the -api and neitron-neutron jobs run up a new tox env ontop of the dsvm and then get the wrong testtools
21:12:52 <tonyb> mriedem: Yeah I looked at that approch rather than https://review.openstack.org/#/c/282146/
21:13:05 <tonyb> but I couldn't find anytthing that worked well
21:13:40 <tonyb> otehr than a just addeding "pip install -U testtools<=2.0.0" in the post-gate-hoo
21:13:43 <tonyb> k
21:13:44 <mriedem> hmm, that failed the LB test
21:13:56 <mriedem> which i think there is another kilo patch backport for nova related to that
21:14:03 <rockyg> bknudson, stable/kilo dies in June???
21:14:18 <mriedem> tonyb: anyway, we should probably try to capture some of this in the stable-tracker etherpad,
21:14:27 <mriedem> since it's pretty sparse on the latest changes / attempts
21:14:36 <tonyb> mriedem: Yeah I've done a terrible job of that.
21:15:05 <mriedem> rockyg: re: kilo, the releases docs say (EOL: 2016-05-02)
21:15:18 <mriedem> http://releases.openstack.org/
21:15:39 <mriedem> there was another action item from last week
21:15:39 <mriedem> bknudson to propose dropping upper-constraints from stable/kilo to see if the reqs job will work without it
21:15:48 <rockyg> mriedem, even better.  In some ways.
21:15:50 <mriedem> that was already sorted out, and we've already talked about it (the py3 thing)
21:15:51 <bknudson> mriedem: I did that
21:16:06 <mriedem> yup
21:16:13 <mriedem> #topic release news
21:16:37 <mriedem> just a reminder to be reviewing stable/liberty things and consider a release sometime soonish to line up with m-3
21:16:38 <bknudson> supposedly there will be no more features in oslo libs
21:16:39 <mriedem> #link http://lists.openstack.org/pipermail/openstack-dev/2016-February/086362.html
21:16:46 <bknudson> so we'll have stable branches sometime.
21:16:53 <mriedem> bknudson: ?
21:17:21 <bknudson> the final releases of non-client libs is past
21:17:22 <mriedem> i'm assuming that's the bknudson dry humor?
21:17:25 <mriedem> ah
21:17:39 <bknudson> y, just for mitaka, not all time
21:17:42 <mriedem> well, oslo.config managed to break stable/liberty last week for 3 projects
21:17:43 <mriedem> but sure
21:18:24 <rockyg> oslo libs are frozen except for critical bugfixes
21:18:25 <bknudson> so if we're still going to do stable releases for libs there will be stable branch
21:18:40 <mriedem> bknudson: yeah
21:18:43 <tonyb> mriedem: right but that's linked to the u-c mistakes we made on liberty that hopefully we won't do again on mitaka
21:19:01 <mriedem> tonyb: sort of...
21:19:21 <mriedem> #topic stuck reviews
21:19:33 <mriedem> there was nothing on the agenda, did anyone have something for this?
21:20:10 <tonyb> mriedem: there are a couple of nova ones but we can talk about that later if you prefer.  Possibly best at a nova meeting
21:20:12 <mriedem> there was a largeish nova backport for an rbd thing
21:20:25 <mriedem> but that merged
21:20:41 <mriedem> https://review.openstack.org/#/c/228505/
21:20:44 <mriedem> tonyb: yeah
21:20:53 <mriedem> #topic tooling
21:20:58 <mriedem> there isn't any news here
21:21:02 <mriedem> i haven't had a chance to work on this
21:21:23 <mriedem> well, specifically, "script to check for backport-potential bugs which are fixed on trunk but backports are not yet proposed"
21:21:33 <tonyb> mriedem: Ooooo I have one of those
21:21:42 <mriedem> rockyg: if you have devs that are looking for things to hack on, ^ would be one
21:21:44 <tonyb> I wrote it last week
21:21:49 <mriedem> tonyb: for that one?
21:21:55 <tonyb> it need to be cleaned up
21:22:07 <mriedem> ok, cool
21:22:17 <mriedem> i'm not sure what repo these go in, maybe release-tools
21:22:20 <tonyb> mriedem: It looks at all the fixed* bugs in LP with a librty-backport-potential tag
21:22:35 <tonyb> and then checks to see if there is an review for that on liberty
21:22:45 <tonyb> mriedem: that's kinda what you wanted right?
21:22:49 <mriedem> yeah
21:22:50 <mriedem> basically
21:22:57 <rockyg> OK.  I supposedly have one.  I'll see if I can light a fire, mriedem I likely will need to intro him to you guys then you can guide him.
21:23:11 <mriedem> rockyg: sounds like tonyb already has what i was looking for
21:23:40 <mriedem> tonyb: i'm not very familiar with the launchpad API but it would be cool if we could also scrape those with a specific release propsed, if the tag isn't there
21:23:43 <rockyg> Yeah.  Just caught up.  In two meetings :P
21:23:45 <tonyb> mriedem: I ran it on nova tl;dr 3 bugs match that criteria which is somethign I wanted to discuss in open-discussion....
21:23:47 <mriedem> but it's a good start
21:23:58 <rockyg> still need to intro him to you guys to get him active.
21:24:09 <tonyb> mriedem: Hmm okay I'll look at adding that.
21:24:25 <tonyb> rockyg: Yeah the more people on #openstack-stable the better :)
21:24:26 <mriedem> tonyb: would be a good start to just get what you have cleaned up i guess and up for review in release-tools
21:24:47 <mriedem> i generally add the release to the bug and tag with liberty-backport-potential
21:25:01 <mriedem> in some projects i can only nominate the bug for a release though, so the tag is always good
21:25:05 <tonyb> mriedem: Yeah I'll do it (adda review) this week
21:25:22 <mriedem> #action tonyb to push up his initial script for scraping backport potential bugs
21:25:36 <mriedem> #topic open discussion
21:25:39 <mriedem> i had one thing
21:25:41 <mriedem> (mriedem): Do we want/need a summit session? I figure there will at least be some overlap with the release and QA teams on how long to keep stable/liberty.
21:26:01 <mriedem> now that i look at the releases docs though, liberty is marked as Current stable release, security-supported (EOL: 2016-11-17)
21:26:16 <rockyg> I'd love a summit session.  Get more people aware, up to date, etc.
21:26:16 <mriedem> which is just ~6 months after kilo eol
21:26:25 <bknudson> how many times can we have the same discussion at the sumit?
21:26:50 <mriedem> rockyg: well, there are at least 2 talk proposals related to stable branch maint
21:26:58 <tonyb> bknudson: the landscape is changing so I think it's a good idea
21:27:16 <bknudson> maybe operators that love using past-eol branches will step up
21:27:20 <tonyb> mriedem: I'd be in favor of it but it'll be hard to get the right people in the room (read PTL/CPLs)
21:27:24 <rockyg> New project.  If we get people, we could make new policies....like extending liberty if there are hands to make it happen.  and mriedem, Oh.  Okay.
21:27:49 <mriedem> well, so far no one new has really stepped up to help out with stable
21:27:56 <mriedem> tonyb: yeah, i need to chat with dhellmann
21:27:58 <tonyb> mriedem: really (2 talks?) /me will be interested in them
21:28:00 <rockyg> The one guy I've got is supposed to help with keeping stable more current/working longer
21:28:15 <mriedem> tonyb: well, they are sort of similar
21:28:28 <tonyb> mriedem: I think sneek it into release/cross-project but I think there shoudl be 45mins to focus on it ...
21:28:41 <rockyg> ++
21:28:50 <mriedem> yeah, i do see value in re-assessing the support timeline on stable/liberty
21:29:04 <mriedem> #action mriedem to talk to dhellmann about cross-project discussion for stable/liberty timeline
21:29:23 <mriedem> ok, tonyb you had something for open discussion?
21:29:29 <tonyb> mriedem: yeah
21:29:36 <rockyg> I also brought up the topic in prod-wg.  Got them up to speed on what ops want/need and how to help.
21:29:37 <dhellmann> mriedem : seems reasonable as a cross-project session topic, when the tc starts putting that list together
21:29:40 <tonyb> So we all knwo the bug-smash is comming up
21:30:12 <tonyb> my focus is going to be on addressing stuff that's appropriate for stabel but isn't backported
21:30:19 <tonyb> I have 3 bugs for nova.
21:30:25 <tonyb> I wanted to check a coupel of things
21:30:34 <tonyb> 1) is that actually helpful?
21:31:03 * mriedem waits for #2
21:31:26 <mriedem> when is the great bug smash happening again?
21:31:32 <tonyb> 2) assuming yes do others want to drop me a list of appropriate bugs to look at
21:31:53 <mriedem> well for stable it's just a matter of backporting the fixes and reviewing them, right?
21:32:13 <tonyb> the bug-smash is a bit of a crap-shoot so I can't make promises but I was hopign to get some attention on stable/liberty befoer we go into phase II support
21:32:21 <mriedem> in general i think it would be helpful to have a concerted effort to review backports for a given project/release and flush them through
21:32:22 <tonyb> mriedem: 7-9March
21:32:23 <mriedem> like this week for stable/liberty
21:33:09 <mriedem> ok, so next week
21:33:16 <tonyb> mriedem: yeah.
21:33:49 <mriedem> i sort of view this as the stable branch liaisons for each project should be driving this kind of thing
21:33:54 <tonyb> I was thinkign that master will be pretty much closed to anything that isn't a release blocker but stable would be a little more open
21:34:06 <rockyg> I think you can distribute bugs to get fixed to some of the etherpads that look like matches.  Like china is nova and neutron
21:34:10 <tonyb> mriedem: I was hopign that a few of them would read this :)
21:34:17 <rockyg> Then, just be ready to review quickly.
21:34:54 <tonyb> rockyg: yeah that's pretty much the point
21:35:27 <mriedem> tonyb: so as a run up to next week, are you thinking about running your script on a bunch of projects and posting the results in an etherpad or something?
21:35:47 <rockyg> Master will be closed for Mitaka, but open for new bugfixes. Might be good to review all fixed bugs that happen during smash and add Backport potential to the ones that should.  Review nightly?
21:36:04 <tonyb> mriedem: https://etherpad.openstack.org/p/OpenStack-Bug-Smash-Mitaka-Bugs
21:36:14 <tonyb> that has the nova/liberty stuff
21:36:30 <tonyb> mriedem: I can do it for cinder/neutron as well
21:37:06 <mriedem> and keystone, and glance...
21:37:09 <tonyb> mriedem: the "challenge" (that you pointed out before) si that we don't fir sure have cores in thos projects on board (yet)
21:37:23 <tonyb> but at least for nova we have me, mikal and (hopefully you)
21:37:27 <bknudson> there's no bugs in keystone
21:37:27 <tonyb> mriedem: Sure
21:37:50 <bknudson> we need people to do reviews not post more patches
21:37:59 <tonyb> bknudson: I'll just run my script .....
21:38:32 <jokke_> tonyb: I'm glance and I will be working again from tomorrow onwards so I will be way more available
21:38:34 <tonyb> bknudson: I hear that.
21:38:51 <mriedem> i can see value in at least putting bugs in the etherpad for people to do the backports,
21:38:58 <rockyg> The China folks have recruited heavily for cores for the areas they are targetting
21:39:01 <tonyb> jokke_: is that something (liberty backports) you're interested in?
21:39:03 <jokke_> I had unexpected month of holidays that bit broke the plans for the start of this year :P
21:39:05 <mriedem> but there is maybe a separate push for the stable team / CPLs to actually review them
21:39:29 <jokke_> tonyb: as stable liaison for glance, yes backports are definitely in my interest
21:39:37 <tonyb> jokke_: okay.
21:39:41 <tonyb> jokke_: Thanks.
21:39:54 <jokke_> tonyb: no no, thank you! ;)
21:40:14 <rockyg> mriedem, ++ for the backports in the etherpads.  Especially if the bug is already on the ehterpad and it gets fixed.
21:40:31 <tonyb> Frankly I'm tryign to turn the bug-smash into somethign *helpful*.  I really don't want to distract
21:41:33 <tonyb> bknudson: this keystone bug is tagg with librty-potential but no patch has been proposed: https://bugs.launchpad.net/keystone/+bug/1525250
21:41:33 <openstack> Launchpad bug 1525250 in OpenStack Identity (keystone) "Failure when federated user name contains non ascii characters" [High,Fix released] - Assigned to Steve Martinelli (stevemar)
21:41:46 <mriedem> tonyb: i think that's fine
21:41:49 <tonyb> bknudson: so for keystone it's the only one
21:42:04 <mriedem> honestly we probably don't have the tag on a lot of bugs,
21:42:11 <mriedem> bugs that are reported in kilo or liberty, but fixed in mitaka
21:42:15 <tonyb> mriedem: Yeah that was my feeling.
21:42:30 <mriedem> but someone could report it in the liberty timeframe and be using juno
21:42:31 <mriedem> idk
21:42:34 <tonyb> mriedem: Hmm I wonder how I could find those ....
21:42:46 <mriedem> i think that's pretty hard given ^ scenario
21:42:50 <mriedem> we see that a lot in nova bugs
21:42:53 <tonyb> mriedem: yeah.
21:43:00 <mriedem> could be reported in december of 2015 but they are using juno or kilo
21:43:06 <rockyg> mriedem, yeah.  Maybe put a note in bold above the list of bugs on each etherpad to check whether the bug getting fixed should be backported?  A reminder when they change the status on the etherpad.
21:44:07 <tonyb> mriedem: Yeah I could look for *juno* or *kilo* in comment0 but I think that'd be messy
21:44:26 <mriedem> tonyb: i don't really think it's worth it
21:44:52 <tonyb> mriedem: okay.
21:44:59 <mriedem> i think teams just have to be on top of tagging things if they seem valid for backporting, like what ihar has been doing in neutron
21:45:07 <tonyb> mriedem: now that I have the ground work playign isn't too hard
21:45:41 <tonyb> +1 neutron is doign great there.  did they start doign frequent releases?
21:45:49 <mriedem> more frequent yes
21:45:58 <rockyg> which is also great
21:46:21 <mriedem> assuming they aren't backporting regressions or bugs, but that's hard to know :)
21:46:51 <rockyg> Uh, yeah ;-)
21:46:55 <tonyb> mriedem: :)
21:47:08 <mriedem> anyway, i don't really have anything else for this topic
21:47:23 <rockyg> I've got a topic....
21:47:26 <mriedem> ok
21:48:13 <rockyg> Prod_wg is working on a "user_story" for longer lasting stable releases.  In fact, I'm writing a good bit about it.  I'll cc you guys when I get the next draft.  It's RST in gerrit
21:48:47 <mriedem> ok
21:49:13 <rockyg> I'm trying to dovetail some needs/desires on the user community side with reality.  And get more people dedicated (at least part time) to this team.
21:49:48 <tonyb> rockyg: cool.
21:49:59 <mriedem> i'm assuming user in this case is more operator/deployer than actual cloud user
21:50:01 <rockyg> So, this is an FYI at the moment, but if you see yourself cc'ed on a review, please review?
21:50:02 <bknudson> it's also infra that has concerns about supporting stable longer
21:50:18 <mriedem> bknudson: yeah, and QA
21:50:22 <rockyg> Yeah.  The HW needed to support it
21:50:25 <mriedem> that's why it's generally 4 teams when talking about timelines
21:50:35 <mriedem> stable, release, QA and infra
21:50:45 <mriedem> b/c tempest changes are tested on all branches too
21:50:50 <rockyg> Ain't that fun;-)
21:51:29 <rockyg> Yeah.  And the tempest changes propagating to stable is an interesting thing.
21:52:14 <mriedem> well, tempest is branchless
21:52:16 <rockyg> Although tempest does get a tag at about the time of each release
21:52:23 <mriedem> so it has to make sure that tempest master works with stable service APIs
21:52:35 <mriedem> and you can't drop things or add things to tempest that would fail on stable
21:53:02 <rockyg> Yeah, but the branchless doesn't work so well for DefCore/Refstack
21:53:15 <mriedem> why?
21:53:18 <rockyg> Because tests change that are not backward compatible.
21:53:28 <mriedem> for like an icehouse cloud?
21:53:52 <rockyg> Like all "extra options" were turned off for Kilo, but Juno still can use them.  So Juno tests fail.
21:54:09 <rockyg> That's nova stuff.
21:54:31 <mriedem> i guess i'd need specific examples/details
21:54:37 <mriedem> which we don't have time to get into here
21:54:49 <mriedem> is this in the review or in the ML?
21:55:01 <rockyg> But, other tests had bugs, so you have to pick and choose git IDs to find the "just right" version where bugs are fixed but tests still work.
21:55:17 <rockyg> It's in the DefCore mailing list.
21:55:31 <rockyg> And in the DefCore meeting minutes.
21:56:03 <mriedem> ok, neither of which are things i've been privy too yet
21:56:18 <mriedem> so a link to the ML would be helpful
21:56:25 <rockyg> I can get you the thread.  Just not before the end of the meeting;-)
21:56:36 <rockyg> Is there an IRC channel for this group?
21:56:43 <rockyg> I can post it there.
21:56:46 <tonyb> #opensatck-stable
21:56:47 <jokke_> rockyg: #openstack-stable
21:56:58 <rockyg> Thanks.  I'll add it to my "favorites.
21:56:58 <tonyb> rockyg: only without typos :)
21:57:07 <mriedem> ok, with that, let's wrap this up
21:57:17 <mriedem> anything else for 3 minutes left
21:57:18 <mriedem> ?
21:57:29 <tonyb> mriedem: not from me
21:57:41 <mriedem> ok, i'm assuming you're speaking for everyone too
21:57:46 <jokke_> :)
21:57:47 <mriedem> thanks everyone
21:57:49 <jokke_> thanks all
21:57:49 <mriedem> #endmeeting