14:01:29 <dhellmann> #startmeeting releaseteam
14:01:29 <openstack> Meeting started Fri Feb  5 14:01:29 2016 UTC and is due to finish in 60 minutes.  The chair is dhellmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:32 <openstack> The meeting name has been set to 'releaseteam'
14:01:52 <dhellmann> our agenda is under R-9 on the etherpad
14:01:53 <dhellmann> #link https://etherpad.openstack.org/p/mitaka-relmgt-plan
14:02:37 <dhellmann> courtesy ping: ttx, dims, lifeless
14:02:45 <ttx> o/
14:02:49 <dhellmann> sorry I'm a bit tardy this morning
14:02:55 <dims> o/
14:03:05 <dhellmann> #topic old business (from R-11)
14:03:12 <dhellmann> #info dhellmann talk to monasca team about releases
14:03:24 <dhellmann> #link governance tag updates: https://review.openstack.org/#/c/274884/
14:03:24 <dhellmann> #link releases repo updates: https://review.openstack.org/#/c/274898/ (merged)
14:03:39 <dhellmann> I think that's taken care of, once the governance change merges
14:03:52 <dhellmann> oh, there was one other update to releases to add monasca-ceilometer
14:04:04 <dhellmann> #info dhellmann talk to mestery about networking-ovn, octavia, vmware-nsx
14:04:11 <dhellmann> #link https://review.openstack.org/#/c/272696/
14:04:40 <dhellmann> I don't think there was a releases repo change for that one
14:05:01 <dhellmann> #info dhellmann change searchlight and aodh to cycle-with-intermediary
14:05:01 <dhellmann> #link https://review.openstack.org/271366 change aodh to cycle-with-milestones (merged)
14:05:01 <dhellmann> #link https://review.openstack.org/271369 change searchlight to cycle-with-milestones (merged)
14:05:12 <dhellmann> #info dhellmann send email encouraging independent projects to record their releases in the releases repository
14:05:19 <dhellmann> #link http://lists.openstack.org/pipermail/openstack-dev/2016-January/084756.html
14:05:37 <dhellmann> we've seen a few updates proposed, but I haven't done any sort of analysis of whether we're complete
14:06:29 <dhellmann> I think we said we'd always expect to have missing data, didn't we?
14:07:18 <dhellmann> ttx: ?
14:07:31 <ttx> yeah
14:07:43 <ttx> But we should have an easy way to sync it back
14:08:11 <dhellmann> yeah, we have some scripts for importing the data but they try to work out the release cycle. we should make a note to fix one up for independent projects
14:08:22 <ttx> although it's obviously manual, given how many erroneous tags are pushed
14:08:55 <dhellmann> true
14:09:15 <dhellmann> #info ttx to file the magnum fix today
14:09:15 <dhellmann> #link https://review.openstack.org/271318 (merged)
14:09:28 <dhellmann> #info dhellmann file governance change to merge freezer deliverables for consistency
14:09:28 <dhellmann> #link https://review.openstack.org/271372 merge freezer deliverables (merged)
14:09:43 <dhellmann> #info dhellmann reorganize releases repository to clean up urls
14:09:43 <dhellmann> #link https://review.openstack.org/271550 rearrange content in preparation to move to new vhost (merged)
14:09:55 <dhellmann> and we skipped our meeting last week
14:10:13 <dhellmann> did I miss anything for old business?
14:10:32 <ttx> nope
14:10:36 <dhellmann> moving one then
14:10:41 <dhellmann> #topic should we make pre-release collapsing the default behavior in reno?
14:11:08 <dhellmann> this week I added the feature to reno to let it optionally collapse pre-releases into the final release, when that final release is available
14:11:30 <dhellmann> I left it off by default, which means turning it on everywhere will require editing rst files in the release notes builds
14:11:47 <dhellmann> I wonder if we should turn it on by default, instead, so we get the consistency we want?
14:12:27 <dhellmann> we can have both flags, so projects can be specific
14:12:35 <dhellmann> sorry, explicit
14:12:49 <dhellmann> thoughts?
14:13:52 <ttx> hm
14:14:13 <ttx> I would say turn it on by default
14:14:24 <dhellmann> that was where I was leaning
14:14:39 <dims> what's the down side dhellmann?
14:15:05 <dhellmann> dims : CD folks might want to know that something changed earlier in the cycle, rather than later
14:15:44 <dims> dhellmann : sure, earlier the better
14:16:07 <dhellmann> well, the change would mean notes released in 0b1 would show up as part of the final release when you build the release notes
14:16:08 <ttx> dhellmann: collapsing is what we always did
14:16:21 <dhellmann> yeah, that's why I thought the default should probably switch
14:16:26 <ttx> also not sure 3 milestones would really help CD
14:16:27 <dhellmann> ok, I'll put that on my list for next week
14:16:41 <dhellmann> yeah, only for folks a few months behind master
14:16:46 <ttx> they should extract the reno from what they use anyway
14:16:54 <dhellmann> that's true
14:17:05 <ttx> so training them not to would not be a good idea
14:17:26 <dhellmann> #agreed change reno default to collapse pre-releases into final releases
14:17:31 <ttx> after all being able to generate notes at any point was a + for reno
14:18:12 <dhellmann> yep
14:18:23 <dhellmann> next up...
14:18:25 <dhellmann> #topic how do we want to handle locking down releases for libraries during the freeze period?
14:18:43 <dhellmann> we've said we would be "more strict" about late library releases this cycle
14:18:54 <dhellmann> how will we implement that? go back to changing ACLs?
14:19:11 <dims> dhellmann : releases not managed by us?
14:19:24 <dhellmann> dims : that's a good part of the question, yeah
14:19:37 <dhellmann> if we only apply the rule to projects we manage, it's easy
14:19:37 <ttx> dims: I think what dhellmann wants is to avoid "accidental" extra releases
14:20:16 <ttx> We could set the acl to release branches like for liberty
14:20:32 <dims> sounds like that worked fine last time
14:20:47 <ttx> also otherwise it ends up on the stable team lap, which is not better
14:21:12 <ttx> although, that's different acls
14:21:13 <dhellmann> "set the acl to release branches"?
14:21:24 <ttx> so... two things
14:21:26 <dims> so when we do that, will teams have to go through us for "exceptions"?
14:21:32 <dhellmann> dims : yeah
14:21:47 <ttx> when you do the first release you cut a stable/mitaka branch
14:21:57 <ttx> master continues on newton
14:22:20 <ttx> that stable/mitaka branch has acls set for approving changes to stable team
14:22:29 <dhellmann> ah, so go ahead and create the stable branch for libraries and assume we'll use backports
14:22:32 <ttx> and the PTL can still tag everywhere.
14:22:43 <ttx> that's what happens if we do nothing
14:22:52 <dhellmann> I didn't remember doing that immediately for liberty. Did we?
14:23:13 <ttx> What we did in the past is switching ACLs for stable/mitaka to be under our control instead of the stable teams
14:23:33 <ttx> so changes can only land there if we approve them
14:23:45 <ttx> although the -release groups also include the ptl
14:24:01 <dhellmann> hmm, ok. I was less concerned about landing patches than releasing things.
14:24:05 <ttx> it doesn't prevent the tags
14:24:15 <ttx> but then you wouldn't tag if there is nothing landed
14:24:25 <dhellmann> true
14:24:26 <ttx> or would you rather also freeze the newton releases ?
14:24:59 <dhellmann> we need to freeze all library releases, because service projects will have mitaka in their master branch for a few more weeks
14:25:17 <ttx> OK, so options;
14:25:22 <dhellmann> we didn't unfreeze lib releases until after the stable branches were cut last time, right?
14:25:49 <ttx> option A: do nothing -- stable teams control what land in stable/mitaka release bracnehs, PTLs can still "accidentally" tag
14:26:37 <ttx> option B: do what we did for kilo/liberty, switch the stable/mitaka ACL to us -- release team controls what lands in stable/mitaka release branches, PTLs can still "accidentally" tag
14:27:15 <dhellmann> hmm
14:27:20 <ttx> option C: do what we did in liberty + claim back tagging on libraries -- release team controls what lands in stable/mitaka release branches, PTLs can't accidentally release libs
14:27:50 <ttx> I suspect what you wanted to discuss now is the B->C
14:27:53 <dhellmann> we've split the stable and release duties this cycle
14:28:11 <dhellmann> I think I want option D: claim back tagging on libraries and let the stable teams land patches
14:28:30 <dhellmann> we have project-specific stable teams now, so we can distribute the review load to them
14:28:38 <ttx> hmm, for libs yes
14:28:46 <dhellmann> but prevent tagging of undesirable releases just during the freeze window
14:29:00 <ttx> We would only switch approval ACLs on cycle-with-milestones projects
14:29:21 <ttx> so that those require $PROJECT-release approval post RC1
14:29:43 <ttx> (which includes us/me but would likely be handled by release liaison there)
14:29:53 <dhellmann> I suppose since the ultimate goal of all of this is to avoid having releases inserted into the gate, we could also just stop approving changes to requirements
14:29:56 <ttx> And we would only claim back tagging on libs
14:29:57 <dhellmann> and leave the ACLs alone
14:30:39 <dims> dhellmann : there are jobs that are not constrained by g-r
14:30:42 <dhellmann> because even for cycle-with-intermediary projects like oslo, we don't want new versions of libs introduced unless absolutely necessary
14:30:45 <ttx> dhellmann: what dims said
14:30:50 <dhellmann> dims : yeah, so that's not enough
14:31:11 <dims> so option C
14:31:13 <dhellmann> so we should *also* stop approving requirements changes, but we'd do that anyway
14:31:18 <dims> right
14:31:33 <ttx> dims: you actually need different things for libs and cycle-with-milestones projects
14:31:34 <dims> stop approving releases and requirements in addition to the ACL
14:31:57 <dims> ah i see
14:31:57 <ttx> the ACL switch on patch approval is an artifact of the RC system, which only cycle-with-milestones uses
14:32:15 <ttx> so let's forget about that for the moment
14:32:22 <dhellmann> yeah, that's a separate issue, I think
14:32:30 <ttx> The question is... should we actively prevent library tagging during the freeze
14:32:37 <dhellmann> I thought we would want to lock down *all* library releases
14:32:42 <dhellmann> not just the ones we manage
14:32:53 <dhellmann> so maybe we should start by addressing the scope question
14:32:59 <ttx> blocking requirements sounds like partial damage control
14:33:07 <ttx> not really a solution
14:33:26 <ttx> we can either trust them and pray... or prevent it using ACLs
14:33:26 <dhellmann> yeah, requirements freeze is part of the process anyway but doesn't yet protect us because we don't have full coverage for constraints
14:33:46 <dhellmann> at the end of liberty when we said we would be more strict, what caused that? did we have an issue?
14:34:05 <ttx> yeah, we had a number of python-*client late releases
14:34:21 <dhellmann> right, I remember now
14:34:36 <ttx> like the week before
14:34:58 <dhellmann> ok, so let's limit the discussion to only libraries, then, and talk about whether it's *all* or *managed* projects that we'll block, because that has an implication on the solution
14:35:09 <ttx> I think we could try what better communication of deadlines leads us to
14:35:36 <dhellmann> so only block releases for managed projects, and communicate more strongly that other projects are expected to not do releases?
14:35:45 <ttx> arguably the liberty situation was partly due to miscommunication of deadlines
14:36:11 <ttx> if /that/ fails we should ACL
14:36:22 <ttx> but by then I expect we'll have all autmated
14:36:39 <dhellmann> next cycle? I hope so.
14:36:59 <dhellmann> ok, I'll write up an email about the release freeze period
14:37:02 <ttx> My point is.. last time it was partly our fault for not being clear enough.
14:37:15 <dhellmann> I know the start date, do we have an end date?
14:37:36 <dhellmann> sure, I think we need to be more clear and explicit this time
14:37:47 <ttx> let me check
14:37:58 <dhellmann> http://docs.openstack.org/releases/mitaka/schedule.html
14:38:04 <ttx> "requirements unfrozen" at R-2
14:38:32 <dhellmann> ok, I'll add that to the schedule, too
14:38:57 <dhellmann> hmm, I don't see "requirements freeze" at all on the schedule; is that implied by something else?
14:39:14 <ttx> see https://etherpad.openstack.org/p/mitaka-relmgt-plan
14:39:26 <dhellmann> ah
14:39:37 <dhellmann> ok, I'll add those dates to the published mitaka schedule today
14:39:47 <dhellmann> #action dhellmann add requirements freeze dates to mitaka calendar
14:40:08 <ttx> dhellmann: start date is implied already
14:40:14 <dhellmann> #action dhellmann announce library release freeze deadlines and thaw on ML
14:40:31 <ttx> R-5: Final release for client libraries
14:40:36 <dhellmann> ttx: Feb 26
14:40:37 <dhellmann> right
14:40:44 <ttx> What's unclear is when the freeze ends
14:41:27 <ttx> not sure that's significant for the schedule though
14:42:00 <ttx> we could add a note on the explanation for " Final release for client libraries"
14:42:23 <ttx> something like: "You can't release until the requirements freeze is lifted, generally at R-2"
14:42:28 <dhellmann> I would like to thaw them when the stable branch for the requirements repo is created
14:42:37 <ttx> might be better since we usually miss that deadline due to $fail
14:43:18 <dhellmann> since that branch makes it safe to start releasing into master test jobs again
14:43:21 <ttx> N-2 is a target week, but we consistently missed it
14:43:35 <ttx> which is why it's in the plan but not the schedule
14:44:09 <dhellmann> so rather than a date, maybe like you say, give the thing that is triggering the thaw
14:44:21 <ttx> +1
14:44:22 <dhellmann> "R-2 at the earliest"
14:44:38 <dhellmann> meh, your phrasing is more optimistic :-)
14:44:56 <dhellmann> ok, I'll draft that and run it past everyone before sending
14:45:01 <ttx> right. Good for the email and the comments in the schedule, but I would leave it out of the schedule grid
14:45:08 <dhellmann> right
14:45:22 <dhellmann> ok, speaking of schedule grid...
14:45:25 <ttx> heh
14:45:29 <dhellmann> #topic starting to think about newton schedule
14:45:29 <dhellmann> it would be good to be able to tell event planners what the dates are
14:45:30 <dhellmann> #link https://review.openstack.org/#/c/275911/
14:45:40 <dhellmann> ttx, you had some comments about the duration of milestones
14:45:41 <ttx> We could work on that one now
14:45:50 <dhellmann> I copied what we had this cycle, just as a place to start
14:46:38 <ttx> right, so the 5/7/6/5 map we used last time was mostly driven by the holiday season
14:46:50 <ttx> i.e. use 7 weeks where we know we'll lose 1.5 week
14:47:01 <dhellmann> right
14:47:11 <ttx> it may or may not make sense to replicate it
14:47:33 <ttx> first thing to discuss would be the FF
14:47:55 <ttx> Used to be R-6. do we think the R-5 week will be successful ?
14:48:14 <ttx> should we wait until we are deep in it to make that call ?
14:48:29 <ttx> (and only publish the dates we know of, like summit and release ?
14:48:30 <dhellmann> that sounds wise
14:48:53 <dhellmann> I started pulling it together to try to identify the n-1 milestone for the bug-squash folks
14:48:55 <ttx> maybe we should just publish the things people can start building on
14:49:20 <ttx> some things are already set in stone:
14:49:21 <dhellmann> so split the patch up?
14:49:31 <ttx> design summit date, election schedule
14:49:51 <ttx> The release date is 99% sure, given the position of the Barcelona summit
14:50:32 <ttx> I think we should publish that already
14:50:46 <dhellmann> ok, I'll split this up into "known dates" and "tentative dates" patches
14:50:51 <ttx> the newton-1 milestone... depends a bit on the map, which depends on the FF position
14:51:11 <dhellmann> do you expect it to be any earlier than r-18?
14:51:13 <ttx> so I don't think we can decide that today... would prefer to have some idea of how well the 5-week thing is going
14:51:35 <dhellmann> I could at least tell the bug-smash folks that as a likely earliest date
14:52:33 <ttx> So one difference with the previous cycle is the distance to the previous release
14:53:08 <ttx> we have one more week pre-summit
14:53:13 <dhellmann> hmm, yeah
14:53:17 <ttx> (which should probably appear on the newton schedule btw)
14:53:23 <dhellmann> I should add the missing dates before and after the actual cycle
14:53:35 <ttx> so we have a r-25
14:53:50 <dhellmann> righty
14:53:54 <dhellmann> er, right :-)
14:54:05 <ttx> By default I would go with a 5/7/7/5 map
14:54:42 <ttx> i.e. or maybe 6/6/7/5
14:54:52 <dhellmann> that feels nicely balanced
14:55:11 <ttx> so R-19 /R-18 for n1
14:55:24 <ttx> still need to crosscheck with holidays
14:55:47 <dhellmann> yeah
14:56:04 <ttx> so yeah, a bit too many unknowns to set the milestones now
14:56:19 <dhellmann> I'll start by adding the extra week and the known fixed dates, and split the unknowns into a separate patch for further discussion for now
14:56:29 <ttx> maybe split it, and we can keep the milestone map in WIP
14:56:35 <dhellmann> right
14:56:36 <ttx> that
14:56:57 <dhellmann> ok, one more thing before we run out of time
14:57:15 <dhellmann> #topic patches to prioritize
14:57:15 <dhellmann> #link https://review.openstack.org/#/c/275829/ show more detail in list-changes job
14:57:15 <dhellmann> #link https://review.openstack.org/#/c/275853/ do not merge stdout & stderr when trying to check ancestry (I think this led to the keystoneclient 2.1.1 issue, but I'm not sure)
14:57:15 <dhellmann> #link https://review.openstack.org/275986 remove the announce flag from the release wrapper scripts
14:57:23 <dhellmann> some of those might already have merged
14:57:35 <dhellmann> yeah, the first two did
14:57:37 <ttx> all of them actually
14:57:42 <dhellmann> heh, ok
14:57:47 <dhellmann> so nevermind :-)
14:57:54 <dhellmann> #topic open discussion
14:58:09 <ttx> nothing to add
14:58:23 <dhellmann> I'll be working with fungi on the release announcement automation again today
14:58:30 <ttx> I'll be traveling Wed-Thu next week
14:58:35 <dhellmann> I expect it will be next week before we get to the releases.o.o work
14:58:44 <ttx> and then in vacation, the week after
14:59:15 <fungi> yeah, i'm around today though i do need to disappear for a good chunk of this afternoon
14:59:17 <ttx> thatand then, the pre-FF sprint
14:59:33 <fungi> dhellmann: you did see the release announcement job worked finally last night though?
14:59:39 <ttx> checking everything seems to be ready for the big thing
14:59:46 <dhellmann> fungi : I hadn't checked yet, but will look
15:00:06 <fungi> i assumed it ended up in the -announce ml moderation queue
15:00:10 <fungi> but the log was good
15:00:19 <fungi> anyway, we can move to -infra
15:00:25 <dhellmann> fungi : great, I'll check the queue (I just signed on right before the meeting)
15:00:29 <dhellmann> yep, we're out of time here
15:00:32 <dhellmann> thanks, everyone!
15:00:36 <dhellmann> #endmeeting