16:00:02 <smcginnis> #startmeeting releaseteam
16:00:03 <openstack> Meeting started Thu Sep 19 16:00:02 2019 UTC and is due to finish in 60 minutes.  The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:07 <openstack> The meeting name has been set to 'releaseteam'
16:00:10 <smcginnis> Ping ttx dhellmann diablo_rojo hberaud evrardjp armstrong tonyb
16:00:20 <armstrong> o/
16:00:32 <smcginnis> #link https://etherpad.openstack.org/p/train-relmgt-tracking Agenda
16:00:33 <fungi> heyhey kids!
16:00:46 <hberaud> o/
16:01:18 <smcginnis> Looks like we're around line 490 in the tracking etherpad.
16:01:29 <ttx> o/
16:01:58 <evrardjp> o/
16:02:34 <smcginnis> OK, looks like that's probably everyone.
16:02:39 <smcginnis> #topic What if anything to do with lib release stragglers
16:03:14 <smcginnis> #link https://etherpad.openstack.org/p/train-library-tracking
16:03:29 <smcginnis> I haven't rerun that to see if anything has changed, but I don't think so.
16:04:00 <smcginnis> Most are probably fine, other than some of these then needing to be backported to stable/train after branching.
16:04:32 <ttx> smcginnis: at one point we said that every change needs to be released
16:05:15 <smcginnis> Some of these are probably minor enough that they could just be considered ussuri changes.
16:05:26 <smcginnis> Some not though.
16:06:57 <ttx> hmm
16:07:37 <ttx> I think it's too late now, and those should be considered Ussuri
16:07:56 <smcginnis> Yeah. :/
16:08:05 <ttx> If they want those in Train, follow the FFE procedure
16:08:26 <ttx> there is only so much we can do to prevent library authors from shooting themselves in the foot
16:08:41 <ttx> I thought we would autorelease libs though
16:08:51 <smcginnis> It's kind of late, but should we send out this list to the ML.
16:09:20 <ttx> line 469 says so
16:09:35 <ttx> and line 443
16:09:46 <ttx> "Generate release requests for all libraries which had changes. "
16:09:50 <ttx> what did we miss?
16:10:13 <smcginnis> Good question.
16:10:34 <ttx> did those changes land after those releases?
16:10:51 <ttx> (and our error is that we should have branched earlier)
16:11:05 <ttx> or did they somehow fall through
16:11:27 <smcginnis> Looking at the timestamps, some did merge after.
16:11:47 <smcginnis> Some were there, but maybe since they were only one or two they didn't get release requests?
16:11:51 <ttx> so the issue is that we failed to branch?
16:12:08 <smcginnis> But for the freeze, we need to make sure even single commits get released.
16:12:33 <ttx> smcginnis: alternatively, lets release all of them
16:12:39 <ttx> but needs to be this week
16:12:47 <ttx> otherwise that would be pushing it
16:13:19 <ttx> hmm that would mean a bunch of requirements bumps... probably not a great idea
16:13:30 <smcginnis> Yeah
16:13:59 <smcginnis> Actually, I was able to remove a few that we did end up doing releases for after I generated that.
16:14:03 <fungi> but better this week than any closer to release
16:14:21 <smcginnis> So most are just trivial changes that probably are OK to count as ussuri changes.
16:14:38 <smcginnis> And the ones that are not, those teams can backport to stable/train and do a stable release once the freeze is over.
16:14:40 <ttx> yeah
16:14:46 <ttx> Let me cross those out
16:14:52 <smcginnis> It's just a matter of a few weeks yet.
16:15:33 <smcginnis> So unless something is completely broken, at this point I say let's get that branching patch merged and treat these as normal stable blackport issues.
16:15:44 <ttx> yeah
16:15:44 <smcginnis> If the teams actually feel like they need to backport, that is.
16:16:20 <ttx> But I'd still check why those were missed in earlier autoreleases
16:16:22 <smcginnis> A few are things for the pdf docs. Those are fine just getting picked up with ussuri I think.
16:16:47 <ttx> like is it all changes that landed after the autorelease
16:17:05 <smcginnis> Realizing the timestamps here are the commit timestamps, not the merge timestamps, so it's possible these actually did land after the autorelease.
16:17:17 <ttx> yes, let's not solve it in-meeting
16:17:37 <ttx> adding to next week tasks
16:17:43 <evrardjp> I am kinda lost in all of that, so I have trouble to follow
16:17:44 <smcginnis> For not, let's just all take a mental note to pay close attention to that next time around too.
16:18:06 <ttx> smcginnis: maybe it's just that we should have branched a deadline time
16:18:12 <ttx> branched at
16:18:12 <smcginnis> evrardjp: Basically the issue is for the autoreleases we do for milestone 1 and 2, if it's only 1 or 2 commits we might skip doing them.
16:18:32 <smcginnis> evrardjp: But for the final freeze, we need to make sure all commits are released so stable branch creation includes all work done.
16:18:59 <smcginnis> The ones that didn't get included now either have to be part of ussuri, or we would need to do a requirements FFE to get those out now.
16:19:12 <evrardjp> got it.
16:19:14 <smcginnis> Which carries the risk that it would have a ripple effect of making others have to do releases.
16:19:19 <ttx> ok task added, let's move on
16:19:22 <evrardjp> yeah
16:19:25 <smcginnis> ++ thanks
16:19:38 <smcginnis> #topic Review accomplished week tasks
16:19:46 <smcginnis> 1- kayobe
16:19:53 <ttx> yeah that was fixed and merged
16:20:13 <smcginnis> 2- old deliverables
16:20:16 <smcginnis> Only ironic
16:20:18 <smcginnis> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-September/009515.html
16:20:22 <ttx> Only Ironic fell in that category
16:20:30 <ttx> PTL warned and acked
16:21:10 <smcginnis> 3- Countdown email content
16:21:13 <ttx> Proposed email is down to line ~463 of https://etherpad.openstack.org/p/relmgmt-weekly-emails
16:21:31 <smcginnis> I reviewed right before the meeting. Everything looks good to me.
16:21:38 <smcginnis> Anyone else have any feedback on it?
16:22:02 <ttx> there is a subtlety beyond what the process mandates here
16:22:13 <smcginnis> If not, I will send this out either later today or tomorrow morning.
16:22:31 <ttx> basically on RC1 week we do cycle-with-rc's RC1s and branching, as well as force things that did not release at all yet
16:23:14 <smcginnis> Which by this point should only include cycle-with-intermediary service deliverables.
16:23:17 <ttx> But cycle-with-intermediary things that have done a train release (that we can fallback to), we give them an extra week to refresh
16:23:19 <smcginnis> Or "other"
16:23:53 <ttx> before we branch
16:24:15 <ttx> so next week all RC things and all libs should be branched
16:24:25 <ttx> and the week after, all intermediary things should be branched
16:24:43 <smcginnis> So branch cycle-with-rc. Release cycle-with-intermediary but wait to force a branch until the following week. Sound about right?
16:24:59 <smcginnis> They can propose the branch creation if they want to before then of course.
16:25:20 <ttx> well, intermediary things that have not released any train, I would include the branch creation in the autorelease
16:25:29 <ttx> (next week)
16:25:54 <ttx> because it's unlikely that they are following along
16:26:03 <smcginnis> Yeah, true.
16:26:13 <ttx> I mean they can always stop us
16:26:23 <ttx> I think we should systematically branch in the autorelease
16:26:27 * smcginnis checks if we've captured the gist of this in next weeks tasks
16:26:34 <ttx> they can -1 it if they object
16:26:43 <ttx> and it would save us extra work down teh line
16:26:49 <ttx> if they don;t
16:26:57 <ttx> yes
16:27:45 <smcginnis> Ah, so that original note we want to change.
16:27:55 <smcginnis> It said no stable branch creation, but now we are saying yes we should.
16:28:00 <ttx> yes
16:28:09 <ttx> I see no reason not to
16:28:19 <ttx> I mean they can still object to it being cut
16:28:29 <ttx> But we should propose it by default imho
16:28:33 <smcginnis> And if they don't object, that's another indication that they are not playing along.
16:29:03 <ttx> hmmm what about cycle-automatic
16:29:09 <ttx> says we should not
16:29:19 <smcginnis> Most of those are brachless (tempest plugins)
16:29:23 <ttx> ah right
16:29:51 <ttx> "Those do not need a stable branch created" says the definition
16:29:59 <smcginnis> ++
16:30:13 <ttx> ok let's continue reviewing tasks there
16:30:23 <smcginnis> #topic Review and assign next weeks tasks
16:30:53 <smcginnis> Really bad timing, but I am at two different conferences on opposite coasts of the US next week.
16:31:03 <smcginnis> So if someone else can sign up for these tasks, that would be best.
16:31:15 <ttx> did you use any script to generate those in the past?
16:31:19 <smcginnis> I will catch up and pitch in whenever I'm able to.
16:31:35 <smcginnis> This is our first cycle-automatic.
16:32:01 <smcginnis> I think Tony had started a script to do batch release proposals, grouping by teams.
16:32:05 <smcginnis> Not sure if that's posted or not.
16:32:36 <smcginnis> evrardjp: Didn't you have a nice bash script or something you used last time?
16:32:55 <evrardjp> mmm
16:33:39 <evrardjp> sorry catching up I was in another  meeting at the same time
16:34:11 <evrardjp> I don't have a script to batch release proposal handy, I probably removed it after use
16:34:37 <ttx> I can easily generate lists of things we need to release
16:34:38 <evrardjp> but I can help
16:34:55 <ttx> then we can pick up / script them somehow
16:34:56 <smcginnis> evrardjp: Would you have the time next week to take the release tasks for next week?
16:35:22 <smcginnis> Help from ttx, and me whenever I'm able to get online without too many distractions.
16:35:41 <evrardjp> as long as it's before wednesday, as I seem to have an extra travel on Thursday friday.
16:36:06 <ttx> has to be done early in the week
16:36:12 <evrardjp> yeah I see
16:36:28 <ttx> I wonder if we should not finalize a quick-and-dirty script
16:36:40 <smcginnis> I have events and travel on Thursday, but think I should be "butt in seat" on Friday.
16:37:03 <evrardjp> ttx: do you mind if we pair along tomorrow to quick-and-dirty the script?
16:37:18 <ttx> evrardjp: I can try...
16:37:28 <ttx> lots of things this week
16:37:36 <evrardjp> yeah I can start writing and you'll say it's plain wrong
16:37:42 <smcginnis> :D
16:37:43 <ttx> Oh I can do that
16:38:07 <ttx> yes, let's sync tomorrow
16:38:21 <ttx> Feels like the number of things to generate justifies it
16:38:57 <ttx> 31 cycle-automatic things
16:39:20 <ttx> and like 80 cycle-with-rc things
16:39:39 <smcginnis> And we'll need to do these cycle-automatic ones every cycle, so having a script will be important.
16:39:40 <ttx> + the cycle-with-intermediary stragglers... like 10 of them
16:39:54 <ttx> I just wish we had identified that need sooner
16:40:07 <ttx> could have worked on that in a plane!
16:41:01 <ttx> smcginnis: can you take the first task of analyzing why we missed things in libraries?
16:41:19 <smcginnis> Yeah, I'll try to take a look through those and see what might have happened.
16:41:31 <smcginnis> I'll add notes to that etherpad if I find anything interesting.
16:42:16 <smcginnis> OK, so I think we're set on next weeks tasks.
16:42:33 <smcginnis> I'm not 100% sure I'll be available at this time next week.
16:42:44 <smcginnis> Would one of you be able to run the meeting if I'm a no show?
16:42:47 <ttx> evrardjp: there are a bunch of scripts to propose releases already (new-release, propose-library-branches)
16:42:53 <evrardjp> I am 100% sure I won't smcginnis
16:42:55 <ttx> so it might just be very close
16:43:07 <evrardjp> ttx: yeah, that's what I used last time.
16:43:13 <ttx> should we move the meetign to Friday so that it's not just me
16:43:26 <smcginnis> So option can be we skip the meeting and just catch up asynchronously in channel as we're each available.
16:43:38 <smcginnis> Or yeah, as a special one time offer, we can move to Fridya.
16:43:46 <smcginnis> I would be available then.
16:43:57 <ttx> I think it would be good to process the autoreleases that did not get any feedback on Friday
16:44:27 <ttx> have a hard stop at 16utc on Friday
16:44:32 <ttx> can do anytime before
16:44:35 <openstackgerrit> Carlos Goncalves proposed openstack/releases master: Releases for Octavia Queens, Rocky and Stein  https://review.opendev.org/683202
16:44:44 <evrardjp> I think it's a good idea, but I cannot ensure I will be there, probably still in conference or in travel.
16:45:00 <smcginnis> ttx: I can do 1500 Friday.
16:45:21 <ttx> ok deal
16:45:36 <smcginnis> Great, updates on my calendar.
16:45:56 <smcginnis> I'll post a notice to the ML to let others know in case they care.
16:46:11 <ttx> ok
16:47:01 <smcginnis> #topic Open discussion
16:47:10 <smcginnis> Anything else for today then?
16:47:53 <ttx> http://lists.openstack.org/pipermail/foundation/2019-September/002794.html
16:48:04 <ttx> now the dats is official
16:48:06 <ttx> date
16:48:31 <smcginnis> Would be good to finalize the U schedule now that we have that.
16:48:40 <smcginnis> Does the current proposal align well?
16:50:24 <ttx> 3 empty weeks between release and event
16:50:42 <smcginnis> Seems pretty good then.
16:50:44 <ttx> which is a bit large, but not usual. And that cycle is long already
16:50:51 <ttx> not unusual I mean
16:50:58 <smcginnis> Better than having it the same week. ;)
16:51:07 <ttx> or the week after
16:51:29 <ttx> nothing like doing a release and packing luggage at the same time
16:51:37 <smcginnis> If we're good with it overall, I can get those nits updated.
16:51:57 <smcginnis> I'm sure some folks would appreciate having a schedule to help plan.
16:52:43 <ttx> yes, please update, then post a new ML post to flag it for final review?
16:52:59 <smcginnis> Good plan.
16:53:08 <smcginnis> OK, nothing more from me then.
16:53:27 <fungi> oh, one bit of good infra news... during the last gerrit restart we implemented a configuration change to disable auto-replication at startup, so you no longer need to be wary of approving releases immediately following a gerrit restart now that there's no longer a multi-hour replication backlog to contend with every time we do that
16:53:28 <ttx> nothing from me
16:53:49 <smcginnis> fungi: Great!
16:54:39 <fungi> so we'll still be wary of restarting the opendev gerrit if there are release jobs in flight so they're not lost, but triggering more right after a restart should be fine
16:54:44 <fungi> anyway, that's all i had
16:55:06 <smcginnis> OK, thanks everyone.
16:55:10 <smcginnis> #endmeeting