15:01:51 #startmeeting releaseteam 15:01:52 Meeting started Fri Feb 2 15:01:51 2018 UTC and is due to finish in 60 minutes. The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:53 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:55 thx all 15:01:56 The meeting name has been set to 'releaseteam' 15:02:03 Ping list: dhellmann, dims, fungi, tonyb, lbragstad, ttx, armstrong 15:02:18 * lbragstad refills coffee quick 15:02:19 o/ 15:02:23 #link 15:02:25 Here 15:02:28 #undo 15:02:29 Removing item from minutes: #link 15:02:33 * dhellmann is on a conference call 15:02:35 #link https://etherpad.openstack.org/p/queens-relmgt-tracking 15:02:43 We are on line 260 15:02:46 dhellmann: ack 15:03:14 * lbragstad is back 15:03:33 I'll wait another minute or so. 15:03:58 I'm here now 15:03:58 * fungi hopes you're not waiting on his account. he's just quiet 15:04:15 oh, right, now i see dhellmann's conf call comment 15:04:18 fungi: :) 15:04:48 Actually hoping ttx would pop in, but I'm sure he'll join us when he can. 15:04:53 #topic Queens-3 Status and Recap 15:05:10 Yeah... it's been a milestone. 15:05:18 more like a millstone 15:05:27 +10 15:05:42 So we now have all libraries released and branched. 15:05:52 Many were late due to various reasons. 15:06:05 rc1 is coming up next thursday, right? 15:06:13 And some never showed up, which I think we will have to discuss more. Probably when we are face to face at the ptg. 15:06:25 dhellmann: Yep, not a lot of time this time around. 15:06:40 maybe we can do a 5 minute talk after lunch one day about why we need releases 15:06:46 of the intermediary projects, that is 15:06:56 On the good side, we batched up all the late lib requirement bumps into one patch, and that went through last night. 15:07:02 oh, good 15:07:05 dhellmann: Oh, I like that idea. 15:07:14 That could be very useful. 15:07:31 we have some of that info in the irc logs from our reasoning through why we needed to do them, so it should be possible to pull together a slide or two 15:07:31 I think there are probably a large chunk of folks that don't really understand why we have the milestones and deadlines. 15:07:43 ++ 15:08:00 I do think we need to discuss what we're going to do with libraries that are stable enough that we aren't going to have releases each cycle, though 15:08:08 I think I closed the tab now, but I seem to remember an etherpad to collect those kinds of lunch talk ideas. 15:08:11 maybe those do just move to independent 15:08:22 That might make sense. 15:08:29 arh 15:08:29 #link https://etherpad.openstack.org/p/dublin-PTG-postlunch 15:08:35 I think we are going to have more of those dormant but stable projects going forward. 15:08:40 That hour just striked too early 15:08:46 indeed 15:08:48 anyway, that's a separate issue, because most of these things *did* have changes 15:09:08 Yes, and some of them not an insignificant number of changes. 15:09:15 maybe we need a gate job for libraries where they can't land more than 10 patches without releasing :-) 15:09:19 Which points to just a complete lack of attention to schedule. 15:09:33 dhellmann pulls out the big hammer. :) 15:09:46 Not a bad idea either though. 15:09:53 That would help get their attention. 15:09:55 ++ 15:09:56 yeah, i think things like requestsexceptions are, well, exceptions. very simple library, chances of not needing any changes in 6 months high 15:10:13 well, I'm sure it would go wrong, but maybe an automatic report would help 15:10:21 I mean blocking patches would go wrong, that is 15:10:25 that might help promote more of a release early, release often pattern? 15:10:29 So we have any remaining open infra issues ? 15:10:31 We do have the unreleased changes script. 15:10:39 fungi : as we stop syncing requirements, I think we're going to find a lot more of the oslo libs are also relatively stable 15:10:41 Do* 15:10:47 I was thinking of adding that to my set of cinder reports I automatically run and publish. 15:10:51 dhellmann: that would be... nice ;) 15:10:58 fungi : exactly 15:11:02 Yes I think it makes a good topic for PTG discussion to be fair 15:11:29 smcginnis : yeah, we could use that or build on its output 15:11:30 releasing in an age of stability and limited resources 15:11:52 ttx: I was thinking just the release team would be involved, but do you want to schedule a session for that? 15:12:06 I figured we'd just talk about it during our half day 15:12:11 dhellmann: first us coming up with a plan I'd say 15:12:14 but I could see it being of interest to a wider audience 15:12:22 makes sense 15:12:39 I mean we'll mention the topic being discussed obviously 15:12:45 We could start there and try to see if we need to follow on with something more. 15:12:48 maybe we hold off on the post-lunch presentation until later in the week then, so we can include the outcome? 15:12:55 ++ 15:12:58 assuming we have one 15:13:11 we may just mention it as a thing we're still working on 15:13:26 anyway, yeah, something to talk about at the ptg 15:13:48 So... q3 done, library release/branching done 15:13:59 Anything left to do on requirement freezing ? 15:14:07 An open infra issue ? 15:14:11 Any* 15:14:18 * smcginnis looks to fungi 15:15:12 nothing i'm aware of 15:15:24 On requirements, I see two conflicting asks for openstacksdk 15:15:29 aside from the releases site publication race 15:15:36 Me neither. I think we only have the rsync issue, but apparently that was always there but hidden. 15:15:52 ttx: For the g-r, not the u-c, right? 15:16:45 one sets u-c to ===0.9.19 15:16:58 the other sets g-r to >=0.11.2 15:17:03 So conflicting I'd say 15:17:21 I commented on the one. 15:17:24 https://review.openstack.org/#/c/540388/ 15:17:25 patch 540388 - requirements - Recap openstacksdk to 0.9.19 15:17:28 https://review.openstack.org/#/c/540343/ 15:17:28 patch 540343 - requirements - Update lower bound for openstacksdk to 0.11.2 15:17:50 I believe the newer one is needed, so raising the lower bounds looks like what is needed. 15:17:56 But maybe mordred can comment on there. 15:18:25 We need that one in before we start cutting RC1s, ideally 15:18:34 (or not in) 15:18:49 that's the only open requirement FFE I could find 15:18:50 I hope it is resolved today. 15:19:04 prometheanfire will likely handle it 15:19:05 oh ... /me looks 15:19:36 I don't think we should cap it back down ... we *definitely* need 0.11 for octavia-dashboard 15:19:54 I'd prefer raising the min - but if we don't it won't kill anyone since pip installs latest anyway 15:20:10 It would be more "correct" though. 15:20:14 exactly 15:20:32 mordred: can you make sure that whatever decision is made, it's made by Monday at the latest ? 15:20:45 We'll likely start to get RCs by then 15:21:03 and I'd like then to be up to date requirements-wise 15:21:07 ttx: sure! I actually defer to y'all on the right thing ... I mostly want to make sure we don't drop the constraints pin sincethat's the worst outcome 15:21:15 ok 15:21:16 I'm fine with any of the other outcomes that people feel are best 15:21:19 it sounds like raising the min is what we need 15:21:22 I'll -2 the constraints change 15:21:32 dhellmann: ack 15:21:34 mordred: With the octavia-dashboard dependency, it sounds like raising the minimum is the right approach to me. 15:21:42 smcginnis: OK looks like past week fun is under control. Let's move to discuss next week fun ? 15:21:49 Yep 15:21:52 #topic Prepping for RC1/ service branch week 15:22:11 #link http://git.openstack.org/cgit/openstack/releases/tree/PROCESS.rst#n190 Reference for current actions 15:22:52 * ttx copies over 15:23:05 smcginnis : in addition to encouraging them to wait, maybe we should encourage them to use depends-on instead of WIP 15:23:25 that made it a lot easier to track dependencies of the pending releases 15:23:31 As far as putting up an RC patch while they are waiting for final commits to land? 15:23:41 right 15:23:42 +1 for the tracking ability 15:24:10 It is nice being able to take a look at where things are, and just a WIP makes it unclear if they are still waiting or just forgot. 15:24:11 maybe that's just me, but the WIP makes the patch fall to the bottom of the dashboard 15:24:18 ++ 15:24:20 that's a good point, too 15:24:43 OK - deliverables missing any intermediary release. 15:24:51 Same kinds of issues we had with libs. 15:25:09 * smcginnis reads latest notes in etherpad 15:25:50 I wonder if patrole should really be cycle-based 15:26:31 we should probably send out a warning email about all of the projects (missing and old) today or monday 15:26:37 That's a tempest plugin, so I would think independent should be OK there. 15:26:51 I can do that. 15:27:18 #action smcginnis to send out email alerting to projects with old or missing intermediary releases 15:27:18 and rather than working through all of the logic again, I think we should just say we're going to tag HEAD if we don't get a release by RC1 15:27:32 That makes sense to me. 15:27:33 I think the same reasoning applies as for libraries 15:27:41 And is kind of our only good option in most cases it seems. 15:27:45 right 15:27:57 * ttx catches up after eterpad formatting fun 15:28:05 if they don't want us to do that, they should let us know what their plan is 15:28:42 so... historically we have been rather flexible for WHEN rc1 hits 15:28:54 ttx: we paste that content into the etherpad every cycle. maybe we should reformat the readme. 15:28:54 The one thing with all of this - I wonder if we should log this somewhere to make sure it's easy to go back and see what was release tema initiated and what was not. 15:28:55 it's been "when ready" with a ideal target week 15:29:08 As far as determining what projects need to be removed. 15:29:36 ttx: that's true. I guess if we wait to branch something the only thing that's going to break is that project, right? 15:30:09 We would still force a release and branch, but wait until just before final release? 15:30:20 hmm, yeah 15:30:29 Well, we need a RC1 for the same reason we need an intermediary release: faility to fall back on a recent tag 15:30:34 ability 15:31:30 so on one side we really want an rc1. On the other we don't want half-baked RC1s 15:31:43 since those are in theory releasable 15:32:01 this is why we have that half/half policy 15:32:27 tag RC1 when ready BUT ideally on that week 15:32:57 Maybe if nothing by Friday morning, we force it? 15:32:58 I think that's been fine for the projects that do it. We were concerned with the projects that don't propose one at all. 15:33:10 I'm not that concerned with RC1s... milestone projects just did a q3 and are aware 15:33:29 I'm more concerned with missing intermediaries, which is why I listed them 15:33:44 what about the intermediary projects? I'm calling those "release candidates" too, I guess, which is confusing. 15:34:29 yes 15:34:31 https://releases.openstack.org/reference/release_models.html#cycle-with-intermediary talks about preparing a release "near the end" of the cycle 15:34:58 historically we've had intermediaries being refreshed up to R-1 15:35:13 which is OK IF you have a fallback 15:35:15 smcginnis: Maybe if nothing by Friday morning, we force it? How? 15:35:21 https://releases.openstack.org/queens/schedule.html#q-finalrc has the dates for final intermediary releases as the week of 19 Feb 15:35:27 right 15:35:39 So I'm just concerned about those we don't have a good fallback for 15:35:43 armstrong : we propose a tag based on the HEAD of master for the project. 15:35:44 like we just did with a bunch of libraries 15:35:46 Missing intermediaries, or very old ones 15:35:49 armstrong: As in, the release team would propose the release instead of coming from the project itself. 15:36:05 For everyone else, we just branch at last intermediary 15:36:21 ttx: what's our cut off for defining "old"? 15:36:40 been using 6 weeks 15:36:50 ok, we should probably write that down 15:36:59 well, I usually consider 2 months as old 15:37:07 I just also listed one that is 6-weeks old 15:37:18 @smcginnis: @ dhellmann: thanks I got it now 15:37:32 I'm sure "old" varied from project to project too. 15:37:44 Based on its "stable" state. 15:38:05 that's true 15:38:07 So... 15:38:17 FOr the ones with OLD it's just a warning 15:38:27 Like making sure they know we'll fall back to that old one 15:38:41 Yeah, I'm more concerned with the ones that have not done any release. 15:38:46 I'm totally fine branching on that very old 15:39:04 ok 15:39:05 It's just good to give a fair warningm so that we can actually execute on that fallback mode 15:39:35 I'm MUCH more concerned with the missing ones. We've been trying to insert various safeguards in the process to not have them 15:39:44 I will make sure to call that out in the ML post I will send. 15:39:45 like reminders in weekly release enails 15:39:59 and yet we still have a bunch of them 15:40:23 how about the wording on lines 355 and 356? 15:40:25 So for the next release cycle I'd force a release earlier, like week before m3 15:40:33 Not sure what to do this cycle though 15:40:43 yeah, I like the idea of an earlier deadline 15:40:44 dhellmann: Looks good to me. 15:41:01 although some of this feeling of scrambling is coming from the fact that it took us an extra week to do the milestone this week 15:41:15 sure 15:41:41 #info should consider adding note pre-milestone-3 to force release and branch cycle-with-intermediary that have not done any releases for the cycle 15:41:59 dhellmann: your wording looks good. I just wish we did that further away from release 15:42:07 (for the unreleased ones) 15:42:26 Has this been as much of an issue in the past? The projects not doing any releases? 15:42:38 smcginnis: no, but we did a lot of cat herding 15:42:46 to ensure there won't be any 15:43:00 Like directly contacting PTLs? 15:43:03 yes 15:43:16 starting a m-2 15:43:57 Maybe we need to bring that back then. For libs and intermediary projects that have no releases, or a large number of outstanding commits. 15:44:09 I think it's fair to require a release by FF. It's just taht it's a busy week already so... 15:44:31 FF-1 maybe 15:44:47 anyway, topic for PTG discussion too 15:44:54 (as in relmgt team discussion 15:44:54 But I could see before FF being too soon for some. 15:45:02 Yep 15:45:12 #link https://review.openstack.org/540425 add release warning and forced release steps to the process 15:45:12 patch 540425 - releases - add release warning and forced release steps to th... 15:45:14 For now let's just run with dhellmann-s wording 15:45:18 we can iterate on the wording on ^^ too 15:45:39 Nice! 15:45:52 smcginnis: let's review tasks, assign them and strike out those we already did 15:46:29 I thihnk you covered some in your release weekly reminder 15:46:35 OK, the pre RC1 ones, I think I covered 1 and 3 in my countdown email. 15:47:16 I will cover 4 and 5 with a new email later today. 15:47:42 What have we done in the past with item 2? 15:48:32 just a message in the weekly email I think 15:48:36 I think that's mostly just been -- yeah 15:48:50 at this point we haven't been adding a lot of independent releases so I'm not too worried about it 15:49:26 And that's for the independent ones that are tagging their own, but now moving to have us do their releases? 15:49:29 So tasks below line 359 are probably not for the coming week 15:49:47 I'll move them to post-RC1 week 15:50:01 ttx: Should we move those into the "Tasks" for next week to make sure we cover them and can mark them done? 15:50:05 yes 15:51:27 done 15:51:55 OK, I think we have a plan then. Or at least a plan for a plan. 15:52:06 Anything else we need to make sure we cover in the meeting? 15:52:29 we moved all of those warning and encouragement emails to next week? instead of sending them today? 15:52:45 I will send the email today. 15:52:47 ok 15:52:52 dhellmann: It's post-meeting tasks :) 15:52:56 ah, I see 15:53:01 you start you week after the meeting :-) 15:53:02 But I think acting on them will next week. 15:53:07 I wanted to quickly discuss final releases extra tasks 15:53:14 Encouraging release highlights, 15:53:15 Checking release notes for completeness 15:53:23 Oh, right. 15:53:29 I think that's mostly weekly email material 15:53:47 we can review for highlights when we start getting rc1 tags, too 15:54:06 Ann sent out a reminder on it yesterday. 15:54:11 but we may want to spend extra time ensuring release highlights end up being usable 15:54:19 And we have a couple patches adding some, so that will help having examples. 15:54:19 ok, good, I probably just haven't hit that point in my inbox 15:54:31 I need to change the formatting for those. It's just confusing right now. 15:54:33 because how it goes the first time will probably determine the future of it 15:54:38 is annabelleB going to review the highlights? or is that up to us? 15:55:15 Hmm, I think mostly us, but I think it would be good if we can get her to review it from a marketing perspective. 15:55:24 annabelleB : o/ 15:55:29 o/ 15:55:41 I do fear they will end up getting to low level technical for what we are targeting. 15:55:44 dhellmann: I don't think we want to review them 15:55:57 just encouraging PTLs to fill them when we talk to them ? 15:55:57 so I was asking whether you're planning to help review release highlights as they come in, or if you want to do that after a bunch are there and maybe tweak the wording? 15:56:08 that makes sense 15:56:22 ttx: I would like to at least make sure what they add makes sense for what is intended for. 15:56:23 if we're trying to guide them to producing useful notes, we may have to change them from what is originally written 15:56:31 smcginnis: ok 15:57:00 happy to help review. I can do my best to keep up with them as they come in, rather than having time pass then people seeing changes 15:57:09 At least to have some suggestions. But maybe annabelleB can do some clean up and tweaking after they've merged? 15:57:13 dhellmann: agree, definitely on this first one 15:57:28 smcginnis: sure, happy to do it either way 15:58:10 sounds good. I'll try to add annabelleB to reviews that have the highlights and give her a chance to look at them. 15:58:18 ++ 15:58:19 I do see already we’ve hit some corners like ‘where does cross project work go?' 15:58:24 maybe we want the policy to be that fixes should come as follow-ups, though, so we don't mess with deadlines 15:58:39 +1, definitely don’t want to hold up deadlines 15:58:51 Yeah, I wouldn't want to hold things up more than necessary for something like this. 15:59:04 Maybe just comment with suggestions, but don't downvote. 15:59:11 And we can clean up after the fact. 15:59:16 yeah 15:59:16 Last thing to discuss - do we do a meeting next week ? dhellmann/smcginnis won't be around. Happy to chair it if needed 15:59:17 works for me 15:59:49 I may or may not be in a good place to still be here for the meeting, but otherwise I will be out. 15:59:55 we could skip or meet on thursday 16:00:21 * I will be out but still checking in if any trailing RC activies. 16:00:25 *activities 16:00:47 I would be fine keeping our normal meeting time. 16:01:06 My online access should be better than last Friday. 16:01:29 ok. I will probably be in the car on the way to the airport but I can check the logs later in the day 16:02:04 OK, over time. Let's plan on normal meeting time unless something comes up before then to change that. 16:02:11 Thanks everyone. 16:02:16 thanks! 16:02:19 #endmeeting