15:01:56 #startmeeting releaseteam 15:01:57 Meeting started Fri Aug 10 15:01:56 2018 UTC and is due to finish in 60 minutes. The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:57 o/ 15:01:58 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:02:00 The meeting name has been set to 'releaseteam' 15:02:01 o/ 15:02:07 #link https://etherpad.openstack.org/p/rocky-relmgt-tracking Agenda 15:02:20 Currently around line 503 15:02:42 I won't do the ping list, but hopefully get by just mentioning ttx and dhellmann 15:03:03 o/ 15:03:03 o/ (late to the party) 15:03:18 O/ 15:03:25 I was there already, see above :) 15:04:20 o/ 15:04:31 Good, the gangs all here. 15:04:44 #topic RC status 15:05:02 We have some details in the etherpad for various deliverables. 15:05:13 prometheanfire: That near the bottom of https://etherpad.openstack.org/p/rocky-relmgt-tracking 15:05:45 I think we have an update on RC1 deliverables except for cyborg. 15:05:53 * fungi is around, just having too many discussions at once 15:06:08 same, two meetings at once :P 15:06:16 * dhellmann is having 2 conversations with fungi 15:06:20 :) 15:06:43 Barbican - they thought deadline was today. 15:06:49 smcginnis: so the question I have is the same as all the FFE things, UC only? probably fine 15:07:32 prometheanfire: We have a few client libraries in the list on the etherpad. 15:07:42 Looks like all should just be u-c only. 15:07:49 ya, I see them, looks like 3 15:08:19 prometheanfire: Do you want us to direct them to send a FFE? 15:08:25 iirc I think karborclient had a release already, but ya, UC only is fine 15:08:40 smcginnis: Looks like only cyborg might need to be cat-herded? 15:08:45 if only to name and shame 15:08:47 prometheanfire: We have a python-designateclient patch up. 15:09:03 (in the cycle-with-milestones ones) 15:09:20 mugsie: Can you send an FFE to openstack-dev for the python-designateclient? Then we can link to that for the release patch. 15:09:33 ttx: Yeah, that is the only one I haven't seen activity from. 15:09:37 they could be batched into one request to cut the spam too 15:10:17 smcginnis: + karbor and tricircle for their client ? 15:10:41 So sticking with the RC list so we don't scatter all over - we are waiting for barbican, we may need to force cyborg, the rest have patches up. 15:10:50 ttx: Karbor is separate. 15:11:02 ttx: Oh, sorry, I see what you're saying now. 15:11:27 I'd just cut a release for both 15:11:47 if that's just CI/reqs changes 15:11:51 I can send the FFE for those two at least as a way to announce the release team is following through with what we had said we would do and are forcing a release of them. 15:11:58 ++ 15:12:26 wfm 15:12:31 and then if karbor fails to release an intermediary release we'd remove both 15:12:45 (karbor and karborclient) 15:12:52 Good point. I will note that as well. 15:13:36 #action smcginnis to send FFE for karbor and tricircle clients noting release is being forced. Also noting karbor will be removed if service release is not requested. 15:13:40 (tricircle ahs a relatively recent release and is branched) 15:14:01 So maybe just some late patches for the client. 15:14:08 ok next cycle-with-intermediary services 15:14:32 we heard for cloudkitty earlier this week, they should send in something 15:14:44 not sure we heard anything from karbor or magnum 15:14:50 I have not. 15:14:55 * ttx checks his ping list 15:14:55 nor i 15:15:07 strigazi: ? 15:15:09 And not sure how cloudkitty missed RC when they had been in touch prior to it. 15:15:38 I think he said he might be one or two days late ? 15:15:53 which for intermediary-released stuff is not so big 15:16:27 ya, triple-o is in that camp 15:16:31 zhipeng is not on this channe; 15:16:33 l 15:17:02 the intermediary server projects aren't technically late, right? 15:17:15 dhellmann: no they aren't 15:17:19 we just set the deadline for them to the final deadline, according to the schedule page 15:17:23 According to the clarification last week, they have until RC final. 15:17:32 right 15:17:37 that's what I meant 15:17:40 we'd still prefer it earlier so we can cut stabel/rocky 15:17:47 agreed 15:17:51 Cuts it close for some of these that have not done any releases yet, but that is what we published. 15:18:04 right, but it's their problem if they run into issues when we go ahead with the other branches 15:18:11 True 15:18:12 that's what we've communicated in the past 15:19:04 So cloudkitty - we know they are working on it, karbor - will warn in client FFE that they need to do a release or risk being removed, magnum - no word yet. 15:19:10 and since we're not syncing requirements any more, we shouldn't have any issues with versions going up on master then back down after the branch 15:20:16 Clients: designate - patch is up, karbor and tricircle - will send FFE and force releases. 15:20:22 should we warn them about the impending deadline and missing being included in rocky? 15:20:35 (them == magnum) 15:20:46 the castellan ffe e-mail just hot the -dev list too 15:21:05 er, just hit 15:21:36 We probably should warn magnum, though we have stated in multiple posts and it is documented. 15:23:19 Horizon plugins? 15:23:33 Cloudkitty is known. 15:23:43 I'm trying to put the notes into the etherpad, and I missed the client ones. are those all warnings? 15:24:09 Hopefully mugsie can send FFE for designate. I will send one for karbor and tricircle ones. 15:24:25 With the warning that they will be removed if no service release by RC final. 15:24:31 smcginnis: I will do that now, before I forget 15:24:38 mugsie: Thanks! 15:25:12 ok, I think I'm caught up 15:25:43 Not sure about tacker-horizon and zun-ui. 15:25:55 zun-ui looks to be the bigger concern. 15:26:03 those looked like there were enough changes to need a warning 15:26:03 But they do still have until RC final. 15:27:26 OK, "others"? 15:27:46 TheJulia might know the bifrost situation 15:28:11 the monasca-kibana-plugin had no real changes except the readme file format 15:28:30 kuryr-* ones have some, but again have until RC final. 15:28:40 I guess that means it's stable? it would be good to have confirmation that it's maintained. 15:28:51 the tempest tag is called out in our process somewhere I think 15:29:27 oh, just for the branching 15:30:18 I do seem to remember that we wait to tag tempest as long as possible though 15:30:30 I have not seen anything for networking-hyperv, and given the recent election results for the Win team, they might need a reminder. 15:30:42 seems likely 15:30:49 I feel like we should (in the future) mandate that intermediary-released stuff has one release and branch done by R-2 15:31:04 that's early for a branch, but a release makes sense 15:31:15 dhellmann: it would be nice to document the why 15:31:18 we used to have that rule, iirc 15:31:21 since the turnaround for removing that deliverable (and update release messaging) is REALLY short 15:31:38 evrardjp : yes, definitely 15:31:41 ttx: RC2 or RC3? It would be nice to make sure something is being done before the end. 15:32:13 smcginnis : rc2 is the inclusion deadline for milestone projects, so that sort of makes sense. although rc3 would be ok, too. 15:32:15 smcginnis: I'd either add a deadline at R-2, or require that anything that is past R-3 requires an exception 15:32:41 Basically I'm fine cutting slack to people that are keeping up and reaching out 15:32:57 I'm worried about peolpe we haven't heard from 15:33:05 Me too. 15:33:35 we don't really do negative messaging, right? so even if we remove the project at r2 folks who aren't checking until the end of the cycle won't notice until then. 15:33:50 re bifrost, I know it needs to be released, but it is mainly geared on tooling to deploy ironic, so I just haven't gotten to that yet since there has a minor breakage 15:34:02 TheJulia: OK, thanks. 15:34:16 TheJulia: It is cycle-with-intermediary, so there's a little time yet. 15:34:54 yeah, my perception was next week 15:35:05 The original idea behind MembershipFreeze and requiring somethign is in by milestone-2 was to have those discussions about what is in the release a lot earlier in the cycle 15:35:23 yeah, that's why I would support an rc2 deadline for intermediary projects 15:35:29 removing stuff the week before release is not making us look good 15:35:30 just make them all the same deadline 15:35:31 That makes sense. 15:35:41 and makes it very difficult to work on release messaging 15:35:44 +1 15:35:47 We really should not have questions about what is in or out this late. 15:36:16 Also... intermediary-released was for things that do multiple releases 15:36:39 things that can't make a release need the cycle-with-milestone baby-wheels 15:36:50 I agree on the idea of having the "same deadline" as message, but it's far from possible due to a chain of dependencies 15:37:01 I'm just ranting out loud 15:37:06 evrardjp : ? 15:37:14 ttx: No, it's all very valid. 15:37:51 maybe it was out of context, continue :) 15:38:13 "them all" was not clear to 15:38:14 me 15:38:38 (I see cyborg rc1 just appeared) 15:38:41 evrardjp : ah, I meant "all cycle-with-intermediary and cycle-with-milestones projects" 15:38:49 thx: thx for the clarification on release models, it’s clear now to me. 15:38:52 dhellmann: I realised that afterwards. 15:39:00 OK, c-w-i outdated releases. 15:39:05 Swift we know is coming. 15:39:11 Not sure about heat. 15:39:20 smcginnis , ttx: perhaps we should change the models for some of these projects that aren't really following the intermediary model expections 15:39:21 We may just need to force a branch from the last release. 15:39:23 The RC dance was so that we have a fallback release on R-3. intermediary-released did not have to go with the RC dance because we were supposed to have a fallback release (the last intermediary) already 15:39:54 And now people use it to release a single thing at the end... and we lose our fallback 15:40:06 dhellmann: I think I'd rather prod them to require an earlier intermediary release. 15:40:08 right 15:40:18 ttx: that's good to clarify this. 15:40:22 smcginnis : well, like ttx said, if they're not actually producing intermediary releases... 15:40:38 I think we should reach out to all those who did a single release and being intermediary 15:40:50 and ask them about going with-milestones again 15:41:00 or independant? 15:41:09 The trick being, of course, that with-milestones is inferior as it's less semver 15:41:10 perhaps we should suggest more strongly than we did in the past when we had those conversations 15:41:11 I'm saying I'd rather force (stronly encourage) them to do intermediary releases like they should or miss being part of the coordinated release. 15:41:29 evrardjp: we want "openstack" main components to be released on the cycle 15:41:35 evrardjp: Yep, good point. If they can't do intermediary, then they probably should be independent. 15:41:36 so that independent option is not open to all 15:41:46 smcginnis : ok, I guess presenting the choice that way works for me. Either actually do intermediary releases, switch to milestones, or switch to independent. 15:41:49 ttx: how many RC should a release have? 15:41:50 It would depend on what it is. 15:41:59 ttx: that was a joke initially -- what I meant behind that is that maybe they are not using the right model 15:42:07 armstrong: As close to 1 as possible. 15:42:13 armstrong : for milestone-based projects we require 1 and often have a second for final translations. 15:42:15 evrardjp: Could be. 15:42:26 * TheJulia finds this entire discussion really frustrating from a contribution standpoint. 15:42:28 dhellmann: in theory, we /could/ fallback to the last release they did. The previous cycle. 15:42:33 ok thx 15:42:39 but yeah, we don't want a lot of them 15:42:40 TheJulia : what about it frustrates you? 15:42:46 TheJulia: Can you share your frustrations? 15:42:58 dhellmann: eating food real quick to make sure it is not low blood sugar 15:43:04 :) 15:43:10 TheJulia: that's why I am saying this: If your project doesn't meet certain procedure,maybe the procedure isn't the right one? 15:43:24 TheJulia: I smiled on blood sugar. 15:43:25 TheJulia : ack. I'm interested in your perspective. I'm probably too close to the other side of the question. 15:43:55 All of this should be around helping teams make sure they are progressing along and able to be released together as a good, quality whole. If it's not encouraging that, then we should reevaluate. 15:44:01 nomming 15:44:03 evrardjp : right, a big purpose of the models is to describe what process the project uses for releases, so if the project isn't actually doing what the model says then the model is wrong 15:44:17 basically I'd prefer not to remove anything from the release after milestone-2 15:44:35 dhellmann: I don't want to look oversimplifying though :) 15:45:06 evrardjp : I think the models are defined well, so what I mean is that they are not matched correctly to what the teams need. 15:45:13 we're using the wrong models, not defining the models incorrectly 15:45:27 that's what I meant. 15:45:35 precisely! 15:45:42 huzzah, we agree! 15:45:49 We have a long list of ones missing stable branches. Some appear to have merged a lot since their last release, but I think we will just have to force the stable branch for those that do not have anything before final. 15:46:13 what is the effect of not forcing stable branches now for projects that are not milestone-based? 15:46:44 These are cycle-with-intermediary, so I don't think we should force until the RC final deadline. 15:46:58 ah, right, I misread what you said 15:47:07 Otherwise if they are planning to release more, they would end up needing to backport potentally a lot. 15:47:09 * dhellmann may need some of TheJulia's noms 15:47:15 dumping a few more thoughts in #openstack-release to avoid disrupting meeting further 15:47:34 Release team brunch. 15:47:35 :) 15:47:41 there's a plan 15:47:43 here for this! 15:47:46 does denver do brunch? 15:48:03 will there be wine? 15:48:06 I'm sure we can find somewhere. 15:48:08 of course! 15:48:10 Mimosas? 15:48:17 bingo 15:48:22 I see it as kind of two fold. We want fallbacks for messaging, but as pointed out we don't get them... because projects have the ability to break eachother kind of hard at times by minor actions or changes. I get the desire for messaging, but if you can't get to something to milestone-3 that is unrelated, then your essentially penalizing in my mind. We're essentially trying to drive things to move in lock-step, but 15:48:22 it is really unreasonable imho. I think we can only really 'release' what we know works together too. 15:48:32 Funny how several channels have all gravitated towards food lately. 15:48:58 smcginnis : the memes own us 15:49:16 * fungi plans to go find food as soon as the meeting ends 15:49:26 TheJulia: We want to make sure a release is done earlier in the cycle, but it would not be their final release. 15:49:29 smcginnis: I am not at fault, but yeah that's my main hobby. 15:49:43 TheJulia: Or maybe I'm misunderstanding what you're saying. 15:49:51 TheJulia : I guess the point is to release early and often, or to use the milestone model which is strictly date-based and tests the release process without producing something that claims to be production ready 15:50:08 smcginnis: so then we force number/release model, stable branch backporting for the final release? 15:50:21 projects using the intermediary model claim to be stable, well tested, and well maintained. If that's not the case, then that's the wrong model for the project. 15:50:28 but that feels like it fragments the deliverable product 15:50:49 put another way, if the project is only creating one release per cycle, then it ought to be cycle-with-milestone? 15:50:58 We don't force release model. That is something the team chooses. We just try to enforce that they follow their chosen model. 15:51:03 fungi: well, yes. 15:51:25 smcginnis : right. we want them to accurately describe how they manage releases. 15:51:38 it's called cycle-with-intermediary, not cycle-with-one-single-release-that-if-you-miss-it-you-are-out 15:51:52 We don't want to impose anything on the teams other than to follow through with what they've declared they would do. 15:51:53 that's spelled "independent" 15:52:08 heh 15:52:15 smcginnis: should we? 15:52:23 ok if we are having that discussion now... I'll paste from #opensatck-release 15:52:50 17:48 In the name of giving teams control over their deliverables, we created a machine that removes components from OpenStack if someone misses a couple of checkboxes. 15:52:51 * dhellmann waits for the paste before responding 15:52:52 17:48 I wonder if the process should not, by default, do releases for people 15:52:54 17:49 and then those who want the extra control can assert it 15:53:07 I agree. 15:53:24 smcginnis: I think that is a good delineation, the conundrum is I think the milestone model is unsustainable without automated point in time "we're releasing this" which is out of control of contributiors aside from trying to get things in early, because we do get the gates bogged down at the end of the cycle, but I also think we have many smaller minor things that just don't have the resources to release any other 15:53:24 way except as "hey, got this thing", but then compatibility is completely unknown to everything else. 15:53:32 it simplifies the model, and doesn't have a bad message. 15:53:44 ttx: we have the split between servers and libraries to consider, too. I would support "milestone by default" for servers, but that doesn't work for libs. :-/ 15:54:04 * TheJulia hits the same idea as ttx 15:54:07 dhellmann: we could do regular library releases too ? 15:54:27 if there have been changes landed, sure 15:54:29 ttx: we could, but when something breaks as the result then that's on us instead of the team that owns the library 15:54:45 we've had a few times where we knew and oslo release would break something, so we waited 15:54:57 hmm 15:55:01 "why did you release that - it's not ready and we've been trying to fix this bug!" 15:55:05 right 15:55:18 This also goes back to the earlier discussion where releasing libraries without a reason felt like the wrong thing to be doing 15:55:35 that's less likely with servers, except the break would happen downstream of us where someone is automatically packaging based on tags 15:55:35 smcginnis: "tough"? 15:55:39 if you don't want your thing to be released, you can flag it 15:55:42 :) 15:55:49 releasing libraries when there are changes is the right thing to be doing. making unnecessary changes to libraries is the wrong thing to be doing 15:56:07 when did we do unnecessary library releases? 15:56:20 dhellmann: we were talking about forcing releases of libraries with no changes 15:56:22 If you don;t flag it as "safe to release anytime", then it's on you to fix it if it breaks ? 15:56:28 i assume it's conflated with releasing libs with just requirements bumps in them in the past 15:56:51 and the idea that we should let our stable libs actually remain stable 15:57:06 TheJulia : ah, maybe because there were CI changes and so if we create the branch without including them it would be broken immediately and require backports? 15:57:39 ttx: isn't that what a PTL of a project can do? Like reviewing things is probably easier than remembering the deadlines in a busy world. 15:57:58 yeah, we still need to talk about what to do with those. I think I'm leaning towards just making stable libraries independent. 15:58:11 dhellmann: That is the risk with any branch or any release. Ironic has had to create its branch in a broken state because it was broken by things outside of our influence or control. We had to backport everything to get things going again 15:58:24 Anyway, that won't apply to rocky, but i think we should spend a few minutes on that in Denver 15:58:30 what was the resolution on branching when there were no changes? did we decide to preemptively branch? 15:58:31 TheJulia : sure. We're trying to reduce the number of times that happens, though. 15:58:53 With 2 minutes left, this does seem like a good thing for the PTG. 15:59:04 agreed 15:59:09 dhellmann: agreed and I believe we should attempt that 15:59:11 agreed 15:59:33 +1 15:59:38 I'll add a topic to our PTG etherpad. 16:00:00 Anyting else in the last few seconds? 16:00:02 fungi : I don't remember, but we should make sure it's written down somewhere 16:00:22 cyborg request is now in 16:00:37 and Barbican's 16:00:43 Great! 16:00:45 so we are good as far as Rocky process goes :) 16:00:51 OK, we're at time. Thanks everyone. 16:00:56 thanks smcginnis! 16:00:58 #endmeeting