15:01:55 #startmeeting tc 15:01:56 Meeting started Thu Dec 10 15:01:55 2020 UTC and is due to finish in 60 minutes. The chair is mnaser. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:01:59 The meeting name has been set to 'tc' 15:02:05 #topic rollcall 15:02:08 o/ 15:02:09 o/ 15:02:11 o/ 15:02:13 o/ 15:02:17 o/ 15:02:28 #link https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee 15:03:05 i count 4 names and an (on time) jungleboyj making it 5 15:03:14 :-) 15:03:17 #topic Follow up on past action items 15:03:29 Didn't have to get the kids to school today. 15:03:34 #link http://eavesdrop.openstack.org/meetings/tc/2020/tc.2020-12-03-15.00.html 15:03:44 Can't wait until Ian can drive himself. 15:03:47 haha :) 15:04:03 mnaser change all reference of meeting time to go towards eavesdrop for single source of truth <= i've removed all references (or as much as i can) and relinked them to eavesdrop 15:04:11 that way, we won't have any confusion if we play in the time for the future 15:04:34 if anyone sees any, i'd appreciate fixing them (or sending them my way!) 15:04:47 ++ 15:04:49 mnaser remove openstacksdk discussions for future meetings <= done 15:05:17 mnaser send email to ML to find volunteers to help drive goal selection <= that's a TODO on me. i will draft up and send the email out today. 15:05:29 #action mnaser send email to ML to find volunteers to help drive goal selection 15:05:31 +1. 15:05:32 for follow up :) 15:05:38 next up, gmann complete retirement of searchlight & qinling 15:06:04 that is close, repo cleanup and governance patches are merged 15:06:21 nice. any project-config patches left at this point or? 15:06:22 I left with few dependencies one left which are up and under review 15:06:25 o/ 15:06:32 no project config patch left 15:06:38 #link https://review.opendev.org/q/topic:%22retire-searchlight%22+status:open 15:06:49 #link https://review.opendev.org/q/topic:%22retire-qinling%22+status:open 15:07:07 one system config is there if you can have look. 15:07:21 ok awesome, i don't have infra-root but maybe we can ask clarkb or fungi nicely :P 15:07:34 i see will ping them 15:07:45 gmann: do you think we should keep it as an action item for next time to track or you feel like it should be marked as 'done' ? 15:08:29 mnaser: we do not need as governance patch is merged only thing is projects need to merge the already up patches for usage ermoval 15:08:31 removal 15:09:04 ack, do you want us to track progress as an action item for next week just to see if we can help you drive this or feel its fine? 15:09:08 i'll take a look at the system-config change now 15:09:37 mnaser: either way is fine, tracking with AI will be more clear. 15:09:42 i agree :) 15:09:51 fungi: thanks, these two https://review.opendev.org/c/opendev/system-config/+/764573 https://review.opendev.org/c/opendev/system-config/+/764553 15:09:55 #action gmann complete retirement of searchlight & qinling 15:10:05 so many diablo_rojo's in here :P 15:10:09 I am behind in my retirement of karbor, but I should have some patches up in the next couple days 15:10:14 Lol sorry 15:10:23 So many laptops 15:10:25 ok awesome, no worries, i'll keep it as an AI to track 15:10:35 Yes please :) 15:10:38 #action diablo_rojo complete retirement of karbor 15:10:51 and finally mnaser remove project retirement from agenda -- that's gone to clean up the agenda now 15:11:08 #topic W cycle goal selection start 15:11:27 We should change this to X cycle goal 15:11:33 very much so, lol 15:11:43 Agreed lol 15:11:48 #undo 15:11:48 Removing item from minutes: #topic W cycle goal selection start 15:11:52 #topic X cycle goal selection start 15:12:10 so -- it's a bit on me having not sent the email to seek volunteers for this 15:12:22 yeah, 15:12:35 the action item is still there and i'll send that out today, but does it make sense to keep it as an open discussion item to track the progoress (i think it makes more sense than action items) 15:12:36 also there is nothing in proposed directory so it may take time 15:12:58 Secure RBAC also not yet ready to be community goal. 15:13:01 yeah, and the etherpad doesn't have a huge selection of ... achivable goals :/ 15:13:15 true 15:13:21 #link https://etherpad.opendev.org/p/community-goals 15:13:37 i think perhaps a small reminder that we don't have to do a community goal for every release. and maybe it makes sense for us to say .. im sure our contributors are tired, and even more since covid 15:14:00 so maybe skipping a community goal for sanity and letting our developers push time towards a feature they want to drive or something that they might _enjoy_ doing a bit more than say, privsep ;) 15:14:12 yeah. 15:14:12 (disclaimer: i think privsep is great but maybe not the most exciting thing to work on) 15:14:32 or we tried a "stablisation" cycle in the past, that might be a good breather goal? 15:14:36 yeah, that is totally fine. 15:14:43 Given how burned out everyone seems to be mnaser 's idea isn't bad. 15:15:00 mugsie, that will make a fine goal indeed 15:15:05 cycle goal: try not to make things worse ;) 15:15:09 mugsie: that's pretty neat, i mean, we can just leave it open and ask folks to update the proposed goal page with something they worked on 15:15:23 I like that :) 15:15:24 +1. goal can be 'spend goal-work time with your family and firend' :) 15:15:25 so that its sort-of a show-and-tell "hey we FINALLY got around doing X" 15:16:19 kinda thinking of thinking the community goal this time around maybe being your '20% time' 15:16:36 and if that doesn't produce anything, that's also fine. times are hard :< 15:16:51 and this can encourage teams to do more progress on OSC and RBAC stuff too 15:17:21 perhaps we can link out to the popup teams and the goals that are not selected 15:17:21 otherwise popup teams does not get much attentions from projects side 15:17:27 mnaser: +1 15:17:34 and say that you could totally help with that 15:17:49 yeah good idea. 15:18:00 this is an awesome idea i think 15:18:06 +100 15:19:01 i will try to summarize some of these ideas, being like either: 1. use this time to rest and relax, no need to do something, 2. use this time to finish up this really cool thing you've been trying to find time to, and share with us to recognize or 3. check out these existing goals / popup teams that might interest you if you want to invest time in something different/new 15:19:37 (all 3 can be what the stabilize 'goal' is) 15:20:28 how do we feel about above ^ before i blast to ML? :) 15:21:02 I like it. 15:21:04 +1, one suggestion to move popup team one as 1st :) 15:21:19 +1 15:21:19 sounds great to me 15:21:24 mnaser, assume we will keep collecting new idea for goals along side with these three idea? 15:21:28 (in any order :D ) 15:22:10 of course 15:22:13 that's a good addition 15:22:18 mnaser, super:) 15:22:29 ok so after all my not sending that email was a 'happy accident' 15:22:31 :P 15:22:42 ok, moving onto next 15:22:45 #topic Audit and clean-up tags (gmann) 15:22:49 Sometimes things happen for a reason. :-) 15:23:06 we can remove the zero upgrade one as it is merged. 15:23:25 #topic Audit and clean-up tags (gmann) 15:23:32 #undo 15:23:32 Removing item from minutes: #topic Audit and clean-up tags (gmann) 15:23:35 for API tag, doc patch is merged. thanks mugsie for pushing that. I will start the ML to encourage projects to start apply for that tag 15:24:04 mnaser: may be we can track only API one for now to know how projects are progressing on this 15:24:28 I am sure manila is interested, may be gouthamr will raise patch soon 15:24:43 +1 15:25:23 ok so should we keep this action to discuss on progress of getting projects in 15:25:25 or split it out to action items? 15:25:55 I think action is fine to know how many projects want/do not want and what next we can do on this tag 15:26:33 gmann: maybe an action item is announcing the tag and seeing if projecst have interest in 'applying' ? 15:27:10 mnaser: I will say to keep 'Audit and clean-up tags' only and we will have each tag one by one to audit carefully 15:27:17 so that we cover each tag this time 15:27:17 sounds good 15:27:38 ok, trying to be mindful of our time 15:27:41 maybe we can flush this quickly 15:27:45 #topic X cycle release name vote recording (gmann) 15:27:57 smcginnis: happen to be around? do you see access to the results or we have to disclose? 15:28:15 (implying that i would .. remember mine) 15:28:31 :-) 15:28:40 :) 15:28:48 It would be good if we had the results as I don't totally remember my vote. 15:29:19 #link https://civs.cs.cornell.edu/cgi-bin/results.pl?id=E_7e6e96070af39fe7 15:29:24 we do have the result link 15:29:25 does organizer get the vote results may be fungi knows? 15:29:26 ^^^ 15:29:36 that's what i'm curious about 15:30:03 (the CIVS email has the link to the results page) 15:30:15 mugsie: ricolin yeah but it does not have individual vote results 15:30:16 civs doesn't provide individual voter choices unless you set that option when creating the poll, otherwise votes are anonymous 15:30:23 ohk 15:30:24 ahhh, ok, so we really need to remember our choices 15:30:58 and only for 6 votes as more vote now can alter the result :) 15:31:12 ok well, we're at 10:30 and a bunch of community members are looking forward for another discussion, so id like to be mindful of time 15:31:22 I am pretty sure I was Xenoblast, Xenomorpth and Xenith 15:31:23 we can keep this for next week or discussion async over ML or IRC 15:31:35 sure 15:31:41 #topic CentOS 8 releases are discontinued / switch to CentOS 8 Stream (gmann/yoctozepto) 15:31:55 all yours ^, also we have apevec from the RDO community who can help inform/explain things too 15:32:21 actually, I didn't propose the topic :) 15:32:27 I think it was from Kolla ? 15:32:39 I can explain what Stream is 15:32:42 yoctozepto started this in QA office hour this week 15:32:59 also we are adding job in devstack also which need more work #link https://review.opendev.org/c/openstack/devstack/+/759122 15:33:01 basically the same content as in centos8 just delivered earlier 15:33:12 apevec: +1, that will be helpful, go head 15:33:16 ahead 15:33:18 so atm no need to change 15:33:27 so, does this mean for stable branches we should remove stream? 15:33:33 so tl;dr the same RHEL process applies 15:33:45 only what is planned to be release will get to Stream 15:33:56 as stream could update under us, and cause a ton of changes to fix the gate? 15:34:05 mugsie, atm we don't think we have 8-stream in AFS 15:34:13 mugsie: as i understand it, stream is not exactly a rolling distro 15:34:17 mugsie, centos updates under us e.g. right now 15:34:31 so it's like how we said we support centos 8 but centos 8 and 8.1 and 8.2 and 8.3 are different 15:34:39 we never really said 'we support centos 8.X' 15:34:41 yeah rolling might have different implications, 15:34:45 we pin to a minor version of centos (wel we did for centos 7.x) 15:34:56 where we pin it? 15:35:06 oh really? TIL, i thought we were always testing against latest centos minor (i remember the painful breakages when upgrades happen) 15:35:10 at least in our OSA times :P 15:35:20 at least nodepool builds latest AFAIK 15:35:26 * yoctozepto still in another meeting, give him 5 mins 15:35:32 in AFS we have just 8 15:35:34 i.e. latest 15:35:54 fungi, please remind me where is AFS content defined? 15:35:55 also the images we build for ci nodes in opendev just use whatever the latest 8.x is at the time 15:36:03 right 15:36:10 and we are getting now all 8.3 updates in bulk 15:36:29 which are breaking things 15:36:32 ask weshay|ruck ;) 15:36:34 :) 15:36:40 :( 15:36:40 tru story 15:37:00 testing runtime also just talk about 7/8 which mean whatever latest .x is can be run #link https://governance.openstack.org/tc/reference/runtimes/ussuri.html 15:37:01 so getting single updates could be better for us in CI 15:37:16 thanks for the link 15:37:21 oh, we did move to the "latest CentOS Major" 15:37:37 switching to stream makes sense IMHO, i already did this locally ~6 month ago. 15:37:40 yeah so CentOS 8 can stay, with 8 Stream repos enabled until 2024 15:37:42 i assume for those targeting centos, life will be easier as it'll be small bumps rather than one big blow at once 15:37:47 #link https://opendev.org/opendev/system-config/src/branch/master/playbooks/roles/mirror-update/files/centos-mirror-update mirror script for centos 15:37:55 apevec: ^ 15:38:00 thanks fungi++ 15:38:25 so rather than one big change on release day, it's a small delta over time.. 15:38:31 oh wait only 7 ? 15:38:42 ah nm, I'm blind 15:38:44 :P\ 15:39:02 apevec: separate sections for 7 and 8 yeah 15:39:09 yep, I see now 15:39:12 so yes, mirror.dal10.us.leaseweb.net/centos/8/ 15:39:20 i think right now from an openstack pov, tripleo will probably continue to give us the signal of 'openstack can run on this stack' in terms of qemu/libvirt/etc as it always has 15:39:32 OK, so it will me RHEL will have to deal with any incompatibilities when building packages for RHEL, but we just keep using CentOS 8 stream, which should stay the same major version for the life cycle of that OpenStack release 15:39:35 we'll need to add http://mirror.dal10.us.leaseweb.net/centos/8-stream/ 15:39:44 yoctozepto and gmann from what i see are working on the devstack improvements to be able to run on it 15:40:12 and stream major versions will have a longer life cycle than our releases anyway, so we should be OK? 15:40:28 yes Stream will be 5 years 15:40:39 anyone working on a change to add 8-stream? I can do it. 15:40:42 mugsie: i think given RHEL's upstream dev happening inside tripleo (and targetting stream?), i can imagine that they'll want to get things landed inside of stream so rhel works 15:40:49 zbr, not change, add 15:40:49 do we need to change in stable branch also or keep centos8? 15:40:52 if we have space 15:41:00 so we don't break current assumption about repo paths 15:41:00 yeah, that was the idea. 15:41:03 so add from wallaby onwards ? 15:41:27 we can think as "8-stream" as some kind of 9-ish. 15:41:35 gmann: thats a good call. doe we have any stable branches that will be supported past Dec 2021? 15:42:01 i think ussuri might be affected, lets check 15:42:11 mugsie: yeah at least Victoria. and ussuri on Extended maintance 15:42:17 #link https://releases.openstack.org/ 15:42:23 I think TripleO will want Train, weshay|ruck marios|rover ? 15:42:31 apevec: yes 15:42:49 * yoctozepto is here 15:42:53 victoria is going to be impacted, ussari in EM is less of a worry 15:42:56 we need to care for both Victoria and Ussuri 15:43:00 I see you are progressing as planned :-) 15:43:03 gmann: ++ 15:43:07 the bulk of the upstream jobs run on ubuntu anyway, so if we end up needing to drop centos-8 jobs from some em branches that doesn't seem like the end of the world 15:43:19 or convert them to use stream 15:43:21 Kolla is going to switch to stream if nothing changes in the first month of 2021 15:43:26 fungi: ++ 15:43:49 yeah. converting victoria to stream may be a good idea. I would just leave ussari as is, and turn off centos next december 15:43:55 i mean, think about it, the packages landing in stream would have also filtered into centos 8 point releases under the old model and caused the same disruption for jobs running on those branches either way 15:43:57 we still expect the situation might change 15:44:16 fungi, exactly 15:44:32 yoctozepto, in what way would it change? 15:44:39 and we can leave decision for Ussuri later if EM team want to move it to Unmaintained or fine with dropping centos8 15:45:01 as i understand it this morning, centos stream 8 is pretty much RHEL 8.LATEST + delta until 8.LATEST+1 15:45:02 so it's business as usual. em branches are either fixed so jobs can continue running, or we drop those jobs if the work to fix them is greater than the benefit they provide us 15:45:25 mnaser, correct, it candidate builds for .LATEST+1 15:45:30 grenade jobs are on ubuntu so we do not need to worry for ussuri updates for victoria grenade testing 15:45:38 i.e. now in c8s you see what will be released as 8.4 15:45:38 apevec: well, we never know whether this decision is not reverted based on pressure 15:45:45 plus also independent projects doing rebuilds 15:45:59 yoctozepto, I can tell you for sure it is not going to be reverted 15:46:12 and there are already other rebuilds 15:46:19 just not sure how viable they would be 15:46:31 again, c8s == same content just earlier 15:46:33 e.g. rockylinux 15:46:44 fungi, is README.md :) 15:46:45 the way i see it is: other rebuilds are there to welcome to bring patches to add support 15:46:52 but yeah we'll see 15:46:57 apevec: yup! plus some vapor 15:47:02 but as of now, we need to seek a route moving forward for centos at least 15:47:10 yeah 15:47:15 thereis some things I suspect will stick around like https://blog.cloudlinux.com/announcing-open-sourced-community-driven-rhel-fork-by-cloudlinux 15:47:38 cloudlinux has been around for ages as a rhel derivative 15:47:46 yes there are clones and that's fine 15:47:48 also scientific linux may end up doing a c8 update eventually 15:48:03 cXs will be still real thing 15:48:07 to be honest, i feel like folks who are interacting with rhel/centos inside our upstream seem to have a pretty good grisp on the best solution moving forward 15:48:09 esp with next 9 cycle 15:48:26 so from a tc pov, it looks like supportability is not disappearing for the platform 15:48:45 and given rh's interest in maintaining older releases, they'll fix what needs to be done from a distro pov 15:49:02 weshay|ruck, marios|rover ^ go fix it :) 15:49:17 ;) 15:49:19 mnaser: ++ 15:49:21 but yes, we are around 15:49:22 and i already see mobilised action and planning from their part there 15:49:48 +1, there are much overlap between these two community in term of contributors. 15:50:01 i think it's important to have had this discussion, i learned a lot about it, but i dont think there is anything immediate from the tc in terms of actionable 15:50:13 apevec: aye; we are more generally fine with stream 15:50:21 just our users seem very upset 15:50:26 * noonedeadpunk was away and missed the party :( 15:50:33 noonedeadpunk: the party is pending 15:50:41 other than to simply be aware that some stable branches may need to switch jobs to a "different" node type 15:50:53 (which isn't really all that different) 15:50:56 for ci it makes a lot of sense, unless you're ci'ing for RHEL builds which isn't the case here 15:51:00 hope we'll get IRL party in 2022 Summit at least! 15:51:02 ^ exactly this - OSA also has end users who have just migrated 7>8 and are really worried 15:51:07 apevec: ++ 15:51:14 i.e. we get the latest stuff earlier rather than all at once 15:51:16 tosky put it this morning in a very neat way, which was: centos 8 stream is stable/ussuri, and rhel 8.x is the tagged release 15:51:17 marios|rover: well, CI for RHEL is the reason we do have centos 15:51:23 jrosser: exactly my words you mean? 15:51:42 this is not just about CI jobs but also the message we give out from projects like Kolla and OSA to our users, quite apart from RH customers 15:52:14 jrosser, users can still consume stream 15:52:16 I do worry abnot people being able to install openstack on rhel without using the RHOP tools 15:52:21 they just might to keep a snapshot 15:52:24 apevec: users don't want stream :D 15:52:32 if they're concerned about the rate of change 15:52:33 that's what they said 15:52:43 they are concerned about multiple things 15:52:44 yoctozepto, want / need is different :) 15:52:47 previously the risk was that because centos lagged rhel we might not be testing against new behaviors in rhel which hadn't trickled into centos yet. the problem will simply get reversed, we may wind up testing against new features in centos-stream which haven't made it into rhel yet 15:52:51 yeah. i can see a point where releasing something to support centos 8 stream.. its probably very different over time 15:52:55 as not all projects are in RHOSP, and people *do* sideload them into installs 15:53:09 apevec: they need a stable, reliable distro; and they feel like it has been discontinued 15:53:10 fungi, there is and will be more RHEL CI 15:53:13 fungi: s/may/will get new features that didn't make into rhel yet 15:53:22 so a release from 1 year ago of OSA that says "supports centos 8 stream" might not actually support it 1 year later since changes 15:53:23 yoctozepto, let's take that offline, but it is not 15:53:32 again same content 15:53:39 apevec: you don't have to persuade me 15:53:40 but, in defense of centos in this case 15:53:46 i don't think any distro 15:53:49 was promising like 15:53:51 apevec: I am running stream 15:53:55 nothing would show up in c8s unless it is planned for .Y+1 15:54:02 OSA train ONLY runs against rhel 8.2 15:54:05 s/rhel/centos/ 15:54:15 we were always very much OSA train runs against centos 8 15:54:47 i don't know of any community project that's actually "pinning" to point release of centos for what it's worth 15:54:59 doing so would be a terrible idea anyway 15:55:05 It's just the matter that it's hard and resourcefull to maintain thing that changes too fast 15:55:10 like it was with Suse 15:55:15 I know projects that say x version works on centos/rhel y.z 15:55:23 because those point releases of centos don't get security support for the same duration as our stable branches 15:55:24 noonedeadpunk: thing is, suse was a rolling distro 15:55:33 mnaser: openhpc 15:55:38 and in case users don't like it and don't wont it in prod it is useless effort moreover 15:55:47 and I believe some others also do pin 15:55:51 kolla never did 15:55:51 i think we were deploying against the rolling version of suse at the time 15:56:01 the centos/rhel model has for a long time been that if you want security support you need to fairly quickly update to the next minor release in the series 15:56:02 it bit us but we can live with that 15:56:08 yoctozepto: TIL! 15:56:10 ubuntu also bit us, not seeing much difference 15:56:36 5 min remaining for meeting 15:56:38 mnaser: well changes between suse 15.1 and 15.2 were not bigger then for centos 8.2 -> 8.3 where repos got renamed... 15:56:46 so two things for us ? 1. move all centos8 josb to centos8-stream for master as well as stable/Victoria (leave stable/ussuri for now) + 2. update wallaby and Victoria testing runtime doc too 15:56:58 noonedeadpunk: i think there was no shortage of packages being renamed inside suse at the time 15:56:59 :P 15:57:09 gmann: ++ 15:57:19 if centos stream rolls more like ubuntu lts now, I will not see a difference 15:57:34 gmann: ok, that's pretty good and i trust your QA-team hat :) 15:57:37 +1 15:57:46 ++ 15:57:58 apevec: i can maybe throw a suggestion for some sort of office hour / ama something like that to explain and talk through these 15:58:09 sure, we can do that 15:58:13 i know that might be quite stressful with a lot of opinions going out but today's discussion was helpful 15:58:16 here in TC channel? 15:58:26 i'm happy to host it here 15:58:27 mnaser: ++ 15:58:28 ok. once we get devstack job working then we can ask teams to start migrating it. I will push testing runtime doc update 15:58:31 gmann: so it will mean that CI has images for both stream and non-strem, right? 15:58:49 and we can ask the deployment projects to be around for that and it will be very useful i think 15:58:52 yes let keep both for now 15:59:02 noonedeadpunk: only stream as those stable release CI continue after dec 2021 also 15:59:20 #action mnaser work to find time for community deployment projects + centos/rdo team 15:59:26 apevec: in that case we need to remove it again 15:59:41 yes but only in Dec ? 16:00:11 (i need to close out at 11, i will end the meeting, but i think we can continue the discussion here) 16:00:24 Yeah, I'd rather live with stable Centos until next release in CI while adding Stream as NV jobs 16:00:26 yeah we can continue after meeting too 16:00:26 (another meeting :<) 16:00:30 to see how it's going 16:00:31 #endmeeting