15:00:59 <mgoddard> #startmeeting kolla 15:00:59 <opendevmeet> Meeting started Wed Jul 28 15:00:59 2021 UTC and is due to finish in 60 minutes. The chair is mgoddard. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:59 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:59 <opendevmeet> The meeting name has been set to 'kolla' 15:01:07 <mgoddard> #topic rollcall 15:01:25 <mgoddard> \o 15:01:43 <yoctozepto> o/ 15:02:27 <priteau> -o 15:02:56 <haerwu> _[o][o]_ 15:04:49 <mgoddard> mgoddard mnasiadka hrw egonzalez yoctozepto rafaelweingartne cosmicsound osmanlicilegi bbezak parallax Fl1nt 15:04:52 <mgoddard> ^ meeting time 15:05:48 <parallax> o/ 15:07:23 <mgoddard> #topic agenda 15:07:30 <mgoddard> * Roll-call 15:07:32 <mgoddard> * Agenda 15:07:34 <mgoddard> * Announcements 15:07:36 <mgoddard> * Review action items from the last meeting 15:07:38 <mgoddard> * CI status 15:07:40 <mgoddard> * Release tasks 15:07:42 <mgoddard> * Xena cycle planning 15:07:44 <mgoddard> ** Xena feature prioritisation https://docs.google.com/spreadsheets/d/1BuVMwP8eLnOVJDX8f3Nb6hCrNcNpRQl57T2ENU9Xao8/edit?usp=sharing 15:07:45 <mnasiadka> o/ 15:07:46 <mgoddard> * Kolla operator pain points https://etherpad.opendev.org/p/pain-point-elimination 15:07:48 <mgoddard> * Open discussion 15:07:50 <mgoddard> #topic announcements 15:07:52 <mgoddard> None from me 15:07:54 <mgoddard> Anyone else? 15:08:39 <mgoddard> #topic Review action items from the last meeting 15:09:03 <mgoddard> None, but I have proposed some stable releases which was a recent action 15:09:11 <mgoddard> #topic CI status 15:09:16 <mgoddard> #link https://etherpad.opendev.org/p/KollaWhiteBoard 15:09:40 <mgoddard> I've seen a few networking issues recently 15:09:47 <mgoddard> I think intermittent 15:10:23 <mgoddard> Are there any other issues? 15:10:37 <parallax> Grafana is still causing issues, probably depending on CI vendor 15:11:13 <yoctozepto> you mean on infra mirrors? 15:11:17 <parallax> Some mirrors might still have the erroneous package metadata cached while others don't 15:11:30 <yoctozepto> could be 15:11:33 <parallax> yoctozepto: yeah - I'm wasn't sure whom to talk to about it 15:11:35 <yoctozepto> it could also be CDN issue 15:11:43 <yoctozepto> #opendev 15:11:51 <parallax> upstream problem is already resolved 15:12:20 <haerwu> iirc there is no cache for grafana 15:12:23 <yoctozepto> yeah, but I mean often it is that some upstream CDN is flaky and some mirrors rely on it (as some mirrors are just caching proxies) 15:12:25 <haerwu> or is it? 15:12:38 <yoctozepto> have not checked, relying on parallax here 15:13:14 <parallax> I'm just saying building it works on my machine :) In CI - intermittent 15:14:37 <mgoddard> #topic Release tasks 15:15:29 <mgoddard> I don't think we have anything this week 15:15:56 <mgoddard> #link https://docs.openstack.org/kolla/latest/contributor/release-management.html 15:16:00 <mgoddard> (yay, merged) 15:16:29 <mgoddard> currently we are R-10 15:16:38 <mgoddard> so binary xena in 2 weeks time 15:16:55 <mgoddard> do we have any more TODOs/cleanup left? 15:17:17 <yoctozepto> no idea, I'm still lagging behind 15:17:24 <mgoddard> does anyone want to check what we've done and what's left? 15:18:27 <mgoddard> tough crowd 15:19:03 <mgoddard> #topic Xena feature prioritisation 15:19:17 <mgoddard> #link https://docs.google.com/spreadsheets/d/1BuVMwP8eLnOVJDX8f3Nb6hCrNcNpRQl57T2ENU9Xao8/edit?usp=sharing 15:19:52 <mgoddard> I extracted patches from gerrit, and put them into a sheet 15:20:19 <yoctozepto> looking 15:20:20 <mgoddard> I went through and assigned a Type to each 15:20:31 <mgoddard> so now we can filter for features, bugfixes, etc 15:20:38 <yoctozepto> wow 15:21:15 <mgoddard> for kayobe, we went through together and assigned some patches to Xena, and gave them a numeric priority 15:21:22 <haerwu> grat 15:21:35 <mgoddard> then they can be ordered and put into the whiteboard for reviewers 15:21:40 <yoctozepto> it's missing one thing 15:21:48 <yoctozepto> the actual project 15:21:53 <mgoddard> ? 15:22:01 <mgoddard> see tabs at the bottom 15:22:03 <yoctozepto> ah, I see tabs 15:22:04 <yoctozepto> yeah 15:22:42 <mgoddard> I'm not sure that a community-wide priority makes sense for kolla 15:23:02 <mgoddard> but what we could do is allow people to register interest to 15:23:04 <haerwu> I think that we can identify quick/trivial changes and get them in to get list shorter 15:23:07 <mgoddard> 1. contribute 15:23:18 <mgoddard> 2. review 15:23:20 <mgoddard> 3. use 15:23:57 <yoctozepto> interesting approach 15:24:01 <mgoddard> ultimately resulting in a system where contributors can get reviewers to sign up to review their code 15:24:32 <yoctozepto> now only need to support payments in bitcoin :-) 15:24:44 <mgoddard> a big downside is that this is a snapshot, and not easily updated for new patches 15:25:18 <mgoddard> but it only took a couple of hours, and it could be done manually from there to add new features to a list 15:26:19 <mgoddard> I suppose the ultimate aim is to allow us to focus on reviewing important patches 15:26:29 <mgoddard> rather than being gerrit email event driven 15:26:47 <mgoddard> or small patch dopamine driven 15:27:14 <mgoddard> if anyone has ideas of how to use this data to help, I'm listening 15:27:16 <yoctozepto> yeah, we are looking for oxytocin here 15:28:06 <yoctozepto> well, it could make sense to propose an overlay system on gerrit to better manage these efforts 15:28:20 <yoctozepto> this looks like the first (manual) PoC on that 15:28:21 <jovial`> Could you make the script you used to generte it smarter so that priorities, etc. were preserved on update? 15:28:39 <yoctozepto> mhm, that would be a good progress forward 15:28:48 <yoctozepto> we can wire something simple up with sqlite 15:29:14 <yoctozepto> or even just json-file-based database 15:29:26 <yoctozepto> that would be easiest 15:29:35 <haerwu> what is wrong with using RP+1/RP+2 for marking trivial/important patches? 15:29:53 <yoctozepto> haerwu: it doesn't really give enough info 15:29:56 <mgoddard> how do we decide which are important? 15:30:05 <yoctozepto> precisely 15:30:08 <haerwu> ok 15:30:18 <mgoddard> gerrit doesn't do a lot to help us here 15:30:30 <parallax> Bugs have their priority 15:30:47 <yoctozepto> it would be cool to have this flow 15:31:16 <yoctozepto> change comes in -> gets assigned a type -> gets assigned the priority (or left for discussion on that) -> gets reviewed thoroughly 15:31:33 <mgoddard> any solution that is itself feature sized would probably negate this work :) 15:31:40 <haerwu> we have hashtags in gerrit 15:31:47 <mgoddard> only the author can set them 15:32:05 <haerwu> shit. 15:32:21 <mgoddard> yeah 15:32:30 <yoctozepto> mayhaps that can be fixed? 15:33:19 <yoctozepto> I'll ask on opendev 15:33:21 <mnasiadka> I'm not convinced that managing that in a spreadsheet instead of using Gerrit custom labels/topics/commit message ,,metadata'' will help with anything, but I'm listening ;) 15:34:03 <mgoddard> the problem is that gerrit does not provide a suitable value to configure 15:34:13 <haerwu> yoctozepto: thanks 15:34:14 <mnasiadka> Can't we do custom labels? 15:34:18 <mgoddard> gerrit would be ideal, since it could be queried 15:34:19 <yoctozepto> hashtags would solve our problem 15:34:28 <yoctozepto> with custom views etc. 15:34:47 <yoctozepto> mnasiadka: labels are numeric; hell no, I don't want +2 for bugfix :-) 15:34:58 <haerwu> https://gerrit.wikimedia.org/r/c/All-Projects/+/416357/4/project.config suggests option for it 15:35:03 <mnasiadka> yoctozepto: you can set that +2 is CI, +1 is something else, and so on 15:35:11 <yoctozepto> 17:34:58 <clarkb> yoctozepto: it can be. Zuul and Ironic have done this. I think we might consider allowing it across the server though? 15:35:16 <yoctozepto> YAY 15:35:42 <mnasiadka> well, we can just raise a change to allow it in kolla projects? 15:35:51 <mnasiadka> before it gets done across the server... 15:36:17 <haerwu> yoctozepto: cool ;) 15:36:27 <yoctozepto> mnasiadka: yeah, /me on it 15:36:29 <mgoddard> sounds good 15:36:41 <yoctozepto> teamwork ftw :-) 15:36:47 <mgoddard> still, I think it is the wrong part of the problem to focus on 15:37:03 <yoctozepto> ok, elaborate 15:37:07 <mgoddard> I'm trying to fill the vacuum left by not having priorities 15:37:46 <mgoddard> a limitation of those priorities was that many did not get implemented 15:38:05 <yoctozepto> well, we can manage tags in the spreadsheet instead 15:38:10 <yoctozepto> topics too 15:38:20 <mgoddard> so I'm suggesting we look instead at what has been proposed already 15:38:29 <mgoddard> or at least what someone has promised to propose 15:38:47 <mgoddard> but the question then is how to we prioritise them 15:39:04 <mgoddard> since we have many more patches being proposed than we can review 15:39:18 <mgoddard> we need to make better use of our time, and focus on the 'right' patches 15:39:51 <haerwu> #cleanup #drop #trivial as things we want to get once and forget 15:40:15 <yoctozepto> 17:39:18 <mgoddard> we need to make better use of our time, and focus on the 'right' patches 15:40:18 <yoctozepto> I could not agree more 15:40:31 <mgoddard> and perhaps create some feedback between contributors and available review bandwidth 15:41:03 <mnasiadka> how do we get from what we have today, to tidied and properly tagged patches? :) 15:41:24 <opendevreview> Skylar Tristan Kelty proposed openstack/kolla-ansible master: Update Manila deploy steps for Wallaby https://review.opendev.org/c/openstack/kolla-ansible/+/802743 15:41:51 <mgoddard> I might not have made it clear - the spreadsheet is just a tool. I expected (pre-hashtags) the outcome to be a YAML-ish list in the whiteboard 15:43:02 <mnasiadka> why not a Gerrit dashboard with categorized entries? 15:43:10 <mgoddard> that would be nice 15:43:17 <mgoddard> what categories would be useful? 15:43:46 <mgoddard> #xena 15:44:03 <mgoddard> #feature 15:44:05 <mgoddard> #bugfix 15:44:08 <haerwu> cleanup, trivial, drop, new feature, ci fix, maintaince (with better word for is as I always forget how to write it)? 15:44:39 <haerwu> #to-backport 15:44:40 <jovial`> #priority-high, #priority-medium, #priority-low? 15:44:47 <mgoddard> we would need a patch triage system to make this work 15:44:49 <haerwu> jovial`: that's handled by RP 15:44:56 <jovial`> true 15:45:10 <yoctozepto> haerwu: and backport by backport candidate 15:45:20 <yoctozepto> anyhow, https://review.opendev.org/c/openstack/project-config/+/802744 15:45:36 <haerwu> yoctozepto: "#to-backport #victoria #bugfix" 15:46:27 <mnasiadka> victoria is a branch, that gets into stable backports category in Gerrit dashboard and is easy to find without hashtags 15:46:30 <jovial`> If you let anyone edit the hashtags is there a danger of vandalism? 15:46:45 <yoctozepto> jovial`: I only allow cores 15:46:47 <mnasiadka> I think we should only let cores edit hashtags 15:47:13 <yoctozepto> mnasiadka: yup, my thoughts exactly 15:47:15 <mnasiadka> so, can somebody summarise what we can't set today? backports and priority is possible now 15:47:15 <haerwu> I have a fix for Victoria. So I send patch to master, add '#victoria #bugfix', set BC+2 and give bug# in commit 15:47:59 <yoctozepto> mnasiadka: kind, but that is easily solvable with standardised hashtags 15:48:06 <yoctozepto> which we will be able to modify now 15:48:09 <haerwu> once it in master, I backport to W and V with same hashtags 15:48:15 <mnasiadka> yoctozepto: agreed 15:48:22 <haerwu> and V gets BC-2 15:48:27 <mnasiadka> wonder if hashtags are copied on cherry pick ;) 15:48:37 <haerwu> can be added by core 15:48:39 <yoctozepto> haerwu: that makes sense to me 15:48:54 <yoctozepto> mnasiadka: yeah, need to check, I think they are 15:48:58 <mnasiadka> So let's define some basic ones we start with, and add additional ones on the way? 15:49:08 <yoctozepto> mnasiadka: +++ 15:49:09 <mgoddard> before we go and add a bunch of extra admin to an already stretche team... 15:49:15 <mnasiadka> I can play with Gerrit dashboard 15:49:26 <mgoddard> ... use cases 15:49:29 <yoctozepto> and document the official tags 15:49:38 <mnasiadka> yeah, in contributor guide 15:50:01 <yoctozepto> so the tags are... 15:50:05 <mgoddard> if we start from the queries we think would actually be useful 15:50:12 <mgoddard> we can work backwards to hashtags 15:50:30 <mgoddard> but there is also the question of how to decide which hashtags to apply to a patch 15:50:48 <yoctozepto> mhm 15:50:56 <mgoddard> if the main goal is to tag important patches to focus on, how do we decide which are important? 15:51:16 <mnasiadka> those related to ,,priority features'' we want to implement in the cycle? 15:51:26 <haerwu> I think that we will go with tagging patches as we review them too 15:51:27 <mnasiadka> and those that unbreak CI, as always - for a starter 15:51:29 <mgoddard> and how do we decide what those are? 15:51:56 <mnasiadka> mgoddard: that's very interesting question :) 15:52:06 <mgoddard> I suppose if we had a #feature tag it would be easier to find features 15:52:21 <yoctozepto> yes, feature is a must 15:52:23 <mgoddard> and easier to rank them 15:52:26 <yoctozepto> we can also have a specialised one 15:52:26 <mnasiadka> yes, but it's also easy to find them today if they have bp/something in topic 15:52:33 <yoctozepto> for features being our priorities 15:52:37 <yoctozepto> just that :-) 15:52:47 <mnasiadka> #priority-feature ? :) 15:52:55 <mgoddard> how about this 15:52:56 <yoctozepto> yeah, something like that 15:52:58 <mnasiadka> or just #feature with RP+2 15:53:14 <mgoddard> a feature is a priority for a release if two cores agree to review it 15:53:33 <yoctozepto> hmm 15:53:43 <mgoddard> this is one of the problems I'd like to solve 15:53:46 <yoctozepto> and then we need a system to collect who agrees to overview the work 15:53:56 <mgoddard> yes 15:54:10 <mgoddard> I think it's not too hard 15:54:20 <mgoddard> we document the proces 15:54:23 <yoctozepto> cores as hashtags anyone? 15:54:25 <yoctozepto> #yoctozepto 15:54:40 <mgoddard> patch author or interested party drives the process to become prioritised 15:54:57 * yoctozepto worried about getting overtagged 15:55:21 <mgoddard> we agree as a team at meetings that criteria are met, and add the tag 15:55:40 <yoctozepto> mgoddard: I mean change owners can set tags freely 15:56:05 <yoctozepto> we need a system of punishments then 15:56:13 <yoctozepto> 1 month of no core reviews for abuses 15:56:17 <mgoddard> it's not the end of the world if it gets abused 15:56:29 <mgoddard> someone might review a patch 15:56:44 <yoctozepto> yeah, I'm just laughing here 15:56:53 <yoctozepto> I think it really is sensible 15:57:21 <mgoddard> of course, a future PTL may throw it out 15:57:57 <yoctozepto> I think mnasiadka likes what we design 15:58:22 <mgoddard> there's some nice feedback involved 15:58:30 <mgoddard> contributor proposes patch 15:58:35 <mgoddard> nobody signs up to review 15:58:42 <mgoddard> hmm, maybe I need some karma 15:58:47 <mgoddard> they do some reviews 15:58:59 <mgoddard> 2 cores sign up 15:59:03 <mgoddard> patch lands 15:59:04 <yoctozepto> well, contributors joining our meetings would be cool as well 15:59:08 <mgoddard> sure 15:59:43 <yoctozepto> oh my, we ate up the whole meeting 15:59:54 <mgoddard> but also it may help reviewers focus on patches they signed up for, and not keep contributors hanging 15:59:57 <yoctozepto> how did THAT happen 16:00:06 <mgoddard> there is one more topic 16:00:17 <mgoddard> #topic Kolla operator pain points 16:00:21 <mgoddard> #link https://etherpad.opendev.org/p/pain-point-elimination 16:00:45 <mgoddard> This is being mooted as a potential Y cycle priority 16:00:59 <mgoddard> Operator pain points 16:01:07 <mgoddard> we would need to decide on 1-2 16:01:28 <mgoddard> if anyone has suggestions, please add them to the linked etherpad 16:01:38 <yoctozepto> need to ask less-internal operators I guess? 16:01:42 <mgoddard> we can review next week 16:01:50 <yoctozepto> ++ 16:02:03 <mgoddard> sure, I could resurrect the kolla-klub mailing list 16:02:13 <yoctozepto> +++ 16:02:24 <mnasiadka> we shouldn't be writing what are operators pain points :) 16:02:27 <mgoddard> #action mgoddard email kolla-klubbers & openstack-discuss about pain points 16:02:35 <mgoddard> mnasiadka: you're not an operator? 16:02:52 <mgoddard> but yes, I see your point 16:03:00 <yoctozepto> I think mnasiadka has a similar stance to mine; we understand the internals and can adapt easily 16:03:06 <mnasiadka> mgoddard: I'm a special type of an operator, that pushes a patch 15 minutes after finding a pain point :) 16:03:15 <yoctozepto> :-) 16:03:18 <mgoddard> if only they were all like mnasiadka 16:03:27 <mgoddard> anyway, time's up 16:03:33 <mgoddard> thanks all 16:03:36 <yoctozepto> thanks mgoddard 16:03:36 <mnasiadka> ideal world doesn't exist and we wol 16:03:38 <mgoddard> #endmeeting