18:01:14 <gouthamr> #startmeeting tc 18:01:14 <opendevmeet> Meeting started Tue Feb 18 18:01:14 2025 UTC and is due to finish in 60 minutes. The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:14 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:14 <opendevmeet> The meeting name has been set to 'tc' 18:01:29 <gouthamr> Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct. 18:01:32 <gouthamr> Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee 18:01:35 <gouthamr> #topic Roll Call 18:01:41 <gmann> o/ 18:01:46 <frickler> \o 18:01:46 <slaweq> o/ 18:01:47 <gtema> o/ 18:01:47 <noonedeadpunk> o/ 18:01:49 <cardoe> o/ 18:02:00 <bauzas> \o 18:03:29 <gouthamr> courtesy-ping: spotz[m] 18:05:00 <gouthamr> alright its that magical time of 18:05, thanks for joining everyone.. lets get started 18:05:01 <spotz[m]> o/ 18:05:09 <gouthamr> #topic Last Week's AI 18:05:47 <gouthamr> we continued discussion around review permissions in openstackdocstheme and os-api-ref 18:05:55 <gtema> this is done. Thanks 18:05:57 <gmann> yeah, this is all done 18:06:07 <gouthamr> nice, thank you gmann gtema 18:06:32 <gouthamr> we took an AI to include tech-committee repos in the roll generation script 18:06:57 <gouthamr> #link https://review.opendev.org/c/openstack/election/+/941612 (Add tc repositories for electoral roll generation) 18:07:21 <bauzas> ++ 18:07:27 <gouthamr> ^ i think the extra-acs here look like the ones in the SIGs, but, i may be missing the schema 18:07:51 <gouthamr> is there a schema we've published anywhere, or is it just in code? 18:08:07 <fungi> one outcome of that worth considering is that anyone who gets a nomination merged to the election repo qualifies as a contributor, which becomes sort of circular (but maybe acceptable anyway) 18:08:18 <gouthamr> ah 18:08:20 <gouthamr> yes 18:08:47 <gmann> that is good point 18:08:59 <gmann> we should not include the election repo in that? 18:09:07 <gouthamr> i think that's fine, the only case would be someone contesting to be on the TC without any gerrit changes 18:09:29 <gmann> it is valid for PTL also right? 18:09:34 <fungi> they become part of the tc electorate (maybe only for the next two cycles rather than the current one?), but anyone who's a foundation member can run for the tc so it doesn't really change that 18:09:44 <gouthamr> no, PTLs should have a change merged 18:09:46 <frickler> why not? PTL candidates need to be contributors anyway? and any valid candidacy for TC IMHO is a valid contribution in itself 18:09:59 <gmann> ohk its APC not AC 18:10:31 <fungi> yeah, getting a change merged in openstack/election doesn't qualify them to run for or vote on ptl elections 18:10:39 <gmann> so it is just solving the 'tc electorate' if anyone only contribute on TC repo and no impact on TC candidacy 18:10:39 <fungi> just to vote in tc elections 18:10:43 <gmann> yeah 18:10:56 <bauzas> yeah this is for the electorate 18:11:02 <bauzas> not for nominations 18:11:11 <gmann> I think then it is fine. agree with what gouthamr mentioned earlier 18:11:53 <gouthamr> ty, i'll address the comments on the patch 18:12:45 <fungi> it does also mean we'll start including them in the list of contributors to the release on the www site release announcement page 18:12:52 <fungi> which seems fine to me 18:12:59 <gouthamr> +1 18:13:47 <fungi> as the person who's been generating that list for years, i see it as a benefit if anything 18:14:06 <gouthamr> nice 18:14:24 <gouthamr> i'll ping you post this meeting if i can test something around extra-acs 18:14:43 <gouthamr> speaking of extra-acs, i see the deadline in the release calendar to be last week 18:14:52 <gouthamr> but, realistically, its until the email deadline, correct? 18:15:40 <fungi> it was supposed to be prior to start of nominations so that extra acs could qualify as ptl candidates 18:16:20 <gouthamr> (email address confirmation deadline is in about 5 hours 45 mins - Feb 19, 2025 00:00 UTC) 18:16:24 <fungi> but if it merges before the tag used to designate the state of the governance repo for the election then they'll at least get ballots 18:16:44 <gouthamr> ah i see 18:17:01 <gouthamr> this would have been helpful is for mistral i think 18:17:08 <gouthamr> s/is// 18:17:28 <gmann> mistral is all good right? 18:17:29 <gmann> #link https://review.opendev.org/c/openstack/election/+/940859 18:17:52 <gouthamr> i noticed the PTL candidate didn't have a patch merged, but they submitted it and got it through after ianychoi pointed the candidacy requirements to them 18:18:00 <gtema> I have a problem with a deadline. Recently there was a broadcast in the oftc with link to https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/7DKEV7IEHOTHED7RVEFG7WIDVUC4MY3Z/ 18:18:11 <gtema> and that stated: Nominations end @ 2025-02-19 23:45:00 UTC 18:18:51 <slaweq> gtema this is correct 18:19:09 <gtema> but that is not in 5 hours 18:19:18 <gtema> it's more then 24 hours to go 18:19:20 <spotz[m]> That’s what it looked like when I looked at Ian’s patch 18:19:31 <slaweq> no, it is 24+ hours 18:19:45 <gtema> yeah, I said "more than" 18:19:45 <gouthamr> gtema: i was referring to the email confirmation deadline 18:19:48 <noonedeadpunk> same here https://governance.openstack.org/election/ 18:19:57 <noonedeadpunk> so it's alighned 18:20:33 <gouthamr> "The electorate is requested to confirm their email address in gerrit, review.opendev.org > Settings > Contact Information > Preferred Email, prior to Feb 19, 2025 00:00 UTC so that the emailed ballots are mailed to the correct email address." 18:20:55 <gtema> ah ok 18:21:07 <gouthamr> fungi: thanks for the clarification on the extra-acs - i think we can do better next time around this 18:22:05 <gouthamr> alright moving on to the next AI: 18:22:05 <gouthamr> show 'INACTIVE' and 'EMERGING' states in project.yaml 18:22:17 <fungi> when there was no "campaigning" period (or when it was very short) we made sure to set the e-mail confirmation deadline far enough in advance of the polls to give election officials time to generate and review the electorate rolls and possibly deal with any last-minute bugs in related tooling 18:22:19 <gouthamr> ^ we said we'd track it in the tracker, assigned to noonedeadpunk 18:22:30 <noonedeadpunk> we mentioned that it's not urgent and can wait a little bit 18:22:54 <gmann> ++ 18:23:07 <gouthamr> yep, i remembered after pasting it from my notes :) all good.. 18:23:11 <noonedeadpunk> i do have that in todo list though 18:23:25 <gouthamr> next one, document the process for EOL transition for the "unmaintained/wallaby" branch in an etherpad 18:24:09 <frickler> #link https://etherpad.opendev.org/p/eol-unmaintained-branches 18:24:17 <gouthamr> ++ ty 18:24:30 <gouthamr> #link https://review.opendev.org/c/openstack/releases/+/941458 (Transition unmaintained/wallaby to EOL) 18:24:49 <frickler> it was pretty easy, so I'll propose the next eol patches soonish, too 18:25:02 <gouthamr> the commit message here doesn't mention a deadline, but i think it's supposed to be 2025-03-14 18:25:18 <gouthamr> is that correct, frickler? 18:25:43 <frickler> I wasn't sure whether we'd really need to full 4 weeks again and again, might be worth some discussion 18:26:35 <frickler> we didn't have any feedback on the v patch other than +1s except for what elodilles did, so not sure how much chance is there for w to be different 18:27:14 <frickler> and elodilles promised to review the w change soon 18:27:31 <bauzas> usually every project doesn't have any concern hence why 18:27:40 <gouthamr> i'd suggest 30 days so we can establish a process.. we may not always need it, but no harm? 18:28:09 <bauzas> also given the very long list of unmaintained-cores, there is no really a concern 18:29:08 <frickler> well the harm is in delaying things like cleanup of config-errors, but maybe that's ok 18:30:33 <gouthamr> frickler: if you're going to propose the xena, yoga and zed, we'd run through these pretty fast 18:30:48 <gouthamr> i.e., the 30-day comment period would overlap 18:31:50 <frickler> yeah, and by the time we're done with that, we eol 2023.2 and start the whole opt-in process right again ;) 18:32:01 <bauzas> I honestly dont't think we need it 18:32:36 <bauzas> what we would solve if we extend the deadline ? 18:33:08 <gouthamr> erm, we're not extending the deadline - we're noting that we haven't set one for this change 18:33:36 <gouthamr> like we did for the victoria EOL patch.. i think you're suggesting that if the unmaintained-cores vote, we can merge this 18:34:09 <bauzas> the naming is bad indeed, my fault 18:34:40 <gouthamr> i don't think everyone on the group would vote, but that's another strategy we can try.. 18:34:46 <bauzas> anyway what's the problem to solve ? 18:35:53 <gouthamr> can attempt rewording this, do we mind discussing this after the meeting? 18:35:59 <frickler> ok, let's just go with the 30 days and then there's no problem 18:36:40 <gouthamr> ^ am good with this.. but, will discuss further if necessary after we get through the rest of the agenda 18:36:51 <gouthamr> one last AI from prior weeks: 18:36:53 <gouthamr> slaweq will look up some CI scripts that dansmith used in the past to review the CI usage across project teams 18:37:12 <gouthamr> slaweq: do you want to fill us in? 18:37:25 <gmann> I think 30 days is more than enough for EOL 18:37:28 <slaweq> yes, I did some script 18:37:35 <slaweq> and put results in: 18:37:40 <slaweq> check queue: https://paste.opendev.org/show/bzS4jJgGxMq9UUBfwKNn/ 18:37:40 <slaweq> gate queue: https://paste.opendev.org/show/bbrKIKUAGiHsntU1cQ7L/ 18:37:51 <slaweq> I used script https://github.com/slawqo/tools/blob/master/jobs_time/infra_usage_stats.py to get those statistics 18:38:21 <slaweq> you can check there how much "nodes" and "hours of using nodes" projects are using in gate and check queue in average 18:38:29 <opendevreview> Merged openstack/election master: Add Artem Goncharov candidacy for Keystone PTL https://review.opendev.org/c/openstack/election/+/942077 18:38:31 <opendevreview> Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for Vitrage PTL https://review.opendev.org/c/openstack/election/+/942078 18:38:33 <slaweq> there are some projects with pretty high numbers there 18:38:41 <gouthamr> super insightful! 18:39:04 <slaweq> like e.g. openstack/tacker which needs 119 nodes in average per patch in check queue 18:39:36 <gouthamr> clarkb's math checks out :) 18:39:43 <fungi> ouch 18:40:08 <fungi> that's something like 25% of our aggregate quota 18:40:16 <slaweq> and looking at random patch from this project https://review.opendev.org/c/openstack/tacker/+/940723 it seems they have a lot of non-voting jobs there 18:40:35 <slaweq> so there is definitely room for optimization there IMO 18:41:16 <slaweq> that's all from me basically 18:41:44 <slaweq> I don't know if we want to send this data to the ML and ask teams to try to optimize it 18:42:01 <gouthamr> i think we should 18:42:15 <frickler> is there an issue that needs solving? I think focusing on CI stability would be much more worthwhile 18:42:36 <clarkb> frickler: if we look at tacker as an example it is due to bad job design imo 18:42:45 <spotz[m]> Overload could be causing a lack of resources? 18:43:00 <clarkb> and the issue arose because it is very noticeable when people push patches to projects like that and use all of the resources for a while 18:43:11 <clarkb> people start wondering why their jobs haven't run yet and I get pinged to debug it for them 18:43:15 <fungi> it's twofold. some projects have unstable tests and end up rerunning jobs a lot more which consumes resources. some projects have a lot of jobs with mostly redundant setup and minimal differences between them 18:43:30 <fungi> both consume unnecessary amounts of resources 18:43:37 <clarkb> also we are our own noisy neighbors in many of these clouds 18:43:53 <clarkb> inefficient use of resources makes jobs run slower when tehy do run not just slow time waiting to run 18:44:12 <clarkb> the more people work together to use the resources appropriately the happier everyone will be 18:44:30 <clarkb> so no there is no critical issue to address. Its part of the thousand cuts problem we suffer from and starting somewhere will improve things 18:44:54 <slaweq> clarkb++ 18:45:35 <slaweq> fungi this script which I did is just checking last comment from zuul which it will find for each patch, so it is not including rechecks or things like that 18:46:20 <slaweq> it just tries to measure how much resources projects are using in the single CI jobs run 18:46:40 <fungi> yep, which is still a great data point 18:47:02 <fungi> i was responding to frickler's point about focusing on job stability instead 18:47:23 <slaweq> ok, just wanted to clarify that :) 18:47:24 <frickler> I think we should care much more about human resources than about compute resources. and in this regard this seems to be the most critical one to me https://governance.openstack.org/tc/reference/upstream-investment-opportunities/2025/quality-assurance-developers.html 18:47:58 <bauzas> a NP-problem to solve 18:48:08 <gouthamr> think having a dialog with project teams about this will be good - there's a good chance some of them will take some action 18:48:37 <fungi> overuse of available compute resources also impacts human energy invested (from the developers waiting to the sysadmins dealing with the added load) 18:48:41 <bauzas> I was leading a project team for years, begging for help 18:48:51 <bauzas> and none of that happened 18:49:47 <bauzas> maybe some new people could join, but we just don't know when 18:50:34 <gmann> I am less and almost no hope of new people could join. we have not seen such help for past 5 year at least 18:51:11 <gmann> contributors are decreasing cycle by cycle and almost no new help in most of the help-needed projects/area 18:51:19 <spotz[m]> We get a few here and there, mainly interns though 18:51:26 <gmann> where? 18:51:52 <gouthamr> i work with like 10 interns right now :) 18:52:09 <gmann> which projects/area 18:52:17 <noonedeadpunk> I was able to pull in help for freezer at least 18:52:31 <noonedeadpunk> comletely new org entering openstack.. hopefully - they will remain 18:53:24 <noonedeadpunk> and having a discussion with a couple more, but we'll see how it will work out 18:53:25 <gouthamr> gmann: i'd be happy to share details, currently these interns are working on UI (Horizon, manila-ui), API (openapi, jsonschema for manila/keystone), documentation 18:54:22 <noonedeadpunk> masakari is another thing they're pretty much interested in... 18:54:34 <gouthamr> gmann: i'd suggest working with diablo_rojo, she constantly highlights interns in OpenStack via blog posts, talks etc 18:54:35 <gmann> ack. testing, CI/CD are really in critical stage of help and nothing worked there for so many years. maybe that is very less important areas 18:54:37 <gouthamr> and we have this: https://openinfra.org/university-partnership-program/ 18:54:39 <fungi> i'm hopeful that regulatory compliance costs downstream will push companies to invest more in upstream collaboration in order to reduce their costs 18:54:42 <JayF> Similarly one of the newest core reviewers and contributors on Ironic, cid, is a former-fellow from the MLH fellowships that GR-OSS runs. 18:55:21 <gmann> I know all these program but I have not see any help in testing, RBAC etc where we really need help than doing extra features 18:55:24 <JayF> gmann: I think what makes CI and Testing contribution difficult is that it can require more context than targeted feature contributions; at least that's the pattern I see emerging when trying to teach newer ironic contributors how to contribute to the devstack plugin of ours. 18:55:34 <spotz[m]> We do have funding for an Outreachy intern, you could definitely submit a project to see if there's interest 18:55:38 <gouthamr> gmann: i'd be happy to work with you to mentor someone into this space 18:56:12 <JayF> gmann: I'm unsure how to solve that problem in the larger case; but a strategy I've taken on is to move my own contributions more towards the common bits and try to offload my project-specific work to newer contributors. 18:56:32 <gouthamr> hmm, that's a good strategy - i realize i'm doing the same 18:57:05 <gmann> gouthamr: mentorship is not problem here, problem is who to mentor when no one interested to know/help in that area :) 18:57:08 <noonedeadpunk> I tend to agree, is that CI/CD is what companies are least eager to contribute 18:57:28 <noonedeadpunk> as rarely they have a decent testing even downstream... 18:57:30 <noonedeadpunk> (if any) 18:57:44 <JayF> noonedeadpunk: it is much more difficult to draw a line from upstream testing to direct downstream benefit, even if we all understand the value of quality CI 18:57:45 <bauzas> they usually prefer to test their own products :( 18:57:55 <fungi> seems like a lack of downstream testing would make upstream testing even more important? 18:58:05 <gouthamr> time check, ~2 mins 18:58:52 <noonedeadpunk> bauzas: or they try to just rely on upstream CI as lack knowledge/resources to make their own 18:59:16 <noonedeadpunk> I perosnally see second even more frequent 18:59:33 <noonedeadpunk> and then - it doesn't benfit their time to market or anything 18:59:35 <gmann> or decrease/no upstream testing will make it realize when everyone need to do more downstream tetsing 18:59:36 <noonedeadpunk> so yeah 19:00:41 <JayF> It's worth noting that upstream integration testing will be more important than ever with the eventlet migration. It'll be a change that will be more likely than ever to cause a headache. 19:00:55 <gouthamr> ++ 19:01:57 <gmann> true but problem here is that we are the only one who understand that and not many/most of the organization who still use openstack but does not help much in upstream activities. 19:02:01 <gouthamr> anothing thing i know is that we have quality-focussed people in each project team, except they haven't been actively maintaining common things like devstack, tempest - these may be too much heavylifting than they desire :( 19:02:24 <gouthamr> s/anothing/another - my keyboard seems hungry 19:02:38 <gouthamr> think this topic certainly needs more room 19:02:45 <gouthamr> but that's our time for today.. 19:03:06 <opendevreview> Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for TC https://review.opendev.org/c/openstack/election/+/942080 19:03:07 <opendevreview> Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for OpenStack-Ansible PTL https://review.opendev.org/c/openstack/election/+/942083 19:03:35 <gouthamr> noonedeadpunk leading half of OpenStack :D ^ /jk 19:03:56 <gouthamr> thanks for sharing your opinions; i think there're some ideas, and needs.. will put something on the PTG agenda and hope we can refine these 19:04:13 <gouthamr> thanks for joining everyone! 19:04:17 <gouthamr> #endmeeting