18:01:14 <gouthamr> #startmeeting tc
18:01:14 <opendevmeet> Meeting started Tue Feb 18 18:01:14 2025 UTC and is due to finish in 60 minutes.  The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:14 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:14 <opendevmeet> The meeting name has been set to 'tc'
18:01:29 <gouthamr> Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.
18:01:32 <gouthamr> Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee
18:01:35 <gouthamr> #topic Roll Call
18:01:41 <gmann> o/
18:01:46 <frickler> \o
18:01:46 <slaweq> o/
18:01:47 <gtema> o/
18:01:47 <noonedeadpunk> o/
18:01:49 <cardoe> o/
18:02:00 <bauzas> \o
18:03:29 <gouthamr> courtesy-ping: spotz[m]
18:05:00 <gouthamr> alright its that magical time of 18:05, thanks for joining everyone.. lets get started
18:05:01 <spotz[m]> o/
18:05:09 <gouthamr> #topic Last Week's AI
18:05:47 <gouthamr> we continued discussion around review permissions in openstackdocstheme and os-api-ref
18:05:55 <gtema> this is done. Thanks
18:05:57 <gmann> yeah, this is all done
18:06:07 <gouthamr> nice, thank you gmann gtema
18:06:32 <gouthamr> we took an AI to include tech-committee repos in the roll generation script
18:06:57 <gouthamr> #link https://review.opendev.org/c/openstack/election/+/941612 (Add tc repositories for electoral roll generation)
18:07:21 <bauzas> ++
18:07:27 <gouthamr> ^ i think the extra-acs here look like the ones in the SIGs, but, i may be missing the schema
18:07:51 <gouthamr> is there a schema we've published anywhere, or is it just in code?
18:08:07 <fungi> one outcome of that worth considering is that anyone who gets a nomination merged to the election repo qualifies as a contributor, which becomes sort of circular (but maybe acceptable anyway)
18:08:18 <gouthamr> ah
18:08:20 <gouthamr> yes
18:08:47 <gmann> that is good point
18:08:59 <gmann> we should not include the election repo in that?
18:09:07 <gouthamr> i think that's fine, the only case would be someone contesting to be on the TC without any gerrit changes
18:09:29 <gmann> it is valid for PTL also right?
18:09:34 <fungi> they become part of the tc electorate (maybe only for the next two cycles rather than the current one?), but anyone who's a foundation member can run for the tc so it doesn't really change that
18:09:44 <gouthamr> no, PTLs should have a change merged
18:09:46 <frickler> why not? PTL candidates need to be contributors anyway? and any valid candidacy for TC IMHO is a valid contribution in itself
18:09:59 <gmann> ohk its APC not AC
18:10:31 <fungi> yeah, getting a change merged in openstack/election doesn't qualify them to run for or vote on ptl elections
18:10:39 <gmann> so it is just solving the 'tc electorate' if anyone only contribute on TC repo and no impact on TC candidacy
18:10:39 <fungi> just to vote in tc elections
18:10:43 <gmann> yeah
18:10:56 <bauzas> yeah this is for the electorate
18:11:02 <bauzas> not for nominations
18:11:11 <gmann> I think then it is fine. agree with what gouthamr mentioned earlier
18:11:53 <gouthamr> ty, i'll address the comments on the patch
18:12:45 <fungi> it does also mean we'll start including them in the list of contributors to the release on the www site release announcement page
18:12:52 <fungi> which seems fine to me
18:12:59 <gouthamr> +1
18:13:47 <fungi> as the person who's been generating that list for years, i see it as a benefit if anything
18:14:06 <gouthamr> nice
18:14:24 <gouthamr> i'll ping you post this meeting if i can test something around extra-acs
18:14:43 <gouthamr> speaking of extra-acs, i see the deadline in the release calendar to be last week
18:14:52 <gouthamr> but, realistically, its until the email deadline, correct?
18:15:40 <fungi> it was supposed to be prior to start of nominations so that extra acs could qualify as ptl candidates
18:16:20 <gouthamr> (email address confirmation deadline is in about 5 hours 45 mins - Feb 19, 2025 00:00 UTC)
18:16:24 <fungi> but if it merges before the tag used to designate the state of the governance repo for the election then they'll at least get ballots
18:16:44 <gouthamr> ah i see
18:17:01 <gouthamr> this would have been helpful is for mistral i think
18:17:08 <gouthamr> s/is//
18:17:28 <gmann> mistral is all good right?
18:17:29 <gmann> #link https://review.opendev.org/c/openstack/election/+/940859
18:17:52 <gouthamr> i noticed the PTL candidate didn't have a patch merged, but they submitted it and got it through after ianychoi pointed the candidacy requirements to them
18:18:00 <gtema> I have a problem with a deadline. Recently there was a broadcast in the oftc with link to https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/7DKEV7IEHOTHED7RVEFG7WIDVUC4MY3Z/
18:18:11 <gtema> and that stated: Nominations end           @ 2025-02-19 23:45:00 UTC
18:18:51 <slaweq> gtema this is correct
18:19:09 <gtema> but that is not in 5 hours
18:19:18 <gtema> it's more then 24 hours to go
18:19:20 <spotz[m]> That’s what it looked like when I looked at Ian’s patch
18:19:31 <slaweq> no, it is 24+ hours
18:19:45 <gtema> yeah, I said "more than"
18:19:45 <gouthamr> gtema: i was referring to the email confirmation deadline
18:19:48 <noonedeadpunk> same here https://governance.openstack.org/election/
18:19:57 <noonedeadpunk> so it's alighned
18:20:33 <gouthamr> "The electorate is requested to confirm their email address in gerrit, review.opendev.org > Settings > Contact Information > Preferred Email, prior to Feb 19, 2025 00:00 UTC so that the emailed ballots are mailed to the correct email address."
18:20:55 <gtema> ah ok
18:21:07 <gouthamr> fungi: thanks for the clarification on the extra-acs - i think we can do better next time around this
18:22:05 <gouthamr> alright moving on to the next AI:
18:22:05 <gouthamr> show 'INACTIVE' and 'EMERGING' states in project.yaml
18:22:17 <fungi> when there was no "campaigning" period (or when it was very short) we made sure to set the e-mail confirmation deadline far enough in advance of the polls to give election officials time to generate and review the electorate rolls and possibly deal with any last-minute bugs in related tooling
18:22:19 <gouthamr> ^ we said we'd track it in the tracker, assigned to noonedeadpunk
18:22:30 <noonedeadpunk> we mentioned that it's not urgent and can wait a little bit
18:22:54 <gmann> ++
18:23:07 <gouthamr> yep, i remembered after pasting it from my notes :) all good..
18:23:11 <noonedeadpunk> i do have that in todo list though
18:23:25 <gouthamr> next one, document the process for EOL transition for the "unmaintained/wallaby" branch in an etherpad
18:24:09 <frickler> #link https://etherpad.opendev.org/p/eol-unmaintained-branches
18:24:17 <gouthamr> ++ ty
18:24:30 <gouthamr> #link https://review.opendev.org/c/openstack/releases/+/941458 (Transition unmaintained/wallaby to EOL)
18:24:49 <frickler> it was pretty easy, so I'll propose the next eol patches soonish, too
18:25:02 <gouthamr> the commit message here doesn't mention a deadline, but i think it's supposed to be 2025-03-14
18:25:18 <gouthamr> is that correct, frickler?
18:25:43 <frickler> I wasn't sure whether we'd really need to full 4 weeks again and again, might be worth some discussion
18:26:35 <frickler> we didn't have any feedback on the v patch other than +1s except for what elodilles did, so not sure how much chance is there for w to be different
18:27:14 <frickler> and elodilles promised to review the w change soon
18:27:31 <bauzas> usually every project doesn't have any concern hence why
18:27:40 <gouthamr> i'd suggest 30 days so we can establish a process.. we may not always need it, but no harm?
18:28:09 <bauzas> also given the very long list of unmaintained-cores, there is no really a concern
18:29:08 <frickler> well the harm is in delaying things like cleanup of config-errors, but maybe that's ok
18:30:33 <gouthamr> frickler: if you're going to propose the xena, yoga and zed, we'd run through these pretty fast
18:30:48 <gouthamr> i.e., the 30-day comment period would overlap
18:31:50 <frickler> yeah, and by the time we're done with that, we eol 2023.2 and start the whole opt-in process right again ;)
18:32:01 <bauzas> I honestly  dont't think we need it
18:32:36 <bauzas> what we would solve if we extend the deadline ?
18:33:08 <gouthamr> erm, we're not extending the deadline - we're noting that we haven't set one for this change
18:33:36 <gouthamr> like we did for the victoria EOL patch.. i think you're suggesting that if the unmaintained-cores vote, we can merge this
18:34:09 <bauzas> the naming is bad indeed, my fault
18:34:40 <gouthamr> i don't think everyone on the group would vote, but that's another strategy we can try..
18:34:46 <bauzas> anyway what's the problem to solve ?
18:35:53 <gouthamr> can attempt rewording this, do we mind discussing this after the meeting?
18:35:59 <frickler> ok, let's just go with the 30 days and then there's no problem
18:36:40 <gouthamr> ^ am good with this.. but, will discuss further if necessary after we get through the rest of the agenda
18:36:51 <gouthamr> one last AI from prior weeks:
18:36:53 <gouthamr> slaweq will look up some CI scripts that dansmith used in the past to review the CI usage across project teams
18:37:12 <gouthamr> slaweq: do you want to fill us in?
18:37:25 <gmann> I think 30 days is more than enough for EOL
18:37:28 <slaweq> yes, I did some script
18:37:35 <slaweq> and put results in:
18:37:40 <slaweq> check queue: https://paste.opendev.org/show/bzS4jJgGxMq9UUBfwKNn/
18:37:40 <slaweq> gate queue: https://paste.opendev.org/show/bbrKIKUAGiHsntU1cQ7L/
18:37:51 <slaweq> I used script https://github.com/slawqo/tools/blob/master/jobs_time/infra_usage_stats.py to get those statistics
18:38:21 <slaweq> you can check there how much "nodes" and "hours of using nodes" projects are using in gate and check queue in average
18:38:29 <opendevreview> Merged openstack/election master: Add Artem Goncharov candidacy for Keystone PTL  https://review.opendev.org/c/openstack/election/+/942077
18:38:31 <opendevreview> Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for Vitrage PTL  https://review.opendev.org/c/openstack/election/+/942078
18:38:33 <slaweq> there are some projects with pretty high numbers there
18:38:41 <gouthamr> super insightful!
18:39:04 <slaweq> like e.g. openstack/tacker which needs 119 nodes in average per patch in check queue
18:39:36 <gouthamr> clarkb's math checks out :)
18:39:43 <fungi> ouch
18:40:08 <fungi> that's something like 25% of our aggregate quota
18:40:16 <slaweq> and looking at random patch from this project https://review.opendev.org/c/openstack/tacker/+/940723 it seems they have a lot of non-voting jobs there
18:40:35 <slaweq> so there is definitely room for optimization there IMO
18:41:16 <slaweq> that's all from me basically
18:41:44 <slaweq> I don't know if we want to send this data to the ML and ask teams to try to optimize it
18:42:01 <gouthamr> i think we should
18:42:15 <frickler> is there an issue that needs solving? I think focusing on CI stability would be much more worthwhile
18:42:36 <clarkb> frickler: if we look at tacker as an example it is due to bad job design imo
18:42:45 <spotz[m]> Overload could be causing a lack of resources?
18:43:00 <clarkb> and the issue arose because it is very noticeable when people push patches to projects like that and use all of the resources for a while
18:43:11 <clarkb> people start wondering why their jobs haven't run yet and I get pinged to debug it for them
18:43:15 <fungi> it's twofold. some projects have unstable tests and end up rerunning jobs a lot more which consumes resources. some projects have a lot of jobs with mostly redundant setup and minimal differences between them
18:43:30 <fungi> both consume unnecessary amounts of resources
18:43:37 <clarkb> also we are our own noisy neighbors in many of these clouds
18:43:53 <clarkb> inefficient use of resources makes jobs run slower when tehy do run not just slow time waiting to run
18:44:12 <clarkb> the more people work together to use the resources appropriately the happier everyone will be
18:44:30 <clarkb> so no there is no critical issue to address. Its part of the thousand cuts problem we suffer from and starting somewhere will improve things
18:44:54 <slaweq> clarkb++
18:45:35 <slaweq> fungi this script which I did is just checking last comment from zuul which it will find for each patch, so it is not including rechecks or things like that
18:46:20 <slaweq> it just tries to measure how much resources projects are using in the  single CI jobs run
18:46:40 <fungi> yep, which is still a great data point
18:47:02 <fungi> i was responding to frickler's point about focusing on job stability instead
18:47:23 <slaweq> ok, just wanted to clarify that :)
18:47:24 <frickler> I think we should care much more about human resources than about compute resources. and in this regard this seems to be the most critical one to me https://governance.openstack.org/tc/reference/upstream-investment-opportunities/2025/quality-assurance-developers.html
18:47:58 <bauzas> a NP-problem to solve
18:48:08 <gouthamr> think having a dialog with project teams about this will be good - there's a good chance some of them will take some action
18:48:37 <fungi> overuse of available compute resources also impacts human energy invested (from the developers waiting to the sysadmins dealing with the added load)
18:48:41 <bauzas> I was leading a project team for years, begging for help
18:48:51 <bauzas> and none of that happened
18:49:47 <bauzas> maybe some new people could join, but we just don't know when
18:50:34 <gmann> I am less and almost no hope of new people could join. we have not seen such help for past 5 year at least
18:51:11 <gmann> contributors are decreasing cycle by cycle and almost no new help in most of the help-needed projects/area
18:51:19 <spotz[m]> We get a few here and there, mainly interns though
18:51:26 <gmann> where?
18:51:52 <gouthamr> i work with like 10 interns right now :)
18:52:09 <gmann> which projects/area
18:52:17 <noonedeadpunk> I was able to pull in help for freezer at least
18:52:31 <noonedeadpunk> comletely new org entering openstack.. hopefully - they will remain
18:53:24 <noonedeadpunk> and having a discussion with a couple more, but we'll see how it will work out
18:53:25 <gouthamr> gmann: i'd be happy to share details, currently these interns are working on UI (Horizon, manila-ui), API (openapi, jsonschema for manila/keystone), documentation
18:54:22 <noonedeadpunk> masakari is another thing they're pretty much interested in...
18:54:34 <gouthamr> gmann: i'd suggest working with diablo_rojo, she constantly highlights interns in OpenStack via blog posts, talks etc
18:54:35 <gmann> ack. testing, CI/CD are really in critical stage of help and nothing worked there for so many years. maybe that is very less important areas
18:54:37 <gouthamr> and we have this: https://openinfra.org/university-partnership-program/
18:54:39 <fungi> i'm hopeful that regulatory compliance costs downstream will push companies to invest more in upstream collaboration in order to reduce their costs
18:54:42 <JayF> Similarly one of the newest core reviewers and contributors on Ironic, cid, is a former-fellow from the MLH fellowships that GR-OSS runs.
18:55:21 <gmann> I know all these program but I have not see any help in testing, RBAC etc where we really need help than doing extra features
18:55:24 <JayF> gmann: I think what makes CI and Testing contribution difficult is that it can require more context than targeted feature contributions; at least that's the pattern I see emerging when trying to teach newer ironic contributors how to contribute to the devstack plugin of ours.
18:55:34 <spotz[m]> We do have funding for an Outreachy intern, you could definitely submit a project to see if there's interest
18:55:38 <gouthamr> gmann: i'd be happy to work with you to mentor someone into this space
18:56:12 <JayF> gmann: I'm unsure how to solve that problem in the larger case; but a strategy I've taken on is to move my own contributions more towards the common bits and try to offload my project-specific work to newer contributors.
18:56:32 <gouthamr> hmm, that's a good strategy - i realize i'm doing the same
18:57:05 <gmann> gouthamr: mentorship is not problem here, problem is who to mentor when no one interested to know/help in that area :)
18:57:08 <noonedeadpunk> I tend to agree, is that CI/CD is what companies are least eager to contribute
18:57:28 <noonedeadpunk> as rarely they have a decent testing even downstream...
18:57:30 <noonedeadpunk> (if any)
18:57:44 <JayF> noonedeadpunk: it is much more difficult to draw a line from upstream testing to direct downstream benefit, even if we all understand the value of quality CI
18:57:45 <bauzas> they usually prefer to test their own products :(
18:57:55 <fungi> seems like a lack of downstream testing would make upstream testing even more important?
18:58:05 <gouthamr> time check, ~2 mins
18:58:52 <noonedeadpunk> bauzas: or they try to just rely on upstream CI as lack knowledge/resources to make their own
18:59:16 <noonedeadpunk> I perosnally see second even more frequent
18:59:33 <noonedeadpunk> and then - it doesn't benfit their time to market or anything
18:59:35 <gmann> or decrease/no upstream testing will make it realize when everyone need to do more downstream tetsing
18:59:36 <noonedeadpunk> so yeah
19:00:41 <JayF> It's worth noting that upstream integration testing will be more important than ever with the eventlet migration. It'll be a change that will be more likely than ever to cause a headache.
19:00:55 <gouthamr> ++
19:01:57 <gmann> true but problem here is that we are the only one who understand that and not many/most of the organization who still use openstack but does not help much in upstream activities.
19:02:01 <gouthamr> anothing thing i know is that we have quality-focussed people in each project team, except they haven't been actively maintaining common things like devstack, tempest - these may be too much heavylifting than they desire :(
19:02:24 <gouthamr> s/anothing/another - my keyboard seems hungry
19:02:38 <gouthamr> think this topic certainly needs more room
19:02:45 <gouthamr> but that's our time for today..
19:03:06 <opendevreview> Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for TC  https://review.opendev.org/c/openstack/election/+/942080
19:03:07 <opendevreview> Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for OpenStack-Ansible PTL  https://review.opendev.org/c/openstack/election/+/942083
19:03:35 <gouthamr> noonedeadpunk leading half of OpenStack :D ^ /jk
19:03:56 <gouthamr> thanks for sharing your opinions; i think there're some ideas, and needs.. will put something on the PTG agenda and hope we can refine these
19:04:13 <gouthamr> thanks for joining everyone!
19:04:17 <gouthamr> #endmeeting