| opendevreview | OpenStack Proposal Bot proposed openstack/openstack-manuals master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-manuals/+/942050 | 04:52 |
|---|---|---|
| opendevreview | Ivan Anfimov proposed openstack/openstack-manuals master: Freezer description additional fix https://review.opendev.org/c/openstack/openstack-manuals/+/942051 | 07:06 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-manuals master: Freezer description additional fix https://review.opendev.org/c/openstack/openstack-manuals/+/942051 | 07:08 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-manuals master: Freezer description additional fix https://review.opendev.org/c/openstack/openstack-manuals/+/942051 | 07:14 |
| opendevreview | OpenStack Proposal Bot proposed openstack/openstack-manuals master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-manuals/+/942050 | 07:26 |
| opendevreview | Juan Larriba proposed openstack/election master: Adding Juan Larriba candidacy for Telemetry https://review.opendev.org/c/openstack/election/+/942068 | 09:15 |
| opendevreview | Merged openstack/election master: [manila] Add Carlos da Silva candidacy for 2025.2 Manila PTL https://review.opendev.org/c/openstack/election/+/942003 | 09:16 |
| opendevreview | Merged openstack/election master: Add Goutham Pacha Ravi candidacy for 2025.2 TC https://review.opendev.org/c/openstack/election/+/942001 | 09:16 |
| opendevreview | Merged openstack/election master: Adding Matt Crees candidacy for Blazar https://review.opendev.org/c/openstack/election/+/941971 | 09:16 |
| opendevreview | Merged openstack/election master: Add Yasufumi Ogawa candidacy for Tacker PTL https://review.opendev.org/c/openstack/election/+/941968 | 09:16 |
| opendevreview | Merged openstack/election master: Add Cyril Roelandt candidacy for Glance PTL 2025.2 https://review.opendev.org/c/openstack/election/+/941937 | 09:16 |
| opendevreview | Merged openstack/election master: Add Artem Goncharov candidacy for OpenStackSDK PTL https://review.opendev.org/c/openstack/election/+/941938 | 09:16 |
| opendevreview | Hemanth N proposed openstack/election master: Adding Hemanth Nakkina PTL candidacy for Sunbeam https://review.opendev.org/c/openstack/election/+/942075 | 10:04 |
| opendevreview | Michal Nasiadka proposed openstack/election master: Add Michal Nasiadka candidacy for 2025.2 TC https://review.opendev.org/c/openstack/election/+/942076 | 10:23 |
| opendevreview | Artem Goncharov proposed openstack/election master: Add Artem Goncharov candidacy for Keystone PTL https://review.opendev.org/c/openstack/election/+/942077 | 10:45 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/election master: Add Dmitriy Rabotyagov candidacy for Vitrage PTL https://review.opendev.org/c/openstack/election/+/942078 | 11:29 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/election master: Add Dmitriy Rabotyagov candidacy for TC https://review.opendev.org/c/openstack/election/+/942080 | 11:40 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/election master: Add Dmitriy Rabotyagov candidacy for OpenStack-Ansible PTL https://review.opendev.org/c/openstack/election/+/942083 | 12:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/election master: Add Dmitriy Rabotyagov candidacy for OpenStack-Ansible PTL https://review.opendev.org/c/openstack/election/+/942083 | 12:21 |
| *** tkajinam is now known as Guest9449 | 13:10 | |
| opendevreview | Merged openstack/election master: Adding Hemanth Nakkina PTL candidacy for Sunbeam https://review.opendev.org/c/openstack/election/+/942075 | 13:52 |
| opendevreview | Merged openstack/election master: Add Michal Nasiadka candidacy for 2025.2 TC https://review.opendev.org/c/openstack/election/+/942076 | 14:00 |
| *** whoami-rajat_ is now known as whoami-rajat | 14:04 | |
| opendevreview | Mauricio Harley proposed openstack/election master: Adding Mauricio Harley candidacy for Barbican https://review.opendev.org/c/openstack/election/+/942110 | 14:55 |
| -opendevstatus- NOTICE: nominations for the OpenStack PTL and TC positions are closing soon, for details see https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/7DKEV7IEHOTHED7RVEFG7WIDVUC4MY3Z/ | 15:59 | |
| gouthamr | tc-members: a gentle reminder that we are meeting here in ~40 minutes | 17:20 |
| gouthamr | #startmeeting tc | 18:01 |
| opendevmeet | Meeting started Tue Feb 18 18:01:14 2025 UTC and is due to finish in 60 minutes. The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot. | 18:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 18:01 |
| opendevmeet | The meeting name has been set to 'tc' | 18:01 |
| gouthamr | Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct. | 18:01 |
| gouthamr | Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee | 18:01 |
| gouthamr | #topic Roll Call | 18:01 |
| gmann | o/ | 18:01 |
| frickler | \o | 18:01 |
| slaweq | o/ | 18:01 |
| gtema | o/ | 18:01 |
| noonedeadpunk | o/ | 18:01 |
| cardoe | o/ | 18:01 |
| bauzas | \o | 18:02 |
| gouthamr | courtesy-ping: spotz[m] | 18:03 |
| gouthamr | alright its that magical time of 18:05, thanks for joining everyone.. lets get started | 18:05 |
| spotz[m] | o/ | 18:05 |
| gouthamr | #topic Last Week's AI | 18:05 |
| gouthamr | we continued discussion around review permissions in openstackdocstheme and os-api-ref | 18:05 |
| gtema | this is done. Thanks | 18:05 |
| gmann | yeah, this is all done | 18:05 |
| gouthamr | nice, thank you gmann gtema | 18:06 |
| gouthamr | we took an AI to include tech-committee repos in the roll generation script | 18:06 |
| gouthamr | #link https://review.opendev.org/c/openstack/election/+/941612 (Add tc repositories for electoral roll generation) | 18:06 |
| bauzas | ++ | 18:07 |
| gouthamr | ^ i think the extra-acs here look like the ones in the SIGs, but, i may be missing the schema | 18:07 |
| gouthamr | is there a schema we've published anywhere, or is it just in code? | 18:07 |
| fungi | one outcome of that worth considering is that anyone who gets a nomination merged to the election repo qualifies as a contributor, which becomes sort of circular (but maybe acceptable anyway) | 18:08 |
| gouthamr | ah | 18:08 |
| gouthamr | yes | 18:08 |
| gmann | that is good point | 18:08 |
| gmann | we should not include the election repo in that? | 18:08 |
| gouthamr | i think that's fine, the only case would be someone contesting to be on the TC without any gerrit changes | 18:09 |
| gmann | it is valid for PTL also right? | 18:09 |
| fungi | they become part of the tc electorate (maybe only for the next two cycles rather than the current one?), but anyone who's a foundation member can run for the tc so it doesn't really change that | 18:09 |
| gouthamr | no, PTLs should have a change merged | 18:09 |
| frickler | why not? PTL candidates need to be contributors anyway? and any valid candidacy for TC IMHO is a valid contribution in itself | 18:09 |
| gmann | ohk its APC not AC | 18:09 |
| fungi | yeah, getting a change merged in openstack/election doesn't qualify them to run for or vote on ptl elections | 18:10 |
| gmann | so it is just solving the 'tc electorate' if anyone only contribute on TC repo and no impact on TC candidacy | 18:10 |
| fungi | just to vote in tc elections | 18:10 |
| gmann | yeah | 18:10 |
| bauzas | yeah this is for the electorate | 18:10 |
| bauzas | not for nominations | 18:11 |
| gmann | I think then it is fine. agree with what gouthamr mentioned earlier | 18:11 |
| gouthamr | ty, i'll address the comments on the patch | 18:11 |
| fungi | it does also mean we'll start including them in the list of contributors to the release on the www site release announcement page | 18:12 |
| fungi | which seems fine to me | 18:12 |
| gouthamr | +1 | 18:12 |
| fungi | as the person who's been generating that list for years, i see it as a benefit if anything | 18:13 |
| gouthamr | nice | 18:14 |
| gouthamr | i'll ping you post this meeting if i can test something around extra-acs | 18:14 |
| gouthamr | speaking of extra-acs, i see the deadline in the release calendar to be last week | 18:14 |
| gouthamr | but, realistically, its until the email deadline, correct? | 18:14 |
| fungi | it was supposed to be prior to start of nominations so that extra acs could qualify as ptl candidates | 18:15 |
| gouthamr | (email address confirmation deadline is in about 5 hours 45 mins - Feb 19, 2025 00:00 UTC) | 18:16 |
| fungi | but if it merges before the tag used to designate the state of the governance repo for the election then they'll at least get ballots | 18:16 |
| gouthamr | ah i see | 18:16 |
| gouthamr | this would have been helpful is for mistral i think | 18:17 |
| gouthamr | s/is// | 18:17 |
| gmann | mistral is all good right? | 18:17 |
| gmann | #link https://review.opendev.org/c/openstack/election/+/940859 | 18:17 |
| gouthamr | i noticed the PTL candidate didn't have a patch merged, but they submitted it and got it through after ianychoi pointed the candidacy requirements to them | 18:17 |
| gtema | I have a problem with a deadline. Recently there was a broadcast in the oftc with link to https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/7DKEV7IEHOTHED7RVEFG7WIDVUC4MY3Z/ | 18:18 |
| gtema | and that stated: Nominations end @ 2025-02-19 23:45:00 UTC | 18:18 |
| slaweq | gtema this is correct | 18:18 |
| gtema | but that is not in 5 hours | 18:19 |
| gtema | it's more then 24 hours to go | 18:19 |
| spotz[m] | That’s what it looked like when I looked at Ian’s patch | 18:19 |
| slaweq | no, it is 24+ hours | 18:19 |
| gtema | yeah, I said "more than" | 18:19 |
| gouthamr | gtema: i was referring to the email confirmation deadline | 18:19 |
| noonedeadpunk | same here https://governance.openstack.org/election/ | 18:19 |
| noonedeadpunk | so it's alighned | 18:19 |
| gouthamr | "The electorate is requested to confirm their email address in gerrit, review.opendev.org > Settings > Contact Information > Preferred Email, prior to Feb 19, 2025 00:00 UTC so that the emailed ballots are mailed to the correct email address." | 18:20 |
| gtema | ah ok | 18:20 |
| gouthamr | fungi: thanks for the clarification on the extra-acs - i think we can do better next time around this | 18:21 |
| gouthamr | alright moving on to the next AI: | 18:22 |
| gouthamr | show 'INACTIVE' and 'EMERGING' states in project.yaml | 18:22 |
| fungi | when there was no "campaigning" period (or when it was very short) we made sure to set the e-mail confirmation deadline far enough in advance of the polls to give election officials time to generate and review the electorate rolls and possibly deal with any last-minute bugs in related tooling | 18:22 |
| gouthamr | ^ we said we'd track it in the tracker, assigned to noonedeadpunk | 18:22 |
| noonedeadpunk | we mentioned that it's not urgent and can wait a little bit | 18:22 |
| gmann | ++ | 18:22 |
| gouthamr | yep, i remembered after pasting it from my notes :) all good.. | 18:23 |
| noonedeadpunk | i do have that in todo list though | 18:23 |
| gouthamr | next one, document the process for EOL transition for the "unmaintained/wallaby" branch in an etherpad | 18:23 |
| frickler | #link https://etherpad.opendev.org/p/eol-unmaintained-branches | 18:24 |
| gouthamr | ++ ty | 18:24 |
| gouthamr | #link https://review.opendev.org/c/openstack/releases/+/941458 (Transition unmaintained/wallaby to EOL) | 18:24 |
| frickler | it was pretty easy, so I'll propose the next eol patches soonish, too | 18:24 |
| gouthamr | the commit message here doesn't mention a deadline, but i think it's supposed to be 2025-03-14 | 18:25 |
| gouthamr | is that correct, frickler? | 18:25 |
| frickler | I wasn't sure whether we'd really need to full 4 weeks again and again, might be worth some discussion | 18:25 |
| frickler | we didn't have any feedback on the v patch other than +1s except for what elodilles did, so not sure how much chance is there for w to be different | 18:26 |
| frickler | and elodilles promised to review the w change soon | 18:27 |
| bauzas | usually every project doesn't have any concern hence why | 18:27 |
| gouthamr | i'd suggest 30 days so we can establish a process.. we may not always need it, but no harm? | 18:27 |
| bauzas | also given the very long list of unmaintained-cores, there is no really a concern | 18:28 |
| frickler | well the harm is in delaying things like cleanup of config-errors, but maybe that's ok | 18:29 |
| gouthamr | frickler: if you're going to propose the xena, yoga and zed, we'd run through these pretty fast | 18:30 |
| gouthamr | i.e., the 30-day comment period would overlap | 18:30 |
| frickler | yeah, and by the time we're done with that, we eol 2023.2 and start the whole opt-in process right again ;) | 18:31 |
| bauzas | I honestly dont't think we need it | 18:32 |
| bauzas | what we would solve if we extend the deadline ? | 18:32 |
| gouthamr | erm, we're not extending the deadline - we're noting that we haven't set one for this change | 18:33 |
| gouthamr | like we did for the victoria EOL patch.. i think you're suggesting that if the unmaintained-cores vote, we can merge this | 18:33 |
| bauzas | the naming is bad indeed, my fault | 18:34 |
| gouthamr | i don't think everyone on the group would vote, but that's another strategy we can try.. | 18:34 |
| bauzas | anyway what's the problem to solve ? | 18:34 |
| gouthamr | can attempt rewording this, do we mind discussing this after the meeting? | 18:35 |
| frickler | ok, let's just go with the 30 days and then there's no problem | 18:35 |
| gouthamr | ^ am good with this.. but, will discuss further if necessary after we get through the rest of the agenda | 18:36 |
| gouthamr | one last AI from prior weeks: | 18:36 |
| gouthamr | slaweq will look up some CI scripts that dansmith used in the past to review the CI usage across project teams | 18:36 |
| gouthamr | slaweq: do you want to fill us in? | 18:37 |
| gmann | I think 30 days is more than enough for EOL | 18:37 |
| slaweq | yes, I did some script | 18:37 |
| slaweq | and put results in: | 18:37 |
| slaweq | check queue: https://paste.opendev.org/show/bzS4jJgGxMq9UUBfwKNn/ | 18:37 |
| slaweq | gate queue: https://paste.opendev.org/show/bbrKIKUAGiHsntU1cQ7L/ | 18:37 |
| slaweq | I used script https://github.com/slawqo/tools/blob/master/jobs_time/infra_usage_stats.py to get those statistics | 18:37 |
| slaweq | you can check there how much "nodes" and "hours of using nodes" projects are using in gate and check queue in average | 18:38 |
| opendevreview | Merged openstack/election master: Add Artem Goncharov candidacy for Keystone PTL https://review.opendev.org/c/openstack/election/+/942077 | 18:38 |
| opendevreview | Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for Vitrage PTL https://review.opendev.org/c/openstack/election/+/942078 | 18:38 |
| slaweq | there are some projects with pretty high numbers there | 18:38 |
| gouthamr | super insightful! | 18:38 |
| slaweq | like e.g. openstack/tacker which needs 119 nodes in average per patch in check queue | 18:39 |
| gouthamr | clarkb's math checks out :) | 18:39 |
| fungi | ouch | 18:39 |
| fungi | that's something like 25% of our aggregate quota | 18:40 |
| slaweq | and looking at random patch from this project https://review.opendev.org/c/openstack/tacker/+/940723 it seems they have a lot of non-voting jobs there | 18:40 |
| slaweq | so there is definitely room for optimization there IMO | 18:40 |
| slaweq | that's all from me basically | 18:41 |
| slaweq | I don't know if we want to send this data to the ML and ask teams to try to optimize it | 18:41 |
| gouthamr | i think we should | 18:42 |
| frickler | is there an issue that needs solving? I think focusing on CI stability would be much more worthwhile | 18:42 |
| clarkb | frickler: if we look at tacker as an example it is due to bad job design imo | 18:42 |
| spotz[m] | Overload could be causing a lack of resources? | 18:42 |
| clarkb | and the issue arose because it is very noticeable when people push patches to projects like that and use all of the resources for a while | 18:43 |
| clarkb | people start wondering why their jobs haven't run yet and I get pinged to debug it for them | 18:43 |
| fungi | it's twofold. some projects have unstable tests and end up rerunning jobs a lot more which consumes resources. some projects have a lot of jobs with mostly redundant setup and minimal differences between them | 18:43 |
| fungi | both consume unnecessary amounts of resources | 18:43 |
| clarkb | also we are our own noisy neighbors in many of these clouds | 18:43 |
| clarkb | inefficient use of resources makes jobs run slower when tehy do run not just slow time waiting to run | 18:43 |
| clarkb | the more people work together to use the resources appropriately the happier everyone will be | 18:44 |
| clarkb | so no there is no critical issue to address. Its part of the thousand cuts problem we suffer from and starting somewhere will improve things | 18:44 |
| slaweq | clarkb++ | 18:44 |
| slaweq | fungi this script which I did is just checking last comment from zuul which it will find for each patch, so it is not including rechecks or things like that | 18:45 |
| slaweq | it just tries to measure how much resources projects are using in the single CI jobs run | 18:46 |
| fungi | yep, which is still a great data point | 18:46 |
| fungi | i was responding to frickler's point about focusing on job stability instead | 18:47 |
| slaweq | ok, just wanted to clarify that :) | 18:47 |
| frickler | I think we should care much more about human resources than about compute resources. and in this regard this seems to be the most critical one to me https://governance.openstack.org/tc/reference/upstream-investment-opportunities/2025/quality-assurance-developers.html | 18:47 |
| bauzas | a NP-problem to solve | 18:47 |
| gouthamr | think having a dialog with project teams about this will be good - there's a good chance some of them will take some action | 18:48 |
| fungi | overuse of available compute resources also impacts human energy invested (from the developers waiting to the sysadmins dealing with the added load) | 18:48 |
| bauzas | I was leading a project team for years, begging for help | 18:48 |
| bauzas | and none of that happened | 18:48 |
| bauzas | maybe some new people could join, but we just don't know when | 18:49 |
| gmann | I am less and almost no hope of new people could join. we have not seen such help for past 5 year at least | 18:50 |
| gmann | contributors are decreasing cycle by cycle and almost no new help in most of the help-needed projects/area | 18:51 |
| spotz[m] | We get a few here and there, mainly interns though | 18:51 |
| gmann | where? | 18:51 |
| gouthamr | i work with like 10 interns right now :) | 18:51 |
| gmann | which projects/area | 18:52 |
| noonedeadpunk | I was able to pull in help for freezer at least | 18:52 |
| noonedeadpunk | comletely new org entering openstack.. hopefully - they will remain | 18:52 |
| noonedeadpunk | and having a discussion with a couple more, but we'll see how it will work out | 18:53 |
| gouthamr | gmann: i'd be happy to share details, currently these interns are working on UI (Horizon, manila-ui), API (openapi, jsonschema for manila/keystone), documentation | 18:53 |
| noonedeadpunk | masakari is another thing they're pretty much interested in... | 18:54 |
| gouthamr | gmann: i'd suggest working with diablo_rojo, she constantly highlights interns in OpenStack via blog posts, talks etc | 18:54 |
| gmann | ack. testing, CI/CD are really in critical stage of help and nothing worked there for so many years. maybe that is very less important areas | 18:54 |
| gouthamr | and we have this: https://openinfra.org/university-partnership-program/ | 18:54 |
| fungi | i'm hopeful that regulatory compliance costs downstream will push companies to invest more in upstream collaboration in order to reduce their costs | 18:54 |
| JayF | Similarly one of the newest core reviewers and contributors on Ironic, cid, is a former-fellow from the MLH fellowships that GR-OSS runs. | 18:54 |
| gmann | I know all these program but I have not see any help in testing, RBAC etc where we really need help than doing extra features | 18:55 |
| JayF | gmann: I think what makes CI and Testing contribution difficult is that it can require more context than targeted feature contributions; at least that's the pattern I see emerging when trying to teach newer ironic contributors how to contribute to the devstack plugin of ours. | 18:55 |
| spotz[m] | We do have funding for an Outreachy intern, you could definitely submit a project to see if there's interest | 18:55 |
| gouthamr | gmann: i'd be happy to work with you to mentor someone into this space | 18:55 |
| JayF | gmann: I'm unsure how to solve that problem in the larger case; but a strategy I've taken on is to move my own contributions more towards the common bits and try to offload my project-specific work to newer contributors. | 18:56 |
| gouthamr | hmm, that's a good strategy - i realize i'm doing the same | 18:56 |
| gmann | gouthamr: mentorship is not problem here, problem is who to mentor when no one interested to know/help in that area :) | 18:57 |
| noonedeadpunk | I tend to agree, is that CI/CD is what companies are least eager to contribute | 18:57 |
| noonedeadpunk | as rarely they have a decent testing even downstream... | 18:57 |
| noonedeadpunk | (if any) | 18:57 |
| JayF | noonedeadpunk: it is much more difficult to draw a line from upstream testing to direct downstream benefit, even if we all understand the value of quality CI | 18:57 |
| bauzas | they usually prefer to test their own products :( | 18:57 |
| fungi | seems like a lack of downstream testing would make upstream testing even more important? | 18:57 |
| gouthamr | time check, ~2 mins | 18:58 |
| noonedeadpunk | bauzas: or they try to just rely on upstream CI as lack knowledge/resources to make their own | 18:58 |
| noonedeadpunk | I perosnally see second even more frequent | 18:59 |
| noonedeadpunk | and then - it doesn't benfit their time to market or anything | 18:59 |
| gmann | or decrease/no upstream testing will make it realize when everyone need to do more downstream tetsing | 18:59 |
| noonedeadpunk | so yeah | 18:59 |
| JayF | It's worth noting that upstream integration testing will be more important than ever with the eventlet migration. It'll be a change that will be more likely than ever to cause a headache. | 19:00 |
| gouthamr | ++ | 19:00 |
| gmann | true but problem here is that we are the only one who understand that and not many/most of the organization who still use openstack but does not help much in upstream activities. | 19:01 |
| gouthamr | anothing thing i know is that we have quality-focussed people in each project team, except they haven't been actively maintaining common things like devstack, tempest - these may be too much heavylifting than they desire :( | 19:02 |
| gouthamr | s/anothing/another - my keyboard seems hungry | 19:02 |
| gouthamr | think this topic certainly needs more room | 19:02 |
| gouthamr | but that's our time for today.. | 19:02 |
| opendevreview | Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for TC https://review.opendev.org/c/openstack/election/+/942080 | 19:03 |
| opendevreview | Merged openstack/election master: Add Dmitriy Rabotyagov candidacy for OpenStack-Ansible PTL https://review.opendev.org/c/openstack/election/+/942083 | 19:03 |
| gouthamr | noonedeadpunk leading half of OpenStack :D ^ /jk | 19:03 |
| gouthamr | thanks for sharing your opinions; i think there're some ideas, and needs.. will put something on the PTG agenda and hope we can refine these | 19:03 |
| gouthamr | thanks for joining everyone! | 19:04 |
| gouthamr | #endmeeting | 19:04 |
| opendevmeet | Meeting ended Tue Feb 18 19:04:17 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 19:04 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/tc/2025/tc.2025-02-18-18.01.html | 19:04 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/tc/2025/tc.2025-02-18-18.01.txt | 19:04 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/tc/2025/tc.2025-02-18-18.01.log.html | 19:04 |
| * noonedeadpunk would love to have some rest one day... | 19:04 | |
| gouthamr | i'm very glad to see the nominations piling up... after chatting with multiple people concerned about the non-candidacies | 19:05 |
| fungi | seems like no matter how long the nomination period is and how far in advance we start reminding everyone it's coming up, the bulk of nominations still only come in the final 24-48 hours | 19:06 |
| gouthamr | not wanting to sound pompous, i actually liked the non-candidacies.. this community's known so many leaders who ensured they mentored someone before they stepped away for a well deserved break.. many of you're on that list :P | 19:06 |
| gouthamr | bauzas: looping back on the unmaintained to EOL thingy - this has been getting discussed a while.. but, we identified some gaps in the TC's resolutions regarding "unmaintained" branches | 19:11 |
| gouthamr | specifically: https://governance.openstack.org/tc/resolutions/20230724-unmaintained-branches.html | 19:12 |
| gouthamr | and https://governance.openstack.org/tc/resolutions/20231114-amend-unmaintained-status.html | 19:12 |
| gouthamr | these established that V,W,X,Y,Z would be renamed "unmaintained/<branch>" from "stable/<branch>" and that there'd be an "openstack-unmaintained-core" that manages changes to these branches | 19:13 |
| gouthamr | they also called out that there ought to be a liaison for each branch, and future "unmaintained" branch transitions will only happen if there's interest from someone to maintain these | 19:14 |
| gouthamr | so ideally if someone wanted "unmaintained/2023.1" for projectX, but not for projectY - the release team would transition projectY to EOL | 19:15 |
| gouthamr | but, in reality that's not what's happening.. | 19:15 |
| gouthamr | so there's a need to clean things up a bit - because lots of code branches were languishing with CI jobs, Zuul configuration errors, and project teams were probably still needing to pay attention to people proposing changes and asking for help | 19:16 |
| gouthamr | unmaintained branches are not the responsibility of project teams unless they want to | 19:16 |
| gouthamr | so, this is the broken situation we're trying to remedy | 19:17 |
| gouthamr | frickler started the effort to EOL old branches.. and we'd ideally fill the gaps in the TC resolutions so this doesn't happen in the future | 19:17 |
| gouthamr | when we did this though, we didn't want to sound like we're reneging on any prior promises/understanding.. that's why these EOL patches had a 30-day comment period | 19:18 |
| gouthamr | we just made it up, allowing wide visibility, and we send emails and reminders to folks, add release liaisons and PTLs to the gerrit change, and wait for objections | 19:19 |
| gouthamr | we probably don't _need_ to do it, but, i'm for erring on the side of transparency and caution here... | 19:20 |
| gouthamr | another opinion of mine, that i think others share as well is that: | 19:22 |
| gouthamr | unmaintained branches are solely the responsibility of the openstack-unmaintained-core team (and other project specific that signed up to "maintain" these); neither the release team nor the TC has to/wants to do anything about this. | 19:22 |
| gouthamr | the current action is solely to clean things up for a situation that festered because of unwritten/misunderstood things | 19:22 |
| gouthamr | if you have anything to ask about that wall of text, do please :) | 19:23 |
| clarkb | from a largely outside perspective looking in it seems like if there is reduced capacity for effort it becomes important to reduce scope as much as possible (maybe don't worry about old branches so much; let them go. Trim back testing to core sets of functionality and users of tertiary functionality either step up or risk things breaking. etc) and prioritize the work that can be | 19:25 |
| clarkb | accomplished by the smaller group. This may also require a shift in thinking about who is responsible for what. Maybe its less about project repo ownership and more about accomplishing goals (eventlet removal and associated CI. SLURP. whatever the collective decides is most important). Then work through that priority list in order of priority and capacity | 19:25 |
| fungi | a possible approach is to declare the eol process the responsibility of the unmaintained-core team, and set a deadline at a point in every cycle where if they haven't taken care of it then anyone is welcome to just go ahead and work on eol'ing it all | 19:25 |
| fungi | basically put the people who need to opt-in in charge of seeking for opt-ins, and if there's nobody to follow that process in a timely manner then the process has failed and can be abandoned | 19:27 |
| gouthamr | true | 19:29 |
| fungi | it just doesi't make sense to add more responsibilities for it to the tc members, release team, et cetera when the whole point of those branches being unmaintained is that they should be making less work for those people | 19:31 |
| JayF | clarkb: yeah, I think it's extremely difficult for folks who have been working on/using openstack for so long to ask the question "what do we /really/ need and what do we /really/ have time to work on" | 19:34 |
| JayF | so it leads to a pattern of (and I have been guilty of this) a few people trying to take on too much responsibility and doing them all to maybe 75% quality | 19:34 |
| gouthamr | fungi: agreed; the tasks are hard and time consuming - the TC has the onus of making the policies, and ensuring they're interpreted correctly and, evolving it because we learned something new.. i for one am for the ideas that clarkb and you're sharing.. will post them into the PTG etherpad | 19:35 |
| JayF | along those lines, I see things like TheJulia's work on OCI container registry support for images for Ironic, and ask myself is there a path to integrate that into nova and potentially remove the need for glance from some clouds? | 19:35 |
| gouthamr | "doing them all to maybe 75% quality" /me chuckles | 19:35 |
| JayF | The only way to get all the work done with less people is to either find efficiencies (I think we've done this a lot already) or to make hard decisions on what to cut | 19:35 |
| TheJulia | A solid thing to keep in mind: Perfection is the enemy of good. | 19:37 |
| * TheJulia appears since summoned... thanks Jay ;) | 19:37 | |
| spotz[m] | hehe | 19:37 |
| JayF | TheJulia: don't wanna take credit for the approximately 90 dozen patches you've written in that direction ;) | 19:38 |
| * JayF lunch brb | 19:38 | |
| fungi | s/credit/blame/ ? ;) | 19:38 |
| clarkb | I'm not going to say glance has always been an easy tool to work with but container registries are almost always terrible ime | 19:38 |
| TheJulia | JayF: Eh, I make up for it with things like I'm doing right now, fixing glaringly obvious small minor things | 19:38 |
| clarkb | the whole container image protocol makes it difficult to cache things | 19:38 |
| clarkb | the way hashes are used makes it difficult to check if the hash you see in a server matches what is in the registry which matches what was originally uploaded | 19:39 |
| TheJulia | clarkb: totally does, people need to do things like.. copy the container to a different registry | 19:39 |
| clarkb | many of the registries seem to think that security scanning is the most fundamental important thing to have in a container registry and forget all of the basic stuff like "I need to be able to prune unneeded data without downtime" | 19:39 |
| TheJulia | clarkb: I'd disagree with the make it difficult comment, IF your explicitly knowing the digest you want | 19:39 |
| TheJulia | clarkb: but I've taken way too much time figuring out how it works under that hood and I'll admit the documentation is not ideal | 19:40 |
| clarkb | TheJulia: right if you know the underlying protocol and know how to map the different shas it is workable | 19:40 |
| TheJulia | (OCI protocol docs, that is) | 19:40 |
| clarkb | the problem is docker and podman show you one hash | 19:40 |
| clarkb | the registry shows you another | 19:40 |
| clarkb | why? because the protocal basically | 19:40 |
| TheJulia | Well | 19:40 |
| clarkb | the registries show you manifest hashes | 19:40 |
| clarkb | the local tools show you image hashes | 19:40 |
| clarkb | you need to open the manifest to see the image hashes | 19:40 |
| TheJulia | your not referring to the last layer or object, your referring to the document's digest which explains everything | 19:41 |
| TheJulia | which is the composition of layers | 19:41 |
| clarkb | the manifest hash for an image can differ between registries if you don't copy every image within the manifest (occurs when you have multiarch images and copy a subset) | 19:41 |
| clarkb | right the problem is almost entirely a UX issue though | 19:41 |
| TheJulia | it is an artifact of the data model | 19:41 |
| clarkb | the hash I see in docker image ls should be easily verified via docker hub's dashboard for that image and tag | 19:41 |
| clarkb | anyway I think the point stands the whole system is a huge pita to work with | 19:41 |
| TheJulia | Also, if you change metadata at all, your digest value changes but your layers might not | 19:42 |
| TheJulia | Indeed if your focused on that level :) | 19:42 |
| clarkb | I mean its a bit of a fundamental problem I want image management to address. I need to be able to confirm that the server has received what I uploaded intact (we've had glance images fail at least once because there was a bitflip in upload) | 19:43 |
| clarkb | and I need to be able to confirm that what I've pull down locally (whether a container or VM image) is what is in the server | 19:43 |
| clarkb | now glance doesn't really solve this problem either. I'm just pointing out that if we're going to replace glance lets replace it with something that solves these major issues | 19:43 |
| clarkb | not make them worse or perpetuate them | 19:43 |
| clarkb | then if your job is to run the registry not being able to prune things is a major headache. The canonical registry is basically append only if you can't take downtime | 19:44 |
| TheJulia | I guess the issue is, the data is there, just not exposed either | 19:44 |
| * TheJulia shrugs | 19:44 | |
| TheJulia | nothing is perfect :) | 19:44 |
| clarkb | and whole disk images can be quite large. Its a smaller problem to be append only when your images average 250MB bigger issue when that bumps to 20GB | 19:45 |
| TheJulia | not sure at what your getting at there | 19:46 |
| clarkb | the reason opendev does not run an official container registry is that there is no way to manage its growth | 19:47 |
| clarkb | we do run an intermediate registry for artifacts belonging to unmerged CI jobs that we can prune aggressively | 19:47 |
| fungi | (without taking it offline periodically) | 19:47 |
| clarkb | glance allows me as a cloud user and operator to delete things | 19:47 |
| clarkb | existing use of that thing will fall off slowly but in the meantime I am able to signal to the system that this is no longer valid pleaase stop wasting resources on it | 19:48 |
| opendevreview | Tim Burke proposed openstack/election master: Tim Burke candidacy for Swift PTL (Flamingo) https://review.opendev.org/c/openstack/election/+/942138 | 19:48 |
| clarkb | there is no way of doing that with many container registries including the canonical implementation unless you take costly downtime | 19:48 |
| TheJulia | oh, yeah, absolutely | 19:48 |
| TheJulia | Totally agree, that is a huge downside to container registries | 19:49 |
| TheJulia | I think there is a mechanism to have a lifetime on an OCI image actually to force expiration | 19:49 |
| TheJulia | but... it is likely server specific | 19:49 |
| JayF | FWIW; I'm not saying in this case we actually should fully replace one tool with another; but instead we should just have that mindset of "does service X need to be an openstack service" | 20:49 |
| JayF | even if something different isn't ideal; it might be better to have something well-supported outside that meets 95% of our needs instead of trying to do it all in-house | 20:50 |
| fungi | this was the discussion we had around the openstack apps site project, which really should have extended better to murano too: plenty of other people outside openstack had already worked out push-button solutions for deploying canned appliations into clouds, we didn't need to waste time maintaining a half-baked alternative of our own | 21:10 |
| fungi | it was under-maintained, then abandoned, then an attractive nuisance encouraging end users to deploy vulnerable outdated applications because they didn't know better, and it reflected very poorly on our community | 21:12 |
| fungi | i'm not saying glance is in that situation, but if someone else has solved the same problems as one of our own projects more thoroughly and is doing a better job of maintaining their thing, maybe we don't need to keep investing time in ours (we could even invest time in helping them with theirs instead) | 21:14 |
| JayF | fungi: the glance/container registry thing was 100% a straw man that was used to demonstrate | 21:55 |
| fungi | yeah, the openstack apps site is a non-straw example we can draw from our history | 21:55 |
| JayF | (that being said; I do think there would be value in nova support for oci container registries as a glance alternative; but I'm not going to suggest that too hard until I know if my downstream would change their usage pattern) | 21:55 |
| fungi | when we started the apps site project, there wasn't really a good solution yet. but a bunch of others cropped up and got traction quickly outside our community, sapping developer interest in the nascent solution we had | 21:57 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!