18:00:52 <gouthamr> #startmeeting tc
18:00:52 <opendevmeet> Meeting started Tue Mar 25 18:00:52 2025 UTC and is due to finish in 60 minutes.  The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:52 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:52 <opendevmeet> The meeting name has been set to 'tc'
18:01:03 <gouthamr> Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.
18:01:09 <gouthamr> Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee
18:01:11 <gouthamr> #topic Roll Call
18:01:18 <bauzas> \o
18:01:19 <spotz[m]> o/
18:01:35 <gtema> o/
18:01:56 <frickler> \o
18:01:58 <noonedeadpunk> o/
18:02:18 <mnasiadka> o/
18:03:35 <gouthamr> courtesy-ping gmann cardoe
18:03:44 <gmann> o/
18:05:49 <gouthamr> alright, its that magical time of 18:05 UTC, lets get started..
18:05:57 <gouthamr> #topic Last Week's AIs
18:07:05 <gouthamr> we had a couple of things:
18:07:05 <gouthamr> 1) take operator engagement concerns to the PTG
18:07:20 <gouthamr> we added a topic to the etherpad:
18:07:23 <gouthamr> #link https://etherpad.opendev.org/p/apr2025-ptg-os-operators
18:07:49 <bauzas> +1
18:07:51 * gouthamr hopes that's the etherpad we'd use :) either case, save the date if you can attend:
18:07:51 <gouthamr> 17 UTC on Friday April 11
18:08:18 <gouthamr> 2) TC meeting time poll
18:08:52 <gouthamr> #link https://framadate.org/os-tc-2025-2
18:09:23 <gouthamr> cardoe and spotz[m] haven't weighed in here
18:09:45 <spotz[m]> Dang though I had hang on
18:10:08 <gouthamr> okay, we can visit this topic at the end of this meeting
18:10:21 <gouthamr> that's all the AIs i was tracking, was anyone else working on anything?
18:12:14 <spotz[m]> Ok looking at my current meeting schedule it's bad:( But might be different after Kubecon
18:12:43 <gouthamr> #topic PTG Planning
18:12:43 <gouthamr> ^ a reminder to add topics to the etherpad here:
18:12:54 <gouthamr> #link https://etherpad.opendev.org/p/apr2025-ptg-os-tc (OpenStack Technical Committee vPTG etherpad)
18:13:48 <cardoe> sorry got stuck on a call.
18:14:57 <gouthamr> i'll slot these into specific times next week.. i'm hoping to get a split of topics that are good for the community to attend/participate in, and regular business where we'd take whatever participation we can get
18:15:31 <gouthamr> ack cardoe, please do fill out https://framadate.org/os-tc-2025-2
18:15:40 <cardoe> doing it now
18:15:41 <spotz[m]> I'll be PTO that week
18:15:57 <gouthamr> ++ i've noted that about you and mnasiadka
18:16:36 <gouthamr> please feel free to add topics nevertheless if you think one of us can seed the discussion
18:16:49 <mnasiadka> Yeah, I'll be in NZST timezone that week
18:20:00 <gouthamr> #topic A check on gate health
18:20:16 <gouthamr> any CI updates to share this week?
18:20:33 <frickler> ubuntu kernel bug breaking jobs in neutron and kolla
18:20:41 <clarkb> the same issue that hit jammy in december
18:20:57 <frickler> we reverted to an old noble image and stopped rebuilds, so we are fine for now
18:21:23 <clarkb> and setuptools 78 rolled out breaking changes that broke many people though impact to openstack seemed minimal. They rolled back the change and now there is much discussion in python land about how to move forward
18:21:39 <clarkb> it is a good reminder that our packages should convert -'s in metadata names to _'s though
18:21:56 <gouthamr> ah, ty for both these updates..
18:21:57 <mnasiadka> We managed to do it in Kolla before they did a revert, so maybe they did succeed ;-)
18:23:05 <noonedeadpunk> just seen a couple of timeouts last week which were not there for quite some time
18:24:00 <fungi> the centos 9 mirror was broken (sync'd an inconsistent state from an upstream mirror) for a few hours yesterday too
18:25:45 <gmann> one thing to update, devstack/grenade/tempest setup for new stable/2025.1 and current master is almost done. main setting are merged but a few more things are in gate
18:25:49 <gmann> #link https://review.opendev.org/q/topic:%22qa-2025-1-release%22
18:27:38 <gouthamr> clarkb: my very quick search on codesearch.o.o shows me that all openstack setup.cfg files are fixed up, there is some boilerplate/tests/examples that need to be addressed.. i see lots of fixes possible for the non openstack/ though: like, https://opendev.org/zuul/zuul-jobs/src/branch/master/setup.cfg
18:28:00 <clarkb> gouthamr: ya an in theory pbr is doing the conversion for us but then setuptools did its own validation again and exploded
18:28:14 <clarkb> which is like we tried to do the right thing the easiest way possible and they broke us anyweay
18:28:56 <clarkb> gouthamr: within openstack I guess the problems were all in dependencies
18:29:40 <gouthamr> ++
18:30:28 <fungi> well, and the aforementioned kolla patch
18:30:34 <gouthamr> ty for all the updates, the grenade one is important, and nice to knock it off as soon as the cycle begins
18:31:07 <gouthamr> #topic TC Tracker
18:31:13 <gouthamr> #link https://etherpad.opendev.org/p/tc-2025.1-tracker (Technical Committee activity tracker - 2025.1)
18:32:19 <gouthamr> frickler++ on the war footing merges last week :D
18:33:05 <frickler> yes, sadly even more zuul config errors now, need to do some follow ups
18:33:10 <gouthamr> #link https://review.opendev.org/c/openstack/releases/+/942218 (Yoga EOL)
18:33:10 <gouthamr> #link https://review.opendev.org/c/openstack/releases/+/942201 (Xena EOL)
18:33:10 <gouthamr> #link https://review.opendev.org/c/openstack/releases/+/941458 (Wallaby EOL)
18:33:30 <gouthamr> ah.. yes, we couldn't know if we didn't start cleaning up
18:34:02 <frickler> also no progress afaict on cleaning up issues for the things we did not eol
18:34:43 <gouthamr> ack
18:35:18 <gouthamr> these are repos that have (un) maintainers.. i suppose we can narrow things down at the PTG
18:36:10 <gouthamr> i'll go down the list and seek updates, because we'll close this etherpad and create a new tracker at the PTG
18:36:18 <gouthamr> https://etherpad.opendev.org/p/tc-2025.1-tracker
18:36:35 <gouthamr> please share any updates if you'd like on items that you've been tagged with
18:37:01 <gouthamr> anything else on the tracker?
18:37:37 <gouthamr> #topic Open Discussion and Reviews
18:37:37 <gouthamr> 
18:37:48 * gouthamr copy-pastes from teh agenda
18:37:50 <gouthamr> Non-auditable process of skyline releases, ie: https://opendev.org/openstack/openstack-ansible-os_skyline/src/branch/master/tasks/skyline_install_yarn.yml#L126-L127 That is a result of building static files with yarn, but potentially it should be completely offloaded to Zuul to prevent malicious code injection during such manual patches.
18:39:39 <fungi> yeah, i recall we discussed it in #openstack-infra recently at length
18:40:35 <fungi> pep 770 will in time provide a mechanism for recording sboms as static data files shipped in sdists/wheels
18:40:53 <mnasiadka> FWIW I don't think we're building static files with yarn in kolla - but I haven't used skyline really. frickler do you have any... experience?
18:40:58 <fungi> there's a yarn plugin apparently to auto-generate cyclonedx sboms
18:41:13 <noonedeadpunk> well we do in osa
18:41:29 <fungi> but also, a short term stop-gap would be to amend the manifest to include the yarn.lock file used at build time
18:41:38 <noonedeadpunk> but the biggest problem is that they do a human made patch for the realease of the amount that is non-verifiable
18:41:40 <frickler> I never did that
18:42:14 <bauzas> I have no context either so far
18:42:27 <clarkb> noonedeadpunk: patch of what? Sorry I don't understand what is being patched
18:43:02 <noonedeadpunk> and that is actully somehow reminds me of xz being compromised in an alike way
18:43:14 <noonedeadpunk> #link https://review.opendev.org/c/openstack/skyline-console/+/945065
18:43:49 <fungi> bauzas: the larger problem is that we have openstack projects (horizon does it too) shipping embedded copies of random libraries developed outside openstack, and these are not easily inspected or tracked for updates, often falling well out of date and including known vulnerabilities, which our users of those files are not notified about in any way
18:43:50 <noonedeadpunk> so they do prepare it for releasing skyline-console so that it was containing the rightfully built content
18:43:53 <clarkb> oh they are committing the build artifacts into the repo. they shouldn't do that either way
18:44:10 <noonedeadpunk> yup...
18:44:16 <clarkb> butthen I agree that is the same sort of vector used by xz. Use opaque gzip data as the transport layetr
18:44:50 <fungi> yeah, the more narrow problem in skyline is that they're committing compiled versions of those libs into git, not even doing it automated at build time
18:45:08 <gmann> If i am recalling correctly but isn't that one of the things to check when skyline project status changed from emerging to active projects ?
18:45:29 <bauzas> I see, a security attack vector indeed
18:45:39 <noonedeadpunk> So sorry if I mislead by original description
18:45:59 <noonedeadpunk> no I think we totally missed the process
18:47:04 <gouthamr> gmann: not the same issue: https://review.opendev.org/c/openstack/governance/+/924109/comments/510391ea_9cf4bc38
18:47:55 <gmann> gouthamr: I mean we missed to check this in that change. I think that was one of the thing we discussed to take it as emerging project and not active
18:48:21 <gmann> and one of the few things they should solve before becoming the Active project
18:48:42 <bauzas> looks important indeed
18:48:53 <frickler> well we made it active, didn't we?
18:50:20 <gmann> yes, we made it active
18:50:21 <noonedeadpunk> I think we did
18:51:05 <gmann> this is good email thread I found where fungi mentioned all points for skyline team to solve
18:51:07 <gmann> #link https://lists.openstack.org/pipermail/openstack-discuss/2021-December/026254.html
18:51:54 <spotz[m]> cardoe: I know you all are using Skyline, is this something you all could possible help with?
18:52:15 <spotz[m]> My thought maybe they just need help and guidance to resolve this
18:52:19 <cardoe> I've really wanted our folks to get involved.
18:55:02 <fungi> it's wholly possible i missed things though, i had limited available time to audit the state of their projects
18:55:08 <noonedeadpunk> so I guess it's a question now on how we should proceed with this, given that project was made active
18:55:30 <noonedeadpunk> as apparently this is a case for TC to step in a way
18:55:31 <clarkb> step 0 might be trying to reproduce what was built
18:55:34 <bauzas> Should we signal it ?
18:55:38 <fungi> looks like i didn't bring up any of the javascript content at all
18:55:46 <clarkb> if that chceks out then the risk is probably low and they can work to fix in the next cycle
18:55:54 <clarkb> if that doesn't check out then you have bigger questions
18:56:00 <bauzas> like a disclosure
18:57:54 <frickler> pretty likely yarn builds are not reproducible bit-by-bit like when deps got updated, what then?
18:58:14 <clarkb> frickler: they should have a lockfile and the diff should probably be minimal if using the same version of the lock?
18:58:24 <clarkb> I mean its effort and I'm not signing up myself for this. But I think it is one path forward
18:58:44 <mnasiadka> Looking at the brief list of items from this Dec 2021 thread - shouldn't there be a resolution that this is the framework that all projects need to comply with? (briefly the list that fungi mentioned there and probably some more)
18:59:31 <fungi> we have a list:
18:59:37 <fungi> #link https://governance.openstack.org/tc/reference/new-projects-requirements.html Requirements for new OpenStack Project applications
18:59:44 <fungi> but it could certainly stand to be improved
19:01:04 <gouthamr> we're at the hour, but we can close out with this topic
19:01:19 <bauzas> please yeah
19:01:34 <gouthamr> can someone take a stab at bringing this issue to the ML?
19:02:40 <gouthamr> we've struggled to get conversations going with skyline contributors on the ML/IRC, but, i can't think of a better way to have a public discussion on something that's not a code change
19:03:21 <gouthamr> we'd do this also to bring attention to deployers/distros and operators apart from the contributors
19:03:31 <fungi> i think project leaders have reached out to them through wechat in the past
19:03:41 <noonedeadpunk> well, I can try to reproduce the process, sure, as we do perform yarn build in osa
19:03:44 <fungi> might at least be able to give them a heads up that it's being discussed
19:04:04 <fungi> (and where, in case they want to participate in the discussion)
19:04:23 <gouthamr> we can alert them wherever to come respond to the ML :D
19:05:12 <gmann> I think language is also one of the challenge for them to be less active on ML
19:05:27 <gmann> at least to read a lengthy emails or so
19:06:45 <gouthamr> yes, we need this to be broken down into problem and suggestion to be helpful.. i think we've identified problems with them in the past, and they don't know what they'd do to fix it? or they may not understand why they should care..
19:07:04 <bauzas> language is a barrier for many of us :-)
19:08:03 <gouthamr> alright, 7 minutes over, don't mean to keep us on this.. let me end the meeting so we can chat async about this
19:08:10 <gouthamr> thank you all for attending
19:08:14 <gouthamr> #endmeeting