Tuesday, 2026-03-17

gouthamrtc-members: a reminder that there's no 0800 UTC meeting today, we'll be catching up at 1700 UTC06:27
mnasiadkayay06:29
elodillesgouthamr: hi, about which deliverables haven't merged any patch since latest release, here is a list: https://paste.opendev.org/show/bFgORyYfUF5yifps7uYW/08:08
elodilles(though, 2 of them is falsly listed: glance-store and openstack-ansible-roles. but the others should be correct)08:09
elodilles(the command is in the openstack/releases repository)08:09
gouthamrtc-members: a gentle reminder that our weekly irc meeting will be held here in ~50 minutes16:09
gouthamrelodilles: late ack, ty! that also probably explains the lack of acks from the barbican release managers/ptl for the hsm repos: https://review.opendev.org/q/topic:%22gazpacho-rc1-deadline%22+status:merged+hsm16:59
gouthamr#startmeeting tc 17:00
opendevmeetMeeting started Tue Mar 17 17:00:57 2026 UTC and is due to finish in 60 minutes.  The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot.17:00
opendevmeetUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:00
opendevmeetThe meeting name has been set to 'tc'17:00
gouthamrWelcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct.17:01
gouthamrToday's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee17:01
gouthamr#topic Roll Call17:01
dansmitho/17:01
bauzaso/17:01
noonedeadpunko/17:02
dansmithI guess it's well-known that the ics feed does not have the meeting today/now?17:02
gouthamrdansmith: yes.. because we moved the 0800 UTC meeting to 1700 UTC and just informed folks here and via email17:02
spotz[m]o/17:03
gouthamrit was tedious to update the ics, we might end up changing the whole thing in a couple of weeks17:03
frickler\o17:03
gouthamrcourtesy-ping:  cardoe, mnasiadka17:03
fungiirc-meetings generates an ics file based on what's configured for your meeting, and serves it from meetings.opendev.org17:03
gouthamrnoted absence: t o n y b17:04
mnasiadkao/17:04
* gouthamr hasn't heard from tonyb here in a few weeks.. despite the tagging last week17:04
gouthamrfungi: yes, i'll be updating irc-meetings when we have consensus on the new meeting time.. last week, we decided to stick to Tuesdays at 1700 UTC until the end of March. 17:05
spotz[m]He was at the board meeting last week17:06
gouthamr(this meeting's the only outlier, the ICS should be fine for next week) 17:06
fungiand showed up in opendev matrix yesterday17:06
gouthamrah, ty he's been spotted somewhere 17:06
gouthamrno TC activity has been noted... 17:07
gouthamrtonyb: i don't want to tack on more than you're probably dealing with at the moment... but, we've some AIs that'll probably need re-assignment if you're away.. 17:07
gouthamra reminder that you can edit https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee and add your absence there, or just leave a note here regarding unavailability if you anticipate it.. 17:08
dansmithgouthamr: I will miss next week17:09
gouthamrdansmith: ack, ty for letting me know17:09
dansmithand the following actually17:09
gouthamr#topic Last week's AIs17:10
gouthamrdansmith: noted17:10
gouthamrwe've updated the chair/vice-chairs on the TC roster: 17:10
gouthamr#link https://governance.openstack.org/tc/#current-members17:10
gouthamrno liaison changes were needed because VMT liaisons continue from the last cycle:17:11
gouthamr#link https://wiki.openstack.org/wiki/CrossProjectLiaisons#Vulnerability_management 17:11
fungithanks!17:12
gouthamrand the rest of the DPL liaisons are in projects.yaml: 17:12
gouthamr#link https://opendev.org/openstack/governance/src/branch/master/reference/projects.yaml17:12
gouthamrspotz[m]: would you submit a change to the election repo to reset the election liaison, here's where it is:17:12
gouthamr#link https://review.opendev.org/c/openstack/election/+/964733 17:12
gouthamrthat's all about liaisoning17:13
fricklerisn't sahara retired? (re VMT)17:13
gouthamryes it is17:13
gouthamrnext are a couple of AIs regarding leaderless teams17:14
gouthamrthe election results are due tomorrow, you'll see PTL updates for nearly 40 project teams posted, we'll need a few eyes to merge them17:15
spotz[m]Yeah I’ll get that in an bit17:15
gouthamrty spotz[m] 17:16
gouthamrwe've some more to discuss around the retirement of venus and vitrage - a release concern, we'll get to that in a sec17:17
gouthamri just wanted consensus that we'll be fast-tracking the merges on the other appointment changes here: 17:17
gouthamr#link https://review.opendev.org/q/hashtag:%222026.2-leaderless%22+(status:open%20OR%20status:merged)17:17
gouthamri don't see any negative votes or concerns expressed in these gerrit changes17:17
gouthamrso, if you'd like to note your approval, or disapproval please do so asap.. once the result changes are posted, i'll rebase these and merge them based on the approvals so far17:18
gouthamrany concerns with this?17:18
noonedeadpunkthere was an email in the morning about masakari situation17:19
noonedeadpunkand I see that appointment patch is missing so far. 17:20
noonedeadpunkI think it makes sense to reply to the thread with TC hat on about proposed options17:20
gouthamrmaybe i need to clarify that the election result change will tag each of these "leaderless" project teams with an "APPOINTMENT NEEDED"  annotation. that's why a rebase is necessary, so it's just process, i don't want to delay these another week since we've had enough soak time.. 17:21
gouthamrnoonedeadpunk++17:22
gouthamr#link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/C2BQ7UAJPPEEY2Q5TKJETG7DSKGVMHXT/17:22
gouthamrtl;dr: we're being asked for advice on whether to allow a past PTL to take over the reigns temporarily while the team reforms, or if we think DPL is better 17:23
noonedeadpunktbh, I am not sure what is better17:24
dansmithnot a past PTL of that project, but past PTL in general?17:24
gouthamrah, an ex-PTL of OpenStack Blazar Project, ex-Core reviewer of OpenStack Congress Project per some social bio17:25
dansmithbut ... mentoring a new PTL, if that's really going to happen, seems like an excellent opportunity.. hate to not capitalize on that17:25
mnasiadkaI’m a bit hesitant to say that DPL for a project that has problems nominating a PTL is going to end well17:26
dansmithyeah ^17:27
noonedeadpunkwell, the main problem with nominating ptl was lack of time for all ptl activities from all interested parties17:27
noonedeadpunkso DPL kinda aims to solve that by splitting responsibilities17:28
gouthamrwe'd prefer an active contributor/maintainer to take on the role, and would welcome mentoring?17:28
mnasiadkanoonedeadpunk: in my mind - there’s a problem to find one person that will take the responsibility (he doesn’t need to do all PTL activities - he can delegate) - are you sure we’ll find three in reasonable time?17:29
noonedeadpunkit's also more if PTL has any awareness of the project, as from what I got, they are more have a general governance knowledge17:29
gmaanDPL for less active team is always risky, maybe we should remove that options from leaderless projects available options17:30
mnasiadkagmaan:+117:30
gouthamrgmaan: woah, that'd affect oslo 17:30
noonedeadpunk++17:30
gouthamr(and requirements)17:30
noonedeadpunk(and potentially more things)17:30
gmaanoslo is less active for sure but still have folks around but i do not disagree on that17:30
mnasiadkaSure, but wouldn’t it raise visibility on the problems we have as a community?17:31
gmaan++ yesh that one17:31
noonedeadpunkmasakari have folks around as well, but not really confident enough in themselves to drive the project17:31
noonedeadpunkso eh, this is so-so argument17:31
noonedeadpunkanyway17:31
fungiglance had a ptl who wasn't even a core reviewer on the project, so i don't know that it holds generally?17:31
noonedeadpunkI am not saying we should prefere DPL for masakari, it's just every approach has it's ups and downs17:32
noonedeadpunkbut indeed, I think it's simpler to get a PTL if there's somebody ready to step in17:32
noonedeadpunkI think nova has/had same17:32
dansmithcurrent nova PTL is not a core17:32
gouthamryet? 17:32
noonedeadpunkfor couple of cycles already17:33
dansmithI thought the offer was for an experienced PTL to take the administrative role and mentor someone else who might be a candidate to take over17:33
dansmithcore or not core, that seems like a very ideal situation if there is actually someone to mentor17:33
noonedeadpunkyeah, okay, agree17:33
noonedeadpunkmakes sense when you put it this way :D17:33
gouthamrhmmm, i support this given we've had success with this approach in other teams17:34
gouthamrnoonedeadpunk: would you be able to summarize this response then? 17:35
noonedeadpunkyep, will do17:35
gouthamrty noonedeadpunk 17:35
noonedeadpunkabout my AI for vitrage and venus retirement17:36
gouthamryes, let's chat about that.. 17:36
noonedeadpunkI was preparing patches, when realized - are we sure we wanna do retirement, and not deprecation?17:36
gouthamrnoonedeadpunk: "inactive" was a kind of deprecation.. the hope was we'd not publish releases17:37
gouthamrbut, we've had a hiccup there17:37
noonedeadpunkas retirement would limit us in proposing any security patches17:37
gouthamr#link https://review.opendev.org/q/topic:%22gazpacho-rc1-deadline%22+status:open17:37
fungideprecation would be fine though17:37
noonedeadpunkdeprecation also prevents projects from releasing17:38
fungiyou only need security fixes on maintained stable branches once deprecated17:38
noonedeadpunk++17:38
gouthamr^ a release for venus and vitrage was tagged earlier in the release, and we're trying to avoid that happening with the coordinated release:17:38
gouthamr#link https://review.opendev.org/c/openstack/releases/+/980900 (Remove vitrage and venus from 2026.1 Gazpacho)17:38
gouthamris that our gap? would folks prefer seeing "inactive" project deliverables deprecated and moved to "legacy.yaml" - would that be a better indicator for the release team 17:39
noonedeadpunkso I'd say that deprecation is potentially a better option overall as we still are able to keep stable branches at least without security issues17:39
noonedeadpunkthough argument here, is that without PTL or security liason - who would be doing patching...17:39
fungiwhich is essentially what happened with murano17:40
noonedeadpunkthough with evolution of LLMs, which are trained also on openstack source code, this becomes easier to handle...17:41
gouthamrhmm, most recently, with monasca, we were able to keep the project around long enough as "inactive", and then drop all the older stable branches alongside master.. but, we won't have that option all the time. 17:42
noonedeadpunkat least I can take care of vitrage, as have an experience with it's code17:42
fungii don't buy that, unless the llm-generated security fixes are any better than the llm-generated vulnerability reports we've been receiving17:43
noonedeadpunkI don't think we should keep as "inactive"17:43
gmaanin such cases of no maintainers for stable branches then it goes to retirement than deprecation. that is not ideal but actual visibility to operators/anyone thinking/relying on fact that stable branches are maintained17:43
fungigoes to retirements *and* we loudly warn operators/users to uninstall the software due to known unfixed exploitable vulnerabilities in the software17:44
gmaanyeah17:44
noonedeadpunkas deprecation removes it from releasing, and drops content17:44
noonedeadpunkbut retirement leaves all stable branches "as-is" - isn't it?17:44
fungideprecation shouldn't prevent it from releasing on stable branches from before the deprecation (until they reach end of maintenance)17:45
gmaanretirement delete everything including stable branches. depecation only do for master content17:45
gouthamrso splitting hairs here, we have a few maintainers for vitrage, but not for venus.. so maybe propose retiring venus and deprecating vitrage?17:45
fungiwe've had stable point releases tagged on deprecated deliverables in the past anyway17:46
gouthamrbasically, be explicit that the approach is motivated by availability of maintainers17:47
noonedeadpunkso basically that was the concern I wanted to raise17:47
noonedeadpunkI'm fine to proceed with any option we agree on17:47
noonedeadpunkbut I think deprecation is a more user-friendly optiopn17:48
noonedeadpunkwhich also allows more easily to pick up maintenance if needed17:48
noonedeadpunkwhile makes state of affairs well-infromed by dropping master branch content17:48
gouthamrnoonedeadpunk: ack. we should let openstack-discuss know, so if there are users in the wild that are interested to keep the lights on, they can step up.. 17:49
gouthamrdoes anyone have strong opinions on this, besides noonedeadpunk? 17:50
JayFI am +1 to the proposed plan.17:50
spotz[m]+117:51
gouthamralright.. ty this was good to sort out. 17:52
gouthamrI'll add my +1 to the release team's proposal here: 17:52
gouthamrhttps://review.opendev.org/c/openstack/releases/+/98090017:52
gouthamryou should too.. 17:52
noonedeadpunkwell, for instance, I can step in for stable branches of vitrage, but not master branches or keeping project active17:53
gouthamrack noonedeadpunk 17:53
noonedeadpunkI don't think these projects should be released anyway :)17:54
gouthamrty, i think we've covered all of the election/leaderless concerns17:55
gouthamranything else to talk about today wrt these?17:55
gouthamrwe're jumping a bit on the agenda, but, that was my bad on the ordering of the topics17:55
gouthamrty for taking all these AIs noonedeadpunk 17:56
gouthamr#topic PTG17:56
gouthamrI've made some reservations and the etherpad is ready for your topics:17:56
gouthamr#link https://etherpad.opendev.org/p/apr2026-ptg-os-tc (TC PTG Etherpad)17:56
gouthamrwe may not need 6 hours, but, i'll adjust that closer to the event.. but please do let me know right away if you have objections17:56
gouthamri'm hoping to discuss community goals again, and discuss new ones (nox? pqc? :D) *ducks* 17:57
spotz[m]I’m the only one who discusses ducks!17:58
gouthamrso if you're a goal champion, or know someone else who is.. please be prepared to do a rundown on where we are, and what else we need to do to.. 17:58
gouthamrthat's all i had about that, really.. 17:59
gouthamrspotz[m]: i need some RDO ducks so i can sell them on ebay in 10 years17:59
gouthamr#topic A check on gate health17:59
spotz[m]There will be some for RH Summit18:00
gouthamrwe don't have time for open discussion today, i know this has been a bad few weeks for us on CI.. just wanted to take a moment to thank folks for the firefighting18:00
gouthamrthe websites going down repeatedly is affecting CI, developers, users and operators. There's no denying that.. we've had very little chatter about that on the ML.. but, if you're plugged into IRC or opendev, you kinda know what opendev folks have been dealing with18:01
spotz[m]FYI I’ll be at Kubecon next week if anyone needs me to talk to anyone let me know18:02
gouthamrwe're past the hour.. but, if you'd like to add anything further to the minutes, please do so now18:02
gouthamralright, thank you all for participating18:04
gouthamrdon't let the discussion stop.. 18:04
gouthamr#endmeeting18:04
opendevmeetMeeting ended Tue Mar 17 18:04:28 2026 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:04
opendevmeetMinutes:        https://meetings.opendev.org/meetings/tc/2026/tc.2026-03-17-17.00.html18:04
opendevmeetMinutes (text): https://meetings.opendev.org/meetings/tc/2026/tc.2026-03-17-17.00.txt18:04
opendevmeetLog:            https://meetings.opendev.org/meetings/tc/2026/tc.2026-03-17-17.00.log.html18:04
gouthamrfungi: clarkb: i'm wondering if you were going to post on the MLs on the gate situation? lots of people are confused and just "recheck"-ing, some people are chiming on #openstack-infra and matrix18:06
gouthamrhttps://www.reddit.com/r/openstack/comments/1ruupac/opesntack_docs_down/18:06
gouthamrand elsewhere ^18:06
clarkbwe have been responding to the people who have talked to us. I will not be posting to reddir18:07
clarkb*reddit18:07
gouthamrofc not18:07
fungii think the people on reddit know where to find us if they want to help out instead of just complain18:07
clarkbthe tl;dr is we've expanded the size of the server and moved docs.openstack.org to a dedicated host18:12
gouthamrah, ty.. curious if you're seeing any success with these:18:13
gouthamr#link https://review.opendev.org/q/hashtag:%22apache-waf%22+(status:open%20OR%20status:merged)18:13
fungiwe've been using some of that functionality to track the ip addresses of abusers and block them temporarily, yes18:14
clarkbthis seems to have helped but hasn't made the problem go away. I've recently identified a new behavior that I think may be contributing and have a change up to try and improve it too18:14
fungialso it increases the memory use on the server18:14
fungihttps://opendev.org/opendev/system-config/src/commit/fb38bb1/playbooks/roles/static/files/50-docs.openstack.org.conf#L21-L4418:16
fungithat's the current ruleset for docs.openstack.org18:16
dansmithgouthamr: added my upcoming absences to the wiki18:17
dansmithyou need not remove me from the ping list on those days, I'll be far away from the computer that dings :)18:17
gouthamrty noted dansmith 18:17
dansmithfungi: clarkb: have you considered the proof-of-work project that sits in front? or are you hoping to just deal with the traffic because you want it scrape-able?18:18
gouthamrty fungi clarkb.. is the new behavior patch this one? https://review.opendev.org/c/opendev/system-config/+/98100918:18
gouthamri think they are: https://review.opendev.org/c/opendev/system-config/+/980993 18:19
fungidansmith: you mean have we thought about the obvious while dealing with these various swarms for about a year? ;)18:19
dansmithouch18:20
fungigouthamr: 981009 is going to see if we free up resources on the servers a little more effectively, apache ends up with a lot of workers hung on old requests which may be a symptom of other timeouts18:20
gouthamr++18:21
fungidansmith: but yeah, anubis, haphash, iocane, et cetera... we've been trying to balance tools that get in the way of users with outages that get in the way of users18:21
dansmithack, I hadn't seen any evidence of them when I've loaded the page so I wasn't sure... didn't mean to insult of course18:22
fungiup until this past week-ish it's not really been an issue, we've dealt with the problems through excluding requests with obviously made-up user agents, instructing non-abusive crawlers where to focus/how to go easy on us, etc18:22
opendevreviewMerged openstack/openstack-manuals master: Change tenant_network_types to project_network_types  https://review.opendev.org/c/openstack/openstack-manuals/+/97783318:22
dansmithI've been fighting it personally for at least a year at this point18:23
clarkbdansmith: https://review.opendev.org/c/opendev/system-config/+/980993 fwiw I put this together earlier as well18:23
fungithe latest incident is a flood of requests from hundreds of thousands of client ip addresses, mostly in mobile provider networks all over the world, trying made-up urls brute-forcing to see if they can find previously unindexed content18:23
dansmithclarkb: ack18:24
fungii'm pretty sure it's a massive bot army made up of backdoored phones18:24
dansmithclarkb: even cloudflare's "I am under attack" human check seems to be no match18:24
fungiyeah, a cdn would be no use in this case because every url they're trying is unique (made of of constructed permutations of path components from nearby urls)18:25
dansmithfungi: well, a CDN that attempts to block bots would help, if it could actually do that18:25
fungimaybe cloudflare's bot detector would flag them before ever letting those requests through, but i feel like this is organized crime trying to bypass cloudflare's mitigations18:26
fungiand also we'd end up paying for cloudflare and handing over the keys for our hosting to them and all of our user ip addresses and browsing behaviors with it18:26
fungiand finally admitting that it's no longer possible to operate a website on the internet without paying the knee-cappers protection money18:28
dansmithyep, I resisted having to do that for a long time, but ran out of options.. and it turns out, they're good but not quite as good as I had hoped18:28
dansmithbut yep, http getting to be like mail18:28
funginothing ever is quite as good as we hope18:29
JayFdansmith: clarkb: fungi: fwiw I'm working with my downstream to try and get someone pointed at opendev to help implement anubis or a similar tech as well; I have no idea whatsoever how successful I likely am to be18:45
JayFwe have a pretty big devops team at ISC (Insight Softmax is the consultancy that powers GR-OSS) and I'm hoping to peel a person or two away to help18:46
fungii'm assuming that's not isc.org (the internet systems consortium)18:47
JayFhttps://insightsoftmax.com/ insight softmax consulting18:47
JayF(If you've heard about Arctos Alliance recently forming a community around Apache Aaron, that's "us" too)18:48
JayF**Arrow18:48
funginot familiar with it, but sounds cool. i know the other isc from bind and dhcpd fame18:50
JayFand from kea infamy18:50
JayF:( 18:50
fungiand also isc created my preferred permissive open source license18:50
JayFI tend to use MIT, which is functionally equivalent aiui18:51
JayFand allows me to not worry if I put that code in gentoo (gpl) or here (apache 2)18:51
fungii assume you mean "mit/expat" ;)18:52
fungi(there's more than one mit license, after all)18:52
JayFhttps://codeberg.org/JayF/jaypardy/src/branch/master/LICENSE this one lol18:53
JayFI probably should use MIT0 or BSD0 but meh18:53
JayF(I know licenses by gentoo identifiers; the 0 is just the no-notice variations)18:54
fungiyeah, debian likes to differentiate between the "expat" license from mit (mit/expat) and the "x11" license from mit (mit/x) and the no-attribution license (mit-0)18:57
fungiisc's license is functionally equivalent to 2-clause bsd but was written before berkeley made a 2-clause version18:57
*** vhari_ is now known as vhari19:39
opendevreviewAmy Marrich proposed openstack/election master: Update TC Laison  https://review.opendev.org/c/openstack/election/+/98104020:53
opendevreviewMerged openstack/openstack-manuals master: Warn crawlers not to touch the tripwire  https://review.opendev.org/c/openstack/openstack-manuals/+/97811522:53
cardoeSorry folks. The sudden freeze messed up some plumbing and I had to deal with that today which is why I missed the TC.23:08
gouthamrah sorry to hear that cardoe23:34

Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!