| *** mikal3 is now known as mikal | 07:08 | |
| opendevreview | Artem Goncharov proposed openstack/governance master: Add Rust use-case analysis for Keystone https://review.opendev.org/c/openstack/governance/+/998652 | 10:48 |
|---|---|---|
| tkajinam | I wonder if we can write something automated to ask fixing Co-Authored-By: <some ai tools> in commit messages... | 14:27 |
| fungi | a flake8 plugin, or add a check to hacking maybe? | 14:28 |
| *** vhari_ is now known as vhari | 14:30 | |
| tkajinam | I'm unsure if these works for commit messages | 14:30 |
| gouthamr | a git commit hook that can reject "noreply" addresses perhaps? | 14:31 |
| fungi | i'm actually surprised hacking doesn't have any commit message checks already, but yeah i guess it's unaware of the git commit context around the state of the code it's being run on | 14:33 |
| fungi | commit hooks are a bit of a pain to distribute | 14:33 |
| fungi | hopefully introduction of guidance in agents.md for tools to avoid adding a co-authored-by entry for themselves will help? | 14:38 |
| tkajinam | yeah. that hopefully helps | 14:39 |
| tkajinam | then I don't have to post 30+ comments "please don't use co-authored-by for AI. AI can't author a commit" | 14:39 |
| tkajinam | git commit hook may be an option but that requires updating every single repo I guess ? | 14:40 |
| tkajinam | (though we need to update each repo anyway, even when we add agents.md | 14:40 |
| fungi | commit hooks aren't normally pulled by cloning | 14:41 |
| fungi | for example the one git-review relies on is retrieved with a separate download (or was originally, now we ship a copy of the hook file inside newer git-review versions because it doesn't really change) | 14:42 |
| tkajinam | you mean it's no longer downloaded from gerrit but is installed locally in advance ? | 14:44 |
| fungi | yes, well "in advance" being the first time you run git review within a repository anyway | 14:44 |
| fungi | same time it used to fetch it from gerrit via scp | 14:45 |
| tkajinam | ah, ok. now I understand | 14:45 |
| tkajinam | so it's installed only once unless you remove the local clone and re-clone. hmm | 14:45 |
| fungi | yes | 14:46 |
| fungi | or if you manually delete the hook script and then run git review again | 14:46 |
| tkajinam | ok. | 14:47 |
| tkajinam | can we add additional validation in gerrit, like the one which requires signed-off tag ? | 14:48 |
| fungi | not really no, that's check is a hard-coded feature | 14:48 |
| tkajinam | ok. that's what I thought | 14:48 |
| tkajinam | having some commit message validation mechanism may be useful, though. it also allows us to catch unwraped commit message line exceeding 300 characters ... annoying in git log | 14:50 |
| tkajinam | For now I just hope agents.md helps but will do some bot-ish work to post that template (to make sure that the author understands our policy) :-P | 14:51 |
| JayF | fungi: can the same thing that rejects on no signoff also reject on co-authored-by [email in known LLMs list] | 14:52 |
| tkajinam | JayF, I think the answer is no. signoff detection is a hard-coded check and we can't use it for generic pattern check | 14:53 |
| tkajinam | if I interpreted what fungi said correctly | 14:53 |
| fungi | JayF: that's literally what tkajinam asked a few lines up | 14:53 |
| fungi | the signed-off-by check is a hard-coded feature not an extensible check framework | 14:54 |
| JayF | what app does that check? gitea? | 14:55 |
| fungi | gerrit | 14:56 |
| JayF | Ah. Was trying to see if it was a small enough project to have a decent chance at getting it as an added feature. Something like git-review would be a lot more accessible to hack a thing into :)\ | 14:57 |
| fungi | https://opendev.org/openstack/project-config/src/branch/master/gerrit/acls/openstack/meta-config.config#L25 | 14:57 |
| fungi | it's a project config option | 14:58 |
| ChristianBerendt[m] | We added a DCO check in our zuul-jobs repository to be able to check for it without using Gerrit: https://github.com/osism/zuul-jobs/blob/main/roles/dcolicense/tasks/main.yml. Maybe something similar can be used for Co-authored-by | 14:59 |
| fungi | yes, zuul can do basically anything, the question is where we would want to do that (i was originally suggesting in hacking or some separate flake8 plugin since we already run jobs with those) | 15:03 |
| fungi | adding a completely separate zuul job just to check commit messages might be overkill, but i guess if we stuck it in one of our standard project-templates it wouldn't require updating the projects themselves regardless | 15:04 |
| fungi | and could run entirely on an executor without needing a node assignment | 15:05 |
| clarkb | fungi: flake8 had commit message checks and they were a scourge | 15:42 |
| clarkb | remember when it demanded you add a '.' to the end of the usbject? or maybe it demanded you not add a '.' | 15:42 |
| fungi | it wanted to not have a trailing . | 15:42 |
| clarkb | anyway I think that created pushback against trying to check things that are subjective in the commit message (this issue is less subjective) | 15:42 |
| fungi | yeah, it's doable and there are a bunch of different places we could inject a check, just a question of which is the most effective, flexible and unn-annoying | 15:44 |
| opendevreview | Elod Illes proposed openstack/openstack-manuals master: [tool] Fix for build-all-rst.sh local run https://review.opendev.org/c/openstack/openstack-manuals/+/1011501 | 15:45 |
| fungi | getting gerrit to reject it at push with an error isn't feasible, if we integrated it into a common linter (plugin) already used by most openstack projects then it would at least allow developers to catch the problem before pushing while also preventing zuul from merging the change in cases where they didn't bother to run checks locally first | 15:47 |
| gouthamr | tc-members: a gentle reminder that our weekly IRC meeting will be hosted here in ~35 minutes | 16:24 |
| gouthamr | my reminders get shorter each week :D sorry, ETOOMANYTHINGS | 16:24 |
| gouthamr | agenda is here: https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee | 16:25 |
| spotz[m] | Gonna make some breakfast and will just eat at the computer:) | 16:44 |
| gouthamr | #startmeeting tc | 17:00 |
| opendevmeet | Meeting started Tue Oct 20 17:00:11 2026 UTC and is due to finish in 60 minutes. The chair is gouthamr. Information about MeetBot at http://wiki.debian.org/MeetBot. | 17:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 17:00 |
| opendevmeet | The meeting name has been set to 'tc' | 17:00 |
| gouthamr | Welcome to the weekly meeting of the OpenStack Technical Committee. A reminder that this meeting is held under the OpenInfra Code of Conduct available at https://openinfra.dev/legal/code-of-conduct. | 17:00 |
| gouthamr | Today's meeting agenda can be found at https://wiki.openstack.org/wiki/Meetings/TechnicalCommittee#Agenda | 17:00 |
| gouthamr | #topic Roll Call | 17:00 |
| noonedeadpunk | o/ | 17:00 |
| ChristianBerendt[m] | o/ | 17:00 |
| frickler | \o | 17:00 |
| gouthamr | Courtesy ping: cardoe mnasiadka dansmith gtema | 17:01 |
| gouthamr | think spotz[m] is grabbing a scone at her desk | 17:01 |
| gtema | o/ | 17:02 |
| dansmith | o/ | 17:02 |
| spotz[m] | o/ eggs | 17:03 |
| * gouthamr ^ is more health conscious than moi | 17:04 | |
| gouthamr | let's get started.. | 17:04 |
| gouthamr | thank you for joining.. I think the first couple of items can be combined | 17:05 |
| gouthamr | we're coming off of the PTG week, so summarizing that would get us to action items | 17:05 |
| gouthamr | #topic PTG follow up | 17:05 |
| gouthamr | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/A2J2EF52JNFKKVCYJK42JUES43PZ4IR6/ (TC PTG Summary) | 17:07 |
| gouthamr | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/2T5ETJFOXPCIM6XLLS7DWHAQGELOTCGN/ (agentic-workflows Summary) | 17:07 |
| gouthamr | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/XQ4FTA4SPILIUKPEWLKYIS262Q2OKPZ2/ (CEWG PTG Summary) | 17:07 |
| gouthamr | these summaries were posted over the weekend and yesterday, think we'll need some time to digest these and make progress on the action items that we've not started on yet.. . | 17:08 |
| gouthamr | however, i'd welcome anyone adding more to the summary captured there.. this could be your own reflection, or refuting my oversimplification or supposed understanding of the consensus where we had some | 17:09 |
| mnasiadka | Sorry, forgot to update that I’m on sick leave this week | 17:09 |
| gouthamr | mnasiadka: ah, np.. hope you feel better soon | 17:09 |
| gouthamr | as you're reading, you may notice several action items that have no owners | 17:10 |
| gtema | at least what was related to me I am pretty satisfied with the summary, thanks | 17:10 |
| gouthamr | please feel free to claim them, and add things to the TC's tracker: | 17:11 |
| gouthamr | #link https://etherpad.opendev.org/p/tc-2027.1-tracker (2027.1 TC tracker) | 17:11 |
| gouthamr | ^ this is brand new as you can tell | 17:11 |
| gouthamr | while you're at it, please also move any items from the prior tracker that are still in progress: https://etherpad.opendev.org/p/tc-2026.2-tracker | 17:13 |
| gouthamr | from the PTG takeaways, we wanted to timebox reviews on gtema's proposed resolution that contains a use case analysis for using rust: | 17:15 |
| gouthamr | #link https://review.opendev.org/c/openstack/governance/+/998652 (Add Rust use-case analysis for Keystone) | 17:15 |
| gouthamr | what time works better for folks here? would 1 week suffice for everyone to have a thumbs-up or thumbs-down vote? | 17:16 |
| gtema | jfyi, yesterday I got a request to do analysis for CRA for Keystone (IAM part) and I added my findings today to the resolution | 17:16 |
| gouthamr | ack, ty for doing that.. | 17:17 |
| gouthamr | i was going to ask fungi to share some insights here too because i'm still learning about the CRA and its implications.. | 17:17 |
| dansmith | gouthamr: 1 week is fine for me I think.. is the proper procedure to RC-1 for the "thumbs down with my TC hat on"? | 17:17 |
| gouthamr | dansmith: yes | 17:17 |
| gtema | also I wanted to ask people to consider the question as if it would come from other project. Just because Keystone is in the core should not mean Keystone is blocked from changes | 17:17 |
| gtema | also if we would say we need to rewrite it in go - all of the raised concerns would still apply while it would not be a TC question at all | 17:18 |
| fungi | gouthamr: gtema: upstream open source projects are not expected to do threat analysis for cra compliance, manufacturers placing products on the european market are responsible for that work | 17:18 |
| gouthamr | s/fungi/fungi JayF and the security SIG folks | 17:19 |
| gouthamr | my request here is to not conflate the existing proposal.. but, help us identify any blind spots during review | 17:19 |
| spotz[m] | That's why I'm in favor of the rewrite as a new project vs as keystone | 17:19 |
| gtema | fungi - you are right, but decisions of upsteam make it easier or harder for downstream | 17:19 |
| JayF | piggybacking on fungi, it's a dangerous precedent to set for us to interact with CRA compliance work directly | 17:19 |
| JayF | It sets an expectation at a project-level that we don't have resources or a charter to fulfill | 17:19 |
| dansmith | agree | 17:19 |
| dansmith | a lot. | 17:19 |
| spotz[m] | We had to add a file into the repo for OKD for CRA and I think an email for reporting but we have one of those already | 17:19 |
| fungi | i think our best course of action is to rely on feedback from our downstream vendors and redistributors in the eu who have compliance challenges | 17:20 |
| JayF | 👍 | 17:20 |
| TheJulia | ++, if anything member companies need to rally their resources to proxy appropriately and not push it entirely upon upstream. | 17:20 |
| fungi | and encourage them to participate upstream (that's really the whole point ofthe open source carve-outs in the cra, encourage companies profiting off open source to make it sustainable by helping upstreams rather than giving them more work() | 17:21 |
| gtema | spotz - it would be much much more than that | 17:21 |
| TheJulia | fungi++ | 17:21 |
| spotz[m] | Well for the initial steps:) | 17:21 |
| gtema | yeah, and so I added this argument that would make it at least to certain extend easier for downstream vendors if we use compiled language | 17:22 |
| JayF | I am a US-ian working for a non-EU company. I don't even want to be in the business of understanding EU law. I do wanna help our downstreams comply with their local policies, I just need them to engage and ask :D | 17:22 |
| gtema | jayf - I am from EU and need to deal with this and this is why I have added it to the resolution as one argument | 17:22 |
| JayF | You-the-human don't owe it to the EU; European companies productizing OpenStack do. | 17:23 |
| fungi | i find it amazing that compiled languages are easier to secure than interpreted languages (or precompiled vs jit-compiled languages really, python is a compiled language after all) | 17:23 |
| gtema | fungi - for compiled languages it is easier to prove the supply-chain control rather than dynamic | 17:23 |
| fungi | i am quite certain it's completely the reverse | 17:24 |
| fungi | we distribute source code, proving the supply chain for our source code is far easier than proving the supply chain for some precompiled opaque binary artifact | 17:24 |
| TheJulia | Perhaps not mingle the topics together? | 17:24 |
| gouthamr | hmm, yeah, worth checking the reasoning in the resolution and responding there directly | 17:25 |
| gtema | fungi - you think from the upstream pov - and it is correct. But it is not the case for downstream that delivers "binaries" | 17:26 |
| gouthamr | alright, let's try a 1-week timebox. Could everyone on the TC please get your RC votes added to this proposal by next Tuesday | 17:26 |
| gouthamr | you can abstain - but please do so explicitly | 17:26 |
| spotz[m] | link for the lazy? | 17:26 |
| gtema | gouthamr - am I supposed to vote on it? | 17:26 |
| gtema | link was above: https://review.opendev.org/c/openstack/governance/+/998652 | 17:27 |
| gouthamr | gtema: yes please | 17:27 |
| gtema | ok | 17:27 |
| fungi | i'll just note that even if we approve the use of a new language, if we expect to distribute precompiled binary artifacts for that project there is a standing resolution they'll have to comply with | 17:30 |
| fungi | #link https://governance.openstack.org/tc/resolutions/20170530-binary-artifacts.html 2017-05-30 Guidelines for Managing Releases of Binary Artifacts | 17:30 |
| gouthamr | ack ty for mentioning, fungi | 17:30 |
| gouthamr | alright, anything else for $topic.. i expect us to focus on other action items going forward, and not have a catch all "PTG Follow up" topic :) | 17:31 |
| gouthamr | was there any feedback to note? | 17:31 |
| gouthamr | OIF Staff sought feedback for the overall PTG organizing itself here: | 17:32 |
| gouthamr | #link https://etherpad.opendev.org/p/oct2026_ptg_feedback (Oct 2026 PTG Feedback) | 17:32 |
| gouthamr | please feel free to add things there, but, for ourselves, was there anything that we should be doing differently? | 17:32 |
| spotz[m] | Not PTO during PTG?:) | 17:33 |
| gouthamr | :D haha; there've got to be penalties for letting us suffer while you were at the beach | 17:34 |
| gouthamr | kidding, /me has no insight into spotz[m]'s itineraries | 17:34 |
| spotz[m] | Scotland but I did see the ocean | 17:34 |
| gouthamr | ah, yeah not there for the beach | 17:34 |
| gouthamr | i hope summaries are helpful and you feel like taking on action items and continuing the discussions even if you weren't able to make it to these sync meetings | 17:35 |
| gouthamr | let's move to the next topic: | 17:36 |
| gouthamr | #topic Closing some goals | 17:36 |
| gouthamr | through the discussions last week, we realized that two open goals would have the same sunset date: | 17:38 |
| gouthamr | #link https://governance.openstack.org/tc/goals/selected/remove-eventlet.html (Remove Eventlet from OpenStack) | 17:38 |
| gouthamr | #link https://governance.openstack.org/tc/goals/selected/migrate-to-privsep.html (Migrate from oslo.rootwrap to oslo.privsep) | 17:38 |
| gouthamr | that's because the oslo team has told us for well over a year that they don't plan to maintain rootwrap going forward | 17:38 |
| clarkb | One idea I've been noodling on is whether or not it makes sense to try and have more topic focused discussions rather than team time blocks. More similar to the design summit scheduling in the past. It feels like we always try to cram so much into a short time and then end up not being able to devote sufficient time to topics. But there are drawbacks to that s well (makes | 17:39 |
| clarkb | scheduling much much more difficult particularly if people aren't all in the same place and timezone) | 17:39 |
| clarkb | so I'm not sure if that would actually help | 17:39 |
| clarkb | any idea if privsep's memory consumption problems have ever been investigated? | 17:39 |
| clarkb | I'm not sure it is necessary to fix that before removing rootwrap but it would be a bit sad if we're switching to a system with a major known drawback and don't understand it with no alternatives | 17:40 |
| fungi | the main problem i recall related to memory consumption is that services were instantiating privsep as a fork with the entire consuming service's codebase resident in memory, so from a footprint perspective it was akin to running multiple copies of e.g. nova | 17:41 |
| gouthamr | clarkb: on the first point, worth discussing.. weve been operating with team blocks for a while now, and we've been nudging folks to do cross project discussions _somewhere_ earlier so teams can do some focussed work in their blocks the rest of the week.. that's starting to see some interesting trends: | 17:43 |
| gouthamr | 1) people seek rooms for specific discussions | 17:43 |
| gouthamr | 2) teams accommodate cross-project discussion in their schedule and folks advertise it as: "Cinder, Glance and Nova cross-project - Nova hosting" or something like that to folks via etherpads.. kinda weird for discoverability | 17:43 |
| dansmith | fungi: fork is going away in python anyway so that would probably not be a long-term thing right? | 17:43 |
| dansmith | I thought nova was actually not forking privsep to start it, but tbh I haven't looked closely in a long time | 17:44 |
| JayF | gouthamr: clarkb: One thing I've suggested before and will again: dedicated time for cross-community topics. This way it doesn't feel like taking away from project-time to have these cross-cutting sessions. | 17:44 |
| TheJulia | There is also a point where some projects need to trigger cross-project discussions or engagement, but also deferring may also be actively harmful so it might be a point of balancing needs and determining if some sort of cross-project discussion needs to happen between PTGs as well. But to JayF's point, there does need to be some time really focused for cross-cutting topics where projects should yield/enable as well. | 17:45 |
| JayF | TheJulia: honestly, you didn't say this, but repurposing our old mid-cycle meetups to a day or two of cross-project planning might be nice | 17:45 |
| TheJulia | hehe | 17:46 |
| TheJulia | I mean, it is not a bad practice to reset alignment in a team either | 17:46 |
| JayF | keeping it away from the technical back-and-forth that can dominate a PTG might be nice | 17:46 |
| TheJulia | we've sure used it as a tool over the years. | 17:46 |
| gouthamr | sync meetings come at a cost of exclusion.. so using them sparingly is good.. but rearranging things may be okay.. and worth trying | 17:48 |
| gouthamr | on privsep issues noted here, the ship has sailed, isn't it.. do we really want to reject oslo team's plan to sunset rootwrap? | 17:49 |
| gouthamr | they haven't made a formal proposal yet in governance (or updated documentation anywhere) | 17:49 |
| clarkb | gouthamr: no I'm not suggesting we reject the plan. I'm more suggesting that if this is the plan then aprt of the plan should be fixing the major issue with the tool we're using | 17:49 |
| gouthamr | their plans have been on the mailing list, in pop up team discussions and at PTGs | 17:50 |
| TheJulia | Trying to get everyone to engage also comes at a cost as well (indecision), trading perfection of agreement over action is the active choice projects need to make. That is almost, situational and entirely on context/comfort/etc. I guess what I'm trying to say is lets be cautious of trying to inherently over-optimize. | 17:50 |
| clarkb | I haven't looked at logs in ~6 months but a major source of CI instability is privsep (due to high memory ocnsumption which leads to swapping which uses up iops which makes us our own noisy neighbors) | 17:50 |
| clarkb | and I've been talking about this for I think half a decade now? | 17:50 |
| clarkb | but no one seems to care because `recheck` is easy | 17:51 |
| TheJulia | Your not wrong, but perhaps there is a better way or a different way which could be lower memory impact while still serving the base goal? That seems deeply technical though. | 17:52 |
| fungi | well, whether or not rootwrap gets retired won't change the problems with privsep. projects are already using it anyway | 17:52 |
| TheJulia | yup | 17:52 |
| fungi | so separate but related concerns | 17:53 |
| gouthamr | clarkb: any chance you can throw something on https://bugs.launchpad.net/oslo.privsep | 17:53 |
| JayF | clarkb: is there a good bug in LP for that, by any chance? Just wondering if there's a place I can look into | 17:53 |
| JayF | jinx with gouthamr lol | 17:53 |
| clarkb | there is an email I wrote years ago | 17:53 |
| clarkb | I will not file an issue | 17:53 |
| clarkb | I've already invested far too much of my time into this problem when it is a fundamental openstack problem beacuse I get asked "why did this ci job explode" | 17:54 |
| clarkb | I've provided the feedback fairly ocnsistently over years and openstack hasn't cared neough to take it from there | 17:54 |
| TheJulia | Could someone on the TC take an action item to go file the bug instead ? | 17:54 |
| clarkb | I will try to find the email in my inbox | 17:55 |
| TheJulia | that way there is a persistent bug and also the weight of the TC itself going "this is an issue" ? | 17:55 |
| dansmith | clarkb: even in today's high memory cost environment, runtime complaints about the memory footprint of privsep (or even the services as a whole) are not something I hear about a lot, FWIW | 17:55 |
| clarkb | dansmith: ack thats reaosnable feedback, but if we know it is a problem internally isn't that sufficient? | 17:55 |
| clarkb | I know that ist a bigger issue if deployments in the real world also hit the problem | 17:56 |
| dansmith | not saying it's not a concern, and I'd like them all to fit in 640kb but in the scheme of things we're asked to do, this is not one of them.. the CI stability is a major pain for sure, but it feels like it's becoming the exception | 17:56 |
| clarkb | I think the bigger problem here is there is not much about privesep's functionality that needs to be so memory inefficient | 17:56 |
| dansmith | meaning, CI is the weird memory-contrained environment that becomes less reflective of relaity | 17:56 |
| dansmith | *reality | 17:56 |
| clarkb | its a clear place where optimizations can be made but we don't and its part of the swapping we do in CI (we ran with 1gb of swap for a while and are up to 8gb by default now I think) | 17:56 |
| dansmith | clarkb: we know about lots of things we'd like to improve that nobody is interested in sponsoring :/ | 17:57 |
| clarkb | fair | 17:57 |
| fungi | and also single-node devstack jobs are going to be running multiple copies of it for each service, so efficiency issues get multiplied | 17:57 |
| dansmith | yep, which is also not reflective of reality | 17:57 |
| fungi | er, multiple copies of it, one for each service | 17:57 |
| * JayF just wanted a summarized bug to use it as fodder for nerd-sniping folks about this problem | 17:57 | |
| dansmith | TBH, I wonder if halving the number of workers and running at 16gb might yield better performance at this point.. certainly may be less frustrating for everyone | 17:58 |
| clarkb | note we cannot run nodes of that size in every cloud so it would be less than half but it is a possibility | 17:58 |
| fungi | clarkb: if you can find a copy of your ml post when you have a moment and at least let me know an approximate date it was sent, that may help me track the public version down. i've gone through 10 pages of privsep matches in the openstack-discuss archives so far and am not having much luck spotting yours | 17:59 |
| clarkb | possibility to try I mean | 17:59 |
| gouthamr | JayF: i think i found a mail thread which you sent out | 17:59 |
| gouthamr | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/PXE4AGUUBCPKR7ZQL5JFR7GQ2MISKLKI/ | 17:59 |
| dansmith | clarkb: ack, understood | 17:59 |
| clarkb | fungi: subject: '[devstack][infra] POST_FAILURE on export-devstack-journal : Export journal' mentions it | 17:59 |
| fungi | thanks | 17:59 |
| clarkb | but in passing. I'm pretty sure there was an earlier dive into this that I haven't found yet | 18:00 |
| gouthamr | yeah, but i share sentiments with you that this was probably discussed and i don't know much about the issue despite the discussions in this context | 18:00 |
| gouthamr | we're at the hour.. but i'll let this lead us to an action item before we close | 18:00 |
| fungi | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/IIT2AX3XJJGTZ2BKTUHCEKKBSUHW6BTY/ POST_FAILURE on export-devstack-journal : Export journal | 18:00 |
| gouthamr | another past investigation that resulted in some changes to privsep: | 18:01 |
| gouthamr | #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/ZPKVN5EQKVRZAAH36ITW66WIFENZS6I4/#IHQFC7BVHUNJBDDF33ESUUQVPFWJHKDH ([oslo] Parallel Privsep is Proposed for Release) | 18:01 |
| gouthamr | i can reprise this topic next week and alert the goal champion that we can coordinate the goal completion at this point | 18:02 |
| gouthamr | there's also the concern that goals like this have a bus factor of 1 | 18:02 |
| clarkb | maybe a better approach is to go down the path dansmith is alluding to: We know CI can be unstable in part due to swapping. Reevaluating what our memory consumption looks like and whether there are clear places where improvments can be made and/or giving up on smaller node sizes to find a solution is broader but maybe more aligned with the goal here(increasing CI stability) | 18:02 |
| clarkb | maybe privsep shakes out as a place tp improve in 2026 or maybe we double the node size and ~halve the node quota | 18:03 |
| dansmith | clarkb: can we pin larger jobs to larger workers and use our trim AIO jobs and unit/functional ones on the small nodes we can't make big? | 18:03 |
| clarkb | dansmith: yes | 18:03 |
| dansmith | I'd be interested in seeing what such an experiment does to our actual throughput and recheck rate, if nothing else | 18:04 |
| clarkb | we currently have does of 4gb memory, 8gb memory, and 16 gb memory | 18:04 |
| gouthamr | we probably need some help trying to identify "wastage" again.. | 18:04 |
| clarkb | so jobs can select from that set based on their expected needs. Note that 8gb will be the most widely available across clouds though | 18:05 |
| gouthamr | lets keep this discussion going.. | 18:06 |
| gouthamr | we're five minutes over, so i'll end this meeting.. | 18:06 |
| gouthamr | does anyone want to add anything else to the minutes today? | 18:06 |
| clarkb | s/does/nodes/ | 18:06 |
| gouthamr | i did have a note of absence for next week, and will be unavailable to chair... i don't know if mnasiadka or cardoe can do that instead | 18:07 |
| gouthamr | i'll let them respond when they see this | 18:07 |
| gouthamr | otherwise, we'll figure out who can! | 18:07 |
| cardoe | Apologies I was making lunch. | 18:07 |
| gouthamr | thank you all for attending, and for the discussion here.. | 18:08 |
| gouthamr | oh, crap, DST changes over the next couple of weeks | 18:08 |
| gouthamr | AH THE MISERY | 18:08 |
| clarkb | but not if you live in british columbia | 18:08 |
| * gouthamr yells into a paper bag on all our behalf | 18:09 | |
| gouthamr | #endmeeting | 18:09 |
| opendevmeet | Meeting ended Tue Oct 20 18:09:13 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 18:09 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/tc/2026/tc.2026-10-20-17.00.html | 18:09 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/tc/2026/tc.2026-10-20-17.00.txt | 18:09 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/tc/2026/tc.2026-10-20-17.00.log.html | 18:09 |
| clarkb | I don't but it will be interesting that north america's west coast will split timezones soon | 18:09 |
| clarkb | fungi: '[all][qa] Tempest jobs are swapping' this is where the concern is first raised but very high level. And I think we fixed the cinder backup problem by removing it from the default services list. It only runs when cinder tests backups now? | 18:09 |
| clarkb | but there are notes in there about looking at where the memory is being consumed | 18:10 |
| JayF | clarkb: which ones of us are not falling back? | 18:10 |
| gouthamr | only knows of gmaan | 18:12 |
| JayF | It sounds like states have passed laws to stay on DST, but feds haven't passed the equivalent law to recognize it. AFAICT I think all the west coast is moving | 18:13 |
| gouthamr | https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/message/YXVOIZV2ZSZV2MM7EVNAK2PCT4T2BAU3/ | 18:13 |
| dansmith | that has been true for years now, but it's still stuck | 18:14 |
| fungi | what's sad is if they'd just decided not to use dst and stick to standard time, the usa federal government already allows that | 18:14 |
| dansmith | BC has decided to make the change already, which is what clarkb is referring to I think | 18:14 |
| JayF | aha | 18:15 |
| JayF | BC makes sense. | 18:15 |
| dansmith | fungi: what's sad is if the politicians had ever written any calendar code ever we'd all be on the UTC-for-all plan by now | 18:18 |
| fungi | touché | 18:19 |
| gmaan | yeah, no time switch in BC from this year | 18:19 |
| gmaan | or at least this year :) | 18:24 |
| gmaan | BC Premier who got elected recently (2 years back) passed law to stay on Standard time but now called for early-election which is this month so let's see how it will go:) | 18:25 |
| TheJulia | ... great, now. I'm thinking of daylight savings time, thanks folks! ;) | 18:27 |
| clarkb | looks like dstat isn't enabled in the main nova tempest compute job? | 18:28 |
| clarkb | oh nevermind the file is named memory_tracker not dstat | 18:29 |
| clarkb | oh wait it should have both. So maybe this tooling stopped working completely | 18:30 |
| clarkb | https://zuul.opendev.org/t/openstack/build/4ab4dee88e0e4ff6ac61f59230ddb563/log/job-output.txt#10560-10575 I think it is deciding that dstat is not enabled | 18:32 |
| clarkb | https://zuul.opendev.org/t/openstack/build/4ab4dee88e0e4ff6ac61f59230ddb563/log/job-output.txt#897 but it is listed in the extra enabled services list so likely a bug | 18:33 |
| mnasiadka | gouthamr: I can | 18:33 |
| gouthamr | mnasiadka: thank you very much :) | 18:35 |
| clarkb | frickler: looks like you disabled dstat by default in https://review.opendev.org/c/openstack/devstack/+/822824 due to the pcp package install problems. That would've been pre jammy release. Do you think we can try enablibg it by default again? | 18:55 |
| clarkb | in the meantime I'll push a chagne up that enables it on a specific change | 18:56 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!