19:00:26 <clarkb> #startmeeting infra
19:00:26 <opendevmeet> Meeting started Tue Jun  3 19:00:26 2025 UTC and is due to finish in 60 minutes.  The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:26 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:26 <opendevmeet> The meeting name has been set to 'infra'
19:00:34 <clarkb> #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/HZE5Z7QO6FJ4FFFH5DXQAUSRC5HYOWEK/ Our Agenda
19:00:45 <clarkb> This agenda is suspciciously similar to the one we had last week
19:01:10 <clarkb> I'll admit that I ended up having a number of non infra distractions and also ran into problems with ubuntu mirrors that interrupted my goal of booting new servers
19:01:20 <clarkb> But we can dive in and figure out what updates there are
19:01:25 <clarkb> #topic Announcements
19:01:52 <clarkb> I will be attempting to help my brother move tomorrow so will be in and out and afk randomly
19:02:14 <corvus> you can borrow my truck
19:02:34 <clarkb> I suspect both the typical start and end of my day will be the time I'm most connected. Please ping me on Matrix if anything needs urgent atetntion. I won't be far from home and can return if necessary
19:03:15 <clarkb> corvus: I'd have to start driving now and not sleep tonight to get it back in time
19:03:20 <clarkb> but I appreciate the offer :)
19:03:27 <corvus> :)
19:03:37 <clarkb> Was there anything else to announce?
19:04:43 <clarkb> #topic Zuul-launcher image builds
19:04:48 <clarkb> I think we can probably dive right in then
19:04:55 <corvus> some updates here!
19:05:01 <corvus> i recently observed a zuul-launcher bug in opendev, fixed in: https://review.opendev.org/951683
19:05:30 <corvus> then there's a stack of changes that's almost merged to expose nodeset request info through the web ui, and also allow us to change node state and delete nodeset requests through the api
19:05:38 <clarkb> "zero division error" that laws of math must not be flouted
19:05:59 <corvus> i have a todo to expose the same feature set through zuul-client for those of us who don't use keycloak
19:06:35 <corvus> those are the only things currently on my "punch list" before i'm comfortable saying we should start switching other tenants, including openstack
19:06:41 <clarkb> does the web ui exposure that is near landing allow authenticated manipulation already?
19:06:49 <corvus> (i will also propose running a second launcher)
19:07:20 <corvus> yes, so those of us with keycloak accounts will immediately be able to click buttons in the web ui and make stuff happen
19:07:35 <corvus> only thing unwritten is "zuul-client delete-nodeset-request", etc
19:07:52 <clarkb> ack. More motivation to get a keycloak account configured if you haven't yet
19:08:04 <corvus> yeah, it's a much nicer experience anyway :)
19:08:22 <clarkb> frickler has also started experimenting with adding debian trixie nodes. And I think this exposes the power of zuul-launcher/nodepool in zuul
19:08:36 <corvus> ++
19:08:41 <clarkb> the changes to dib that enable building trixie are able to be speculatively gated in zuul-providers with all of our elements
19:08:49 <clarkb> and not just the minimal elements that dib tests itself against
19:09:18 <clarkb> a good illustration of some of the motivation behind these changes (there are other motiviations too)
19:09:30 <corvus> certainly a big one
19:10:08 <clarkb> anything else zuul-launcher/nodepool in zuul related?
19:10:39 <corvus> nope, just expect me to propose expanded use soon :)
19:11:06 <clarkb> exciting
19:11:12 <clarkb> #topic Gerrit shutdown problems
19:11:51 <clarkb> Last week we basically said lets land https://review.opendev.org/c/opendev/system-config/+/950595 then do a restart using sighup to see if we hit the timeout. Then the time after that go back to using sigint to see if shorter h2 compaction time helps
19:12:05 <clarkb> unfortunately I've not been in a good place where I can focus on Gerrit properly to do that
19:12:22 <clarkb> Looking ahead at my week I hesitate to say I can do this Friday but that may be doable.
19:12:29 <clarkb> so thats my goal we'll see if I can hit it
19:12:43 <fungi> i can try to help with it too
19:12:50 <clarkb> that change has the reviews I need too. its just waiting on my +A for when I can monitor so nothing really necessary from anyone else until we're ready to restart it
19:12:53 <clarkb> fungi: thanks!
19:13:15 <clarkb> #topic Gerrit 3.11 Upgrade Planning
19:13:26 <clarkb> similarly I haven't been looking any closer at the Gerrit upgrade recently
19:13:32 <clarkb> #link https://www.gerritcodereview.com/3.11.html
19:13:47 <clarkb> still a good idea for those with time to look over the release notes here and add any notes or concerns on our planning document
19:14:02 <clarkb> #link https://etherpad.opendev.org/p/gerrit-upgrade-3.11 Planning Document for the eventual Upgrade
19:14:30 <clarkb> Assuming the signal handling related restart(s) happen this week i should be able to pick this back up again next week maybe.
19:14:45 <clarkb> #topic Upgrading old servers
19:14:55 <clarkb> As mentioned at the start of the meeting I did start poking at this a bit last week
19:15:12 <clarkb> I landed some updates to the mirror update server and zookeeper servers that allow them to run on old and nwe ubuntu
19:15:50 <clarkb> my plan was to follow that up with booting new servers but then ubuntu mirrors were having issues and I didn't want to boot new servers that launch-node deletes due to an inability to install packages (we run some minimal ansible and also do a full system package update after the first boot)
19:16:15 <clarkb> but once I have some breathing room I'd like to start on that again as I think ubuntu mirrors are happier now ( at least I haven't seen failures recently)
19:16:46 <clarkb> pretty sure there are no refstack cleanup updates but chime in if I'm wrong
19:17:52 <clarkb> Any other server upgrades notes?
19:19:06 <fungi> (narrator: there were not)
19:19:15 <clarkb> #topic Working through our TODO list
19:19:19 <clarkb> #link https://etherpad.opendev.org/p/opendev-january-2025-meetup
19:19:36 <clarkb> just our weekly reminder and a call out to any interested contributors that this etherpad has a good high level overview of stuff that needs doing
19:19:48 <clarkb> I'm happy to help provide more detail and assist anyone looking into tackling items on that list
19:19:56 <clarkb> #topic OFTC Matrix bridge no longer supporting new users
19:20:46 <clarkb> as mentioned last week things got better but its clearly not a service that gets fixes urgently. We've had a bit of time to noodle on this more. Do we think we'd like to proceed with running an opendev matrix room as our primary synchronous comms point?
19:21:03 <clarkb> I don't think I got an PMs or emails for feedback but if I missed them I apologize
19:21:31 <fungi> i'm indifferent but dont wish to block the wheels of progress (nor be ground beneath them)
19:21:38 <clarkb> One thing I thought I'd mention is the openstack operators group is experimenting with a matrix room that fungi set up for them
19:21:51 <clarkb> they might be good canaries for how able people who aren't use are able to make use of a matrix room?
19:22:08 <clarkb> though we have some feedback on that via zuul and there aren't major concerns there so may be overkill to wait for feedback from the openstack operators
19:22:51 <clarkb> I'll admit part of my willingness to defer a bit here is simply how full my plate is suddenly with other stuff and I'm not sure Id be able to care and feed for a new channel and potential problems associated with that properly
19:23:05 <clarkb> I'm hoping all of that eases up in a week or two though
19:23:09 <corvus> i'm happy to help in any way
19:23:20 <corvus> not that i'm not busy too, but i am volunteering :)
19:23:59 <clarkb> thanks. Proposal: given my hopes that next week will be better and we have volunteers. Maybe plan to create the new room next week and actively drive people on irc #opendev to that location?
19:24:14 <corvus> and work on some announcements, messaging?
19:24:19 <clarkb> ++
19:24:41 <corvus> sgtm
19:24:52 <clarkb> maybe the order of operations is write an announcement and update the irc channel topic? create the room, start using the room and actively push people that direction
19:25:34 * clarkb scribbles some notes so that ends up on the todo list
19:25:55 <corvus> yeah.  to avoid confusion, i'd probably want to try to set a cutover timeframe for clarity
19:26:34 <clarkb> maybe next week is get ducks in a row time (write an announcement, communicate in general, create the room) then the week after is the switch in comms location?
19:26:53 <corvus> ++
19:28:04 <clarkb> sounds like a plan. Anything else on thsi topic?
19:29:03 <clarkb> #topic Enabling hashtags globally
19:29:14 <clarkb> #link https://lists.opendev.org/archives/list/service-announce@lists.opendev.org/thread/SVXT6X4WRYE6XQPB5PTWFUKUATICLO56/ Announcement that this is happening today
19:29:22 <clarkb> #link https://review.opendev.org/c/opendev/system-config/+/951601 Docs update to reflect the update being made manually
19:29:42 <clarkb> 951601 captures the change that corvus has volunteered to make in all-projects meta config
19:30:05 <clarkb> it lgtm and I think we can proceed with the actual change whenever convenient. But if others are concerend please review the docs change asap
19:30:35 <corvus> happy to update it to match topic as clark mentioned... lmk
19:30:55 <clarkb> I don't think we need to since the existing acls do it the way you've done it
19:31:10 <clarkb> (so we know that will work, it was just odd to me that they would be in different locations when I first looked)
19:31:39 <clarkb> oh and ya my comment on that change links to followup cleanups that we can make to remove redundant configs from project-config/gerrit/acls
19:32:28 <corvus> since we said we'd do that today...  do we wait for more +2s or do i +w it?
19:32:57 <clarkb> corvus: maybe give people until 20:00 since I think we'll finish the meeting early today and that gives anyone time to quickly reivew it they want to
19:33:05 <clarkb> and if not I'm good with proceeding at that point
19:33:08 <corvus> ++
19:33:52 <clarkb> anything else on gerrit hash tags?
19:34:12 <fungi> sorry, reviewing now!
19:34:47 <clarkb> #topic Adding CentOS 10 Stream Support to Glean, DIB, and Nodepool
19:34:50 <fungi> lgtm
19:34:57 <clarkb> I don't have any personal updates on this topic
19:35:15 <clarkb> I think the glean change is ready for review though
19:35:23 <clarkb> #link https://review.opendev.org/c/opendev/glean/+/941672 Glean updates for centos 10 stream ready for review
19:35:35 <clarkb> then I saw tonyb pushed changes to test dib without nodepool
19:36:06 <clarkb> https://review.opendev.org/c/openstack/diskimage-builder/+/949942 it is marked wip and doesn't pass yet but progress is being made on that side of the effort
19:37:11 <clarkb> tonyb when you are awake maybe let us know when that change is ready for initial review
19:37:49 <clarkb> but ya if we get glean updated, dib testing updated, then we can land the centos 10 stream updates that use specific cpu options
19:37:56 <tonyb> I will let you know.   I expect it'll be tomorrow at this rate
19:38:04 <clarkb> then we can have a disussion over whether/how we're going boot those images in opendev
19:38:06 <clarkb> tonyb: ack
19:38:16 <clarkb> Anything else on this topic?
19:39:17 <clarkb> #topic Open Discussion
19:40:53 <clarkb> June 13 is the open infra summit cfp deadline
19:41:11 <fungi> just a heads up that we've got an assortment of changes landing soon for openstack and starlingx to switch from enforcing the old openinfra icla to signed-off-by for the dco
19:41:15 <clarkb> I think they will be opening up forum session submissions too
19:41:31 <clarkb> fungi: those are scheduled to land July 1?
19:41:43 <fungi> the final one yes
19:41:58 <fungi> tooling changes and acl cleanup hopefully far sooner
19:42:16 <clarkb> ++ I know I reviewed some of them. Let me know if my reviews have gone stale for updates and I can rereview
19:42:17 <fungi> and starlingx may switch sooner than openstack too
19:42:59 <clarkb> also I discovered that something like 15 yaers ago spearce and an android dev decided that any matching signed-of-by was sufficient signed-of-by checking in case anyone gets questions later
19:43:29 <clarkb> (basically you oinly need one of author, committer, person making the change in gerrit to have a signed-of-by)
19:43:37 <clarkb> and I can't spell off
19:44:26 <clarkb> ok last call. Otherwise I'll end the meeting in a minute or two
19:45:58 <clarkb> thanks everyone! We'll be back here same time and location next week. Until then thanks again for all your help working on opendev
19:46:02 <clarkb> #endmeeting