19:01:14 #startmeeting infra 19:01:14 Meeting started Tue Feb 22 19:01:14 2022 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:14 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:14 The meeting name has been set to 'infra' 19:01:21 #link http://lists.opendev.org/pipermail/service-discuss/2022-February/000322.html Our Agenda 19:01:26 #topic Announcements 19:01:46 The service coordinator nomination period ended last week and I was the only one who was crazy enough to jump on it :) 19:03:03 #topic Actions from last meeting 19:03:07 #link http://eavesdrop.openstack.org/meetings/infra/2022/infra.2022-02-15-19.01.txt minutes from last meeting 19:03:12 #link https://review.opendev.org/c/opendev/system-config/+/829882 Restore gerrit mergeability checking 19:03:33 frickler: got that chagne pushed up. I asked about doing some additional testing but after talkign to gerrit upstream over slack about it I think it is probably safe to proceed as is if we prefer 19:03:52 basically the behavior we should expect is that changes that are modified after the config update will get mergability checked. Changes that aren't updated won't have that info 19:05:34 Then we'll need to monitor for overall load induced by the chagne. THough we don't expect it to be a problem 19:05:40 #topic Topics 19:05:47 #topic Improving OpenDev's CD throughput 19:06:00 I didn't update this topic to reflect the gpg encryption of logs work, but I think that is realyl close now? 19:06:45 yes, after a few mis-steps in templates it worked :) 19:06:47 https://zuul.opendev.org/t/openstack/build/307b0e0e097c48daa959ee3b960f7ff5/logs 19:07:10 what I plan to do now is write up a change with some documentation, and to switch it to apply to all prod jobs 19:07:13 cool. I guess infra-root should push up changes with their keys? 19:07:35 yep, can either do that now to test if you like, or after i write up some docs 19:08:15 I'm not in a rush 19:08:19 same 19:08:55 Anything else on this topic? 19:09:11 i'm hopeful it can help us get some more people interested in helping with prod jobs ... build it and they will come :) that's the theory anyway :) 19:09:35 if nothing else I think it continues to show how we can make use of zuul to help us operate things 19:09:39 nope ... although waiting for the various prod jobs to finish has increased my interest in finishing this parallel work! 19:09:50 oh, there were a few comments on the spec @ 19:10:18 #link https://review.opendev.org/c/opendev/infra-specs/+/821645 19:10:36 something to read and ponder 19:10:39 that's all, thanks 19:11:15 #topic Container maintenance 19:11:44 No real news on this other than I actually set time aside for looking at this later today with jentoio. No concrete time, but sometime this afternoon :) Hoping to start making progress on this 19:12:20 #topic Spring Cleaning of Old Reviews 19:12:28 #link https://review.opendev.org/q/topic:retirement+status:open Changes to retire all unused the repos. 19:12:44 This is where I'm at on that. I got all the repos running noop jobs and then pushed up retirement changes for them all. 19:13:04 If they still look retireable please approve and land those changes. Then I'll followup with a change that removes them from zuul and marks them retired in projects.yaml 19:13:12 And we can abandon changes at that point too 19:14:11 abandoning should be done before approving the acl update, just a reminder 19:14:13 Please ping me somehow if you find something that shouldn't be retired that has been going through the process. I'll need to update some of my notes and rollback some updates 19:14:19 fungi: ++ good point 19:15:28 #topic Gitea 1.16.1 19:15:35 #link https://review.opendev.org/c/opendev/system-config/+/828184 Change to upgrade to 1.16.1 when we are ready 19:15:39 #link https://104.130.74.7:3081/opendev/system-config Test site via held node here 19:15:50 I think this is likely landable, but I was really hoping to get many eyes on this 19:16:01 Our testing should represent good coverage but the large changelog scares me :) 19:16:12 If you have time to take a look it is much appreciated 19:16:41 Cross checking the changelog against our expectations is likely a good idea too (in addition to general functioanltiy via the test site) 19:17:29 #topic Gerrit Gitea links 19:17:40 Hey this happened. Thank you to everyone who kept pushing it along 19:17:55 thank you for the heavy lifting fixing gerrit upstream! 19:17:55 This required quite a bit more tweaking than I would've expected 19:18:01 clarkb: heh, just on the previous, i'm seeing that jinja thing agains system-config again 19:18:38 ianw: interesting. If I had to guess they reindex things periodically (maybe as a new change) and depending on how complete it is per repo maybe yo uget back weird results. That may be worth asking upstream about 19:18:51 we can try filing an issue with them in github I guess 19:18:52 as part of the gitea linking work, we aldo disabled gitiles links in the gerrit webui (because they're effectively redundant) 19:19:03 s/aldo/also/ 19:19:28 ya I think at this point we're mostly waiting for anyone to notice and if they notice hopefully without issues :) 19:19:28 or, maybe it's just ... right - https://imgur.com/a/Yi79MSx 19:19:41 i couldn't find any way to disable the gitiles plug-in because it's "core" to gerrit, but i have a change proposed to forbid access to /plugins/gitiles as a location in our apache proxy layer 19:19:49 ianw: heh it is a lot of nasible. But the flip flopping is curious 19:22:19 #topic Rocky Linux 19:22:47 Just a heads up that rocky images continue to be a work in progress. The most recent thing was we needed epel to install haveged 19:23:14 I updated dib's epel role to support rocky so that we can get that in our builds and ianw made a dib release. We've updated nodepool to include that dib release and are just waiting for images to build now 19:23:34 I think we're stuck behind all the other image builds right now so may be a bit. Might be nice to be able to prioritize a build in nodepool but I don't think that is currently possible 19:24:24 yesterday i almost deleted the centos-8 wheel afs volumes, but then i paused thinking rocky could use them 19:24:34 can't even cancel the other builds to move it up the queue, i don't think 19:24:41 but then i realised it is better to start them fresh anyway 19:24:58 if there is actually interest in that, because they have a lot of old stuff in them 19:25:15 from before fungi updated us to only build wheels that aren't upstream, etc. 19:25:36 so, unless there's objections, my plan is still to delete those volumes 19:25:45 no objection from me 19:26:01 if things are signed the signatures are likely different too? 19:26:10 oh wheels ya those aren't signed 19:26:13 sorry I had in my head rpms 19:26:32 But ya I agree starting over is probably better 19:26:42 and I think rocky will be a good test case for mirroring less stuff and seeing how it goes 19:28:11 #topic Open Discussion 19:28:21 That was what we had on the agenda. Anything else? 19:28:40 I'm working on fixing Gerrit's server-sig-algs ssh key exchange extension support 19:28:52 Eventually Gerrit will be able to do modern rsa :) 19:29:16 i've got openstack's release artifact signing key proposed for the zed cycle 19:29:31 if anyone wants to update it with additional attestations, you can find it here: 19:29:43 #link https://review.opendev.org/829933 Publish the Zed Cycle signing key for future use 19:30:22 the process for exporting the signed key with your added signature is in a comment in the index.rst file at the top of the list of key entries 19:30:32 https://gerrit-review.googlesource.com/c/gerrit/+/331019 just pushed the gerrit fix for kex server sig algs 19:30:43 awesome, thanks! 19:30:57 i assume best case that will only be available in 3.5.something? 19:31:11 seems like they're not backporting the mina-sshd update 19:32:15 I think 3.6 19:32:15 clarkb's gitea stats are going to start flipping to "java" soon :) 19:32:34 ianw: its ok gitea stopped syncing the gerrit repo :) 19:32:47 well we stopped syncing the gerrit repo so gerrit stopped pushing updates to gitea :) 19:33:09 did we ever retire that repo? should we? 19:33:23 I thought about retiring it but keeping it for now seems fine 19:33:45 I think my main concern is that we might need ti again for some reason. Though we have a good way to carry patches more distro packaging like in system-config if necessary 19:33:46 i find it tends to come up with higher priority in searches than the actual gerrit repos 19:34:01 that could be problematic 19:34:28 i should add for esoteric things where the only reference is in the actual code 19:35:02 or changelog 19:36:27 ianw: as you can probably tell I've just ended up with a local checkout :/ 19:36:55 worth noting you need bazel 5.0 for gerrit master and I had to do a whole dance for that and ended up with a 5GB docker image in the process 19:37:32 yikes 19:38:15 reminds me of when i tried to `tox -e docs` in zuul/zuul recently and it died with enospc after consuming some 4gb of disk 19:39:30 ya similar situation. Turns out that build stuff is big 19:39:39 Anyway sounds like that may be all for the meeting? 19:39:45 thank you everyone. We'll see you here next week 19:39:53 ++ thanks for hosting clarkb! 19:40:00 But then the week after I'll likely have to skip due to other engagements 19:40:01 thanks clarkb! 19:40:08 #endmeeting