19:01:10 #startmeeting infra 19:01:10 Meeting started Tue Jun 27 19:01:10 2023 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:10 The meeting name has been set to 'infra' 19:01:17 #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/77NQ27ARPLEVBLT7ZSUVDWER6YYACCXN/ Our Agenda 19:01:20 #topic Announcements 19:02:22 Next tuesday, July 4, is a US holiday. I won't be able to attend this meeting on that day as I'll either be traveling (you might have noticed my timezone hasn't shifted and that is because I caught covid and shift all my travel back by two weeks), or celebrating the holiday wondering how much longer it will take to get over covid 19:03:41 I suspect that others will also be afk so we can probably cancel that meeting 19:04:39 #topic Bastion Host Changes 19:04:52 #link https://review.opendev.org/q/topic:bridge-backups This set of changes still needs additonal reviewers 19:05:02 #topic Mailman 3 19:05:22 I suspect fungi isn't able to join today and I also doubt much new has happened on this one due to the summit and travel etc 19:05:33 fungi: please let us know if these assumptions are bad 19:06:35 #topic Gerrit Updates 19:06:56 #link https://review.opendev.org/c/opendev/system-config/+/885317 Update Gerrit 3.8 image to final 3.8 release 19:07:16 This is a bit of a bookkeeping change to bump up gerrit versions. We were previously testing with a hacked up gerrit 3.8 rc build 19:07:23 #link https://review.opendev.org/c/opendev/system-config/+/884779 Revert bind mounts for Gerrit plugin data 19:07:40 And this undoes the bind mounting of plugin data so that we clear out the leaked replication task files when we down/up the container 19:07:57 I continue to think this isn't urgent but the longer we put it off the more files there will be to clean up in the end 19:08:26 #topic Upgrading Servers 19:08:46 tonyb's efforts to make the ci registry more testable have resulted in better testing and a new host replacing the old one. THank you for that ! 19:09:23 The old host still needs to be removed https://review.opendev.org/c/opendev/system-config/+/887001 but we can queue that up behind the zuul executor replacements 19:09:36 since they all update the inventory file it the prod job queues can get fairly long 19:09:56 once that chagne lands we can delete the old server too. The new server is in production though due to dns updates and the rest of this is lower priority 19:10:08 Thanks again for the help and happy to help work through more of these with interested volunteers 19:10:16 #topic Fedora Cleanup 19:10:29 I haven't seen any new movement on this and don't think we have tonyb here today 19:10:56 However, it is probably worth noting that frickler expanding afs volumes to move the logjam out of the way for bookworm 19:11:04 thank you for pushing that forward frickler 19:11:49 #topic Quo vadis Storyboard 19:12:16 There was discussion in #openstack-infra ealier today about making projects inactive possibling removing those projects from search lists making it difficult to find the project IDs to perform cleanup activities 19:12:23 around for a moment, assumptions about mm3 are spot on, unfortunately :( 19:12:37 I don't think we understand all the details here, but probably worth clarifying with projects as we mark them inactive if this is the case 19:13:02 so that they are aware any potential cleanup actions they want ot take may be more complicated (though it doesn't seem like any of the actions were prevented just more difficult to find identifiers) 19:13:17 we can also look up sb project ids with a db query if the api isn't sufficient in that case 19:14:38 (but does require a root sysadmin) 19:15:02 ack, considering this is he first time it has come up that may be good enough 19:15:09 doesn't seem to be a common occurrence in any case 19:15:18 #topic Gitea 1.20 19:15:27 Gitea 1.20 exists in RC form now (up to rc2) 19:15:46 I've pushed a change to start testing things, but we don't have a proper change log yet so this is unlikely to be comprehensive 19:15:51 #link https://review.opendev.org/c/opendev/system-config/+/886993 19:16:24 The "good" news is that this has exposed a problem in the oauth2 implementation where despite us disabling it it insists a JWT_SECRET value be set then tries to write it back to disk which fails because we don't allow gitea to write to its config file 19:16:29 #link https://github.com/go-gitea/gitea/issues/25377 19:16:52 looks like upstream may treat this as a bug, but they say the undocumented behavior of gitea writing back new config to its own config file is intended and won't be changed 19:17:05 people really need to stop designing software this way. Makes it impossible to use ocnfiguration management sanely 19:17:36 Poking around at it I think the only real issue there is the JWT_SECRET though and the other normalization I did doesn't appera to be required. It doesn't result in failed writes back to disk at least 19:18:10 also super unsafe 19:18:35 doesn't allow privilege separation between write access to configs and processes for the service 19:19:26 ya I think we should continue to set up perms on that file os that gitea can't write back to it then rely on our ci system to ensure we're configuring things properly 19:19:31 aka make it fail if it wants to write back 19:19:42 #topic Etherpad 1.9.1 19:19:49 #link https://review.opendev.org/c/opendev/system-config/+/887006 19:20:21 There is a new etherpad release. However I'm not sure we want to upgrade at this point... Number lists don't increment the number values. They are always just 1. 1. 1. etc apparently (though I've not tested this myself) 19:20:45 Also the tagged commit for 1.9.1 doesn't actually build. So 887006 checks out a commit a few commits later after they fixed the lock file contents so that it can build 19:20:54 i guess this is per a bug report somewhere? 19:21:01 I'm hoping we get a 1.9.2 that fixes the build issue at least and then we can consider building that 19:21:05 🎶...one is the onliest number that you'll ever know...🎶 19:21:11 fungi: yes there is an issue for the numbered list thing 19:23:40 I've got github release event alerts enabled for both gitea and etherpad so hopefully we see new releases with fixes for things 19:23:55 unitl then be aware of the change and feel free to review them. But we probably won't upgrade until a few things get fixed 19:24:00 #topic Open Discussion 19:24:02 Anything else? 19:24:10 re server upgrades: i've resumed the attempt to update the zuul-executors to jammy 19:24:58 (after fungi's fix to the apt config which is what caused the previous rollback) 19:25:18 er, to be clear, the incorrect apt config caused the rollback; fungi fixed it. :) 19:25:18 not caused it :) 19:25:50 just a reminder that i'm not around much this week, popping in from time to time as i get a moment (like now obviously). i'll be my usual self again next week 19:26:23 be whoever you want to be, i say 19:27:42 sage advice 19:28:32 I'll be around but also trying to take it easy. I'm still testing positive for covid as of today 19:28:53 hope you and the fam feel better soon 19:29:12 ++ 19:29:21 and in theory I'm traveling a week from now. That should be long enough to shake this off 19:29:34 and then timezones will be weird. But for this week I'm around 19:29:48 i'll be home and caught up by the time you're travelling, so that should work out 19:30:35 sounds good 19:30:58 I don't need to take up any more of our time today. Thank you everyone! We'll likely skip next week (I won't be here anyway) and then try to pick this back up July 11 19:31:01 #endmeeting