19:01:19 <clarkb> #startmeeting infra 19:01:19 <opendevmeet> Meeting started Tue Apr 5 19:01:19 2022 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:19 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:19 <opendevmeet> The meeting name has been set to 'infra' 19:01:28 <ianw> o/ 19:01:31 <clarkb> #link https://lists.opendev.org/pipermail/service-discuss/2022-April/000330.html Our Agenda 19:01:39 <clarkb> #topic Announcements 19:02:12 <clarkb> As noted earlier the PTG is happening right now and all week. I've ended up in more PTG sessions than anticipated which probably means I anticipated poorly :) but so far decently productive so yay 19:03:16 <clarkb> I expect to continue to participat ein PTG thing sthrough the week. There will be discussion about ci resources and translation tools and more that I'm forgetting about 19:04:02 <clarkb> One thing I noticed was that the openstack TC discussion with project PTLs ended up having some feedback for OpenDev. I found it a bit odd that we've tried to have time for that every ptg until this one and largely been ignored so we skipped it this time around and then a different venue gets found 19:04:27 <clarkb> If anyone has ideas on how we might solicit that feedback better I'm all ears. (we have the mailing list and our irc channel and previously the ptg office hour sessions) 19:05:18 <clarkb> #topic Actions from last meeting 19:05:28 <clarkb> #link http://eavesdrop.openstack.org/meetings/infra/2022/infra.2022-03-29-19.01.txt minutes from last meeting 19:05:42 <clarkb> ianw has an action to clean up our gerrit fork repo's branches and retire the repo 19:05:52 <clarkb> Looks like thta happened? 19:05:55 <ianw> that is done, yep 19:06:04 <clarkb> #link https://opendev.org/opendev/gerrit has been retired 19:06:25 <clarkb> thank you! 19:06:47 <clarkb> #topic Topics 19:06:53 <clarkb> #topic Improving OpenDev CD Throughput 19:07:26 <clarkb> I did want to call out Zuul's latest spec: https://review.opendev.org/c/zuul/zuul/+/834198 in relation to this 19:08:03 <clarkb> In particular I think we need to consider the spec for managing bridge more directly in zuul within the contetx of ^ and determine if we need to make any edits to the spec or potentially change our approach in general 19:09:32 <clarkb> don't need to sort that out in this meeitng, but maybe after we've all had some time to think it over a meeting/call/thread to discuss it would be a good idea? 19:09:57 <clarkb> essentially a more explicit followup for us to consider the potential changing risk there 19:11:02 <clarkb> anything else to add to this topic? 19:11:31 <ianw> are you thinking things like publishing jobs too? 19:12:23 <fungi> mainly the question of whether zuul's container-based isolation alone is sufficient security to trust deployment on our production servers to 19:12:27 <clarkb> right 19:12:36 <clarkb> in a shared zuul deployment 19:13:24 <fungi> noting that this question is merely triggerd by the discussion of the ansible module spec, it's the situation we're already in and effectively have been since we started doing cd with zuul 19:13:40 <ianw> right, i'm just thinking, a bit like "reflections on trusting trust", we're pulling zuul images that have been published via that "level" of isolation 19:13:56 <clarkb> ianw: yup that is good to call out too. 19:13:58 <fungi> and gerrit images, and so on 19:14:45 <clarkb> in my mind it was a good reminderfor us to take stock of the security stance and determine if anything needs adjusting 19:15:07 <clarkb> but I think that deserves its own conversation and if others agree I can work to schedule something forthat 19:15:40 <ianw> right, so ultimately it feels like our consideration is to (continue) trusting publish/deploy pipelines in general, or not, just because we ultimately end up there 19:16:16 <clarkb> yup that makes sense 19:16:38 <clarkb> anyway maybe think it over for the next week and at our netx meeting I can schedule discussion on this if we want to do that (or start a thread) 19:17:03 <ianw> ++ 19:17:06 <fungi> i guess the reason not to discuss it on the ml is that we don't want to unnecessarily shake users' faith in our systems management toolchain? 19:17:41 <ianw> heh yes, or write out a "here's a great way to compromise us, now give it a go" instructions :) 19:18:00 <clarkb> I also think synchronous comms can be useful when it come sto this sort of thing as a sort of brainstorming exercise 19:18:10 <clarkb> I find that asynchronous comms tend to be less candid 19:18:22 <clarkb> maybe that has to do more with the recording aspect though 19:20:11 <clarkb> Basically take some time to think about it so we're aware of the related issues, then meet together synchronously to have candid discussion/brainstorming where silly ideas don't filtered out by editing an email 19:20:33 <ianw> sounds good 19:21:03 <clarkb> #topic Container Maintenance 19:21:40 <clarkb> I don't see any new changes on this yet. I think jentoio is busy though. Feel free to ping me if/when you manage to look at this again 19:22:09 <jentoio> yeah, still working on it 19:22:48 <clarkb> #topic Cleaning up old reviews 19:23:09 <clarkb> #link https://review.opendev.org/q/project:opendev/system-config+status:open+topic:system-config-cleanup resurrected reviews that should be easy to land that need review 19:23:20 <clarkb> would be great if we could try to review a few of these and land them 19:23:50 <clarkb> fungi: on https://review.opendev.org/c/opendev/system-config/+/654040 it isn't clear to me if you want thos efiles to be removed in that chagne or if we can land that change then do a followup to delete unused files 19:24:04 <clarkb> mostly the lack of a value vote making me wonder which you would prefer there 19:24:51 <fungi> i was +/-0 on the change, as almost everything it's updating should just be deleted instead 19:24:59 <fungi> (possibly everything it's updating) 19:25:21 <clarkb> everything but the readme file update I think 19:25:30 <clarkb> oh and prep-apply.sh 19:25:33 <clarkb> so half the chagne would stick around 19:25:43 <fungi> do we use prep-apply.sh any longer? 19:25:53 <clarkb> yes in our puppet noop apply tests 19:26:00 <fungi> i wasn't sure if that only got called when launching new servers. okay 19:26:03 <clarkb> which will exist until we remove the puppet apply testing I think 19:26:30 <fungi> got it. i thought we used ansible to install it 19:26:53 <fungi> to install/configure puppet i mean 19:27:26 <clarkb> we do, if you read that script it uses ansible 19:27:35 <clarkb> but ya that script is still used by the noop apply tests iirc 19:27:52 <fungi> anyway, i figured if people concurred on deleting some of those files, i could just push a revision to remove any which aren't needed, or a separate deletion change as a parent of that 19:28:17 <fungi> and if there was any input on other things we should also delete from the repo, add those too 19:28:50 <clarkb> ok I can push a new ps that removes email_stats.py and maintain-infra-groups.py 19:28:58 <clarkb> then maybe do a separate followup for any additional files to remove 19:29:22 <clarkb> the problem I'm trying to avoid is makign each of these super perfect since they have been ignored for so long. Better to land what is good enough then followup I think 19:29:33 <clarkb> otherwise we may go another 3 years not merging anything :) 19:29:36 <fungi> that's fair, easier to just approve this one for now 19:30:35 <clarkb> Some of the other changes under that topic may need more scrutiny lik ethe gerrit config updates 19:30:53 <clarkb> but I'll try to keep up with the reviews on them regardless of who the original owner is. I can be a proxy owner 19:31:09 <clarkb> #topic Retiring the Gerrit Repo 19:31:18 <clarkb> I think we can say "DONE" and move on 19:31:35 <clarkb> #topic Scheduling project renames 19:32:03 <clarkb> the school let me know that the 15th is a day off for the kids which I susppose I should've expected. I don't think it is that big of a deal to continue to do the rename on that day for me though 19:32:31 <clarkb> if there are no objections at this point I should send email about the plan and make the scheduled time official 19:32:39 <clarkb> fungi: ^ does this day still work for you? 19:32:44 <fungi> yep 19:33:15 <clarkb> cool let me make a note here to send email about that soon (probably tomorrow at this rate). Is there a good time of day? 19:33:29 <clarkb> I think I can do anything after like 1500 UTC 19:34:02 <fungi> that works for me 19:34:23 <fungi> unless later is better for others who want to help 19:35:33 <clarkb> cool I can send email for 1500-1600 on the 15th then we can always adjust from there if necessary 19:35:41 <clarkb> #topic Open Discussion 19:35:48 <ianw> i can't guarantee much for my 16th as that's school holidays here 19:35:54 <clarkb> ianw: ya no worries 19:36:03 <clarkb> I'm good with it as long as we have two people around 19:36:22 <clarkb> and then we can schedule the gerrit upgrade on another day. (which I keep meaning to start catcing up on) 19:36:37 <clarkb> Note we updated the udev rule for glean to fix centos 9 boots in rax 19:36:53 <clarkb> that seems to have worked and I havne't seen any unexpected fallout which is always a concern when modifying boot processes 19:38:12 <ianw> thanks for that. i sent a kernel patch in to mark those interfaces as "enumerated", which is a flag to systemd that basically they should be renamed 19:38:49 <ianw> the chance of that 1) getting merged, 2) making its way back into centos 8 to get the interfaces renamed there rounds down to about 0% i think 19:39:00 <ianw> but still, hopefully going forward it may not flip-flop again 19:39:25 <clarkb> is that a backport of an existing fix since centos 9 doesn't have that behavior? 19:39:37 <clarkb> or are we working by chance? 19:40:41 <ianw> i think that centos 9 systemd/udev decides to rename them anyway; despite their /sys flag not marking them as "unstably" named 19:40:47 <clarkb> ah 19:41:01 <ianw> i couldn't find a smoking gun change, probably could by git bisecting systemd and booting over and over 19:41:31 <ianw> but who knows, maybe something happens and the behaviour flips back again, unclear if it was even intentional 19:41:45 <clarkb> got it 19:44:13 <clarkb> Sounds like that may be it for this week. I'll give it a few more minutes before ending 19:46:41 <fungi> i've just sent e-mail messages to service-announce and openstack-discuss about the new -d with depends-on behavior 19:46:50 <fungi> er, -2 with depends-on 19:48:56 <clarkb> thank you for putting that together 19:51:24 <clarkb> #endmeeting