19:01:33 #startmeeting infra 19:01:33 Meeting started Tue May 3 19:01:33 2022 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:01:33 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:01:33 The meeting name has been set to 'infra' 19:01:37 o/ 19:01:40 #link https://lists.opendev.org/pipermail/service-discuss/2022-May/000334.html Our Agenda 19:01:52 o/ 19:02:06 #topic Announcements 19:02:21 There were no announcements on the agenda. 19:02:49 #topic Actions from last meeting 19:02:52 though we are now running zuul/nodepool 6.0.0 19:03:00 yup 19:03:02 #link http://eavesdrop.openstack.org/meetings/infra/2022/infra.2022-04-26-19.01.txt minutes from last meeting 19:03:09 There were no actions recorded last meeting 19:03:14 #topic Topics 19:03:25 #topic Improving OpenDev's CD Throughput 19:03:54 As mentioned Zuulv6 and nodepool v6 have happend and we are running the released shas. Our versions won't show v6 until the next restarts we do as we restarted prior to tagging 19:04:37 I'd still like to have a less formal more brainstorm like discussion wtih infra root on how this might affects us and so on. Is that something that might work for next week? 19:04:51 maybe we could do Monday UTC at the same time block as our Tuesday meeting? 19:04:58 next week works for me 19:06:05 I'd prefer something earlier, but that depends on what ianw thinks I guess 19:06:27 ya or maybe we do two chats. I can go as early as ~1530 UTC Monday 19:06:46 that'd be much better for me 19:06:55 yeah feel free to do it then, and i can catch up later 19:07:08 ok fungi ^ does 15:30 UTC work for you? 19:07:30 sure thing 19:07:50 great we can just use meetpad probably? and I can share a link then 19:08:23 Anything else on this topic or should we move on? 19:09:42 #topic Container Maintenance 19:09:58 I'm not aware of any movement on this since the base job cleanups and addition of python3.10 base images 19:10:22 jentoio isn't here today, but I haven't seen any new changes either we can probably continue on 19:10:29 #topic Spring Cleaning Old Reviews 19:10:35 #link https://review.opendev.org/q/project:opendev/system-config+status:open+topic:system-config-cleanup could use second reviews 19:11:13 fungi: frickler ^ If you have time to review those changes that would be much apprciated. I'm acting like an adopted change owner so didn't want to be the second reviewer on them. I also think that due to their age having some eyeballs on them is useful to catch things like what ianw found with the gerrit config updates 19:11:17 hey 19:11:25 jmorgan: oh hey I was looking for the wrong nick :) 19:11:36 sorry for slacking but have been onboarding new role 19:11:49 no problem. We've always got plenty going on too. I know how it goes 19:11:51 I found my old password recently 19:12:04 and I have a meeting conflict now sometimes 19:12:17 anyway, I hope to be back at it sometime soon 19:12:22 jmorgan: sounds good thanks 19:13:04 Last week we also cleaned up ELK services and their puppetry and that came with additional change abandonments. I've been meaning to take a pass at system-config proper to abandon anything ELK/health/status/subunit2sql related that just doesn't apply anymore 19:13:42 Any other change cleanup related ideas/options/etc ? 19:14:47 sounds good to me. i'm sure i have some sub 19:14:56 subunit2sql changes around that i'll clear out 19:16:10 we took down status.o.o and associated broken things like reviewday 19:16:55 also i'm in the process of retiring about half the mailing lists for the openstack mailman site (any unused for 3 years or more) 19:17:30 good points any related changes for old lists or reviewday can be abandoned too 19:18:20 so ya I'll try to make another bulk pass at that soon since there are things that can simply be abandoned due to not existing 19:18:23 #topic Support for Jammy Jellyfish 19:18:48 i hope you like jammin' too 19:18:48 We've got ubuntu mirroring in place for x86 and arm64 now 19:19:05 There are images for x86 at least, not sure about arm64 yet 19:19:34 and in some of my limited testing with zuul x86 appears to work. There are some small things I noticed like needing to specify python_version in jobs as a string not a float because 3.10 as a float is 3.1 and pythone 3.1 != 3.10 19:19:39 the reprepro issue still exists, but seems to clear up after repeated iterations 19:20:03 #link https://review.opendev.org/c/opendev/base-jobs/+/840355 Add jammy nodesets to base-jobs 19:20:11 the devstack job on jammy also works fine except for distro OVN, yatin is working on that 19:20:20 Once we're happy enough with the bootstrapped state we should probably land that change or something like it to add nodesets 19:20:42 frickler: devstack has pretty broad coverage of all things linux too so that is a good indicator it largely works 19:20:51 i expect the libzstd errors will trigger any time an affected package is pulled, which won't be all the time since reprepro will only pull changed packages on rerunning 19:21:16 As far as things that are missing goes we should add arm64 images and labels, and then we also need a wheel mirror 19:21:21 fungi: that's what it looked like to me, too 19:21:37 ianw: after thinking about the openafs packaging it probably makes sense to add jammy builds to our ppa for consistency then we don't have to special case jammy 19:21:49 for wheel the builds in general were broken, but maybe got fixed today 19:21:51 ianw: I realized that all the jobs and tools that install openafs would have to have conditions and that just seems like extra work 19:22:39 yeah i think given the history it's worth it to keep the ppa around 19:22:53 frickler: is that related to the centos 7 issue? if so I think we are really close to getting that addressed in dib 19:23:16 we've had to emergency push changes into it before, and although https://launchpad.net/~openafs is currently looking up-to-date, we've been caught with it not being before 19:23:17 clarkb: not sure, it was related to afs for c9 missing iiuc 19:24:05 yeah, the bit i missed was the publishing steps on c9 wheels was failing 19:24:20 i've updated the rpms and https://review.opendev.org/c/opendev/system-config/+/839841, which has merged now, should fix todays run 19:24:45 i'm thinking of adding installing openafs to the base wheel jobs 19:25:21 the problem is that takes quite a while to compile, and isn't useful other than for testing we didn't forget to add things like with a new distro. it's probably overkill for the amount of times we actually do that 19:25:42 s/base/test/ -- you get the idea; do it in the gate 19:26:19 ah ok so nothing that would stop jammy 19:26:25 the big jammy thing would be the ppa for openafs 19:26:49 ianw: does that require uploading a new source package entirely? Or can we just tell lp to build for jammy too? 19:26:51 yeah i can do that today, should be straight forward 19:26:56 thanks 19:27:16 I can probably write up some changes to get arm64 jammy stuff moving along too 19:27:18 i *think* you have to add the distro then re-trigger with an upload 19:27:41 i don't think we added jammy to the arm64 dib functional tests; that's probably a good start 19:27:49 ++ 19:28:55 Any other Jammy related items? We're running jobs successfully on jammy now which is great progress. Thank you frickler for getting this started 19:30:11 #topic Shutting Down Ethercalc 19:30:29 Our list of services that are not bionic or newer is very small now. One service that stands out is ethercalc 19:31:04 The ethercalc service itself doesn't seem super well maintained (last I looked they haven't been modernizing the build tooling) and the software lacks historical records for the sheets which people expect due to etherpad 19:31:22 on top of that it is mostly used for the PTG and other tools can be used there instead (PTGBot for example) 19:31:37 all of these has me leaning strongly towards shutting down this service entirely rather than updating it to focal/jammy 19:31:43 *all of these things 19:32:11 my opinions on the matter mirror yours 19:32:39 yeah it's a shame, i've used it from time to time when we have data that has benefited from some simple analysis. but practically it seems to be an abandonded project 19:32:44 Are there any concerns from other infra-root for doing this? In my head we'd send an announcement for this to service-announce@lists.opendev.org planning to shut it down at the end of May. Then we can shutdown the instance, snapshot the instance, and delete the instance. Then do all the related config mgmt clean up 19:33:01 it was a neat idea in order to try to persuade community members not to rely on google spreadsheets, but it's unfortunately a liability 19:35:11 ok I'm not hearing any "please no!!!" :) I'll work to send out an announcement that this is our plan to communicate it with our user base 19:35:28 fwiw I did bring it up with people who do a lot of the PTG planning like diablo_rojo_phone and there wasn't any major concern from them either 19:36:07 We can totally just use the bot for teams to sign up with. 19:36:31 So as to avoid google sheets 19:37:28 sounds like that may be it for ethercalc. 19:37:35 both in this meeting and long term :) 19:37:40 #topic Open Discussion 19:37:42 Anything else? 19:38:03 i have a quick thing 19:38:17 https://zuul-ci.org/docs/zuul/6.0.0/releasenotes.html#relnotes-4-1-0-deprecation-notes 19:38:40 i don't believe that the opendev tenants have universally completed that migration 19:39:06 corvus: probably not. In fact just a month ago it was the OSA project saying that queues are pipeline specific and being confused about it 19:39:12 i plan on sending out a reminder to zuul-discuss about that, along with a script ppl can use to find places where updates are necessary 19:39:14 (I tried to clarify) 19:39:54 we should have removed that already, but the v6 release needed to happen before the next ansible release and i didn't want to hold it up 19:40:17 but i think it's fairly likely that we'll complete that before the next major rev 19:40:25 so would be good for opendev tenants to update 19:41:14 agreed, thanks! that script will help us double-check 19:41:27 just skimming codesearch there are a few places I see. The script would be great 19:42:14 freezer, zaqar, tripleo-ci, os-net-config, ironic and so on 19:42:29 that list wasn't exhaustive 19:43:05 Anything else? 19:43:20 (nak from me) 19:43:26 nope 19:44:54 In that case thanks everyone. We'll be back here next week same time and location. As always feel free to reach out on the mailing list (service-discuss@lists.opendev.org) or IRC (#opendev) if something comes up. We don't need to wait for the meeting to talk to each other :) 19:45:03 thanks clarkb! 19:45:09 And we can all have 15 minutes of breakfast/lunch/dinner/etc back 19:45:11 #endmeeting