| frickler | I'm going to skip the meeting and call it a day, will read backlog tomorrow | 18:56 |
|---|---|---|
| fungi | have a good evening frickler! | 18:56 |
| clarkb | good night! | 18:58 |
| clarkb | it may be a small group then | 18:58 |
| clarkb | tonyb also mentioend not being able to make it today | 18:59 |
| clarkb | #startmeeting infra | 19:00 |
| opendevmeet | Meeting started Tue Jul 30 19:00:15 2024 UTC and is due to finish in 60 minutes. The chair is clarkb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 19:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 19:00 |
| opendevmeet | The meeting name has been set to 'infra' | 19:00 |
| clarkb | #link https://lists.opendev.org/archives/list/service-discuss@lists.opendev.org/thread/GBQUFAFSFBGZGLEYVTCCSUABJXCS6ZUJ/ Our Agenda | 19:00 |
| clarkb | #topic Upgrading old servers | 19:00 |
| clarkb | #undo | 19:00 |
| opendevmeet | Removing item from minutes: #topic Upgrading old servers | 19:00 |
| clarkb | I'm getting ahead of myself here | 19:00 |
| clarkb | #topic Announcements | 19:00 |
| clarkb | As mentioned above frickler and tonyb won't make the meeting today. But we'll take notes here so that they and others can review them later | 19:01 |
| clarkb | I've realized that I made a bit of a mistake in planning family obligations and will likely miss the next two Tuesdays... My brother has midweek "weekends" and so late summer family stuff all got planned midweek | 19:01 |
| clarkb | Anyway I probably won't be around on August 6 or 13 to run this meeting. | 19:02 |
| fungi | i expect to be around and can chair either/both if we want to meet | 19:02 |
| clarkb | thanks! | 19:02 |
| fungi | we can probably defer that decision until closer to the end of the week since we have some people not around today | 19:03 |
| clarkb | I'm also waffling on whether or not to attend Fossy whcih starts Thursday afternoon. I probably should go and use it is an opportunity to thank the folks that work on the osuosl stuff | 19:03 |
| clarkb | fungi: yup no rush on that | 19:03 |
| clarkb | I do have a working laptop again though so if I do go to fossy I won't be completely disconnected from the world. Just highly distracted :) | 19:03 |
| clarkb | #topic Upgrading Old Servers | 19:04 |
| clarkb | tonyb can't make it but did mention that noble images have been uploaded to all cloud regions but one. I'm not sure which that one is, but this implies that at least two rax regions were uploaded to which is good news as I expected those to be most difficult | 19:05 |
| fungi | that's exciting progress. thanks tonyb! | 19:06 |
| clarkb | ++ | 19:06 |
| clarkb | looking forward to more info when tonyb is able to share | 19:06 |
| clarkb | anything else to mention about upgrading servers? I've been distracted by services and clouds and stuff not servers lately | 19:07 |
| fungi | i don't think so | 19:07 |
| clarkb | #topic AFS Mirror Cleanups | 19:08 |
| clarkb | we just landed the change to cleanup centos 8 stream nodesets and similar stuff from opendev/base-jobs | 19:08 |
| clarkb | The next step there is to clean the centos 8 stream images out of nodepool entirely. | 19:09 |
| clarkb | #link https://review.opendev.org/c/openstack/project-config/+/924790 Next step in CentOS 8 Stream Cleanup | 19:09 |
| clarkb | also topic:drop-centos-8-stream | 19:09 |
| clarkb | before we merge 924790 we should check if there are any stuck nodes that need deleting, but Ithink the solution to that is likely not going to chagne regardless of how we merge that change | 19:10 |
| clarkb | just somethign to check on and clean up | 19:10 |
| clarkb | apologies that progress has been so slow on this, but we are slowly making progress which I'll take as a small win | 19:11 |
| clarkb | #topic Testing Rackspaces New Cloud Offering | 19:11 |
| clarkb | I'm going to send a followup email today/tomorrowish after double checking that this is more likely to be a better time for them | 19:12 |
| clarkb | I was told to wait a week and its been a week. Just want to check first before things get lost in the ether again. But otehrwise nothing new here | 19:12 |
| clarkb | #topic Reconsider queue depth for integrated gate | 19:13 |
| clarkb | The alternative proposal for this to implement early failure detection has been implemented for tempest jobs | 19:13 |
| clarkb | The reason for doing this in tempest is that those jobs are long running so have a larger impact if we can detect failures early and reset ASAP, but also beacuse these tempest base jobs are used everywhere and we can make a small change in one spot and have a large impact across many jobs | 19:13 |
| clarkb | So far haven't seen any complaints or even questions about why things are failing before they are done. This would imply that it is working in a manner that isn't too intrusive which is great | 19:14 |
| fungi | yeah, a tempest job can fail on a test 25 minutes in and then keep running more tests for a further hour or more | 19:14 |
| clarkb | a good next step may be to update openstack-tox-pyXY jobs to do similar though their impact will be smaller | 19:15 |
| clarkb | other than that I guess try to observe gate behavior the next time you notice a large queue and we can see if this has helped at all | 19:16 |
| clarkb | #topic Removing the Works on ARM Linaro Cloud from Nodepool | 19:16 |
| clarkb | Last week I received an email from the Works on ARM folks letting us know that the hardware hosting the linaro cloud will be removed/no longer accessible on August 10. I committed to shutting down our usage of that cloud this week which is now done as of this morning local time for me | 19:17 |
| clarkb | I have since sent a followup email to Kevin at linaro letting him know the cleanup on our side is done and that we can clean up the cloud stuff if necessary. I also offered to help | 19:18 |
| clarkb | I suspect that might end up being a die on the vine situation and the cloud will go away when the hardware goes away | 19:18 |
| clarkb | This does mean we are down to operating arm jobs in one cloud region (the osuosl arm cloud) | 19:19 |
| clarkb | fungi: you've been helping out with this process anything else to add? | 19:19 |
| fungi | nope, i think we're basically done other than whatever kevin comes back with | 19:21 |
| clarkb | #topic Service Coordinator Election | 19:22 |
| clarkb | It is that time of year again. I looked at when we had the last election and added 6 months and ended up with this proposal: Open candidate nominations from August 6 to August 20. Then if necessary we can have an election August 21 to 28. This timing also works out nicely to avoid the summit | 19:23 |
| clarkb | considering that attendance is light today, maybe we can see if others can ack this proposal as reasonable and then I'll send out email to the list making it official later this week? | 19:23 |
| fungi | sounds good, thanks for keeping up with that! | 19:24 |
| clarkb | and if there are any concerns or questions feel free to raise those too | 19:24 |
| clarkb | thanks! | 19:25 |
| clarkb | #topic Open Discussion | 19:25 |
| clarkb | I have more thinsg too that never made it to the agenda proper (you might have noticed the last item was missed too, I was very distracted by laptop reinstall things) | 19:25 |
| clarkb | First up dansmith is trying to clarify some confusion he ran into creating a new project here: https://review.opendev.org/c/opendev/infra-manual/+/925275 | 19:25 |
| clarkb | I think landing that now if we can is a good idea just because it is strightforward and I don't want to forget later | 19:26 |
| clarkb | fungi maybe you have time to review that? it should be a quick one | 19:26 |
| fungi | on it | 19:26 |
| clarkb | Next I've been looking ahead to mid term goals I've got for OpenDev: I would like to see us bump the default nodeset to ubuntu noble and change the default ansible version in our zuultenants to ansible 9 for example. A late october early november ish upgrade of Gerrit to 3.10 would also be great. I've also been thinking about trying to have more "OpenDev Zuul for our users" | 19:29 |
| clarkb | content this is still very early brainstorming but I've noticed a lot of people struggling with zuul concepts recently... | 19:29 |
| clarkb | of these the most straightforward might be the ansible bump. Noble presents challenges due to it having python3.12 which is a fairly big leap in the python workd | 19:30 |
| fungi | content in the infra-manual? | 19:30 |
| clarkb | fungi: maybe? another thought was some sort of video | 19:30 |
| clarkb | addressing things like people not understanding depends on and speculative git states, rarely providing direct links to debug failures, understanding when jobs run and why, and so on are what I want to address | 19:32 |
| fungi | makes sense | 19:32 |
| clarkb | Zuul is this really great powerful tool and I want people to be able to take advantage of it | 19:33 |
| clarkb | anyway all of that gets mixed in with things like the summit and the ptg and the next openstack release. A lot of stuff is happening over the next few months and I wanted to bring these ideas up early so that we can try and fit them in | 19:34 |
| corvus | ++ | 19:34 |
| clarkb | feedback and ideas welcome. | 19:35 |
| clarkb | Anything else to go over in the meeting today? | 19:36 |
| fungi | i didn't have anything else | 19:37 |
| clarkb | sounds like that is everything. Thank you for your time today. And we can figure out what we are doing next week as we get closer to that point in time. | 19:39 |
| clarkb | #endmeeting | 19:39 |
| opendevmeet | Meeting ended Tue Jul 30 19:39:39 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 19:39 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/infra/2024/infra.2024-07-30-19.00.html | 19:39 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/infra/2024/infra.2024-07-30-19.00.txt | 19:39 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/infra/2024/infra.2024-07-30-19.00.log.html | 19:39 |
| corvus | thanks clarkb ! | 19:39 |
| fungi | thanks clarkb!1 | 19:39 |
| fungi | !!1!eleven | 19:39 |
| opendevmeet | fungi: Error: "!1!eleven" is not a valid command. | 19:39 |
| fungi | even opendevmeet doesn't get my jokes | 19:40 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!