15:00:42 #startmeeting openstack_ansible_meeting 15:00:42 Meeting started Tue May 20 15:00:42 2025 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:42 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:42 The meeting name has been set to 'openstack_ansible_meeting' 15:00:47 #topic rollcall 15:00:48 o/ 15:01:28 o/ hello 15:02:58 #topic office hours 15:03:06 so I have good and bad news kinda 15:03:20 good one, is that we've branched 2025.1 15:03:41 And we're pretty much in time for the release in 2 weeks 15:04:26 despite most things were not done at last minute, we still having a release with no time to spare 15:04:45 And I wanted to say with way lower amount of stress as well 15:04:52 if not... EL10 situation 15:05:06 i think we just have to let that go for a while 15:05:18 seems like something that will eventually come (CI nodes) 15:05:35 and here bad news are coming. As python 3.11 is the minimal requirement, I did include drop of all EL9 related jobs to 15:05:39 #link https://review.opendev.org/c/openstack/openstack-ansible/+/950261 15:05:56 i did a patch also to drop jammy 15:06:11 I actually decided it to squash with this one 15:06:17 as it would be conflicting anyway 15:06:20 no problem, yes 15:06:38 I did not do any deep clean-up for EL though 15:06:48 IT was more of docs/matrix/jobs thing 15:07:02 and I wanna start looking locally at el10 support asap 15:07:30 Merged openstack/openstack-ansible stable/2025.1: Update .gitreview for stable/2025.1 https://review.opendev.org/c/openstack/openstack-ansible/+/950417 15:07:31 nasty part, that we will have to backport it for upgrade reasons... 15:08:13 and I really don't like this specific part of it 15:08:42 But, well... we have what we have I guess... 15:08:53 we certainly do 15:08:59 also I left rocky 9 molecule jobs 15:09:25 as where they are used, they should not hurt just for sanity checks of the codepath 15:10:02 i am not sure how much i like having to backport a ton of stuff either 15:10:32 it is currently so much work to keep up with the EOL / unmaintained fall out that i just don't know if it is a good idea 15:10:33 it can be not _that_ bad if it's the next thing we land 15:10:58 as more or less clean backports can be done with a single button in gerrit 15:11:27 but then we're dropping an upgrade path, which is also not good 15:12:04 and I think it's me to blame, as I didn't manage to work on support in time, totally missed that we can't do this next cycle 15:13:00 Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Start development of 2025.2 (Flamingo) https://review.opendev.org/c/openstack/openstack-ansible/+/950261 15:13:06 thats not entirely fair - to land support for a whole OS with major changes that maybe you don't even use for deployments 15:13:25 imho we really need more hands on the rh derived stuff in total 15:14:06 the debian and ubuntu OS upgrades are almost a no-op in comparison 15:14:08 Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Start development of 2025.2 (Flamingo) https://review.opendev.org/c/openstack/openstack-ansible/+/950261 15:14:30 well, I don't use, but it come really handy for me couple of times in arguments 15:15:24 where it was - "we want EL, as we trust in it" - "well, maybe let;s start with deb, and if it won't work - we can always seamingly migrate to EL?" - "lgtm" kind of conversatrions 15:16:20 we do OS upgrades through re-install anyway btw, so if not mentioning amount of changes required in code/packages/etc - for operators upgradfe process is the same 15:16:26 but I totally get what you're saying 15:16:32 and I agree with it 15:16:54 just from the "product" prespective better to have such upgrade release... 15:17:00 ah! 15:17:03 yes thats very true 15:17:05 * NeilHanlon got distracted by prometheus 15:17:07 sorry :D 15:17:13 easy to happen :) 15:18:07 we're discussing dropping Rocky support :D 15:18:10 sry reading back :D 15:18:18 so you haven't missed anything important :p 15:18:21 hah 15:18:35 no it's OK, i mean.. I get it 15:18:56 seems maybe the immediate issue is getting more horsepower on the CI node issue 15:19:06 But in seriosness, I wanna start doing patches this week 15:19:12 as anyone can do local dev with a cloud image and make patches 15:19:22 but we can't test/merge anything right now sensibly 15:19:40 and I agree w/ jrosser about it not being really all on your plate noonedeadpunk especially when it's _not_ something you deploy actively 15:19:59 but passing blame around isn't probably productive :D 15:20:39 the immediate need is to fix for our current users and make sure we can keep pushing code we are confident in because it's passing CI 15:20:43 I'd actually vote on landing EL10 even without testing.... Just openly stating that everywhere to manage expectations 15:21:23 as I believe that CI might be solved sooner then later 15:21:35 another possible interesting thing re: major version upgrades is Oracle recently did this: https://github.com/openela/leapp-repository 15:21:43 we can add jobs as NV to find issues and them set as voting 15:22:10 but that's tangential at this point as it's just more work we'd need to do to maintain actors for doing openstack upgrades atop rocky and i prefer not doing that anyways 15:22:33 right, I prefer not upgrading ubuntu either, whenever I can 15:22:53 jsut re-install is way more trivial if you have all internal things automated anyway 15:23:05 (and take less time) 15:23:11 i think i miswrote earlier about OS upgrades, i really meant introducing jobs for a new ubuntu/debian tends to be very trivial 15:23:20 right, that's sorta the whole reason we use ansible ! 15:23:37 jrosser: yeash, that's true 15:23:38 jrosser: yeah I think that also threw me off but i still wanted to mention it's technically more possible now 15:24:02 apparmor issues were not trivial this time around though 15:24:58 anyway :) if you want to make some patches regardless that is OK 15:25:10 so yeah, I will check on what it would take to add EL10 support with local aio 15:25:16 have we poked mnasiadka about the python3.11 stuff btw? 15:25:19 and we'll see if/how/when to land them 15:25:22 i was thinking to look at the usgi script thing (maybe even later today) 15:25:28 we were chatting about NFV sig stuff this morning.. 15:25:39 as i see that some services are migrating with no backward compatibility 15:25:48 i'm almost at the point where like.. i'll just go rebuild all the things we need to make python3.11 work 15:25:56 What python3.11 stuff? :) 15:26:02 I now for fact that python3-libvirt is required by nova, and it's shipped only for 3.9 15:26:03 * NeilHanlon signing myself up for more work as always 15:26:15 Ah, you tried the same thing as me 15:26:35 mnasiadka: I learned that with EL8 I think 15:26:47 Merged openstack/openstack-ansible stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1 https://review.opendev.org/c/openstack/openstack-ansible/+/950418 15:26:49 under very alike circumstances 15:26:59 (or maybe it was 7? who knows) 15:27:02 Enterprise Linux: the pain that keeps on giving 15:27:13 why do i do this to myself? lol 15:27:29 and I have memories of libselinux bindings for python as well, but there were some ways around... 15:27:38 NeilHanlon: great question, lol "D 15:28:17 but I'm really impressed how ppl continue shooting themselves in the leg 15:28:28 by building against cnetos for EL, and things like that 15:28:43 dropping cpu flags for still used hardware... 15:28:48 doh 15:29:05 anyway 15:29:12 psh isn't everyone replacing their servers every 3 years to fit more pci lanes to inference AI or something? 15:29:19 Should we all stop supporting EL?:) 15:29:35 mnasiadka: in protest? :P 15:29:57 NeilHanlon: and also efficiency per watt, and save costs on cooling, sure 15:30:18 nah i mean, we all know it's been a weird situation the past few years and hasn't gotten really any clearer esp. with the CI situation and not really having any idea what's up with that all 15:30:37 mnasiadka: frankly? I'd internally love to do that, but I kinda realize that it's not gonna end well 15:31:08 internally, in terms of very personal opinion 15:32:03 as when trying to sell openstack, more then half are not ready to consider anything except EL 15:33:14 or at the very least they require the ability of rollback to EL 15:33:15 i just had that conversation with a colleague in the rocky project on our testing team 15:33:47 his boss wants him to investigate openstack and talk to Canonical and he is firmly planted he'd get it from RH instead 15:34:02 i'm always trying to abduct more people into our cult :P 15:34:20 right :) 15:35:11 I especially like cases of moving from VMware due to screwed policies to RH OpenStack 15:35:21 mnasiadka: while you are around - did i understand right that you were looking at the rh-10-ish nodpool images? 15:35:37 Just as if IBM is any better then Broadcom at global scale of screwing up customers 15:36:06 Yes, I’m trying to help tonyb in CS10 work 15:37:50 I can try to help if there's a need 15:38:16 but I did not have time yet to double check the state after asking in #opendev 15:38:57 noonedeadpunk: yeah i chose not to mention that to that colleague :P 15:39:22 mnasiadka: and, sorry i fell off, but did you end up working on Rocky 9 nodepool images, too? 15:39:53 @NeilHanlon: yup, that’s done ;) 15:40:24 mnasiadka: add another beer to my tab for you 15:45:27 ok, so.. 15:46:01 short term, we drop rocky jobs and backport to the N-1(-1,-1?) releases so we can land changes 15:46:40 at the same time, we maybe work on landing el10? I volunteer DavidGomez to help. :) 15:47:38 noonedeadpunk: does that seem like a reasonable next steps? 15:47:38 of course, happy to help 15:48:48 NeilHanlon: we don't need to backport jobs removal 15:48:57 we'd need to backport EL10 patches 15:49:04 ah.. 15:49:04 ideally 15:49:07 i had that reversed in my head 15:49:15 well, more experience for DavidGomez! lol 15:49:33 lol, I won't refuse help, sure :) 15:49:39 sorry david, you can get me back at the next onsite :P 15:50:30 but also adding rocky 10 image to nodepool would be amazing 15:51:33 haha not at all, sounds like a good learning opportunity 15:52:10 ok, cool, sounds like a plan then 15:53:51 #endmeeting