15:00:42 <noonedeadpunk> #startmeeting openstack_ansible_meeting 15:00:42 <opendevmeet> Meeting started Tue May 20 15:00:42 2025 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:00:42 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:00:42 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting' 15:00:47 <noonedeadpunk> #topic rollcall 15:00:48 <noonedeadpunk> o/ 15:01:28 <jrosser> o/ hello 15:02:58 <noonedeadpunk> #topic office hours 15:03:06 <noonedeadpunk> so I have good and bad news kinda 15:03:20 <noonedeadpunk> good one, is that we've branched 2025.1 15:03:41 <noonedeadpunk> And we're pretty much in time for the release in 2 weeks 15:04:26 <noonedeadpunk> despite most things were not done at last minute, we still having a release with no time to spare 15:04:45 <noonedeadpunk> And I wanted to say with way lower amount of stress as well 15:04:52 <noonedeadpunk> if not... EL10 situation 15:05:06 <jrosser> i think we just have to let that go for a while 15:05:18 <jrosser> seems like something that will eventually come (CI nodes) 15:05:35 <noonedeadpunk> and here bad news are coming. As python 3.11 is the minimal requirement, I did include drop of all EL9 related jobs to 15:05:39 <noonedeadpunk> #link https://review.opendev.org/c/openstack/openstack-ansible/+/950261 15:05:56 <jrosser> i did a patch also to drop jammy 15:06:11 <noonedeadpunk> I actually decided it to squash with this one 15:06:17 <noonedeadpunk> as it would be conflicting anyway 15:06:20 <jrosser> no problem, yes 15:06:38 <noonedeadpunk> I did not do any deep clean-up for EL though 15:06:48 <noonedeadpunk> IT was more of docs/matrix/jobs thing 15:07:02 <noonedeadpunk> and I wanna start looking locally at el10 support asap 15:07:30 <opendevreview> Merged openstack/openstack-ansible stable/2025.1: Update .gitreview for stable/2025.1 https://review.opendev.org/c/openstack/openstack-ansible/+/950417 15:07:31 <noonedeadpunk> nasty part, that we will have to backport it for upgrade reasons... 15:08:13 <noonedeadpunk> and I really don't like this specific part of it 15:08:42 <noonedeadpunk> But, well... we have what we have I guess... 15:08:53 <jrosser> we certainly do 15:08:59 <noonedeadpunk> also I left rocky 9 molecule jobs 15:09:25 <noonedeadpunk> as where they are used, they should not hurt just for sanity checks of the codepath 15:10:02 <jrosser> i am not sure how much i like having to backport a ton of stuff either 15:10:32 <jrosser> it is currently so much work to keep up with the EOL / unmaintained fall out that i just don't know if it is a good idea 15:10:33 <noonedeadpunk> it can be not _that_ bad if it's the next thing we land 15:10:58 <noonedeadpunk> as more or less clean backports can be done with a single button in gerrit 15:11:27 <noonedeadpunk> but then we're dropping an upgrade path, which is also not good 15:12:04 <noonedeadpunk> and I think it's me to blame, as I didn't manage to work on support in time, totally missed that we can't do this next cycle 15:13:00 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Start development of 2025.2 (Flamingo) https://review.opendev.org/c/openstack/openstack-ansible/+/950261 15:13:06 <jrosser> thats not entirely fair - to land support for a whole OS with major changes that maybe you don't even use for deployments 15:13:25 <jrosser> imho we really need more hands on the rh derived stuff in total 15:14:06 <jrosser> the debian and ubuntu OS upgrades are almost a no-op in comparison 15:14:08 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Start development of 2025.2 (Flamingo) https://review.opendev.org/c/openstack/openstack-ansible/+/950261 15:14:30 <noonedeadpunk> well, I don't use, but it come really handy for me couple of times in arguments 15:15:24 <noonedeadpunk> where it was - "we want EL, as we trust in it" - "well, maybe let;s start with deb, and if it won't work - we can always seamingly migrate to EL?" - "lgtm" kind of conversatrions 15:16:20 <noonedeadpunk> we do OS upgrades through re-install anyway btw, so if not mentioning amount of changes required in code/packages/etc - for operators upgradfe process is the same 15:16:26 <noonedeadpunk> but I totally get what you're saying 15:16:32 <noonedeadpunk> and I agree with it 15:16:54 <noonedeadpunk> just from the "product" prespective better to have such upgrade release... 15:17:00 <NeilHanlon> ah! 15:17:03 <jrosser> yes thats very true 15:17:05 * NeilHanlon got distracted by prometheus 15:17:07 <NeilHanlon> sorry :D 15:17:13 <jrosser> easy to happen :) 15:18:07 <noonedeadpunk> we're discussing dropping Rocky support :D 15:18:10 <NeilHanlon> sry reading back :D 15:18:18 <noonedeadpunk> so you haven't missed anything important :p 15:18:21 <NeilHanlon> hah 15:18:35 <NeilHanlon> no it's OK, i mean.. I get it 15:18:56 <jrosser> seems maybe the immediate issue is getting more horsepower on the CI node issue 15:19:06 <noonedeadpunk> But in seriosness, I wanna start doing patches this week 15:19:12 <jrosser> as anyone can do local dev with a cloud image and make patches 15:19:22 <jrosser> but we can't test/merge anything right now sensibly 15:19:40 <NeilHanlon> and I agree w/ jrosser about it not being really all on your plate noonedeadpunk especially when it's _not_ something you deploy actively 15:19:59 <NeilHanlon> but passing blame around isn't probably productive :D 15:20:39 <NeilHanlon> the immediate need is to fix for our current users and make sure we can keep pushing code we are confident in because it's passing CI 15:20:43 <noonedeadpunk> I'd actually vote on landing EL10 even without testing.... Just openly stating that everywhere to manage expectations 15:21:23 <noonedeadpunk> as I believe that CI might be solved sooner then later 15:21:35 <NeilHanlon> another possible interesting thing re: major version upgrades is Oracle recently did this: https://github.com/openela/leapp-repository 15:21:43 <noonedeadpunk> we can add jobs as NV to find issues and them set as voting 15:22:10 <NeilHanlon> but that's tangential at this point as it's just more work we'd need to do to maintain actors for doing openstack upgrades atop rocky and i prefer not doing that anyways 15:22:33 <noonedeadpunk> right, I prefer not upgrading ubuntu either, whenever I can 15:22:53 <noonedeadpunk> jsut re-install is way more trivial if you have all internal things automated anyway 15:23:05 <noonedeadpunk> (and take less time) 15:23:11 <jrosser> i think i miswrote earlier about OS upgrades, i really meant introducing jobs for a new ubuntu/debian tends to be very trivial 15:23:20 <NeilHanlon> right, that's sorta the whole reason we use ansible ! 15:23:37 <noonedeadpunk> jrosser: yeash, that's true 15:23:38 <NeilHanlon> jrosser: yeah I think that also threw me off but i still wanted to mention it's technically more possible now 15:24:02 <noonedeadpunk> apparmor issues were not trivial this time around though 15:24:58 <jrosser> anyway :) if you want to make some patches regardless that is OK 15:25:10 <noonedeadpunk> so yeah, I will check on what it would take to add EL10 support with local aio 15:25:16 <NeilHanlon> have we poked mnasiadka about the python3.11 stuff btw? 15:25:19 <noonedeadpunk> and we'll see if/how/when to land them 15:25:22 <jrosser> i was thinking to look at the usgi script thing (maybe even later today) 15:25:28 <NeilHanlon> we were chatting about NFV sig stuff this morning.. 15:25:39 <jrosser> as i see that some services are migrating with no backward compatibility 15:25:48 <NeilHanlon> i'm almost at the point where like.. i'll just go rebuild all the things we need to make python3.11 work 15:25:56 <mnasiadka> What python3.11 stuff? :) 15:26:02 <noonedeadpunk> I now for fact that python3-libvirt is required by nova, and it's shipped only for 3.9 15:26:03 * NeilHanlon signing myself up for more work as always 15:26:15 <mnasiadka> Ah, you tried the same thing as me 15:26:35 <noonedeadpunk> mnasiadka: I learned that with EL8 I think 15:26:47 <opendevreview> Merged openstack/openstack-ansible stable/2025.1: Update TOX_CONSTRAINTS_FILE for stable/2025.1 https://review.opendev.org/c/openstack/openstack-ansible/+/950418 15:26:49 <noonedeadpunk> under very alike circumstances 15:26:59 <noonedeadpunk> (or maybe it was 7? who knows) 15:27:02 <NeilHanlon> Enterprise Linux: the pain that keeps on giving 15:27:13 <NeilHanlon> why do i do this to myself? lol 15:27:29 <noonedeadpunk> and I have memories of libselinux bindings for python as well, but there were some ways around... 15:27:38 <noonedeadpunk> NeilHanlon: great question, lol "D 15:28:17 <noonedeadpunk> but I'm really impressed how ppl continue shooting themselves in the leg 15:28:28 <noonedeadpunk> by building against cnetos for EL, and things like that 15:28:43 <noonedeadpunk> dropping cpu flags for still used hardware... 15:28:48 <noonedeadpunk> doh 15:29:05 <noonedeadpunk> anyway 15:29:12 <NeilHanlon> psh isn't everyone replacing their servers every 3 years to fit more pci lanes to inference AI or something? 15:29:19 <mnasiadka> Should we all stop supporting EL?:) 15:29:35 <NeilHanlon> mnasiadka: in protest? :P 15:29:57 <noonedeadpunk> NeilHanlon: and also efficiency per watt, and save costs on cooling, sure 15:30:18 <NeilHanlon> nah i mean, we all know it's been a weird situation the past few years and hasn't gotten really any clearer esp. with the CI situation and not really having any idea what's up with that all 15:30:37 <noonedeadpunk> mnasiadka: frankly? I'd internally love to do that, but I kinda realize that it's not gonna end well 15:31:08 <noonedeadpunk> internally, in terms of very personal opinion 15:32:03 <noonedeadpunk> as when trying to sell openstack, more then half are not ready to consider anything except EL 15:33:14 <noonedeadpunk> or at the very least they require the ability of rollback to EL 15:33:15 <NeilHanlon> i just had that conversation with a colleague in the rocky project on our testing team 15:33:47 <NeilHanlon> his boss wants him to investigate openstack and talk to Canonical and he is firmly planted he'd get it from RH instead 15:34:02 <NeilHanlon> i'm always trying to abduct more people into our cult :P 15:34:20 <noonedeadpunk> right :) 15:35:11 <noonedeadpunk> I especially like cases of moving from VMware due to screwed policies to RH OpenStack 15:35:21 <jrosser> mnasiadka: while you are around - did i understand right that you were looking at the rh-10-ish nodpool images? 15:35:37 <noonedeadpunk> Just as if IBM is any better then Broadcom at global scale of screwing up customers 15:36:06 <mnasiadka> Yes, I’m trying to help tonyb in CS10 work 15:37:50 <noonedeadpunk> I can try to help if there's a need 15:38:16 <noonedeadpunk> but I did not have time yet to double check the state after asking in #opendev 15:38:57 <NeilHanlon> noonedeadpunk: yeah i chose not to mention that to that colleague :P 15:39:22 <NeilHanlon> mnasiadka: and, sorry i fell off, but did you end up working on Rocky 9 nodepool images, too? 15:39:53 <mnasiadka> @NeilHanlon: yup, that’s done ;) 15:40:24 <NeilHanlon> mnasiadka: add another beer to my tab for you 15:45:27 <NeilHanlon> ok, so.. 15:46:01 <NeilHanlon> short term, we drop rocky jobs and backport to the N-1(-1,-1?) releases so we can land changes 15:46:40 <NeilHanlon> at the same time, we maybe work on landing el10? I volunteer DavidGomez to help. :) 15:47:38 <NeilHanlon> noonedeadpunk: does that seem like a reasonable next steps? 15:47:38 <DavidGomez> of course, happy to help 15:48:48 <noonedeadpunk> NeilHanlon: we don't need to backport jobs removal 15:48:57 <noonedeadpunk> we'd need to backport EL10 patches 15:49:04 <NeilHanlon> ah.. 15:49:04 <noonedeadpunk> ideally 15:49:07 <NeilHanlon> i had that reversed in my head 15:49:15 <NeilHanlon> well, more experience for DavidGomez! lol 15:49:33 <noonedeadpunk> lol, I won't refuse help, sure :) 15:49:39 <NeilHanlon> sorry david, you can get me back at the next onsite :P 15:50:30 <noonedeadpunk> but also adding rocky 10 image to nodepool would be amazing 15:51:33 <DavidGomez> haha not at all, sounds like a good learning opportunity 15:52:10 <noonedeadpunk> ok, cool, sounds like a plan then 15:53:51 <noonedeadpunk> #endmeeting