15:00:34 <noonedeadpunk> #startmeeting openstack_ansible_meeting
15:00:34 <opendevmeet> Meeting started Tue May 13 15:00:34 2025 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:34 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:34 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting'
15:00:38 <noonedeadpunk> #topic rollcall
15:00:40 <noonedeadpunk> o/
15:00:53 <David_Gomez> o/
15:03:45 <jrosser> o/
15:06:31 <noonedeadpunk> #topic office hours
15:06:44 <noonedeadpunk> so I think we are super close today
15:06:54 <noonedeadpunk> to get roles branched witrh 2025.1
15:07:37 <noonedeadpunk> I'm not sure we do have hard blockers actually
15:07:49 <NeilHanlon> o/
15:08:29 <jrosser> it is a shame to miss the swift stuff
15:08:29 <noonedeadpunk> there are some Swift outstanding improvements, but I think we may be able to backport them after branching
15:08:35 <jrosser> hah :)
15:08:38 <jrosser> ^ that
15:08:56 <jrosser> its basically broken so this is bugfix work
15:08:58 <noonedeadpunk> today the blocker for them are 0775 permissions in /etc on Rocky
15:09:17 <noonedeadpunk> we can try to address it in ssh_keypairs role
15:09:46 <noonedeadpunk> but I feel weird changing permissions for etc
15:10:01 <jrosser> can we do it in zuul pre task or something
15:10:06 <jrosser> thats less wierd
15:10:17 <jrosser> as its a CI specific fix, in a CI specific place
15:10:30 <NeilHanlon> yeah.. i can get it fixed but it probably will take too long to be meaningful for us
15:10:45 <NeilHanlon> i have to build and test and publish containers, then nodepool would have to rebuild them... then propagate that out...
15:11:16 <noonedeadpunk> I'm thinking if this can be done in CI for upgrade jobs...
15:11:23 <noonedeadpunk> if in some pre_tasks
15:11:32 <noonedeadpunk> it can work, yeah, I will try this out
15:11:49 <noonedeadpunk> and the next confusing/broken thing is OVS for Noble.
15:12:00 <noonedeadpunk> #link https://zuul.opendev.org/t/openstack/build/9199af1212034c2fbc4f086c0569416a
15:12:17 <noonedeadpunk> and I was able to reproduce this, it seems
15:12:51 <noonedeadpunk> so potentially this might be quite valid bug
15:13:38 <jrosser> hmm i wonder if we have the right version
15:14:09 <jrosser> like assumption of ovs version vs distro repo vs UCA .....
15:14:27 <noonedeadpunk> in AIO I got 3.5.0
15:14:33 <noonedeadpunk> but also
15:14:36 <noonedeadpunk> metal is passing
15:14:51 <noonedeadpunk> https://zuul.opendev.org/t/openstack/build/8f30d7d8d8e64d1ba25e6aa46f09f4cf
15:15:29 <jrosser> having said all this - the other jobs are all OVN which uses OVS under the hood
15:16:01 <jrosser> so that side of things is OK on noble
15:16:29 <noonedeadpunk> well. OVN is kinda different, as it's not neutron code that need to deal with OVS wiring and namespaces and etc
15:16:51 <noonedeadpunk> its Neutron -> OVN -> OVS
15:17:43 <jrosser> and i just catch up with this - it's the LXC version of this job which fails but metal is OK
15:18:12 <noonedeadpunk> yeah, which is weird
15:18:22 <noonedeadpunk> but seems reproducible at very least
15:18:32 <noonedeadpunk> I jsut did not check on logs in my AIO yet
15:19:11 <noonedeadpunk> Other then that... I guess I'd love to land https://review.opendev.org/c/openstack/openstack-ansible/+/946281 regardless, before branching happens
15:19:39 <noonedeadpunk> Probably it's a bit too late, but also I'm not sure what we are actually testing now
15:19:53 <noonedeadpunk> this should have been done right after the beta
15:20:23 <jrosser> theres some errors about dropped packets on bt-int https://3e3b9bb51c4bbfcca1ce-b2ab9ab260082d15bc4e79c36fac49d2.ssl.cf5.rackcdn.com/openstack/9199af1212034c2fbc4f086c0569416a/logs/host/openvswitch/ovs-vswitchd.log.txt
15:21:45 <noonedeadpunk> I have same log pretty much on aio, yes
15:24:18 <noonedeadpunk> it can boil down to some brdige setup and veth
15:24:50 <noonedeadpunk> as we connect br-provider in ovs with br-vlan in lxb thrtough veth eth12
15:30:31 <noonedeadpunk> but yeah - other then that things look pretty good
15:31:12 <jrosser> it will be good to get the release done to work on new ansible and also cutting the number of jobs back where possible
15:32:06 <noonedeadpunk> I don't think we will in fact drop anything
15:32:12 <noonedeadpunk> as Debian 13 is pretty much released
15:32:26 <noonedeadpunk> so I assume we'll need to look into it as well
15:33:53 <jrosser> i have a few patches outstanding for zuul errors cleanup in stable and unmaintained
15:34:06 <jrosser> and also a few that can't be merged
15:34:10 <noonedeadpunk> wich are not passing
15:34:12 <noonedeadpunk> yeah
15:34:39 <jrosser> idk if we want to get them force merged or not
15:34:53 <jrosser> but there is a problem that the 2023.2 branch is deleted from the service repos
15:34:57 <jrosser> but not from ours
15:35:03 <jrosser> so i think thats a big cause of the trouble
15:36:04 <noonedeadpunk> oh... right...
15:36:26 <noonedeadpunk> I think I need to look on EOM-ing the branch
15:36:54 <jrosser> yep - there are several processes, EOM-ing, unmaintaining etc
15:37:10 <jrosser> and of course we are somewhat offset from those today compared to the rest of the projects
15:37:19 <jrosser> zed is messy right now in particular
15:38:38 <noonedeadpunk> I frankly did not look into CI results except saw they're not feeling good
15:40:55 <noonedeadpunk> ok, so to sum up plan for this week - branch roles, propose rc1, EOM 2023.2, check what's up with Zed
15:43:56 <noonedeadpunk> and try to figure out OVS/Swift
15:51:14 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Ensure /etc permissions in CI  https://review.opendev.org/c/openstack/openstack-ansible/+/949647
15:51:47 <opendevreview> Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Migrate ring distribution to SSHCA  https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946990
15:51:58 <noonedeadpunk> let's see if that will help
15:52:41 <noonedeadpunk> oh, one thing I wanted to discuss actually
15:53:04 <noonedeadpunk> which is this patch
15:53:06 <noonedeadpunk> #link https://review.opendev.org/c/openstack/openstack-ansible/+/949497
15:53:38 <noonedeadpunk> and if we wanna go this route, or better to properly patch tooling and add support for deploy-guide folders import to Zanata instead
15:53:48 <noonedeadpunk> as eventually - this patch is pretty much a workaround
15:54:38 <noonedeadpunk> over stuff related to the translations overall
15:56:43 <noonedeadpunk> so any input/ideas on how we prefer moving forward is really appreciated
15:59:19 <noonedeadpunk> or we can try this way and revert if it does not work...
15:59:33 <noonedeadpunk> #endmeeting