15:00:20 <noonedeadpunk> #startmeeting openstack_ansible_meeting
15:00:20 <opendevmeet> Meeting started Tue Sep 17 15:00:20 2024 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:20 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:20 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting'
15:00:24 <noonedeadpunk> #topic rollcall
15:00:26 <noonedeadpunk> o/
15:06:20 <noonedeadpunk> #topic office hours
15:06:37 <noonedeadpunk> so, we're quite long overdue with the first minor release of Caracal
15:07:11 <noonedeadpunk> while we do really have plenty of fixes already merge - I think the biggest one is still Apache MPM which are inconsistent with Skyline on metal
15:07:39 <noonedeadpunk> I'm already about to have a patch (based on repo container), which should cover all others
15:08:00 <NeilHanlon> hiya
15:08:05 <noonedeadpunk> and this I assume also blocks us with testing of upgrade from the proper branch
15:08:23 <noonedeadpunk> (now tests run from 2023.2)
15:09:11 <noonedeadpunk> then I've also proposed testing Ceph for Rocky as well in https://review.opendev.org/c/openstack/openstack-ansible/+/929636
15:09:17 <noonedeadpunk> though it fails :(
15:11:16 <noonedeadpunk> and I _think_ I already saw that some time ago
15:11:58 <noonedeadpunk> and we even discussed that like couple of years ago with yu NeilHanlon
15:12:10 <noonedeadpunk> `The error was: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'discovered_interpreter_python'`
15:14:09 <noonedeadpunk> would need to revive in my memory wtf is wrong... I can recall ansible having some mapping for python by distros, but I kinda thought they got rid of it or smth.
15:14:39 <noonedeadpunk> regarding upcoming dalmatian release - we need to switch to 2024.2 branch for tracking
15:15:31 <noonedeadpunk> though, there's a batch right before branching as well: https://review.opendev.org/c/openstack/openstack-ansible/+/927841
15:15:44 <NeilHanlon> ah yeah.. i vaguely recall that interpreter thing.. python and ansible in EL has been ... fun
15:15:48 <noonedeadpunk> which fails for metal installs
15:19:49 <noonedeadpunk> we had some progress with ladning deb822 migrations
15:20:22 <noonedeadpunk> I wonder what else do we miss covering with it, except openstack_hosts, rabbitmq and galera
15:20:27 <noonedeadpunk> And ceph_client
15:23:45 <NeilHanlon> are there downsides to missing some conversions?
15:26:44 <noonedeadpunk> well, apt issues a warning of storing gpg keys in shared storage which is deprecated
15:27:12 <noonedeadpunk> as then each repo can use any gpg key basically, which is fixed by this new format
15:29:50 <NeilHanlon> ah, i see
15:30:01 <NeilHanlon> i tried to do some research on what it all was about but got lost
15:31:15 <noonedeadpunk> so nothing particulary bad, I guess, but as change is quite breaking, would bee good to do tis one time
15:31:48 <noonedeadpunk> I also need to deprecate couple of repos, ie: senlin, murano, sahara as underlying projects were also deprecated
15:35:32 <NeilHanlon> anything I can do to help? I've got some time I can spend on reviews and other things
15:39:37 <noonedeadpunk> um.. any updates on repo mirroring?:)
15:40:08 <noonedeadpunk> as that is smth which would be totally nice to have/improve on
15:40:25 <NeilHanlon> i did start working on diving through grafana
15:40:31 <NeilHanlon> err, opensearch
15:42:08 <NeilHanlon> https://opensearch.logs.openstack.org/_dashboards/app/data-explorer/discover?security_tenant=global#?_a=(discover:(columns:!(_source),isDirty:!t,sort:!()),metadata:(indexPattern:'94869730-aea8-11ec-9e6a-83741af3fdcd',view:discover))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-1M,to:now))&_q=(filters:!(('$state':(store:appState
15:42:08 <NeilHanlon> ),meta:(alias:!n,disabled:!f,index:'94869730-aea8-11ec-9e6a-83741af3fdcd',key:build_status,negate:!f,params:(query:FAILURE),type:phrase),query:(match_phrase:(build_status:FAILURE))),('$state':(store:appState),meta:(alias:'build_name%20~%20rocky',disabled:!f,index:'94869730-aea8-11ec-9e6a-83741af3fdcd',key:query,negate:!f,type:custom,value:'%7B%22wi
15:42:08 <NeilHanlon> ldcard%22:%7B%22build_name%22:%22*rocky*%22%7D%7D'),query:(wildcard:(build_name:'*rocky*')))),query:(language:kuery,query:'project:%20(%22openstack%2Fopenstack-ansible*%22%20OR%20%22openstack%2Fkolla*%22)%20AND%20message:%20(%22*cannot%20install*%22%20OR%20%22*Depsolve%20Error*%22)'))
15:42:16 <NeilHanlon> .. bleh
15:43:49 <NeilHanlon> https://drop1.neilhanlon.me/irc/uploads/f810495797312453/image.png there
15:44:25 <NeilHanlon> i need to spend some more time finding other failures as 9 seems low
15:44:41 <NeilHanlon> but, yeah.. progress
15:46:35 <noonedeadpunk> well, 8 of them were during one day?
15:46:53 <NeilHanlon> true..
15:47:05 <NeilHanlon> i bet i can correlate that with rocky releaseing stuff...
15:47:09 <noonedeadpunk> and probably worth checking for couple of month or smth...
15:47:22 <noonedeadpunk> yeah, true
15:47:36 <noonedeadpunk> but then also worth checking amount of jobs in total maybe?
15:47:45 <NeilHanlon> i think opensearch only has ~1week of logs
15:47:57 <NeilHanlon> err, i guess two
15:47:59 <noonedeadpunk> as local mirrors for sure would reduce latency/throughput to the external mirrors
15:48:03 <NeilHanlon> farthest back being the 4th
15:48:19 <noonedeadpunk> which is already a win as well, if there're enough jobs running to justify that
15:48:21 <noonedeadpunk> but dunno
15:48:45 <NeilHanlon> yeah i will make some pretty graph showing total jobs vs failed /timeout/etc
15:51:09 <noonedeadpunk> as low error rate can be used to promote using rocky in PTI instead of CentOS :D
16:01:23 <noonedeadpunk> #endmeeting