15:03:02 <noonedeadpunk> #startmeeting openstack_ansible_meeting
15:03:02 <opendevmeet> Meeting started Tue Jun 25 15:03:02 2024 UTC and is due to finish in 60 minutes.  The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:03:02 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:03:02 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting'
15:03:07 <noonedeadpunk> #topic office hours
15:03:10 <jrosser> o/ hello
15:03:13 <noonedeadpunk> o/
15:03:15 <noonedeadpunk> hey
15:03:30 <damiandabrowski> hi!
15:03:38 <noonedeadpunk> I clean forgot that I will need to leave shortly - so let's make this quick :)
15:04:16 <noonedeadpunk> So ceph-ansible 9.0 issue I think Damian you was working on
15:04:26 <noonedeadpunk> and had some progress from what I recall?
15:04:29 <NeilHanlon> o/ - am only kinda around today
15:04:29 <noonedeadpunk> sorry, 8.0
15:04:35 <noonedeadpunk> ++
15:05:00 <jrosser> yes it looks like there was a renaming of the service for rgw which we need to accomodate
15:05:01 <damiandabrowski> yeah, this patch is responsible for the issue: https://github.com/ceph/ceph-ansible/commit/1121e6d98a41a913886240bcb873380008afcafc
15:05:18 <damiandabrowski> I proposed a solution above ^
15:05:53 <jrosser> would you like to make a patch - else i can do that?
15:06:52 <damiandabrowski> if we apply a fix in a separate patch, we'll probably have circular dependency :/
15:07:02 <damiandabrowski> i can push a commit to your change if you don't have anything against it
15:07:04 <damiandabrowski> https://review.opendev.org/c/openstack/openstack-ansible/+/921976
15:07:13 <jrosser> sure thats no problem, i don't mind
15:07:57 <noonedeadpunk> so, I guess I'd define `rgw_zone` to value of ceph_cluster_name if it makes any sense?
15:08:05 <noonedeadpunk> https://opendev.org/openstack/openstack-ansible-ceph_client/src/branch/master/defaults/main.yml#L52-L54
15:08:11 <jrosser> i think `default`
15:08:30 <jrosser> you may have multiple zone groups in one cluster
15:08:38 <noonedeadpunk> aha
15:08:43 <damiandabrowski> okok, I'll try to figure out how to apply the fix in a separate change but if I won't be able to come up with anything wise, I'll just push a commit to your change
15:08:45 <noonedeadpunk> ok, fair enough
15:09:01 <jrosser> you can just edit my change, that will be easier
15:09:10 <noonedeadpunk> nah, you should edit Jonathan's one
15:09:15 <noonedeadpunk> otherwise it won't pass CI
15:09:15 <damiandabrowski> ok
15:09:29 <jrosser> thankyou for finding that :)
15:09:31 <damiandabrowski> I also need to figure out how to not break environments that use recent OSA version with ceph-ansible 7.0
15:09:44 <damiandabrowski> no problem at all ;)
15:09:48 <noonedeadpunk> there's a ML asking for some assistance with Central Logging in OSA
15:09:59 <noonedeadpunk> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/B26UYEP7NA5YTCPABLO5TSKU63JLCVJR/
15:10:43 <jrosser> i also did reply to mariusz about mcapi on debian but did not have change to test that myself yet
15:10:55 <noonedeadpunk> I was going to answer something about ELK role, but then thought if maybe jrosser you had some ideas about what reply would be the best
15:11:14 <jrosser> yeah, elk role is kind of expert level stuff somehow
15:11:31 <noonedeadpunk> yeah, I didn't come to a point of testing capi yet :(
15:11:51 <jrosser> hah no worries, at this moment i am working on tooling for offline installs
15:12:27 <noonedeadpunk> so - better say that you can grab journald from metal hosts with whatever you want, but also we have elk role just in case?
15:12:56 <jrosser> yes so it depends - one aspect is inventory integration
15:13:12 <jrosser> perhaps thats something we should have an example of, how to deploy some easy collector but not a whole stack
15:13:37 <jrosser> and then the other aspect is deploying a whole elk stack which is way more scope than that
15:13:48 <noonedeadpunk> yeah...
15:14:11 <noonedeadpunk> though I think deploying elk stack is also quite valuable
15:14:23 <jrosser> yes, the roles are pretty complete and very flexible
15:14:40 <noonedeadpunk> ok, I will try come up with something unless you want to :)
15:14:43 <jrosser> but they do only get very irregular maintainance
15:14:56 <jrosser> sure no problem
15:15:15 <noonedeadpunk> I was going on to check on mariadb 11.4 - but didn't manage yet
15:16:08 <jrosser> right i did have some time on that
15:16:16 <noonedeadpunk> and also - by end of the week (or likely somewhere during weekends) we'd need to have a bugfix release - or at least sha bump proposed. Will see how things land, but would be nice to also land known backports until then
15:16:39 <jrosser> ^ so on that andrewbonney is just preparing for a lab deploy of C
15:16:46 <jrosser> we may find some thing this week
15:16:51 <noonedeadpunk> (how things land for services)
15:17:01 <jrosser> and on mariadb the issue seems to be around TLS
15:17:07 <noonedeadpunk> yep
15:17:16 <jrosser> well, first the client name is changed mariadb<>myqsl but that is an easy patch
15:17:55 <jrosser> but the startup script fails due to tls trouble, i think we drop our custom certs and something is broken with the client not knowing about the ca
15:17:56 <noonedeadpunk> But tls is kinda weird
15:18:06 <noonedeadpunk> Yeah
15:18:12 <damiandabrowski> regarding central logging: we also have https://github.com/openstack/openstack-ansible-plugins/tree/master/roles/journald_remote which may be helpful
15:18:24 <noonedeadpunk> Potentially it expects different format or smth
15:18:41 <noonedeadpunk> Oh, yes, that is great point Damian!
15:19:13 <noonedeadpunk> Journald remote on itself not very well developed iirc, but yeah
15:19:30 <noonedeadpunk> There was also some playbook for journal2gelf for graylog
15:19:39 <noonedeadpunk> Which I used... 4y ago
15:22:43 <damiandabrowski> yeah, i remember some issues with logs rotation
15:22:44 <noonedeadpunk> Also, talking about logs... Ovn still does not use journal in our setup...
15:22:47 <damiandabrowski> https://github.com/systemd/systemd/issues/5242
15:22:50 <damiandabrowski> porbably this one
15:24:35 <noonedeadpunk> Also I have patches regarding oslo_concurrency in my backlog
15:25:06 <noonedeadpunk> And, overall, seems this release might be quite busy with adding new os support
15:25:17 <noonedeadpunk> In terms of el10 and Ubuntu 24.04
15:26:01 <noonedeadpunk> But likely makes sense to do el10 for the next one even
15:26:17 <noonedeadpunk> As it will appear too late, imo
15:26:44 <jrosser> yes i have a list too with these on
15:27:09 <jrosser> i can do some - was thinking to look at 24.04 quite soon
15:27:23 <noonedeadpunk> ++, that would be sweet
15:27:41 <jrosser> on also we did a bobcat upgrade last week
15:28:04 <jrosser> and when upgrading nova, performance (horizon responsiveness) seems to be quite a lot worse
15:28:46 <noonedeadpunk> Huh
15:28:48 <jrosser> so there is something odd happening there and we did not yet dig into it
15:28:57 <jrosser> but i know you have done rally stuff in the past?
15:29:19 <noonedeadpunk> Well, we do have one bobcat region, but it's a new one, so hard to judge on performance so far
15:29:23 <jrosser> andrewbonney: was looking at this a small amount
15:29:51 <noonedeadpunk> Well, yeah, I did used rally in CI periodically some time ago
15:30:15 <noonedeadpunk> So have a comparable results and to judge on API responsiveness
15:30:56 <noonedeadpunk> But not now... Though might make total sense to do that before upgrades to caracal then...
15:31:51 <jrosser> i think the first place we might look is the database, perhaps sqlalchemy changes bring somethng there
15:32:55 <noonedeadpunk> Well, it's still sqlalchemy <2 there
15:33:09 <noonedeadpunk> Just some work was done for compatibility with 2.0
15:33:23 <noonedeadpunk> Even in caracal it's not 2.0 yet
15:33:44 <noonedeadpunk> So sqlalchemy should be pretty much same version as for Antelope just in xase
15:33:55 <jrosser> hmmmm
15:34:11 <noonedeadpunk> But mariadb version?
15:34:32 <noonedeadpunk> Did we used same lts as for Antelope?
15:36:06 <jrosser> yes 10.11 in both cases
15:36:38 <noonedeadpunk> Well....
15:36:59 <noonedeadpunk> Some compatibility things could bring regression ofc...
15:37:02 <noonedeadpunk> But dunno
15:40:16 <noonedeadpunk> Sorry, I need to run now
15:40:38 <noonedeadpunk> So if anything - let's catch up slightly later :)
15:41:28 <noonedeadpunk> #endmeeting