15:03:02 <noonedeadpunk> #startmeeting openstack_ansible_meeting 15:03:02 <opendevmeet> Meeting started Tue Jun 25 15:03:02 2024 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:03:02 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:03:02 <opendevmeet> The meeting name has been set to 'openstack_ansible_meeting' 15:03:07 <noonedeadpunk> #topic office hours 15:03:10 <jrosser> o/ hello 15:03:13 <noonedeadpunk> o/ 15:03:15 <noonedeadpunk> hey 15:03:30 <damiandabrowski> hi! 15:03:38 <noonedeadpunk> I clean forgot that I will need to leave shortly - so let's make this quick :) 15:04:16 <noonedeadpunk> So ceph-ansible 9.0 issue I think Damian you was working on 15:04:26 <noonedeadpunk> and had some progress from what I recall? 15:04:29 <NeilHanlon> o/ - am only kinda around today 15:04:29 <noonedeadpunk> sorry, 8.0 15:04:35 <noonedeadpunk> ++ 15:05:00 <jrosser> yes it looks like there was a renaming of the service for rgw which we need to accomodate 15:05:01 <damiandabrowski> yeah, this patch is responsible for the issue: https://github.com/ceph/ceph-ansible/commit/1121e6d98a41a913886240bcb873380008afcafc 15:05:18 <damiandabrowski> I proposed a solution above ^ 15:05:53 <jrosser> would you like to make a patch - else i can do that? 15:06:52 <damiandabrowski> if we apply a fix in a separate patch, we'll probably have circular dependency :/ 15:07:02 <damiandabrowski> i can push a commit to your change if you don't have anything against it 15:07:04 <damiandabrowski> https://review.opendev.org/c/openstack/openstack-ansible/+/921976 15:07:13 <jrosser> sure thats no problem, i don't mind 15:07:57 <noonedeadpunk> so, I guess I'd define `rgw_zone` to value of ceph_cluster_name if it makes any sense? 15:08:05 <noonedeadpunk> https://opendev.org/openstack/openstack-ansible-ceph_client/src/branch/master/defaults/main.yml#L52-L54 15:08:11 <jrosser> i think `default` 15:08:30 <jrosser> you may have multiple zone groups in one cluster 15:08:38 <noonedeadpunk> aha 15:08:43 <damiandabrowski> okok, I'll try to figure out how to apply the fix in a separate change but if I won't be able to come up with anything wise, I'll just push a commit to your change 15:08:45 <noonedeadpunk> ok, fair enough 15:09:01 <jrosser> you can just edit my change, that will be easier 15:09:10 <noonedeadpunk> nah, you should edit Jonathan's one 15:09:15 <noonedeadpunk> otherwise it won't pass CI 15:09:15 <damiandabrowski> ok 15:09:29 <jrosser> thankyou for finding that :) 15:09:31 <damiandabrowski> I also need to figure out how to not break environments that use recent OSA version with ceph-ansible 7.0 15:09:44 <damiandabrowski> no problem at all ;) 15:09:48 <noonedeadpunk> there's a ML asking for some assistance with Central Logging in OSA 15:09:59 <noonedeadpunk> #link https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/B26UYEP7NA5YTCPABLO5TSKU63JLCVJR/ 15:10:43 <jrosser> i also did reply to mariusz about mcapi on debian but did not have change to test that myself yet 15:10:55 <noonedeadpunk> I was going to answer something about ELK role, but then thought if maybe jrosser you had some ideas about what reply would be the best 15:11:14 <jrosser> yeah, elk role is kind of expert level stuff somehow 15:11:31 <noonedeadpunk> yeah, I didn't come to a point of testing capi yet :( 15:11:51 <jrosser> hah no worries, at this moment i am working on tooling for offline installs 15:12:27 <noonedeadpunk> so - better say that you can grab journald from metal hosts with whatever you want, but also we have elk role just in case? 15:12:56 <jrosser> yes so it depends - one aspect is inventory integration 15:13:12 <jrosser> perhaps thats something we should have an example of, how to deploy some easy collector but not a whole stack 15:13:37 <jrosser> and then the other aspect is deploying a whole elk stack which is way more scope than that 15:13:48 <noonedeadpunk> yeah... 15:14:11 <noonedeadpunk> though I think deploying elk stack is also quite valuable 15:14:23 <jrosser> yes, the roles are pretty complete and very flexible 15:14:40 <noonedeadpunk> ok, I will try come up with something unless you want to :) 15:14:43 <jrosser> but they do only get very irregular maintainance 15:14:56 <jrosser> sure no problem 15:15:15 <noonedeadpunk> I was going on to check on mariadb 11.4 - but didn't manage yet 15:16:08 <jrosser> right i did have some time on that 15:16:16 <noonedeadpunk> and also - by end of the week (or likely somewhere during weekends) we'd need to have a bugfix release - or at least sha bump proposed. Will see how things land, but would be nice to also land known backports until then 15:16:39 <jrosser> ^ so on that andrewbonney is just preparing for a lab deploy of C 15:16:46 <jrosser> we may find some thing this week 15:16:51 <noonedeadpunk> (how things land for services) 15:17:01 <jrosser> and on mariadb the issue seems to be around TLS 15:17:07 <noonedeadpunk> yep 15:17:16 <jrosser> well, first the client name is changed mariadb<>myqsl but that is an easy patch 15:17:55 <jrosser> but the startup script fails due to tls trouble, i think we drop our custom certs and something is broken with the client not knowing about the ca 15:17:56 <noonedeadpunk> But tls is kinda weird 15:18:06 <noonedeadpunk> Yeah 15:18:12 <damiandabrowski> regarding central logging: we also have https://github.com/openstack/openstack-ansible-plugins/tree/master/roles/journald_remote which may be helpful 15:18:24 <noonedeadpunk> Potentially it expects different format or smth 15:18:41 <noonedeadpunk> Oh, yes, that is great point Damian! 15:19:13 <noonedeadpunk> Journald remote on itself not very well developed iirc, but yeah 15:19:30 <noonedeadpunk> There was also some playbook for journal2gelf for graylog 15:19:39 <noonedeadpunk> Which I used... 4y ago 15:22:43 <damiandabrowski> yeah, i remember some issues with logs rotation 15:22:44 <noonedeadpunk> Also, talking about logs... Ovn still does not use journal in our setup... 15:22:47 <damiandabrowski> https://github.com/systemd/systemd/issues/5242 15:22:50 <damiandabrowski> porbably this one 15:24:35 <noonedeadpunk> Also I have patches regarding oslo_concurrency in my backlog 15:25:06 <noonedeadpunk> And, overall, seems this release might be quite busy with adding new os support 15:25:17 <noonedeadpunk> In terms of el10 and Ubuntu 24.04 15:26:01 <noonedeadpunk> But likely makes sense to do el10 for the next one even 15:26:17 <noonedeadpunk> As it will appear too late, imo 15:26:44 <jrosser> yes i have a list too with these on 15:27:09 <jrosser> i can do some - was thinking to look at 24.04 quite soon 15:27:23 <noonedeadpunk> ++, that would be sweet 15:27:41 <jrosser> on also we did a bobcat upgrade last week 15:28:04 <jrosser> and when upgrading nova, performance (horizon responsiveness) seems to be quite a lot worse 15:28:46 <noonedeadpunk> Huh 15:28:48 <jrosser> so there is something odd happening there and we did not yet dig into it 15:28:57 <jrosser> but i know you have done rally stuff in the past? 15:29:19 <noonedeadpunk> Well, we do have one bobcat region, but it's a new one, so hard to judge on performance so far 15:29:23 <jrosser> andrewbonney: was looking at this a small amount 15:29:51 <noonedeadpunk> Well, yeah, I did used rally in CI periodically some time ago 15:30:15 <noonedeadpunk> So have a comparable results and to judge on API responsiveness 15:30:56 <noonedeadpunk> But not now... Though might make total sense to do that before upgrades to caracal then... 15:31:51 <jrosser> i think the first place we might look is the database, perhaps sqlalchemy changes bring somethng there 15:32:55 <noonedeadpunk> Well, it's still sqlalchemy <2 there 15:33:09 <noonedeadpunk> Just some work was done for compatibility with 2.0 15:33:23 <noonedeadpunk> Even in caracal it's not 2.0 yet 15:33:44 <noonedeadpunk> So sqlalchemy should be pretty much same version as for Antelope just in xase 15:33:55 <jrosser> hmmmm 15:34:11 <noonedeadpunk> But mariadb version? 15:34:32 <noonedeadpunk> Did we used same lts as for Antelope? 15:36:06 <jrosser> yes 10.11 in both cases 15:36:38 <noonedeadpunk> Well.... 15:36:59 <noonedeadpunk> Some compatibility things could bring regression ofc... 15:37:02 <noonedeadpunk> But dunno 15:40:16 <noonedeadpunk> Sorry, I need to run now 15:40:38 <noonedeadpunk> So if anything - let's catch up slightly later :) 15:41:28 <noonedeadpunk> #endmeeting