13:00:24 <mnasiadka> #startmeeting kolla
13:00:24 <opendevmeet> Meeting started Wed Sep 25 13:00:24 2024 UTC and is due to finish in 60 minutes.  The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:24 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:24 <opendevmeet> The meeting name has been set to 'kolla'
13:00:29 <mnasiadka> #topic rollcall
13:00:30 <mnasiadka> o/
13:00:32 <SvenKieske> o/
13:00:33 <jovial> \o
13:00:34 <bbezak> o/
13:00:37 <kevko> \o/
13:00:43 <mhiner> o/
13:00:47 <r-krcek> 0/
13:01:35 <mnasiadka> #topic agenda... (full message at <https://matrix.org/oftc/media/v1/media/download/AfvCuoxbKukDvfMYa1XqelHiI6JA2oy1yrxh5OX-liMapP23Mk5VHpYGNwZDdFVV4TB1g_TTAeUxVm6vFfY2Ur9CeSKv1GuQAG1hdHJpeC5vcmcvSEVTTFBoWldjanVBSWtJT0xKSWlqV0xS>)
13:01:38 <mnasiadka> #topic CI status
13:01:49 <mnasiadka> Green? Red? Amber? Watermelon?
13:02:37 <frickler> o/
13:02:50 <frickler> that agenda topic was mangled in the matrix
13:02:53 <SvenKieske> what about honeydew melon? :D
13:03:14 <mnasiadka> Yeah, switched to Element, probably it's mangling ;-)
13:03:23 <mnasiadka> I'll try better next time
13:03:25 <SvenKieske> yeah, you need to post agenda line by line for poor old IRC
13:03:28 <mnasiadka> CI looks not bad, except aarch64
13:04:32 <mnasiadka> frickler: do you know if there's any outlook on aarch64 improvements?
13:04:59 <frickler> mnasiadka: none that I'm aware of
13:05:08 <mnasiadka> ok, it is what it is
13:05:16 <mnasiadka> #topic Current cycle planning
13:06:17 <mnasiadka> So, Ubuntu 24.04 seems to be closer, some small CI failures to revisit and I guess I'll ask for reviews next week
13:06:21 <mnasiadka> anything else major that we should be thinking of?
13:06:27 <mnasiadka> There's RMQ 4.0
13:06:37 <mnasiadka> As in no patches, but it got released
13:07:00 <mnasiadka> Since 4.0 release - 3.13 community support has ended
13:07:12 <SvenKieske> yes, we need to shift everything over from the old queues afaik for that to work, so afaik we need to finish the fanout/reply queue stuff
13:07:23 <mnasiadka> Sounds fantastic
13:07:31 <SvenKieske> for that to work in turn, we need the queue manager which kevko is working on
13:07:40 <opendevreview> Michal Arbet proposed openstack/kolla-ansible master: Drop support for py38, py39  https://review.opendev.org/c/openstack/kolla-ansible/+/929408
13:07:40 <opendevreview> Michal Arbet proposed openstack/kolla-ansible master: Add ansible-core as a dependency  https://review.opendev.org/c/openstack/kolla-ansible/+/922369
13:07:53 <SvenKieske> but he seems busy :P ;)
13:08:13 <kevko> with queue manager it's little bit hard ..we need to bind /dev/shm to containers using oslo.messaging
13:08:28 <mnasiadka> is that secure?
13:08:45 <SvenKieske> just..bind mount it? :) /dev/shm is actually just a fancy way to say "tmpfs".
13:09:07 <kevko> well, i was thinking about the option to create tmpfs and mount as /dev/shm into container
13:09:14 <SvenKieske> I hope oslo has guarded it and there are no race conditions with concurrent access :D
13:09:20 <kevko> but then I realized we have already some containers which has /dev/shm already mounted
13:09:42 <mnasiadka> Isn't --ipc=host for that?
13:09:44 <kevko> and ...moreover ...in classic openstack installations from packages ..there is shared memory also ...
13:09:56 <kevko> no, ipc turning more than /dev/shm
13:10:28 <kevko> but yes ..if you set ipc_mode ...it's effectively turn on bind /dev/shm into container
13:10:40 <kevko> but we don't need it ...
13:10:44 <mnasiadka> there's the alternative of shared ipc for containers
13:11:06 <mnasiadka> but that was a bit... tricky, because you start a container pointing to other running containers ipc
13:11:09 <frickler> maybe we should stop doing containers if we share everything again anyway in the end? (almost, but not completely non-serious)
13:11:10 <SvenKieske> I think the short term conclusion is we need some time before we can truly merge rmq 4?
13:11:18 <mnasiadka> yeah, that's another point
13:11:39 <mnasiadka> if we need shared mem for all containers, why even containers?
13:12:37 <mnasiadka> so, is that bloody queue manager completely required for RMQ 4?
13:13:30 <SvenKieske> well if we wanna split hairs it's afaik not required for rmq, but for how oslo/openstack uses rmq :)
13:13:44 <kevko> frickler: It is still an effective way to deploy OpenStack services and, for example, run two different versions on one system... it's containerized. Anyway... we use also like net=host namespace...it's similar
13:14:00 <mnasiadka> the oslo.messaging patch says it's optional
13:14:01 <kevko> frickler: But yes, I understand your comment.. :)
13:14:19 <SvenKieske> what it does is essentially keeping track of queue names, so they always stay the same "per host" (that's the tricky part with containers). so consumers can just always bind to the same queue
13:14:58 <kevko> mnasiadka: it's optional yes ... but we also provides ability to override configuration per user ...so if user choose to override and turn on the queue_manager ...he brokes his cloud because we are kolla ..right ?
13:15:08 <SvenKieske> xD
13:15:12 <kevko> and it's also very nice feature :)
13:15:26 <mnasiadka> ok, but it doesn't sound like required
13:15:43 <SvenKieske> well for first release we could put big fat warning that queue manager is not yet supported? if it works?
13:15:46 <kevko> so should we  add precheck if user override includes queue_manager = true and fail the reconfigure ? :D
13:16:05 <mnasiadka> user overrides mean user knows what he's doing
13:16:06 <kevko> but agree that it's not important issue for now
13:16:20 <mnasiadka> we never checked what is in user overrides
13:16:28 <kevko> don't forget it's also requirement for streams for fanout
13:16:34 <mnasiadka> so let's not create a drama
13:16:56 <kevko> agree ...
13:17:23 <kevko> it's mainly my downstream project which i am (as always) tracking in gerrit ... and I like that feature
13:18:35 <mnasiadka> Well, if we can create a tmpfs volume and mount that as shm, that would probably be a bit better
13:19:56 <kevko> That's most clean i think ...but i need to test if  - when you turn on the ipc if it's not automatically mounted host /dev/shm and overrides your tmpfs mount
13:20:19 <mnasiadka> well, we only run libvirt with ipc=host IIRC
13:20:26 <mnasiadka> that doesn't interact with RMQ
13:20:43 <kevko> hacluster ?
13:20:53 <kevko> cinder ?
13:21:09 <kevko> masakari ?
13:21:12 <mnasiadka> ah, and nova-compute
13:21:16 <kevko> manila ?
13:21:36 <mnasiadka> well then let's drop containers
13:21:37 <mnasiadka> :)
13:21:41 <kevko> we already has bunch of containers host's /dev/shm accessible
13:22:16 <kevko> just saying .. :D
13:22:24 <SvenKieske> various neutron stuff also has quite some host access (necessary for SRIOV etc) afaik
13:22:39 <mnasiadka> Sounds like a lot of work
13:22:49 <mnasiadka> Maybe it's easier to write a container-friendly queue_manager? :)
13:22:52 <SvenKieske> it's just inherent complexity, the k8s deployment tools are basically doing the same thing
13:24:44 <mnasiadka> so then it probably means we need to identify which containers need to have "/dev/shm" mounted
13:24:52 <kevko> Containerization is still great for delivering code; we don't necessarily have to use everything. Moreover, we also copy config files into the container instead of using binds as it should be... for example.
13:24:52 <mnasiadka> and that's it
13:25:38 <kevko> I just want to say that we, as Kolla, definitely don't do everything the way it should be done... and with the complexity of OpenStack, I don't think it's even possible.
13:25:45 <mnasiadka> well, binds have the problem of permissions
13:26:10 <mnasiadka> so probably that's why the project started with this
13:26:38 <mnasiadka> anyway, we have probably other priorities this cycle
13:27:22 <kevko> Anyway, I will work on it and track it in Gerrit. It's probably not worth discussing it much. I’d rather ask about ProxySQL. I’ve fixed everything, including IPv6, and I think it’s OK to switch.
13:27:22 <mnasiadka> MariaDB bump is not happening, unless they release a new version of 11.4 where they fixed SST between old MariaDB and new MariaDB
13:28:52 <mnasiadka> Well, I think I commented we need to remove the unused mariadb-clustercheck running container in upgrade tasks
13:29:11 <mnasiadka> if we're changing the default
13:29:48 <kevko> mnasiadka: yeah, I replied that it can be used by external scripts maybe ... and in reno it's specified that it can be removed manually
13:30:12 <kevko> mnasiadka: but I agree ... i will write something in followup ..so i will add on the top of relation chain ..
13:30:20 <mnasiadka> ok then
13:30:28 <kevko> +1
13:30:57 <mnasiadka> I've added my +2
13:31:11 <mnasiadka> https://review.opendev.org/c/openstack/kolla-ansible/+/913724
13:31:25 <mnasiadka> Do we need to backport any of the fixes you had to do?
13:32:03 <kevko> yep ..we should one
13:32:35 <kevko> this one https://review.opendev.org/c/openstack/kolla-ansible/+/929848 only
13:32:40 <kevko> it's just ipv6 ...
13:33:30 <kevko> nothing more ..because that dance with --initial and removal of db file << ..this was only in master ...lower versions still working fine ...
13:33:55 <kevko> and maybe this ... https://review.opendev.org/c/openstack/kolla-ansible/+/930077/4 << but this sounds like a feature ....
13:34:06 <kevko> mnasiadka: thanks
13:34:36 <opendevreview> Merged openstack/kolla-ansible stable/2024.1: Fix links in docs for unmaintained releases  https://review.opendev.org/c/openstack/kolla-ansible/+/928579
13:34:38 <opendevreview> Merged openstack/kolla-ansible stable/2023.2: Fix links in docs for unmaintained releases  https://review.opendev.org/c/openstack/kolla-ansible/+/928581
13:35:14 <opendevreview> Merged openstack/kolla-ansible stable/2023.1: Fix links in docs for unmaintained releases  https://review.opendev.org/c/openstack/kolla-ansible/+/928587
13:35:14 <mnasiadka> frickler, SvenKieske : around backporting 930077 - it would be useful for my $downstream - are you fine with doing that?
13:35:33 <mnasiadka> doesn't include any inventory changes, so won't include any problems hopefully
13:36:22 <kevko> nice to hear that you also using ProxySQL ;-)
13:36:29 <mnasiadka> Not yet, but planning to.
13:36:35 <kevko> ah, ok
13:36:36 <SvenKieske> we need an exception for those, no? via ML? but looks okayish I think
13:36:38 <mnasiadka> And monitoring is an important aspect
13:37:15 <mnasiadka> kevko: propose the backports please for the ipv6 and the prometheus endpoint via Gerrit and we'll sort it then
13:37:16 <kevko> ML ? what is the shortcut for ? :D
13:37:21 <mnasiadka> mailing list
13:37:25 <kevko> ah ok
13:37:26 <mnasiadka> but no, never did it that way
13:37:28 <kevko> mnasiadka: ack
13:37:52 <mnasiadka> #topic Additional agenda (from whiteboard)
13:38:13 <mnasiadka> mhiner - please review: Move to high level client in DockerWorker https://review.opendev.org/c/openstack/kolla-ansible/+/908295
13:38:19 <opendevreview> Grzegorz Koper proposed openstack/kolla-ansible stable/2023.1: Fixing typo in etc/kolla/globals.yml  https://review.opendev.org/c/openstack/kolla-ansible/+/930466
13:39:25 <mnasiadka> mhiner: commented, will also do a proper review
13:39:34 <mhiner> thanks
13:39:40 <opendevreview> Grzegorz Koper proposed openstack/kolla-ansible stable/2023.2: Fixing typo in etc/kolla/globals.yml  https://review.opendev.org/c/openstack/kolla-ansible/+/930467
13:39:52 <mnasiadka> r-krcek:  https://review.opendev.org/c/openstack/kolla-ansible/+/923110
13:40:04 <opendevreview> Grzegorz Koper proposed openstack/kolla-ansible stable/2024.1: Fixing typo in etc/kolla/globals.yml  https://review.opendev.org/c/openstack/kolla-ansible/+/930468
13:41:16 <mnasiadka> ok, commented - would be nice to get a python based CLI this cycle I guess
13:42:12 <mnasiadka> SvenKieske: I've seen you did some reviews - can you help r-krcek to get those over the line?
13:42:40 <mnasiadka> Another one from r-krcek - https://review.opendev.org/c/openstack/kolla-ansible/+/599735
13:43:38 <mnasiadka> reviewed
13:43:56 <SvenKieske> yeah, probably over the next days, still stuck in upgrading stuff, sadly have considerable less time currently during working hours for reviews :-/
13:44:15 <SvenKieske> will make some room for reviews in my free time I guess
13:44:53 <mnasiadka> Last one is a question if anybody is interested in Swift and would like to help defining tests in https://review.opendev.org/c/openstack/kolla-ansible/+/797498
13:46:23 <mnasiadka> If yes, wuchunyang and r-krcek would be probably grateful
13:46:25 <mnasiadka> #topic Open discussion
13:46:30 <mnasiadka> anything else to cover today?
13:47:16 <r-krcek> Thank you guys for the reviews and comments :)
13:48:31 <kevko> i would like to ask again for some reviews ...but you know which one right :D
13:49:04 <mnasiadka> r-krcek: if you are interested in being a more permanent part of Kolla community - given your not small contributions - you could review other people patches and become a core some day
13:49:25 <mnasiadka> kevko: yeah, will have a look on your patches tomorrow
13:50:15 * frickler will be on holiday for the coming three weeks, just fyi. will be back in time for the ptg
13:50:39 <mnasiadka> Remember PTG for Kolla starts on Tue - I already booked the slots
13:50:46 <frickler> ack
13:50:48 <mnasiadka> #link https://etherpad.opendev.org/p/kolla-epoxy-ptg
13:50:48 <kevko> ack
13:50:54 <mnasiadka> Please try to populate the topics
13:50:59 <mnasiadka> we have 2x3 hours
13:51:08 <mnasiadka> (and there's kayobe as well)
13:51:10 <mnasiadka> jovial: ^^
13:51:38 <mnasiadka> Ok then, that's all for me - see you next week (have fun frickler !)
13:51:55 <jovial> awesome - I'll populate a few topics. Would definitely appreciate any input there.
13:51:59 <SvenKieske> see you :)
13:52:15 <mmalchuk> o/
13:52:20 <mnasiadka> #endmeeting