14:00:07 <bbezak> #startmeeting kolla
14:00:07 <opendevmeet> Meeting started Wed Oct 30 14:00:07 2024 UTC and is due to finish in 60 minutes.  The chair is bbezak. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:07 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:07 <opendevmeet> The meeting name has been set to 'kolla'
14:00:32 <bbezak> #topic rollcall
14:00:38 <SvenKieske> o/
14:00:41 <frickler> \o
14:00:44 <Pcmalih_> Having hard time booting node using ironic (redfish)
14:00:52 <mattcrees[m]> o/
14:00:54 <kevko> \o/
14:01:27 <mnasiadka> o/
14:01:34 <luca_> kevko: i did not yet tried a custom policy file, my issue is that with default ironic policy ironic-inspector service created by kolla is not able to create port, the policy in question is baremetal:port:create (role:admin and system_scope:all) or (role:service and system_scope:all) and i am not sure on how to fix it
14:01:41 <Pcmalih_> Wondering if ironic really works with baremetal when deployed using kolla-Ansible?
14:02:04 <mnasiadka> Guys, there's a meeting
14:02:08 <bbezak> luca_, Pcmalih_ let's discuss that after the meeting
14:02:10 <mnasiadka> Questions after the meeting
14:02:14 <luca_> ok
14:02:25 <Pcmalih_> Appreciate if anyone share their experience.
14:02:41 <mnasiadka> #topic agenda
14:02:52 <mnasiadka> * CI status
14:03:01 <mnasiadka> * Release tasks
14:03:13 <mnasiadka> * Open discussion
14:03:25 <mnasiadka> #topic CI status
14:03:32 <mnasiadka> Seems it's sort of green, Kayobe has some clock sync issues, but apart of that looks ok
14:03:41 <mnasiadka> #topic Release tasks
14:03:52 <mnasiadka> the only task now is to get from rc1 to final for Dalmatian
14:04:12 <mnasiadka> there are some Kayobe related patches that we merged, and once Kayobe is all green - I'll post rc2 and then final
14:04:30 <mnasiadka> Once we have final, we need to remember about the deploy guide patch
14:04:47 <mnasiadka> Anything I missed regarding making final release?
14:05:26 <kevko> Well, I don't even want to reach out anymore. :)
14:05:41 <mnasiadka> reach out?
14:06:01 <kevko> https://review.opendev.org/c/openstack/kolla/+/915440 < second +2 missing
14:06:29 <mnasiadka> Well, this needs to wait for Epoxy
14:06:41 <kevko> okay, no problem
14:07:20 <mnasiadka> ok then
14:08:10 <mnasiadka> Next week probably we can look at those unmerged patches by kevko and others - once we have final release
14:08:20 <mnasiadka> #topic Open discussion
14:08:22 <mnasiadka> Anybody anything?
14:09:06 <chembervint> Hi. yes, I have one question ... just a second :)
14:09:11 <mattcrees[m]> Could we priorities some reviews for this patch please? Upgrades to Caracal break on Octavia without it right now: https://review.opendev.org/c/openstack/kolla-ansible/+/932408
14:10:01 <mnasiadka> kevko frickler bbezak can you have a look? ^^
14:11:09 <jovial> I have a couple of topics after we've finished with this one
14:11:14 <frickler> ack, on it
14:11:20 <chembervint> we are currently looking for a ability to control OOM score for kolla containers. we are ready to add it to kolla. couple of questions: 1) is there difference to put in into systemd units or into docker\podman-worker modeule 2) do we need only OOM-score or OOM-disable also? 3) do we need ability to adjust OOM-score for each container, or kolla-wide?
14:12:30 <kevko> yep
14:13:24 <mnasiadka> chembervint: you're being extremely vague, but I assume you want to put OOMScoreAdjust=xxx in systemd unit?
14:14:26 <mnasiadka> I mean probably nobody currently has insight what is possible in docker/podman, and what is possible in systemd apart that variable
14:14:53 <mnasiadka> So you'd need to tell us about pros and cons
14:14:54 <chembervint> mnasiadka: either OOMScoreAdjust=xxx in systemd unit or oom_score_adj into kolla_dcocker.py (docker run option)
14:15:11 <mnasiadka> does podman have the same option?
14:15:34 <chembervint> yes, like a docker - the same
14:15:58 <mnasiadka> so which one is a better solution?
14:17:00 <SvenKieske> :)
14:17:02 <kevko> mnasiadka: hmm, shouldn't we move creation of database to some separate task and call in upgrade and also bootstrap ?
14:17:27 <mnasiadka> kevko: this is a one off, it's needed only in that stable branch
14:17:38 <frickler> kevko: this is a one off only in 2023.2 and 2024.1, so worth no extra effort IMO
14:17:45 <kevko> mnasiadka: because if we merge that https://review.opendev.org/c/openstack/kolla-ansible/+/932408 ...we will have same tasks in two files ... in bootstrap.yaml and upgrade.yaml
14:17:49 <mnasiadka> I'm fine if anybody wants to go the extra mile on this, but I don't :)
14:17:57 <chembervint> I'm not sure for the moment. I've wanted to discuss it with you :) but ok - I will research more on systemd vs docker - where we have to apply this. And what do you think - do we need ability to adjust it for each kolla container, or kolla-wide setting will be enough?
14:18:29 <mnasiadka> chembervint: let's start with kolla-wide, no need to overcomplicate if there's no requirement
14:18:49 <frickler> chembervint: what's your use case for this in general? don't you have enough memory on your controllers?
14:18:58 <kevko> mnasiadka: well, what I am trying to say ...we can call creation of DB and user everytime ...during the config phase ...or somewhere around ...as that ansible module works fine ...create if db not exist ..and just touch if already exist
14:19:13 <chembervint> ok! thank you! will do my best and bring the patch soon
14:20:06 <mnasiadka> kevko: if you want to rework all roles in master - I'm fine with that, but then we won't backport this - this is a one off for upgrade from Antelope to Bobcat/Caracal, and it's perfectly fine it's not perfect
14:20:12 <kevko> i mean ..every time in context deploy, config, upgrade ... (in deploy phase before boostrap-service)
14:21:45 <kevko> hmm, okay ..i am just trying to review every time :D
14:21:57 <SvenKieske> mhm
14:22:31 <mnasiadka> Added +w
14:22:37 <mnasiadka> jovial: your turn?
14:23:06 <jovial> thanks. Firstly just an update from the PTG. I emailed openstack-dicusss about the recommended tooz driver: https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/JTYWG37I4OS7QSVIPY5DPMF5SFTVMY5I/
14:23:26 <mnasiadka> No answers at all :)
14:23:38 <jovial> Indeed
14:23:43 <SvenKieske> why is this octavia thing only a single release thing though? I mean I believe you, but I don't understand the issue it seems :D
14:24:06 <jovial> I didn't miss any important tags in the subject did I?
14:24:14 <SvenKieske> re: tooz: it might be better to ping individual contributors to that codebase? :)
14:24:21 <mnasiadka> we introduced a new database in MariaDB just for Octavia, we didn't add creating it to upgrade, just to deploy
14:24:23 <kevko> SvenKieske: persistence_database (jobboard) was added in some release ... don't remember which
14:24:29 <chembervint> frickler: we have a lot of memory on controllers. but, you know, different services could waste a huge amount of memory esecially if they experiencing the problems. bugs, memory leaks ... we've faced such problems even with a ethernet kernel module how eat all memory ..
14:25:02 <mnasiadka> jovial: no, I'll try to ask Oslo folks, maybe we'll get an answer
14:25:05 <mnasiadka> jovial: anything else?
14:25:15 <SvenKieske> jovial: well you could have added some context to your mail: without it, it just reads like some curiosity, not that you really need the information urgently :)
14:25:52 <jovial> Just a request to get this patch merged for 2024.2: https://review.opendev.org/c/openstack/kolla-ansible/+/920294 (might be a long shot)
14:25:53 <SvenKieske> but also might be oslo people didn't read it, maybe tagging it with oslo might move more eyes
14:26:16 <mnasiadka> well, posted something on oslo channel, let's see
14:26:32 <jovial> thanks mnasiadka - it would be good to get an answer :)
14:26:39 <frickler> the oslo team is very small, similar to reqs team
14:27:28 <chembervint> jovial: our experience is to use tooz + redis - redis is much more faster than a etcd. kolla os already ready for redis. it works really fine. and also we've tested tooz+consul - also works fine, if you already using the consul.
14:27:38 <kevko> +1
14:27:39 <mnasiadka> jovial: I'm afraid it's rc1, I wouldn't like to make the release process more complicated - can we come back to it in 2025.1 (and if it's a bug - we could backport)
14:28:21 <mnasiadka> I think so many people use Redis, and I think Octavia jobboard (at least in kolla-ansible) requires redis, so we shouldn't really overcomplicate things
14:28:33 <jovial> Yeah, fair enough. I'm calling it a bug, but its a matter of interpretation there :laughs:
14:28:36 <kevko> if not all people :D
14:28:48 <mnasiadka> Once valkey is in Debian packages - we'll move to valkey in 2025.1 and that's it
14:29:12 <kevko> Hmm, maybe I should create and build valkey for debian :)
14:29:13 <mnasiadka> ok, anything else? :)
14:29:33 <mnasiadka> kevko: it's built for unstable, but seems it's not on the FTP - I have no clue why it takes so long
14:30:07 <kevko> mnasiadka: it's in trixie already
14:30:23 <mnasiadka> ok, it wasn't there like two weeks ago
14:30:46 <kevko> mnasiadka: yeah, it's debian policy to have some package in unstable for some time ..then it's automatically moved
14:31:11 <mnasiadka> Do we need to think about moving to Trixie in 2025.1? When is it due for releasing?
14:32:08 <kevko> 2025 i think
14:32:33 <mnasiadka> Ok, then maybe we need to jump in 2025.1, we'll see
14:32:38 <kevko> june maybe :)
14:32:43 <mnasiadka> or in 2025.2
14:32:49 <mnasiadka> ok then
14:32:53 <mnasiadka> I see no more topics
14:33:12 <mnasiadka> Thanks for coming guys
14:33:24 <mnasiadka> jovial: let's focus on kayobe rc1 and kolla-ansible rc2
14:33:36 <mnasiadka> See you next week
14:33:38 <mnasiadka> #endmeeting
14:34:01 <frickler> bbezak needs to end it
14:34:12 <bbezak> #endmeeting