| *** logan_ is now known as Guest31561 | 03:28 | |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug rocky zuul_console issue https://review.opendev.org/c/openstack/kolla-ansible/+/963657 | 07:06 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug rocky zuul_console issue https://review.opendev.org/c/openstack/kolla-ansible/+/963657 | 07:06 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: DNM: Debug rocky zuul_console issue https://review.opendev.org/c/openstack/kolla-ansible/+/963657 | 07:07 |
| opendevreview | Merged openstack/kolla master: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967475 | 07:09 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.2: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967656 | 07:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.1: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967657 | 07:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.1: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967657 | 07:27 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2024.2: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967658 | 07:27 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2024.1: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967659 | 07:28 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2024.1: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967659 | 07:28 |
| mnasiadka | Ok then, let’s see | 07:30 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2025.2: Fix versions in Ansible bump release note https://review.opendev.org/c/openstack/kayobe/+/967667 | 09:53 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2025.2: Bump minimum Python version to 3.12 https://review.opendev.org/c/openstack/kayobe/+/967668 | 09:53 |
| opendevreview | Pierre Riteau proposed openstack/kayobe stable/2025.2: Move nvidia.nvue to 1.2.9 for cumulus 5.13 https://review.opendev.org/c/openstack/kayobe/+/967669 | 09:54 |
| opendevreview | Merged openstack/kolla stable/2024.2: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967658 | 10:53 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Debug space issues https://review.opendev.org/c/openstack/kolla/+/967675 | 10:58 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 11:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Debug space issues https://review.opendev.org/c/openstack/kolla/+/967675 | 11:40 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 11:45 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 12:46 |
| *** EugenMayer4401806 is now known as EugenMayer440180 | 13:19 | |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: DNM: Debug space issues https://review.opendev.org/c/openstack/kolla/+/967675 | 13:26 |
| mnasiadka | mnasiadka bbezak frickler kevko mmalchuk gkoper jovial mattcrees dougszu darmach pabloclsn ravlew amir58118 r-krcek BertrandLanson - kolla meeting in 4 minutes | 13:56 |
| mnasiadka | #startmeeting kolla | 14:01 |
| opendevmeet | Meeting started Wed Nov 19 14:01:26 2025 UTC and is due to finish in 60 minutes. The chair is mnasiadka. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:01 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:01 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:01 |
| mnasiadka | #topic rollcall | 14:01 |
| mnasiadka | o/ | 14:01 |
| bbezak | o/ | 14:01 |
| BertrandLanson[m] | hello ! | 14:01 |
| ravlew | o/ | 14:01 |
| mattcrees | o/ | 14:01 |
| darmach | \o | 14:01 |
| AlexWelsh[m] | o/ | 14:02 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 14:02 |
| *** ravlew is now known as Guest31601 | 14:02 | |
| mnasiadka | #topic agenda | 14:02 |
| mnasiadka | * Announcements | 14:03 |
| mnasiadka | * CI status | 14:03 |
| mnasiadka | * Current cycle planning | 14:03 |
| mnasiadka | * Additional agenda (from whiteboard) | 14:03 |
| mnasiadka | * Open discussion | 14:03 |
| mnasiadka | #topic Announcements | 14:03 |
| mnasiadka | Kolla/Kolla-Ansible Flamingo 2025.2 has been released - thank you for all that contributed to that | 14:03 |
| mnasiadka | #topic CI status | 14:04 |
| mnasiadka | So… the CI is well… red | 14:04 |
| mnasiadka | RabbitMQ removed the PPA repo we used, and now we need to switch to their deb repo - that’s in progress | 14:04 |
| mnasiadka | Master should be better now, but stable back ports are still in progress | 14:04 |
| mnasiadka | #link https://review.opendev.org/c/openstack/kolla/+/967475 | 14:05 |
| mnasiadka | 2024.1 is problematic, because the tox jobs check out requirements branch stable/2024.1 (which doesn’t exist anymore, since it’s moved to unmaintained/2024.1) | 14:05 |
| mnasiadka | bbezak: did you raise a patch for moving 2024.1 to unmaintained? | 14:05 |
| bbezak | I will, too many meetings today ;) | 14:06 |
| mnasiadka | Ok then | 14:06 |
| mnasiadka | I’m trying to land that RMQ repo patch also in unmaintained/2023.1 | 14:06 |
| mnasiadka | But there are some other things failing | 14:06 |
| BertrandLanson[m] | I'm assuming the rabbitmq stuff prevents us from building older releases until backported ? | 14:06 |
| mnasiadka | BertrandLanson[m]: yup - the last good known images are on quay.io - but nothing newer ;) | 14:07 |
| mnasiadka | And people locally can’t build the images as well | 14:07 |
| mnasiadka | (Well, they can’t without local changes) | 14:07 |
| mnasiadka | The other thing is - Docker 29 decided to switch default storage engine to containerd | 14:08 |
| BertrandLanson[m] | and anything self-built aswell then, does this warrant new tags for older releases with the fix ? Or do we just install from HEAD of whatever release branch we use ? | 14:08 |
| mnasiadka | That means images are now stored in /var/lib/containerd instead of /var/lib/docker | 14:08 |
| mnasiadka | BertrandLanson[m]: we recommend users to use stable branches directly from git - but surely we should also tag point releases once we merge the fixes | 14:08 |
| mnasiadka | And that means our build jobs run out of storage space, because images get written outside of /var/lib/docker :) | 14:09 |
| mnasiadka | Trying to get that fixed in a-c-k | 14:09 |
| mnasiadka | #link https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 14:09 |
| mnasiadka | No other breakages I know of, unless anybody has something to add to the burning pile :) | 14:09 |
| bbezak | Some reasoning is here #link https://www.docker.com/blog/docker-engine-version-29/ | 14:10 |
| mnasiadka | Ok then, let’s move on | 14:10 |
| mnasiadka | #topic Current cycle planning | 14:10 |
| mnasiadka | Using that RMQ is fixed for master - and 2025.2 final is released - we can now accept/review new features patches | 14:11 |
| mnasiadka | So if anybody has something that needs attention - please add it in the additional agenda on the whiteboard - or post here | 14:11 |
| BertrandLanson[m] | there is the old inventory bug I linked a few days ago for cyborg | 14:12 |
| BertrandLanson[m] | #link https://review.opendev.org/c/openstack/kolla-ansible/+/934429 | 14:12 |
| BertrandLanson[m] | which deploys everything everywhere for no reason since it was introduced | 14:13 |
| mnasiadka | well, we have no cyborg jobs, so nobody would notice | 14:13 |
| mnasiadka | Out of interest - what do you use Cyborg for? | 14:13 |
| mnasiadka | bbezak: it already has my +2 - can you have a look? | 14:13 |
| bbezak | ok | 14:14 |
| BertrandLanson[m] | we used to use it rather than nova for passing NVME directly to VMs | 14:14 |
| mnasiadka | Ok, does it clean up these NVMes? | 14:14 |
| BertrandLanson[m] | now we can do it with nova but... legacy | 14:14 |
| mnasiadka | (Like after removing the instance that has it passed-through) | 14:15 |
| BertrandLanson[m] | cleanup as in, wipe them after delete ? | 14:15 |
| BertrandLanson[m] | I think so | 14:15 |
| BertrandLanson[m] | need to recheck if we have some custom stuff for it but I don't think so | 14:15 |
| mnasiadka | That would be great, sometimes I have such questions from customers ;) | 14:16 |
| BertrandLanson[m] | there are critical bugs inside cyborg itself tho | 14:16 |
| BertrandLanson[m] | past Antelope I believe ? | 14:16 |
| BertrandLanson[m] | with oslo db not updated and the nautocommit behavior that changed | 14:17 |
| mnasiadka | I’d probably would like some reviews on https://review.opendev.org/q/kolla-ansible-lint starting with https://review.opendev.org/c/openstack/kolla-ansible/+/966790 | 14:17 |
| mnasiadka | Bbezak: if you have some free time during next coffee break ;) | 14:17 |
| mnasiadka | BertrandLanson[m]: ah, another maintained project ;) | 14:17 |
| bbezak | ;) | 14:18 |
| BertrandLanson[m] | quick question idk if that's the place but what are all these ansible-lint changes for ? I saw like dozens of these and didn't really understand why you were making them | 14:19 |
| BertrandLanson[m] | (other the the truthy ones these are pretty obvious) | 14:19 |
| mnasiadka | Basically we have exceptions for like 80% of the linting rules now | 14:19 |
| mnasiadka | I’d like to stop being the linter :) | 14:19 |
| BertrandLanson[m] | oh,lol | 14:19 |
| BertrandLanson[m] | yh that makes sense | 14:20 |
| BertrandLanson[m] | ok thanks | 14:20 |
| mnasiadka | Ok then, no entries in additional agenda | 14:20 |
| mnasiadka | Let’s move to open discussion | 14:20 |
| mnasiadka | #topic Open discussion | 14:21 |
| mnasiadka | Anybody anything? | 14:21 |
| BertrandLanson[m] | podman jobs in ci ? | 14:22 |
| BertrandLanson[m] | do we have any or do we want any ? | 14:23 |
| bbezak | 2024.1 transition to unmaintained #link https://review.opendev.org/c/openstack/releases/+/967694 | 14:23 |
| mnasiadka | Well, we have a variant of each AIO job using podman | 14:23 |
| mnasiadka | Do we need more? | 14:24 |
| mnasiadka | bbezak: thank you very much | 14:24 |
| BertrandLanson[m] | hm, cause we keep finding weird behavior in the PodmanWorker, I fixed I think most of it but it passed without any issue before | 14:24 |
| mnasiadka | I think we need to ask ourselves the question why we don’t see that in CI | 14:25 |
| BertrandLanson[m] | there are patches open for it cause currently it's impossible to not restart the entire cluster on reconfigure | 14:25 |
| BertrandLanson[m] | cause of idempotence issues | 14:25 |
| BertrandLanson[m] | do we reconfigure ? | 14:25 |
| mnasiadka | There is a reconfigure step | 14:26 |
| mnasiadka | It doesn’t fail | 14:26 |
| mnasiadka | But maybe we don’t trigger a real reconfigure | 14:27 |
| mnasiadka | Or maybe we don’t check that something that should not be restarted got restarted | 14:27 |
| opendevreview | Merged openstack/kolla-ansible master: Fix inventory file for cyborg control services https://review.opendev.org/c/openstack/kolla-ansible/+/934429 | 14:27 |
| mnasiadka | That’s probably the case | 14:27 |
| BertrandLanson[m] | hm, I'm very bad with zuul still trying to wrap my head around how it all works. But yh podmanworker keeps finding diffs between current and desired state because of defaults imposed on podman's side that end-up "polluting" the api response | 14:28 |
| BertrandLanson[m] | I've made unit tests for it in the patches I openend that should see if it breaks cause the old ones were assuming the api response was similar to the docker one which it isn't. But it will only work for as long as the api doesn't change | 14:29 |
| mnasiadka | Yeah, so we would need a check after the reconfigure step - to check if any containers that shouldn’t restart have been restarted | 14:29 |
| mnasiadka | #link https://opendev.org/openstack/kolla-ansible/src/branch/master/roles/kolla-ansible-reconfigure/tasks | 14:30 |
| mnasiadka | Currently we only try to break OVN containers | 14:30 |
| mnasiadka | So all the rest should not be restarted | 14:30 |
| mnasiadka | You would need probably to dump all containers info and check their restart/start time | 14:30 |
| mnasiadka | Or systemd units | 14:31 |
| BertrandLanson[m] | Idk if there's a proper start time in the podman api, I can try to see what would work. | 14:31 |
| BertrandLanson[m] | that would also force me to leanr it | 14:31 |
| mnasiadka | haha | 14:33 |
| mnasiadka | Happy to help if you’ll have questions | 14:33 |
| mnasiadka | Ok then, let’s finish for today - no other questions and this chit-chat can continue outside the meeting | 14:33 |
| mnasiadka | #endmeeting | 14:33 |
| opendevmeet | Meeting ended Wed Nov 19 14:33:59 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:33 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-11-19-14.01.html | 14:33 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-11-19-14.01.txt | 14:33 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2025/kolla.2025-11-19-14.01.log.html | 14:33 |
| mnasiadka | Thanks for coming! | 14:34 |
| BertrandLanson[m] | thanks! | 14:44 |
| opendevreview | Merged openstack/kayobe-config stable/2025.2: Update .gitreview for stable/2025.2 https://review.opendev.org/c/openstack/kayobe-config/+/967494 | 14:49 |
| opendevreview | Merged openstack/kayobe-config stable/2025.2: Update TOX_CONSTRAINTS_FILE for stable/2025.2 https://review.opendev.org/c/openstack/kayobe-config/+/967495 | 14:53 |
| opendevreview | Merged openstack/kayobe-config-dev stable/2025.2: Update .gitreview for stable/2025.2 https://review.opendev.org/c/openstack/kayobe-config-dev/+/967496 | 14:53 |
| opendevreview | Merged openstack/kayobe-config-dev stable/2025.2: Update TOX_CONSTRAINTS_FILE for stable/2025.2 https://review.opendev.org/c/openstack/kayobe-config-dev/+/967497 | 14:53 |
| opendevreview | Jack Hodgkiss proposed openstack/kayobe master: feat: add command to view passwords https://review.opendev.org/c/openstack/kayobe/+/956440 | 15:12 |
| opendevreview | Jack Hodgkiss proposed openstack/kayobe master: feat: add command to view passwords https://review.opendev.org/c/openstack/kayobe/+/956440 | 15:26 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 16:01 |
| opendevreview | Verification of a change to openstack/kayobe stable/2025.2 failed: Move nvidia.nvue to 1.2.9 for cumulus 5.13 https://review.opendev.org/c/openstack/kayobe/+/967669 | 16:03 |
| opendevreview | Merged openstack/kayobe stable/2025.2: Update .gitreview for stable/2025.2 https://review.opendev.org/c/openstack/kayobe/+/967491 | 16:10 |
| opendevreview | Merged openstack/kayobe stable/2025.2: Update TOX_CONSTRAINTS_FILE for stable/2025.2 https://review.opendev.org/c/openstack/kayobe/+/967492 | 16:10 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 16:15 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 16:18 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 16:20 |
| opendevreview | Verification of a change to openstack/kayobe stable/2024.2 failed: Move nvidia.nvue to 1.2.9 for cumulus 5.13 https://review.opendev.org/c/openstack/kayobe/+/967431 | 17:10 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 18:01 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 19:13 |
| opendevreview | Bertrand Lanson proposed openstack/kolla-ansible master: DNM: Test idempotence of reconfigure command in CI https://review.opendev.org/c/openstack/kolla-ansible/+/967737 | 19:22 |
| BertrandLanson[m] | mnasiadka: just pushed a test for the idempotence thingy we talked about earlier, it should trigger on its own right ? | 19:25 |
| opendevreview | Cristhian Garcia proposed openstack/kolla stable/2025.1: Pin pip version to 25.2.* in openstack base image. https://review.opendev.org/c/openstack/kolla/+/967740 | 19:39 |
| opendevreview | Cristhian Garcia proposed openstack/kolla stable/2025.1: Pin pip version to 25.2.* in openstack base image. https://review.opendev.org/c/openstack/kolla/+/967740 | 19:52 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: docker: Configure path in containerd config.toml https://review.opendev.org/c/openstack/ansible-collection-kolla/+/967676 | 19:52 |
| mnasiadka | BertrandLanson[m]: It seems we forgot to update triggers to trigger on changes to that role - just add a proper regex to zuul.d/scenarios/aio.yaml | 19:53 |
| opendevreview | Bertrand Lanson proposed openstack/kolla-ansible master: DNM: Test idempotence of reconfigure command in CI https://review.opendev.org/c/openstack/kolla-ansible/+/967737 | 20:01 |
| BertrandLanson[m] | either this works or it's a problem for tomorrow-me | 20:03 |
| opendevreview | Verification of a change to openstack/kolla stable/2025.1 failed: Transition to the new rabbitmq Debian package archives. https://review.opendev.org/c/openstack/kolla/+/967657 | 21:31 |
| BertrandLanson[m] | <BertrandLanson[m]> "either this works or it's a..." <- ay it actually works! only fails for podman job (which was expected), because it restarts containers it shouldn't | 21:34 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!