| mnasiadka | morning | 06:27 |
|---|---|---|
| mnasiadka | If there’s anything that needs my attention - please shout ;-) | 06:27 |
| blanson[m] | haaaaaaaaaa | 06:46 |
| blanson[m] | good morning :) | 06:46 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Replace /etc/localtime bind mount with TZ env variable https://review.opendev.org/c/openstack/kolla-ansible/+/989435 | 06:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: Replace /etc/localtime bind mount with TZ env variable https://review.opendev.org/c/openstack/kolla-ansible/+/989435 | 06:55 |
| mnasiadka | ok, that one should be good to go ^^ - added comparison for all env vars | 06:56 |
| blanson[m] | https://review.opendev.org/c/openstack/kolla-ansible/+/995693 this little guy needs love | 07:56 |
| mnasiadka | blanson[m]: podman sounds fantastic | 08:08 |
| blanson[m] | I'm in a toxic relationship with it | 08:09 |
| mnasiadka | Sounds like high toxicity | 08:12 |
| Vii_ | I don't know if you've seen it... so the fun will probably continue :) | 08:12 |
| Vii_ | https://blog.podman.io/2026/07/introducing-podman-v6-0-0/ | 08:13 |
| blanson[m] | Vii_: I hadn't seen it, now I'm scared | 08:18 |
| opendevreview | Merged openstack/kolla master: Add opt-in SPICE support for Rocky via COPR. https://review.opendev.org/c/openstack/kolla/+/986283 | 08:21 |
| opendevreview | Merged openstack/kolla master: kolla-build: normalize mtime in archive creation https://review.opendev.org/c/openstack/kolla/+/993961 | 08:21 |
| mikal | blanson[m]: it is now five days since we've seen a libvirtd rate limit message in CI for master. So I think we can declare that one defeated? Where will I get my graphs from now?!? | 08:23 |
| mnasiadka | mikal: nice work ;) | 08:47 |
| blanson[m] | mikal: that's nice ! + the config options are actually really good on their own as well so double win | 08:53 |
| opendevreview | Will Szumski proposed openstack/kayobe master: Update Docker storage driver default https://review.opendev.org/c/openstack/kayobe/+/995926 | 08:54 |
| blanson[m] | for the graphs I wouldn't worry to much, we should get some catastrophic CI failure soon enough | 08:54 |
| blanson[m] | :) | 08:55 |
| mnasiadka | Well, we could update the graphs in grafana.opendev.org if anybody wants to have a go :) | 08:57 |
| *** jhorstmann is now known as Guest12725 | 09:06 | |
| blanson[m] | grafana states: 'No datapoints' in a graph means there have been no job failures reported yet. all of our panels have no data so this means we're perfect right ? | 09:06 |
| blanson[m] | right ? | 09:06 |
| mnasiadka | That means dashboard definitions are using wrong job names: https://opendev.org/openstack/project-config/src/branch/master/grafana/kolla.yaml ;-) | 09:09 |
| kevko | Hi, blanson[m]: maybe second +2 here ? https://review.opendev.org/c/openstack/kolla-ansible/+/992478 and mnasiadka reapply your +2 ? | 09:25 |
| kevko | morning folks | 09:25 |
| blanson[m] | hi ! | 09:29 |
| blanson[m] | this is some close lazy loading variables stuff you're doing for galera_status I'm surprised this actually works reliably lol | 09:30 |
| kevko | blanson[m]: why ? :D i think it's elegant check that cluster is working OK | 09:34 |
| blanson[m] | it is smart, I'm just surprised the lazy loading works in this setup, but it is actually elegant | 09:43 |
| blanson[m] | and the check is much better than it previously was aswell so, win-win | 09:46 |
| opendevreview | Merged openstack/kolla-ansible stable/2026.1: UnicodeDecodeError in PodmanWorker.compare_config https://review.opendev.org/c/openstack/kolla-ansible/+/995693 | 10:00 |
| opendevreview | Merged openstack/kayobe master: Fix YAML indentation in documentation https://review.opendev.org/c/openstack/kayobe/+/995837 | 10:00 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: neutron: Remove OVN metadata agent https://review.opendev.org/c/openstack/kolla-ansible/+/996078 | 11:33 |
| mnasiadka | blanson[m]: regarding podman - doesn’t this look a bit weird? https://56ba38d194e87e183512-a125ba43f78b0a3b8db7156167084af5.ssl.cf5.rackcdn.com/openstack/471a65407b98423b9851f7bdaeff2459/primary/logs/kolla/all-ERROR.txt | 11:39 |
| blanson[m] | lemme finish my lovely meeting and I'll check it out | 11:57 |
| mnasiadka | I’ve only seen this in podman patches :D | 12:32 |
| mnasiadka | as in podman CI jobs | 12:32 |
| mnasiadka | I need my vacation after vacation | 12:32 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: podman: Add kolla-make-root-rshared.service https://review.opendev.org/c/openstack/ansible-collection-kolla/+/993903 | 12:35 |
| mnasiadka | blanson[m]: but maybe it’s just some cpu starvation and ML2/OVS being tricky | 12:36 |
| blanson[m] | this rings a bell but not on podman in particular | 12:37 |
| blanson[m] | just | 12:37 |
| blanson[m] | neutron workers being too busy | 12:38 |
| blanson[m] | which is weird | 12:38 |
| blanson[m] | do we have values for these worker things ? | 12:38 |
| mnasiadka | Well, neutron-server logs complains about workers being zero, but the code path implies it’s ovn hash ring... | 12:42 |
| mnasiadka | (Which does not apply on ML2/OVS) | 12:43 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: neutron: Run metadata extension in ovn agent https://review.opendev.org/c/openstack/kolla-ansible/+/996083 | 12:45 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: neutron: Run metadata extension in ovn agent https://review.opendev.org/c/openstack/kolla-ansible/+/996083 | 12:45 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: neutron: Run metadata extension in ovn agent https://review.opendev.org/c/openstack/kolla-ansible/+/996083 | 12:46 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: neutron: Remove OVN metadata agent https://review.opendev.org/c/openstack/kolla-ansible/+/996078 | 12:46 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: neutron: Remove OVN metadata agent https://review.opendev.org/c/openstack/kolla-ansible/+/996078 | 12:46 |
| blanson[m] | hm | 12:47 |
| blanson[m] | maybe this has to do with that | 12:48 |
| blanson[m] | I can check here what we tweaked for this cause I remember us having this issue on prod | 12:49 |
| blanson[m] | which was not fun | 12:49 |
| frickler | mnasiadka: if you are bored, the propose-update jobs seemed to work, but now the nasty reqs team (:]) wants us to implement pyproject.toml files https://zuul.opendev.org/t/openstack/build/224d83c4ebb64c9ab313e4051037d7d8 | 13:12 |
| mnasiadka | Oh boy, let me hit myself in the face with pyproject.yaml :) | 13:13 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Migrate to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996091 | 13:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add docker and podman as optional deps to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996092 | 13:29 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add docker and podman as opt deps to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996092 | 13:29 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible master: WIP: neutron: Implement ovn-migration https://review.opendev.org/c/openstack/kolla-ansible/+/996096 | 13:40 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Migrate to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996091 | 13:43 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add docker and podman as opt deps to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996092 | 13:43 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 13:43 |
| opendevreview | Doug Szumski proposed openstack/kolla-ansible master: Increase Glance `chunked-input-limit` https://review.opendev.org/c/openstack/kolla-ansible/+/995931 | 13:50 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Migrate to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996091 | 13:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add docker and podman as opt deps to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996092 | 13:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 13:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 13:57 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 13:58 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 13:58 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 13:58 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 14:01 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: podman: Add kolla-make-root-rshared.service https://review.opendev.org/c/openstack/ansible-collection-kolla/+/993903 | 14:13 |
| blanson[m] | mnasiadka: for the ovs timeout issue, we bumped openstack_service_rpc_workers: "{{ [ansible_facts.processor_vcpus, 3] | min }}" to like | 14:29 |
| blanson[m] | a lot | 14:29 |
| blanson[m] | https://github.com/openstack/kolla-ansible/blob/1d479409c3e2db3bae868ef06e4055135722fab8/ansible/group_vars/all/common.yml#L328 | 14:29 |
| blanson[m] | it says the default is half the cpu cores | 14:30 |
| blanson[m] | we bumped to 128, but maybe we could bumpthe default to a bit more and see how it goes | 14:31 |
| blanson[m] | I'll do that and spam the gate jobs | 14:31 |
| mnasiadka | Makes sense, I just reused it from old approach, but maybe now it needs more | 14:32 |
| opendevreview | Piotr Milewski proposed openstack/kolla master: Upgrade Prometheus Server to LTS version 3.13.0 https://review.opendev.org/c/openstack/kolla/+/996108 | 15:11 |
| opendevreview | Merged openstack/kolla master: Install and start virtlogd in nova-libvirt-spice. https://review.opendev.org/c/openstack/kolla/+/993067 | 15:22 |
| opendevreview | Jed Preist proposed openstack/kolla-ansible master: CI: Emulate Redfish baremetal with Tenks https://review.opendev.org/c/openstack/kolla-ansible/+/995952 | 15:27 |
| opendevreview | Michal Nasiadka proposed openstack/ansible-collection-kolla master: podman: Add kolla-make-root-rshared.service https://review.opendev.org/c/openstack/ansible-collection-kolla/+/993903 | 16:05 |
| mnasiadka | frickler: seems pyproject.toml for Kolla is not going to be that trivial | 16:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Migrate to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996091 | 16:25 |
| mnasiadka | frickler: that’s why ^^ | 16:25 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add docker and podman as opt deps to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996092 | 16:26 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Introduce mypy https://review.opendev.org/c/openstack/kolla/+/996097 | 16:26 |
| opendevreview | Ettore Simone proposed openstack/kolla-ansible master: Create clouds.yaml entry for Ironic standalone https://review.opendev.org/c/openstack/kolla-ansible/+/996130 | 16:27 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Deploy MariaDB before configuring loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/992478 | 16:51 |
| opendevreview | Merged openstack/kayobe-config stable/2025.1: [2025.1-only] Synchronise with latest kayobe https://review.opendev.org/c/openstack/kayobe-config/+/995752 | 17:04 |
| frickler | oh wow | 17:17 |
| opendevreview | Salman proposed openstack/kolla-ansible master: Sort blackbox exporter targets in Prometheus config https://review.opendev.org/c/openstack/kolla-ansible/+/996139 | 17:24 |
| blanson[m] | holy... | 17:59 |
| opendevreview | Merged openstack/kolla-ansible master: Increase Glance `chunked-input-limit` https://review.opendev.org/c/openstack/kolla-ansible/+/995931 | 18:32 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Migrate to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996091 | 19:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Migrate to pyproject.toml https://review.opendev.org/c/openstack/kolla/+/996091 | 19:59 |
| opendevreview | Salman proposed openstack/kolla-ansible master: Fix RabbitMQ restart in check mode https://review.opendev.org/c/openstack/kolla-ansible/+/996170 | 20:09 |
| opendevreview | Salman proposed openstack/kolla-ansible master: Fix RabbitMQ restart in check mode https://review.opendev.org/c/openstack/kolla-ansible/+/996170 | 22:18 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!