| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 04:25 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed openstack/kolla master: repos: Merge user-provided with default repos.yaml https://review.opendev.org/c/openstack/kolla/+/993499 | 04:47 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: repos: Merge user-provided with default repos.yaml https://review.opendev.org/c/openstack/kolla/+/993499 | 04:47 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 04:47 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: repos: Merge user-provided with default repos.yaml https://review.opendev.org/c/openstack/kolla/+/993499 | 04:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 04:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 04:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 04:55 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 04:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 04:56 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 05:21 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 05:22 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 05:33 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 05:51 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2026.1: CI: Switch periodics to periodic-stable https://review.opendev.org/c/openstack/kolla/+/993500 | 05:57 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: WIP: Use buildset registry and run multiarch build https://review.opendev.org/c/openstack/kolla/+/990887 | 05:58 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Don't create MariaDB monitor user in loadbalancer tasks https://review.opendev.org/c/openstack/kolla-ansible/+/993362 | 07:16 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 07:23 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 07:26 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Deploy MariaDB before configuring loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/992478 | 07:29 |
| kevko | mnasiadka: thank you | 07:29 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 07:42 |
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2025.1: [rabbitmq] Use stop instead of drain on single-node https://review.opendev.org/c/openstack/kolla-ansible/+/993519 | 07:54 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.2: Install and start virtlogd in nova-libvirt. https://review.opendev.org/c/openstack/kolla/+/993520 | 08:02 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.1: Install and start virtlogd in nova-libvirt. https://review.opendev.org/c/openstack/kolla/+/993521 | 08:02 |
| opendevreview | Verification of a change to openstack/kolla stable/2026.1 failed: Install and start virtlogd in nova-libvirt. https://review.opendev.org/c/openstack/kolla/+/993460 | 08:02 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 08:14 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 08:26 |
| opendevreview | Merged openstack/kolla-ansible master: Don't create MariaDB monitor user in loadbalancer tasks https://review.opendev.org/c/openstack/kolla-ansible/+/993362 | 08:34 |
| opendevreview | Alexey Kuchinskiy proposed openstack/kolla-ansible master: Fix container unhealthy checks https://review.opendev.org/c/openstack/kolla-ansible/+/991842 | 08:35 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 08:39 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 08:59 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 08:59 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 09:00 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support for overriding base repos for Debian/Ubuntu https://review.opendev.org/c/openstack/kolla/+/993407 | 09:01 |
| opendevreview | Jed Preist proposed openstack/kolla master: Create Ironic temp directory https://review.opendev.org/c/openstack/kolla/+/993369 | 09:06 |
| kevko | is it possible that zuul is not trigerred by recheck ? :/ | 09:09 |
| priteau | mnasiadka: Kayobe CI is broken, most likely caused by some change in kolla-ansible? https://review.opendev.org/c/openstack/kolla-ansible/+/993066 depends on https://review.opendev.org/c/openstack/kolla/+/993065 which merged yesterday, but https://quay.io/repository/openstack.kolla/nova-libvirt?tab=tags&tag=latest shows that some master images have not been built for 4 days | 09:11 |
| priteau | Daily push have stopped running: https://zuul.opendev.org/t/openstack/builds?job_name=kolla-publish-rocky-10-quay | 09:16 |
| mnasiadka | priteau: are you sure you are not using gazpacho images with master k-a? | 09:25 |
| *** jhorstmann is now known as Guest11421 | 09:26 | |
| priteau | quay.io/openstack.kolla/nova-libvirt:master-rocky-10 | 09:26 |
| priteau | Looks fine to me | 09:26 |
| priteau | Anyway, there should be new master images daily on quay | 09:26 |
| priteau | They are all the same: new images are available for stable branches and master aarch64, but none for master/amd64 | 09:27 |
| mnasiadka | Well, then probably the publish job failed | 09:27 |
| mnasiadka | I don’t have time today to manage this | 09:27 |
| priteau | I raised it on Matrix, maybe it's a Zuul issue as I don't see any kolla CI change that would cause this | 09:29 |
| mnasiadka | I’ll try to have a look in the evening though | 09:29 |
| mnasiadka | Well, anybody can check the state of periodic daily publish jobs in Kolla | 09:30 |
| priteau | I have linked to the build list above, they are not running anymore? https://zuul.opendev.org/t/openstack/builds?job_name=kolla-publish-rocky-10-quay | 09:37 |
| mnasiadka | https://zuul.opendev.org/t/openstack/build/0495b334ab6d47d59ddcd23d793d3756 | 09:37 |
| priteau | Yes, that was 4 days ago | 09:37 |
| mnasiadka | That’s the last one | 09:37 |
| priteau | That's my point :) | 09:40 |
| mnasiadka | Well, then it’s most probably a Zuul problem | 09:40 |
| priteau | There was also a gap of 4 days for stable/2026.1 | 09:40 |
| priteau | One build on June 12 and another today, but none in the meantime | 09:40 |
| mnasiadka | Unfortunately the only people having an idea are in US timezone, so we need to be patient | 09:42 |
| mnasiadka | It might be related to some stuck requests in another provider… | 09:43 |
| frickler | answered in #opendev, in short arm jobs were blocked | 11:08 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Deploy MariaDB before configuring loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/992478 | 11:10 |
| mnasiadka | Ah, and they are in the same pipeline | 11:19 |
| opendevreview | Alexey Kuchinskiy proposed openstack/kolla-ansible master: Fix container unhealthy checks https://review.opendev.org/c/openstack/kolla-ansible/+/991842 | 11:24 |
| opendevreview | Merged openstack/kolla stable/2026.1: Install and start virtlogd in nova-libvirt. https://review.opendev.org/c/openstack/kolla/+/993460 | 12:11 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Deploy MariaDB before configuring loadbalancer https://review.opendev.org/c/openstack/kolla-ansible/+/992478 | 12:43 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 12:43 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: repos: Merge user-provided with default repos.yaml https://review.opendev.org/c/openstack/kolla/+/993499 | 12:44 |
| tafkamax | Hi Kolla gurus! I have a question about upgrading from 2025.1 -> 2025.2. We upgraded our test-cluster to 2025.2. Thanks for the proxysql fix last week! Also redis migradted to valkey aswell. We now have an issue that neutron-ovn-metadata-agent are not coming up. | 12:50 |
| tafkamax | TASK [nova-cell : Waiting for nova-compute services to register themselves] **************************************************************************** | 12:51 |
| tafkamax | Fails at this stage | 12:51 |
| tafkamax | might there be an option we are missing? | 12:51 |
| tafkamax | or an open patch that might fix this? | 12:51 |
| tafkamax | Though networking + instance creation works. | 12:51 |
| kevko | and why it's not registered ? | 12:51 |
| tafkamax | We don't know yet. Has anybody seen this when migrating from 2025.1 to 2025.2? | 12:53 |
| tafkamax | Logs from /var/log/kolla/neutron/neutron-ovn-metadata-agent.log https://paste.opendev.org/show/blXOWmi19AOGXeNk7I86/ | 12:57 |
| mnasiadka | tafkamax: sounds like https://review.opendev.org/c/openstack/kolla-ansible/+/961510 is related | 12:58 |
| mnasiadka | tafkamax: can you enable debug in ovn metadata agent and post debug logs? | 12:59 |
| tafkamax | sure | 12:59 |
| tafkamax | my colleague maybe found an issue, will wait for his response, what he has to say | 13:00 |
| priteau | tafkamax: are you sure your nova_compute container is running correctly? | 13:05 |
| opendevreview | Pierre Riteau proposed openstack/kolla master: CI: Disable arm64 publish jobs https://review.opendev.org/c/openstack/kolla/+/993594 | 13:18 |
| priteau | mnasiadka: I proposed to completely disable the arm64 periodic jobs, but if I switch them to periodic-weekly, would it stop blocking the periodic ones? | 13:19 |
| priteau | And then we would get weekly aarch64 builds which might be good enough | 13:21 |
| frickler | +1 for weekly | 13:30 |
| opendevreview | Pierre Riteau proposed openstack/kolla master: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993594 | 13:43 |
| priteau | frickler: I updated my change | 13:43 |
| blanson[m] | weekly is nice | 13:49 |
| mnasiadka | Then weekly it is for now, but I don’t think we should make arm64 a second class citizen for long | 15:12 |
| priteau | Of course, but making it first class citizen without a first class CI will be difficult :( | 15:17 |
| priteau | Thanks for merging | 15:17 |
| opendevreview | Alexey Kuchinskiy proposed openstack/kolla-ansible master: Fix container unhealthy checks https://review.opendev.org/c/openstack/kolla-ansible/+/991842 | 15:32 |
| opendevreview | Alexey Kuchinskiy proposed openstack/kolla-ansible master: Fix container unhealthy checks https://review.opendev.org/c/openstack/kolla-ansible/+/991842 | 15:41 |
| opendevreview | Alexey Kuchinskiy proposed openstack/kolla-ansible master: Fix container unhealthy checks https://review.opendev.org/c/openstack/kolla-ansible/+/991842 | 15:45 |
| opendevreview | Merged openstack/kolla master: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993594 | 15:51 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2026.1: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993624 | 15:53 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2026.1: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993624 | 15:54 |
| opendevreview | Verification of a change to openstack/kolla-ansible master failed: CI: Improve ProxySQL test https://review.opendev.org/c/openstack/kolla-ansible/+/978221 | 15:58 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 15:58 |
| priteau | I don't understand the Zuul failure here: https://review.opendev.org/c/openstack/kolla/+/993624 | 16:02 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add --retries 5 to pip installation macro https://review.opendev.org/c/openstack/kolla/+/993633 | 16:05 |
| mnasiadka | priteau: it’s in PS1, because it was a merge conflict, I removed it in PS2 ;) | 16:05 |
| priteau | OK | 16:06 |
| opendevreview | Aleksey proposed openstack/kolla-ansible master: Fix container unhealthy checks https://review.opendev.org/c/openstack/kolla-ansible/+/991842 | 16:09 |
| frickler | fixing things faster than zuul can complain, good job :) | 16:14 |
| opendevreview | Seunghun Lee proposed openstack/kolla-ansible master: CI: Improve ProxySQL test https://review.opendev.org/c/openstack/kolla-ansible/+/978221 | 16:22 |
| mnasiadka | priteau: fwiw - rocky 10 quay.io images updated an hour ago | 16:25 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.2: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993638 | 16:34 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.2: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993638 | 16:34 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.1: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993639 | 16:34 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.1: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993639 | 16:36 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.2: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993638 | 16:37 |
| opendevreview | Michal Nasiadka proposed openstack/kolla stable/2025.2: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993638 | 16:37 |
| mnasiadka | Ok, that should now look as it should | 16:37 |
| opendevreview | Seunghun Lee proposed openstack/kolla-ansible master: Fix nova-compute startup after reprovisioning https://review.opendev.org/c/openstack/kolla-ansible/+/936785 | 16:40 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 16:45 |
| opendevreview | Merged openstack/kolla stable/2026.1: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993624 | 16:49 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override baseos and appstream repos for RedHat family https://review.opendev.org/c/openstack/kolla/+/993400 | 17:02 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 17:03 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 17:03 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 17:03 |
| opendevreview | Merged openstack/kolla stable/2025.1: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993639 | 17:10 |
| opendevreview | Merged openstack/kolla stable/2025.2: CI: Switch arm64 publish jobs to periodic-weekly https://review.opendev.org/c/openstack/kolla/+/993638 | 17:10 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:00 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:06 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:15 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:27 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:33 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:39 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:48 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 18:48 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Add support to override base repos https://review.opendev.org/c/openstack/kolla/+/993400 | 19:51 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: CI: Remove npm proxy configuration https://review.opendev.org/c/openstack/kolla/+/993671 | 19:53 |
| blanson[m] | mnasiadka: for the virtlogd backport thing, should we add a config option for it ? I'm not sure we really can determine if the image has the new file early enough for it to be automatic | 19:55 |
| blanson[m] | I was thinking of adding something like nova_libvirt_virtlog_enabled which would default to false or something | 19:55 |
| mnasiadka | blanson[m]: well, it’s a security bug, I would prefer it being enabled by default - maybe a LABEL on the image and a precheck would be enough? | 20:01 |
| opendevreview | Verification of a change to openstack/kayobe master failed: Remove external_net_name variable https://review.opendev.org/c/openstack/kayobe/+/992570 | 20:12 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: Move PIP_* env variables to ARG and config https://review.opendev.org/c/openstack/kolla/+/993675 | 20:15 |
| opendevreview | Verification of a change to openstack/ansible-collection-kolla master failed: Add Docker config option for Prometheus endpoint https://review.opendev.org/c/openstack/ansible-collection-kolla/+/936717 | 20:17 |
| opendevreview | Michal Nasiadka proposed openstack/kolla master: kolla-build: Add Docker BuildKit support via docker buildx build https://review.opendev.org/c/openstack/kolla/+/993034 | 20:24 |
| opendevreview | Verification of a change to openstack/kayobe master failed: Fix paths to EPEL repositories https://review.opendev.org/c/openstack/kayobe/+/992770 | 20:26 |
| blanson[m] | mnasiadka or we could fallback the command something like this || that . I wonder if that would work | 22:06 |
| blanson[m] | eh probbly would cause a bunch of unforseen issue | 22:07 |
| shermanm | https://review.opendev.org/c/openstack/kolla/+/993034 | 22:52 |
| shermanm | nice, this is much cleaner than my ancient PR trying to use the python-on-whales library | 22:52 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!