| opendevreview | Merged openstack/openstack-ansible-os_magnum master: Fix Vexxhost CAPI installation for Debian https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/990619 | 00:05 |
|---|---|---|
| f0o | Good morning, I noticed another quirk with the haproxy_vip_binds - The goal is to have our public domains dualstack, backed with certificates from certbot. However the 2nd External IP serves the local-pki certificates and not the certbot ones. Did I miss something? | 04:46 |
| noonedeadpunk | mornings... eh... really good question...there's indeed a logic around loop order which might be weird.... | 06:56 |
| noonedeadpunk | I wonder why though your original solution didn't work | 06:57 |
| noonedeadpunk | as I would expect it to | 06:57 |
| noonedeadpunk | as I see our regions do `extra_lb_tls_vip_addresses` for IPv6 | 06:58 |
| noonedeadpunk | and we're running 2025.1 now | 06:58 |
| noonedeadpunk | so it could be that original configuration you was having was closer to working one... | 07:01 |
| noonedeadpunk | f0o: ^ | 07:01 |
| f0o | :D | 07:03 |
| f0o | my concern with `extra_lb_tls_vip_addresses` was that I hit a bug that interface was None in the dict. I guess I will have to plan upgrade to 2025.1 ? | 07:04 |
| noonedeadpunk | I don;t think there were much changes in this area | 07:04 |
| f0o | so why did I hit the issue... does it need a combination of `haproxy_bind_external_lb_vip_address` and `extra_lb_tls_vip_addresses` ? | 07:05 |
| f0o | today we dont specify `haproxy_bind_external_lb_vip_address` | 07:05 |
| noonedeadpunk | well `haproxy_bind_external_lb_vip_address: "{{ external_lb_vip_address }}"` by default | 07:07 |
| noonedeadpunk | so you have it defined through external_lb_vip_address I guess | 07:07 |
| f0o | let me rerun it with just external_lb_vip_address + extra_lb_tls_vip_addresses set | 07:09 |
| noonedeadpunk | so fair enough, extra_lb_tls_vip_addresses does not actually add interface | 07:10 |
| noonedeadpunk | but I think it should not fail..... | 07:10 |
| noonedeadpunk | can you also provide a task which was failing? was that template? | 07:10 |
| f0o | fatal: [n1_2]: FAILED! => {"msg": "The task includes an option with an undefined variable.. 'dict object' has no attribute 'interface'\n\nThe error appears to be in '/etc/ansible/roles/haproxy_server/tasks/haproxy_ssl_letsencrypt.yml': line 61, column 3, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n\n- | 07:11 |
| f0o | name: Create new pem file for haproxy\n ^ here\n"} | 07:11 |
| f0o | sec | 07:12 |
| f0o | https://paste.opendev.org/show/b3mQS4m2Q7EncTvU7H0q/ | 07:12 |
| f0o | the v6 is added as `extra_lb_tls_vip_addresses` - the v4's external_lb_vip_address seem to pass for certbot | 07:14 |
| f0o | runnig all of `extra_lb_tls_vip_addresses` + `external_lb_vip_address` + `haproxy_vip_binds` does actually make it pass | 07:15 |
| f0o | yeah all three together makes a valid cerbot certified v6 :F | 07:16 |
| f0o | should the ipv6 address in haproxy config be escaped? | 07:18 |
| f0o | bind 2a0a:3507:1:2999::10:443 - I'm amazed haproxy didnt bark at this, I think the "right" way is to use bind [2a0a:3507:1:2999::10]:443 | 07:19 |
| f0o | I mean... it works.. I'm just curious at this point | 07:20 |
| noonedeadpunk | sry in a meeting | 07:27 |
| noonedeadpunk | hm | 07:29 |
| f0o | no worries mate, I think you helped me a lot already | 07:29 |
| noonedeadpunk | I am not getting why are you hitting failure | 07:29 |
| noonedeadpunk | f0o: do you have this patch? https://opendev.org/openstack/openstack-ansible-haproxy_server/commit/e22724aa2b3815ddad2e815cb8c44a59a3c5360d | 07:29 |
| noonedeadpunk | it was backported to 2024.2 https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/940962 | 07:30 |
| noonedeadpunk | as it could be answer to your original issue | 07:31 |
| f0o | doesnt seem like, I'm on f4846463a5ccabfefc3d23bdb6e1779247e21d45 | 07:31 |
| f0o | well that explains it then :D | 07:38 |
| noonedeadpunk | yeah, it right before it | 07:38 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.2: Update role requirements for stable branch https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991289 | 08:02 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.2: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991245 | 08:02 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2025.2: Populate physical_host_addr by dynamic_inventory https://review.opendev.org/c/openstack/openstack-ansible/+/991290 | 08:05 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs: clarify deployment configuration examples https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 08:07 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs: clarify deployment configuration examples https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 08:55 |
| opendevreview | Damian DÄ…browski proposed openstack/openstack-ansible-os_manila master: Remove manila v1 API https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/985230 | 09:12 |
| damiandabrowski | omfg, manila was failing on gates because I changed the nodeset only for check pipeline... | 09:13 |
| damiandabrowski | it should work now, but unfrotunately the votes has been reset: https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/985230 | 09:13 |
| damiandabrowski | noonedeadpunk: what happened here? :D https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/981710 | 09:19 |
| damiandabrowski | I see some unresolved comments but you still left +2 there | 09:19 |
| noonedeadpunk | aha, good catch with manila | 09:32 |
| noonedeadpunk | but then also we need to rebase the whole stack | 09:32 |
| noonedeadpunk | regarding magnum, shit, I think I have pushed an update to it to resolve comments | 09:33 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: Add vexxhost mcapi proxy supporting tasks https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/981710 | 09:42 |
| noonedeadpunk | fixed ^ | 09:42 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server stable/2025.2: Add certbot package variable for RHEL based systems https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/991210 | 09:52 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Fix shibboleth defenition for http role https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/991302 | 10:05 |
| damiandabrowski | what about this one? https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/981710/comment/1a939d51_f0717d40/ | 10:13 |
| noonedeadpunk | right | 10:13 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: Add vexxhost mcapi proxy supporting tasks https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/981710 | 10:14 |
| opendevreview | Merged openstack/openstack-ansible-os_skyline master: Remove redundant vars line https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/973143 | 11:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_magnum master: [doc] Refactor documentation for Magnum https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/989853 | 11:16 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Revert "Disable CI jobs for the role" https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/990646 | 11:32 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs: clarify deployment configuration examples https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 11:43 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs: clarify deployment configuration examples https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 11:50 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Use infra mirror for LXC debootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/951338 | 11:59 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: Update repositories URL to opendev.org https://review.opendev.org/c/openstack/openstack-ansible/+/974618 | 11:59 |
| opendevreview | Merged openstack/openstack-ansible-os_trove master: Handle missing provider network definition safely https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/980638 | 11:59 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_trove stable/2025.2: Handle missing provider network definition safely https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991314 | 12:00 |
| f0o | maybe a silly question but is there a support matrix showing which OSA release supports which base OS when using the distro-based installation methods? | 12:02 |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts master: Remove redundant vars line https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/990808 | 12:03 |
| opendevreview | Merged openstack/openstack-ansible master: Use openstack_hosts_apt_pinned_packages in user_variables https://review.opendev.org/c/openstack/openstack-ansible/+/977881 | 12:03 |
| f0o | I'm setting up a new node and wonder if I can just take Ubuntu 24.04 or even 26.04 instead of what we got today (22.04) | 12:03 |
| noonedeadpunk | 26.04 - no | 12:07 |
| noonedeadpunk | not until 2026.1 | 12:07 |
| noonedeadpunk | 24.04 - yes | 12:07 |
| noonedeadpunk | you just wuould need to mess a bit with gluster if you're using it | 12:07 |
| f0o | I take it there's no matrix? :D | 12:07 |
| noonedeadpunk | and upgrade existing nodes to gluster 11 from 11 from ppa | 12:07 |
| noonedeadpunk | f0o: there is | 12:07 |
| noonedeadpunk | https://docs.openstack.org/openstack-ansible/latest/admin/upgrades/compatibility-matrix.html | 12:08 |
| f0o | luckily no glusterfs, burned my fingers a few years ago with k8s on it. staying away from it. guess bricks was a good name afterall | 12:08 |
| f0o | aaah under ungrades | 12:08 |
| f0o | hrm 2025.2 is not supported at all with distro? | 12:09 |
| f0o | well 2025.1 neither I take it? | 12:10 |
| noonedeadpunk | so for 2025.1 it's working and passing | 12:10 |
| noonedeadpunk | it's non-voting in CI though | 12:10 |
| noonedeadpunk | 2025.2 - has broken glance in packages | 12:11 |
| noonedeadpunk | it probably is fixed now though | 12:11 |
| noonedeadpunk | https://bugs.launchpad.net/cloud-archive/+bug/2136411 | 12:11 |
| f0o | funny enough, I'm on 2024.2 with distro on Ubuntu 22.04 just fine haha | 12:12 |
| noonedeadpunk | I am not sure in what state it is right now, as there was a bit of back-forcing | 12:12 |
| noonedeadpunk | f0o: and 2026.1 has another issue with distro install, is that packages are still only in proposed | 12:14 |
| noonedeadpunk | and not in updates, so when you add UCA you still get 2025.2 and not 2026.1 | 12:14 |
| f0o | I looked at moving from distro to source before but found no easy way (~2 years ago), has that changed? | 12:15 |
| f0o | it feels like building on distro was a footbullet in the long run | 12:16 |
| noonedeadpunk | I'd guess you should be able convert node-by-node | 12:16 |
| noonedeadpunk | jsut dropping containers on one control, adding to group_vars override, deploy it | 12:17 |
| f0o | might bite with the nova-hypervisors since those are metal right? | 12:17 |
| noonedeadpunk | I checked 2025.2 state now, and it fails with `ModuleNotFoundError: No module named 'glance.wsgi'` -> https://zuul.opendev.org/t/openstack/build/40e8858cddd548fca6f5f60c68b89f92/log/logs/host/glance-api.service.journal-09-26-02.log.txt#554 | 12:18 |
| noonedeadpunk | but then when I tried to backport logic, it was still failing | 12:18 |
| noonedeadpunk | https://review.opendev.org/c/openstack/openstack-ansible-os_glance/+/989182 | 12:18 |
| noonedeadpunk | with `failed to open python file glance-wsgi-api` | 12:19 |
| noonedeadpunk | so it seems there's just no working packaged glance in UCA.... | 12:19 |
| f0o | 🙃 | 12:23 |
| noonedeadpunk | maybe there's differnet path in packages version or smth now... but it works on 2026.1 when I actually getting 2026.1 packages from noble-proposed | 12:26 |
| noonedeadpunk | I had a bug there: https://bugs.launchpad.net/cloud-archive/+bug/2154466 | 12:26 |
| f0o | haha | 12:27 |
| f0o | cant help but laugh at this | 12:27 |
| f0o | guess I'll be stuck on 22.04 a while longer | 12:27 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server master: Improve feature enablement layout https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/980551 | 12:43 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Remove manila v1 API https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/985230 | 12:45 |
| damiandabrowski | yay \o/ | 12:52 |
| noonedeadpunk | I wonder if we should be rebasing | 12:53 |
| opendevreview | Damian DÄ…browski proposed openstack/openstack-ansible-os_manila master: Add hashi_vault pki backend support https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/981675 | 12:55 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade RabbitMQ to 4.2.7 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989625 | 13:01 |
| opendevreview | Takashi Kajinami proposed openstack/openstack-ansible-os_manila master: Avoid leaking internal url for authentication error https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/976027 | 13:15 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Do not remove policy.yaml file https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/967186 | 13:16 |
| opendevreview | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_manila master: Updated from OpenStack Ansible Integrated repository https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/985587 | 13:16 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-os_manila master: Replace git-core with git for debian https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/860354 | 13:16 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-os_manila master: Move default quotes in quota group https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/977864 | 13:17 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-os_manila master: Update service description https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/975573 | 13:17 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-os_manila master: Remove redundant vars line https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/857647 | 13:17 |
| opendevreview | Merged openstack/openstack-ansible master: Add Resolute CI jobs https://review.opendev.org/c/openstack/openstack-ansible/+/988591 | 14:22 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs: clarify deployment configuration examples https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 14:22 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Document glusterfs_reset_peers https://review.opendev.org/c/openstack/openstack-ansible/+/966209 | 14:41 |
| drarvese | noonedeadpunk: I saw your update -- bummer. :( Here's a paste of some of the haproxy logs: https://paste.opendev.org/show/bvpaZO8CgsBv2y4562pS/ | 15:09 |
| noonedeadpunk | drarvese: I'd need logs from Horizon container | 15:10 |
| noonedeadpunk | haproxy doesn't contain anything except statement that it's 500 | 15:10 |
| drarvese | https://paste.opendev.org/show/bzCetYPMOR5PutyblEpc/ | 15:13 |
| drarvese | Horizon is returning a 200, and I can curl the individual container IPs successfully, but haproxy is throwing 500s | 15:14 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server stable/2025.2: Update role requirements for stable branch https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991289 | 15:24 |
| noonedeadpunk | hm | 15:49 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server stable/2025.2: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991245 | 15:50 |
| opendevreview | Merged openstack/openstack-ansible stable/2025.2: Populate physical_host_addr by dynamic_inventory https://review.opendev.org/c/openstack/openstack-ansible/+/991290 | 16:27 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Add hashi_vault pki backend support https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/981675 | 17:00 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Add details about Khepri https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991478 | 17:05 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Add details about Khepri https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991478 | 17:05 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Add details about Khepri https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991478 | 17:07 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Add details new deployment and Khepri https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991478 | 17:07 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Add details for new deployment about Khepri https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991478 | 17:08 |
| opendevreview | Merged openstack/openstack-ansible master: Remove galera_disable_privatedevices and memcached_disable_privatedevices https://review.opendev.org/c/openstack/openstack-ansible/+/984348 | 18:12 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Updated from OpenStack Ansible Integrated repository https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/985587 | 18:54 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Update service description https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/975573 | 18:54 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Replace git-core with git for debian https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/860354 | 18:54 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Avoid leaking internal url for authentication error https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/976027 | 18:54 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Remove redundant vars line https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/857647 | 19:53 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-ceph_client master: Re-runs of the ceph_client role would fail on RHEL https://review.opendev.org/c/openstack/openstack-ansible-ceph_client/+/870027 | 20:10 |
| opendevreview | Merged openstack/openstack-ansible-os_manila master: Move default quotes in quota group https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/977864 | 20:58 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_manila master: Do not remove policy.yaml file https://review.opendev.org/c/openstack/openstack-ansible-os_manila/+/967186 | 21:08 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server stable/2025.1: Add certbot package variable for RHEL based systems https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/991211 | 22:03 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!