| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade RabbitMQ to 4.2.7 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989625 | 00:12 |
|---|---|---|
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Refactor compressed backup creation https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/960387 | 00:41 |
| opendevreview | Merged openstack/openstack-ansible-galera_server master: Remove galera_disable_privatedevices https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/984312 | 00:41 |
| opendevreview | Merged openstack/openstack-ansible-repo_server master: Filter upper-constraints content https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/989979 | 02:01 |
| opendevreview | OpenStack Proposal Bot proposed openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-ansible/+/991008 | 04:07 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Fix hosts healthcheck https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/991016 | 05:51 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-lxc_hosts master: Ensure ubuntu_virt hook not passed to LXC https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/990070 | 05:52 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: Use openstack_hosts_apt_pinned_packages in user_variables https://review.opendev.org/c/openstack/openstack-ansible/+/977881 | 05:57 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-plugins master: Remove redundant vars line https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/990807 | 05:58 |
| opendevreview | Merged openstack/openstack-ansible master: Imported Translations from Zanata https://review.opendev.org/c/openstack/openstack-ansible/+/991008 | 06:01 |
| noonedeadpunk | mornings - I think we need also https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/991016 to unblock hosts scenario | 06:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-plugins master: Fix hosts healthcheck https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/991016 | 06:07 |
| opendevreview | Merged openstack/openstack-ansible-memcached_server master: Remove memcached_disable_privatedevices https://review.opendev.org/c/openstack/openstack-ansible-memcached_server/+/984282 | 08:04 |
| f0o | is there a bug within extra_lb_tls_vip_addresses and certbot? I get an error that 'dict object' has no attribute 'interface'\n\nThe error appears to be in '/etc/ansible/roles/haproxy_server/tasks/haproxy_ssl_letsencrypt.yml' - I have it set to extra_lb_tls_vip_addresses: [ "1:2:3:4::5" ] | 09:24 |
| noonedeadpunk | um, I haven't used that for ipv6 for a while... | 09:29 |
| opendevreview | Merged openstack/ansible-role-pki master: Enable openbao job for ansible-role-pki https://review.opendev.org/c/openstack/ansible-role-pki/+/989947 | 09:33 |
| f0o | noonedeadpunk:what's the preffered way? swap the external IP for v6 and use extra_lb for v4? | 09:34 |
| noonedeadpunk | f0o: I would say in general I'd just used `haproxy_vip_binds` instead: https://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/defaults/main.yml#L295-L304 | 09:38 |
| noonedeadpunk | But I'm not sure why you hit the issue tbh | 09:38 |
| f0o | I had the feeling vip_binds would be harder to maintain but I guess its just a matter of adding both externals and the single internal with interface mapping right> | 09:39 |
| f0o | haproxy_vip_binds seems to have worked great | 10:08 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Allow to reset peer during it's re-install https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/966117 | 10:12 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Add .py extension to modules https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/988147 | 10:26 |
| f0o | Hrm I'm just in the process of adding a new HAProxy host; and out of the blue I'm struck with haproxy_nova_spice_console_service not being set. Not sure how that happened or what it supposed to be. I'm pretty sure I'm on the same git-commit that I've always been on with osa | 10:33 |
| f0o | could be related to https://bugs.launchpad.net/openstack-ansible/+bug/2122778 - just odd that I'm hitting it now and not earlier | 10:35 |
| f0o | or maybe not | 10:36 |
| f0o | nvmd turns out I cant do -l on nova without breaking it | 11:15 |
| noonedeadpunk | haproxy_nova_spice_console_service should have been fixed | 12:44 |
| noonedeadpunk | we had bunch of patrches around that, but not sure what and if all of them were backported | 12:44 |
| noonedeadpunk | as it required quite some refactoring to untackle that\ | 12:44 |
| noonedeadpunk | f0o: ^ | 12:44 |
| noonedeadpunk | jrosser: andrewbonney mgariepy damiandabrowski can we quickly merge this one? https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/991016 | 12:45 |
| noonedeadpunk | and then close these topics | 12:46 |
| noonedeadpunk | https://review.opendev.org/q/topic:%22osa/resolute%22+status:open | 12:46 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade RabbitMQ to 4.2.7 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989625 | 12:51 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: Use openstack_hosts_apt_pinned_packages in user_variables https://review.opendev.org/c/openstack/openstack-ansible/+/977881 | 13:00 |
| f0o | noonedeadpunk: could be I'm just on a very old commit (2024.2) to not introduce changes - or well, more changes haha | 13:08 |
| f0o | noonedeadpunk: haproxy_vip_binds seems to be implied TLS; our internal IP doesnt use TLS however, is there a way to disable TLS for the type:internal | 13:19 |
| noonedeadpunk | f0o: that is really good question.... | 13:21 |
| f0o | or is internal tls now always the case, if so what's the migration - I dont recall this being the case prior | 13:21 |
| f0o | interesting, it seems that it has it all backwards | 13:22 |
| f0o | let me paste | 13:22 |
| noonedeadpunk | it should not be required now... but indeed, I don't see a logic which would allow to skip the tls for haproxy_vip_binds | 13:22 |
| noonedeadpunk | I wonder how it wasn't implemented | 13:23 |
| noonedeadpunk | we're discussing the requirement for internal tls going forward though | 13:23 |
| f0o | https://paste.opendev.org/show/bAPuiwPfHRElyEKX3YuG/ | 13:24 |
| f0o | I think theres a broader issue | 13:24 |
| noonedeadpunk | The thing is that extra_lb_tls_vip_addresses does also populate haproxy_vip_binds | 13:24 |
| f0o | Just confused why only one of the frontends has the TLS but not the other | 13:26 |
| f0o | let me see if I can fully revert the vips one to what I had prior | 13:26 |
| f0o | reason I even stumbled on it was that I wanted to switchover to the new Haproxy node with the dualstack but it all just exploded because it started requiring TLS for the internal VIP | 13:28 |
| noonedeadpunk | so smth is off apparently | 13:29 |
| noonedeadpunk | so logic right now also relies on order of things indeed | 13:31 |
| noonedeadpunk | try that https://paste.openstack.org/show/bSy7hTXCVikeFkgmrTQU/ | 13:32 |
| f0o | that was my next Q since it looked like the first was always TLS'd; will give it a shot | 13:33 |
| noonedeadpunk | `(loop.index == 1 or vip_address in extra_lb_tls_vip_addresses or (service.haproxy_ssl_all_vips | default(false)` | 13:33 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-haproxy_server/src/branch/master/templates/service.j2#L56 | 13:33 |
| noonedeadpunk | I want to refactor all that for 2 cycles now | 13:34 |
| f0o | the haproxy templates gives me headaches haha | 13:35 |
| noonedeadpunk | I want to get rid of all these extra_lb_tls_vip_addresses/extra_lb_vip_addresses/haproxy_bind_external_lb_vip_address etc | 13:37 |
| noonedeadpunk | just in fwavor of haproxy_vip_binds and have a separate key to control tls | 13:37 |
| f0o | well this has sortof fixed it but only one of my externals have TLS now haha | 13:41 |
| f0o | I gues that's fine, I just FW off the other as well | 13:41 |
| f0o | since we ideally only care for traffic matching the service hostnames on 443 | 13:42 |
| noonedeadpunk[e] | I will try to work on that on upcoming weekends... And make changes backprotable somehow | 13:58 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:00 |
| opendevmeet | Meeting started Tue Jun 2 15:00:15 2026 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:00 |
| noonedeadpunk | #topic rollcall | 15:00 |
| noonedeadpunk | o/ | 15:00 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server master: Install rabbitmq for 26.04 from distro repos https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989933 | 15:00 |
| opendevreview | Merged openstack/openstack-ansible-plugins master: Fix hosts healthcheck https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/991016 | 15:02 |
| noonedeadpunk | #topic releasing | 15:05 |
| noonedeadpunk | So, we should be having quite healthy CI right now | 15:05 |
| noonedeadpunk | most of important changes are merged already | 15:05 |
| noonedeadpunk | one thing that escaped attention is mistral | 15:07 |
| noonedeadpunk | #link https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/950513 | 15:07 |
| noonedeadpunk | we never merged wsgi module there | 15:07 |
| noonedeadpunk | this is the last patch in the topic | 15:07 |
| noonedeadpunk | eventually mistral is gonna be non-functional without it | 15:08 |
| noonedeadpunk | and apparently we don't do any tempest tests there | 15:08 |
| damiandabrowski | hi! sorry for joining late | 15:10 |
| noonedeadpunk | no worries | 15:11 |
| noonedeadpunk | There are also bunch of outstanding patches for manila actually | 15:11 |
| noonedeadpunk | #link https://review.opendev.org/q/project:openstack/openstack-ansible-os_manila+status:open | 15:12 |
| noonedeadpunk | thought they're all dependent on v1 removal, which just failed gates ( | 15:12 |
| damiandabrowski | yeah, for the second time it passed check pipeline but failed on gates... | 15:13 |
| noonedeadpunk | out of magnum topic only couple of patches left | 15:14 |
| noonedeadpunk | #link https://review.opendev.org/q/topic:%22osa/k8s-mcapi%22+status:open | 15:14 |
| noonedeadpunk | out of hashi vault patches only manila and zun left | 15:15 |
| noonedeadpunk | #link https://review.opendev.org/q/topic:%22osa_hashi_vault%22+status:open | 15:15 |
| noonedeadpunk | I believe zun is still broken on horizon plugin | 15:15 |
| noonedeadpunk | so I'd say let's backport it once it get fixed... | 15:15 |
| noonedeadpunk | I think we also need to merge galera version bump as it's having 3CVEs one with 10 score | 15:16 |
| noonedeadpunk | #link https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/990811 | 15:16 |
| noonedeadpunk | we should just restore CI fisrt | 15:16 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-galera_server master: Upgrade MariaDB to 11.8.8 https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/990811 | 15:17 |
| noonedeadpunk | and I guess that's pretty much it | 15:19 |
| noonedeadpunk | I was already pinged on branching requirement, so we're on a borrowed time now | 15:19 |
| noonedeadpunk | I think this is most important topic we have right now | 15:20 |
| noonedeadpunk | #topic office hours | 15:20 |
| noonedeadpunk | I don't have much to discuss here | 15:21 |
| noonedeadpunk | so if there're no more topics - I'd finish early :) | 15:21 |
| damiandabrowski | okok, thanks Dmitry! | 15:24 |
| noonedeadpunk | #endmeeting | 15:29 |
| opendevmeet | Meeting ended Tue Jun 2 15:29:41 2026 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:29 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2026/openstack_ansible_meeting.2026-06-02-15.00.html | 15:29 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2026/openstack_ansible_meeting.2026-06-02-15.00.txt | 15:29 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2026/openstack_ansible_meeting.2026-06-02-15.00.log.html | 15:29 |
| drarvese | Hello. I've got a fresh 2025.2 deployment and Horizon is failing to load the CSS files, returning a 500 error on the requests. It seems to be an issue with haproxy, since requests to the individual containers work. | 15:32 |
| drarvese | I've used the same haproxy & Horizon settings on a number of previous deployments with no issues. I compared the haproxy configs with a 2025.1 install and nothing's changed between the two. | 15:32 |
| drarvese | I was able to get the CSS files to load by removing "connect-src 'self' our-fqdn:* wss://our-fqdn:6083" from haproxy's Content-Security-Policy-Report-Only header, but that setting is present on our other deployments so I'm not sure if that's the right solution... | 15:33 |
| noonedeadpunk | there were changes in horizon dependencies afaik | 15:33 |
| noonedeadpunk | there were series of patches to horizon plugins to get in line with this | 15:33 |
| drarvese | Here's a paste of the relevant user_variables.yml settings: https://paste.opendev.org/show/bQ9NDllQQGfI4Fx3lBsF/ | 15:34 |
| noonedeadpunk | so if you set `haproxy_security_headers_csp_report_only: true` does this solve the horizon issue? | 15:36 |
| drarvese | that's already set | 15:36 |
| noonedeadpunk | So I would assume that the issue you is having is not in deployment, but in horizon version or plugins | 15:37 |
| noonedeadpunk | I am fully aware there were issues on 2026.1 which were fixed only by patches | 15:41 |
| drarvese | I'm installing 32.1.1 if that makes any difference | 15:49 |
| noonedeadpunk | I would need to spawn a test environment to check on that | 15:49 |
| noonedeadpunk | like things like https://review.opendev.org/c/openstack/magnum-ui/+/983751 | 15:52 |
| noonedeadpunk | but I would really guess that the reason is in horizon/individual dashboards | 16:02 |
| f0o | noonedeadpunk: seems like `Set haproxy service state` doesnt honor -l params, I currently have it set to 'all:!n2_2' but it still tries to connect to n2_2 which is currently offline and the task fails | 16:12 |
| noonedeadpunk | No it does not, as it's designed to pass when your're limiting to the service hosts | 16:13 |
| noonedeadpunk | ie - when you do -l control01-nova-api-container you still got it disabled on all haproxies | 16:13 |
| f0o | guess I get to edit the haproxy files by hand until tomorrow :S | 16:13 |
| noonedeadpunk | I guess you could do --skip-tags common-haproxy | 16:14 |
| f0o | the sed -i was quicker - heh | 16:15 |
| noonedeadpunk | but if haproxy is unavailable, why not to drop it from the inventory? | 16:15 |
| f0o | its only unavailable temporarily | 16:15 |
| noonedeadpunk | well, still? | 16:15 |
| f0o | wouldnt that cause all sorts of issues when I re-add it? | 16:15 |
| noonedeadpunk | I don't think it should? | 16:16 |
| noonedeadpunk | or well keepalived potentially can failover vip | 16:16 |
| noonedeadpunk | ah | 16:17 |
| noonedeadpunk | you can also try overriding `haproxy_target_hosts` | 16:17 |
| noonedeadpunk | https://opendev.org/openstack/openstack-ansible-plugins/src/branch/master/roles/haproxy_endpoint_manage/tasks/main.yml#L29 | 16:17 |
| noonedeadpunk | add that to user_variables.yml and explicitly state hosts that are reachable | 16:17 |
| noonedeadpunk | drarvese: I am spawning a sandbox now to check, but will likely able to get back to you only tomorrow with results | 16:29 |
| drarvese | Thanks, I appreciate it | 16:29 |
| f0o | noonedeadpunk: thanks for all the pointers, I'm gonna check out for today | 17:00 |
| opendevreview | Merged openstack/openstack-ansible-os_mistral master: Switch from wsgi script to wsgi module https://review.opendev.org/c/openstack/openstack-ansible-os_mistral/+/950513 | 17:09 |
| opendevreview | Merged openstack/openstack-ansible-lxc_hosts master: Ensure ubuntu_virt hook not passed to LXC https://review.opendev.org/c/openstack/openstack-ansible-lxc_hosts/+/990070 | 17:31 |
| noonedeadpunk | drarvese: I deployed an AIO with 32.1.1 and horizon looks like working as expected :( | 18:23 |
| noonedeadpunk | At least I don't see any issues right away | 18:24 |
| noonedeadpunk | so probably some more detauls on the error/logs is needed | 18:24 |
| opendevreview | Merged openstack/openstack-ansible-haproxy_server master: Add certbot package variable for RHEL based systems https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/967195 | 19:08 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-haproxy_server stable/2025.2: Add certbot package variable for RHEL based systems https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/991210 | 19:45 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-haproxy_server stable/2025.1: Add certbot package variable for RHEL based systems https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/991211 | 19:45 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs:wip https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 21:48 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: wip: add information about Ubuntu 26.04 support https://review.opendev.org/c/openstack/openstack-ansible/+/989095 | 22:31 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: wip: add information about Ubuntu 26.04 support https://review.opendev.org/c/openstack/openstack-ansible/+/989095 | 22:31 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: wip: add information about Ubuntu 26.04 support https://review.opendev.org/c/openstack/openstack-ansible/+/989095 | 22:31 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: wip: add information about Ubuntu 26.04 support https://review.opendev.org/c/openstack/openstack-ansible/+/989095 | 22:34 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: Add information about Ubuntu 26.04 support https://review.opendev.org/c/openstack/openstack-ansible/+/989095 | 22:35 |
| opendevreview | Dmitriy Chubinidze proposed openstack/openstack-ansible-os_trove master: docs: clarify deployment configuration examples https://review.opendev.org/c/openstack/openstack-ansible-os_trove/+/991229 | 23:11 |
| opendevreview | Merged openstack/openstack-ansible-rabbitmq_server master: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/988733 | 23:24 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.2: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991245 | 23:53 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade RabbitMQ to 4.2.7 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989625 | 23:53 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade RabbitMQ to 4.2.7 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989625 | 23:53 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server master: Upgrade RabbitMQ to 4.2.7 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/989625 | 23:54 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.2: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991245 | 23:56 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.1: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991246 | 23:57 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.1: Add erlang package repository providing packages for arm64 https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/991246 | 23:58 |
Generated by irclog2html.py 4.1.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!