| opendevreview | Andrew Bonney proposed openstack/openstack-ansible master: DNM: Add minimal AIO files for k8s clusters https://review.opendev.org/c/openstack/openstack-ansible/+/966666 | 10:28 |
|---|---|---|
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server master: Clean-up rabbitmq_gpg_keys variable https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/967508 | 11:23 |
| noonedeadpunk | seems that https://ppa1.novemberain.com was wiped fully or partially.... | 11:23 |
| noonedeadpunk | at least affects old branches (and ubuntu 22.04) badly | 11:24 |
| noonedeadpunk | well... now it's 500... | 11:36 |
| noonedeadpunk | so could be accidential.... | 11:36 |
| jrosser | "welcome to nginx" | 11:41 |
| mgariepy | shall we start archiving everything locally.. | 11:42 |
| noonedeadpunk | I was thinking about that for repo_server for a while now... | 11:42 |
| noonedeadpunk | though storage amount could be weird.... | 11:43 |
| jrosser | thats how it used to be - caching apt proxy etc | 11:43 |
| noonedeadpunk | any recollections why it was dropped? | 11:44 |
| mgariepy | the issue is that caching would allow you to re-deploy stuff but won't allow you to upgrade. | 11:44 |
| jrosser | i think in part simplification | 11:44 |
| jrosser | and in part it was pretty much duplicate of any local mirror you might have | 11:44 |
| noonedeadpunk | right... | 11:45 |
| jrosser | i.e if you mirror locally anyway outside OSA then having another cache in the repo server was a bit pointless | 11:45 |
| jrosser | though mirroring might always remove content that was removed upstream, so not a guarantee | 11:45 |
| noonedeadpunk | I don't have mirrors for each of deployment though | 11:45 |
| jrosser | errr | 11:49 |
| jrosser | what about ppa1.rabbitmq.com | 11:49 |
| jrosser | is that also broken? | 11:49 |
| noonedeadpunk | well, for master we flipped it: https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/965196/3/vars/debian.yml#19 | 11:49 |
| noonedeadpunk | but only for rabbitmq - not erlang | 11:50 |
| noonedeadpunk | so likely we need to do same for erlang now.... | 11:50 |
| noonedeadpunk | and potentially backport... | 11:50 |
| noonedeadpunk | but really not sure | 11:50 |
| mgariepy | err aren't ll these doing the same server error on cloudflare? | 11:51 |
| noonedeadpunk | I was really thinking about having some optional pulp... but yeah.... | 11:51 |
| noonedeadpunk | it was 404 at first for https://ppa1.novemberain.com | 11:53 |
| mgariepy | Investigating - Cloudflare is aware of, and investigating an issue which potentially impacts multiple customers. Further detail will be provided as more information becomes available. | 11:53 |
| mgariepy | Nov 18, 2025 - 11:48 UTC | 11:53 |
| noonedeadpunk | ah | 11:53 |
| noonedeadpunk | well. does not explain 404 | 11:53 |
| jrosser | i am not seeing any cloudflare errors at all | 11:56 |
| noonedeadpunk | I do | 11:56 |
| jrosser | but anyway - just looking at a patch to switch the erlang repo across | 11:57 |
| noonedeadpunk | https://paste.openstack.org/show/bNNo9vozIorW1HhQj4z7/ | 11:57 |
| noonedeadpunk | `deb https://ppa1.novemberain.com/rabbitmq/rabbitmq-server/deb/ubuntu jammy main` was 404-ing for me, yeah | 11:58 |
| noonedeadpunk | and same for rabbitmq-erlanf | 11:58 |
| jrosser | ooooh yes from something other than the fqdn yes | 11:58 |
| noonedeadpunk | why I did not switch rabbitmq-erlang - as deb1 was not having erlang for trixie | 11:58 |
| jrosser | i think cloudflare are having a bad day | 11:59 |
| noonedeadpunk | there's also a ML kolla having same issue | 12:13 |
| noonedeadpunk | one thing I've already figured out, is that changing mirror URL will not fully solve it, as it will add a new mirror on older branches | 12:14 |
| noonedeadpunk | rather then swap it... | 12:14 |
| noonedeadpunk | jrosser: are you looking on the patch? or should I? | 12:37 |
| mnasiadka | Yeah well, no brainer we have the same issue | 12:53 |
| mnasiadka | Although I would appreciate if the whole internet wouldn’t be against us (cloudflare outage) | 12:53 |
| noonedeadpunk | right... | 13:12 |
| noonedeadpunk | mnasiadka: the only thing is that I had issues with this new rpo and erlang and debian 13 | 13:12 |
| mnasiadka | We don’t support debian 13 yet | 13:13 |
| noonedeadpunk | as the version I wanted was not there, like whole major release was not there | 13:13 |
| mnasiadka | So at least this is not going to bite me :) | 13:13 |
| noonedeadpunk | but it was on old one | 13:13 |
| noonedeadpunk | yeah, this was the only reason why we did not switch it for master even... | 13:13 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server master: Switch erlang repo to deb1.rabbitmq.com https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/967523 | 13:17 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.1: Change repository for deb to deb1.rabbitmq.com https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/967525 | 13:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server stable/2025.1: Change repository for deb to deb1.rabbitmq.com https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/967525 | 13:25 |
| jrosser | noonedeadpunk: thanks for making the patch, got into meetings here | 14:09 |
| noonedeadpunk | I'm more concerned on how to backport it further | 14:10 |
| noonedeadpunk | #startmeeting openstack_ansible_meeting | 15:00 |
| opendevmeet | Meeting started Tue Nov 18 15:00:27 2025 UTC and is due to finish in 60 minutes. The chair is noonedeadpunk. Information about MeetBot at http://wiki.debian.org/MeetBot. | 15:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 15:00 |
| opendevmeet | The meeting name has been set to 'openstack_ansible_meeting' | 15:00 |
| noonedeadpunk | #topic roll call | 15:00 |
| damiandabrowski | hi! | 15:00 |
| noonedeadpunk | o/ | 15:00 |
| jrosser | o/ hello | 15:04 |
| noonedeadpunk | #topic office hours | 15:04 |
| noonedeadpunk | so I think the biggest issue right now, except approaching deadling for 2025.2 release, is broken rabbitmq repos | 15:05 |
| noonedeadpunk | which affects stable and master branch | 15:05 |
| noonedeadpunk | while for master it's easy to fix, everything before 2025.1 (and deb822) is not | 15:05 |
| noonedeadpunk | as changing URI for the repo will add just a new record to the existing file, rather then replace it | 15:06 |
| jrosser | is that how the deb822 module works? | 15:06 |
| noonedeadpunk | yeah, deb822 will just update the URI in the existing repo file I believe | 15:07 |
| jrosser | thats strange isnt it | 15:07 |
| jrosser | oh you mean deb822 module is additive or is not additive for uri | 15:08 |
| noonedeadpunk | but in old /etc/apt/sources.list.d/RabbitMQ.list you'll get just 2 records | 15:08 |
| noonedeadpunk | ie https://paste.openstack.org/show/btvdsjM7xIvaztJOv0cH/ | 15:08 |
| noonedeadpunk | I don't see any description of "exclusive" for https://docs.ansible.com/projects/ansible/latest/collections/ansible/builtin/apt_repository_module.html | 15:10 |
| noonedeadpunk | so we need to somehow handle a clean-up on older backports | 15:10 |
| jrosser | we can probably lineinfile delete the old things | 15:13 |
| noonedeadpunk | yeah... | 15:13 |
| noonedeadpunk | I think it might be the best option | 15:13 |
| jrosser | using the name of the var for the repo, as it might be overidden | 15:14 |
| noonedeadpunk | we should use the default value to avoid this, not var | 15:14 |
| noonedeadpunk | or well. old default | 15:14 |
| jrosser | yep | 15:14 |
| noonedeadpunk | as we're changing the var | 15:14 |
| noonedeadpunk | ok, will do that | 15:15 |
| noonedeadpunk | we're coming really to a point we need to do role branching | 15:15 |
| noonedeadpunk | it's jsut 2 week till the deadling now | 15:15 |
| jrosser | we really should merge that big bunch of policy patches | 15:15 |
| noonedeadpunk | yeah | 15:15 |
| noonedeadpunk | and I think at this point we have to branch | 15:16 |
| jrosser | https://review.opendev.org/q/topic:%22bug/2112559%22 | 15:16 |
| jrosser | ^ andrewbonney | 15:16 |
| noonedeadpunk | the gnocchi one should be "fixed" though... but yeah | 15:16 |
| noonedeadpunk | there are also some other bug fixes I proposed last week worth looking at, like consoles slight refactoring... | 15:17 |
| noonedeadpunk | https://review.opendev.org/q/topic:%22bug/2122778%22 | 15:18 |
| noonedeadpunk | and at this point I think we have to leave openbao for pki to 2026.1 :( | 15:19 |
| damiandabrowski | that's super sad :/ | 15:20 |
| noonedeadpunk | but it seems there are quite some comments left to be addressed at this point? | 15:21 |
| noonedeadpunk | do we feel like merging all the topic by end of the week? | 15:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-rabbitmq_server master: Clean-up rabbitmq_gpg_keys variable https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/967508 | 15:23 |
| damiandabrowski | only 3 I guess, I aimed to propose patches tomorrow | 15:23 |
| damiandabrowski | we can merge the most important patches, that adds hashi_vault support to ansible-role-pki and to the integrated repo | 15:23 |
| noonedeadpunk | well, let's see, but we have to branch like early next week to have a chance to be in time | 15:23 |
| noonedeadpunk | and we already like one week behind with this | 15:23 |
| damiandabrowski | we probably won't be able to merge patches in all services, but maybe they can be merged in 25.0.2? | 15:24 |
| noonedeadpunk | but can it be used without the roles patches? as I think it makes sense to have everything or nothing? | 15:24 |
| noonedeadpunk | like we are not passing a backend to pki role otherwise | 15:25 |
| noonedeadpunk | so you can't use `openstack_pki_backend` | 15:26 |
| noonedeadpunk | yo ucan try with `pki_backend` | 15:26 |
| jrosser | tbh i think we need to spend time this week making all the roles working | 15:27 |
| noonedeadpunk | manila and magnum seem to be most problematic this time | 15:28 |
| noonedeadpunk | based of polic patches | 15:28 |
| jrosser | andrewbonney has been looking at magnum in the last week | 15:28 |
| * noonedeadpunk failed to spend any time on weekend | 15:28 | |
| damiandabrowski | yeah, you can't just switch openstack_pki_backend if we don't merge patches in service roles | 15:29 |
| damiandabrowski | , but with overriding *_certificates and *_install_certificates for each service, you would be able to leverage hashi_vault support | 15:29 |
| damiandabrowski | anyway, I'll work on this tomorrow | 15:29 |
| noonedeadpunk | ok, anything else? | 15:32 |
| jrosser | gnocchi role looks sad too | 15:32 |
| noonedeadpunk | well, it's result of the patch | 15:32 |
| noonedeadpunk | (I think it is) | 15:33 |
| jrosser | oh ok | 15:33 |
| noonedeadpunk | it has weird logic which no other role has regarding policy files | 15:33 |
| noonedeadpunk | `msg: No user [ src ] or [ content ] was provided` | 15:33 |
| noonedeadpunk | for /etc/gnocchi/policy.yaml-32.0.0.0b2.dev34 | 15:33 |
| noonedeadpunk | actually | 15:33 |
| noonedeadpunk | it has completely different approach as I said | 15:34 |
| noonedeadpunk | at that was the reason for it: | 15:34 |
| noonedeadpunk | #link https://opendev.org/openstack/openstack-ansible-os_gnocchi/src/commit/87cc1bf816689230f9678378c824dd48d5cd480d/handlers/main.yml#L36-L42 | 15:34 |
| noonedeadpunk | so it's completely opposite from what we;'re doing everywhere else | 15:34 |
| noonedeadpunk | as here is a tradeoff between upgrade vs operations | 15:35 |
| NeilHanlon | 🤦well i've actually updated my calendar now, so that's good | 15:35 |
| noonedeadpunk | and I think I'm in favor of operations right now | 15:35 |
| noonedeadpunk | hehe | 15:35 |
| noonedeadpunk | and I'm thinking to align gnocchi to others as well | 15:36 |
| * NeilHanlon stares at Thunderbird thinking it is Sunday, Nov 16 still | 15:37 | |
| noonedeadpunk | I wish it was... It would give way more time on hands to work on things... | 15:37 |
| NeilHanlon | same... | 15:37 |
| noonedeadpunk | so it's smart :) | 15:37 |
| NeilHanlon | re: gnocchi -- that sounds reasonable re: favoring ops | 15:38 |
| noonedeadpunk | ++ k, will propose patche then | 15:38 |
| noonedeadpunk | so if that's it, I'd propose to wrap it up | 15:39 |
| noonedeadpunk | and as cloudflare seems to recover - we may do bunch of rechecks now | 15:39 |
| NeilHanlon | seems good to me | 15:40 |
| noonedeadpunk | #endmeeting | 15:44 |
| opendevmeet | Meeting ended Tue Nov 18 15:44:03 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:44 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2025/openstack_ansible_meeting.2025-11-18-15.00.html | 15:44 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/openstack_ansible_meeting/2025/openstack_ansible_meeting.2025-11-18-15.00.txt | 15:44 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/openstack_ansible_meeting/2025/openstack_ansible_meeting.2025-11-18-15.00.log.html | 15:44 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_ceilometer master: Add coordination support to the role https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/967064 | 15:48 |
| noonedeadpunk | we already finished :) | 15:57 |
| DavidGomez | ah I wonder if it's a daylight savings thing where I need to update my calendar time for this meeting | 16:02 |
| noonedeadpunk | it is :) | 16:02 |
| *** jonher_ is now known as jonher | 16:20 | |
| damiandabrowski | noonedeadpunk: how would you install venv for pki role? | 16:26 |
| damiandabrowski | I'm asking because this role doesn't have any standard "installation" playbook, it's triggered only for issuing/installing certs | 16:26 |
| *** starkis is now known as Guest31500 | 16:36 | |
| damiandabrowski | considering that each service can have different setup_host(we use vars like nova_pki_setup_host), we need to prepare pki venv on each of them | 16:38 |
| noonedeadpunk | I'd be thinking the same way as for the uwsgi? | 16:49 |
| noonedeadpunk | But indeed, that would trigger python_venv_build role include for each include of pki role | 16:50 |
| noonedeadpunk | which would increase runtime | 16:50 |
| noonedeadpunk | (at least when vault is used, but probably in general as well) | 16:50 |
| *** kleini- is now known as kleini | 17:17 | |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!