| jrosser | o/ morning | 07:04 |
|---|---|---|
| spotz[m] | Hey | 07:08 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-openstack_hosts master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-openstack_hosts/+/907434 | 07:12 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/907752 | 07:40 |
| jrosser | noonedeadpunk: ^ interested on your opinion here | 07:40 |
| jrosser | i noticed that we have the "external repo" logic in a conditional block for both galera and rabbitmq | 07:40 |
| jrosser | so that only runs when we set external_repo to true, but will not do the proper cleanup of repo config if external_repo is changed from true to false | 07:41 |
| jrosser | i changed the deb822 galera role to manage the state (present/absent) of the repo based on external_repo | 07:43 |
| noonedeadpunk | oh, that is really nice thing to do | 07:58 |
| jrosser | i think i should look again at the rabbitmq patch and make it the same | 07:59 |
| noonedeadpunk | the only thing is that we should do the same for redhat | 08:02 |
| noonedeadpunk | yum_repository also does support state, but we don't pass it at all | 08:02 |
| noonedeadpunk | so probably doing that in follow-up not a bad idea | 08:03 |
| jrosser | sure i can do that | 08:03 |
| jrosser | do you know why we have the `when: galera_repo` condition | 08:14 |
| jrosser | is there some time it may be `[]` or `False` | 08:14 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Improve handling of galera_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/927901 | 08:18 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-apt_package_pinning master: Add ability to control the state of apt pins. https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/927903 | 08:27 |
| noonedeadpunk | jrosser: I think it was a way to use `distro` before it become a thing | 08:31 |
| noonedeadpunk | as galera_install_method was added quite recently to galera role | 08:32 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/907833 | 08:34 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Improve handling of galera_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/927906 | 08:40 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Improve handling of rabbitmq_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/927906 | 08:41 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/907752 | 08:51 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Improve handling of galera_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/927901 | 08:51 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/907833 | 08:52 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Improve handling of rabbitmq_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/927906 | 08:52 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/907833 | 09:05 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-rabbitmq_server master: Improve handling of rabbitmq_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-rabbitmq_server/+/927906 | 09:05 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-ops master: Update magnum-cluster-api version https://review.opendev.org/c/openstack/openstack-ansible-ops/+/926347 | 09:09 |
| jrosser | this looks ugly https://paste.opendev.org/show/btGCszJq0wmxBqPX40gI/ | 09:17 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Add Ubuntu Noble to the list of supported LXC operating systems https://review.opendev.org/c/openstack/openstack-ansible/+/927909 | 09:32 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Test on Ubuntu Noble https://review.opendev.org/c/openstack/openstack-ansible/+/924342 | 09:32 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Test on Ubuntu Noble https://review.opendev.org/c/openstack/openstack-ansible/+/924342 | 09:33 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Add support for Ubuntu 24.04 to docs https://review.opendev.org/c/openstack/openstack-ansible/+/924829 | 09:33 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-apt_package_pinning master: Fix linters https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/927910 | 09:35 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_magnum master: Add high availability k8s control plane test on Ubuntu Noble https://review.opendev.org/c/openstack/openstack-ansible-os_magnum/+/927911 | 09:39 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Manage apt repositores and keys using deb822_repository module https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/907752 | 09:49 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-galera_server master: Improve handling of galera_install_method changing https://review.opendev.org/c/openstack/openstack-ansible-galera_server/+/927901 | 09:49 |
| noonedeadpunk | is that from CI? | 09:50 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-apt_package_pinning master: Fix linters https://review.opendev.org/c/openstack/openstack-ansible-apt_package_pinning/+/927910 | 09:51 |
| noonedeadpunk | ah, recalled to revise skyline patch... | 09:51 |
| noonedeadpunk | which you commented looking ugly as well :D | 09:51 |
| jrosser | yes that was from one of the stepca failed jobs | 09:51 |
| jrosser | just in basic_server_ops tempest test | 09:51 |
| noonedeadpunk | yeah, looks not good | 09:51 |
| jrosser | also unusal error in your master SHA bump patch too | 09:57 |
| jrosser | it's almost like the contents of the utility venv are inconsistent | 09:59 |
| rambo | rambo2412 | 09:59 |
| *** rambo is now known as rambo2412 | 10:00 | |
| rambo2412 | Hi Dmitriy | 10:00 |
| noonedeadpunk | o/ | 10:00 |
| noonedeadpunk | jrosser: I haven't checked SHA bump reuslts yet though | 10:01 |
| noonedeadpunk | was more into our 2024.1 upgrade which was ... fun ?:) | 10:01 |
| rambo2412 | As discussed yesterday I am preparing for Ussuri to Victoria upgrade. I have some more queries on it. | 10:02 |
| rambo2412 | In our user_variable.yml file there is no apply_security_hardening parameter, so by default it is true or false? | 10:03 |
| rambo2412 | also, on setup-hosts.yml playbook which create and updates lxc containers, what impact is expected if we already have a openstack ussuri installed existing LXC containers? | 10:04 |
| jrosser | noonedeadpunk: make patches for managing the "fun"! | 10:04 |
| rambo2412 | will it restart the LXC containers , if yes what will be the sequence of that? | 10:05 |
| noonedeadpunk | it wasn't _that_ fun - just missed uwsgi disablement. And this somehow resulted in net nodes spawning 400 namespaces after reboot for 3 hours | 10:09 |
| noonedeadpunk | switching back to eventlet jsut solved it... | 10:09 |
| noonedeadpunk | and another "fun" were stream queues and our monitoring not ready to deal with streams - so was alerting as crazy by non-consumed messages in fanout queues | 10:09 |
| noonedeadpunk | rambo2412: apply_security_hardening should be true by default | 10:10 |
| noonedeadpunk | here's the default for it: https://opendev.org/openstack/openstack-ansible/src/branch/unmaintained/victoria/inventory/group_vars/all/all.yml#L126 | 10:12 |
| noonedeadpunk | I don't think it will restart containers? | 10:13 |
| noonedeadpunk | but you can also set `lxc_container_allow_restarts: false` jsut to be extra sure they won't | 10:14 |
| rambo2412 | okay so it is true , during major upgrade shall we keep it false . as I see in minor update there is recommendation to keep it security hardening as false. | 10:14 |
| noonedeadpunk | Also, possible containers restart is placed after backend is disabled in HAProxy from what I see | 10:14 |
| noonedeadpunk | again - there's a note that on Victoria not all playbooks do disable haproxy or run in serial... | 10:15 |
| noonedeadpunk | No idea why you would disable hardening for upgrades.... | 10:15 |
| rambo2412 | yeah is there a way to check which LXC containers will be restarted and in which order? | 10:16 |
| rambo2412 | it is mentioned in this page https://docs.openstack.org/openstack-ansible/ussuri/admin/upgrades/minor-upgrades.html | 10:17 |
| noonedeadpunk | so there is a common-task which can restart container if it's config has changed | 10:17 |
| noonedeadpunk | such task is included in individual playbooks | 10:18 |
| noonedeadpunk | let's take glance as example: https://opendev.org/openstack/openstack-ansible/src/branch/unmaintained/victoria/playbooks/os-glance-install.yml#L46-L53 | 10:18 |
| noonedeadpunk | as you see - it comes after `haproxy-endpoint-manage.yml` which disables the backend first | 10:18 |
| noonedeadpunk | and restart is triggered only if mounts or container config is changed: https://opendev.org/openstack/openstack-ansible/src/branch/unmaintained/victoria/playbooks/common-tasks/os-lxc-container-setup.yml#L114-L116 | 10:19 |
| rambo2412 | okay got it , os-glance-install.yml is part of setup-openstack.yml playbook, so that quite clear that it will follow serial parameter and ha_proxy will disable the backend before restart. | 10:20 |
| rambo2412 | I am more concerned about setup-hosts.yml , which has one playbook container-deploy.yml. | 10:21 |
| rambo2412 | just wondering if this playbook will make any change or restart if we already have LXC containers deployed as part of Ussuri installation. | 10:23 |
| jrosser | rambo2412: you could test a bunch of this stuff out in an all-in-one build if you want to see what does/doesnt happen with the upgrade | 10:29 |
| jrosser | it's possible to make one with multiple containers for each service if your VM is big enough so that the way serial/not serial is done | 10:30 |
| jrosser | can be seen | 10:30 |
| noonedeadpunk | rambo2412: container-deploy.yml should not restart containers for sure | 10:30 |
| noonedeadpunk | or well, dunno, but I never seen issues there | 10:30 |
| noonedeadpunk | it would be a bit tricky to spawn AIO with Ussuri though.. but quite alike to how upgade anyway | 10:31 |
| rambo2412 | okay thanks yeah I would love to have one AIO lab , yeah Ussuri installation is not possible these days looks like some repos are broken. | 10:32 |
| noonedeadpunk | I guess it should be possible, but would require some effort | 10:36 |
| rambo2412 | is it possible to deploy openstack AIO on a VMware VM? | 10:38 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Remove ceph-server common tasks https://review.opendev.org/c/openstack/openstack-ansible/+/927917 | 10:52 |
| noonedeadpunk | rambo2412: if it allows nested virtualization - then yes | 10:54 |
| noonedeadpunk | probably it's possible even without nested virt though | 10:54 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-plugins master: Add infrastructure playbooks to openstack-ansible-plugins collection https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/924171 | 11:00 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_skyline master: Enable SSLProxyProtocol for internal backends behind TLS https://review.opendev.org/c/openstack/openstack-ansible-os_skyline/+/927248 | 11:09 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Use infrastructure playbooks from openstack-ansible-plugins repo https://review.opendev.org/c/openstack/openstack-ansible/+/924253 | 11:12 |
| noonedeadpunk | yeah, `ValueError: check_hostname requires server_hostname` is very weird in SHA bump | 11:39 |
| jrosser | i saw it was installing setuptools into the utility venv so maybe we end up with a broken combination of the venv vendored things and some other dependencies | 11:40 |
| noonedeadpunk | and affects ubuntu/tls jobs only | 11:41 |
| noonedeadpunk | stackoverflow says it has smth to do with requests version | 11:41 |
| noonedeadpunk | and proxy and schema | 11:41 |
| noonedeadpunk | worth try to reproduce | 11:42 |
| noonedeadpunk | btw | 11:46 |
| noonedeadpunk | centos fsailing quite consistently on DB creation for Keystone | 11:46 |
| jrosser | also horizon static file compression is failing quite a few jobs https://paste.opendev.org/show/bzuVIZE8bISWHMqKXfED/ | 12:01 |
| jrosser | i wonder if we should just start by dropping the use of async on that task | 12:20 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-os_horizon master: Remove use of async when compressing static files https://review.opendev.org/c/openstack/openstack-ansible-os_horizon/+/927953 | 12:47 |
| noonedeadpunk | well removal of async won't help in any way frankly speaking | 12:56 |
| noonedeadpunk | with `An error occurred during rendering serial_console.html` at least will not... | 12:57 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Use infrastructure playbooks from openstack-ansible-plugins repo https://review.opendev.org/c/openstack/openstack-ansible/+/924253 | 13:02 |
| gokhan_ | hello noonedeadpunk, it seems we can not install garbd with with galera role. how you are installing garbd on your env? are you overriding galera role or installing with different methods. | 13:02 |
| noonedeadpunk | I ended up just with 3 galera nodes, but wrote down add garbd support in todo list | 13:03 |
| noonedeadpunk | so image is pretty much design idea but things like garbd and proxysql not around yet :( | 13:04 |
| noonedeadpunk | and eventually garbd probably doesn't make sense without proxysql | 13:04 |
| noonedeadpunk | but dunno | 13:04 |
| noonedeadpunk | (forget it, it was stupid assumption) | 13:04 |
| jrosser | i am making a guess that "Syntax error: Found 'inline-blo'" is somehow threading related | 13:04 |
| jrosser | as it seems to be a trunction of 'inline-block' | 13:05 |
| noonedeadpunk | ah, huh | 13:05 |
| jrosser | having said that, its kind of always the same error when it does break | 13:06 |
| noonedeadpunk | I more wonder if there's something in django that does corrupt things when doing compilation | 13:07 |
| jrosser | could easily be that | 13:09 |
| gokhan_ | ok thanks noonedeadpunk, we will also work on adding garbd and proxysql. | 13:13 |
| noonedeadpunk | I actually do have a starting point for proxysql support | 13:14 |
| noonedeadpunk | though it's now like 3yo :( | 13:14 |
| noonedeadpunk | I really want to get it done though once some free time appear on my hands | 13:15 |
| noonedeadpunk | most tricky part I've stopped on was auth, as proxysql requires to have a transition user | 13:15 |
| noonedeadpunk | but garbd support - that would be really nice | 13:15 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible-plugins master: Add infrastructure playbooks to openstack-ansible-plugins collection https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/924171 | 13:30 |
| opendevreview | Jonathan Rosser proposed openstack/openstack-ansible master: Use hosts setup playbooks from openstack-ansible-plugins repo https://review.opendev.org/c/openstack/openstack-ansible/+/924259 | 13:56 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Use cgroupsv2 for L3 cleanup https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/927989 | 14:20 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Do not kill ipsec on L3 cleanup https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/927992 | 14:22 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_neutron master: Remove ns-metadata-proxy cleanuop handler https://review.opendev.org/c/openstack/openstack-ansible-os_neutron/+/927993 | 14:24 |
| opendevreview | Merged openstack/openstack-ansible stable/2023.1: Remove the get_md5 parameter from ansible stat tasks https://review.opendev.org/c/openstack/openstack-ansible/+/927721 | 16:31 |
| *** rambo is now known as Guest2508 | 17:57 | |
| *** rambo is now known as Guest2524 | 20:27 | |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!