| f0o | Morning! | 06:12 |
|---|---|---|
| f0o | I noticed swift-object-expirer wasnt installed with distro on Ubuntu Jammy and it turns out it has its own package now. No idea how it is with Rocky/Centos | 06:12 |
| f0o | Also saw that swift tries to deploy mlocate daily cron but doesnt actually install it. mlocate is deprecated (at least on ubuntu) for plocate. How is it used inside swift? Do we actually need it? | 06:13 |
| f0o | quick grep over the swift codebase didnt find any references to [mp]locate or locatedb or similar | 06:20 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Remove seemingly unsused mlocate cronjob https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/947050 | 06:22 |
| opendevreview | Merged openstack/openstack-ansible master: [doc] Update docs on how to use inventory scripts https://review.opendev.org/c/openstack/openstack-ansible/+/946030 | 06:50 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-plugins master: Move regenerate handler into task to trigger more reliably https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/946998 | 07:37 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-plugins master: Flush Handlers to regenerate trusted_ca early https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/946998 | 07:37 |
| noonedeadpunk | f0o: are you using distro install? or was just checking different code paths? | 07:40 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Move zuul preparation for role/collection bootstrap https://review.opendev.org/c/openstack/openstack-ansible/+/939151 | 07:40 |
| f0o | noonedeadpunk: I am | 07:41 |
| f0o | but if you refer to the mlocate thing, I was looking at the swift codebase itself. The packages dont recommend/suggest m/plocate at all either | 07:42 |
| f0o | so this daily-cron is at best a noop since crontab isnt installed but in worst case it will just fail because [mp]locate isnt installed | 07:42 |
| f0o | and I cant find any references to it anywhere other than "by default mlocate excludes xfs" from the commit-msgs which doesnt explain why it was needed to start with | 07:43 |
| noonedeadpunk | yeah, might be fair to drop it | 07:43 |
| noonedeadpunk | I guess it was just added for operator to search for files on the storage | 07:44 |
| f0o | possibly | 07:44 |
| f0o | was a bit funny to see it checking for battery state | 07:44 |
| noonedeadpunk | I wonder if removal of mlocate really depends on all these other changes | 07:50 |
| noonedeadpunk | it looks safe to rebase it to master directly | 07:51 |
| noonedeadpunk | and also it's the easiest one to vote on right away :D | 07:51 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-haproxy_server master: Add custom errorfiles https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/946617 | 07:52 |
| f0o | ^ just now saw your comment on this one, sorry for the delay | 07:52 |
| f0o | noonedeadpunk: I can probably remove it from the set and make it a standalone change | 07:53 |
| noonedeadpunk | it;s also in merge conflict it seems | 07:53 |
| f0o | (the mlocate thing) | 07:53 |
| noonedeadpunk | you can rebase i gerrit directly | 07:53 |
| f0o | oh neat | 07:53 |
| noonedeadpunk | *in gerrit | 07:53 |
| f0o | noonedeadpunk: regarding the whitespace, isnt it needed for the | block to continue? I'm not YAML native | 07:54 |
| noonedeadpunk | I think not? | 07:55 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Remove seemingly unsused mlocate cronjob https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/947050 | 07:55 |
| f0o | no clue :D | 07:55 |
| f0o | but I can remove it, it's only an example anyway | 07:55 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-haproxy_server master: Add custom errorfiles https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/946617 | 07:56 |
| noonedeadpunk | btw, you can also `content: lookup('file', 'etc/openstack_deploy/haproxy/500.http')` | 07:56 |
| f0o | Ah yes that's what I was going to use for when its merged but I think the example should just show that the contents are almost verbatim | 07:57 |
| noonedeadpunk | oh, just "{{ lookup(...) }}" ofc | 07:57 |
| noonedeadpunk | ++ | 07:57 |
| noonedeadpunk | ok, cool | 07:57 |
| noonedeadpunk | right | 07:57 |
| f0o | I mean either way. if it's verbatim then non-ansible people (like me) understand that its just a long string | 07:57 |
| f0o | if its a function you need to know what the function is | 07:57 |
| noonedeadpunk | yes, absolutely. jsut decided to mention that it's a thing | 07:57 |
| noonedeadpunk | was not suggestign to change the example :) | 07:58 |
| f0o | I can add it in the comments too | 07:58 |
| f0o | add a 2nd one with the func | 07:58 |
| f0o | just to cover all bases | 07:58 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-haproxy_server master: Add custom errorfiles https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/946617 | 07:59 |
| noonedeadpunk | about generating rings on repo server - I was actually looking at imports here https://opendev.org/openstack/openstack-ansible-os_swift/src/branch/master/templates/swift_rings.py.j2#L18-L19 | 08:02 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-haproxy_server master: Add custom errorfiles https://review.opendev.org/c/openstack/openstack-ansible-haproxy_server/+/946617 | 08:03 |
| f0o | rebased :) | 08:03 |
| noonedeadpunk | and realized that we'd need then to install swift on repo server, which is not good at all | 08:03 |
| f0o | that is true | 08:03 |
| noonedeadpunk | from other side - I realized why we needed to copy to the rings to deploy host and then to swift | 08:03 |
| f0o | you can install swift on the utility server and then move the rings to the repo server :| | 08:03 |
| noonedeadpunk | as in case of distro upgrade - it's vital not to drop the first swift container | 08:04 |
| noonedeadpunk | as rings are pretty much stateful? | 08:04 |
| f0o | I dont think they are actually | 08:04 |
| noonedeadpunk | ah, ok | 08:04 |
| f0o | if the same disks and zones and hosts exist then you can just rebuild them straight away | 08:04 |
| f0o | at least that's what my back of the head is telling me from Mitaka times | 08:05 |
| noonedeadpunk | but still some safegurads are probably needed in case the first container is unreachable | 08:05 |
| noonedeadpunk | (but maybe not) | 08:05 |
| f0o | the rings will always rebuild entirely when the topology changes (add/remove hosts or disks or zones) | 08:05 |
| f0o | and then the object replicator will handle the shuffling of data afaik | 08:05 |
| noonedeadpunk | I need to keep thinking on it :D | 08:05 |
| noonedeadpunk | yeah, actually building on utitilty might be good | 08:05 |
| noonedeadpunk | as potentially swift-common is installed with pythonclient even... or maybe not... | 08:06 |
| f0o | that I dont know :S | 08:06 |
| noonedeadpunk | it's not | 08:09 |
| noonedeadpunk | ok, so if it's safe to rebuild rings - then I guess what you did makes most sense | 08:09 |
| f0o | I guess we can always check with the swift people | 08:11 |
| f0o | I think I can also rebase the systemd timers out of the set, those are pretty standalone too | 08:14 |
| noonedeadpunk | yeah, probably | 08:23 |
| noonedeadpunk | btw | 08:24 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Migrate cronjobs to systemd-timers https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946991 | 08:24 |
| noonedeadpunk | I think you'd need to add task to clean-up old cron jobs? | 08:24 |
| f0o | hrm possibly | 08:24 |
| f0o | probably the same with the mlocate thing then | 08:26 |
| noonedeadpunk | I left the comment | 08:26 |
| noonedeadpunk | just also plase add some comment to drop this clean-up in 2025.2 | 08:26 |
| noonedeadpunk | yeah, makes sense for mlocate as well | 08:27 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Migrate cronjobs to systemd-timers https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946991 | 08:28 |
| f0o | ^ like this? | 08:28 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Remove seemingly unsused mlocate cronjob https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/947050 | 08:31 |
| f0o | noonedeadpunk: speaking of a different change; it seems that using meta:flush_handlers in the ssh_keypairs role does work; at least the CI is passing | 08:40 |
| noonedeadpunk | f0o: tbh I was thinking of adding it as the last task of main.yml | 09:37 |
| f0o | the flush_handlers? sure | 09:39 |
| noonedeadpunk | flush_handlers, yes | 09:40 |
| noonedeadpunk | and I left comment ofr https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946991 | 09:40 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-plugins master: Flush Handlers to regenerate trusted_ca early https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/946998 | 09:40 |
| f0o | ^ moved it to main.yml | 09:40 |
| f0o | Fixing your remarks | 09:41 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Migrate cronjobs to systemd-timers https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946991 | 09:42 |
| f0o | noonedeadpunk: I looked into why rocky is failing and the only thing I can think of is that sshd isnt restarted/reloaded after the swift principal config was added. I dont know why it would matter for rocky but not ubuntu but maybe there's some OS specific shenanigans going on there | 09:57 |
| f0o | but this is also the only step in the entire CI run that actively tries to use the sshca to log in | 09:59 |
| f0o | keystone's fernet distribution is skipped so that's not using it | 09:59 |
| noonedeadpunk | iirc - ssh.conf.d is not included for rocky by default | 11:12 |
| noonedeadpunk | or smth like that | 11:12 |
| noonedeadpunk | so sshd restart isrequired there | 11:12 |
| noonedeadpunk | but I think you should be able to add `Depends-On: https://review.opendev.org/c/openstack/openstack-ansible-plugins/+/946998` to the patch | 11:13 |
| noonedeadpunk | f0o: ^ | 11:13 |
| noonedeadpunk | then these 2 will be tested together, and I'd guess it will also trigger ssh restart timely | 11:13 |
| f0o | gotcha | 11:25 |
| f0o | I'm a bit confused on which of the changes tho | 11:26 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Migrate ring distribution to SSHCA https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/947054 | 11:27 |
| f0o | hrm this didnt work | 11:27 |
| opendevreview | Daniel Preussker proposed openstack/openstack-ansible-os_swift master: Migrate ring distribution to SSHCA https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946990 | 11:28 |
| f0o | so like this ^ ? | 11:28 |
| noonedeadpunk | yeah, that looks good | 11:39 |
| f0o | that didnt do the trick | 11:50 |
| f0o | oh nvmd non-voting failed unable to install stuff | 11:50 |
| f0o | doesnt seem to work | 12:06 |
| f0o | it still flushes all handlers | 12:06 |
| f0o | so it tries to restart the swift services before they're configured | 12:06 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: WIP https://review.opendev.org/c/openstack/openstack-ansible/+/947055 | 12:10 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: WIP https://review.opendev.org/c/openstack/openstack-ansible/+/947055 | 12:11 |
| opendevreview | Ivan Anfimov proposed openstack/openstack-ansible master: docs: fix for small error https://review.opendev.org/c/openstack/openstack-ansible/+/947055 | 12:11 |
| f0o | noonedeadpunk: I'm trying to alter the CORS headers in haproxy for only one service (Swift obviously) do you happen to know how easy/hard this is to accomplish? | 13:14 |
| f0o | I can leverage `haproxy_backend_options` but I dont know where I should set this | 13:21 |
| jrosser | f0o: each haproxy service is setup in the inventory group vars like this https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/swift_all/haproxy_service.yml | 13:22 |
| f0o | can I just override that by defining haproxy_swift_proxy_service>haproxy_backend_options in my user_variables.yml ? | 13:25 |
| jrosser | you have to override the whole thing | 13:25 |
| f0o | :< | 13:25 |
| f0o | https://github.com/openstack/openstack-ansible/blob/master/inventory/group_vars/swift_all/haproxy_service.yml#L35 how does this work? | 13:26 |
| jrosser | wait no thats wrong | 13:26 |
| f0o | the haproxy_swift_proxy_service_overrides specifically | 13:26 |
| jrosser | ^ yes that will do it | 13:26 |
| f0o | haproxy_swift_proxy_service_overrides>haproxy_backend_options:[] ? | 13:26 |
| f0o | I mean I can just yolo it really.. nobody using this yet | 13:26 |
| jrosser | well kind of - that pattern is used for all of the different haproxy services, swift or otherwise | 13:27 |
| noonedeadpunk | f0o: but isn't cors should be adjusted in swifgt config? | 13:49 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_swift master: Migrate ring distribution to SSHCA https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946990 | 13:57 |
| f0o | noonedeadpunk: hrm possibly? I cant remember tbh | 14:41 |
| f0o | it was quite easy to just inject the headers with the haproxy setting | 14:41 |
| noonedeadpunk | I just know that for glance, for instance, CORS is handled in a separate section: https://opendev.org/openstack/openstack-ansible-os_glance/src/branch/master/templates/glance-api.conf.j2#L159-L164 | 16:21 |
| noonedeadpunk | and siwft also have cors-related options: https://docs.openstack.org/swift/latest/config/proxy_server_config.html#default | 16:22 |
| f0o | noonedeadpunk: yeah there is a cors middleware for swift but I dont think the role supports it | 16:58 |
| f0o | at least the role has nothing regarding cors | 16:59 |
| f0o | I'll add that in on monday | 16:59 |
| noonedeadpunk | Well, role supports everything :D | 16:59 |
| noonedeadpunk | Just use overrides | 16:59 |
| f0o | just needed to take a break from the ansible stuff and get on with my PoC for swift workloads :D | 16:59 |
| f0o | possibly | 16:59 |
| noonedeadpunk | ++ | 16:59 |
| f0o | I think for swift it should be solved like horizon does CSP with an explicit setting | 17:00 |
| f0o | because it's not uncommon to use object storage for SPAs and such which then relies on CORS to a degree | 17:00 |
| f0o | https://github.com/openstack/openstack-ansible-os_swift/blob/master/templates/proxy-server.conf.j2 doesnt seem to have overrides | 17:02 |
| f0o | or am I just blind? | 17:02 |
| noonedeadpunk | https://github.com/openstack/openstack-ansible-os_swift/blob/master/defaults/main.yml#L546 | 17:12 |
| noonedeadpunk | don't tell you never used overrides so far? | 17:12 |
| f0o | I'm not super used to the concept tbh | 17:13 |
| noonedeadpunk | pretty much the doc is here: https://docs.openstack.org/ansible-config_template/latest/ | 17:13 |
| noonedeadpunk | so you can add/modifyu anything in template | 17:13 |
| f0o | oh ok | 17:13 |
| noonedeadpunk | as we're using custom module which supports that | 17:13 |
| f0o | I always assumed it was parsed in some way | 17:13 |
| f0o | and not just dumped into the config | 17:13 |
| f0o | TIL | 17:13 |
| noonedeadpunk | so you can do smth like https://paste.openstack.org/show/buXeWlZnA51Ua0j9hx5U/ | 17:14 |
| noonedeadpunk | we support that for yaml, json and regular conf files only though | 17:14 |
| noonedeadpunk | so in case of haproxy - we had to do tons of weird complexity, as haproxy conf files are in "weird" format | 17:15 |
| noonedeadpunk | but, we have ability for such overrides for all systemd service files | 17:15 |
| noonedeadpunk | for proxy server: https://github.com/openstack/openstack-ansible-os_swift/blob/master/defaults/main.yml#L363 | 17:16 |
| noonedeadpunk | so you really don't need role support for some fine-tunning | 17:17 |
| opendevreview | Merged openstack/openstack-ansible-os_swift master: Migrate cronjobs to systemd-timers https://review.opendev.org/c/openstack/openstack-ansible-os_swift/+/946991 | 17:20 |
| f0o | 0makes sense | 17:30 |
| opendevreview | Merged openstack/openstack-ansible master: Update roles for 31.0.0.0b1 release https://review.opendev.org/c/openstack/openstack-ansible/+/947014 | 20:38 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!