| noonedeadpunk | hey folks | 12:08 |
|---|---|---|
| noonedeadpunk | so, I think I will be slowly starting with our bug figthing activity | 12:08 |
| noonedeadpunk | just in case - the etherpad is here: | 12:08 |
| noonedeadpunk | #link https://etherpad.opendev.org/p/osa-nov-2025-bug-triage | 12:08 |
| noonedeadpunk | so if you're taking part - please fill yourself in the participants list | 12:09 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible stable/2025.1: Bump SHAs for 2025.1 https://review.opendev.org/c/openstack/openstack-ansible/+/966496 | 12:22 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Ensure shibboleth utils are installed https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/967018 | 12:33 |
| noonedeadpunk | there are already 2 quite good bugs actually | 12:42 |
| noonedeadpunk | question - do we actually want to do anything wrt https://bugs.launchpad.net/openstack-ansible/+bug/2048659 ? | 12:55 |
| noonedeadpunk | As the bug kinda sounds gross, but in fact I feel lazy about doing smth there | 12:55 |
| noonedeadpunk | as then what are we kinda going to even if we find it, but there are no other repos available | 12:58 |
| noonedeadpunk | I wonder how can I reproduce this one: https://bugs.launchpad.net/openstack-ansible/+bug/2122778 | 13:22 |
| noonedeadpunk | as I totally saw it during upgrades internally as well | 13:22 |
| noonedeadpunk | aha, ok, just drop facts... | 13:25 |
| noonedeadpunk | and this is related to https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/group_vars/all/nova.yml#L27-L29 | 13:27 |
| noonedeadpunk | I actually wonder if this is correct at all... what are wetrying to achieve here? find out if there are arm computes at all? | 13:35 |
| noonedeadpunk | jrosser: maybe you have some idea here? as I can recall we've discussed it quite a bit in past ^ | 13:36 |
| jrosser | in the past arm vm only did serial consoles | 13:37 |
| jrosser | that might not be the case any more | 13:37 |
| noonedeadpunk | but like... | 13:37 |
| noonedeadpunk | how it makes sense in context of haproxy endpoints? https://opendev.org/openstack/openstack-ansible/blame/commit/2d66e80f111758c63838dd690ce6efdfe8467bfa/inventory/group_vars/nova_all/haproxy_service.yml#L73 | 13:38 |
| noonedeadpunk | as I assume that nova_console_proxy_types gonna be unique per compute host? or at least intended to be? | 13:38 |
| noonedeadpunk | because it should not be resolved as intended when it's defining haproxy service | 13:40 |
| noonedeadpunk | like we need to get from all hostvars for the nova_compute group[ | 13:44 |
| noonedeadpunk | should not these be 2 separate tasks? https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/master/tasks/nova_install.yml#L78-L93 | 13:48 |
| noonedeadpunk | as we might need to run both? | 13:49 |
| noonedeadpunk | and again nova_console_type is kinda irrelevant... | 13:49 |
| noonedeadpunk | or incorrect in context of setup | 13:50 |
| noonedeadpunk | /o\ | 13:50 |
| * noonedeadpunk thinking where to start unpacking this | 13:51 | |
| jrosser | well | 13:52 |
| jrosser | the console proxies run in the control plane | 13:52 |
| jrosser | and currently we have logic to decide which of those proxies get run (spice/novnc/serial) | 13:53 |
| jrosser | but tbh this is a very awkward set of logic to deal with | 13:53 |
| noonedeadpunk | so if your all controllers are x86_64 - what will be this one? nova_console_type: "{{ (ansible_facts['architecture'] == 'aarch64') | ternary('serialconsole', 'novnc') }}" | 13:53 |
| jrosser | as it has been the case that "if arm hosts you need serial" | 13:53 |
| noonedeadpunk | or what if one controller is arm64 | 13:53 |
| jrosser | "if ironic consoles you need serial" | 13:53 |
| jrosser | and so on | 13:54 |
| noonedeadpunk | but now we take fact for current host | 13:54 |
| noonedeadpunk | and assume we verify all computes | 13:54 |
| noonedeadpunk | and take decisions on that | 13:54 |
| jrosser | so, it might just end up being simpler to install all the proxies, i'm not sure that there is a good answer (other than some kind of optimisation) to do that selectively | 13:54 |
| jrosser | the logic in the haproxy role was previously wrong and horrible | 13:54 |
| jrosser | tbh i think the best answer here is to simplify | 13:55 |
| noonedeadpunk | it's still wrong on all levels... I'd assume you just having override for nova_console_proxy_types | 13:55 |
| jrosser | yes we do, in user_variables so it's global for * | 13:56 |
| noonedeadpunk | right. as if you don't current code will likely screw things | 13:56 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Include both novnc and spice tasks if needed https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/967036 | 14:01 |
| mgariepy | hmm. any issue with https://github.com/openstack/openstack-ansible-os_nova/blob/stable/2024.1/tasks/nova_post_install.yml#L126-L133 ? | 14:10 |
| mgariepy | i'm deploying new compute and it fails because the files don't exist. | 14:10 |
| noonedeadpunk | um, nothing I'm aware of? | 14:10 |
| noonedeadpunk | try -e venv_rebuild=true | 14:10 |
| mgariepy | it's copied from venv to /etc/nova/ ? | 14:11 |
| noonedeadpunk | I think that /etc/nova is a symlink to venv | 14:17 |
| mgariepy | yep indeed. | 14:17 |
| mgariepy | not sure what happen to the first time it did install. | 14:18 |
| mgariepy | but whatever. | 14:18 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Improve nova_console_proxy_types logic https://review.opendev.org/c/openstack/openstack-ansible/+/967047 | 14:53 |
| noonedeadpunk | I hope I am thinking in the correct direction here^ | 14:53 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Improve nova_console_proxy_types logic https://review.opendev.org/c/openstack/openstack-ansible/+/967047 | 14:56 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Improve nova_console_proxy_types logic https://review.opendev.org/c/openstack/openstack-ansible/+/967047 | 14:57 |
| jrosser | i do wonder if more modern arm/libvirt supports graphical console | 15:09 |
| jrosser | i'm not sure though | 15:10 |
| jrosser | it should be obvious though with a vga device in the vm? | 15:10 |
| noonedeadpunk | um. I have nowhere to check that | 15:12 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_nova master: Ensure nova_console_proxy_types respects all computes https://review.opendev.org/c/openstack/openstack-ansible-os_nova/+/967052 | 15:12 |
| noonedeadpunk | but I'd say that these patches, ideally, should be getting us to behavior which I think was intended | 15:14 |
| noonedeadpunk | by extracting arch vars from all computes and making `nova_console_proxy_types` and deployment-wide variable | 15:15 |
| noonedeadpunk | and also closing bug :) | 15:16 |
| andrewbonney | noonedeadpunk: I can't test exhaustively, but I've tried that on one of our deployments and it appears to behave correctly | 15:28 |
| noonedeadpunk | nice, thanks | 15:30 |
| jrosser | heres the arm pci devices i have currently https://paste.opendev.org/show/bj9NRPY3Z6YJ0dj0RPGs/ | 15:37 |
| jrosser | so i have serial console on that, which is working | 15:40 |
| noonedeadpunk | yeah, I was more about default logic, which should not work, unless you override it to be correct | 15:45 |
| noonedeadpunk | anyway. if we wanna change default console type for arm - we can do that as a follow-up | 15:46 |
| damiandabrowski | o/ | 15:55 |
| *** jizaymes_ is now known as jizaymes | 16:04 | |
| noonedeadpunk | so... | 16:06 |
| noonedeadpunk | what are we going to do with this one https://bugs.launchpad.net/openstack-ansible/+bug/2112559 | 16:06 |
| noonedeadpunk | I've just confirmed it's still valid | 16:07 |
| noonedeadpunk | we can kinda drop https://opendev.org/openstack/openstack-ansible-os_nova/src/branch/master/tasks/nova_post_install.yml#L86-L94 | 16:07 |
| noonedeadpunk | but I'm afraid we don't have any good way to say that clean up file content if it already exists | 16:08 |
| noonedeadpunk | The simplest thing - to always have an empty policy.yaml regardless | 16:08 |
| noonedeadpunk | Next what we can do - add notify to this task | 16:08 |
| noonedeadpunk | And we have that for pretty much each and every role: https://codesearch.openstack.org/?q=Remove%20legacy%20policy&i=nope&literal=nope&files=&excludeFiles=&repos= | 16:09 |
| noonedeadpunk | Hard way could be try to fix the oslo_policy.... | 16:10 |
| damiandabrowski | tbh, I'm fine with both(empty policy.yaml file and adding notify) | 16:15 |
| noonedeadpunk | I will probably switch now to looking through really old bug reports to mainly close them | 16:21 |
| damiandabrowski | noonedeadpunk: I wonder what to do with https://bugs.launchpad.net/openstack-ansible/+bug/2087759 | 16:24 |
| damiandabrowski | so if I understand correctly, OSA will never show up in "deployment projects" section in https://docs.openstack.org/2025.2/projects.html for the development release | 16:25 |
| damiandabrowski | it's really surprising that tripleo is there :D | 16:25 |
| damiandabrowski | but I wonder if we can do something to always be there, even for releases that are in development | 16:25 |
| damiandabrowski | i think it's okay to not show in "deployment guides" (https://docs.openstack.org/2025.2/deploy/) until we do OSA release for the particular openstack version | 16:26 |
| noonedeadpunk | yes, tripoleo is weird | 16:26 |
| noonedeadpunk | I'm not really sure if we can do anything wrt | 16:26 |
| damiandabrowski | but it would be nice to always be listed in deployment projects section | 16:26 |
| noonedeadpunk | whenever we release a new version, I am going and patching openstack-manuals | 16:26 |
| noonedeadpunk | to ensure osa is shown there | 16:26 |
| noonedeadpunk | And kolla does pretty much the same today | 16:27 |
| noonedeadpunk | as we are not part of coordinated release, and kinda no point to show up for the verison, which we don't have yer | 16:27 |
| noonedeadpunk | and actually we show up for development branch: https://docs.openstack.org/2026.1/projects.html | 16:28 |
| noonedeadpunk | just 2025.2 is not development anymore :) | 16:28 |
| damiandabrowski | ahh right... | 16:28 |
| damiandabrowski | should be close that bug report then? | 16:28 |
| noonedeadpunk | So I think we should say we probably can't fix it... | 16:28 |
| damiandabrowski | ack, will do that | 16:30 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-repo_server master: [doc] Add documentation around repo_server role https://review.opendev.org/c/openstack/openstack-ansible-repo_server/+/967059 | 16:46 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ceilometer master: Add coordination support to the role https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/967064 | 17:10 |
| opendevreview | Merged openstack/openstack-ansible stable/2025.1: Bump SHAs for 2025.1 https://review.opendev.org/c/openstack/openstack-ansible/+/966496 | 17:12 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ceilometer master: Add coordination support to the role https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/967064 | 17:13 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_ceilometer master: Add coordination support to the role https://review.opendev.org/c/openstack/openstack-ansible-os_ceilometer/+/967064 | 17:14 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_aodh master: Add coordination support to the role https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/967067 | 17:20 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_aodh master: Add coordination support to the role https://review.opendev.org/c/openstack/openstack-ansible-os_aodh/+/967067 | 17:21 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Test coordination for Telemetry stack https://review.opendev.org/c/openstack/openstack-ansible/+/967070 | 17:24 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: [doc] Fix link to supported distributions https://review.opendev.org/c/openstack/openstack-ansible/+/967076 | 17:39 |
| damiandabrowski | noonedeadpunk: any ideas what to do here? https://bugs.launchpad.net/openstack-ansible/+bug/2074196 | 17:58 |
| damiandabrowski | https://review.opendev.org/c/openstack/keystone/+/930589 is cherry-picked to 2024.1 and 2025.1 | 17:58 |
| damiandabrowski | so if you were right about that, the problem is fixed. But hard to fully confirm it | 17:58 |
| noonedeadpunk | I am not 100% sure, but I think it was related | 17:59 |
| noonedeadpunk | But also there are probably multiple reports regarding the same issue | 17:59 |
| noonedeadpunk | also backprot to 2024.2 was not merged | 18:00 |
| noonedeadpunk | I think I'd suggest to mark as fix released | 18:00 |
| noonedeadpunk | as no idea what else we can do there anyway | 18:01 |
| noonedeadpunk | and iirc there were actually migrations in 2024.1 which were related to the issue | 18:01 |
| damiandabrowski | yeah....I was also leaning towards marking it as fixed. Will do that | 18:01 |
| noonedeadpunk | ++ | 18:01 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Replace set_fact with vars defenition https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/967088 | 18:06 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible-os_keystone master: Use dynamic includes for conditional tasks https://review.opendev.org/c/openstack/openstack-ansible-os_keystone/+/967089 | 18:13 |
| noonedeadpunk | jrosser: need your Ironic knowledge to evaluate https://bugs.launchpad.net/openstack-ansible/+bug/1896355 | 18:21 |
| noonedeadpunk | does it make sense to have ironic-conductor to run separately from API? | 18:21 |
| jrosser | centos7 / train -> invalid :) | 18:22 |
| jrosser | but yeah | 18:22 |
| noonedeadpunk | I'm just thinking it still might make sense to fix it? | 18:23 |
| noonedeadpunk | like looking at current env.d and there's no good way to do that today: https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/env.d/ironic.yml | 18:23 |
| jrosser | atm it’s kind of simplistic | 18:24 |
| jrosser | but there is a concept in ironic of conductor groups that I don’t think we support at all | 18:25 |
| jrosser | so there is one thing there is we want to be able to split conductor/api apart, and for scale out that might totally be worth it | 18:25 |
| jrosser | and then a second is how to have conductor groups and what that would mean | 18:26 |
| noonedeadpunk | that is potentially doable with overrides... but it's gonna be quite confusing | 18:27 |
| jrosser | so first step is certainly to improve the existing env.d | 18:27 |
| noonedeadpunk | yeah | 18:27 |
| noonedeadpunk | ok, thanks, so it;s still valid then | 18:28 |
| jrosser | yeah, particularly if you don’t use swift for images | 18:28 |
| jrosser | ie the conductor has to locally host and serve them | 18:28 |
| noonedeadpunk | ok, cool, I wrote that down to etherpad | 18:33 |
| noonedeadpunk | kinda wonder wtf is this one https://bugs.launchpad.net/openstack-ansible/+bug/1833066 | 18:57 |
| jrosser | this could be just something unrelated to the lock dir | 19:03 |
| jrosser | looks very odd tbh | 19:03 |
| noonedeadpunk | any guesses wtf is that? https://opendev.org/openstack/openstack-ansible/src/branch/master/inventory/env.d/glance.yml#L29-L30 | 19:53 |
| noonedeadpunk | or well, I know it's size of cow or lvm volume for glance | 19:53 |
| noonedeadpunk | but like... wtf it's doing there.... | 19:54 |
| opendevreview | Dmitriy Rabotyagov proposed openstack/openstack-ansible master: Remove hardcoded container_fs_size for glance https://review.opendev.org/c/openstack/openstack-ansible/+/967107 | 20:01 |
| noonedeadpunk | so we went down from smth around 154 bugs to 129 | 20:24 |
| noonedeadpunk | which is not great, but given it was only me and Damian - it's not that bad either :) | 20:24 |
| noonedeadpunk | And we totally need to handle found issue with policy removal. As this sounds like a nasty thing to have... | 20:25 |
| -opendevstatus- NOTICE: The OpenDev team will be restarting Gerrit at approximately 2130 UTC in order to pick up the latest 3.10 bugfix release. | 20:30 | |
| noonedeadpunk | will check out for today | 20:44 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!