| opendevreview | Merged openstack/kolla-ansible master: Add removal of --key parameter to CLI rewrite note https://review.opendev.org/c/openstack/kolla-ansible/+/933864 | 06:50 |
|---|---|---|
| opendevreview | Michal Nasiadka proposed openstack/kolla-ansible stable/2024.2: Add removal of --key parameter to CLI rewrite note https://review.opendev.org/c/openstack/kolla-ansible/+/934100 | 06:53 |
| opendevreview | Merged openstack/kolla-ansible stable/2024.2: Add removal of --key parameter to CLI rewrite note https://review.opendev.org/c/openstack/kolla-ansible/+/934100 | 07:15 |
| yosef | mnasiadka: Hi, can you check out my reply on: https://review.opendev.org/c/openstack/kolla-ansible/+/929646 | 08:16 |
| mnasiadka | Will do later today, it's 9am here :) | 08:16 |
| mnasiadka | need my coffee | 08:16 |
| yosef | thank you :) | 08:17 |
| opendevreview | Jan Horstmann proposed openstack/kolla-ansible master: Improve grafana dashboard deployment https://review.opendev.org/c/openstack/kolla-ansible/+/921097 | 08:18 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add TLS support for MariaDB Connections https://review.opendev.org/c/openstack/kolla-ansible/+/933924 | 08:53 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Align Kayobe Ansible version with Kolla Ansible https://review.opendev.org/c/openstack/kayobe/+/926639 | 09:05 |
| opendevreview | yosef proposed openstack/kolla-ansible master: Add election-timer option to ovn-db role https://review.opendev.org/c/openstack/kolla-ansible/+/934106 | 09:08 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Align Kayobe Ansible version with Kolla Ansible https://review.opendev.org/c/openstack/kayobe/+/926639 | 09:11 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Align Kayobe Ansible version with Kolla Ansible https://review.opendev.org/c/openstack/kayobe/+/926639 | 09:20 |
| opendevreview | Roman Krcek proposed openstack/kolla-ansible master: Move to high level client in DockerWorker https://review.opendev.org/c/openstack/kolla-ansible/+/908295 | 10:07 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Align Kayobe Ansible version with Kolla Ansible https://review.opendev.org/c/openstack/kayobe/+/926639 | 10:21 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/2023.2: Prevent accidental downgrades of RabbitMQ https://review.opendev.org/c/openstack/kolla-ansible/+/933875 | 11:02 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/2023.2: Fix: add common options to RabbitMQ version check https://review.opendev.org/c/openstack/kolla-ansible/+/934113 | 11:02 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/2023.1: Prevent accidental downgrades of RabbitMQ https://review.opendev.org/c/openstack/kolla-ansible/+/933876 | 11:07 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible stable/2023.1: Fix: add common options to RabbitMQ version check https://review.opendev.org/c/openstack/kolla-ansible/+/934114 | 11:07 |
| mattcrees[m] | Any cores fancy a second +2 for a Bobcat bugfix? The Caracal one merged, so should be trivial :) https://review.opendev.org/c/openstack/kolla-ansible/+/933757 | 11:18 |
| SvenKieske | done | 11:21 |
| mattcrees[m] | Cheers | 11:25 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Remove upgrade note about Rocky Linux 9 support https://review.opendev.org/c/openstack/kayobe/+/934118 | 11:41 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add TLS support for MariaDB Connections https://review.opendev.org/c/openstack/kolla-ansible/+/933924 | 11:46 |
| kevko | folks, do you think it's ok if we are generating two separate TLS certs for kolla_internal_vip ? | 11:46 |
| kevko | haproxy and proxysql ? | 11:47 |
| kevko | shouldn't they use the same cert as it's same internal VIP ? | 11:47 |
| kevko | like, it's working ... there is no reason why it shouldn't ... as it's generate against same CA ...but just wondering ... shouldn't be the cert same ? | 11:48 |
| frickler | different services should use different certs IMO. you could even tie these to different IPs in larger deployments? | 11:48 |
| kevko | frickler: I am not sure if I can agree with you ...for example kolla is now generating backend.crt and backend.key ...and it's copied into the container for specific service ...keystone , cinder, glance ...etc ...etc ..... but it's same cert ... | 11:50 |
| kevko | frickler: I can agree that different IP = different TLS cert ... i am okay with this | 11:51 |
| kevko | frickler: but let's say i am connecting to internal_vip:5000 for keystone ... cert A .... mysql connect to same internal_vip:3306 ...cert B | 11:51 |
| kevko | same VIP different port | 11:52 |
| kevko | frickler: it's similar as you have horizon on port 443 which is providing dashboard ...but you have nova vnc or spice on 6032 (if I remember correctly the port) ...which is also under the same TLS cert | 11:52 |
| kevko | frickler: *same | 11:54 |
| kevko | (it's 6082 regarding above... ) | 11:56 |
| frickler | maybe more certs would be better indeed. also backend TLS might be yet another question. but anyway, I think the current state as you describe it wouldn't be improved by using a smaller number of certs | 11:57 |
| kevko | frickler: It would improve because the code would be simplified and unified. | 12:00 |
| kevko | TLS validation is verified against fqdn or IP | 12:01 |
| kevko | I'm not saying it doesn't work, but it simply doesn't make sense to make life harder for yourself. | 12:02 |
| opendevreview | Merged openstack/kolla-ansible stable/2023.2: Fix Octavia service upgrade failure from 2023.1 https://review.opendev.org/c/openstack/kolla-ansible/+/933757 | 12:47 |
| opendevreview | Merged openstack/kolla-ansible master: Fix cinder etcd3gw backend_url https://review.opendev.org/c/openstack/kolla-ansible/+/933726 | 12:47 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible stable/2024.2: Fix cinder etcd3gw backend_url https://review.opendev.org/c/openstack/kolla-ansible/+/934122 | 12:56 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible stable/2024.1: Fix cinder etcd3gw backend_url https://review.opendev.org/c/openstack/kolla-ansible/+/934123 | 12:56 |
| opendevreview | Pierre Riteau proposed openstack/kolla-ansible stable/2023.2: Fix cinder etcd3gw backend_url https://review.opendev.org/c/openstack/kolla-ansible/+/934124 | 12:57 |
| zigo | kevko: Hey there! Could you tell me where's the topic agenda stuff for the Kolla meetings? | 13:11 |
| zigo | (unrelated to what we discussed yesterday) | 13:11 |
| opendevreview | Mickael Razzouk proposed openstack/ansible-collection-kolla master: add docker_apt_key_set_env variable to docker role https://review.opendev.org/c/openstack/ansible-collection-kolla/+/934126 | 13:46 |
| mattcrees[m] | zigo: We keep the agenda on the whiteboard: https://etherpad.opendev.org/p/KollaWhiteBoard#L80 | 13:53 |
| zigo | mattcrees[m]: Cheers ! | 13:53 |
| opendevreview | Merged openstack/kayobe master: Align Kayobe Ansible version with Kolla Ansible https://review.opendev.org/c/openstack/kayobe/+/926639 | 14:26 |
| opendevreview | Bertrand Lanson proposed openstack/kolla-ansible master: set haproxy for cyborg module https://review.opendev.org/c/openstack/kolla-ansible/+/890642 | 14:38 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible master: Add retries to `Waiting for rabbitmq to start` https://review.opendev.org/c/openstack/kolla-ansible/+/934135 | 14:51 |
| opendevreview | Bertrand Lanson proposed openstack/kolla-ansible master: set haproxy for cyborg module https://review.opendev.org/c/openstack/kolla-ansible/+/890642 | 15:00 |
| kevko | zigo: https://etherpad.opendev.org/p/KollaWhiteBoard | 15:13 |
| zigo | Yeah, got it. | 15:13 |
| kevko | zigo: sorry, I had a meeting | 15:13 |
| zigo | Sure, no pb. | 15:13 |
| kevko | I'm really desperate at this point , I'm using Neutron with OVN, and OVS starts spinning at 100% CPU, with the log showing entries https://paste.openstack.org/show/bnleYUdDaH5W7Ohk1LiS/ and the time just keeps increasing. The ovs-appctl command hangs, so I can’t use coverage. While ovs-vsctl show works, any attempt to change anything causes it | 15:13 |
| kevko | to freeze. If I delete OVS (including the volume) and restart the server, when I reconfigure OVS (re-create it), everything is initially fine because there are no rules in OVS. At this stage, if I remove the interface from br-ex and call reconfigure on OVN (which sets the metadata), everything is still OK. However, as soon as I put the interface | 15:13 |
| kevko | back into br-ex, it crashes again | 15:13 |
| kevko | Kolla-ansible, kolla, images ubuntu 22.04, ovs 3.3.0 , 24.03.2 | 15:13 |
| kevko | btw zigo ^^ ? | 15:13 |
| kevko | for me it sounds like an ovs bug or something | 15:13 |
| zigo | Maybe. | 15:14 |
| zigo | Are we talking about OVS + OVN Debian packages? | 15:14 |
| zigo | These days, they are maintained in Debian by Ubuntu people. | 15:14 |
| zigo | (the Ubuntu server team) | 15:15 |
| zigo | I have no experience with OVN, so I can't tell... | 15:15 |
| kevko | openvswitch-common 3.3.0-1ubuntu3~cloud0 | 15:15 |
| kevko | ovn-common 24.03.2-0ubuntu0.24.04.1~cloud0 | 15:15 |
| zigo | In Bookworm, we have 3.1.0-2+deb12u1 | 15:16 |
| zigo | and 23.03.1-1~deb12u2 | 15:16 |
| zigo | It's been a *very* long time since we had issues with OVS (though, as I mentioned, we aren't using OVN) | 15:17 |
| shermanm | hey, I had a question about the best way to include arm64 ipxe binaries in the ironic_pxe container. It looks like I'd need to modify the extend_start script to load them into the correct path during bootstrap? | 15:19 |
| kevko | zigo: yes, we also haven't seen issues until now :D | 15:20 |
| SvenKieske | kevko: maybe you can try our osism downstream images? afaik we rebuild ovs and ovn ourselves, just in case you want to rule out some compilation bugs | 15:21 |
| SvenKieske | but don't know how easily you can swap images in that deployment | 15:21 |
| zigo | kevko: Is there a way you can use Debian packages instead of the Ubuntu ones? | 15:22 |
| frickler | kevko: also which kernel version? | 15:22 |
| SvenKieske | yes, if I would need to throw a stone at someone to blame, I would aim for the kernel before ovs :D | 15:23 |
| kevko | frickler: Linux controller0 5.15.0-124-generic #134-Ubuntu SMP Fri Sep 27 20:20:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux | 15:23 |
| SvenKieske | I'd recommend running the ubuntu HWE kernels (not hwe edge, those don't get security updates) | 15:23 |
| kevko | zigo: it's easy ... but I am out of time ..so don't want to try everything at once ..as I am in serious trouble :/ | 15:23 |
| SvenKieske | as a datapoint we have ovn 24.3.3 and ovs 3.3.1 | 15:25 |
| kevko | SvenKieske: do you have images somewhere publicly ? | 15:29 |
| SvenKieske | kevko: yes, e.g. https://quay.io/repository/osism/openvswitch-db-server | 15:29 |
| SvenKieske | you can search in that org etc | 15:30 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible master: Add retries to `Waiting for rabbitmq to start` https://review.opendev.org/c/openstack/kolla-ansible/+/934135 | 15:31 |
| SvenKieske | or you can use our mirror, see our sboms here: https://github.com/osism/sbom/blob/main/8.0.2/openstack.yml (this is the latest official release) | 15:31 |
| kevko | SvenKieske: some specific tags ? | 15:31 |
| SvenKieske | kekvo: that would be the tag, e.g. for ovn: osism.harbor.regio.digital/kolla/release/ovn-controller:24.3.3.20240909 (see above sbom link) | 15:32 |
| kevko | SvenKieske: thank you | 15:32 |
| kevko | I will try it probably | 15:32 |
| SvenKieske | kevko: you're welcome :) hope it works, and if it also triggers a bug, happy to hear about that too and we can try to fix it then, at least it's not distro specific then | 15:33 |
| SvenKieske | afaik we compile sources directly from upstream | 15:33 |
| SvenKieske | kevko: if you want to look into the ovs build process yourself, you can find it here: https://github.com/osism/deb-packaging | 15:35 |
| kevko | SvenKieske: do you use ubuntu also ? | 15:35 |
| kevko | as base ? | 15:35 |
| SvenKieske | still ubuntu, but debian and centos stuff should also work in general | 15:36 |
| SvenKieske | frickler wants to switch to debian, which I kind of agree with :D but many people want/need ubuntu | 15:36 |
| SvenKieske | and some need centos, so that is the latest addition. there are some issues, especially with ARM :D | 15:37 |
| kevko | SvenKieske: well yeah, I like also debian more ... | 15:38 |
| kevko | SvenKieske: Yuck, CentOS | 15:39 |
| kevko | SvenKieske: https://review.opendev.org/c/openstack/kolla-ansible/+/890642 << | 15:57 |
| SvenKieske | kevko: was that related to your issue? or just a bugfix where another +2 is needed? | 16:27 |
| kevko | SvenKieske: just bugfix ..mnasiadka gave +2 before .. | 16:28 |
| SvenKieske | ok. looking into ti | 16:34 |
| SvenKieske | it* | 16:34 |
| SvenKieske | kevko: how much do you care about TLS in cyborg? see reply on the change | 16:44 |
| kevko | SvenKieske: i replied :) | 16:45 |
| kevko | SvenKieske: check my answer | 16:45 |
| yosef | kevko: reading your issue with ovn-ovs, what is your workload? recently upgraded to ovs 3.3.0 without any problem although had a problem with ovn 24.03.2, forced me to bump to 24.03.4 | 16:46 |
| kevko | SvenKieske: i think the commiter did what he proposed as patch and added reno ... fixed availability of cyborg on VIP ... TLS can be added in separate patch | 16:46 |
| kevko | yosef: what issue did you have ? | 16:47 |
| yosef | kevko: https://github.com/ovn-org/ovn/issues/261 | 16:48 |
| SvenKieske | kevko: ack, got your +2 :) but wait for CI for W+1 | 16:50 |
| kevko | SvenKieske: of course | 16:52 |
| kevko | SvenKieske: btw, after we will release D ...there is whole relation chain to merge i think | 16:52 |
| kevko | SvenKieske: https://review.opendev.org/c/openstack/kolla-ansible/+/933924 | 16:52 |
| kevko | SvenKieske: to merge/to review | 16:54 |
| mnasiadka | D final release is done | 17:06 |
| opendevreview | Michal Arbet proposed openstack/kolla-ansible master: Add TLS support for MariaDB Connections https://review.opendev.org/c/openstack/kolla-ansible/+/933924 | 17:06 |
| mnasiadka | Time to start merging things in master - E cycle for Kolla/Kolla-Ansible is now open ;-) | 17:07 |
| kevko | oh, i see https://review.opendev.org/c/openstack/releases/+/934054 | 17:08 |
| kevko | mnasiadka: Awesome, I love this freedom of a fresh master branch where reviews will be quick and the vibe relaxed, after all, it’s just master again! :) :D | 17:09 |
| SvenKieske | kevko: :D well I like that optimism :) | 17:18 |
| kevko | SvenKieske: I'm such a spammer; I'll always remind you of myself, and I'll always try to break the ice to be more smooth :) | 17:20 |
| kevko | :D | 17:20 |
| kevko | SvenKieske: Subtle sarcasm is close to my heart, and so is humor. :) | 17:21 |
| mnasiadka | kevko: can you add a reno to https://review.opendev.org/c/openstack/kolla/+/930152 - so we can merge that one as well? | 17:43 |
| mnasiadka | fix handling configs is gating | 17:44 |
| kevko | mnasiadka: thanks, yeah of course | 17:50 |
| kevko | mnasiadka: I will in few minutes, right after I will have a keyboard under my hands | 17:50 |
| mnasiadka | No problem, will have a look then | 17:51 |
| mnasiadka | Other patches for tomorrow ;-) | 17:51 |
| mnasiadka | kevko: we just need to state that the logging format changed from THIS to THAT, so users are aware (maybe somebody is parsing those logs or something) | 17:52 |
| kevko | Yep | 17:53 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Enhance logging format for better readability https://review.opendev.org/c/openstack/kolla/+/930152 | 18:13 |
| kevko | mnasiadka: ^^ i am not good at reno writing ..so please ..check it :D | 18:13 |
| opendevreview | Bertrand Lanson proposed openstack/kolla-ansible master: set haproxy for cyborg module https://review.opendev.org/c/openstack/kolla-ansible/+/890642 | 19:02 |
| opendevreview | Verification of a change to openstack/kolla master failed: Fix handling configs in base image https://review.opendev.org/c/openstack/kolla/+/915440 | 19:11 |
| opendevreview | Merged openstack/kayobe stable/2024.1: CI: Fix Ubuntu 24.04 version check https://review.opendev.org/c/openstack/kayobe/+/933881 | 19:37 |
| opendevreview | Michal Arbet proposed openstack/kolla master: Fix handling configs in base image https://review.opendev.org/c/openstack/kolla/+/915440 | 21:43 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!