| opendevreview | Merged openstack/kayobe master: Prevent accidental overriding of Ansible extensions https://review.opendev.org/c/openstack/kayobe/+/917762 | 01:04 |
|---|---|---|
| *** fungi is now known as Guest9249 | 01:33 | |
| *** kinrui is now known as fungi | 01:41 | |
| Fyrie | How do I modify the base image to accept root CA? I am trying to build the kolla images via `kayobe overcloud container image build` I think I have to override Jinja2 blocks? | 05:55 |
| Fyrie | need to add company's cert | 06:00 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible master: Update user role assignments https://review.opendev.org/c/openstack/kolla-ansible/+/934414 | 09:26 |
| opendevreview | Matt Crees proposed openstack/kolla-ansible master: Give ironic-inspector system scope `all` https://review.opendev.org/c/openstack/kolla-ansible/+/934415 | 09:26 |
| opendevreview | Pierre Riteau proposed openstack/kayobe master: Add loop control to custom DNF repos https://review.opendev.org/c/openstack/kayobe/+/933761 | 09:58 |
| lanson | Hello guys, back at it again wwith https://review.opendev.org/c/openstack/kolla-ansible/+/934429 which I believe should be good to go, and https://review.opendev.org/c/openstack/kolla-ansible/+/934747 which was split out of the cyborg fix and should also be fairly straight forward. cheers | 09:59 |
| SvenKieske | mnasiadka: replied, there are quite some errors in at least one multinode job, not sure if I get around to dig into them today. | 09:59 |
| SvenKieske | ( regarding https://review.opendev.org/c/openstack/kolla/+/934749 ) | 09:59 |
| mnasiadka | SvenKieske: I won't, I'm off today and tomorrow, and I just posted that so we start thinking about this | 10:17 |
| SvenKieske | sure, no problem :) thanks for doing so! | 10:21 |
| opendevreview | Rafal Lewandowski proposed openstack/kolla-ansible master: Enable ML2/OVN and distributed FIP by default. https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 11:18 |
| opendevreview | Mickael Razzouk proposed openstack/kolla master: Added libvirt dependency to nova-libvirt container https://review.opendev.org/c/openstack/kolla/+/934952 | 11:35 |
| moot | Hello after upgrading my cluster to 2024.2 all mdev management broke, I submited a fix and wanted to ask if adding the mdevctl package here was the right way of doing it | 11:38 |
| opendevreview | Merged openstack/kayobe stable/2024.1: Add internal VIP address to no_proxy https://review.opendev.org/c/openstack/kayobe/+/934729 | 11:46 |
| opendevreview | Merged openstack/kayobe stable/2023.2: Add internal VIP address to no_proxy https://review.opendev.org/c/openstack/kayobe/+/934730 | 12:03 |
| opendevreview | Merged openstack/kayobe stable/2023.1: Add internal VIP address to no_proxy https://review.opendev.org/c/openstack/kayobe/+/934731 | 12:03 |
| PrzemekK | Maybe You can teach about "From zero to code: A hands-on guide to making your first OpenStack code contribution" from kolla-ansible side ? It would be interested how we can propose changes to master code and do local adjustments for example to images/ code https://lists.openstack.org/archives/list/openstack-discuss@lists.openstack.org/thread/EYGHAMW765EFCXD7JLVYVBCHINPZP3ZS/ | 12:30 |
| opendevreview | Mickael Razzouk proposed openstack/kolla master: Added libvirt dependency to nova-libvirt container https://review.opendev.org/c/openstack/kolla/+/934952 | 12:51 |
| opendevreview | Rafal Lewandowski proposed openstack/kolla-ansible master: Enable ML2/OVN and distributed FIP by default. https://review.opendev.org/c/openstack/kolla-ansible/+/904959 | 13:00 |
| opendevreview | Jakub Darmach proposed openstack/kolla-ansible master: Exclude NFS backend mount from ownership change https://review.opendev.org/c/openstack/kolla-ansible/+/923945 | 13:27 |
| SvenKieske | bbezak: meeting? :) | 14:00 |
| bbezak | certainly ;) | 14:00 |
| bbezak | #startmeeting kolla | 14:00 |
| opendevmeet | Meeting started Wed Nov 13 14:00:34 2024 UTC and is due to finish in 60 minutes. The chair is bbezak. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'kolla' | 14:00 |
| bbezak | #topic rollcall | 14:00 |
| *** bbezak is now known as Guest9306 | 14:01 | |
| SvenKieske | o/ | 14:01 |
| frickler | \o | 14:01 |
| Guest9306 | I can see that core team is around, good :) | 14:02 |
| Guest9306 | #topic agenda | 14:03 |
| Guest9306 | * Roll-call... (full message at <https://matrix.org/oftc/media/v1/media/download/Aaudh7XK0DL4kQY_zkWsYCLPncW_1ApJNdaQnSNYQH0FUwwGMopjXbBNAYF1ZXJN3K2wNqjWndpaEAzyoxh2jx5CeTbe8zCAAG1hdHJpeC5vcmcvbmtpdEZJUURZdVlObk13eExmaVFsU1Jj>) | 14:03 |
| Guest9306 | (i hope rendering is not very bad) | 14:03 |
| frickler | it is | 14:03 |
| SvenKieske | rendering is very bad (just a link) :D | 14:03 |
| frickler | you need to paste each line on it's own | 14:04 |
| frickler | but for now I guess we can live with it | 14:04 |
| Guest9306 | yeah. I'll try to remember that | 14:04 |
| Guest9306 | #topic CI status | 14:04 |
| Guest9306 | whiteboard says green | 14:04 |
| Guest9306 | I didn't see massive issues recently | 14:05 |
| frickler | there were some infra issues yesterday, but should be fine now again | 14:05 |
| Guest9306 | ok | 14:05 |
| Guest9306 | #topic Release tasks | 14:06 |
| Guest9306 | how is kayobe doing | 14:06 |
| Guest9306 | jovial | 14:06 |
| Guest9306 | I'll check with him | 14:06 |
| Guest9306 | #topic Current cycle planning | 14:07 |
| Guest9306 | I guess we're just after PTG and in the beginning of the cycle | 14:08 |
| Guest9306 | rabbitmq is a good topic, but I saw discussion in the change SvenKieske | 14:08 |
| SvenKieske | yes, so on one hand CI is "green" but looking at actual logs it's not that green and I wonder why that many errors are reported as green, but I didn't have time to dive deep into the issues and if they are even related. | 14:09 |
| Guest9306 | any particular example? | 14:10 |
| SvenKieske | and if I'm not missing something it's really weird that CI is green, as afaik upstream removed some of our used queue types, so I wonder how that should work. | 14:10 |
| SvenKieske | #link https://zuul.opendev.org/t/openstack/build/0037175825154a0bb76e0c6c65eab144 | 14:10 |
| SvenKieske | specifically https://zuul.opendev.org/t/openstack/build/0037175825154a0bb76e0c6c65eab144/log/secondary1/logs/kolla/all-ERROR.txt but it's a very very long error log so as I said I didn't deep dive into it | 14:11 |
| SvenKieske | there's errors all over the place, no sql connection, keystone domain not found..various stuff, might be related to rmq, might be something else | 14:12 |
| Guest9306 | might be related to bootstrap. I'd compare other changes flows | 14:12 |
| SvenKieske | not sure why that's reported as "success" though. afaik in k-a that should be a hard error. maybe zuul ci in kolla is different than in kolla-ansible? not sure. | 14:13 |
| frickler | lot's of those errors are during the startup phase, so they are ignored | 14:13 |
| Guest9306 | yes, we're ignoring a lots of errors. recently we added some as well | 14:13 |
| frickler | and keystone domain/user/whatever not found is normal ops | 14:14 |
| Guest9306 | in check-logs.sh | 14:14 |
| Guest9306 | tests/check-logs.sh that is | 14:14 |
| Guest9306 | sorry for autoparsing link | 14:14 |
| SvenKieske | it's at least hard to distinguish multinode jobs from single node jobs in kolla, these seem to be differently labeled from the kolla-ansible jobs somehow | 14:14 |
| SvenKieske | because I don't think we would find any issues in single node jobs. | 14:15 |
| Guest9306 | yes, multinodes are very helpful in that regard | 14:15 |
| SvenKieske | or this: 2024-11-12 14:28:53.801 31 ERROR oslo_service.periodic_task oslo_messaging.exceptions.MessageDeliveryFailure: Unable to connect to AMQP server on 192.0.2.1:5672 after inf tries: Basic.publish: (404) NOT_FOUND - no exchange 'cinder-scheduler_fanout' in vhost '/' | 14:16 |
| SvenKieske | basically last line in: https://zuul.opendev.org/t/openstack/build/19efb602a7734f06b495a0b2346bf345/log/secondary2/logs/kolla/all-ERROR.txt | 14:16 |
| SvenKieske | so that doesn't look like a healthy rmq to me | 14:16 |
| SvenKieske | still reported as success. I'm not sure what's filtering out "harmless" errors in our CI, but I think it's a little bit greedy | 14:17 |
| SvenKieske | and in general I would be very careful about filtering out errors at all, because the required patterns often spin out of control and tend to shadow real issues. | 14:18 |
| Guest9306 | indeed. however some errors in bootstrap will occur anyway | 14:19 |
| SvenKieske | but whatever, someone would have to dedicate some time, maybe I can make some on sunday, to look into it, no promises though | 14:19 |
| Guest9306 | ok. let's continue discussion in that change | 14:19 |
| Guest9306 | it make sense | 14:19 |
| SvenKieske | sure, I understand the very good reasons to hide some errors, we can't fix all of them. we just need to be very careful :) | 14:19 |
| Guest9306 | to make it clearer | 14:19 |
| Guest9306 | #topic Additional agenda (from whiteboard) | 14:20 |
| SvenKieske | I'll update the link to the latter link, that seems like a more real issue, thx | 14:20 |
| Guest9306 | I can see some changes to review from SvenKieske | 14:20 |
| Guest9306 | and backports | 14:21 |
| Guest9306 | that is on my list | 14:21 |
| Guest9306 | I'll take a look into that | 14:21 |
| SvenKieske | nice, yeah most of those are backports I think :) | 14:22 |
| SvenKieske | no reason to go over them individually imho. would just be happy to receive some reviews, thx. | 14:22 |
| Guest9306 | yeap, will take a look | 14:23 |
| Guest9306 | #topic Open discussion | 14:23 |
| Guest9306 | I guess we did some open discussion already :) | 14:24 |
| Guest9306 | I think we won't have meeting next week, as there are at least couple of conferences | 14:25 |
| SvenKieske | okay | 14:25 |
| Guest9306 | I'll be in Prague on OVS/OVN con | 14:25 |
| Guest9306 | and some folks will be on Supercomputing in the US | 14:25 |
| frickler | let's skip it then, I'd think | 14:25 |
| Guest9306 | I'm wondering why today's attendance is so low ;) | 14:25 |
| SvenKieske | can you motivate someone at ovn con to revive the ovn-exporter project, please? ;) | 14:26 |
| Guest9306 | :) | 14:26 |
| Guest9306 | I'll try | 14:26 |
| SvenKieske | upstream is dead and I don't feel particular inclined to maintain my one-off-patch fork, which isn't even finished :D | 14:26 |
| Guest9306 | it is a problem of one-people project. I think ovs/ovn should have and maintain that kind of stuff | 14:27 |
| SvenKieske | btw, your IRC name somehow changed to "Guest9306" just if you're wondering bbezak :D | 14:27 |
| Guest9306 | lovely | 14:27 |
| Guest9306 | again :) | 14:28 |
| SvenKieske | right, would be nice if ovn/ovs project could step up. no idea how many ressources they got though. | 14:28 |
| *** Guest9306 is now known as bbezak | 14:28 | |
| SvenKieske | that looks better, but I guess we are finished? :) | 14:29 |
| bbezak | I don't have anything else | 14:29 |
| bbezak | let's wrap up then, thank you all! | 14:29 |
| bbezak | #endmeeting | 14:29 |
| opendevmeet | Meeting ended Wed Nov 13 14:29:57 2024 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 14:29 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-11-13-14.00.html | 14:29 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-11-13-14.00.txt | 14:29 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/kolla/2024/kolla.2024-11-13-14.00.log.html | 14:29 |
| frickler | o/ | 14:30 |
| SvenKieske | o/ | 14:30 |
| SvenKieske | thx for running the meeting bbezak | 14:31 |
| bbezak | ;) | 14:31 |
| greatgatsby | Hello. I did another test upgrade from Zed to Antelope and we experienced another 5 minute outage to our VMs via the floating IPs. This time we had kolla-ansible logging with timestamps and the outage occurred just as the "Restart openvswitch-db server container" and "Restart openvswitchd container" tasks were happening. Really hoping there's something we can do to prevent this, appreciate any tips or help. Thanks. | 15:24 |
| opendevreview | Antony Messerli proposed openstack/kolla-ansible master: Enables override of Octavia notification topics https://review.opendev.org/c/openstack/kolla-ansible/+/934827 | 15:33 |
| xiaolong | hello | 15:44 |
| xiaolong | I encountered an issue regarding deploying OpenStack using Kolla | 15:44 |
| xiaolong | When deploying OpenStack 2024.1 and 2024.2 using CentOS 9 Steam and Rocky 9.4 | 15:45 |
| xiaolong | Bootstrap servers' prompts error when installing 'dbus python' | 15:47 |
| xiaolong | Is 2024.2 really applicable to the Rocky9.4 system? The system's built-in Python 3.9 cannot even install Ansible core>=2.16 | 15:49 |
| SvenKieske | there was a fix regarding the dbus package but that should been included in 2024.1 and .2 | 16:02 |
| SvenKieske | https://review.opendev.org/c/openstack/ansible-collection-kolla/+/922614 | 16:02 |
| xiaolong | Do you have a document address? | 16:02 |
| xiaolong | Thank You | 16:03 |
| SvenKieske | not so sure about your rocky issue, afaik kolla should take care of the appropriate python version inside the venv | 16:04 |
| SvenKieske | maybe above dbus fix is not enough for all scenarios, but you should be able to adapt it to your scenario, bug reports are welcome, if you can confirm something is missing :) | 16:05 |
| xiaolong | ok | 16:06 |
| xiaolong | Is there an article about deploying OpenStack starting from installing the system? | 16:07 |
| SvenKieske | most assume you have a basic operating system installed somewhere I think | 16:07 |
| SvenKieske | I think our downstream guide is probably one of the best which also mentions the operating system installation and prerequisites, but the disadvantage is, that it assumes you use our open source kolla wrapper: https://osism.tech/docs/guides/deploy-guide/seed | 16:08 |
| SvenKieske | xiaolong: I assume you have read https://docs.openstack.org/kolla-ansible/latest/user/quickstart.html already? | 16:10 |
| xiaolong | yes | 16:11 |
| xiaolong | But there is no detailed explanation of the system version, such as whether it is written as Rocky9 in Quick Start, 9.0, 9.2, or 9.4? | 16:11 |
| xiaolong | rocky | 16:11 |
| SvenKieske | mhm all should work but from what I still know about rocky, afaik only their latest release is supported, no? so if 9.4 is released 9.3 is out of support? at least that was the case with centos some years ago. | 16:12 |
| SvenKieske | yes, only 9.4 is supported: https://wiki.rockylinux.org/rocky/version/#__tabbed_1_2 | 16:12 |
| SvenKieske | 9.3 is EOL since May 2024 | 16:13 |
| xiaolong | yes | 16:13 |
| xiaolong | OSISM may be useful in data center deployment, but I want to deploy OpenStack on a laptop, and OSISM requires RAM>=16 | 16:13 |
| SvenKieske | ah okay | 16:13 |
| SvenKieske | we have a special deployment method for laptops actually, but let me check the ram requirements | 16:13 |
| xiaolong | really? | 16:14 |
| SvenKieske | yeah, that even needs 32 GB of ram, due to ceph, unfortunately :( | 16:14 |
| xiaolong | devstack? | 16:14 |
| xiaolong | ...... | 16:14 |
| SvenKieske | so it's a complete cloud, including ceph: https://osism.tech/docs/guides/other-guides/cloud-in-a-box/ | 16:14 |
| SvenKieske | you can make it work with 8GB if you don't start any payloads I think :D | 16:15 |
| xiaolong | Yes, I think it must work very well in a production environment | 16:15 |
| SvenKieske | there was a blog where someone made it run on his laptop | 16:15 |
| SvenKieske | ah yeah, but he had also 64 GB of ram: https://scs.community/2023/08/04/ciab-on-a-laptop/ but it's quite educational xiaolong. the next best thing would be devstack I think | 16:17 |
| xiaolong | hahaha | 16:19 |
| xiaolong | yes | 16:19 |
| xiaolong | Have you ever deployed OpenStack on VMware using the Quick Start feature? | 16:23 |
| xiaolong | hello | 17:02 |
| xiaolong | Have you ever deployed OpenStack on VMware using the Quick Start feature? | 17:02 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!