| opendevreview | Takashi Kajinami proposed openstack/octavia-dashboard master: Use systemctl command to manage service https://review.opendev.org/c/openstack/octavia-dashboard/+/955639 | 02:55 |
|---|---|---|
| opendevreview | Takashi Kajinami proposed openstack/octavia-dashboard master: Use yaml format policy file https://review.opendev.org/c/openstack/octavia-dashboard/+/955651 | 04:46 |
| johnsom | #startmeeting Octavia | 16:00 |
| opendevmeet | Meeting started Wed Jul 23 16:00:03 2025 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'octavia' | 16:00 |
| gthiemon1e | o/ | 16:00 |
| johnsom | Hello everyone! | 16:00 |
| johnsom | #topic Announcements | 16:00 |
| johnsom | I don't have any announcements this week. Any others? | 16:01 |
| gthiemon1e | nothing here | 16:01 |
| johnsom | #topic Brief progress reports / bugs needing review | 16:01 |
| johnsom | I have been focused on the latest gate breakage. This is another issue related to log offloading and rsyslog in Ubuntu | 16:02 |
| johnsom | They are making changes to apparmor for rsyslog and it is breaking us in various ways. | 16:03 |
| johnsom | I fixed the issues in the amphora, but now the nodepool instance running devstack is seeing issues with permission denied when rsyslog attempts to create some log files. It's impacting the haproxy package in addition to our logs from what I see. | 16:03 |
| johnsom | It's been hard to track down as locally I have not yet reproduced it. Anyway, that has been a good chunk of what I am working on this week. | 16:04 |
| johnsom | Aside from downstream work | 16:04 |
| gthiemon1e | I proposed a patch that fixes random failures in the unit test | 16:05 |
| gthiemon1e | https://review.opendev.org/c/openstack/octavia/+/955265 | 16:05 |
| fungi | note that there are no more nodepool test nodes in opendev, all test nodes are booted by zuul from images built in zuul jobs, as of the past week | 16:06 |
| gthiemon1e | I also opened a launchpad for the client, when a user wants to see their quotas, using the project name doesn't work, they have to pass the ID of the project | 16:06 |
| gthiemon1e | (someone reported it here in the channel) | 16:06 |
| johnsom | Ah, interesting. I missed that memo. I wonder if that is related given the timing of the issue | 16:06 |
| fungi | (this has been a gradual migration off nodepool, and the openstack tenant was left for last but has been underway for about a month-ish) | 16:06 |
| gthiemon1e | https://bugs.launchpad.net/octavia/+bug/2117394 | 16:06 |
| fungi | zuul image build jobs still rely on the same diskimage-builder elements nodepool-builder did, so the images should be essentially the same | 16:07 |
| johnsom | Ok, Yeah, this started on the 17th, but like I said earlier, I see in the changelog that the Ubuntu packages are changing the apparmor rules | 16:08 |
| fungi | right, changes to ubuntu's packages are a far more likely suspect for something like this | 16:08 |
| johnsom | I will continue to track this down. I don't want to just do something in our plugin to fix it and not understand the "Why" part | 16:09 |
| fungi | mainly just pointing out that "nodepool instance" is a bit of a misnomer at this point | 16:09 |
| fungi | there is no nodepool, only zuul | 16:09 |
| johnsom | Cool, thank you! | 16:09 |
| johnsom | lol | 16:09 |
| fungi | ;) | 16:09 |
| gthiemon1e | can we disable apparmor? | 16:10 |
| johnsom | We could.... It might be a bit tricky to do on the "test node" instance. Though I'm not 100% sure the current issue is apparmor as I pushed a test that should have moved rsyslog to complain only and it still failed. | 16:11 |
| johnsom | In general I don't like just disabling it as I would expect most deployments are going to have it on and run the defaults. | 16:11 |
| johnsom | More investigating to do.... | 16:13 |
| johnsom | My plan is to build a fresh VM from the ground up again today and look at a few things | 16:14 |
| fungi | keep in mind that if you disable apparmor in testing, then you may be deviating from how users are actually deploying your software so just hiding the problems they'll trip over | 16:14 |
| johnsom | +1 | 16:14 |
| fungi | probably useful for figuring out the problem, not so much a great long-term strategy | 16:15 |
| johnsom | This is one of those "butterfly flapped it's wings in the rain forest and now our tests break" situations | 16:16 |
| johnsom | Any other progress updates today? | 16:17 |
| gthiemon1e | no | 16:17 |
| johnsom | #topic Open Discussion | 16:17 |
| johnsom | Any other topics this week? | 16:17 |
| gthiemon1e | nothing on my side | 16:17 |
| johnsom | Thanks Greg for finding that octavia-status has an oslo config issue. I will poke at that as well. | 16:18 |
| gthiemon1e | oh yeah octavia-status is broken, I forgot it | 16:18 |
| johnsom | Ok, thanks for another week of finding and smashing bugs! Have a good one. | 16:18 |
| gthiemon1e | o/ | 16:18 |
| johnsom | #endmeeting | 16:19 |
| opendevmeet | Meeting ended Wed Jul 23 16:19:02 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:19 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/octavia/2025/octavia.2025-07-23-16.00.html | 16:19 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/octavia/2025/octavia.2025-07-23-16.00.txt | 16:19 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/octavia/2025/octavia.2025-07-23-16.00.log.html | 16:19 |
| fungi | happy to talk about any follow-up questions or ideas that may have come out of the contributor and maintainer surveys and metrics analysis a few weeks ago, but if everyone's still mulling it over that's fine too. might be a good ptg topic | 16:19 |
| johnsom | Agreed, it could be a good PTG topic. There is more regular attendance there as well. | 16:19 |
| fungi | we do have a forum session in for the summit as well, for a more openstack-wide discussion | 16:20 |
| fungi | though i know not everyone can make it to paris(-saclay) | 16:20 |
| johnsom | Yeah, I will not be in attendance | 16:21 |
| fungi | i can't even guarantee i'll be there, viability of air travel and especially travel in and out of the usa is... up in the air | 16:22 |
| johnsom | Sigh | 16:22 |
| gthiemon1e | I might be there | 16:22 |
| johnsom | For me it's budget and other life issues blocking my travel | 16:22 |
| QG | Hello, i have a quick question, did you already saw this error in Octavia Worker : | 16:49 |
| QG | Error: [('asn1 encoding routines', 'asn1_check_tlen', 'wrong tag'), ('asn1 encoding routines', 'asn1_item_embed_d2i', 'nested asn1 error')] | 16:49 |
| QG | https://github.com/openstack/octavia/blob/zed-eom/octavia/certificates/common/pkcs12.py#L31 | 16:49 |
| QG | We are still in Zed and this is still using Openssl | 16:49 |
| johnsom | Hmmm, no, this does not ring a bell for me | 16:50 |
| johnsom | asn1 is the meta data format in the pkcs12 bundle. It's as if there is something off about the pkcs12 bundle meta data. | 16:51 |
| QG | okay, thanks anyway :-) | 16:51 |
| johnsom | On that host can't you read the meta data with "openssl pkcs12" ? | 16:52 |
| johnsom | For example "openssl pkcs12 -in <pkcs12 filename> -noout", does that pass? | 16:53 |
| QG | i have not output, so no meta ? | 16:57 |
| QG | openssl is just asking about "Enter Import Password" | 16:57 |
| QG | but if i transform it in pem : openssl pkcs12 -in <pkcs12 filename> -out certificate.pem -clcerts -nodes | 16:58 |
| johnsom | Yeah, or maybe the -info flag | 16:58 |
| QG | it also ask me the same and then i have the content in certificate.pem | 16:59 |
| QG | ah yeah it does work with the -info flag | 17:00 |
| johnsom | I am a bit rust on my pkcs12 memory, it's been a while since I had to go deep on it. | 17:00 |
| johnsom | Hmm, so if openssl doen't throw a error via command line on that, the file is probably fine. | 17:01 |
| johnsom | Next I would check the version of the pyOpenSSL (we had to use that back then as cryptography didn't have pkcs12 support yet, we switched in later versions): https://github.com/pyca/pyopenssl/blob/main/CHANGELOG.rst | 17:04 |
| johnsom | To see if there was a known bug | 17:04 |
| johnsom | FYI: https://github.com/m-click/requests_pkcs12/issues/4 | 17:08 |
| johnsom | They had a similar error and found a problem with an intermediate cert in the file | 17:09 |
| QG | We are using pyOpenSSl in version 22.0.0 | 17:10 |
| johnsom | Yeah, that was what was in the upper constraints for Zed | 17:11 |
| johnsom | I didn't see anything obvious in the change log | 17:11 |
| QG | me neither | 17:12 |
| QG | Thanks anyway johnsom for the time and the help :-) | 17:33 |
| johnsom | Sure, no problem | 17:34 |
| opendevreview | Michael Johnson proposed openstack/octavia master: Remove duplicate policy file format check https://review.opendev.org/c/openstack/octavia/+/955729 | 20:21 |
| johnsom | ^^ fix for the octavia-status issue. It should be backported too | 20:23 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!