| opendevreview | Gregory Thiemonge proposed openstack/octavia master: WIP Don't wait for the completion of jobs in the consumers https://review.opendev.org/c/openstack/octavia/+/960378 | 13:44 |
|---|---|---|
| skraynev | @gthiemonge hi. do you know : why this endpoint do not set PENDING status for balancer? | 14:40 |
| skraynev | https://github.com/openstack/octavia/blob/master/octavia/api/v2/controllers/amphora.py#L180-L198 | 14:40 |
| gthiemonge | skraynev: hey I knew it, but i don't remember. I know that it's expected, and we have a similar behavior in the cert rotation flow of the housekeeping service | 14:47 |
| gthiemonge | johnsom knows | 14:49 |
| skraynev | interesting. thanks. I hope, that johnsom will shed of light on this behaviour | 14:50 |
| gthiemonge | https://bugs.launchpad.net/octavia/+bug/2111857 | 14:54 |
| gthiemonge | hmm | 14:54 |
| johnsom | @skraynev It's because this API does not impact the load balancer, it only updates the amphora agent config and for the most part does it transparently by mutating the config. | 15:00 |
| gthiemonge | johnsom++ | 15:01 |
| johnsom | It is also a single API vs the multiple endpoints that affect change to the load balancer config | 15:01 |
| johnsom | #startmeeting Octavia | 16:00 |
| opendevmeet | Meeting started Wed Sep 10 16:00:36 2025 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| opendevmeet | The meeting name has been set to 'octavia' | 16:00 |
| gthiemonge | o/ | 16:00 |
| johnsom | Hi everyone, happy RC1 week! | 16:01 |
| johnsom | #topic Announcements | 16:01 |
| johnsom | It's RC1 week. lol | 16:02 |
| gthiemonge | :) | 16:02 |
| johnsom | I think there are still bug fixes we should get in, but we can do an RC2 as well. | 16:03 |
| johnsom | Thoughts? | 16:03 |
| gthiemonge | can you mark those reviews in the etherpad? I'll prioritize them | 16:04 |
| johnsom | Yes, I can work on that today. | 16:04 |
| johnsom | I am thinking the GRAP bug for example, the controller IP list update | 16:05 |
| johnsom | Just to name a few off the top of my head | 16:05 |
| gthiemonge | unless there's a regression, i'm not found of RC2 | 16:05 |
| gthiemonge | ack | 16:05 |
| johnsom | Yeah, it's a pain | 16:05 |
| johnsom | We should keep the list as small as possible | 16:06 |
| gthiemonge | yeah | 16:06 |
| gthiemonge | simple bugfixes are backportable too | 16:07 |
| johnsom | Yeah, everything is a backport after RC1 lol | 16:07 |
| gthiemonge | right | 16:08 |
| johnsom | #topic Brief progress reports / bugs needing review | 16:08 |
| whershberger | Hey friends, wondering if I could get another review on https://review.opendev.org/c/openstack/octavia/+/958808 since the failure isn't related to any changes proposed against 2025.1/stable. I'll rebase https://review.opendev.org/c/openstack/octavia/+/956930 as soon as that merges and would appreciate a review of the backports after that (we would really like to get these fixes delivered in Ubuntu) | 16:08 |
| johnsom | I have been working on reviews and trying to finish the controller IP list update patch | 16:08 |
| johnsom | whershberger Ok, thanks for letting us know that is a priority. I will look after the meeting | 16:10 |
| gthiemonge | whershberger: I +2ed the patch that makes grenade non-voting, note that you don't need to rebase your patch (because the gerrit branch is always merged into the target branch in the CI) | 16:10 |
| whershberger | Oh cool, that's good to hear. Thanks johnsom, really appreciate it | 16:11 |
| gthiemonge | i've been working on improving the graceful termination of the worker when jobboard is enabled, not that it doesn't work well, but I think we can handle it in a better way | 16:11 |
| johnsom | Yes, this underscore vs dot issue with setuptools is impacting a lot of things. There is an open setuptools bug for it, but I have not checked the status yet | 16:12 |
| whershberger | I think it made lwn some time ago :) | 16:12 |
| johnsom | Oh really? lol | 16:13 |
| johnsom | Well, the fun of working in python.... | 16:13 |
| johnsom | Any other progress reports this week? | 16:13 |
| johnsom | #topic Open Discussion | 16:14 |
| johnsom | How about any other topics this week? | 16:14 |
| gthiemonge | nothing from me | 16:15 |
| johnsom | I have been tracking the mariadb discussion the TC has been having about the database collation changes. There is an OpenStack discuss thread about it. I need to go do more research on the changes on the mariadb side to fully get my head around it. | 16:16 |
| johnsom | In Octavia, we do not specify a collation, we rely on the collation used at DB creation time by the deployment tools. | 16:17 |
| johnsom | So we may/may not be too impacted or their might need to be some kind of upgrade collation migration. To be determined.... | 16:17 |
| johnsom | Ok, that is all I have this week. Thanks for joining! | 16:18 |
| gthiemonge | interesting topic | 16:18 |
| gthiemonge | thanks johnsom | 16:18 |
| johnsom | #endmeeting | 16:18 |
| opendevmeet | Meeting ended Wed Sep 10 16:18:44 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:18 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/octavia/2025/octavia.2025-09-10-16.00.html | 16:18 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/octavia/2025/octavia.2025-09-10-16.00.txt | 16:18 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/octavia/2025/octavia.2025-09-10-16.00.log.html | 16:18 |
| skraynev | @johnsom regarding API for update amphora config. I found an interesting bug, when run locally tempest tests. Test creates LB, later it makes several amphora config updates and exactly after it runs delete LB. in one of X runs it could goes to race condition, when Lb be removed, but flow of update amphora config stuck in jobboard. There is here second part of issue: task tries to connect to deleted VM during 30 m | 16:37 |
| skraynev | inutes, but jobboard re-schedule task to another worker after timeout of 30 seconds. | 16:37 |
| skraynev | setting LB to PENDING_UPDATE on update amphora config will prevent initial error. however the second part with infinite task in jobboard could be still reproduced. | 16:39 |
| johnsom | Yeah, there could be a bug in that flow there. Feel free to open a launchpad for it | 16:39 |
| johnsom | Yeah, I don't think locking the LB is the right solution. Probably just need better error logic in the amphora flow | 16:40 |
| skraynev | @johnsom ok, got it, I will create a bug. What about the case with stuck task in jobboard. I thought, that octavia will update TTL of task , but when it goes in AmphoraConfigUpdate looks like TTL update skipped. I did not debug why it happens yet. may be you met similar case? | 16:48 |
| gthiemonge | skraynev: ttl should be updated until the task is complete, or the worker that runs it is stopped | 16:49 |
| gthiemonge | skraynev: is it the redis backend? | 16:50 |
| skraynev | yeah. I expected the same, but observed another situation. maybe it happens, because LB was deleted, as I mentioned above. or may be because task retry connect to already deleted amphora. | 16:51 |
| skraynev | yes, it's the redis | 16:51 |
| gthiemonge | which release of octavia/openstack do you use? | 16:52 |
| johnsom | I suspect it is the retry to connect running with a long retry timeout. | 16:52 |
| skraynev | @gthiemonge 2023.1 it's not fresh actually, but I did not find significant changes in base_taskflow or in taskflow releasenotes. | 16:57 |
| skraynev | @johnsom timing is the following: 30 minutes - totally try to connect to VM (120 retries with 10 timeout and 5 sleep duration), 30 seconds - expiration time in taskflow. so exactly after 30 seconds task are marked as without parent key and assigned to another worker | 16:59 |
| gthiemonge | the last major fix regarding the ttl or any related issues was included in zed | 17:00 |
| skraynev | @johnsom https://bugs.launchpad.net/octavia/+bug/2122533 , if some necessary details are missed, please let me know. | 17:08 |
| johnsom | Thank you! | 17:08 |
| skraynev | @gthiemonge yeah. I will try to make some debug around it and come back with more information. | 17:09 |
| skraynev | thank you for the help | 17:09 |
| opendevreview | Merged openstack/octavia stable/2025.1: zuul: octavia-grenade voting false https://review.opendev.org/c/openstack/octavia/+/958808 | 18:33 |
| tt | Hey, trying to setup redis with the jobboard and it doesn't appear to like my redis cluster. A redis cluster should be supported? I'm getting a MOVED 655 error pointing to the master host. Anyone have any tips? | 18:57 |
| johnsom | Are you using Redis Sentinel? | 18:57 |
| tt | I am not, but maybe I need to? | 18:58 |
| johnsom | I think the Redis driver in Taskflow only supports standalone or Sentinel based clustering | 19:00 |
| tt | Ah! Thanks for the infor!! | 19:04 |
| johnsom | It ultimately uses this library: https://pypi.org/project/redis/ | 19:05 |
| johnsom | We test with standalone, but others are using sentinel | 19:06 |
| johnsom | I'm not sure about other clustering mode/models. | 19:06 |
| tt | Gotcha, we're currently testing with ot/redis-cluster | 19:07 |
| tt | they have a sentinel version, going to give that a try! | 19:07 |
| opendevreview | Michael Johnson proposed openstack/octavia master: Fix Amphora controller IP list update https://review.opendev.org/c/openstack/octavia/+/958933 | 20:36 |
| opendevreview | Merged openstack/octavia master: Use native String type https://review.opendev.org/c/openstack/octavia/+/957097 | 21:22 |
| opendevreview | Merged openstack/octavia master: Fixed missing port_id in additional_vips API https://review.opendev.org/c/openstack/octavia/+/946866 | 21:26 |
| opendevreview | Merged openstack/octavia master: Move log offload files to /var/log/octavia https://review.opendev.org/c/openstack/octavia/+/955959 | 21:36 |
| opendevreview | Merged openstack/octavia master: Use overridden defaults cors options when generating a config file https://review.opendev.org/c/openstack/octavia/+/935410 | 21:36 |
| opendevreview | Merged openstack/octavia master: Fix L7Rules with FILE_TYPE and EQUAL_TO https://review.opendev.org/c/openstack/octavia/+/957853 | 21:39 |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!