16:00:06 <johnsom> #startmeeting Octavia
16:00:07 <openstack> Meeting started Wed Jul 22 16:00:06 2020 UTC and is due to finish in 60 minutes.  The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:10 <openstack> The meeting name has been set to 'octavia'
16:00:20 <johnsom> Hi everyone
16:00:24 <rm_work> o/
16:00:27 <aannuusshhkkaa> hello!
16:00:29 <gthiemonge> Hi
16:01:02 <johnsom> #topic Announcements
16:01:17 <johnsom> I don't have much in the announcement category today.
16:01:38 <johnsom> I will remind everyone that next week is the Victoria milestone 2 week.
16:01:47 <johnsom> #link https://releases.openstack.org/victoria/schedule.html#v-2
16:01:58 <cgoncalves> hi
16:02:08 <johnsom> Any other announcements today?
16:02:51 <johnsom> #topic Brief progress reports / bugs needing review
16:03:27 <johnsom> I have been focused on doing reviews and working on the failover v2 patch. It is now fully functional (haven't pushed that yet), but I still have some cleanup and test work to do.
16:05:01 <gthiemonge> I have #link https://review.opendev.org/#/c/735126/ (add a config setting for a default connection_limit in the amphora driver) in review, perhaps we could discuss it later
16:05:03 <johnsom> Any other updates this week?
16:05:25 <cgoncalves> I have posted a patch to fix a bug introduced recently in master that also impacts stable/train. if you run on SINGLE topology and nova anti-affinity enabled, please consider testing https://review.opendev.org/#/q/I757d84e4624f488382ec4ff9af3f489cd9a85ed3
16:05:28 <johnsom> Yeah, we should discuss that. Let's do that next
16:05:45 <johnsom> cgoncalves Thanks!
16:06:15 <gthiemonge> sure
16:06:25 <cgoncalves> we also released octavia-lib 2.1.0 and 2.1.1
16:07:30 <johnsom> I think there has been some work on the metrics side as well
16:08:26 <aannuusshhkkaa> Yeap, we have 3 patches up
16:08:46 <aannuusshhkkaa> 1 - delta changes, 2 - Refactoring driver layer, 3 - Added new metric - response time
16:09:05 <aannuusshhkkaa> We still have to merge the three together and in that order
16:09:05 <johnsom> Nice
16:09:07 <shtepanie> the delta changes are pretty much complete, as well as the driver layer refactor
16:09:28 <johnsom> #topic default connection_limit in the amphora driver
16:09:39 <johnsom> Greg you have the floor
16:09:44 <gthiemonge> thanks
16:10:06 <gthiemonge> for reference, the story is #link https://storyboard.openstack.org/#!/story/2007794
16:10:33 <gthiemonge> basically, default connection_limit in the api is -1, which is translated to 1,000,000 in the amphora driver
16:11:29 <rm_work> yeah we ran into this internally -- had to get people to start setting it manually to a lower value as a workaround
16:11:37 <rm_work> would be good to get a fix in
16:11:40 <gthiemonge> it leads to a huge memory consumption in recent haproxy releases, that might trigger bugs when reloading the configuration (combined with some features, such as stateful table in HA)
16:11:51 <rm_work> it happens very easily with terraform since it does a bunch of stuff back-to-back
16:11:55 <johnsom> I raised some concern about the proposed patch in  that it secretly changes -1 to some number defined by the operator.
16:12:15 <gthiemonge> so the idea is to add a default_connection_limit option in octavia that could be 50000 (admin can set it back to 1M if he needs)
16:12:48 <rm_work> I don't see it being realistic to have it higher honestly
16:12:56 <cgoncalves> first, thank you Greg for finding and reporting this issue and for proposing a patch!
16:13:18 <gthiemonge> in my last patch, -1 sets 50k in the amphora, but the user doesn't know that it is 50k
16:13:33 <johnsom> Well, with one core, agreed. But, with future versions multi-core will do more
16:14:02 <johnsom> Agreed, thank you for working on this.
16:14:21 <gthiemonge> ;-)
16:14:28 <cgoncalves> my concern is about violating the API contract by starting returning a different  connection limit. for example, user sets -1 but API returns 50,000 or whatever is the default
16:15:26 <johnsom> Yeah, it's a tough one. If we don't swap it and show them the number set, people will think whatever the operator set is the limit of the load balancer, which it may not be.
16:16:17 <cgoncalves> I have to be consistent and fair with everyone since I pushed back on a Neutron SG API change not so long ago very similar to this case
16:17:40 <johnsom> Well, that was a bit different IMO as it was changing user input data and this is changing the meaning of -1
16:18:58 <cgoncalves> it would still change the return value if users set to -1
16:19:12 <rm_work> yeah we should change the default (and note that) but not *translate* -1
16:19:14 <johnsom> I just feel that users should be able to see what the actual value is set to.
16:20:22 <johnsom> So I lean towards either taking -1 and returning the user the actual value, both at create and future "show" calls. Or we just eliminate -1 as an option and set some other default value
16:22:09 <gthiemonge> 1. would require to translate the value before entering the amphora driver, right?
16:22:47 <johnsom> Yes, I think so
16:25:59 <johnsom> I think it's a bit better transition if we still accept -1 but translate it. Just in case someone has coded something like terraform to use -1.
16:26:51 <cgoncalves> +1
16:27:49 <rm_work> i ... feel like that's unlikely
16:28:07 <rm_work> but ok
16:28:49 <cgoncalves> I'm just agreeing with continue to accept -1
16:29:24 <johnsom> I'm throwing ideas out. Please if you have input share.
16:31:29 <johnsom> Do we need a vote?
16:31:31 <johnsom> grin
16:31:43 <cgoncalves> if we determine 50,000 is an unattainable connection limit amphora load balancers, that would still be an unlimited connection limit
16:31:54 <rm_work> i believe it is
16:32:01 <rm_work> but /shrug
16:32:07 <cgoncalves> rm_work, I'm disappointing. you should be screaming "API v3!!"
16:32:08 <johnsom> Yeah, I think 50k is probably a decent default value.
16:32:15 <rm_work> lol i mean yeah
16:33:10 <johnsom> I think the only question is how we handle the API part.
16:33:40 <johnsom> The current patch leaves -1 in place in all of the API return values, but secretly changes it in the amphora to 50k
16:34:38 <cgoncalves> we may need to consider a fix needs to be backported to stable branches, so either we come up with *one* solution that is backportable or *two* solutions (master / stable branches)
16:35:03 <cgoncalves> johnsom, it is secretly changing it to 1M today
16:35:55 <johnsom> Yeah, which is effectively "unlimited" lol But, hey with 2.2 you might be able to throw enough cores at it....
16:36:55 <cgoncalves> right, that is why I mentioned that we may need to come up with two solutions: one for master and another for stable branches
16:36:59 <johnsom> 1. Leave the patch as is and secretly set the value to whatever the operator picks.
16:37:18 <johnsom> 2. Change -1 to the value the operator picks.
16:37:32 <johnsom> #vote Which path should we take? 1, 2
16:37:47 <johnsom> #startvote Which path should we take? 1, 2
16:37:48 <openstack> Begin voting on: Which path should we take? Valid vote options are 1, 2.
16:37:49 <openstack> Vote using '#vote OPTION'. Only your last vote counts.
16:38:02 <johnsom> lol, it's been to long since we needed to vote.
16:38:21 <cgoncalves> #vote 1
16:38:23 <johnsom> #vote 2
16:39:01 <gthiemonge> #vote 1
16:39:06 <cgoncalves> option 1 allows us to still take option 2. taking option 2 today is irreversible, I think
16:39:08 <gthiemonge> (i'm lazy)
16:39:49 <johnsom> aannuusshhkkaa, shtepanie, rm_work?
16:40:24 <johnsom> Anyone else that may be in the meeting...
16:40:41 <aannuusshhkkaa> i dont think i have enough knowledge about the consequences to vote..
16:40:51 <ZhuXiaoYu> #vote 1
16:41:33 <rm_work> hmm
16:41:34 <shtepanie> i dont have enough knowledge / context either
16:41:38 <rm_work> sorry doordash arrived
16:41:44 <johnsom> lol
16:42:03 <rm_work> uhh
16:42:27 <rm_work> #vote 2
16:42:32 <rm_work> i think i get it
16:42:33 <cgoncalves> doordash also gets to vote if they use octavia
16:42:46 <johnsom> True
16:43:11 <johnsom> Ok, I think that is enough time and everyone that I know is in the meeting.
16:43:15 <johnsom> #endvote
16:43:16 <openstack> Voted on "Which path should we take?" Results are
16:43:17 <openstack> 1 (3): ZhuXiaoYu, cgoncalves, gthiemonge
16:43:18 <openstack> 2 (2): rm_work, johnsom
16:44:05 <johnsom> So it looks like folgers crystals it is.
16:44:10 <johnsom> Thanks Greg!
16:44:18 <johnsom> #topic Open Discussion
16:44:24 <johnsom> Other topics today?
16:44:28 <gthiemonge> Thanks Folks!
16:45:09 <ZhuXiaoYu> Excuse me. About my Active/Active spec, I want to if there is
16:45:10 <ZhuXiaoYu> anything else we need to provide
16:45:34 <johnsom> Ah, yes. That needs some reviews I suspect.
16:45:57 <johnsom> #link https://review.opendev.org/723864
16:45:57 <cgoncalves> #link https://review.opendev.org/#/c/723864/
16:46:19 <johnsom> Yeah, that one.
16:46:48 <johnsom> I had put this on the meeting agenda a few weeks back. It doesn't look like it got the reviews it needs.
16:46:55 <cgoncalves> I have only read quickly the first version, sorry
16:47:34 <johnsom> #action Please review the Act/Act spec https://review.opendev.org/723864
16:47:55 <ZhuXiaoYu> We are working on a POC right now, it's coming up in August.
16:48:12 <johnsom> It got an update last week, so is due for another review pass
16:48:25 <cgoncalves> #link https://storage.bhs.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_14a/723864/5/check/openstack-tox-docs/14aaa63/docs/contributor/specs/version1.1/alternative-active-active-l3-distributor.html
16:48:26 <johnsom> ZhuXiaoYu Thank you for reminding us
16:48:31 <cgoncalves> ^ HTML rendered page
16:50:32 <johnsom> Any other items today?
16:50:57 <rm_work> oh did we link the delta patch?
16:50:59 <rm_work> it's ready to go
16:51:04 <rm_work> failing on the docs issue
16:51:09 <rm_work> but we'll wrap that today hopefully?
16:51:20 <johnsom> It wasn't linked, but mentioned
16:51:23 <rm_work> (the octavia-lib docs issue, not with the delta patch)
16:51:25 <rm_work> #link https://review.opendev.org/#/c/740815/
16:52:43 <johnsom> Ok then, thanks everyone for the good discussion today.
16:52:53 <johnsom> #endmeeting