16:00:59 <gthiemonge> #startmeeting Octavia
16:00:59 <opendevmeet> Meeting started Wed Feb 16 16:00:59 2022 UTC and is due to finish in 60 minutes.  The chair is gthiemonge. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:00 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:00 <opendevmeet> The meeting name has been set to 'octavia'
16:01:06 <gthiemonge> Hi everyone
16:01:13 <johnsom> o/
16:01:28 <tweining___> hi
16:01:38 <rm_work> o/
16:03:41 <gthiemonge> #topic Announcements
16:03:47 <gthiemonge> * PTL nominations are closed
16:04:03 <gthiemonge> and it seems that I'm the only candidate for this role!
16:04:06 <tweining___> congrats! :)
16:04:10 <johnsom> Congratulations Greg in being our PTL for the Zed release!
16:04:58 <gthiemonge> Thanks for your support ;-)
16:05:29 <johnsom> Happy to give it!
16:05:30 <johnsom> grin
16:06:14 <gthiemonge> * Yoga Release Schedule
16:06:20 <gthiemonge> ** Final release for non-client libraries
16:06:34 <gthiemonge> FYI we will releasea octavia-lib 2.5.0 for Yoga soon
16:06:51 <johnsom> Just waiting on the release team to hit the button
16:07:01 <gthiemonge> (it includes the definitions for PROMETHEUS support)
16:07:14 <gthiemonge> ** Next week is Yoga-3 milestone
16:07:26 <gthiemonge> This is: Feature freeze and Final release for client libraries
16:07:42 <gthiemonge> regarding the features, there's an open review for the PROMETHEUS support (from johnsom):
16:07:47 <gthiemonge> #link https://review.opendev.org/c/openstack/octavia/+/812258
16:07:58 <gthiemonge> and there are still open reviews for python-octaviaclient:
16:08:12 <gthiemonge> #link https://review.opendev.org/q/project:openstack/python-octaviaclient+is:open+NOT+label:Code-Review%253C%253D-1
16:08:23 <johnsom> Yes, I need to switch the HTTP threading to be compatible with 3.6, but otherwise ready for review
16:08:40 <gthiemonge> We need to focus on those reviews
16:08:55 <gthiemonge> I will take a first look at your patch johnsom!
16:09:02 <rm_work> Ah, I have a thing to bring up, but call on me last 😀
16:09:40 <johnsom> Thank you. I would really be great to get the Prometheus support into yoga.
16:10:06 <johnsom> It is fully functional and has test coverage at this point.
16:10:55 <gthiemonge> ** Zed PTG
16:11:02 <gthiemonge> (Yeah, Z is Zed)
16:11:13 <gthiemonge> I've booked a room for Thurday and Friday (2 x 3h)
16:11:23 <gthiemonge> I will confirm the schedule next week
16:12:30 <gthiemonge> Reminder: you can register for the Virtual PTG at
16:12:37 <gthiemonge> #link https://www.eventbrite.com/e/project-teams-gathering-april-2022-tickets-246804447747
16:13:48 <gthiemonge> any other announcements? rm_work?
16:14:21 <rm_work> Ah yes
16:16:15 <rm_work> So, we're considering proposing a new feature similar to allowed_cidrs on Listeners: allowed_address_groups
16:16:53 <rm_work> This would allow for easily tracking centralized lists that can be shared between LBs and even other services
16:17:31 <rm_work> I ran this by johnsom and I know he raised a valid concern about how we would handle non-neutron-network setups with this
16:17:38 <johnsom> Ah, so we are done with announcements?
16:17:51 <rm_work> ah sorry, I just saw myself pinged and went 😀
16:18:15 <gthiemonge> #topic allowed_address_groups
16:18:28 <rm_work> heh thanks
16:19:21 <rm_work> anyway that is basically it
16:19:21 <opendevreview> Merged openstack/python-octaviaclient master: Add Python3 yoga unit tests  https://review.opendev.org/c/openstack/python-octaviaclient/+/808232
16:19:21 <rm_work> Does anyone have any other feelings about whether this is good?
16:19:33 <gthiemonge> rm_work: so... an address_group is a resource/object that contains a list of cidrs?
16:19:52 <rm_work> essentially yes, but it is added as a native object on a SG
16:20:00 <johnsom> Yeah, our current model/spec for third party provider drivers is that Octavia collects all of the necessary information and passes it down to the driver. In the case of the allowed_cidrs, we pass the list down to the driver so hardware offloaded security groups can be used in the appliances.
16:20:04 <rm_work> so when it is updated, the SG is automatically updated
16:21:03 <rm_work> https://specs.openstack.org/openstack/neutron-specs/specs/victoria/address-groups-support-in-security-group-rule.html
16:21:12 <johnsom> If we start allowing neutron address_groups, how would that work with third party drivers? If we pass the list down to the driver, it could become out of date should someone update the address group in neutron.
16:21:51 <gthiemonge> NotImplemented :D
16:22:14 <rm_work> Basically yeah, if the driver doesn't support that
16:22:34 <johnsom> Do we just pass the neutron reference down to the driver and they are on their own? (breaks our current driver spec)
16:22:50 <rm_work> no, I think if the driver doesn't support that option we don't let it be set?
16:23:11 <johnsom> gthiemonge So you are saying force the drivers to never support that API feature? Block it in the o-api?
16:24:22 <gthiemonge> the driver (not the api) could deny the request, right?
16:24:24 <rm_work> Is it not already the case that if a driver doesn't support something, we already check in the API and refuse certain features
16:24:27 <rm_work> I thought?
16:24:35 <rm_work> which IS the driver denying the API request
16:25:49 <johnsom> rm_work Yes, that is not an issue for this feature. We already have specs/code for if a driver doesn't support something. That is not the question here.
16:26:08 <rm_work> wait, then what is the question?
16:26:18 <johnsom> The question here is how we would allow a driver to implement this feature should they want to.
16:26:38 <rm_work> Oh, then yes, we would pass the address_group ID I think
16:26:49 <rm_work> and let them deal with it
16:27:13 <rm_work> maybe they DO use SGs in the backend?
16:27:14 <johnsom> That breaks our spec/model for drivers.
16:27:17 <rm_work> it's possible
16:27:23 <rm_work> how?
16:27:58 <johnsom> We explicitly called out that we do not require drivers to have tokens and access the cloud for Octavia API features.
16:28:10 <rm_work> we don't REQUIRE them to
16:28:21 <johnsom> For example, with TLS, we collect the cert/key on behalf of the driver and provide it to them.
16:28:53 <rm_work> and if they don't have that access, they're certainly not required to implement the feature 😛
16:29:01 <johnsom> This is why we implemented allowed_cidrs instead of taking neutron SGs.
16:29:05 <rm_work> i think it's not so critical as TLS Term
16:29:26 <rm_work> I definitely didn't think of that as the reason to not take neutron SGs, at the time 😛
16:29:44 <johnsom> We had a long discussion about it.
16:30:16 <rm_work> the reason I remember was that we didn't want the users to be able to manipulate the SG in a way that would allow them to shoot themselves in the foot
16:31:13 <johnsom> Well, that is also another issue, that neutron doesn't support AND with SG
16:31:42 <johnsom> The use case I remember debating was for hardware offload of SGs in third party appliances.
16:32:17 <rm_work> so we allow A way to do this without neutron access, with allowed_cidrs
16:32:25 <rm_work> so it's possible to do without a neutron-requiring feature
16:33:03 <rm_work> why does that block us adding a second way to do it WITH neutron that's slightly more convenient?
16:33:41 <johnsom> I'm just saying we need to discuss how we would handle this feature with third party drivers.
16:34:05 <johnsom> 1. We break our model and spec for third party drivers and pass the driver the neutron address group ID.
16:34:45 <rm_work> (where did we explicitly notate that model/spec BTW?)
16:34:45 <johnsom> 2. We scrape the neutron address group at creation time, pass the list of cidrs down to the driver just like we do for the existing allowed-cidrs feature.
16:34:52 <johnsom> 3. ???? Other options?
16:35:25 <johnsom> 4. Build some kind of notification sink with neutron to hope to capture updates to the address groups and push that down to the drivers as updates?
16:35:35 <rm_work> I'd be for #1 but with the default of NotImplemented unless providers opt in
16:35:37 <rm_work> 4 is madness 😀
16:35:41 <rm_work> and 2 is inconsistent
16:35:43 <johnsom> It's in the driver development guide
16:35:57 <rm_work> (would lead to user frustration and essentially be false advertising)
16:36:00 <johnsom> Not implemented is already there. Greg was joking I think
16:37:00 <johnsom> 5. Don't implement this and stick to allowed_cidrs which provides this capability, maybe not as convenient.
16:37:06 <rm_work> Octavia Provider API endpoint to have octavia re-fetch the address_groups contents and return it for that LB?
16:37:51 <johnsom> Hmm, that is interesting. We could provide that via the driver API.
16:38:46 <johnsom> So we would pass them a scrape, then it would be up to the driver to pool that API as they see fit?
16:38:53 <johnsom> Is that the proposal
16:38:55 <rm_work> yeah
16:38:55 <johnsom> ?
16:39:03 <rm_work> I mean, that's my current thought
16:39:14 <rm_work> I'm not married to it 😀
16:39:19 <rm_work> but it seems workable
16:39:45 <johnsom> This is why we have design discussions. Collect ideas and spec out the best one.
16:40:37 <johnsom> Greg or Tom, any thoughts/comments/ideas?
16:41:21 <gthiemonge> not yet, I need to think about it first
16:41:23 <rm_work> so, adding a parallel to `allowed_cidrs` to Listeners called `allowed_address_groups` that provides similar functionality, but by using the address groups feature in neutron rather than adding a huge list of individual rules to the SGs; providers that aren't neutron-aware would get a snapshot and be able to refresh that via the provider-api
16:41:45 <opendevreview> Tom Weininger proposed openstack/octavia-dashboard master: Display Draining state correctly  https://review.opendev.org/c/openstack/octavia-dashboard/+/826905
16:41:46 <gthiemonge> but I agree that using address groups would be a good feature
16:42:11 <rm_work> I believe the address-groups thing is, besides just more convenient for users with large lists of CIDRs and lots of LBs / other services using them, also more performant/efficient
16:42:23 <johnsom> rm_work Maybe leave allowed_cidrs, pass the addr-group ID(s) to the driver, with the API they can use the ID to poll
16:43:10 <johnsom> That way we aren't forcing them to get tokens, etc. but if they already have some path to neutron they have the IDs
16:43:12 <tweining___> sorry, I know too little about the subject to comment
16:43:33 <rm_work> right now we have users with hundreds of CIDRs in their allowed_cidrs list, with many LBs using the same list, and also VMs and other services too, and any time they have to modify it they have to update every single LB... and then they update their address_group and everything else besides octavia is instantly done 😀
16:43:45 <rm_work> johnsom: yeah that seems doable too
16:44:10 <johnsom> I can live with that.
16:44:41 <johnsom> So next step would be write up a quick spec and propose some patches.
16:44:55 <johnsom> It won't make yoga, but Zed
16:45:02 <rm_work> yeah np
16:45:09 <rm_work> will poke at a spec then
16:45:17 <johnsom> Yoga octavia-lib is basically frozen now.
16:45:20 <rm_work> (when did we start requiring specs? :D)
16:45:29 <rm_work> yeah Zed is totally fine
16:45:31 <johnsom> (since day one)
16:45:43 <johnsom> It doesn't need to be a novel
16:45:48 <rm_work> not aiming at having code out the door for this even within a month prolly
16:45:50 <rm_work> heh
16:46:10 <rm_work> man, one of the things I appreciated about working on Octavia was that we didn't need a whole spec written up for every little thing 😉
16:46:10 <johnsom> #link https://github.com/openstack/octavia/tree/master/specs
16:46:57 <johnsom> The reason I personally lean towards a spec is this impacts provider drivers and a spec would give them an opportunity to comment before it's all coded up.
16:47:37 * johnsom thinks rm_work remembers it that way because other people wrote the specs for him.....
16:48:07 <gthiemonge> lol
16:49:33 <gthiemonge> Ok Folks, thanks for this discussion!
16:49:44 <rm_work> ahaha maybe 😀
16:49:53 <rm_work> yeah I think that's fine, I'll start on a spec
16:49:54 <johnsom> Feels good to have an actual design discussion again....
16:49:55 <gthiemonge> maybe this can be a topic for our PTG in april
16:49:59 <rm_work> thanks for the productive discussion
16:51:09 <gthiemonge> #topic Brief progress reports / bugs needing review
16:51:46 <johnsom> On the Octavia front, I have mostly be focused on updating the metrics patch based on the PTG comments and getting that ready for review.
16:52:01 <gthiemonge> I have a small fix for the new network interface management tool (that is include in wallaby, xena and master)
16:52:32 <gthiemonge> It was not waiting for the ipv6 address to leave the tentative state, so a member may have appeared in ERROR During a few seconds
16:52:40 <gthiemonge> #link https://review.opendev.org/c/openstack/octavia/+/828606
16:53:07 <gthiemonge> ^ it fixes some random failures in our jobs so it would be great to merge it soon
16:53:59 <gthiemonge> And I also wrote a quick hack to reduce the duration of the noop-api tests:
16:54:17 <gthiemonge> #link https://review.opendev.org/c/openstack/octavia-tempest-plugin/+/828963
16:54:43 <gthiemonge> we are frequently hitting the 3h(and a few minutes) timeout in the noop-api tests
16:55:06 <gthiemonge> splitting the MemberAPITest into 2 classes helps us to spread the load on different workers
16:56:03 <gthiemonge> so the patch changes the name of the tests, but it doesn't change the IDs of the tests, I was wondering if you would have some concerns about it
16:57:04 <johnsom> As long as the ID stays the same, it's ok to change the test name.
16:57:32 <johnsom> I still wish I new why some of the noop tests take so long. They should be fast, not a minute or more to run
16:58:43 <tweining___> couldn't we run it with a profiler to see what makes them so slow?
16:59:37 <gthiemonge> I guess the answer would be: SQLAlchemy
17:00:41 <johnsom> Yeah, it tanked in performance 40% in 1.4
17:01:48 <tweining___> https://review.opendev.org/c/openstack/octavia-dashboard/+/826905 could use another reviewer
17:02:38 <gthiemonge> tweining___: feel free to take a look with a profiler if you have experiences with it, I don't how if it is doable beside our unit tests
17:02:51 <johnsom> gthiemonge Are we going to revive the priority review list for the end of yoga?
17:03:12 <gthiemonge> ho yeah, I forgot this point, I will restore the list
17:03:59 <gthiemonge> I don't know if I can do it before the end of the week, but perhaps on Monday we will have it
17:04:48 <gthiemonge> I will ping you on this channel when it's ready
17:05:02 <gthiemonge> Ok Folks, we're late :D
17:05:16 <gthiemonge> #topic Open Discussion
17:05:22 <gthiemonge> anything to add?
17:05:44 <johnsom> Nope, thanks!
17:06:51 <gthiemonge> Ok Thanks!
17:06:55 <gthiemonge> #endmeeting