16:00:01 <rm_work> #startmeeting Octavia
16:00:02 <openstack> Meeting started Wed Oct 23 16:00:01 2019 UTC and is due to finish in 60 minutes.  The chair is rm_work. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:03 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:05 <openstack> The meeting name has been set to 'octavia'
16:00:08 <johnsom> o/
16:00:08 <rm_work> o/
16:00:12 <colin-> hi
16:01:08 <haleyb> hi
16:02:35 <rm_work> #topic Announcements
16:02:50 <rm_work> Queens is transitioning to Extended Maintenance!
16:02:54 <rm_work> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-October/010161.html
16:03:13 <rm_work> Any other announcements today?
16:03:54 <cgoncalves> hi
16:03:58 <ataraday_> hi
16:04:06 <johnsom> I have proposed a 1.11.0 python-octaviaclient release: https://review.opendev.org/690386
16:04:09 <johnsom> #link https://review.opendev.org/690386
16:04:41 <johnsom> There is no real functional changes, it is just to get PDF docs going and fix a help string bug that was breaking python-openstackclient docs.
16:04:51 <johnsom> So no need for people to rush and update.
16:05:11 <rm_work> cool
16:05:19 <johnsom> That is the only announcement I have
16:05:21 <rm_work> k
16:05:28 <rm_work> anyone else?
16:06:06 <rm_work> #topic Brief progress reports / bugs needing review
16:06:31 <rm_work> I started to refresh our Priority etherpad
16:06:42 <rm_work> #link https://etherpad.openstack.org/p/octavia-priority-reviews
16:07:21 <johnsom> I have been focused on a series of issues related to certificate handling. One of those patches has merged, the other I posted yesterday:
16:07:29 <johnsom> #link https://review.opendev.org/690444
16:08:27 <johnsom> I also fixed an issue with the gates Friday related to version 2.4 of networkx (taskflow uses this). I have also submitted a patch to Taskflow to prepare it for the 2.x series of networkx.
16:08:38 <johnsom> #link https://review.opendev.org/689611
16:08:40 <ataraday_> Working on taskflow changes queue... Checking errors that job octavia-v2-dsvm-scenario-amphora-v2 got on them - if there are relevent etc..
16:09:10 <ataraday_> Looking forward reviews for #link https://review.opendev.org/#/c/689128/
16:09:32 <johnsom> I have a few more TLS related patches to work on, then hopefully moving back to fixing the failover flow.
16:10:36 <rm_work> ataraday_: how close are those patches to us being able to merge the whole set?
16:10:47 <johnsom> ataraday_ Done, makes sense
16:10:57 <rm_work> i was kinda poking at them the other day
16:11:16 <rm_work> would love to get moving on those, and stuff tends to move faster once we get the base work merged
16:11:48 <ataraday_> all changes ready for review :)
16:13:30 <ataraday_> I rebased jobboard controller on the top of all refactor, so when extra change will be merged - like retry and tempest and job will be green - we can merge
16:13:43 <rm_work> cool
16:14:01 <rm_work> might try to get through those soon
16:14:15 <ataraday_> Retry change #link https://review.opendev.org/#/c/662791/
16:14:46 <rm_work> I just have one bugfix i'd like to see merged
16:14:48 <rm_work> #link https://review.opendev.org/#/c/688548/
16:15:04 <rm_work> and its dep in octavia-lib
16:16:18 <rm_work> alright, moving on...
16:16:20 <rm_work> #topic Open Discussion
16:16:55 <rm_work> how is this meeting time for folks? any chance it could be a couple hours later? or is this pushing it?
16:17:05 <johnsom> FYI, we have an agenda etherpad for the Shanghai PTG:
16:17:07 <johnsom> #link https://etherpad.openstack.org/p/octavia-shanghai-U-ptg
16:17:42 <colin-> i am flexible
16:17:54 <johnsom> I am fine with a later time
16:18:20 <ataraday_> for me this time is fine, but I can do later as well.
16:18:54 <cgoncalves> question would be what time and possibly different date. doodle?
16:19:37 <rm_work> well, yes, but mostly would like... 2 hours later for instance be workable for folks?
16:20:49 <cgoncalves> fine for me during non-DST (starting next week)
16:22:01 <rm_work> k, i can make a doodle but not sure what other times i'd like to put on there
16:22:13 <rm_work> might just be this time and that time :D
16:25:07 <cgoncalves> I'd say add as many time slots as possible
16:25:08 <rm_work> alright, anything else?
16:25:20 <johnsom> FYI, I have -2'd this feature patch to stable/rocky:
16:25:23 <johnsom> #link https://review.opendev.org/#/c/690498
16:25:52 <cgoncalves> feature patch and straight to rocky? hmmmm
16:26:12 <johnsom> Right, it also references a blueprint that doesn't exist
16:26:14 <rm_work> probably running a rocky cloud
16:26:30 <rm_work> wouldn't mind some of that if it were proposed to master :)
16:27:02 <johnsom> Well, I wonder if they know how Octavia works. Monitoring amphora directly is....  questionable value.
16:27:11 <johnsom> But, yes, I commented that it could be proposed to master
16:27:20 <rm_work> right
16:27:22 <johnsom> Feel free to add additional comments, etc.
16:27:25 <rm_work> but sometimes they get lost
16:27:34 <rm_work> and it'd be nice info to have stamped on them :)
16:27:46 <rm_work> anywho, yes, can't be on rocky
16:28:07 <maciejjozefczyk> johnsom, rm_work hey! can you please take a look on this one? https://review.opendev.org/#/c/672264/ For now all the tests have ROUND_ROBIN algorithm hardcoded. That makes OVN tests not possible to successfully run (all the tests will be failing). That would be great to start running OVN gate first. Can we do a refactor of this issue later?
16:29:43 <maciejjozefczyk> I know thats not mona lisa, but that could unblock us, folks  :)
16:31:14 <cgoncalves> maciejjozefczyk, octavia-tempest-plugin is branchless, hence there's no time-constraint (i.e. Usurri feature freeze)
16:31:46 <johnsom> I think the other cores should comment. I still stand behind my comment that adding a config option for which algorithm is tested is counter productive for the test suite in general.
16:32:21 <cgoncalves> plus, your patch also adds a new option. this means that sooner or later we would need to deprecate which comes at a cost
16:32:42 <rm_work> technically not ALL options get deprecated :D
16:33:14 <johnsom> Yeah, deprecation on a branchless repo is... painful
16:33:25 <cgoncalves> plus^2, agreeing with johnsom that we should test as many algorithms as possible. we don't want to run reconfigure and run tempest multiple times to test different algos
16:34:06 <maciejjozefczyk> cgoncalves, johnsom so that would need a refactor of whole octavia-tempest-plugin
16:34:10 <rm_work> yeah, might just want to have tests for each and skip them based on what's available?
16:34:22 <johnsom> rm_work That is what I proposed
16:34:32 <rm_work> so we can do a provider list
16:34:44 <rm_work> and detect that way which ones we can run
16:34:45 <johnsom> rm_work We already have a config for "provider"
16:34:49 <rm_work> ah
16:35:56 <johnsom> #link https://github.com/openstack/octavia-tempest-plugin/blob/master/octavia_tempest_plugin/config.py#L96
16:36:56 <maciejjozefczyk> But its not only about skipping tests that are not accurate for given provider driver, for example this part: https://review.opendev.org/#/c/672264/9/octavia_tempest_plugin/tests/test_base.py is about checking if members are balanced
16:36:59 <johnsom> What I proposed is for each test that is using RR, we create another test that uses the "SOURCE_IP_PORT", then do a skip check for each based on the configured provider.
16:37:23 <johnsom> Since OVN doesn't support RR and amphora doesn't support "SOURCE_IP_PORT"
16:37:35 <rm_work> yep, seems sane to me
16:38:17 <rm_work> though that still makes it impossible to test multiple in one run
16:38:44 <johnsom> Yes, a "check member balanced" will need to be created for the "SOURCE_IP_PORT" algorithm. Or the test to at least be appropriate to test that algorithm.
16:38:49 <rm_work> we COULD do something smart and inspect providers and flavors and see if we should try testing with multiple flavors or something, or allow that to be configured
16:39:00 <rm_work> but at least just adding both tests would be a start
16:39:25 <maciejjozefczyk> So what about api tests? Like this? https://review.opendev.org/#/c/672264/9/octavia_tempest_plugin/tests/api/v2/test_member.py
16:39:31 <johnsom> rm_work Why would it? It would run all the tests that apply to that provider. The config option proposed is the only one that would limit it to only testing one per run.
16:40:04 <rm_work> no, you linked to the config option already merged that sets what provider to use
16:40:18 <rm_work> which is a single provider
16:41:05 <johnsom> rm_work Yes, test runs are validate against one "provider" at a time. The proposed patch would make test runs only valid for one provider and one LB algorithm.
16:41:26 <rm_work> yeah, but what you proposed still just runs one algorithm test
16:41:32 <rm_work> it does pick it correctly
16:41:43 <johnsom> rm_work Run per-provider seems valid given the other infrastructure requirements for them.
16:41:48 <rm_work> but if we want to run BOTH, we'd need to detect flavors, etc
16:42:03 <johnsom> rm_work I think we have confused you.
16:42:32 <rm_work> [09:33:25] cgoncalves:	plus^2, agreeing with johnsom that we should test as many algorithms as possible. we don't want to run reconfigure and run tempest multiple times to test different algos
16:42:38 <johnsom> rm_work We should leave the provider config. That is valid. I'm just saying the algorithm tests should run all algorithms that are valid for a given provider.
16:42:41 <rm_work> [09:38:17] rm_work:	though that still makes it impossible to test multiple in one run
16:43:10 <rm_work> [09:36:59] johnsom:	What I proposed is for each test that is using RR, we create another test that uses the "SOURCE_IP_PORT", then do a skip check for each based on the configured provider.
16:43:12 <rm_work> ^^ that's two
16:43:15 <rm_work> one for each
16:43:37 <rm_work> and requires two runs and a reconfigure inbetween
16:43:47 <rm_work> it's technically possible to test both in one run
16:43:55 <rm_work> if there exist flavors to allow it
16:44:27 <johnsom> rm_work only because OVN doesn't support anything other than "SOURCE_IP_PORT". For amphora, one run should test multiple algorithms that it supports. Unlike the proposed patch.
16:44:47 <johnsom> rm_work No, this has nothing to do with flavors.
16:44:49 <maciejjozefczyk> johnsom, I don't understand. How?
16:44:57 <maciejjozefczyk> johnsom, Looks like everything is hardcoded, no?
16:45:03 <rm_work> yeah i'm talking less about the proposed patch, and more about what we WANT to see
16:45:21 <rm_work> johnsom: flavors are the only way to actually run with multiple providers at the same time right now
16:45:26 <rm_work> so yes, it has to do with flavors?
16:45:27 <johnsom> maciejjozefczyk That is what I am proposing instead of your patch.
16:46:08 <johnsom> rm_work We don't want to run multiple providers in one test run. That is not what I am proposing and I think it is a bad idea.
16:46:26 <cgoncalves> rm_work, disregard test of multiple providers in same run
16:46:36 <johnsom> rm_work We want one test run per provider, each running all of the algorithm tests it can.
16:46:59 <rm_work> k, then that requires us to duplicate our test a bunch of times
16:47:03 <rm_work> because right now it just tests RR
16:47:19 <rm_work> not just add SOURCE_IP
16:47:26 <cgoncalves> we want to have the ability to test multiple algorithms for the same provider in one run. e.g. test RR and LEAST_CONNECTION for a given provider
16:47:32 <johnsom> rm_work Right, today we only have tests for one algorithm. I am saying we should fix that.
16:47:40 <cgoncalves> +1
16:47:47 <johnsom> cgoncalves +1
16:47:52 <maciejjozefczyk> +1 cgoncalves
16:47:58 <cgoncalves> otherwise it also comes at a cost of exploding our CI job matrix
16:47:59 <rm_work> maciejjozefczyk: +1
16:48:18 <cgoncalves> seems we're all in agreement, in a round-robin fashion :D
16:48:21 <maciejjozefczyk> johnsom, rm_work so how we could do it? inspect provider algorithm and then generate classes dynamically based on current code? that would have only changed unsed algorithm?
16:48:42 <rm_work> nah, just write the test a bunch
16:48:51 <rm_work> as long as it's fairly decomposed, it won't be a ton of code reuse
16:49:05 <rm_work> and then do skiptests depending on the config value for provider
16:49:51 <rm_work> we COULD try to be fancy and use one test and detect the provider and switch out the algorithm+test_func based on a dictionary of options
16:49:52 <rm_work> but
16:50:03 <rm_work> ¯\_(ツ)_/¯
16:50:04 <johnsom> Well, that is an implementation detail. There would be a run for each algorithm and a validation for each algorithm. So, maybe separate tests is easier/better.
16:50:28 <haleyb> maciejjozefczyk: can you just create sub-classes with a single line selecting the algorithm, then @skip some?  i think that's what ^^^ is ?
16:50:39 <rm_work> yes
16:50:44 <maciejjozefczyk> hum, so we have ~17 testfiles here https://review.opendev.org/#/c/672264/; for every class we're just inherit with given subclass? It will produce a lot of new classes
16:51:09 <rm_work> just another method
16:51:18 <rm_work> not another full class, I think
16:51:28 <rm_work> i mean, I guess you could do it that way too O_o
16:51:29 * johnsom wonders if supporting the tempest suite for third party drivers is a good idea....
16:52:29 <rm_work> well, for a ton of tests (everything but traffic, pretty much) the only change is literally which default algorithm to use
16:52:50 <rm_work> so you *could* at a very high level (base test class?) just detect provider and select a "default algorithm"
16:53:16 <rm_work> and then for the tests where it *matters*, create copies and skip the ones that don't run
16:53:28 <johnsom> You would still want per-algorithm tests though
16:53:42 <rm_work> not for a ton of them
16:53:55 <johnsom> right, we said the same thing at the same time-ish
16:54:02 <rm_work> most tests don't actually CARE about the algorithm, they just provide it so it will be valid create syntax
16:54:20 <maciejjozefczyk> rm_work, like an API tests
16:54:26 <rm_work> yes, like all API tests
16:54:29 <maciejjozefczyk> maybe not all,
16:54:35 <cgoncalves> yeah but now we'd be actually testing the algorithm
16:54:37 <johnsom> It's really too bad there isn't support for RR in OVN
16:54:51 <maciejjozefczyk> johnsom, yeah...
16:54:52 <rm_work> well, we already have exactly one test for algorithm testing
16:54:56 <rm_work> the traffic-ops test
16:55:12 <rm_work> that's the one that'd actually benefit from having copies for each alg
16:55:19 <cgoncalves> right
16:55:25 <rm_work> but the majority of tests just need to fill that field with SOMETHING valid
16:55:38 <rm_work> so they can do their real testing which is "does a member create work or ERROR"
16:55:42 <johnsom> Agreed, something valid
16:56:02 <cgoncalves> algorithm validation with traffic is important also for certification purposes
16:56:05 <rm_work> so yes, for 99% of tests, selecting a default algorithm in a base class based on the provider would work
16:56:21 <johnsom> yep
16:56:31 <rm_work> then for the traffic tests, you'd want to actually put in the work to make them validate each algorithm
16:56:39 <rm_work> and skip as appropriate
16:56:46 <johnsom> Well, maybe a bit more than 99%, but yes.
16:56:54 <maciejjozefczyk> rm_work +1
16:57:06 <maciejjozefczyk> rm_work I own you a beer in Shanghai
16:57:08 <johnsom> rm_work Yep. That is what I proposed in a comment on that patch.
16:57:23 <rm_work> lol yeah i guess i'll take it since johnsom isn't there :D
16:57:29 <maciejjozefczyk> johnsom, you too :)
16:57:32 <johnsom> lol
16:57:41 <cgoncalves> he can take the week after Shanghai :P
16:57:42 <maciejjozefczyk> ahh. :P
16:57:53 <maciejjozefczyk> ah, right, Barcelona
16:57:55 <maciejjozefczyk> ;)
16:58:38 <rm_work> k well, we went through the meeting time pretty well there
16:58:41 <rm_work> any parting comments?
16:58:50 <rm_work> we've got one more meeting before the summit
16:59:07 <rm_work> then will probably cancel the meeting during the summit, per usual
16:59:24 <johnsom> +1
16:59:44 <rm_work> alright
16:59:48 <rm_work> thanks for coming folks!
16:59:54 <rm_work> #endmeeting