*** yamamoto has joined #openstack-lbaas | 00:03 | |
*** yamamoto has quit IRC | 00:09 | |
*** longkb1 has joined #openstack-lbaas | 00:33 | |
*** yamamoto has joined #openstack-lbaas | 01:06 | |
openstackgerrit | Michael Johnson proposed openstack/octavia master: Implement provider drivers - Driver Library https://review.openstack.org/571358 | 01:06 |
---|---|---|
*** kobis has joined #openstack-lbaas | 01:08 | |
*** kobis has quit IRC | 01:09 | |
*** kobis has joined #openstack-lbaas | 01:10 | |
*** harlowja has quit IRC | 01:10 | |
*** yamamoto has quit IRC | 01:11 | |
*** kobis has quit IRC | 01:15 | |
*** JudeC_ has quit IRC | 01:20 | |
*** threestrands_ has joined #openstack-lbaas | 01:33 | |
*** threestrands has quit IRC | 01:36 | |
*** hongbin has joined #openstack-lbaas | 01:48 | |
*** blake has joined #openstack-lbaas | 01:55 | |
*** yamamoto has joined #openstack-lbaas | 02:07 | |
*** yamamoto has quit IRC | 02:12 | |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Correctly guess amp count based on detected topo https://review.openstack.org/572661 | 02:22 |
*** blake has quit IRC | 02:26 | |
*** SumitNaiksatam has joined #openstack-lbaas | 02:29 | |
*** SumitNaiksatam has quit IRC | 02:44 | |
*** SumitNaiksatam has joined #openstack-lbaas | 02:50 | |
*** yamamoto has joined #openstack-lbaas | 03:08 | |
*** kobis has joined #openstack-lbaas | 03:12 | |
*** kobis has quit IRC | 03:12 | |
*** kobis has joined #openstack-lbaas | 03:12 | |
*** yamamoto has quit IRC | 03:14 | |
*** kobis has quit IRC | 03:17 | |
*** bzhao__ has joined #openstack-lbaas | 03:38 | |
*** links has joined #openstack-lbaas | 04:02 | |
*** yamamoto has joined #openstack-lbaas | 04:10 | |
*** hongbin has quit IRC | 04:13 | |
*** yamamoto has quit IRC | 04:16 | |
*** harlowja has joined #openstack-lbaas | 04:17 | |
*** kobis has joined #openstack-lbaas | 04:28 | |
*** kobis has quit IRC | 04:39 | |
*** harlowja has quit IRC | 04:45 | |
*** yamamoto has joined #openstack-lbaas | 04:56 | |
openstackgerrit | Adit Sarfaty proposed openstack/octavia master: Use object instead of object id in the drivers delete callbacks https://review.openstack.org/571974 | 05:01 |
openstackgerrit | Adit Sarfaty proposed openstack/octavia master: Remove a duplicated key in unit test dict https://review.openstack.org/572669 | 05:03 |
*** JudeC_ has joined #openstack-lbaas | 05:24 | |
*** ptoohill1 has quit IRC | 05:28 | |
*** ptoohill1 has joined #openstack-lbaas | 05:28 | |
*** links has quit IRC | 05:33 | |
*** yboaron has joined #openstack-lbaas | 05:43 | |
*** links has joined #openstack-lbaas | 05:49 | |
openstackgerrit | Carlos Goncalves proposed openstack/octavia master: Add grenade support https://review.openstack.org/549654 | 05:51 |
*** kobis has joined #openstack-lbaas | 06:00 | |
*** kbyrne has quit IRC | 06:02 | |
*** JudeC_ has quit IRC | 06:03 | |
*** kbyrne has joined #openstack-lbaas | 06:04 | |
bzhao__ | Hi, guys, do we have a plan to support ca certificate for listener? It is necessary that client authentication case in real LB scenario | 06:09 |
openstackgerrit | Merged openstack/octavia master: Fix amp failover where failover already failed https://review.openstack.org/548989 | 06:29 |
*** links has quit IRC | 06:44 | |
*** pcaruana has joined #openstack-lbaas | 06:44 | |
*** ispp has joined #openstack-lbaas | 06:48 | |
rm_work | bzhao__: i think i would like that too... | 06:49 |
bzhao__ | rm_work: cool. For our current implementation, we support to upload services side certificate , as the existing fields "default_tls_container_ref", "sni_container_refs" in Listener API. We support single direction authentication, but not double direction. Right? | 06:54 |
rm_work | right | 06:57 |
bzhao__ | rm_work: So the new ca certificates specified for client side in listener is necessary for double direction authentication. Ha, I will post a RFE for this in storyborad, but not very clear should we need to expose something about <"double direction authentication enable" and "ca files" fields introduced> or just <"ca files" fields> in Listener API? | 07:03 |
*** links has joined #openstack-lbaas | 07:03 | |
*** rcernin has quit IRC | 07:07 | |
*** links has quit IRC | 07:12 | |
*** openstackgerrit has quit IRC | 07:19 | |
*** nmanos has joined #openstack-lbaas | 07:22 | |
*** ispp has quit IRC | 07:25 | |
cgoncalves | grenade job passed! | 07:27 |
rm_work | make it pass again :P | 07:29 |
*** links has joined #openstack-lbaas | 07:33 | |
cgoncalves | rechecked :) | 07:46 |
rm_work | bzhao__: yeah hmm, i would look to see what other services call it | 07:55 |
cgoncalves | rm_work, uh-oh! http://logs.openstack.org/54/549654/35/check/octavia-grenade/fd5303f/logs/screen-o-hm.txt.gz#_Jun_06_06_57_48_337890 | 07:56 |
rm_work | erm | 07:57 |
rm_work | it couldn't find a listener by that ID lol | 07:57 |
rm_work | wut | 07:57 |
rm_work | i don't ... hold on | 07:57 |
rm_work | ah right | 07:58 |
rm_work | it was deleted | 07:58 |
rm_work | "Amphora e112a159-13b8-4cf7-a053-6cffb7113230 health message reports 1 listeners when 0 expected" | 07:58 |
rm_work | one sec | 07:58 |
rm_work | might be a legit bug | 07:58 |
*** yboaron has quit IRC | 07:59 | |
rm_work | must be a race | 07:59 |
rm_work | but anyway yeah it's a bug, one sec | 08:00 |
*** threestrands_ has quit IRC | 08:03 | |
*** JudeC_ has joined #openstack-lbaas | 08:10 | |
rm_work | sorry this is more complex than i thought lol | 08:18 |
rm_work | obviously a bug | 08:18 |
rm_work | fixing it | 08:18 |
*** lxkong has quit IRC | 08:29 | |
*** ispp has joined #openstack-lbaas | 08:34 | |
*** openstackgerrit has joined #openstack-lbaas | 08:46 | |
rm_work | cgoncalves: this: | 08:46 |
openstackgerrit | Adam Harwell proposed openstack/octavia master: Fix stats update when missing listener in DB https://review.openstack.org/572702 | 08:46 |
rm_work | it's kinda dumb | 08:46 |
rm_work | I KNOW we have this info... and I could be a purist and say "we shouldn't look it up like this", but that's dumb IMO | 08:46 |
rm_work | will see what johnsom thinks in the morning | 08:46 |
cgoncalves | rm_work, that was quick :) | 08:47 |
*** ispp has quit IRC | 08:47 | |
rm_work | eh I thought it'd just be a minute but it took like 45 <_< | 08:47 |
cgoncalves | I'll rebase my grenade on top of that | 08:47 |
rm_work | eugh and i missed a bad thing | 08:48 |
rm_work | https://review.openstack.org/#/c/572661/ | 08:48 |
rm_work | didn't work for active-standby topo T_T | 08:49 |
rm_work | which is annoying because that is what i run, and i should know better | 08:49 |
cgoncalves | go easy on yourself. you've done a magnificent work on the tempest tests! | 08:51 |
rm_work | yeah but it cost me an hour today, which is annoying, lol | 08:57 |
numans | cgoncalves, hey | 09:16 |
numans | I am working on octavia ovn driver. I have a question. | 09:16 |
numans | When i delete a listener, the ovn driver returns the proper status code and the listener gets deleted. | 09:17 |
numans | when i run "openstack loadbalancer listener list", it is empty | 09:17 |
numans | but the entry in the octavia db for the listener is still there. Why isn't it getting deleted from the listener table ? | 09:18 |
sapd | rm_work: How can I get loadbalancer id inside amphora instance ? | 09:21 |
rm_work | numans: is `provisioning_status` == "DELETED"? | 09:24 |
rm_work | sapd: hmm ... i am not sure if we save that in there? | 09:24 |
numans | rm_work, ues it is | 09:24 |
rm_work | the listener ID will be there | 09:24 |
numans | yes it is | 09:24 |
rm_work | numans: then, that sounds normal? though ... i thought we deleted them immediately... so, I'm not sure if we're changing things a bit or if i was mistaken | 09:25 |
rm_work | numans: normally we have our housekeeping process clean up deleted entries after a configured expiry period | 09:25 |
numans | rm_work, i have noticed with other resources as well i.e pool | 09:25 |
rm_work | yeah | 09:26 |
numans | rm_work, http://paste.openstack.org/show/722782/ | 09:26 |
numans | ok. let me just wait for a while and see if it gets deleted. the issue though is if i create a listener again with the same port, i get error | 09:26 |
rm_work | we might be doing things like that now, with the new provider system | 09:26 |
rm_work | erk, right, we probably didn't check on that T_T | 09:27 |
rm_work | you might have a bug | 09:27 |
rm_work | let me take a look at how we are doing this | 09:27 |
rm_work | numans: oh, also I think there is a patch up to change how deletes work | 09:27 |
cgoncalves | numans, hi! | 09:27 |
rm_work | we are still solidifying that code | 09:27 |
numans | cgoncalves, hi. rm_work answered my questions :) | 09:28 |
cgoncalves | numans, ah, ok. reading the backlog | 09:28 |
rm_work | ah but that doesn't actually deal with this | 09:28 |
numans | rm_work, thanks. i will look out the patches. right now i have applied the patch manually https://review.openstack.org/#/c/571358/ | 09:29 |
numans | which now is merged | 09:29 |
numans | rm_work, thanks. | 09:29 |
rm_work | so i'm not even sure how you are able to set the status to deleted yet, since the code for the callbacks isn't complete still? | 09:29 |
rm_work | ahh ok yeah | 09:29 |
rm_work | that is the one | 09:29 |
*** yboaron has joined #openstack-lbaas | 09:30 | |
rm_work | yeah ok | 09:32 |
rm_work | numans: congratulations! you found a major bug | 09:32 |
rm_work | because deletes are also just treated as a "status update", it updates the status now for listeners to DELETED instead of actually doing a delete, which is what we did before | 09:32 |
rm_work | and because it doesn't ACTUALLY delete the listener (or pool or anything), some of our code that expected that (like the listener API that ensured no duplicate ports) is failing | 09:33 |
numans | glad that you found the reason | 09:33 |
rm_work | need to talk to johnsom in the morning and decide how to deal with that | 09:33 |
cgoncalves | rm_work, why is that? (listeners marked as DELETED) | 09:33 |
numans | ok. i was wondering if something is wrong in the driver code | 09:33 |
rm_work | we could change the driver callback lib to do an actual `delete` action if that's the new status, OR we could update the API code to filter on only non-deleted listeners when checking port availability | 09:34 |
numans | i mean the ovn provider driver | 09:34 |
rm_work | yeah, ignore that for now | 09:34 |
rm_work | we'll fix it ASAP | 09:34 |
numans | rm_work, thanks. | 09:34 |
openstackgerrit | Carlos Goncalves proposed openstack/octavia master: Add grenade support https://review.openstack.org/549654 | 09:35 |
*** JudeC_ has quit IRC | 09:35 | |
sapd | rm_work: I would like to use custom collector to collect more metrics in amphora. :D | 09:35 |
rm_work | i am leaning towards the latter fix, in the API and actually leave the objects there as DELETED status (which honestly I feel like we should have done before) but that will require some thought, because it could have other unintended consequences | 09:35 |
rm_work | sapd: you can do that! make an element for it, add it to the element list when you build the image, and it will work :) | 09:35 |
rm_work | you can look at how we add our amphora-agent element | 09:36 |
sapd | rm_work: Yes. I know it. but I would like to know these metrics belong to which loadbalancer. So I need to know loadbalancer id inside amphora | 09:36 |
rm_work | you can know the listener ID easily... so you can query the Octavia API to get the loadbalancer ID from that? | 09:37 |
rm_work | but, it does seem like it would be useful to log that somewhere... :/ | 09:38 |
*** JudeC_ has joined #openstack-lbaas | 09:38 | |
rm_work | eugh, i think we are doing the port validation as a DB constraint... which will make fixing it that way painful | 09:38 |
*** links has quit IRC | 09:39 | |
rm_work | probably we need to change the bulk_update method to do a real delete if the status is DELETED, rather than just updating the status | 09:39 |
rm_work | i guess that's that | 09:40 |
rm_work | i'll poke michael about that in the morning if he hasn't figured it all out already | 09:40 |
*** JudeC_ has quit IRC | 09:43 | |
*** ispp has joined #openstack-lbaas | 09:53 | |
*** ispp has quit IRC | 09:55 | |
*** links has joined #openstack-lbaas | 09:57 | |
*** nmanos has quit IRC | 10:00 | |
*** kobis has quit IRC | 10:01 | |
*** nmanos has joined #openstack-lbaas | 10:01 | |
*** yboaron_ has joined #openstack-lbaas | 10:01 | |
*** kobis has joined #openstack-lbaas | 10:02 | |
openstackgerrit | chenge proposed openstack/octavia master: Amend the spelling error of a word https://review.openstack.org/572718 | 10:04 |
*** yboaron has quit IRC | 10:04 | |
*** links has quit IRC | 10:16 | |
*** annp has quit IRC | 10:21 | |
*** kobis has quit IRC | 10:22 | |
*** links has joined #openstack-lbaas | 10:29 | |
*** lxkong has joined #openstack-lbaas | 10:45 | |
*** kobis has joined #openstack-lbaas | 10:46 | |
sapd | rm_work: So how to get listener ID :D | 10:46 |
*** longkb1 has quit IRC | 10:47 | |
rm_work | sapd: in /var/local/octavia/ the folders that look like UUIDs and have haproxy.cfg inside, are the listener ID names | 10:47 |
rm_work | if you create two listeners, there will be two folders, etc | 10:48 |
sapd | rm_work: how can I push loadbalancer id to amphora instance? | 10:49 |
rm_work | interesting... you could probably change the templates | 10:51 |
rm_work | let me see | 10:51 |
rm_work | so, like this maybe | 10:52 |
rm_work | sorry, taking me a minute longer because tests | 10:57 |
*** yboaron_ has quit IRC | 11:02 | |
rm_work | sapd: ok, check this out | 11:02 |
openstackgerrit | Adam Harwell proposed openstack/octavia master: Add lb_id comment to amp haproxy listener config https://review.openstack.org/572730 | 11:03 |
rm_work | I THINK that should work | 11:03 |
rm_work | didn't test it for real or anything | 11:03 |
rm_work | but it should put the lb id in the listener config file, after the name | 11:03 |
rm_work | i thought it might have just been a template change, but we pass a kind of weird translated version of the LB-dict into the renderer, unfortunately :/ | 11:07 |
rm_work | so have to actually change a little bit of code | 11:07 |
*** kobis has quit IRC | 11:11 | |
cgoncalves | rm_work, your patch fixed the exception seen in the grenade job | 11:25 |
*** yboaron_ has joined #openstack-lbaas | 11:28 | |
*** kobis has joined #openstack-lbaas | 11:30 | |
*** amuller has joined #openstack-lbaas | 11:58 | |
*** kobis has quit IRC | 12:00 | |
*** kobis has joined #openstack-lbaas | 12:10 | |
*** fnaval has joined #openstack-lbaas | 12:14 | |
*** AlexeyAbashkin has joined #openstack-lbaas | 12:29 | |
*** links has quit IRC | 12:33 | |
*** atoth has joined #openstack-lbaas | 12:46 | |
*** hongbin has joined #openstack-lbaas | 12:57 | |
*** kobis has quit IRC | 13:30 | |
*** kobis has joined #openstack-lbaas | 13:30 | |
*** yboaron_ has quit IRC | 13:44 | |
*** Alexey_Abashkin has joined #openstack-lbaas | 13:51 | |
*** amuller_ has joined #openstack-lbaas | 13:53 | |
*** AlexeyAbashkin has quit IRC | 13:53 | |
*** amuller has quit IRC | 13:53 | |
*** Alexey_Abashkin is now known as AlexeyAbashkin | 13:53 | |
*** nmanos has quit IRC | 14:05 | |
*** hvhaugwitz has quit IRC | 14:08 | |
*** hvhaugwitz has joined #openstack-lbaas | 14:14 | |
*** AlexeyAbashkin has quit IRC | 14:28 | |
*** AlexeyAbashkin has joined #openstack-lbaas | 14:36 | |
xgerman_ | ls | 14:48 |
*** kobis has quit IRC | 15:02 | |
dayou_ | . | 15:03 |
*** lxkong has quit IRC | 15:12 | |
*** AlexeyAbashkin has quit IRC | 15:14 | |
*** AlexeyAbashkin has joined #openstack-lbaas | 15:16 | |
*** pcaruana has quit IRC | 15:23 | |
*** amuller_ is now known as amuller | 15:36 | |
*** irenab has quit IRC | 15:44 | |
*** irenab has joined #openstack-lbaas | 15:47 | |
*** phuoc_ has quit IRC | 15:53 | |
*** phuoc_ has joined #openstack-lbaas | 15:54 | |
johnsom | Busy night | 15:56 |
*** kobis has joined #openstack-lbaas | 15:58 | |
*** hongbin has quit IRC | 16:10 | |
*** yamamoto has quit IRC | 16:14 | |
*** yamamoto has joined #openstack-lbaas | 16:14 | |
*** yamamoto has quit IRC | 16:17 | |
*** yamamoto has joined #openstack-lbaas | 16:19 | |
*** SumitNaiksatam has quit IRC | 16:43 | |
*** irenab has quit IRC | 16:44 | |
*** irenab has joined #openstack-lbaas | 16:46 | |
*** AlexeyAbashkin has quit IRC | 16:57 | |
*** mstrohl has joined #openstack-lbaas | 16:58 | |
*** JudeC_ has joined #openstack-lbaas | 17:02 | |
*** kobis has quit IRC | 17:03 | |
*** yamamoto has quit IRC | 17:10 | |
*** mastrohl has joined #openstack-lbaas | 17:11 | |
*** yamamoto has joined #openstack-lbaas | 17:13 | |
*** SumitNaiksatam has joined #openstack-lbaas | 17:13 | |
*** mstrohl has quit IRC | 17:16 | |
*** mstrohl has joined #openstack-lbaas | 17:16 | |
*** kobis has joined #openstack-lbaas | 17:16 | |
*** yamamoto has quit IRC | 17:18 | |
*** mastrohl has quit IRC | 17:19 | |
*** mastrohl has joined #openstack-lbaas | 17:19 | |
*** mstrohl has quit IRC | 17:21 | |
*** sapd1 has joined #openstack-lbaas | 17:24 | |
*** sapd2 has joined #openstack-lbaas | 17:25 | |
*** yamamoto has joined #openstack-lbaas | 17:28 | |
*** yamamoto has quit IRC | 17:30 | |
*** yamamoto has joined #openstack-lbaas | 17:30 | |
*** yamamoto_ has joined #openstack-lbaas | 17:43 | |
*** yamamoto has quit IRC | 17:47 | |
*** sapd1 has quit IRC | 17:48 | |
openstackgerrit | Michael Johnson proposed openstack/octavia master: Improve the error logging for zombie amphora https://review.openstack.org/561369 | 18:01 |
*** yamamoto_ has quit IRC | 18:18 | |
*** yamamoto has joined #openstack-lbaas | 18:28 | |
*** yamamoto has quit IRC | 18:32 | |
rm_work | johnsom: did you catch upon backlog? there are a few things | 18:35 |
johnsom | yes, I have commented on a few. Need to look at the status issue after the meeting. | 18:35 |
rm_work | k | 18:36 |
*** yamamoto has joined #openstack-lbaas | 18:36 | |
johnsom | rm_work I already had this open: https://review.openstack.org/#/c/561369/ I think we should chain yours on top of mine or merge them into one patch. | 18:36 |
rm_work | ah k | 18:36 |
rm_work | yeah i see it does similar | 18:37 |
rm_work | i can just abandon mine | 18:37 |
rm_work | unless you like how i did it | 18:37 |
rm_work | eh now that i realize it's just doing stuff for the neutron queue.... meh | 18:40 |
rm_work | i don't really care how hard we try to make it right | 18:41 |
rm_work | we can just do your thing | 18:41 |
*** amuller has quit IRC | 19:12 | |
*** SumitNaiksatam has quit IRC | 19:15 | |
*** SumitNaiksatam has joined #openstack-lbaas | 19:15 | |
*** sapd2 has quit IRC | 19:31 | |
*** atoth has quit IRC | 19:32 | |
nmagnezi | rm_work, p/ | 19:38 |
nmagnezi | rm_work, o/ | 19:38 |
nmagnezi | rm_work, plz just take a look at https://review.openstack.org/#/c/549263/10/octavia/network/drivers/neutron/allowed_address_pairs.py@274 so we can merge :) | 19:38 |
*** issp has joined #openstack-lbaas | 19:39 | |
*** mastrohl has quit IRC | 19:39 | |
johnsom | Yeah, I need to re-review that one | 19:43 |
*** kobis has quit IRC | 19:43 | |
*** SumitNaiksatam has quit IRC | 19:46 | |
rm_work | nmagnezi: i mean... | 19:49 |
nmagnezi | rm_work, if i got that comment wrong please let me know | 19:50 |
nmagnezi | rm_work, everything else looked okay to me | 19:50 |
rm_work | you aren't explicitly wrong? | 19:50 |
rm_work | but like | 19:50 |
*** blake has joined #openstack-lbaas | 19:50 | |
rm_work | ¯\_(ツ)_/¯ | 19:50 |
nmagnezi | LOL | 19:50 |
nmagnezi | I think it's fair you nuke all ports for that sec-group, but we cannot declare that those owned by Octavia :P | 19:51 |
* rm_work sighs | 19:51 | |
*** yamamoto has quit IRC | 19:51 | |
nmagnezi | rm_work, don't hate me. O_O | 19:51 |
openstackgerrit | Adam Harwell proposed openstack/octavia master: When SG delete fails on vip deallocate, try harder https://review.openstack.org/549263 | 19:53 |
rm_work | already had german's +2, could have merged it T_T | 19:53 |
rm_work | but now gates and waiting again | 19:53 |
rm_work | lol | 19:53 |
nmagnezi | lol | 19:54 |
nmagnezi | rm_work, I can just +1 to give johnsom time to review before you merge ;) | 19:54 |
rm_work | augh lol | 19:55 |
rm_work | you should +2 and let johnsom review and merge :P | 19:55 |
*** SumitNaiksatam has joined #openstack-lbaas | 19:57 | |
*** yamamoto has joined #openstack-lbaas | 19:58 | |
johnsom | #startmeeting Octavia | 20:00 |
openstack | Meeting started Wed Jun 6 20:00:04 2018 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 20:00 |
openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 20:00 |
*** openstack changes topic to " (Meeting topic: Octavia)" | 20:00 | |
openstack | The meeting name has been set to 'octavia' | 20:00 |
xgerman_ | o/ | 20:00 |
johnsom | Hello everyone | 20:00 |
johnsom | I have a list of announcements today... | 20:00 |
johnsom | #topic Announcements | 20:00 |
*** openstack changes topic to "Announcements (Meeting topic: Octavia)" | 20:00 | |
johnsom | There is interest in starting a Zun compute driver for Octavia | 20:01 |
nmagnezi | o/ | 20:01 |
johnsom | #link http://lists.openstack.org/pipermail/openstack-dev/2018-June/131056.html | 20:01 |
johnsom | Good stuff there. I hope we can support that effort | 20:01 |
johnsom | The next OpenStack summit is in Berlin and the one after will be in Denver (downtown and not at the train, grin) | 20:02 |
johnsom | It sounds likely that in 2019 the PTG and summit will merge back into one event. | 20:02 |
*** yamamoto has quit IRC | 20:02 | |
johnsom | (These are all notes I have captured from the mailing list BTW | 20:03 |
johnsom | There is a proposal for Barbican to become a base service for OpenStack | 20:03 |
johnsom | #link https://review.openstack.org/#/c/572656/ | 20:03 |
johnsom | You can vote and comment on that patch | 20:03 |
johnsom | Also of note, today is Rocky milestone 2 | 20:04 |
johnsom | That means I will be cutting milestone releases | 20:04 |
johnsom | #link https://releases.openstack.org/rocky/schedule.html | 20:04 |
johnsom | Any other announcements today? | 20:04 |
*** blake has quit IRC | 20:05 | |
*** blake__ has joined #openstack-lbaas | 20:05 | |
johnsom | #topic Brief progress reports / bugs needing review | 20:05 |
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)" | 20:05 | |
johnsom | I have been busy working on the provider driver activities. I think all of the provider driver spec is now implemented. There are still things we need to work out and fix, notably status updates and the update calls. | 20:06 |
johnsom | I have also been working on the neutron-lbaas to octavia load balancer migration tool. As it is today, it *should* work for octavia provider load balancers. I still need to finish the support for migrating other driver LBs and some testing. | 20:07 |
nmagnezi | this is great! | 20:08 |
johnsom | Any other updates? | 20:08 |
rm_work | re: Milestone releases -- i think you should wait a day if possible, we have some fixes to merge for some important bugs related to stuff that just merged IMO | 20:08 |
nmagnezi | I'll try to spawn a node for that. hopefully devstack can handle installing both octavia and n-lbaas | 20:08 |
johnsom | rm_work I need to get that in today, but we have the rest of the day. Can you put together a list? | 20:09 |
nmagnezi | I have a story that needs review, maybe worth to discuss in the open discussion part | 20:09 |
johnsom | Ok | 20:09 |
nmagnezi | #link https://storyboard.openstack.org/#!/story/2002167 | 20:09 |
nmagnezi | If it's acceptable, I can try to make it work | 20:10 |
johnsom | Other updates? I know we have made some progress on grenade. | 20:10 |
nmagnezi | but there are open questions there | 20:10 |
johnsom | Hmm, yeah, this leads into another conversation I wanted to start in open discussion. | 20:12 |
nmagnezi | np | 20:12 |
johnsom | If there are no other progress updates I guess we can jump in there | 20:13 |
johnsom | Ok | 20:13 |
johnsom | #topic Brief progress reports / bugs needing review | 20:13 |
johnsom | opps | 20:13 |
*** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)" | 20:13 | |
johnsom | #topic Open Discussion | 20:13 |
*** openstack changes topic to "Open Discussion (Meeting topic: Octavia)" | 20:13 | |
xgerman_ | that was quick | 20:13 |
johnsom | Ok, so my topic was about API versioning | 20:13 |
*** blake__ is now known as blake | 20:13 | |
johnsom | Yeah, I didn't get the right topic cut and pasted, so duplicate topic for progress reports. | 20:14 |
johnsom | Nir did you want to go first or should we just talk about it in the context of the API versions? | 20:14 |
nmagnezi | i'm fine with both | 20:14 |
nmagnezi | johnsom, lead the way :) | 20:15 |
johnsom | Ok, So we have been bad | 20:16 |
johnsom | Currently we have only one version for the v2 API "v2.0" though we have added new capability | 20:16 |
xgerman_ | yeah, we should be at 2.1 | 20:17 |
johnsom | I think it is time we really get serious about versioning the API so that clients, etc. can work with our API across deployments | 20:17 |
johnsom | I proposed | 20:17 |
johnsom | #link https://review.openstack.org/#/c/559460/ | 20:17 |
johnsom | But there was pushback about the 2.1 | 20:18 |
johnsom | It's probably not the right answer anyway. | 20:18 |
johnsom | I think it might be time we consider microversioning | 20:19 |
johnsom | #link https://specs.openstack.org/openstack/api-wg/guidelines/microversion_specification.html | 20:19 |
johnsom | I'm not the biggest fan of this | 20:19 |
johnsom | Most notably if people make a request without a micro version they get the oldest version of the API by default. | 20:19 |
*** yamamoto has joined #openstack-lbaas | 20:20 | |
*** yamamoto has quit IRC | 20:20 | |
johnsom | I think that is a bit lame | 20:20 |
johnsom | It also means we have to think about how we want to handle version in the API code. | 20:20 |
nmagnezi | johnsom, just for context, when you say that we added new capability, which API capability you refer? cascade delete? asking just to get a sense of how major/minor that feature was.. | 20:20 |
johnsom | As well as in tempest as Nir mentioned | 20:20 |
xgerman_ | amphora API | 20:21 |
johnsom | Well, Queens added amphora failover | 20:21 |
nmagnezi | gotcha | 20:21 |
rm_work | yeah the amp API, which ... i don't know if anyone uses? it definitely isn't end-user | 20:21 |
johnsom | Rocky adds timeouts and listing provider drivers | 20:21 |
rm_work | and might not be in the right place given changes made with provider anyway <_< | 20:21 |
nmagnezi | rm_work, yeah but it's still an API change. a totally new API call | 20:22 |
nmagnezi | johnsom, yup. those listener timeouts will cause tests that we currently have in our tempest plugin to fail against stable/queens | 20:22 |
rm_work | yeah i guess we can't "undo" adding it ... but i wonder if it should be in a different place when we finally actually put up v2.1 or whatever | 20:23 |
nmagnezi | which makes sense. we just need to find a way to skip it somehow | 20:23 |
rm_work | eh, this is not relevant for right now, so ignore me | 20:23 |
rm_work | nmagnezi: wait, why do they cause test failures? because extra data comes back? | 20:23 |
*** SumitNaiksatam_ has joined #openstack-lbaas | 20:23 | |
*** SumitNaiksatam has quit IRC | 20:23 | |
*** SumitNaiksatam_ is now known as SumitNaiksatam | 20:23 | |
rm_work | i feel like that's badly designed testing :/ | 20:23 |
rm_work | but ... ehh | 20:24 |
nmagnezi | rm_work, because some listener attrs just didn't exist in Queens | 20:24 |
rm_work | oh | 20:24 |
johnsom | There is some docs for tempest testing with microversions | 20:24 |
johnsom | #link https://docs.openstack.org/tempest/latest/microversion_testing.html | 20:24 |
rm_work | ah right you mean the NEW tests (which test those) | 20:24 |
rm_work | and yeah obviously they don't work :P | 20:24 |
nmagnezi | rm_work, yes. it's in the story I submitted https://storyboard.openstack.org/#!/story/2002167 | 20:25 |
nmagnezi | johnsom, will read it. | 20:25 |
johnsom | To be honest, I have not read up on the microversion stuff recently, so can't talk in too much detail. | 20:25 |
rm_work | yeah i hate the "oldest" thing too | 20:26 |
rm_work | i don't suppose we could just ... do it differently? or is that code that's in the client / etc | 20:26 |
rm_work | and not really our choice | 20:26 |
nmagnezi | johnsom, if we don't want to go with micro versioning (because of what you wrote above), can't we just go with 2.1 for that reason? | 20:26 |
rm_work | yeah i don't particularly mind a v2.1 | 20:27 |
xgerman_ | So the way I undertstand microversions is that you rev up to 2.X and someday call it fine and then release 2.X as 3.0 | 20:27 |
johnsom | There was discussion at the summit about how to fix it to not be the "oldest" | 20:27 |
xgerman_ | and if somebody needs some of the “new” functionality they need to do that call with the appropriate microversion | 20:27 |
xgerman_ | which all sounds pretty crazy to me | 20:28 |
johnsom | I am fine with v2.1, it's what I proposed, but we need to figure out how to support the older and newer in our API code | 20:28 |
rm_work | yeah | 20:28 |
rm_work | i mean ermm | 20:28 |
johnsom | xgerman Yeah, so we are all in agreement on microversions kind of suck | 20:28 |
johnsom | On option is a full copy | 20:28 |
rm_work | i suppose it's naive / bad to just make a v2.1 directory in the api module, and make new classes that just inherit and pass from the old ones? >_> | 20:28 |
johnsom | Right | 20:29 |
rm_work | i guess that could lead to a mess | 20:29 |
johnsom | A bit, but at least it would be kind of clear | 20:29 |
rm_work | eugh i don't want to be in the business of fixing bugs in multiple classes at once | 20:29 |
xgerman_ | well, the people behind microversions have little concerns for the plights of us programmers having to implement those schemes | 20:30 |
rm_work | the microversions stuff has like... decorators with version numbers and allows multiple functions with the same name? | 20:30 |
rm_work | and they live side-by-side? | 20:30 |
johnsom | I'm not up to date enough on it to say | 20:32 |
rm_work | i need to do some research but, this is something we do need <_< | 20:32 |
rm_work | do we need it BY ROCKY? | 20:32 |
johnsom | I would likely need to dig into nova code to see how they do it | 20:32 |
xgerman_ | I think like it or not but we probably have to do microversions | 20:32 |
johnsom | I'm not convinced he have to do microversions. | 20:33 |
rm_work | err i am thinking of broader than openstack | 20:33 |
johnsom | neutron does not | 20:33 |
nmagnezi | maybe we can learn from neutron https://specs.openstack.org/openstack/neutron-specs/specs/liberty/microversioning.html | 20:33 |
xgerman_ | you want to do extensions? | 20:33 |
rm_work | like, i would like to research how this is done in general | 20:33 |
johnsom | Yeah, ok. I am fine with putting it on the next agenda | 20:33 |
johnsom | I do think we need something in place by the end of Rocky | 20:34 |
nmagnezi | yeah. | 20:34 |
rm_work | k | 20:34 |
rm_work | though i will maybe miss next week | 20:34 |
rm_work | i'm in an all week internal summit thing next week | 20:34 |
xgerman_ | sure, but it’s a major change and R-2… | 20:34 |
nmagnezi | johnsom, as for https://storyboard.openstack.org/#!/story/2002167 , not sure if this needs to depend on API versioning. what do you think? | 20:35 |
johnsom | nmagnezi We do need to have that running in queens. I am fine with moving forward to add the gates non-voting | 20:36 |
rm_work | yeah, we should do that | 20:37 |
nmagnezi | great | 20:37 |
rm_work | and i can noodle fixing the tests to be a little more compatible | 20:37 |
rm_work | we do have SOME api versioning | 20:37 |
rm_work | there's dates | 20:37 |
rm_work | I generally tried to update those on api changes but may not always have remembered :( | 20:37 |
nmagnezi | I'll try to make it happen and keep you all posted | 20:37 |
rm_work | but, probably i can tag on whatever queens uses | 20:37 |
rm_work | i hope it's different | 20:37 |
johnsom | Ok. Sounds good | 20:38 |
johnsom | Any other discussion today? | 20:38 |
rm_work | ummm | 20:39 |
rm_work | eh | 20:40 |
rm_work | i was gonna plug some stuff | 20:40 |
rm_work | but i think we're on it | 20:40 |
johnsom | Ok | 20:40 |
rm_work | i'll try to get a list of stuff i'd like by R2 | 20:40 |
johnsom | Excellent | 20:41 |
johnsom | It looks like we finally have movement on the zuul job update to collect the nodejs coverage reports, so the coverage patches can merge. | 20:41 |
johnsom | I also cut a python-octaviaclient in case you missed it | 20:41 |
johnsom | Otherwise, I don't have any more topics | 20:42 |
rm_work | k | 20:42 |
johnsom | Going once... | 20:42 |
rm_work | i feel like i'm forgetting something | 20:42 |
rm_work | but i'll remember after you close the meeting and we can discuss later :P | 20:43 |
johnsom | grin | 20:43 |
johnsom | #endmeeting | 20:43 |
*** openstack changes topic to "Discussion of OpenStack Load Balancing (Octavia) | https://etherpad.openstack.org/p/octavia-priority-reviews" | 20:43 | |
openstack | Meeting ended Wed Jun 6 20:43:11 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 20:43 |
openstack | Minutes: http://eavesdrop.openstack.org/meetings/octavia/2018/octavia.2018-06-06-20.00.html | 20:43 |
openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/octavia/2018/octavia.2018-06-06-20.00.txt | 20:43 |
openstack | Log: http://eavesdrop.openstack.org/meetings/octavia/2018/octavia.2018-06-06-20.00.log.html | 20:43 |
nmagnezi | o/ | 20:43 |
nmagnezi | johnsom, btw, what do you think about option #3 I listed https://storyboard.openstack.org/#!/story/2002167 ? is it something that we can even do? | 20:44 |
johnsom | nmagnezi You mean a git tag for tempest? | 20:44 |
johnsom | No, the tempest plugin is supposed to be branchless. | 20:45 |
johnsom | Which is just, odd anyway | 20:45 |
nmagnezi | johnsom, I know we can tag. the question is whether or not it is acceptable in OpenStack to release new a tag for an already version | 20:45 |
openstackgerrit | Merged openstack/octavia-tempest-plugin master: Correctly guess amp count based on detected topo https://review.openstack.org/572661 | 20:45 |
nmagnezi | johnsom, not talking about branches. just git tags | 20:46 |
nmagnezi | johnsom, say we decide which HEAD makes sense to work ok against stable/queens.. | 20:46 |
johnsom | Yeah, technically it is possible, but I don't think it is in the spirit of tempest. I think we would get pushback | 20:47 |
rm_work | nmagnezi: so the problematic tests are.... well, i guess the whole API-TEST thing | 20:47 |
rm_work | right? | 20:47 |
rm_work | since it sends those fields | 20:47 |
johnsom | Right, it's testing new stuff | 20:47 |
rm_work | i could ... ummm | 20:47 |
rm_work | we could configure tempest with a release | 20:48 |
rm_work | like | 20:48 |
rm_work | octavia_release = queens | 20:48 |
rm_work | under loadbalancer | 20:48 |
nmagnezi | rm_work, I don't think that will be accepted | 20:48 |
rm_work | and we could explicitly handle some stuff in code | 20:48 |
nmagnezi | rm_work, https://docs.openstack.org/tempest/latest/HACKING.html#skipping-tests | 20:48 |
rm_work | not skipping | 20:48 |
rm_work | like... | 20:48 |
rm_work | let me just show you | 20:49 |
nmagnezi | ok | 20:49 |
*** JudeC__ has joined #openstack-lbaas | 21:01 | |
*** JudeC_ has quit IRC | 21:01 | |
rm_work | nmagnezi: this | 21:06 |
openstackgerrit | Adam Harwell proposed openstack/octavia-tempest-plugin master: Compatibility with past releases (option 1) https://review.openstack.org/573003 | 21:06 |
*** issp has quit IRC | 21:06 | |
rm_work | what do you think? | 21:06 |
*** yamamoto has joined #openstack-lbaas | 21:20 | |
*** yamamoto has quit IRC | 21:33 | |
cgoncalves | rm_work, https://docs.openstack.org/tempest/latest/HACKING.html#skipping-tests | 21:45 |
rm_work | right, i don't think this applies | 21:46 |
rm_work | A) Based on configuration (see the config flag?) | 21:46 |
rm_work | B) Not skipping the test | 21:46 |
cgoncalves | btw, sorry I missed the meeting. I forgot to give a heads up | 21:46 |
cgoncalves | rm_work, yeah... although I'm not that keen on that option 1 | 21:47 |
cgoncalves | a decorator to skip could be interesting to explore | 21:48 |
rm_work | we don't want to SKIP the test | 21:48 |
rm_work | that'd be lame | 21:48 |
rm_work | then we would have no tests :P | 21:48 |
rm_work | just modify the test to match the target cloud intelligently | 21:48 |
cgoncalves | but that would imply writing multiple similar tests, that that one with the timeouts | 21:48 |
rm_work | why not just do this <_< | 21:49 |
cgoncalves | for some cases you can modify the test, for others you have to skip (e.g. amphora failover) | 21:49 |
rm_work | that seems like a lot more work | 21:49 |
rm_work | we already do essentially this (change tests based on cloud features) in the *exact same way* | 21:49 |
rm_work | https://github.com/openstack/octavia-tempest-plugin/blob/master/octavia_tempest_plugin/tests/api/v2/test_member.py#L139-L149 | 21:50 |
rm_work | two things right there | 21:50 |
rm_work | that are based on config | 21:50 |
rm_work | the first one does EXACTLY this | 21:50 |
cgoncalves | hmmm in that case, if I had to choose between that and your option 1 patch, I'd say option 1 patch because it's more explicit to users | 21:53 |
* rm_work shrugs | 21:54 | |
rm_work | will see what johnsom thinks | 21:54 |
rm_work | oh right i was in the middle of making alist | 21:54 |
*** rcernin has joined #openstack-lbaas | 21:54 | |
openstackgerrit | Adam Harwell proposed openstack/octavia master: Add lb_id comment to amp haproxy listener config https://review.openstack.org/572730 | 21:55 |
openstackgerrit | Adam Harwell proposed openstack/octavia master: Improve the error logging for zombie amphora https://review.openstack.org/561369 | 21:59 |
rm_work | johnsom: ^^ fixed your unit test failures | 21:59 |
johnsom | Oh, thanks | 21:59 |
*** lxkong has joined #openstack-lbaas | 22:01 | |
rm_work | and reviewing that | 22:03 |
rm_work | as it's one of the ones i want to get in | 22:03 |
rm_work | as well as, it'd be nice to merge my SG thing | 22:03 |
rm_work | though not as urgent | 22:03 |
rm_work | (just, it's ready, and fairly straightforward) | 22:03 |
xgerman_ | rm_work: in my practice I am seeing the Warn utput on busy amps (https://github.com/openstack/octavia/blob/master/octavia/controller/healthmanager/health_drivers/update_db.py#L137-L142) | 22:04 |
xgerman_ | wonder if we should ignore the heartebeat when an amp is busy | 22:04 |
rm_work | xgerman_: yeah that's not good :( | 22:04 |
rm_work | no, because then when it comes out of busy, it might immediately try to failover | 22:04 |
rm_work | unless we also update the code that does the un-busy to also update the time | 22:05 |
xgerman_ | k, but in my. case it just count up the secs since last update and keeps skipping | 22:05 |
rm_work | but that is like | 22:05 |
rm_work | so minimal | 22:05 |
rm_work | the real issue is that your stuff is crazy overloaded somehow :/ | 22:05 |
rm_work | err | 22:05 |
rm_work | that is not what that does | 22:05 |
rm_work | it's the time from packet receipt, until processing | 22:05 |
rm_work | nothing to do with the health table data | 22:05 |
xgerman_ | ah | 22:05 |
rm_work | it means you are receiving the packet, and it is taking a LONG TIME to process | 22:06 |
rm_work | which is bad-news-bears | 22:06 |
xgerman_ | yep | 22:06 |
rm_work | your processes are still broken | 22:06 |
xgerman_ | thsi time we think it’s the DB | 22:06 |
rm_work | did we get all the patches backported and are you up-to-date? | 22:06 |
rm_work | the DB traffic really isn't that heavy, i thought that a few times but it turns out it's really not a big deal for MySQL unless you have like 1000000 amps | 22:06 |
rm_work | what are the times you're seein | 22:07 |
xgerman_ | no, mysql being broken | 22:07 |
rm_work | like, 10s? | 22:07 |
rm_work | what is your heartbeat interval? | 22:07 |
xgerman_ | 10 | 22:09 |
xgerman_ | Ibut we have seen galera to go crazy — so not us | 22:09 |
*** fnaval has quit IRC | 22:15 | |
rm_work | ummmmmm | 22:15 |
rm_work | how many amps | 22:15 |
rm_work | xgerman_: something is *wrong* | 22:16 |
rm_work | obviously | 22:16 |
xgerman_ | yeah, mysql went out to lunch | 22:16 |
rm_work | but like | 22:16 |
rm_work | <_< | 22:16 |
johnsom | Sigh, xgerman_ your issue in your lab has nothing to do with Octavia. You can stop the process and everything is still hosed up | 22:16 |
rm_work | yeah i think he's getting that :P | 22:16 |
rm_work | but i'm surprised it's mysql too | 22:16 |
johnsom | octavia is handling it properly. The DB steps are taking 40 seconds, so it's throwing stuff out. | 22:17 |
rm_work | yep, that's what he said actually | 22:17 |
rm_work | still surprising | 22:17 |
xgerman_ | yeah, I was just curious why we don;t ignore heartbeats from amps which are busy… | 22:17 |
xgerman_ | which rm_work answered | 22:17 |
johnsom | Yeah, in the mysql container it's at 1200% CPU according to top, so.... | 22:19 |
rm_work | loool | 22:19 |
rm_work | eh, johnsom maybe it doesn't matter, no one should be installing the R2 anyway honestly... | 22:36 |
rm_work | I hope... | 22:36 |
johnsom | So, no list? | 22:37 |
xgerman_ | better safe than sorry | 22:37 |
rm_work | i was compiling one but basically all that ended up on it were things that are meh, and things that are just not going to be ready today | 22:38 |
rm_work | the zombie thing for one (as it fixes a bug that causes explosions in the stats update) | 22:38 |
rm_work | but i had to -1 it | 22:38 |
rm_work | and the SG one really doesn't matter much | 22:38 |
rm_work | and the driver callback stuff and the delete api updates are not going to happen for a bit | 22:39 |
xgerman_ | The SG one would be useful for me but I go of shas so no real pressure | 22:41 |
rm_work | right | 22:44 |
rm_work | i assume people running so new that they would do RCs, would ... actually just use master or SHAs | 22:44 |
xgerman_ | unless you pip install | 22:46 |
rm_work | err | 22:46 |
rm_work | would the RC actually go out to people | 22:46 |
xgerman_ | if you install from pypi | 22:47 |
rm_work | erg | 22:48 |
*** threestrands has joined #openstack-lbaas | 22:51 | |
*** gigo has quit IRC | 22:51 | |
*** gigo has joined #openstack-lbaas | 22:56 | |
*** eandersson_ is now known as eandersson | 22:59 | |
johnsom | I have posted the MS2 release patch | 23:16 |
johnsom | rm_work Struggling with your comment on the zombie patch | 23:28 |
rm_work | erm | 23:30 |
johnsom | I see a bug, but not the amp deleted thing | 23:31 |
rm_work | so you say you're looking for spare amps | 23:31 |
johnsom | I am saying, if it doesn't come back with an LBs owning the amp, and the amp isn't a spare, ignore the health message | 23:33 |
openstackgerrit | Michael Johnson proposed openstack/octavia master: Improve the error logging for zombie amphora https://review.openstack.org/561369 | 23:37 |
rm_work | so | 23:37 |
rm_work | amp = self.amphora_repo.get(session, id=health['id']) | 23:37 |
rm_work | that gets the amp | 23:37 |
rm_work | if amp.load_balancer_id | 23:37 |
rm_work | that's the spare check | 23:38 |
johnsom | not a spare check | 23:38 |
rm_work | err wait | 23:38 |
rm_work | hold on | 23:38 |
rm_work | this whole thing... | 23:39 |
rm_work | are you expecting this whole thing to execute if the amp is deleted? | 23:39 |
johnsom | If it is deleted, it should log and return, not process the heartbeat | 23:40 |
rm_work | erm | 23:40 |
rm_work | if it's deleted, it still won't execute this | 23:40 |
johnsom | Because if it is deleted, it shouldn't be sending heartbeats | 23:40 |
rm_work | if it's deleted, it will still get a return value from db_lbs_on_amp = self.amphora_repo.get_all_lbs_on_amphora | 23:40 |
rm_work | that doesn't filter by non-deleted | 23:41 |
rm_work | i think i just put the comment in the wrong place | 23:41 |
johnsom | That is true, it would match if the load balancer is deleted, so it would still process the heartbeat | 23:42 |
rm_work | yeah, LB+amp could both be deleted | 23:43 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!