Wednesday, 2017-09-13

*** dayou has joined #openstack-lbaas00:01
openstackgerritAdam Harwell proposed openstack/octavia master: Gatefix: Migrate from testr to stestr  https://review.openstack.org/50315600:02
openstackgerritAdam Harwell proposed openstack/octavia master: Gatefix: Migrate from testr to stestr  https://review.openstack.org/50315600:04
*** salmankhan has quit IRC00:19
*** ianychoi_ is now known as ianychoi00:39
dayou@johnsom, I remember you have a script to automate the creation of loadbalancers/listeners/members in devstack, would like to use it but don't know where it is.00:41
*** cody-somerville has joined #openstack-lbaas00:54
*** cody-somerville has quit IRC00:54
*** cody-somerville has joined #openstack-lbaas00:54
*** atoth has quit IRC01:00
*** catintheroof has quit IRC01:13
openstackgerritHengqing Hu proposed openstack/neutron-lbaas master: Use interface opts from the right place  https://review.openstack.org/50349801:39
*** yamamoto_ has joined #openstack-lbaas01:40
openstackgerritMerged openstack/python-octaviaclient master: Move to auto-generated CLI reference  https://review.openstack.org/49682701:40
openstackgerritMerged openstack/octavia-dashboard master: Update links in README  https://review.openstack.org/50111501:42
*** cody-somerville has quit IRC01:44
*** bbbbzhao_ has quit IRC02:00
*** yamamoto_ has quit IRC02:03
*** sticker has joined #openstack-lbaas02:12
*** cody-somerville has joined #openstack-lbaas02:23
*** fnaval has joined #openstack-lbaas02:27
*** csomerville has joined #openstack-lbaas02:32
*** yamamoto has joined #openstack-lbaas02:32
*** cody-somerville has quit IRC02:35
*** reedip_afk is now known as reedip02:41
*** yamamoto has quit IRC02:47
*** ramishra has joined #openstack-lbaas03:37
ramishrahi guys, it seems we need https://review.openstack.org/#/c/352471 to merge after https://review.openstack.org/#/c/347867/52 landed in neutron.03:40
ramishrabut it seems to be stuck for a long time, probably needs a rebase and +W ?03:41
openstackgerritHengqing Hu proposed openstack/neutron-lbaas master: Updating import for linux/interface opts  https://review.openstack.org/35247103:46
*** links has joined #openstack-lbaas03:48
ramishraoh.. it was dependant on the neutron patch03:51
dayouThat patch is not correct03:53
*** gans has joined #openstack-lbaas03:57
*** ramishra has quit IRC04:00
openstackgerritHengqing Hu proposed openstack/neutron-lbaas master: Updating import for linux/interface opts  https://review.openstack.org/35247104:01
*** ramishra has joined #openstack-lbaas04:11
*** ramishra has quit IRC04:21
*** ramishra has joined #openstack-lbaas04:23
*** aojea has joined #openstack-lbaas04:34
*** aojea has quit IRC04:38
*** armax has joined #openstack-lbaas04:50
johnsomdayou http://paste.openstack.org/show/620996/05:01
*** chlong has quit IRC05:01
johnsomThat is a mostly converted to openstack client script I use for the Pike Octavia endpoint (no neutron-lbaas)05:01
stickerhmm... i've bumped into a sort of race condition when creating a number of resources quickly05:07
dayouThank you very much, quite handy, will save a lot of time with this :-D05:08
stickerif i create a lot of members quickly (eg. in a heat template or autoscaling group), the requests get fired off from the neutron_lbaas octavia driver but the first one gets in, sets the load balancer to PENDING_UPDATE and any other requests that hit get the 'immutable' response and are dropped05:09
stickerleads to my neutron database getting out of sync with octavia05:09
stickeranyone know if this is a known thing at the moment?05:09
johnsomsticker If you are still using neutron-lbaas you have to enable the sync options in octavia to push status/status information back into neutron database to stop neutron from getting out of sync.05:10
stickerthe event_streamer?05:11
johnsomBest option is to use Pike or newer and not use neutron-lbaas anymore.05:11
stickeri'm on ocata so the loadbalancer stats bit of code isn't there (i think that landed in 11.0.1?)05:11
johnsomsticker event_streamer and sync_provisioning_status05:12
stickeri dont think that syncs the member configuration though does it?05:12
stickerthats the problem i have.  if i create a bunch of members, neutron db will show all of them, octavia will show *most* of them05:12
stickerjohnsom: i am really looking forward to upgrading to Pike but that is going to take a fair amount of time, this is a production setup that is not allowed downtime05:14
johnsomneutron should be giving you a 409 when the LB is in PENDING_UPDATE and allowing addition updates to the same LB.  However, I will also note: https://review.openstack.org/#/c/477034/ is really what you want....05:14
stickeryes, neutron sees the 409 but the driver doesn't handle it and the member configuration is not applied05:15
*** chlong has joined #openstack-lbaas05:15
stickerjohnsom: it might be related to the multiple listeners we have for each load balancer.  The other reports I have seen of this all have multiple listeners too05:16
johnsomYes, in 409, the member config should not be applied.  I wonder if this is one of the cases where neutron is ignoring the status code the drivers are returning and pretending things were fine05:16
stickerjohnsom: it totally does.. i have patched ours to at least retry05:16
stickerhttps://github.com/openstack/neutron-lbaas/blob/master/neutron_lbaas/drivers/octavia/driver.py#L426-L428 <- the call to create the member in octavia05:17
stickerhttps://github.com/openstack/neutron-lbaas/blob/master/neutron_lbaas/drivers/octavia/driver.py#L128-L148 <- the request code05:17
stickerit makes the call, logs the response code, but no retry and no checking of the result05:18
johnsomYeah, the no checking of the result is not good05:18
johnsomYou know you can run Pike octavia with a Ocata clouds, so if you have HA control plane for octavia you likely can do a rolling upgrade.  We haven't really tested it or have gates for it yet, but in theory it should be possible05:19
stickeri've slapped a loop in there to try again (after a short delay) if r.status_code is > 299 for now05:20
stickeroh okay.. i might look into that05:20
johnsomYeah, there is one outfit running master octavia on a liberty cloud...  Only for the brave IMO05:20
stickerhaha wow05:21
stickeri'd submit a bug report and suggest my loop-iness for review but i have no idea how i would write a test for it05:22
johnsomIf you want to upstream fixes for neutron-lbaas and backport to ocata you can still do that.  I'm not sure the retry is really a "fix", but if you wanted to spend some time on it.05:22
stickeryeah, it's still a workaround05:22
johnsomYeah, a bug would be good.  Note, we moved our bugs to the new Storyboard project (all of openstack is going to over time)05:22
johnsomhttps://storyboard.openstack.org/#!/05:23
stickeri haven't read the upgrade notes for octavia yet for going to Pike.  neutron_lbaas is gone is it?05:23
stickerjohnsom: thanks, will submit something now05:23
johnsomIt is no longer required.  Octavia has it's own endpoint now.  neutron-lbaas isn't yet in deprecation because we didn't get the vendor driver interface done in Octavia in Pike.  Likely we will start the deprecation clock towards the end of Queens.05:24
johnsomMuch faster API, no DB issues, openstack client CLI, etc.05:24
stickerahh okay. nice05:25
johnsomOk, I'm going to sign off for the evening.  Need to get some sleep before I Octavia sessions tomorrow at the PTG.05:26
*** armax has quit IRC05:31
*** armax has joined #openstack-lbaas05:37
*** rcernin has joined #openstack-lbaas05:44
*** pcaruana has joined #openstack-lbaas05:50
*** csomerville has quit IRC06:00
*** csomerville has joined #openstack-lbaas06:03
*** aojea has joined #openstack-lbaas06:03
*** aojea has quit IRC06:10
*** armax has quit IRC06:11
*** eezhova has joined #openstack-lbaas06:26
*** aojea has joined #openstack-lbaas06:35
openstackgerritHengqing Hu proposed openstack/octavia-dashboard master: Convert octavia-dashboard to use Octavia v2 API  https://review.openstack.org/49985806:46
*** sticker has quit IRC06:52
*** tesseract has joined #openstack-lbaas06:58
*** eezhova has quit IRC07:21
*** csomerville has quit IRC07:24
*** eezhova has joined #openstack-lbaas07:44
*** logan- has quit IRC08:05
*** logan- has joined #openstack-lbaas08:06
*** yamamoto has joined #openstack-lbaas08:27
*** gans819 has joined #openstack-lbaas08:29
*** gans has quit IRC08:31
openstackgerritHengqing Hu proposed openstack/octavia-dashboard master: Convert octavia-dashboard to use Octavia v2 API  https://review.openstack.org/49985808:35
*** salmankhan has joined #openstack-lbaas09:17
*** yamamoto has quit IRC09:26
*** eezhova has quit IRC09:29
*** yamamoto has joined #openstack-lbaas09:32
*** yamamoto has quit IRC09:34
*** eezhova has joined #openstack-lbaas09:35
*** aojea_ has joined #openstack-lbaas09:48
*** aojea has quit IRC09:51
*** reedip has quit IRC10:17
*** reedip has joined #openstack-lbaas10:17
*** eezhova_ has joined #openstack-lbaas10:28
*** eezhova has quit IRC10:29
*** eezhova_ has quit IRC10:38
*** eezhova has joined #openstack-lbaas10:39
*** reedip has quit IRC10:39
*** reedip has joined #openstack-lbaas10:52
*** reedip is now known as reedip_afk11:02
*** bbbbzhao_ has joined #openstack-lbaas11:06
*** gans819 has quit IRC11:15
nmagnezijohnsom, o/11:16
nmagnezijohnsom, working on those devstack examples11:16
nmagnezijohnsom, we need to update our devstack plugin as well11:16
nmagnezijohnsom, we still use the old client via cli11:17
nmagnezijohnsom, i'll try to cover that as well11:17
openstackgerritLingxian Kong proposed openstack/octavia master: Fix inappropriate provisioning_status of l7 policy  https://review.openstack.org/50360411:42
openstackgerritZhaoBo proposed openstack/octavia master: WIP: Support UDP load balance  https://review.openstack.org/50360611:59
*** reedip_afk has quit IRC12:02
*** reedip_afk has joined #openstack-lbaas12:14
*** catintheroof has joined #openstack-lbaas12:38
openstackgerritOpenStack Proposal Bot proposed openstack/python-octaviaclient master: Updated from global requirements  https://review.openstack.org/50002912:44
openstackgerritNir Magnezi proposed openstack/octavia master: WIP: Update devstack plugin and examples  https://review.openstack.org/50363812:57
openstackgerritOpenStack Proposal Bot proposed openstack/octavia master: Updated from global requirements  https://review.openstack.org/50204712:57
*** eezhova has quit IRC12:59
openstackgerritOpenStack Proposal Bot proposed openstack/python-octaviaclient master: Updated from global requirements  https://review.openstack.org/50002913:02
johnsomnmagnezi I have a patch up for the devstack plugin (though it doesn't work)13:04
johnsomI think I did the host binding wrong13:06
*** fnaval has quit IRC13:24
*** leitan has joined #openstack-lbaas13:27
*** eezhova has joined #openstack-lbaas13:38
*** armax has joined #openstack-lbaas13:43
*** yamamoto has joined #openstack-lbaas13:48
*** fnaval has joined #openstack-lbaas13:49
*** links has quit IRC14:05
openstackgerritYAMAMOTO Takashi proposed openstack/neutron-lbaas master: Update after the recent refactoring of Neutron options  https://review.openstack.org/50368814:11
openstackgerritHengqing Hu proposed openstack/octavia-dashboard master: Able to change lb algorithm on pool updating  https://review.openstack.org/50368914:27
*** atoth has joined #openstack-lbaas14:35
*** yamamoto has quit IRC14:38
*** apuimedo has joined #openstack-lbaas14:38
apuimedojohnsom: Hi14:39
apuimedoI know you're in the PTG, but if you could take a look at https://review.openstack.org/#/c/352471/1714:40
johnsomHi14:40
apuimedodulek reports that it fixes our gate14:40
johnsomOk, yeah, those changes were in and out yesterday.  We will look soon14:40
apuimedoin and out?14:41
*** dulek has joined #openstack-lbaas14:41
*** armax has quit IRC14:47
*** tonygunk has joined #openstack-lbaas14:49
tonygunkGot Pike standalone without Neutron set up mostly but getting error on LB creation - something around certificates14:51
tonygunkhttp://paste.openstack.org/raw/621043/14:51
tonygunkError: [('PEM routines', 'PEM_read_bio', 'no start line'), ('SSL routines', 'SSL_CTX_use_PrivateKey_file', 'PEM lib')]14:51
tonygunksomething I'm doing I'm sure - but passphrase is correct for private ca key14:52
tonygunkideas on where to troubleshoot?14:54
*** yamamoto has joined #openstack-lbaas14:58
*** yamamoto_ has joined #openstack-lbaas15:01
*** yamamoto has quit IRC15:04
openstackgerritYAMAMOTO Takashi proposed openstack/neutron-lbaas master: Update after the recent refactoring of Neutron options  https://review.openstack.org/50368815:05
*** Alex_Staf has joined #openstack-lbaas15:06
*** tongl has joined #openstack-lbaas15:06
*** armax has joined #openstack-lbaas15:07
xgerman_ /join openstack-ptg15:16
xgerman_\join openstack-ptg15:17
rm_workStoryboard for Octavia: https://storyboard.openstack.org/#!/project/90815:51
*** rcernin has quit IRC15:51
openstackgerritBrian Haley proposed openstack/neutron-lbaas master: Updating import for linux/interface opts  https://review.openstack.org/35247115:52
rm_workProject group: https://storyboard.openstack.org/#!/project_group/7015:53
johnsomhttps://www.openstack.org/videos/boston-2017/storyboard-101-survival-guide-to-the-great-migration16:00
*** ajo has joined #openstack-lbaas16:06
*** Alex_Staf has quit IRC16:09
*** ipsecguy_ has joined #openstack-lbaas16:14
*** ipsecguy has quit IRC16:15
*** eezhova has quit IRC16:17
*** yamamoto_ has quit IRC16:26
*** links has joined #openstack-lbaas16:28
*** belharar has joined #openstack-lbaas16:29
*** belharar has quit IRC16:31
*** Alex_Staf has joined #openstack-lbaas16:31
*** belharar has joined #openstack-lbaas16:31
*** yamamoto has joined #openstack-lbaas16:31
*** belharar has quit IRC16:33
*** tesseract has quit IRC16:36
*** sapd_ has joined #openstack-lbaas16:42
*** apuimedo has quit IRC16:43
*** sapd has quit IRC16:44
*** eezhova has joined #openstack-lbaas16:47
*** JudeC has joined #openstack-lbaas16:55
*** sshank has joined #openstack-lbaas17:06
*** rstarmer has joined #openstack-lbaas17:09
openstackgerritAdam Harwell proposed openstack/octavia master: Gatefix: Migrate from testr to stestr  https://review.openstack.org/50315617:12
*** yamamoto has quit IRC17:17
*** chlong_ has joined #openstack-lbaas17:22
*** chlong has quit IRC17:24
*** links has quit IRC17:27
*** armax has quit IRC17:40
*** sshank has quit IRC17:42
*** apuimedo has joined #openstack-lbaas17:45
*** Alex_Staf has quit IRC17:47
*** Alex_Staf has joined #openstack-lbaas17:49
*** Alex_Staf has quit IRC17:54
*** sshank has joined #openstack-lbaas17:58
*** tongl has quit IRC18:01
*** ramishra has quit IRC18:14
*** mixos has joined #openstack-lbaas18:24
*** eezhova has quit IRC18:25
*** rstarmer has quit IRC18:30
*** rstarmer has joined #openstack-lbaas18:31
*** rstarmer has quit IRC18:35
*** med_ is now known as med18:37
*** med is now known as Guest2031718:38
*** Guest20317 is now known as med_18:38
*** salmankhan has quit IRC18:39
nmagnezirm_work, what is stestr ?18:47
*** sshank has quit IRC18:51
rm_worknmagnezi: fork of testr :)18:55
*** yamamoto has joined #openstack-lbaas18:59
*** eezhova has joined #openstack-lbaas19:01
*** eezhova has quit IRC19:01
openstackgerritMerged openstack/neutron-lbaas master: Updating import for linux/interface opts  https://review.openstack.org/35247119:01
*** chlong__ has joined #openstack-lbaas19:02
*** chlong_ has quit IRC19:04
*** armax has joined #openstack-lbaas19:04
*** rcernin has joined #openstack-lbaas19:06
nmagnezirm_work, :D19:08
tonygunkFYI issue with PEM error above was because I was using amphora_haproxy_rest_driver19:10
tonygunkchanged to noop and went away19:11
rm_workerrr19:11
*** mixos has quit IRC19:13
*** mixos has joined #openstack-lbaas19:15
*** mixos has quit IRC19:18
*** pcaruana has quit IRC19:20
*** tonygunk has quit IRC19:34
*** rstarmer has joined #openstack-lbaas19:37
*** Alex_Staf has joined #openstack-lbaas19:40
*** Alex_Staf has quit IRC19:47
*** mixos has joined #openstack-lbaas19:49
*** rstarmer has quit IRC19:51
*** khamtamtun has joined #openstack-lbaas19:53
*** yamamoto has quit IRC19:53
*** ramishra has joined #openstack-lbaas19:57
*** sshank has joined #openstack-lbaas19:57
*** atoth has quit IRC20:02
*** yamamoto has joined #openstack-lbaas20:03
*** rstarmer has joined #openstack-lbaas20:07
*** khamtamtun has quit IRC20:08
*** tonygunk has joined #openstack-lbaas20:08
*** salmankhan has joined #openstack-lbaas20:12
*** bzhao has quit IRC20:12
*** ianychoi has quit IRC20:14
*** Alex_Staf has joined #openstack-lbaas20:15
*** rcernin has quit IRC20:18
*** ramishra has quit IRC20:19
*** ajo_ has joined #openstack-lbaas20:22
*** tonygunk has quit IRC20:22
ajoapuimedo: ping20:22
*** tonygunk has joined #openstack-lbaas20:23
ajodo you want to connect with us to the octavia cointainers discussion via bluejeans20:23
ajo?20:23
apuimedoajo: when does it start?20:23
ajoit is now20:25
*** yamamoto has quit IRC20:26
apuimedoajo: link?20:27
*** yamamoto has joined #openstack-lbaas20:29
*** csomerville has joined #openstack-lbaas20:32
*** Alex_Staf has quit IRC20:36
nmagneziajo, o/20:36
*** cody-somerville has joined #openstack-lbaas20:38
*** cody-somerville has quit IRC20:38
*** cody-somerville has joined #openstack-lbaas20:38
*** csomerville has quit IRC20:41
*** aojea_ has quit IRC20:42
*** aojea has joined #openstack-lbaas20:42
*** catintheroof has quit IRC20:42
*** aojea_ has joined #openstack-lbaas20:43
*** aojea has quit IRC20:44
*** leitan has quit IRC20:45
*** blogan has joined #openstack-lbaas20:45
*** tongl has joined #openstack-lbaas20:47
*** Alex_Staf has joined #openstack-lbaas20:50
*** tonygunk has quit IRC20:52
*** ajo_ has quit IRC20:57
*** Alex_Staf has quit IRC21:00
*** tongl has quit IRC21:00
*** Alex_Staf has joined #openstack-lbaas21:07
*** ajo_ has joined #openstack-lbaas21:08
*** rstarmer has quit IRC21:09
*** bbbbzhao_ has quit IRC21:12
*** yamamoto has quit IRC21:27
*** mixos has quit IRC21:28
*** armax has quit IRC21:28
*** mixos has joined #openstack-lbaas21:32
*** cody-somerville has quit IRC21:35
*** armax has joined #openstack-lbaas21:36
*** sshank has quit IRC21:40
*** sshank has joined #openstack-lbaas21:47
*** yamamoto_ has joined #openstack-lbaas21:53
openstackgerritLingxian Kong proposed openstack/octavia master: Fix inappropriate provisioning_status of l7 policy  https://review.openstack.org/50360421:56
*** mixos has quit IRC21:58
*** salmankhan has quit IRC22:02
*** mixos has joined #openstack-lbaas22:02
*** mixos has quit IRC22:02
*** yamamoto_ has quit IRC22:03
*** KeithMnemonic2 has joined #openstack-lbaas22:11
openstackgerritMerged openstack/neutron-lbaas master: Stop registering "graphs" resource to quota  https://review.openstack.org/50164722:12
*** KeithMnemonic1 has quit IRC22:14
*** armax has quit IRC22:17
*** rstarmer has joined #openstack-lbaas22:21
*** Alex_Staf has quit IRC22:24
*** armax has joined #openstack-lbaas22:25
*** yamamoto_ has joined #openstack-lbaas22:27
*** armax has quit IRC22:30
*** rstarmer has quit IRC22:31
*** ianychoi has joined #openstack-lbaas22:32
*** ipsecguy_ has quit IRC22:32
*** ipsecguy has joined #openstack-lbaas22:34
*** armax has joined #openstack-lbaas22:35
*** armax has quit IRC22:36
*** rstarmer has joined #openstack-lbaas22:38
*** armax has joined #openstack-lbaas22:38
openstackgerritMerged openstack/neutron-lbaas master: Updated from global requirements  https://review.openstack.org/50269722:42
*** fnaval has quit IRC22:46
*** ajo_ has quit IRC22:46
*** rstarmer has quit IRC22:47
*** armax has quit IRC22:49
*** rstarmer has joined #openstack-lbaas22:53
*** salmankhan has joined #openstack-lbaas22:54
-openstackstatus- NOTICE: CentOS 7.4 point release today has resulted in some mirror disruption, repair underway; expect jobs on centos7 nodes to potentially fail for a few hours longer23:04
*** KeithMnemonic2 has quit IRC23:07
konghello23:20
*** mixos has joined #openstack-lbaas23:24
*** jniesz has joined #openstack-lbaas23:25
*** mixos has quit IRC23:29
*** catintheroof has joined #openstack-lbaas23:34
*** catintheroof has quit IRC23:35
*** catintheroof has joined #openstack-lbaas23:37
kongIf i understand correctly, the current octavia tempest test cases are only covering v1 endpoint, right?23:39
*** armax has joined #openstack-lbaas23:39
kongI see there is octavia-tempest-plugin repo, but nothing inside, seems it's still in working in progress23:39
*** jniesz has quit IRC23:41
johnsomCorrect, there are patches up for review23:42
rm_workkong: there are a lot of patches23:42
rm_workwe're working on getting those going soon23:42
rm_workso yes, octavia-tempest-plugin is the right place to look for the future tests for v223:42
kongjohnsom, rm_work, and octavia gate is broken?23:43
konghttp://logs.openstack.org/04/503604/2/check/gate-octavia-v1-dsvm-scenario-ubuntu-xenial/ae74f20/console.html23:43
rm_workkong: fix is merging now23:43
rm_workhttps://review.openstack.org/#/c/503156/23:43
konghmm, ok23:43
rm_workas soon as that merges you can recheck23:43
kongi am in a good timing :-(23:43
rm_workshould be about an hour23:44
rm_worki hope23:44
kongseems i need to walk through current patches in review in different repos, to get an overview of current octavia development status23:45
kongi found a bug, btw https://review.openstack.org/#/c/503156/23:45
kongSorry, this one https://review.openstack.org/#/c/503604/23:46
*** sshank has quit IRC23:46
johnsomkong This might help: https://review.openstack.org/#/q/(project:openstack/octavia+OR+project:openstack/octavia-dashboard+OR+project:openstack/python-octaviaclient+OR+project:openstack/octavia-tempest-plugin)+AND+status:open+AND+NOT+label:Code-Review%253C%253D0+AND+NOT+label:Verified%253C%253D0+AND+NOT+label:Workflow%253C%253D023:50
johnsomPatches ready for review across our gerrits23:51
kongyeah, i have a similar one. and i already watched all related projects to get notified if there is new patch proposed23:51
*** armax has quit IRC23:52
kongbut thanks anyway for the useful link :-)23:53

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!