Friday, 2016-08-19

*** rudrajit has quit IRC00:09
*** rudrajit has joined #openstack-lbaas00:09
*** rudrajit has quit IRC00:14
johnsomgreghaynes Thanks for poking at the elements again.  I think the issue with the haproxy merged element is it is installing the non-backports version of haproxy which we don't support.00:15
*** rudrajit has joined #openstack-lbaas00:23
*** sticker has quit IRC00:27
*** chenli has joined #openstack-lbaas00:34
*** sticker has joined #openstack-lbaas00:49
*** rudrajit has quit IRC00:52
*** rudrajit has joined #openstack-lbaas00:53
*** rudrajit has quit IRC00:58
*** kevo has quit IRC01:05
*** bana_k has quit IRC01:05
greghaynesjohnsom: ah, interesting01:23
greghaynesjohnsom: I wonder if I messed up the package-map then01:23
*** chlong has joined #openstack-lbaas01:24
*** ducttape_ has joined #openstack-lbaas01:28
*** ducttape_ has quit IRC01:53
openstackgerritzhangyanxian proposed openstack/neutron-lbaas: Change assertTrue(isinstance()) by optimal assert  https://review.openstack.org/35757002:00
*** diogogmt has quit IRC02:00
openstackgerritzhangyanxian proposed openstack/neutron-lbaas: Change assertTrue(isinstance()) by optimal assert  https://review.openstack.org/35757002:01
*** ducttape_ has joined #openstack-lbaas02:01
*** armax has joined #openstack-lbaas02:10
*** ducttape_ has quit IRC02:11
*** chlong has quit IRC02:29
*** chlong has joined #openstack-lbaas02:46
*** yuanying has quit IRC02:56
*** yuanying has joined #openstack-lbaas02:58
*** ducttape_ has joined #openstack-lbaas03:12
*** ducttape_ has quit IRC03:17
*** yuanying has quit IRC03:22
*** woodster_ has quit IRC03:29
*** eandersson_ has joined #openstack-lbaas03:30
*** eandersson_ has quit IRC03:37
*** chlong has quit IRC03:39
*** chenli has quit IRC03:44
*** rudrajit has joined #openstack-lbaas03:48
*** csomerville has joined #openstack-lbaas03:48
*** cody-somerville has quit IRC03:48
*** chlong has joined #openstack-lbaas03:52
*** yuanying has joined #openstack-lbaas03:54
*** armax has quit IRC04:06
*** armax has joined #openstack-lbaas04:07
*** armax has quit IRC04:07
*** armax has joined #openstack-lbaas04:07
*** armax has quit IRC04:08
*** chlong has quit IRC04:10
*** ducttape_ has joined #openstack-lbaas04:13
*** links has joined #openstack-lbaas04:15
*** ducttape_ has quit IRC04:18
openstackgerritPhillip Toohill proposed openstack/octavia: Updating network base utils for more functionality  https://review.openstack.org/35658804:18
*** chlong has joined #openstack-lbaas04:23
*** rudrajit has quit IRC04:44
*** rudrajit has joined #openstack-lbaas04:44
*** rudrajit has quit IRC04:49
*** rudrajit has joined #openstack-lbaas04:50
*** armax has joined #openstack-lbaas04:52
*** numans has joined #openstack-lbaas04:55
*** armax has quit IRC04:58
*** armax has joined #openstack-lbaas05:01
*** chlong has quit IRC05:12
*** ducttape_ has joined #openstack-lbaas05:14
*** nmagnezi has joined #openstack-lbaas05:16
*** ducttape_ has quit IRC05:18
*** chenli has joined #openstack-lbaas05:19
*** bana_k has joined #openstack-lbaas05:26
*** chlong has joined #openstack-lbaas05:26
*** fnaval has quit IRC05:28
*** armax has quit IRC05:36
reedipblogan: ping05:47
openstackgerritYAMAMOTO Takashi proposed openstack/neutron-lbaas: Update disable_ssl_certificate_validation reference  https://review.openstack.org/35761905:51
*** chlong has quit IRC05:52
openstackgerritReedip proposed openstack/neutron-lbaas: Add Timeout on Listeners  https://review.openstack.org/27389605:52
*** anilvenkata has joined #openstack-lbaas05:54
*** ducttape_ has joined #openstack-lbaas06:15
*** ducttape_ has quit IRC06:21
*** chlong has joined #openstack-lbaas06:21
*** rudrajit has quit IRC06:26
*** rudrajit has joined #openstack-lbaas06:26
*** sticker has quit IRC06:26
*** rudrajit has quit IRC06:31
*** yamamoto has quit IRC06:31
*** rcernin has joined #openstack-lbaas06:41
*** fnaval has joined #openstack-lbaas06:44
*** tesseract- has joined #openstack-lbaas06:56
*** bana_k has quit IRC07:16
*** ducttape_ has joined #openstack-lbaas07:17
*** ducttape_ has quit IRC07:21
openstackgerritli,chen proposed openstack/neutron-lbaas: Update loadbalancer stats when receive stats streamed event  https://review.openstack.org/35765307:40
*** rtheis has joined #openstack-lbaas07:51
*** amotoki has joined #openstack-lbaas07:53
*** ihrachys has joined #openstack-lbaas07:53
openstackgerritReedip proposed openstack/neutron-lbaas: Add Timeout on Listeners  https://review.openstack.org/27389607:53
*** amotoki_ has joined #openstack-lbaas07:57
*** amotoki has quit IRC07:59
*** openstackgerrit has quit IRC08:03
*** openstackgerrit has joined #openstack-lbaas08:04
*** cody-somerville has joined #openstack-lbaas08:07
*** cody-somerville has quit IRC08:07
*** cody-somerville has joined #openstack-lbaas08:07
*** csomerville has quit IRC08:10
*** ducttape_ has joined #openstack-lbaas08:18
*** noshankus has joined #openstack-lbaas08:20
*** armax has joined #openstack-lbaas08:22
*** ducttape_ has quit IRC08:22
*** armax has quit IRC08:23
*** rtheis_ has joined #openstack-lbaas08:35
*** rtheis__ has joined #openstack-lbaas08:37
*** rtheis has quit IRC08:39
*** eandersson_ has joined #openstack-lbaas08:41
*** rtheis_ has quit IRC08:41
*** bcafarel has quit IRC08:47
*** bcafarel has joined #openstack-lbaas08:49
*** nmagnezi has quit IRC08:59
*** jsheeren has joined #openstack-lbaas09:00
*** chlong has quit IRC09:03
*** numans has quit IRC09:18
*** ducttape_ has joined #openstack-lbaas09:18
openstackgerritli,chen proposed openstack/octavia: Add a new API to list loadbalancer statistics  https://review.openstack.org/35626809:21
openstackgerritli,chen proposed openstack/neutron-lbaas: Update loadbalancer stats when receive stats streamed event  https://review.openstack.org/35765309:21
*** ducttape_ has quit IRC09:24
*** chenli has quit IRC09:28
*** numans has joined #openstack-lbaas09:55
openstackgerritThomas Bechtold proposed openstack/octavia: Remove oslo.rootwrap from requirements.txt  https://review.openstack.org/35773210:04
openstackgerritDoug Wiegley proposed openstack/neutron-lbaas: DO NOT MERGE - ci check  https://review.openstack.org/35773610:12
openstackgerritDoug Wiegley proposed openstack/octavia: DO NOT MERGE - ci check  https://review.openstack.org/35773710:13
*** ducttape_ has joined #openstack-lbaas10:19
*** ducttape_ has quit IRC10:24
*** ihrachys has quit IRC10:31
*** ihrachys has joined #openstack-lbaas10:40
*** yamamoto has joined #openstack-lbaas10:58
*** jsheeren has quit IRC11:03
gokhanhi team, I am trying to install octavia on ubuntu. I need your helps. therefore, I have questions about octavia. I installed octavia 0.8.0 with pip. so I see its setting file from /usr/local/etc/octavia/octavia.conf11:13
gokhando you any sample file for octavia.conf? Do I need creating octavia service and database ?11:14
gokhanI tried also installing on devstack I got an error like http://paste.openstack.org/show/561267/ keystone failed to get token11:17
*** ducttape_ has joined #openstack-lbaas11:20
openstackgerritYAMAMOTO Takashi proposed openstack/neutron-lbaas: Update disable_ssl_certificate_validation reference  https://review.openstack.org/35761911:22
*** ducttape_ has quit IRC11:25
*** jsheeren has joined #openstack-lbaas11:28
*** rtheis__ is now known as rtheis11:31
*** ihrachys has quit IRC11:38
gokhanisn't there anyone to help ?11:43
*** ihrachys has joined #openstack-lbaas12:04
*** yamamoto has quit IRC12:05
*** yamamoto has joined #openstack-lbaas12:06
*** yamamoto has quit IRC12:11
*** ducttape_ has joined #openstack-lbaas12:21
*** ducttape_ has quit IRC12:26
*** ducttape_ has joined #openstack-lbaas12:29
*** yamamoto has joined #openstack-lbaas12:36
*** ihrachys has quit IRC12:50
*** ihrachys has joined #openstack-lbaas12:50
*** diogogmt has joined #openstack-lbaas12:51
*** ducttape_ has quit IRC13:00
*** matt-borland has joined #openstack-lbaas13:00
*** nmagnezi has joined #openstack-lbaas13:06
*** BjoernT has joined #openstack-lbaas13:21
*** ducttape_ has joined #openstack-lbaas13:23
*** ducttape_ has quit IRC13:28
*** nmagnezi has quit IRC13:47
*** woodster_ has joined #openstack-lbaas13:53
*** ducttape_ has joined #openstack-lbaas13:53
*** gokhan has quit IRC13:54
*** catintheroof has joined #openstack-lbaas13:54
*** jsheeren has quit IRC13:56
*** catintheroof has quit IRC14:01
*** diogogmt has quit IRC14:03
*** diogogmt has joined #openstack-lbaas14:04
*** eandersson_ has quit IRC14:07
*** eandersson_ has joined #openstack-lbaas14:08
*** chlong has joined #openstack-lbaas14:13
*** ihrachys has quit IRC14:18
*** matt-borland has quit IRC14:20
*** ihrachys has joined #openstack-lbaas14:21
*** anilvenkata has quit IRC14:26
diltrammorning14:39
*** matt-borland has joined #openstack-lbaas14:54
*** hockeynut has joined #openstack-lbaas15:00
*** hockeynut has quit IRC15:02
*** rcernin has quit IRC15:24
*** pcaruana has quit IRC15:26
*** bana_k has joined #openstack-lbaas15:29
*** armax has joined #openstack-lbaas15:31
*** daneyon_ has quit IRC15:36
*** rtheis has quit IRC15:42
*** yamamoto has quit IRC15:47
*** links has quit IRC15:49
*** yamamoto has joined #openstack-lbaas15:50
*** rudrajit has joined #openstack-lbaas15:51
*** rtheis has joined #openstack-lbaas15:54
*** rtheis has quit IRC15:54
*** yamamoto has quit IRC15:59
*** yamamoto has joined #openstack-lbaas16:02
*** yamamoto has quit IRC16:08
*** ihrachys has quit IRC16:17
*** amotoki_ has quit IRC16:22
*** noshankus has quit IRC16:30
*** numans has quit IRC16:38
johnsomdiltram Morning.  How is that spec going?16:41
diltramjohnsom: hey, still working on16:41
diltrambefore summit will be ready16:42
diltramI found out that os-api-ref sphinx extension16:42
johnsomOk, just checking in16:42
diltramand I'm adding it16:42
diltramto our documentation16:42
diltramhttp://docs.openstack.org/developer/os-api-ref/16:42
*** krotscheck is now known as the_other_clarkb16:42
johnsomCool!  Yeah, there is an Oslo.confg extension to generate the config docs too that I want to switch over as well.16:43
*** the_other_clarkb is now known as krotscheck16:43
diltramyeah, we need to make all standarized16:45
diltramand even that os-api-ref is great because writing api is much easier that fighting with normal RST tables16:46
diltram:)16:46
*** tesseract- has quit IRC16:53
*** eezhova has quit IRC16:54
*** amotoki has joined #openstack-lbaas16:55
*** matt-borland has quit IRC16:56
*** kev0 has joined #openstack-lbaas17:05
*** fnaval has quit IRC17:08
*** fnaval has joined #openstack-lbaas17:08
*** bana_k has quit IRC17:10
*** bana_k has joined #openstack-lbaas17:33
*** BjoernT is now known as Bjoern_zZzZzZzZ17:35
*** kev0 has quit IRC17:38
*** kev0 has joined #openstack-lbaas17:39
*** yamamoto has joined #openstack-lbaas17:47
openstackgerritMichael Johnson proposed openstack/octavia: Fixes failover flow with namespace driver  https://review.openstack.org/31227017:48
*** yamamoto has quit IRC17:52
*** fnaval has quit IRC18:00
*** Bjoern_zZzZzZzZ is now known as BjoernT18:05
*** BjoernT is now known as Bjoern_zZzZzZzZ18:05
*** Bjoern_zZzZzZzZ is now known as BjoernT18:06
*** fnaval has joined #openstack-lbaas18:13
*** amotoki has quit IRC18:16
*** armax has quit IRC18:18
*** armax has joined #openstack-lbaas18:23
*** anilvenkata has joined #openstack-lbaas18:24
*** numans has joined #openstack-lbaas18:28
*** yamamoto has joined #openstack-lbaas18:32
*** armax has quit IRC18:34
*** Chenhao has joined #openstack-lbaas18:36
*** yamamoto has quit IRC18:38
*** amotoki has joined #openstack-lbaas18:47
*** amotoki has quit IRC18:50
*** amotoki has joined #openstack-lbaas18:56
openstackgerritBanashankar k proposed openstack/neutron-lbaas: validate that delay and timeout Make sure delay must be greater than or equal to timeout  https://review.openstack.org/35511518:57
ChenhaoHi guys, I found that in the amphora VM there is an "amphora-haproxy" namespace. But that namespace disappeared if I reboot the amphora VM.19:01
ChenhaoAnyone know when this namespace was created?19:02
bana_kChenhao https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/plug.py#L12719:03
*** amotoki has quit IRC19:04
ChenhaoOh~ Thanks very much bana_k :)19:07
rm_workChenhao: I think there should have been code in place to make that work properly19:12
rm_workChenhao: but if reboots are causing problems, this is possibly a known issue19:12
rm_workhttps://bugs.launchpad.net/octavia/+bug/151729019:12
openstackLaunchpad bug 1517290 in octavia "Not able to ssh to amphora or curl the vip after rebooting" [High,Incomplete]19:12
*** yamamoto has joined #openstack-lbaas19:18
ChenhaoHi rm_work, based on the code point by bana_k, I assume the namespace should be created when plug_vip() was called. Seems like there is a rest api call for plug_vip https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/server.py#L11119:21
rm_workyes, but on a reboot it should retain that configuration19:22
*** yamamoto has quit IRC19:22
ChenhaoIn side the amphora vm, I only saw the api was called once 2016-08-19 18:52:49.923 1374 INFO werkzeug [-] 192.168.254.3 - - [19/Aug/2016 18:52:49] "POST /0.5/plug/vip/10.0.0.48 HTTP/1.1" 202 -"19:22
bana_kChenhao and there is still an ongoing discussion about  do we need to reboot a LB VM.19:23
Chenhaoyeah~ That could be the situation :)19:23
rm_workChenhao: yes, it would only be called once by the controller at create time19:24
rm_workon reboot, it would not be called again -- but it SHOULD retain the existing network configuration -- if something else is happening, it might be something specific to your environment? do you have a network agent that could be rewriting the configurations on boot?19:24
rm_workat rackspace we ran into that problem somewhat19:24
ChenhaoI am trying to debugging Octavia with DVR. And try to see whether I can find the root cause of the problem why Octavia doesn't work in an DVR environment19:25
*** rudrajit has quit IRC19:26
ChenhaoI hit this problem during my debugging :)19:26
*** rudrajit has joined #openstack-lbaas19:26
ChenhaoThanks for the information rm_work bana_k , it is very helpful~ :)19:27
rm_workah yes, DVR :(19:29
*** rudrajit has quit IRC19:31
*** cody-somerville has quit IRC19:31
bana_kChenhao I think it was a prob with allowed address pairs in DVR19:31
*** yamamoto has joined #openstack-lbaas19:48
*** yamamoto has quit IRC19:52
*** rudrajit has joined #openstack-lbaas19:56
*** rudrajit has quit IRC20:05
*** sputnik13 has quit IRC20:05
*** rudrajit has joined #openstack-lbaas20:05
*** sputnik13 has joined #openstack-lbaas20:05
*** sputnik13 has quit IRC20:06
ChenhaoI saw that ticket that complaining about DVR doesn't support allow ip pairs. But when I test this blog http://blog.aaronorosen.com/?p=320, I can actually make the vrrp settings working for the fix ip address.20:08
*** rudrajit has quit IRC20:10
*** numans has quit IRC20:10
ChenhaoSo it makes feel like DVR can at least support some part of allow ip pairs. I just trying to see whether I can at least make the amphora working without floating ips.20:11
ChenhaoActually the problem I saw in my environment is "upstart/network-interface-eth1.log:Error getting hardware address for "eth1": No such device"20:13
ChenhaoIn devstack I saw "eth1.cfg" exist in '/etc/netns/amphora-haproxy/network/interfaces.d" in the amphora VM, but somehow in my DVR environment I saw "eth1.cfg" exist in "/etc/network/interfaces.d"20:18
ChenhaoThis is a little weird, because eth1 should be an interface inside the amphora-haproxy namespace, its configuration should under the namespace folder which is "etc/netns/amphora-haproxy/network/interfaces.d" just like devstack did20:22
*** anilvenkata has quit IRC20:29
*** yamamoto has joined #openstack-lbaas20:32
*** saju_m has joined #openstack-lbaas20:36
*** yamamoto has quit IRC20:37
openstackgerritMerged openstack/octavia: Updating network base utils for more functionality  https://review.openstack.org/35658820:40
openstackgerritMerged openstack/octavia: Updated from global requirements  https://review.openstack.org/35693420:45
*** bana_k has quit IRC20:57
*** bana_k has joined #openstack-lbaas20:58
*** yamamoto has joined #openstack-lbaas21:07
*** yamamoto has quit IRC21:11
johnsomchenhao Your reboot issue is fixed in https://review.openstack.org/#/c/312270/21:17
ChenhaoGreat~ Thanks very much johnsom :)21:17
johnsomchenhao I'm am pretty sure that that blog post about allowed address pairs is broken with DVR.  Putting a floating IP in front of the vip IP address will fail if DVR is enabled.21:19
johnsomOctavia should work on DVR networks, just DVR has a bug with floating IPS21:20
*** ducttape_ has quit IRC21:21
Chenhaojohnsom I haven't test the floating ip with that vrrp settings. What my test did is just verify the fix vip and mac address could be moved using that vrrp settings in an DVR environment. Also looking at some possible solutions to make DVR working with vrrp floating ips.21:23
ChenhaoSeems like we need to fix the vrrp floating ip for DVR first.21:24
johnsomYeah, I think just moving the IP back and forth, kind of works with DVR.21:24
*** bana_k has quit IRC21:28
ChenhaoFor now I think the only mechanism I could come up is like launch two normal VMs which don't use VRRP and try to set an aodh alarm monitoring the amphora VMs if the first amphora VMs dead move the floating ip to the second VM. This could be done automatically by using some heat template.21:29
johnsomYeah, but that is horribly slow.  With VRRP and DVR disabled we failover in about a second.21:30
ChenhaoI agree.21:30
ChenhaoAnyway it seems like a long way to go, base on the architecture of DVR it won't be an easy fix.21:32
johnsomMy off the head answer was (back when were were talking about this) DVR should light up the flip on all instances that have the allowed address pair.  It should then have something watching for the GARPs that updates DVR's static arp table.  Much like normal switches do21:32
johnsomThere are probably reasons that won't work as I haven't done a deep dive on the DVR implementation in a long time.21:34
*** nmagnezi has joined #openstack-lbaas21:34
ChenhaoI need to digging more before I can say whether it will work or not...21:35
ChenhaoAnother question, anyone notice we have a race condition here? https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/plug.py#L13321:35
johnsomDo you mean if the interface has already been moved?21:36
Chenhaowhat will happen if the link is not ready when we try to make this call?21:37
ChenhaoI mean the interface hasn't been created21:37
johnsomBut it is created in a previous step.  This is the "post VIP plug" method that is called after it has been created21:38
ChenhaoNot sure whether it is only in my environment, I saw this error "./kern.log:Aug 19 18:52:45 amphora-e11d8b3b-a3c6-421d-9936-d7cba58073f6 kernel: [    4.987287] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready"21:39
Chenhaoand eth1 was not moved to the amphora-haproxy namespace.21:39
johnsomYeah, that is a bogus error.  It's there because I don't delete the eth1.cfg out of the global network namespace21:39
Chenhaohmm... Can you point me the place where the interface was created?21:41
johnsomChenhao I would test with the failover patch I sent you.  There was a numbering issue during failovers that could cause the eth1 to not move correctly, but otherwise I haven't seen a problem there.  I have recently run a lot of tests for the failover patch.21:41
ChenhaoSounds cool~ :)21:42
johnsomThis task: https://github.com/openstack/octavia/blob/master/octavia/controller/worker/tasks/network_tasks.py#L26321:42
ChenhaoThanks very much johnsom, let me try to learn that first :)21:43
johnsomOnce that task completes nova will have attached the network interface and the kernel will see the hot-plug.  eth1 with be present in the kernel at that point.  Not configured, but present.21:43
*** diogogmt has quit IRC21:48
*** ducttape_ has joined #openstack-lbaas21:57
*** BjoernT has quit IRC21:59
*** bana_k has joined #openstack-lbaas22:02
*** ducttape_ has quit IRC22:41
*** ducttape_ has joined #openstack-lbaas22:45
*** ducttape_ has quit IRC22:50
*** nmagnezi has quit IRC22:54
*** saju_m has quit IRC22:58
*** Chenhao has quit IRC23:02
*** armax has joined #openstack-lbaas23:05
*** amotoki has joined #openstack-lbaas23:09
*** ducttape_ has joined #openstack-lbaas23:21
*** amotoki has quit IRC23:23
*** yamamoto has joined #openstack-lbaas23:53

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!