Wednesday, 2015-04-08

*** xgerman has quit IRC00:08
madhu_akblogan, dougwig, fnaval: I get created a devstack setup on my new vm, after pulling the right tempest version using :1f88ecec48dd89cc101fdf30458ed177bd7f20ef, when I run the tests "tox -v -epy27", I could see bunch of errors : "from neutron.tests.unit import test_api_v200:13
madhu_akImportError: cannot import name test_api_v2" and "from neutron.tests.unit.openvswitch import test_agent_scheduler00:13
madhu_akImportError: No module named openvswitch00:13
madhu_akThe test run didn't actually run any tests00:13
madhu_akERROR: InvocationError: '/bin/sh tools/ '"00:13
madhu_akexact error can be seen:
dougwigmadhu_ak: the fix for that is in the gate right now00:29
madhu_akthanks for confirming dougwig. could you let me know once that fix is merged?00:32
*** madhu_ak has quit IRC01:22
*** sbfox has joined #openstack-lbaas01:31
*** vivek-eb_ has quit IRC01:44
*** sbfox has quit IRC01:51
*** mugu has quit IRC02:01
*** Varun_Lodaya has quit IRC02:06
*** mugu has joined #openstack-lbaas02:15
*** vivek-ebay has joined #openstack-lbaas02:17
openstackgerritMerged openstack/neutron-lbaas: Rename imports based on neutron tests reorganization.
dougwigmadhu_ak: ^^02:22
*** sbfox has joined #openstack-lbaas02:26
*** mwang2 has quit IRC02:32
*** fnaval has quit IRC02:47
*** Varun_Lodaya has joined #openstack-lbaas02:54
*** fnaval has joined #openstack-lbaas02:54
*** sbfox has quit IRC03:04
*** vivek-ebay has quit IRC03:06
*** xgerman has joined #openstack-lbaas03:16
*** xgerman has quit IRC03:18
*** vivek-ebay has joined #openstack-lbaas03:19
*** vivek-ebay has quit IRC03:30
*** vivek-ebay has joined #openstack-lbaas03:31
*** madhu_ak has joined #openstack-lbaas03:33
*** xgerman has joined #openstack-lbaas03:43
*** xgerman has quit IRC03:54
*** vivek-ebay has quit IRC04:00
*** rm_you| has joined #openstack-lbaas04:02
*** rm_you has quit IRC04:05
*** woodster_ has quit IRC04:20
*** sbfox has joined #openstack-lbaas04:35
*** woodster_ has joined #openstack-lbaas04:46
*** xgerman has joined #openstack-lbaas04:48
*** Varun_Lodaya has quit IRC04:55
*** TrevorV|Home has joined #openstack-lbaas04:57
*** xgerman has quit IRC05:03
*** TrevorV|Home has quit IRC05:05
*** fnaval has quit IRC05:06
*** madhu_ak has quit IRC05:07
*** Varun_Lodaya has joined #openstack-lbaas05:10
*** Varun_Lodaya has quit IRC05:41
*** apuimedo has joined #openstack-lbaas06:07
*** kobis has joined #openstack-lbaas06:21
*** kiran_ has joined #openstack-lbaas06:22
*** _kiran_ has joined #openstack-lbaas06:22
*** _kiran_ has quit IRC06:22
*** kiran_ is now known as kiran-r06:26
*** kobis has quit IRC06:28
*** kobis has joined #openstack-lbaas07:09
*** sbfox has quit IRC07:12
openstackgerritMerged openstack/neutron-lbaas: Updated from global requirements
*** amotoki has joined #openstack-lbaas07:50
*** woodster_ has quit IRC07:50
*** Tiancheng has joined #openstack-lbaas08:27
*** mugu has quit IRC09:03
*** mugu has joined #openstack-lbaas09:06
openstackgerritSantosh Sharma proposed openstack/neutron-lbaas: changes to skip negative test cases with invalid/empty tenant_id
*** rm_work has quit IRC10:04
*** rm_work has joined #openstack-lbaas10:06
*** rm_work has quit IRC10:06
*** rm_work has joined #openstack-lbaas10:06
openstackgerritenikanorov proposed openstack/neutron-lbaas: Use id attribute of the object in assert_modifications_allowed
*** f13o has joined #openstack-lbaas10:52
*** Tiancheng has quit IRC12:02
openstackgerritSantosh Sharma proposed openstack/neutron-lbaas: changes to skip negative test cases with invalid/empty tenant_id
*** woodster_ has joined #openstack-lbaas12:14
*** apuimedo has quit IRC13:50
*** chlong has quit IRC13:58
*** vivek-ebay has joined #openstack-lbaas14:24
*** kiran-r has quit IRC14:27
*** vivek-eb_ has joined #openstack-lbaas14:34
*** vivek-ebay has quit IRC14:36
*** Varun_Lodaya has joined #openstack-lbaas14:59
*** xgerman has joined #openstack-lbaas15:07
*** xgerman has quit IRC15:09
*** xgerman has joined #openstack-lbaas15:10
*** vivek-eb_ has quit IRC15:23
openstackgerritTrevor Vardeman proposed stackforge/octavia: Amphora SSH Driver
*** kiran has joined #openstack-lbaas15:58
*** kiran has quit IRC16:12
*** madhu_ak has joined #openstack-lbaas16:14
*** kiran has joined #openstack-lbaas16:17
*** Varun_Lodaya has quit IRC16:22
*** kiran has quit IRC16:25
*** kiran has joined #openstack-lbaas16:25
*** kiran has quit IRC16:30
*** mlavalle has joined #openstack-lbaas16:36
*** bharath has joined #openstack-lbaas16:37
*** enikanorov_ has quit IRC16:42
*** mwang2 has joined #openstack-lbaas16:43
*** kobis has quit IRC16:47
openstackgerritAl Miller proposed stackforge/octavia: Add devstack plugin for octavia
*** amotoki has quit IRC17:20
*** vivek-ebay has joined #openstack-lbaas17:27
*** bharath has quit IRC17:31
*** bharath has joined #openstack-lbaas17:31
dougwigmorning all.  do we want to plan a summer mid-cycle for lbaas/octavia coding?17:39
*** Varun_Lodaya has joined #openstack-lbaas17:42
xgermanI would aslo like to invite the VPN and FwaaS folks - since they probbaly don't have enough mass -- unless they are doing the Neutron midcycle17:43
xgermanI can get probably HP to host in Seattle17:44
xgermanput it on the agenda for today17:47
*** Aish has joined #openstack-lbaas18:04
*** mwang2 has quit IRC18:05
*** mwang2 has joined #openstack-lbaas18:06
sballexgerman: sounds like a good plan18:18
sballeboth inviting FW and VPN and hosting it in Seattle18:18
Santosh_NSI would like to go through all docs/desgins for octavia . Starting with
Santosh_NSIf there are other pointers plz share18:20
openstackgerritMerged openstack/neutron-lbaas: changes to skip negative test cases with invalid/empty tenant_id
*** SumitNaiksatam has joined #openstack-lbaas18:22
xgermanSantosh_NS those are our main documents but we haven't done a good job with keeping up on documentation18:27
*** vivek-ebay has quit IRC18:31
openstackgerritmin wang proposed openstack/neutron-lbaas: Introduced tempest API tests for openstack/neutron-lbaas
openstackgerritMerged stackforge/octavia: Adds rise threshold to configuration generation
*** sbalukoff has joined #openstack-lbaas18:54
*** vivek-ebay has joined #openstack-lbaas18:55
*** vivek-ebay has quit IRC18:57
openstackgerritMerged stackforge/octavia: Updates Jinja templater with additional optional params
*** fnaval has joined #openstack-lbaas19:23
openstackgerritFranklin Naval proposed openstack/neutron-lbaas: Neutron LBaaS: Load Balancer Basic Scenario Test
openstackgerritFranklin Naval proposed openstack/neutron-lbaas: Neutron_LBaaS: Update README for Tests
*** madhu_ak has quit IRC19:31
*** enikanorov has joined #openstack-lbaas19:51
*** xgerman has quit IRC19:53
*** xgerman has joined #openstack-lbaas19:54
*** jorgem has joined #openstack-lbaas19:58
openstackgerritFranklin Naval proposed openstack/neutron-lbaas: Neutron_LBaaS: Update README for Tests
openstackgerritFranklin Naval proposed openstack/neutron-lbaas: WIP: Neutron LBaaS: TLS Barbican Scenario Test
*** madhu_ak has joined #openstack-lbaas20:05
*** vivek-ebay has joined #openstack-lbaas20:16
*** ptoohill-oo has joined #openstack-lbaas20:28
*** crc32 has joined #openstack-lbaas20:30
*** ptoohill-oo has quit IRC20:35
*** apuimedo has joined #openstack-lbaas20:43
*** sbfox has joined #openstack-lbaas20:47
*** vivek-ebay has quit IRC20:51
*** sbfox has quit IRC21:00
johnsomblogan, you add it, I will call it.  Are you going to update the spec and noop driver?21:02
johnsomAfter you push this through of course:
bloganyeah, im goign to make a new review for it too21:03
johnsomSounds good.  Was there another topic?21:04
bloganwhoops, i missed that, thanks for covering21:04
bloganjohnsom: yeah, so the plug_vip method in the network driver is going to return a list of data models call AmpGroups (Please come up with a better name for them)21:05
bloganjohnsom: basically it will contain the amphora_id, the bind_ip, and the ha_ip21:05
bloganthe bind_ip will be the ip that the amphora will be listening on, while the ha_ip will be the ip that something like keepalived would use as a virtual ip21:05
bloganthe ips can both be the same if the network driver decides that21:06
openstackgerritMerged stackforge/octavia: Adding post_network_plug to the noop amphora driver
bloganthere just has to be a way to tell the amphora what ip it's haproxy should be listening on, and what ha_ip it should use if it needs that, so this is what i came up with21:07
xgermanI think we can dynamically reconfigure keepalived - so no need for an ha ip?21:08
bloganbut the controller worker would take that list of AmpGroups and insert the bind_ip and ha_ip into the db for each amphora21:08
johnsomOk, so I would take that and send it down to the amp in post vip right?21:08
bloganxgerman: not for all network driver implementatiosn, for instance the allowed address pairs driver ive worked on allocates antoher ip for that, there's no way to dynamically figure out which ip that is21:09
xgermanJust trying to wrap my head around that:21:10
bloganjohnsom: once you insert it and get a refresh load_balancer tree, you should just pass the load_balancer to the post_vip as it will probably need all the amphorae and the vip information21:10
openstackgerritMerged openstack/neutron-lbaas: Use id attribute of the object in assert_modifications_allowed
bloganxgerman: read this blog post21:11
* xgerman reading21:12
bloganxgerman: its essentially what the network driver will do, with a few minor changes21:12
bloganwell the driver i've written21:13
bloganthe driver we will use internally probably wont' even use the ha_ip field, even with an active active topology21:14
dougwigi love vrrp of nova instances, with no provision for ensuring separate compute nodes.21:15
johnsomdougwig yeah, there is still scheduler work to do21:15
bloganbut that driver will still work with these data structures and workflow, and so will the allowed-address-pairs driver, and from what i remember will handle the network architecure yall mapped out in seattle, along with bluebox'es21:15
bloganbabysteps dougwig21:16
bloganif yall could kind of map this out for what yall plan on doing with your implementation, and see if it will work for yall that'd be great, if it doesn't more work will need to be done to make sure it works for all of us21:18
*** SumitNaiksatam has quit IRC21:21
johnsomblogan Does this work with dvr?21:22
johnsomblogan also, can we keep the option open to use unicast for the vrrp?21:25
johnsomvrrp_unicast_bind   # Internal IP of this machine21:26
johnsomvrrp_unicast_peer   # Internal IP of peer21:26
xgermanunicast ++ -- I don't want to set up a subnet for each haproxy cluster21:26
xgermanif I can help it ;-)21:26
johnsomI have heard rumblings of multicast issues in openstack deployments, so it might be best avoided if we can.21:28
xgerman+1 (we share the same source)21:31
*** jorgem has quit IRC21:31
openstackgerritCarlos Garza proposed stackforge/octavia: Implementation for bp/api-to-oslo-messing-handler
openstackgerritCarlos Garza proposed stackforge/octavia: Implementation for bp/api-to-oslo-messing-handler
*** vivek-ebay has joined #openstack-lbaas21:51
*** vivek-ebay has quit IRC21:53
*** vivek-ebay has joined #openstack-lbaas21:53
*** amotoki has joined #openstack-lbaas21:54
*** jorgem has joined #openstack-lbaas22:00
*** chlong has joined #openstack-lbaas22:01
bloganjohnsom: those ips would essentially be all the amphora ips correct?22:01
bloganwell all the amphora bind_ips22:01
openstackgerritmin wang proposed stackforge/octavia: Add health check  amphora-driver-interface
xgermanblogan, yes22:02
johnsomThe vrrp_unicast ips?  They would be the HA IPs right?22:02
bloganthe bind ip being the ip that haproxy would be listening on22:03
xgermanisn't ha_ip the one we trade around?22:03
bloganxgerman: yeah22:03
bloganthe ha_ip would be the virtual ip basically, but not necessarily the vip the user sees22:04
blogandepends on the driver implementation22:04
openstackgerritMadhusudhan Kandadai proposed openstack/neutron-lbaas: Introduced admin_api tempest tests
johnsomNow I am confused.  If the HA IP is the one that floats between the Master/Slave AMP that is the address HAProxy will bind to right?22:06
bloganjohnsom: in some implemetnations, yes, in others no22:06
johnsomIt's a non-arp lo address22:06
xgermanlet's talk about the one for devstack22:06
bloganso the one for devstack, the bind_ip would be in the haproxy config as the listen ip22:07
xgermanand the ha_ip would be the same?22:07
bloganthe ha_ip would be the keepalvied conf22:07
bloganno they'd be different22:07
johnsomblogan assuming no vrrp?22:07
bloganassuming no vrrp, ha_ip deosn't really matter, unless you're donig some kind of active active topology that might need that ha_ip, which i dont know of any examples22:08
xgermanok, so bind_ip is the one we are trading22:08
bloganxgerman: what do you mean by trading?22:09
xgermanif haproxy1 fails haproxy2 will grab22:09
blogannope, thats the ha_ip22:09
bloganwell it doesn't grab it22:09
bloganin this network driver22:10
johnsomBorrowing this:
johnsomhaproxy will listen on on both master and slave22:11
xgermanyep, that was my thought, too22:11
johnsomtypically it is on a non-arping lo interface22:11
johnsom141 and 142 are used for vrrp communication22:11
xgerman(and potentially for our management network if wqe want to save a subnet)22:12
johnsomand outbound to the members22:12
xgermanbind_ip is the one outbound to the members?22:12
johnsomThat would be my thought, but I think blogan has a different thought22:13
xgermanhe usually has the same thought but we misunderstand him in the beginning :-)22:13
johnsomI guess you could do nonlocal bind and skip the lo interface22:14
xgermanwell, I also like the ha_ip be in the tenant network22:15
bloganxgerman: ha in this case i don't think we have the same thought22:15
xgermanok, explain22:15
fnavalhi - is the jenkins gate broken22:15
dougwigfnaval: is it a day ending in 'y' ?22:15
fnavalin english, yes22:16
bloganso in the allowed address pairs example, the ha_ip would not be an interface at all on teh amphora, just an entry in the keepalived conf as the virtual_ipaddress22:16
bloganthe bind_ip would have an interface, and be different for each amphora, and that is what haproxy would listen on22:17
bloganso in that image 141 and 142 would be the bind_ips and 140 the ha_ip22:17
bloganbut that doesn't mean another network driver can't implement it antoher way22:18
johnsomThat works?  I know you can get away with ip_nonlocal_bind, but I thought the listener had to have the IP 14022:18
xgermanyep + from the blog you cited the ha_ip needs to be bound to some port on one of the josts22:18
bloganso i have tested out the allowed-address-pairs, and it does work, i'm not sure of all the mechanics behind it, but i believe it just sets firewall rules to allow traffic destined for the ip specified in the allowed-address-pair port attribute, to go to the fixed ip22:19
xgermanyeah, I believe that22:20
bloganxgerman: actually the ha_ip doesn't even need to be bound to a port, he's jsut doing it so that ip address is reserved and won't be allocated in the future22:20
xgermanbut haproxy needs to listen on some ip?22:20
Aishblogan: again to confirm… As a demo tenant, should we be able to create a listener with an empty/invalid tenant id?22:21
bloganyeah the fixed_ip for each of the ports that the nova-interface-attach call created22:21
bloganAish: if by empty you mean providing an empty string then no it probably won't bc it will think you're tyring to add something under a tenant you dont have permissions to do so, but thats going through keystonemiddleware to do i believe22:23
xgermanok, I think I understand22:24
xgermanhaproxy listens on and repectively22:25
xgerman10.0.0.201   get's added as an allowed address to both22:25
bloganand like i said before this is not meant for any kind of production environment, just for the demo to get it out quickly, and if we want to show HA then we have that opportunity22:26
bloganotherwise it will still work with only one amphora22:26
xgermanok, we can definitely build on that but in our cloud it's hard to say what works without actually truing it ;-)22:27
bloganxgerman: lol yeah we dont even have an environment to test our custom network driver out yet so we dont even know if it will work yet22:28
xgermanI will bring my shirt "works in devstack" :-)22:29
bloganso the case of haproxy binding to the ha_ip and the amphora using the bind_ip woudl require a different amphora driver than the ssh driver22:29
bloganand different network driver too probably22:30
xgermanwell, I think what you did is fine22:30
bloganwould still work, but i was hoping just a new network driver woudl be needed, not having to change the amp driver22:30
xgermanpractically customers care more about from which address the packages come to their members22:31
Aishblogan: ok then, I think the existing tests are correct.22:31
AishThe failure occurs because while running tox -e tempest,  it is actually creating the listener even when we give empty/invalid tenant id.22:31
AishAnd, this newly created listener is not being deleted. That is why the subsequent tests fails with the “already has a listener with protocol_port of 80”.  While running nosetests, there wasn’t any problem with the existing tests.22:31
AishI am not sure why this happens though.22:32
bloganAish: that is odd22:32
Aishblogan: yes..22:33
bloganAish: that is under the demo user?22:33
Aishyes blogan22:33
bloganAish: it is actually allowing the creation of a listener with an empty string tenant_id?22:33
Aishnosetests doesn’t. but seems like tempest allows it22:34
Aishi mean tox -e tempest22:34
Aishblogan: ^^22:34
bloganAish: i bet when you run nose its using the /etc/tempest/tempest.conf which may have a different user, and tox -e tempest uses the octavia/tests/tempest/etc/tempest.conf22:35
Aishblogan: octavia/tests….??22:36
Aishi ll take a look22:36
bloganAish: yeah running tox -e tempest uses the octavia/tests/tempest/etc/tempest.conf config file22:37
Aishi ll check that blogan22:38
Aishblogan  from the tox.ini I oculd see that it uses /neutron_lbaas/tests/tempest/etc/tempest.conf22:49
Aishand it is using admin credentials..22:50
bloganand it passes with tox, and fails with nose right?22:50
bloganerr, it depends on what pass and fail is22:51
Aishexisting tests fails with tox and passes with nosetests as nosetests points to demo user22:51
bloganso it passes bc it expects a failure22:51
bloganis nosetests indeed using the /etc/tempest/tempest.conf?22:52
Aishyes. the test is actually for demo user.. and it is expected to pass. But, since the config file used by tox has admin credentials, it lets u create a listener with empty/invalid tenant id and that is not being deleted …22:52
Aishhence the subsequent tests faild22:53
Aishblogan: ^^22:54
bloganAish: that config file should probably be changed to use the demo user then22:54
bloganAish: that one is my fault22:54
*** jorgem has quit IRC22:54
*** sbfox has joined #openstack-lbaas22:54
Aishok. i can do that.. But, when we have to test for admin user, how do we do it then?22:54
bloganAish: that i am unsure, i feel like tempest has config values for admin_users to tailor tests for in the same test suite, but im unsure22:55
blogani am no tempest expert22:55
Aishhmmm.. .Okay.. So for now, can i change the config file to have demo user22:56
bloganAish: yes please22:56
Aishokay blogan Thnk you,,22:57
bloganAish: thank you22:57
xgermanso back to logging23:02
xgermanI am still trying to wrap my head around it23:02
xgermanwe fire up one haproxy per listener so for me to get logs they need to have different identifiers23:03
xgermanaka log    <identifier>  notice23:03
xgermanhow does that chime with the jinja engine?23:03
*** chlong has quit IRC23:05
*** mlavalle has quit IRC23:08
xgermanI guess foosball23:09
*** sbfox has quit IRC23:09
bloganxgerman: lol no he's gone, he'll be back later im sure23:13
xgermanyeah, I think we want logging to be like /var/log/haproxy-<listener-id>.log23:13
bloganxgerman: its 6pm bro! you west coasters live in the past23:13
xgermanjust work on WestCoast time ;-)23:14
xgermangive you nice long mornings23:14
bloganand then leave on east coast time?23:14
xgermansquare that with jorgem23:15
bloganxgerman: is there a review where you're having this discussion with ptoohill?23:15
bloganjorgem is just a puppet23:15
xgermansoon there will be ;-)23:15
bloganxgerman: ah okay, is it currently storing the logs in one file for all haproxy processes?23:16
openstackgerritGerman Eichberger proposed stackforge/octavia: Adds a log identifier to the template
xgerman^^^ blogan23:19
openstackgerritTrevor Vardeman proposed stackforge/octavia: Amphora SSH Driver
bloganxgerman: that just allows the location to be overridden in the template right?23:21
xgermanno, that allows to add an identifier23:21
xgermanthen we need to make some rsyslogfile23:21
bloganwell before it was always local023:22
*** vivek-ebay has quit IRC23:22
bloganso now its just defaults to it, but the driver should override it23:22
bloganfor each listener23:22
xgermanabd then according to
xgermanwe need some syslogd file telling the computer that23:23
xgermanlocal1.* -/var/log/haproxy_1.log23:23
xgermanaka identifier -> different file23:23
bloganill need to read up on that23:24
xgermanyeah, caught me by surprise, too23:25
bloganbut you need tests :)23:25
bloganor modify the test23:25
xgermanthey passed23:25
xgermanbut feel free to -123:25
blogana test that tests the identifier gets rendered correctly23:25
bloganit deafults to local0 though, so at that point you'd kind of just be testing python default arg overridden23:26
xgermanwell it will also render23:26
xgermanso I felt it would be ok-ish the way it is23:27
bloganyeah but i wast hinking a test to make sure you passing in a different value for identifier would end up in an haproxy with that identifier, but the fact that it defaults to 'local0' and the tests already assume 'local0' and they pass, that tests it23:27
bloganso i retract my test comment23:27
*** amotoki has quit IRC23:29
blogangonna give ptoohill a chance to review it before i +2, but I +1'ed it23:29
blogani gotta go now23:30
*** sbfox has joined #openstack-lbaas23:32
openstackgerritMerged stackforge/octavia: Add health check  amphora-driver-interface
*** sbfox has quit IRC23:43
*** apuimedo has quit IRC23:44
*** sbfox has joined #openstack-lbaas23:46
ptoohillxgerman, good catch on that. When i updated for defaults i must have overwrote it and didnt notice. Though, i do have a comment on this that im about to make23:54
xgermanyeah, I am still wrapping my head around this logging thing23:54
ptoohillagreed, i need to look into it more if we want individual files for each listener23:57
ptoohillbut how it is now:23:57
ptoohillthe log0 line will log all http requests to the defined file, and local1 all haproxy (server) specific logs to the defined file23:58
ptoohillim sure theres a way to specidify per listener if thats what we want to accomplish. Again, keep in mind this was a cp/tweak from neutron so there are probably more gotchas that we want done23:59

Generated by 2.14.0 by Marius Gedminas - find it at!