| johnsom | Cores, can you guys review this: https://review.openstack.org/#/c/379001/ It's a bit of a chicken/egg with our gates/cross-repo dependencies. To see that it worked, look at https://review.openstack.org/#/c/379000/ which depends on it. | 00:03 |
|---|---|---|
| *** bana_k has quit IRC | 00:08 | |
| *** amotoki has joined #openstack-lbaas | 00:08 | |
| *** ducttape_ has joined #openstack-lbaas | 00:08 | |
| *** sticker has quit IRC | 00:16 | |
| johnsom | That should fix the scenario test gate issues with get-pip.py | 00:19 |
| *** ducttape_ has quit IRC | 00:20 | |
| *** amotoki_ has joined #openstack-lbaas | 00:30 | |
| *** amotoki has quit IRC | 00:34 | |
| *** sticker has joined #openstack-lbaas | 00:39 | |
| *** yamamoto has joined #openstack-lbaas | 00:52 | |
| *** ducttape_ has joined #openstack-lbaas | 01:05 | |
| *** ducnc has joined #openstack-lbaas | 01:23 | |
| *** ducttape_ has quit IRC | 01:27 | |
| *** ducttape_ has joined #openstack-lbaas | 01:36 | |
| *** ducttape_ has quit IRC | 01:50 | |
| *** yuanying has quit IRC | 02:47 | |
| *** ducttape_ has joined #openstack-lbaas | 02:51 | |
| *** bdeschenes has quit IRC | 02:51 | |
| openstackgerrit | He Qing proposed openstack/octavia: Remove dumplicated config option 'cert_generator' https://review.openstack.org/379903 | 02:54 |
| *** yuanying has joined #openstack-lbaas | 02:55 | |
| *** ducttape_ has quit IRC | 02:56 | |
| *** amotoki_ has quit IRC | 03:15 | |
| *** yuanying has quit IRC | 03:38 | |
| *** yuanying has joined #openstack-lbaas | 03:49 | |
| *** ducttape_ has joined #openstack-lbaas | 03:52 | |
| *** amotoki has joined #openstack-lbaas | 03:53 | |
| *** amotoki has quit IRC | 03:57 | |
| *** ducttape_ has quit IRC | 03:57 | |
| *** links has joined #openstack-lbaas | 03:58 | |
| *** amotoki has joined #openstack-lbaas | 04:10 | |
| *** portdirect_ has joined #openstack-lbaas | 04:21 | |
| *** portdirect has quit IRC | 04:21 | |
| *** portdirect_ is now known as portdirect | 04:22 | |
| openstackgerrit | Merged openstack/neutron-lbaas: Use a cached get-pip.py if it is availble https://review.openstack.org/379001 | 04:38 |
| *** ducttape_ has joined #openstack-lbaas | 04:53 | |
| *** ducttape_ has quit IRC | 04:58 | |
| rm_work | heh i'm independent now so I can merge stuff as a second reviewer to RAX :P | 05:12 |
| openstackgerrit | Merged openstack/octavia: Use a cached get-pip.py if it is availble https://review.openstack.org/379000 | 05:39 |
| johnsom | Bonus | 05:55 |
| *** amotoki has quit IRC | 06:01 | |
| *** rcernin has joined #openstack-lbaas | 06:07 | |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 06:11 |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 06:11 |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 06:13 |
| ptoohill | heh | 06:13 |
| *** anilvenkata has joined #openstack-lbaas | 06:28 | |
| *** sticker has quit IRC | 06:30 | |
| *** pcaruana has joined #openstack-lbaas | 06:39 | |
| *** amotoki has joined #openstack-lbaas | 06:51 | |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 06:54 |
| *** amotoki has quit IRC | 07:04 | |
| *** ducnc has quit IRC | 07:15 | |
| *** ihrachys has joined #openstack-lbaas | 07:16 | |
| *** ducnc has joined #openstack-lbaas | 07:17 | |
| *** coolias has joined #openstack-lbaas | 07:32 | |
| *** amotoki has joined #openstack-lbaas | 07:46 | |
| *** coolias has quit IRC | 07:49 | |
| *** ihrachys has quit IRC | 07:54 | |
| *** bdeschenes has joined #openstack-lbaas | 08:10 | |
| *** coolias has joined #openstack-lbaas | 08:14 | |
| *** amotoki_ has joined #openstack-lbaas | 08:27 | |
| *** amotoki has quit IRC | 08:29 | |
| *** ihrachys has joined #openstack-lbaas | 08:38 | |
| openstackgerrit | Stephen Balukoff proposed openstack/octavia: Add support for PKCS7 bundles and encrypted keys https://review.openstack.org/380020 | 08:49 |
| openstackgerrit | Stephen Balukoff proposed openstack/octavia: Add support for PKCS7 bundles and encrypted keys https://review.openstack.org/380020 | 08:53 |
| *** bdeschenes has quit IRC | 08:57 | |
| openstackgerrit | Stephen Balukoff proposed openstack/octavia: Add support for PKCS7 bundles and encrypted keys https://review.openstack.org/380020 | 09:05 |
| openstackgerrit | Stephen Balukoff proposed openstack/octavia: Add support for PKCS7 bundles and encrypted keys https://review.openstack.org/380020 | 09:11 |
| *** eezhova has joined #openstack-lbaas | 09:17 | |
| *** carrbs has quit IRC | 09:26 | |
| *** ducnc1 has joined #openstack-lbaas | 09:53 | |
| *** ducnc has quit IRC | 09:55 | |
| *** ducnc1 is now known as ducnc | 09:56 | |
| *** ducnc1 has joined #openstack-lbaas | 10:18 | |
| *** ducnc has quit IRC | 10:18 | |
| *** ducnc1 is now known as ducnc | 10:18 | |
| *** yamamoto has quit IRC | 10:19 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/neutron-lbaas: Updated from global requirements https://review.openstack.org/378853 | 10:20 |
| *** coolias has quit IRC | 10:23 | |
| *** coolias has joined #openstack-lbaas | 10:24 | |
| *** fnaval has joined #openstack-lbaas | 10:45 | |
| *** coolias has quit IRC | 10:55 | |
| *** ducttape_ has joined #openstack-lbaas | 10:58 | |
| *** ducttape_ has quit IRC | 11:02 | |
| *** nagyz has quit IRC | 11:10 | |
| *** nagyz has joined #openstack-lbaas | 11:11 | |
| *** gcheresh_ has joined #openstack-lbaas | 11:20 | |
| *** nmagnezi has joined #openstack-lbaas | 11:32 | |
| *** anilvenkata has quit IRC | 11:34 | |
| *** yamamoto has joined #openstack-lbaas | 11:34 | |
| *** yamamoto_ has joined #openstack-lbaas | 11:36 | |
| *** yamamoto has quit IRC | 11:40 | |
| *** numans has joined #openstack-lbaas | 11:41 | |
| *** nmagnezi has quit IRC | 11:45 | |
| *** Kiall has joined #openstack-lbaas | 11:54 | |
| rm_work | sbalukoff: nits on your PKCS7 patch and then it'll be +2 from me | 11:57 |
| *** ducttape_ has joined #openstack-lbaas | 11:58 | |
| *** ipsecguy has quit IRC | 12:02 | |
| *** ipsecguy has joined #openstack-lbaas | 12:03 | |
| *** ducttape_ has quit IRC | 12:03 | |
| *** ducttape_ has joined #openstack-lbaas | 12:12 | |
| ajo_ | ptoohill, I've noticed that scenario test doesn't work on DVR (floating ip stays marked as "DOWN") may be it's just my env | 12:13 |
| *** amoralej is now known as amoralej|lunch | 12:14 | |
| *** gcheresh_ has quit IRC | 12:22 | |
| *** yamamoto_ has quit IRC | 12:26 | |
| *** fnaval has quit IRC | 12:27 | |
| *** ducttape_ has quit IRC | 12:27 | |
| *** fnaval has joined #openstack-lbaas | 12:39 | |
| openstackgerrit | Monty Taylor proposed openstack/octavia: Use get-pip.py from cache if it exists https://review.openstack.org/380255 | 12:42 |
| rm_work | huh... thought https://review.openstack.org/#/c/379000/ already did that, but I guess not | 12:48 |
| openstackgerrit | Monty Taylor proposed openstack/octavia: Use get-pip.py from cache if it exists https://review.openstack.org/380255 | 12:51 |
| *** matt-borland has joined #openstack-lbaas | 12:52 | |
| openstackgerrit | Merged openstack/neutron-lbaas: Updated from global requirements https://review.openstack.org/378853 | 12:52 |
| *** yamamoto has joined #openstack-lbaas | 12:56 | |
| *** links has quit IRC | 12:56 | |
| *** yamamoto has quit IRC | 13:01 | |
| *** yamamoto has joined #openstack-lbaas | 13:03 | |
| *** yamamoto has quit IRC | 13:03 | |
| *** yamamoto has joined #openstack-lbaas | 13:10 | |
| *** yamamoto has quit IRC | 13:15 | |
| johnsom | I did already fix that | 13:19 |
| johnsom | mugsie around? | 13:20 |
| mugsie | yo | 13:20 |
| johnsom | I was thinking about your security group issue | 13:20 |
| mugsie | Oh, cool | 13:21 |
| mugsie | been meaning to write it up a little better | 13:21 |
| johnsom | Have you tried passing us a pre-made port with your SG on it already? | 13:21 |
| mugsie | no... can we do that? | 13:21 |
| johnsom | Yes | 13:21 |
| mugsie | (this is all via the LBaaS v2 API) | 13:22 |
| *** amoralej|lunch is now known as amoralej | 13:22 | |
| johnsom | It might work for your needs | 13:22 |
| mugsie | and all traffic from the LB -> Backends will come out that port? | 13:22 |
| johnsom | Yes, lbaasv2 api with Octavia | 13:22 |
| johnsom | No | 13:22 |
| *** yamamoto has joined #openstack-lbaas | 13:22 | |
| mugsie | where does that traffic come from? | 13:22 |
| rm_work | johnsom: looks like he's pulling a slightly different get-pip.py location | 13:22 |
| johnsom | Lb to backends is a port we create on each backend network | 13:23 |
| mugsie | but so is the VIP Port? | 13:23 |
| rm_work | johnsom: and his matches the path from https://review.openstack.org/#/c/378999/ and others | 13:24 |
| rm_work | johnsom: so maybe we did the wrong location previously? | 13:24 |
| johnsom | We allow you to optionally pass us a vip port | 13:24 |
| *** yamamoto has quit IRC | 13:24 | |
| *** fnaval has quit IRC | 13:25 | |
| johnsom | rm_work hmmm, I used the path infra gave me and it seems to work. I will take a look at the patch when I am off my mobile | 13:25 |
| rm_work | AFAIK monty is basically an infra guru :P so I trust his judgement there -- but maybe we need to just use his version and essentially revert the previous patch? :/ dunno | 13:26 |
| rm_work | his might just be simpler / more generic | 13:27 |
| mugsie | johnsom: ah, we pass a VIP subnet | 13:27 |
| mugsie | https://github.com/kubernetes/kubernetes/blob/master/pkg/cloudprovider/providers/openstack/openstack_loadbalancer.go#L364-L368 | 13:27 |
| johnsom | mugsie try passing a port | 13:27 |
| *** ducttape_ has joined #openstack-lbaas | 13:28 | |
| mugsie | and Ocavia won't add its own SG to that custom port? | 13:28 |
| johnsom | rm_work well, I -1'd his nodepool patch for this, so... yeah | 13:28 |
| johnsom | mugsie it will, but you are trying to use transitive trust right? Making the vip a member of the same SG as your other ports? | 13:29 |
| johnsom | It won't help with the backend traffic though. I am still not clear on the use case | 13:30 |
| *** fnaval has joined #openstack-lbaas | 13:32 | |
| mugsie | no, we want to add a rule to the backend SG, that allows traffic from the VIP SG | 13:33 |
| mugsie | and we cant do that with the octavia created one, as we cannot see it | 13:34 |
| rm_work | johnsom: hmmmmmm | 13:34 |
| johnsom | You want to loop back through the lb? | 13:34 |
| mugsie | nope | 13:34 |
| mugsie | as the LB is in another sec group | 13:35 |
| mugsie | it cannot access the backends. as their sec group does not ahve an allow from 0.0.0.0/0 | 13:35 |
| johnsom | Hahaha, ok, I am just confused now. I will wait for your update on the bug | 13:35 |
| *** ducttape_ has quit IRC | 13:35 | |
| johnsom | So, the backends, are they on the same subnet as the VIP on you load balancer? | 13:37 |
| johnsom | ajo_ DVR has an open bug against it's floating IP implementation when used with neutron allowed address pairs ports. This impacts Octavia. | 13:38 |
| johnsom | I can dig for a bug number if you would like it | 13:38 |
| *** portdirect has quit IRC | 13:38 | |
| johnsom | rm_work Ok, I see that he is adding another location to check. That is cool. Gerrit on my mobile is less than usable sometimes. | 13:40 |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 13:41 |
| *** fnaval_ has joined #openstack-lbaas | 13:43 | |
| *** fnaval_ has quit IRC | 13:43 | |
| *** fnaval has quit IRC | 13:43 | |
| *** fnaval has joined #openstack-lbaas | 13:44 | |
| ajo_ | johnsom, ahhhh thanks :) | 13:48 |
| ajo_ | johnsom, no worries, I just spawned octavia via vagrant which we default to DVR=on, and I saw scenario not passing | 13:49 |
| ajo_ | I disabled DVR and saw scenario passing | 13:49 |
| mugsie | johnsom: is DVR *still* not working? | 13:50 |
| mugsie | -_- | 13:50 |
| *** fnaval has quit IRC | 13:50 | |
| johnsom | Yeah, still broken | 13:50 |
| mugsie | and CVR HA is still ... problematic | 13:51 |
| *** ducnc has quit IRC | 13:54 | |
| *** crc32 has quit IRC | 13:58 | |
| *** amotoki_ has quit IRC | 14:04 | |
| *** ducttape_ has joined #openstack-lbaas | 14:04 | |
| *** gcheresh_ has joined #openstack-lbaas | 14:08 | |
| *** crc32 has joined #openstack-lbaas | 14:11 | |
| *** crc32 has quit IRC | 14:12 | |
| *** crc32 has joined #openstack-lbaas | 14:12 | |
| *** amotoki has joined #openstack-lbaas | 14:24 | |
| *** yamamoto has joined #openstack-lbaas | 14:29 | |
| *** matt-borland has quit IRC | 14:31 | |
| *** amotoki has quit IRC | 14:39 | |
| *** yamamoto has quit IRC | 14:41 | |
| *** amotoki has joined #openstack-lbaas | 14:46 | |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 14:52 |
| *** yamamoto has joined #openstack-lbaas | 14:57 | |
| *** yamamoto has quit IRC | 14:58 | |
| openstackgerrit | Merged openstack/octavia: Use get-pip.py from cache if it exists https://review.openstack.org/380255 | 15:02 |
| *** pglass has joined #openstack-lbaas | 15:05 | |
| *** amotoki has quit IRC | 15:07 | |
| *** numans has quit IRC | 15:11 | |
| openstackgerrit | Miguel Angel Ajo proposed openstack/octavia: Fix nova image-list which is unavailable https://review.openstack.org/379493 | 15:14 |
| *** yamamoto has joined #openstack-lbaas | 15:15 | |
| openstackgerrit | Merged openstack/octavia: Fix typo in active-active-distributor.rst https://review.openstack.org/379077 | 15:24 |
| *** amotoki has joined #openstack-lbaas | 15:24 | |
| openstackgerrit | Paul Glass proposed openstack/octavia: Basic listener scenario test https://review.openstack.org/378922 | 15:39 |
| *** gcheresh_ has quit IRC | 15:43 | |
| *** woodster_ has joined #openstack-lbaas | 15:53 | |
| openstackgerrit | Paul Glass proposed openstack/octavia: Basic session persistence scenario test https://review.openstack.org/379826 | 16:05 |
| openstackgerrit | Paul Glass proposed openstack/octavia: Basic shared pools scenario test https://review.openstack.org/378993 | 16:05 |
| *** rcernin has quit IRC | 16:08 | |
| *** yamamoto has quit IRC | 16:12 | |
| *** bana_k has joined #openstack-lbaas | 16:12 | |
| amoralej | johnsom, i've been testing lbaas-dashboard with newton rc2 using RDO packages and i've found some bugs | 16:24 |
| amoralej | i'm reported in LB | 16:24 |
| amoralej | LP | 16:24 |
| johnsom | Great, thank you | 16:25 |
| *** bana_k has quit IRC | 16:27 | |
| *** bcafarel_ has quit IRC | 16:27 | |
| *** bana_k has joined #openstack-lbaas | 16:28 | |
| *** bcafarel_ has joined #openstack-lbaas | 16:28 | |
| *** ihrachys has quit IRC | 16:28 | |
| *** bana_k has quit IRC | 17:02 | |
| *** ducttape_ has quit IRC | 17:09 | |
| *** yamamoto has joined #openstack-lbaas | 17:13 | |
| *** yamamoto has quit IRC | 17:19 | |
| *** eezhova has quit IRC | 17:20 | |
| *** ducttape_ has joined #openstack-lbaas | 17:25 | |
| *** amoralej is now known as amoralej|off | 17:31 | |
| *** fnaval has joined #openstack-lbaas | 17:56 | |
| *** fnaval has quit IRC | 17:57 | |
| *** fnaval has joined #openstack-lbaas | 17:58 | |
| *** fnaval has quit IRC | 17:59 | |
| *** fnaval has joined #openstack-lbaas | 17:59 | |
| openstackgerrit | Merged openstack/neutron-lbaas-dashboard: Subnet dropdown list has empty fields https://review.openstack.org/357322 | 18:11 |
| *** eezhova has joined #openstack-lbaas | 18:24 | |
| *** bana_k has joined #openstack-lbaas | 18:35 | |
| *** kbyrne has quit IRC | 19:01 | |
| pglass | in devstack, there is a neutron database with lbaas* tables, and there is a separate octavia database. | 19:05 |
| pglass | when creating resources with the `neutron lbaas*` commands, I see resources placed in both the neutron tables and the octavia tables. | 19:05 |
| johnsom | Correct | 19:05 |
| johnsom | This is part of what we are trying to solve with the merge.... | 19:05 |
| pglass | are there known situations where those two databases could get out of sync? | 19:06 |
| johnsom | Yes | 19:06 |
| *** kbyrne has joined #openstack-lbaas | 19:06 | |
| johnsom | Especially if the event synchronization isn't enabled | 19:07 |
| pglass | what is event synchronization? | 19:07 |
| pglass | in this context | 19:08 |
| johnsom | Enabling the "event_streamer_driver" in your octavia.conf | 19:08 |
| johnsom | The default is noop driver | 19:08 |
| johnsom | It pushes certain events from Octavia back up to neutron lbaas over oslo messaging. | 19:09 |
| pglass | is this typically enabled in the openstack ci jobs? | 19:09 |
| pglass | i guess I can just check | 19:10 |
| pglass | so based on, http://logs.openstack.org/22/378922/2/check/gate-octavia-v1-dsvm-scenario-ubuntu-xenial-nv/43844bd/logs/etc/octavia/octavia.conf.txt.gz it uses the noop driver | 19:10 |
| johnsom | Yep, there you go. | 19:11 |
| johnsom | What are you looking for? health changes? | 19:11 |
| pglass | well I created a health monitor and it quickly put my two member nodes into an error status (looking at logs). the members in the octavia database have their operating status set to ERROR. the members in the neutron database are set to ONLINE however. | 19:13 |
| pglass | trying out the `neutron lbaas-loadbalancer-status <lb>` command (which lists everything) doesn't show the error statuses - presumably because it's using neutron's api which checks neutron's database. | 19:15 |
| *** openstackgerrit has quit IRC | 19:18 | |
| johnsom | Correct. Enable the event streamer and you will get what you want | 19:19 |
| *** openstackgerrit has joined #openstack-lbaas | 19:19 | |
| sbalukoff | Hmmm... might be a good idea to set this to enabled in devstack by default? Any down-sides to doing that? | 19:24 |
| pglass | well, does this actually affect any behavior other than gets via neutron's api? | 19:26 |
| pglass | like our tempest tests are consuming octavia's api directly, so they wouldn't see inconsistencies there? | 19:27 |
| pglass | the new tempest tests, that is. | 19:27 |
| *** carrbs has joined #openstack-lbaas | 19:33 | |
| pglass | okay. that's better. | 19:35 |
| pglass | are there plans to create an octavia-specific client? | 19:36 |
| *** anilvenkata has joined #openstack-lbaas | 19:36 | |
| *** FransUrbo has joined #openstack-lbaas | 19:44 | |
| FransUrbo | I'm having trouble with one of my LBaaSv1.. I've installed a second controller node, put keepalived in front of (most) services (not Neutron - 9696 - which didn't work for some reason) and put my virtual router into distributed mode (can't seem to change it into 'HA' mode). | 19:46 |
| FransUrbo | And now one of my loadbalancers (HTTPS) can't seem to forward the request when called with the floating IP. With the internal IP it works. | 19:47 |
| FransUrbo | When using curl, I get: curl: (35) Unknown SSL protocol error in connection to fqdn:port | 19:48 |
| FransUrbo | The request never reaches the real server behind it.. | 19:48 |
| FransUrbo | Running strace on the haproxy process, I do see that it tries to connect to the backend server on the correct port (EINPROGRESS). | 19:51 |
| FransUrbo | In the syslog, I get: neutron-lbaas-agent[25825]: 2016-09-30 20:52:18.098 25825 WARNING neutron_lbaas.services.loadbalancer.drivers.haproxy.namespace_driver [-] Error while connecting to stats socket: [Errno 111] ECONNREFUSED | 19:52 |
| sbalukoff | pglass: Yes with the merge we intend to create an Octavia CLI, which will operate a lot like the neutron-lbaas CLI, but will be part of the unified openstack CLI system. | 19:56 |
| sbalukoff | I don't think anyone is working on that just yet. | 19:56 |
| johnsom | FransUrbo LbaaSv1 has been deprecated for a while and has been removed from the codebase in newton, so there aren't a lot of folks that have much recent experience with it. We highly recommend you use LBaaSv2 instead. That said, not being able to connect to the stats socket implies the haproxy process has a problem. | 19:58 |
| FransUrbo | Yeah, I tried that a few weeks ago, but i couldn't get it to work. | 19:58 |
| johnsom | pglass Yes and no. With the merge the current python-neutronclient will work directly as will the openstack client (I think someone is working on that), but it will use the LBaaSv2 API | 19:59 |
| pglass | :q | 19:59 |
| FransUrbo | I posted a couple of issues that I've been seeing, but apparently I'm "seeing things" (so to speak :). I was planning on getting back to it 'eventually', but I had other more important things on my mind. | 19:59 |
| pglass | (ehh, vim commands) | 19:59 |
| johnsom | FransUrbo, where did you post them? Here? | 20:00 |
| FransUrbo | No, in the tracker. | 20:00 |
| johnsom | Ah | 20:00 |
| johnsom | Well, with v2 we can help you here at least | 20:00 |
| johnsom | What version of OpenStack are you using? | 20:01 |
| FransUrbo | Mitaka. | 20:01 |
| johnsom | Oh, good stuff | 20:01 |
| FransUrbo | yeah, but I'm pretty sure I'll be forced to upgrade 'shortly'. I Need VPNaaS and a few other bits and pieces that's only available in Newton.. | 20:02 |
| johnsom | Ok, yeah, if you want to give LBaaSv2 ago we can help. Do you want to keep using the namespace/agent driver or Octavia (I'm guessing Octavia as you want HA) | 20:02 |
| johnsom | Even better. | 20:03 |
| *** anilvenkata has quit IRC | 20:03 | |
| FransUrbo | Not today. When I go to LBaaSv2, I need to recreate almost all my infrastructure, and I want it to actually _WORK_ first :). Even if it's not perfect (I need the listeners from LBaaSv2), but for now I can survive with only one listener.. | 20:04 |
| *** anilvenkata has joined #openstack-lbaas | 20:04 | |
| FransUrbo | s/need the listeners/need the multiple listeners/ | 20:04 |
| johnsom | Yeah, note, you will not find LBaaSv1 in newton. | 20:05 |
| FransUrbo | Because this worked before my second controller and distributed router, I'm pretty sure it's something I did, but I'm unsure of how to rectify it. | 20:05 |
| johnsom | Your issue doesn't sound DVR related, but there are a number of bugs in DVR. You might try it on a router with DVR disabled. | 20:06 |
| FransUrbo | It's taken all day to cleanup other bits and pieces that broke with this, and from what I can tell, this is the last piece. And I'm _pretty_ sure it's only this one LBaaSv1 that's at fault.. | 20:06 |
| FransUrbo | "try it on a router with DVR disabled"? I do remember something about DVR in my fixes today (or yesterday).. | 20:07 |
| FransUrbo | Ah, yes: l3_agent.ini:DEFAULT/agent_mode=dvr_snat (was 'legacy'). | 20:12 |
| FransUrbo | But if that was the case, then I'd figure _all_ of my LBaaSv1 would break.. Right? | 20:12 |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 20:21 |
| johnsom | sbalukoff Are you doing development on xenial? I'm seeing this strange thing where tox -e py27 is only running 725 tests instead of 1200+ | 20:22 |
| johnsom | Just curious if it's something in my environment or a general xenial thing | 20:22 |
| sbalukoff | johnsom: Yeah, I'm on xenial, and yes, I'm seeing that too. | 20:22 |
| johnsom | Ok. Project for me this afternoon | 20:23 |
| sbalukoff | Also, good catch on that. It hadn't occurred to me until just now that there should be a *lot* more tests running... | 20:23 |
| johnsom | Yeah, the coverage gate dropped to like 66% which made me do a o-shucks moment until I realized it didn't run half the tests | 20:24 |
| *** eezhova has quit IRC | 20:25 | |
| FransUrbo | I just noticed that the curl command works just fine from outside of OS.. ! | 20:31 |
| FransUrbo | It seems like it's only if I do it from an instance that it fails.. | 20:32 |
| FransUrbo | Which seems to indicate the router.. | 20:32 |
| johnsom | hmm, ok, not xenial. I just produced it on my trusty box. It must be a python package update. | 20:33 |
| FransUrbo | And it works from the instance if I'm using the private IP of the VIP (but not the floating one).. | 20:35 |
| pglass | in devstack, i have a load balancer with a vip of 10.0.0.7 and a member with ip 10.0.0.5 and load balancing works just fine. however, if I ssh into the amp, I am unable to curl that member directly. | 20:40 |
| johnsom | Yes, it's in a network namespace | 20:40 |
| johnsom | sudo ip netns exec amphora-haproxy curl ... | 20:40 |
| * johnsom wonders if he remembered that command line right | 20:41 | |
| FransUrbo | The 'amphora-haproxy' should be the namespace name.. | 20:42 |
| FransUrbo | ip netns exec qrouter-4b3639a1-880f-4b55-989f-c6f654e562a7 curl --insecure https://fqdn:port | 20:42 |
| johnsom | FransUrbo pglass is using Octavia which is a bit different | 20:42 |
| FransUrbo | But this sounds vagely like that problem I'm having.. | 20:42 |
| FransUrbo | Ah, sorry | 20:42 |
| openstackgerrit | Phillip Toohill proposed openstack/octavia: Add quota support to Octavia https://review.openstack.org/360794 | 20:45 |
| pglass | well that is something i need to read about. but that makes sense - `sudo ip netns exec amphora-haproxy route -n` | 20:47 |
| johnsom | pglass With Octavia the haproxy and keepalived processes live in an isolated network namespace. The VIP and member networks get plugged into that namespace | 20:47 |
| FransUrbo | How do you debug a (possible) problem with a namespace? | 20:51 |
| *** anilvenkata has quit IRC | 20:53 | |
| johnsom | FransUrbo I usually do sudo ip netns exec <name> bash and look around. But in your case, if it works without the floating IP, it's not likely a namespace issue. | 20:54 |
| johnsom | Did you try disabling DVR? | 20:54 |
| FransUrbo | Well, it works inside the namespace, outside of it but not from an instance. | 20:54 |
| FransUrbo | How do I disable DVR? | 20:55 |
| johnsom | FransUrbo neutron router-update --distributed False <router> | 20:57 |
| johnsom | Then recreate the floating up | 20:57 |
| johnsom | ip | 20:57 |
| FransUrbo | Ah... Doh! | 20:58 |
| openstackgerrit | Stephen Balukoff proposed openstack/octavia: Add support for PKCS7 bundles and encrypted keys https://review.openstack.org/380020 | 20:58 |
| FransUrbo | I was thinking about changes to the l3_agent.ini file.. Probably have to do that to, don't I? | 20:59 |
| FransUrbo | s/agent_mode = dvr_snat/agent_mode = legacy/ ? | 20:59 |
| johnsom | Hmm, not sure. I think the command line will work until it restarts at a minimum | 21:00 |
| FransUrbo | Right, I forgot. I've actually tried that: Bad router request: Migration from distributed router to centralized is not supported. | 21:00 |
| johnsom | HA | 21:01 |
| pglass | okay. i figured out my issue: https://bugs.launchpad.net/octavia/+bug/1600326. whenever I added a health monitor, it would put my member into error. the health monitor's timeout was 5 - which meant 5 milliseconds. | 21:04 |
| openstack | Launchpad bug 1600326 in neutron "neutron-lbaas health monitor timeout and delay values interpreted as milliseconds" [Medium,Confirmed] | 21:04 |
| pglass | then haproxy would timeout on the health checks. | 21:05 |
| pglass | and octavia uses something like "3s" (three seconds) for haproxy's health check interval, but doesn't do the same for the timeout | 21:09 |
| FransUrbo | I'm pretty sure my probem is in the router and not on the loadbalancer so I'll go over to the FWaaS channel instead. Thanx johnsom for the input! | 21:10 |
| pglass | https://github.com/openstack/octavia/blob/master/octavia/common/jinja/haproxy/templates/macros.j2#L186 | 21:12 |
| pglass | but okay. maybe that will solve my tempest test issues. | 21:13 |
| *** FransUrbo has left #openstack-lbaas | 21:14 | |
| johnsom | pglass hmmm, I thought that got fixed. Can you confirm that is the issue? | 21:19 |
| *** eezhova has joined #openstack-lbaas | 21:20 | |
| pglass | i'm pretty confident about it now. let me paste some things for you | 21:22 |
| pglass | johnsom: http://paste.openstack.org/show/583686/ | 21:24 |
| pglass | so i use the neutron client to update the timeout to "5", and the haproxy.cfg on the amp is then updated to include "timeout check 5" | 21:25 |
| pglass | and per haproxy docs, that is in milliseconds, unless suffixed by another unit: https://cbonte.github.io/haproxy-dconv/1.5/configuration.html#4.2-timeout%20check | 21:26 |
| johnsom | Ok, yeah, I see that too | 21:26 |
| johnsom | Yeah, it looks like both the legacy driver and octavia have that issue | 21:32 |
| johnsom | Do you want to push patches up or want me to? | 21:32 |
| pglass | uhh, i can do it. just one to neutron-lbaas and one to octavia? | 21:33 |
| johnsom | pglass ^^? | 21:33 |
| johnsom | Yeah, I'm thinking add the "s" to the jinja templates | 21:33 |
| johnsom | https://github.com/openstack/neutron-lbaas/blob/master/neutron_lbaas/drivers/haproxy/templates/haproxy_proxies.j2 | 21:34 |
| johnsom | https://github.com/openstack/octavia/blob/master/octavia/common/jinja/haproxy/templates/macros.j2 | 21:35 |
| *** ducttape_ has quit IRC | 21:36 | |
| openstackgerrit | Paul Glass proposed openstack/neutron-lbaas: Switch HAProxy health check timeout to seconds https://review.openstack.org/380658 | 21:38 |
| openstackgerrit | Paul Glass proposed openstack/octavia: Switch HAProxy health check timeout to seconds https://review.openstack.org/380660 | 21:40 |
| johnsom | Thanks pglass | 21:43 |
| pglass | no problem! heading home now | 21:43 |
| johnsom | Have a good weekend | 21:43 |
| *** pglass has quit IRC | 21:48 | |
| *** ducttape_ has joined #openstack-lbaas | 22:42 | |
| *** eezhova has quit IRC | 23:27 | |
| *** bana_k has quit IRC | 23:36 | |
| *** yamamoto has joined #openstack-lbaas | 23:48 | |
| *** fnaval has quit IRC | 23:54 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!