*** leitan has quit IRC | 00:04 | |
*** leitan has joined #openstack-lbaas | 00:07 | |
*** fnaval has quit IRC | 00:34 | |
*** fnaval has joined #openstack-lbaas | 00:36 | |
*** leitan has quit IRC | 00:42 | |
*** armax has quit IRC | 00:48 | |
*** armax has joined #openstack-lbaas | 00:56 | |
*** armax has quit IRC | 00:59 | |
*** ipsecguy has quit IRC | 01:07 | |
*** csomerville has joined #openstack-lbaas | 01:19 | |
*** cody-somerville has quit IRC | 01:20 | |
*** leitan has joined #openstack-lbaas | 01:21 | |
openstackgerrit | Merged openstack/python-octaviaclient master: Add member commands to client https://review.openstack.org/463035 | 01:28 |
---|---|---|
*** ipsecguy has joined #openstack-lbaas | 01:28 | |
*** sanfern has quit IRC | 01:32 | |
rm_work | eandersson: still failing i guess? https://review.openstack.org/#/c/471611/ | 01:33 |
*** JudeC has joined #openstack-lbaas | 01:34 | |
*** dayou has quit IRC | 01:36 | |
openstackgerrit | Jude Cross proposed openstack/python-octaviaclient master: Fix listener command to include PROXY https://review.openstack.org/471991 | 01:40 |
*** dayou has joined #openstack-lbaas | 01:56 | |
*** csomerville has quit IRC | 01:57 | |
*** fnaval has quit IRC | 02:01 | |
*** armax has joined #openstack-lbaas | 02:02 | |
*** fnaval has joined #openstack-lbaas | 02:04 | |
*** yamamoto_ has quit IRC | 02:14 | |
*** eanderson_ has joined #openstack-lbaas | 02:15 | |
*** eanderson_ is now known as eandersson_ | 02:15 | |
eandersson_ | rm_work, yea, but it isn't using the new patch | 02:15 |
eandersson_ | > 2017-06-07 21:39:33.472349 | inst_method = types.MethodType(go_to_method, None, Navigation) | 02:16 |
*** JudeC has quit IRC | 02:16 | |
*** yamamoto_ has joined #openstack-lbaas | 02:17 | |
eandersson_ | Since it's just getting the latest tar.gz file and installing that | 02:18 |
eandersson_ | > Collecting horizon from http://tarballs.openstack.org/horizon/horizon-master.tar.gz#egg=horizon | 02:18 |
eandersson_ | We will need to wait until the patch has been merged | 02:18 |
openstackgerrit | Merged openstack/python-octaviaclient master: Add l7policy commands to client https://review.openstack.org/462431 | 02:22 |
openstackgerrit | Merged openstack/python-octaviaclient master: Add l7rule commands to client https://review.openstack.org/462808 | 02:22 |
openstackgerrit | Merged openstack/python-octaviaclient master: Add healthmonitor commands to client https://review.openstack.org/463830 | 02:22 |
openstackgerrit | Merged openstack/python-octaviaclient master: Update help text for all commands https://review.openstack.org/466441 | 02:23 |
*** sanfern has joined #openstack-lbaas | 02:54 | |
*** yamamoto_ has quit IRC | 03:04 | |
*** leitan has quit IRC | 03:23 | |
*** links has joined #openstack-lbaas | 03:34 | |
*** gans has joined #openstack-lbaas | 03:48 | |
*** gcheresh_ has joined #openstack-lbaas | 04:17 | |
*** gcheresh_ has quit IRC | 04:24 | |
*** pcaruana has joined #openstack-lbaas | 04:31 | |
*** kobis has joined #openstack-lbaas | 04:34 | |
*** kobis has quit IRC | 04:39 | |
*** kobis has joined #openstack-lbaas | 04:43 | |
*** kobis has quit IRC | 04:45 | |
*** yamamoto has joined #openstack-lbaas | 04:48 | |
*** reedip is now known as reedip_afk | 04:54 | |
*** gans has quit IRC | 04:55 | |
*** pcaruana has quit IRC | 04:57 | |
*** leitan has joined #openstack-lbaas | 05:00 | |
*** gans has joined #openstack-lbaas | 05:01 | |
*** leitan has quit IRC | 05:04 | |
*** gcheresh_ has joined #openstack-lbaas | 05:13 | |
openstackgerrit | Merged openstack/octavia master: Handle log message interpolation by the logger https://review.openstack.org/471947 | 05:21 |
*** aojea has joined #openstack-lbaas | 05:26 | |
*** reedip_afk is now known as reedip | 05:39 | |
*** belharar has joined #openstack-lbaas | 05:45 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/neutron-lbaas-dashboard master: Updated from global requirements https://review.openstack.org/472068 | 05:50 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/neutron-lbaas-dashboard master: Updated from global requirements https://review.openstack.org/472094 | 06:06 |
*** aojea has quit IRC | 06:07 | |
*** armax has quit IRC | 06:08 | |
*** armax has joined #openstack-lbaas | 06:08 | |
*** armax has quit IRC | 06:09 | |
*** rcernin has joined #openstack-lbaas | 06:15 | |
*** belharar has quit IRC | 06:54 | |
*** pcaruana has joined #openstack-lbaas | 06:57 | |
*** JudeC has joined #openstack-lbaas | 06:58 | |
*** gans has quit IRC | 07:08 | |
*** tesseract has joined #openstack-lbaas | 07:12 | |
*** belharar has joined #openstack-lbaas | 07:15 | |
*** JudeC has quit IRC | 07:21 | |
*** aojea has joined #openstack-lbaas | 07:22 | |
*** leitan has joined #openstack-lbaas | 07:24 | |
*** leitan has quit IRC | 07:28 | |
eandersson_ | rm_work: > The old method of installing a version of _() in the builtins namespace is deprecated | 07:28 |
eandersson_ | I was just browsing the olso.i18n documentation and noticed the above dislaimar | 07:29 |
eandersson_ | https://docs.openstack.org/developer/oslo.i18n/usage.html#creating-an-integration-module | 07:29 |
*** eandersson_ has quit IRC | 07:32 | |
*** links has quit IRC | 08:25 | |
*** links has joined #openstack-lbaas | 08:38 | |
*** yamamoto has quit IRC | 08:58 | |
*** belharar has quit IRC | 09:08 | |
*** gans has joined #openstack-lbaas | 09:16 | |
*** gans has quit IRC | 09:25 | |
*** gans has joined #openstack-lbaas | 09:30 | |
*** gongysh has joined #openstack-lbaas | 09:32 | |
*** gcheresh_ has quit IRC | 10:03 | |
*** gcheresh_ has joined #openstack-lbaas | 10:03 | |
*** med_ has quit IRC | 10:31 | |
*** Guest72612 has joined #openstack-lbaas | 10:35 | |
*** sanfern has quit IRC | 10:52 | |
*** aojea has quit IRC | 11:15 | |
*** gongysh has quit IRC | 11:37 | |
*** gans has quit IRC | 11:41 | |
*** kobis has joined #openstack-lbaas | 11:43 | |
*** atoth has joined #openstack-lbaas | 12:03 | |
*** ianychoi has quit IRC | 12:27 | |
*** ianychoi has joined #openstack-lbaas | 12:28 | |
*** catintheroof has joined #openstack-lbaas | 12:32 | |
*** sanfern has joined #openstack-lbaas | 12:41 | |
*** yamamoto has joined #openstack-lbaas | 12:55 | |
*** aojea has joined #openstack-lbaas | 13:08 | |
*** yamamoto has quit IRC | 13:10 | |
*** yamamoto has joined #openstack-lbaas | 13:14 | |
*** links has quit IRC | 13:15 | |
*** sanfern has quit IRC | 13:20 | |
*** sanfern has joined #openstack-lbaas | 13:21 | |
*** cpuga has joined #openstack-lbaas | 13:23 | |
*** cpuga has quit IRC | 13:25 | |
*** cpuga has joined #openstack-lbaas | 13:25 | |
*** leitan has joined #openstack-lbaas | 13:35 | |
openstackgerrit | Merged openstack/octavia-tempest-plugin master: Updated from global requirements https://review.openstack.org/471180 | 13:35 |
*** leitan has quit IRC | 13:35 | |
*** leitan has joined #openstack-lbaas | 13:35 | |
*** fnaval_ has joined #openstack-lbaas | 13:49 | |
*** fnaval has quit IRC | 13:52 | |
*** ianychoi_ has joined #openstack-lbaas | 13:54 | |
*** ianychoi has quit IRC | 13:57 | |
*** ianychoi_ is now known as ianychoi | 13:57 | |
*** matt-borland has joined #openstack-lbaas | 13:57 | |
*** chlong has joined #openstack-lbaas | 14:02 | |
*** aojea has quit IRC | 14:12 | |
openstackgerrit | Merged openstack/octavia master: Add check O346 to disallow backslash line continuation https://review.openstack.org/471503 | 14:20 |
*** kobis has quit IRC | 14:26 | |
openstackgerrit | Merged openstack/octavia master: Minor code cleanup in l7policy controller https://review.openstack.org/471176 | 14:33 |
*** aojea has joined #openstack-lbaas | 14:34 | |
*** aojea has quit IRC | 14:36 | |
*** aojea has joined #openstack-lbaas | 14:40 | |
*** armax has joined #openstack-lbaas | 14:47 | |
*** gcheresh_ has quit IRC | 14:54 | |
*** fnaval_ has quit IRC | 15:01 | |
*** aojea has quit IRC | 15:12 | |
*** rcernin has quit IRC | 15:16 | |
*** fnaval has joined #openstack-lbaas | 15:18 | |
*** cpuga has quit IRC | 15:28 | |
*** cpuga has joined #openstack-lbaas | 15:28 | |
*** yamamoto has quit IRC | 15:30 | |
*** aojea has joined #openstack-lbaas | 15:32 | |
*** cpuga_ has joined #openstack-lbaas | 15:32 | |
*** yamamoto has joined #openstack-lbaas | 15:35 | |
*** cpuga has quit IRC | 15:35 | |
*** bzhao_ has joined #openstack-lbaas | 15:35 | |
*** gans has joined #openstack-lbaas | 15:37 | |
*** yamamoto has quit IRC | 15:39 | |
*** armax has quit IRC | 15:40 | |
*** leitan has quit IRC | 15:42 | |
*** cpuga_ has quit IRC | 15:42 | |
*** cpuga has joined #openstack-lbaas | 15:43 | |
*** aojea has quit IRC | 15:53 | |
*** reedip_ has joined #openstack-lbaas | 16:06 | |
openstackgerrit | Michael Johnson proposed openstack/octavia-dashboard master: Replace SortedDict with OrderedDict and fixing Python 3.5 test https://review.openstack.org/471611 | 16:08 |
johnsom | Let's see what we get with that | 16:09 |
*** cpuga has quit IRC | 16:12 | |
*** cpuga has joined #openstack-lbaas | 16:13 | |
*** leitan has joined #openstack-lbaas | 16:18 | |
openstackgerrit | Michael Johnson proposed openstack/octavia-dashboard master: Replace SortedDict with OrderedDict and fixing Python 3.5 test https://review.openstack.org/471611 | 16:23 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/octavia master: Updated from global requirements https://review.openstack.org/472325 | 16:23 |
openstackgerrit | OpenStack Proposal Bot proposed openstack/neutron-lbaas-dashboard master: Updated from global requirements https://review.openstack.org/472326 | 16:23 |
*** tesseract has quit IRC | 16:32 | |
*** yamamoto has joined #openstack-lbaas | 16:37 | |
*** armax has joined #openstack-lbaas | 16:43 | |
*** yamamoto has quit IRC | 16:43 | |
*** fnaval has quit IRC | 16:49 | |
*** gans has quit IRC | 16:53 | |
*** pcaruana has quit IRC | 17:00 | |
*** aojea has joined #openstack-lbaas | 17:14 | |
*** eanderson_ has joined #openstack-lbaas | 17:16 | |
*** eanderson_ is now known as eandersson_ | 17:17 | |
*** aojea has quit IRC | 17:19 | |
openstackgerrit | Erik Olof Gunnar Andersson proposed openstack/octavia-dashboard master: Replace SortedDict with OrderedDict and fixing Python 3.5 test https://review.openstack.org/471611 | 17:21 |
eandersson_ | johnsom: Added horizon back now as the upstream fix was merged | 17:22 |
*** amotoki is now known as amotoki_away | 17:28 | |
johnsom | Well, I was waiting for the tar to actually be built, it is still in the queue | 17:30 |
eandersson_ | Ah | 17:30 |
eandersson_ | I was just looking at the status of that | 17:30 |
eandersson_ | I assumed it was done already as I slept in :D | 17:31 |
johnsom | Yeah, it is sitting in the "post" queue to build the tar | 17:31 |
*** amotoki_away is now known as amotoki | 17:35 | |
*** sanfern has quit IRC | 17:39 | |
leitan | hi guys | 17:42 |
leitan | trying to test the brand new generated amphora image | 17:42 |
leitan | to test the keepalived patches | 17:42 |
leitan | getting on the log "Cannot connect to instance retrying" | 17:43 |
johnsom | Hi ther | 17:43 |
johnsom | there | 17:43 |
leitan | but i can connect to the instances just fine, via the mgmt network | 17:43 |
leitan | ill try rollabking to the old image, to check if something else got broken in the meantime | 17:43 |
leitan | but any thoughts ? | 17:43 |
johnsom | Well, if you can connect over the mgmt net, take a look at the amphora-agent logs. /var/log/amphora-agent.log and /var/log/syslog | 17:44 |
eandersson_ | The tar is finally there... fingers crossed :D | 17:45 |
johnsom | Yeah, waiting to force a recheck | 17:45 |
eandersson_ | bah didn't realize it was still running -_- | 17:45 |
*** bzhao_ has quit IRC | 17:45 | |
leitan | johnsom: great, ill take a look there and let you know | 17:45 |
johnsom | That last run isn't finish yet | 17:45 |
leitan | johnsom: guess nothing weird on the agent http://paste.openstack.org/show/612037/ | 17:47 |
*** eandersson_ has quit IRC | 17:49 | |
johnsom | How about the syslog? | 17:49 |
leitan | johnsom: checking | 17:50 |
leitan | hmm, inside the amphora image, the api on the port 9443 but if i curl it inside the same VM i get curl: (56) Recv failure: Connection reset by peer | 17:58 |
leitan | doing it to localhost or the actual mgmt ip | 17:58 |
leitan | inside or outside the amphorae vms same result | 17:59 |
*** aojea has joined #openstack-lbaas | 18:02 | |
leitan | hmm, maybe coause its SSL , ill try with https instead | 18:03 |
leitan | yep, was that, getting a 404 now, curling the amphorae vm, from the controller | 18:05 |
leitan | so its reaching it ... | 18:05 |
leitan | i dont understand why the rest_api.driver says that connot reach the vm | 18:05 |
*** reedip_ has quit IRC | 18:10 | |
*** fnaval has joined #openstack-lbaas | 18:15 | |
eandersson | Success :D | 18:17 |
johnsom | Oh good! | 18:18 |
eandersson | or wel... npm-run-test is probably gonna fail | 18:18 |
eandersson | So we still need that project update | 18:19 |
johnsom | Which? | 18:19 |
*** cpuga has quit IRC | 18:19 | |
eandersson | https://review.openstack.org/#/c/471887/ | 18:19 |
*** cpuga has joined #openstack-lbaas | 18:19 | |
johnsom | Ah, ok. | 18:19 |
leitan | 9443 answering same 404 inside the container where the api is running, im going to debug there to see what "connection error" is returning | 18:22 |
*** aojea has quit IRC | 18:22 | |
*** ipsecguy_ has joined #openstack-lbaas | 18:23 | |
*** ipsecguy has quit IRC | 18:24 | |
eandersson | johnsom, I'll switch to chrome and update the dependencies to match horizon | 18:28 |
eandersson | and see if that fixes the issue | 18:28 |
johnsom | Cool, thanks! | 18:29 |
rm_work | eandersson: ah fix it so the test is actually PASSING? :P | 18:29 |
johnsom | I like passing tests.... Grin | 18:30 |
eandersson | Yea - I mean sounds crazy I know | 18:31 |
robcresswell | eandersson: I'd try just the deps first before switching | 18:33 |
robcresswell | I think its just a fix to karma.conf, at a guess... | 18:33 |
eandersson | I managed to get the tests running, but hit a new issue | 18:47 |
leitan | well | 18:55 |
leitan | i managed to debug the issue | 18:55 |
leitan | and i think its a bug | 18:55 |
*** amotoki is now known as amotoki_away | 18:55 | |
leitan | for some reason, the amphorae image that is spawned first (maybe the master?) get assigned two or more ips from the management network | 18:57 |
leitan | if the first ip, is assigned as a port first to the VM, its reacheable and the balancer gets configured ok | 18:58 |
leitan | but, if for some reason the rest_api driver tries to contact the other binded ip address that are not the one that the VM actually HAS inside | 18:58 |
leitan | fails | 18:58 |
xgerman_ | sounds weird | 18:58 |
leitan | so the real issue here is why the amphorae that is launched first, gets assigned more than one ip from the management network | 18:59 |
leitan | im looking at all the balancers i created | 18:59 |
leitan | and for all is the same | 18:59 |
leitan | the first one, has two or more ip addresses from the mgmt network | 18:59 |
xgerman_ | huh, they should only get one IP but two on the VIP network | 18:59 |
leitan | the second one, has one and one on the tunneled (OK) | 18:59 |
leitan | xgerman_: thats correct, for the tunneled network the behaviour is correct, 1 ip for each and the vip as another address on the allowed_address pairs | 19:00 |
leitan | but the issue here, is the first amphorae getting more than one ip on the management network | 19:00 |
xgerman_ | yeah, we ask nova to spin it up in that network and assign IP | 19:01 |
leitan | this causes that my loadbalancers, are created ok ACTIVE only if the ip that uses the rest_api driver to contact the balancer is the same as the one that finally got inside the vm from the two or more that got assigned | 19:01 |
xgerman_ | yep, if the REST can’t talk to the VM it will be shot and result in ERROR | 19:01 |
leitan | xgerman_: thats correrct and thats whats happening, but i realized the behaviour adding some print debugs where the rest_api was trying to connect, and saw the multiple ips stuff | 19:02 |
leitan | xgerman_: to be more graphic, here is a pic: http://i66.tinypic.com/10nb7u0.png | 19:04 |
xgerman_ | yeah, that is definitely wrong | 19:04 |
leitan | as you can see the first LB got the .200 and .198 ip address, the one that is on the vm and working is the .200, but the rest_api_driver is trying to contact the .198, without success of course | 19:04 |
xgerman_ | mmh, I don’t know where that seconf ip could come from… we allocate/plug all our IP on tenant networks only (unless you passed in mgmt network for some subnet) | 19:06 |
rm_work | leitan: does your nova add a default vif? | 19:07 |
rm_work | some deployments do | 19:07 |
rm_work | for example, mine does :P | 19:07 |
leitan | rm_work: no | 19:07 |
rm_work | ok | 19:07 |
*** cpuga has quit IRC | 19:07 | |
leitan | rm_work xgerman_ if i manually launch many instances, on the lb_mgmt_net, the issue is not present | 19:09 |
*** cpuga has joined #openstack-lbaas | 19:09 | |
leitan | so im discarding the nova -> neutron circuit | 19:09 |
rm_work | hmm | 19:09 |
leitan | but this happen with every single balancer pair i create with octavia, with the first vm, two or more ip address on the mgmt net, some LBs has 3 ips hahaha | 19:11 |
*** gcheresh_ has joined #openstack-lbaas | 19:13 | |
rm_work | what does your config look like for "lb_mgmt_networks" | 19:14 |
leitan | rm_work: one sec | 19:15 |
leitan | amp_boot_network_list = 6e668339-70ef-49c6-82ad-ad3f2ad432c5 | 19:17 |
leitan | rm_work: thats the only place when i refer to a network | 19:19 |
rm_work | hmmm | 19:19 |
rm_work | set debug=True and post your whole worker log maybe for one create | 19:19 |
rm_work | i can scan through and see what's going on maybe... | 19:19 |
johnsom | maybe do a "neutron net-list | grep 6e668339-70ef-49c6-82ad-ad3f2ad432c5" | 19:19 |
johnsom | See if there is multiple matching? | 19:20 |
leitan | rm_work: doing it right now | 19:20 |
leitan | johnsom: no, | 6e668339-70ef-49c6-82ad-ad3f2ad432c5 | lb-mgmt-net | 122d58d6-ffa5-49ed-99b0-be71aa899f3e 172.30.3.0/24 | | 19:20 |
rm_work | ok | 19:20 |
johnsom | And "neutron net-show 6e668339-70ef-49c6-82ad-ad3f2ad432c5" just to make sure it doesn't have multiple subnets | 19:20 |
rm_work | unfortunately i'm about to head to lunch, so | 19:20 |
rm_work | after i get back i can look | 19:20 |
leitan | rm_work: k | 19:21 |
leitan | johnsom: | subnets | 122d58d6-ffa5-49ed-99b0-be71aa899f3e | | 19:21 |
leitan | only one | 19:21 |
johnsom | And a "nova show" on the amp with two IPs? | 19:22 |
leitan | johnsom: buuuuuut, the allocation pool is not contiguos | 19:22 |
leitan | johnsom: maybe something with that ? | 19:22 |
leitan | johnsom: allocation_pools | {"start": "172.30.3.165", "end": "172.30.3.250"} | 19:22 |
leitan | {"start": "172.30.3.10", "end": "172.30.3.160"} | 19:22 |
leitan | that cant be it ... but just saying | 19:22 |
johnsom | How about the nova show? | 19:23 |
leitan | johnsom: http://paste.openstack.org/show/612050/ | 19:25 |
johnsom | So that one actually has three mgmt Ips, contiguous | 19:27 |
leitan | johnsom: yes, that discard the non-contigous alloc pool malefic theory | 19:28 |
johnsom | Check the flavor, since you are using a different flavor than we create, it might be something in there, but otherwise a run with debug=true and a copy of that o-cw log would be my next step. | 19:28 |
johnsom | Very unusual. FYI, we take that network ID from the config and pass it to nova, we don't hot plug it or even ask for anything special, just pass it in as the boot network | 19:29 |
leitan | johnsom: only an anti-affinity spec on the flavor http://paste.openstack.org/show/612051/ | 19:30 |
leitan | johnsom: yes, but in that case, the fact that i have spinned 50 vms in the lb-mgmt-network without getting dup ips ... make it weirder | 19:32 |
leitan | and happens always with the first amphorae, always | 19:32 |
leitan | ill do the full debbuged run now | 19:33 |
*** blogan has joined #openstack-lbaas | 19:34 | |
johnsom | What version of Octavia are you running? | 19:37 |
leitan | should be ocata, its a kolla container | 19:40 |
leitan | octavia-0.10.1.dev106 | 19:41 |
johnsom | leitan Can you check this: https://review.openstack.org/#/c/358928/2/octavia/controller/worker/tasks/compute_tasks.py | 19:51 |
johnsom | It should be in that 0.10.1 build, but I am wondering if it is missing | 19:51 |
*** JudeC has joined #openstack-lbaas | 19:52 | |
*** leitan has quit IRC | 20:01 | |
eandersson | https://review.openstack.org/#/c/471611/7 | 20:04 |
eandersson | https://review.openstack.org/#/c/471611/ | 20:04 |
eandersson | ^ It's passing finally | 20:04 |
*** harlowja has quit IRC | 20:08 | |
johnsom | eandersson That doesn't seem to have the schemaform fix, are you working on another patch for that? | 20:12 |
*** matt-borland has quit IRC | 20:17 | |
*** matt-borland_ has joined #openstack-lbaas | 20:17 | |
*** gcheresh_ has quit IRC | 20:18 | |
*** borland__ has joined #openstack-lbaas | 20:27 | |
*** matt-borland_ has quit IRC | 20:28 | |
openstackgerrit | Merged openstack/python-octaviaclient master: Add hacking checks https://review.openstack.org/471168 | 20:29 |
johnsom | JudeC Any chance you can address this comment so we can do a client release soon? https://review.openstack.org/#/c/471991/ | 20:33 |
JudeC | Ha literally 2 sec away from pushing that | 20:33 |
johnsom | Awesome | 20:34 |
JudeC | I noticed some other crap that needed fixing so I added it to that. | 20:34 |
johnsom | Ok | 20:34 |
JudeC | just documentation stuff | 20:34 |
openstackgerrit | Jude Cross proposed openstack/python-octaviaclient master: Fix command to include PROXY and update metavars https://review.openstack.org/471991 | 20:34 |
JudeC | Boom | 20:34 |
johnsom | Yeah, I skimmed the docs. So there might be a few things I missed. I did have a fail on not noticing the proxy proto though.. Sigh | 20:35 |
JudeC | Hey its my code, if anyone should be bummed it me. :P | 20:35 |
johnsom | Ah, found an issue.... | 20:36 |
eandersson | Yea - johnsom - I rather keep that separate as well | 20:37 |
johnsom | Ok, NP. If you weren't going to do it, I was going to try Rob's advice, but if you have it I can move on to other issues... | 20:38 |
eandersson | I managed to get it past the schema thing, but ran into a new issue | 20:38 |
johnsom | JudeC PROXY is a backend protocol only. It sends extra info on backend requests about the front end client | 20:38 |
*** leitan has joined #openstack-lbaas | 20:39 | |
eandersson | but I probably won't have much more time today, so if you want have a go at it | 20:39 |
eandersson | I say we keep them separate though as that isn't blocking anymore | 20:39 |
JudeC | johnsom: ohhhh ok fixing now. | 20:39 |
johnsom | Maybe, I need to check on some bugs that were filed, so not sure if I would have a lot of time either | 20:40 |
johnsom | Yeah, separate is great with me. | 20:40 |
johnsom | JudeC http://logs.openstack.org/99/471499/1/gate/gate-python-octaviaclient-pep8-ubuntu-xenial/eb227be/console.html#_2017-06-08_20_29_04_191549 | 20:42 |
johnsom | While you are there.... | 20:42 |
JudeC | oh weird why does my pep8 pass hmm | 20:43 |
johnsom | Because that patch adds the check for them. | 20:43 |
*** leitan has quit IRC | 20:44 | |
*** kbyrne has quit IRC | 20:48 | |
*** kbyrne has joined #openstack-lbaas | 20:50 | |
openstackgerrit | Jude Cross proposed openstack/python-octaviaclient master: Fix command to include PROXY and update metavars https://review.openstack.org/471991 | 20:54 |
*** borland__ has quit IRC | 21:03 | |
rm_work | johnsom / JudeC: OK if Proxy is backend only then it was right before and I told him wrongly -- but I thought it was not | 21:07 |
rm_work | johnsom: that means I need to change MY patch here: https://review.openstack.org/#/c/463851/5/octavia/common/constants.py | 21:08 |
rm_work | PROXY should not be a LISTENER protocol? | 21:08 |
rm_work | only POOL? | 21:08 |
johnsom | Ah, ok. Yeah, it takes info from the frontend connection (client IP, etc.) and sendings it to the backend before the main TCP connection. | 21:08 |
rm_work | JudeC: ok my bad, sorry about that :P | 21:08 |
rm_work | I thought proxy was both | 21:08 |
johnsom | Wow, bummer, ok, somewhere we introduced a regression | 21:24 |
johnsom | failovers with public IP vips don't work. | 21:24 |
johnsom | Elena reported them here: https://bugs.launchpad.net/octavia/+bug/1694440 | 21:24 |
openstack | Launchpad bug 1694440 in octavia "Spares pool failover errors for a load balancer created in public subnet" [Undecided,New] | 21:24 |
johnsom | Well, with spares pool or act/stdby | 21:24 |
johnsom | So, I'm going to dig into this.... | 21:25 |
johnsom | I hate that the gates don't have the horse power to test this... We need to get containers working.... | 21:27 |
*** blogan has quit IRC | 21:31 | |
*** leitan has joined #openstack-lbaas | 21:39 | |
leitan | guys | 21:42 |
leitan | sorry got in a meeting | 21:42 |
leitan | longer than expected | 21:42 |
leitan | rm_work johnsom here is the full debug run: https://pastebin.com/uz07YwcC | 21:42 |
leitan | johnsom: ill check what you send me when i got home | 21:43 |
rm_work | johnsom: yeah I'm tracking the lxd stuff in the background | 21:43 |
rm_work | johnsom: seems nova-lxd devstack plugin broke, it no longer builds a working nova+lxd | 21:43 |
rm_work | was trying to get someone to talk to / work with us | 21:43 |
*** leitan has quit IRC | 21:44 | |
rm_work | but also busy with a bunch of other stuff T_T | 21:44 |
*** catintheroof has quit IRC | 21:50 | |
*** chlong has quit IRC | 21:53 | |
*** bzhao_ has joined #openstack-lbaas | 22:06 | |
*** cpuga has quit IRC | 22:32 | |
*** marcin123 has joined #openstack-lbaas | 22:41 | |
marcin123 | What are the plans of implementing in Neutron/Octavia dynamically updated of "neutron lbaas-loadbalancer-status <lb_name>" ? To actually see offline member in API manner | 22:47 |
johnsom | marcin123 What driver are you using? This works fine with the octavia driver | 22:48 |
*** tonygunk has quit IRC | 22:53 | |
marcin123 | I am using Octavia driver I am checking my config because I saw this not working. | 22:54 |
johnsom | leitan There is something in your nova config adding the extra IPs. grep that log for "a3c64b9d-79eb-48b0-b2fd-fb6e57ab0824". You will see that we poll nova to see when the status goes active so there are a number of debug messages while nova is building the instance, noted by "status": "BUILD". As the build progresses nova is adding those interfaces to the instance. If you grep for REQ you can see how we are | 22:55 |
johnsom | asking nova to build it with one network. | 22:55 |
johnsom | marcin123 Did you enable the event streamer? octavia.conf event_streamer_driver = queue_event_streamer | 22:55 |
johnsom | marcin123 That tells octavia to send update events to neutron | 22:56 |
johnsom | https://docs.openstack.org/developer/octavia/main/configref.html#health_manager.event_streamer_driver | 22:59 |
eandersson | I just had him configure it | 23:00 |
johnsom | Yeah it's an evil required to keep neutron up to date. That mess goes away with the new Octavia v2 API | 23:00 |
johnsom | Though we haven't finished the status API in that yet | 23:01 |
*** tonygunk has joined #openstack-lbaas | 23:01 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack/neutron-lbaas-dashboard master: Updated from global requirements https://review.openstack.org/472470 | 23:02 |
johnsom | Oh, this hasn't merged yet... https://review.openstack.org/#/c/471611/ rm_work xgerman_ if you have a minute | 23:03 |
marcin123 | so I was missing that "event_streamer_driver = queue_event_streamer" in /etc/octavia/octavia.conf | 23:09 |
rm_work | sounds like that'd be the problem, yep! working now? | 23:11 |
rm_work | johnsom: +A'd | 23:11 |
*** armax_ has joined #openstack-lbaas | 23:12 | |
johnsom | Cool, that should fix our proposal bot spam | 23:12 |
*** armax has quit IRC | 23:14 | |
*** armax_ is now known as armax | 23:14 | |
openstackgerrit | Merged openstack/octavia-dashboard master: Replace SortedDict with OrderedDict and fixing Python 3.5 test https://review.openstack.org/471611 | 23:15 |
*** armax_ has joined #openstack-lbaas | 23:33 | |
*** armax has quit IRC | 23:33 | |
*** armax_ is now known as armax | 23:33 | |
openstackgerrit | Michael Johnson proposed openstack/octavia master: Fix an issue with failover on VIP net without DHCP https://review.openstack.org/472486 | 23:49 |
*** JudeC has quit IRC | 23:49 | |
johnsom | That seems to fix the issue, but I want to do a bit more testing. WIP | 23:49 |
*** tonygunk has quit IRC | 23:53 | |
rm_work | AHH I have a similar fix in my own flow johnsom | 23:53 |
rm_work | https://review.openstack.org/#/c/435612/25/octavia/controller/worker/flows/amphora_flows.py | 23:54 |
rm_work | I found the default one to be a bit broken | 23:54 |
rm_work | (for our network, without DHCP) | 23:54 |
rm_work | I found I always needed to do a ReloadLoadBalancer then ReloadAmphora or stuff wasn't working right | 23:55 |
rm_work | johnsom: you need to rebase your other octavia-dashboard commit? | 23:56 |
johnsom | Oh at a minimum. I kind of started that and didn't finish because of the horizon bugs we just squished. It's marked WIP. | 23:57 |
johnsom | Juggling priorities. I will probably get back to that soon. | 23:57 |
*** harlowja has joined #openstack-lbaas | 23:57 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!