| korean101 | ajo: well. different timezone... | 00:09 |
|---|---|---|
| korean101 | ajo: i'll try master branch and report to you | 00:09 |
| korean101 | ajo: thanks for reply | 00:09 |
| johnsom | korean101 I don't know anyone using the vagrant stuff, so it might be stale | 00:11 |
| korean101 | johnsom: OMG | 00:11 |
| korean101 | johnsom: any other documents ? | 00:12 |
| johnsom | korean101 documents for what? | 00:12 |
| korean101 | johnsom: I get a lot of help with devstack. | 00:12 |
| korean101 | johnsom: installation manuals | 00:13 |
| korean101 | johnsom: i already deploy octavia in Newton Releases | 00:13 |
| johnsom | Our docs are here: https://docs.openstack.org/developer/octavia/ | 00:13 |
| korean101 | johnsom: and now i try Ocata releases | 00:13 |
| johnsom | There is an overview here: https://docs.openstack.org/developer/octavia/guides/dev-quick-start.html | 00:13 |
| johnsom | Most of us day-to-day for development use devstack (without vagrant) | 00:14 |
| johnsom | This is the main octavia setup script for devstack: https://github.com/openstack/octavia/blob/master/devstack/plugin.sh | 00:14 |
| korean101 | i use this file (https://github.com/openstack/octavia/blob/stable/ocata/devstack/samples/singlenode/Vagrantfile) | 00:14 |
| korean101 | johnsom: not freshly file? | 00:14 |
| korean101 | OK. plugin.sh | 00:15 |
| korean101 | johnsom: other questions | 00:15 |
| korean101 | johnsom: any solutions about Octavia HA? | 00:15 |
| johnsom | Yeah, I don't know if the vagrant stuff is current or out of date | 00:15 |
| johnsom | What about octavia HA? We support HA on both control and data plane | 00:16 |
| johnsom | Have since Mitaka if I remember correctly | 00:16 |
| korean101 | johnsom: Octavia daemons dead? and So what happens? | 00:20 |
| *** aojea has quit IRC | 00:20 | |
| johnsom | korean101 You can run multiple instances of all of the octavia processes. | 00:20 |
| johnsom | We recommend three or more | 00:21 |
| *** aojea has joined #openstack-lbaas | 00:21 | |
| korean101 | instances? | 00:21 |
| korean101 | octavia-api, octavia-health-manager, octavia-housekeeping, octavia-worker | 00:22 |
| johnsom | Correct | 00:22 |
| korean101 | that daemons on network nodes only | 00:23 |
| korean101 | but network nodes die? | 00:23 |
| johnsom | You will need to run a load balancer in front of the API instances. You need to configure all of the health manager endpoints in the configuration file so the amps use them all | 00:23 |
| johnsom | No, none of those need to run on network nodes | 00:23 |
| korean101 | johnsom: but o-hm0 device needs br-int? | 00:24 |
| johnsom | The old neutron-lbaas haproxy driver has to run on network nodes, but octavia does not. It's an advantage to using octavia. | 00:25 |
| *** armax has quit IRC | 00:26 | |
| johnsom | Not necessarily, no. That is just how devstack is setup. the lb-mgmt-net (which in devstack we setup as o-hm0) is just a private network between the control plane processes (o-cw, o-hm, o-hk) and the amphora-agent. It can be a private neutron network or a provider (VLAN, etc.) network | 00:27 |
| korean101 | johnsom: hmmmm | 00:28 |
| johnsom | It us up to the deployer how they setup that network | 00:28 |
| johnsom | It can be routable too | 00:29 |
| korean101 | johnsom: so... hard to me | 00:30 |
| korean101 | johnsom: some examples for me? | 00:31 |
| johnsom | OpenStack Ansible has a playbook for Octavia | 00:32 |
| korean101 | johnsom: this one? (http://git.openstack.org/cgit/openstack/openstack-ansible-os_octavia/) | 00:33 |
| johnsom | Yep | 00:33 |
| korean101 | johnsom: Oh thanks!!!! | 00:34 |
| korean101 | johnsom: i'wll try that | 00:34 |
| johnsom | Sure, NP | 00:34 |
| korean101 | johnsom: last questions | 00:34 |
| johnsom | Ok | 00:34 |
| korean101 | johnsom: in Pike releases 1.0 versions come out? | 00:34 |
| johnsom | Yes, it will have it's own endpoint | 00:35 |
| johnsom | https://developer.openstack.org/api-ref/load-balancer/v2/index.html | 00:36 |
| johnsom | It is compatible with neutron-lbaas LBaaS v2 API | 00:36 |
| korean101 | johnsom: OH that's good | 00:37 |
| korean101 | johnsom: OMG... really really last questions | 00:37 |
| johnsom | Much easier deployment as it doesn't plug into neutron | 00:37 |
| korean101 | johnsom: DVR + Octavia problems was solved? | 00:37 |
| johnsom | That document isn't done yet, still L7 and quota to write | 00:37 |
| korean101 | johnsom: floating IPs | 00:37 |
| johnsom | Oh, the neutron bug with floating IPs, DVR, and allowed-address-pairs ports? | 00:38 |
| korean101 | johnsom: well this one (https://docs.openstack.org/ocata/networking-guide/config-lbaas.html#associating-a-floating-ip-address) | 00:40 |
| korean101 | neutron floatingip-associate FLOATINGIP_ID LOAD_BALANCER_PORT_ID | 00:40 |
| johnsom | Ok, what about it? | 00:41 |
| korean101 | https://bugs.launchpad.net/neutron/+bug/1583694 | 00:42 |
| openstack | Launchpad bug 1583694 in neutron "[RFE] DVR support for Allowed_address_pair port that are bound to multiple ACTIVE VM ports" [Wishlist,In progress] - Assigned to Swaminathan Vasudevan (swaminathan-vasudevan) | 00:42 |
| johnsom | Yeah, that is the neutron bug with floating IPs, DVR, and allowed-address-pairs ports | 00:44 |
| johnsom | DVR is still broken | 00:44 |
| johnsom | Though I think the neutron folks are working on it for Pike | 00:45 |
| korean101 | johnsom: ok. i wish fix that bugs | 00:45 |
| korean101 | johnsom: and i wish (DVR + Octavia)'s problem solved in Pike releases | 00:46 |
| korean101 | johnsom: many many thanks! | 00:46 |
| johnsom | Yeah, it breaks more than just octavia. I hope they get it fixed soon too | 00:47 |
| *** Dave has quit IRC | 00:53 | |
| *** Dave____ has joined #openstack-lbaas | 00:55 | |
| *** amotoki_away is now known as amotoki | 01:10 | |
| *** sanfern has quit IRC | 01:21 | |
| *** sanfern has joined #openstack-lbaas | 01:27 | |
| *** dayou has joined #openstack-lbaas | 01:30 | |
| *** sanfern has quit IRC | 01:37 | |
| *** armax has joined #openstack-lbaas | 01:40 | |
| johnsom | Oh brother, these functional tests are screwed up. | 02:10 |
| rm_work | johnsom: seems like healthchecks aren't ... actually working right | 02:10 |
| rm_work | we have http checks configured and it's still doing ping checks it seems | 02:11 |
| johnsom | Hmmm, wonder what is up with that. They were working when I tested the alternate port thing | 02:11 |
| rm_work | looking at the haproxy config and we have: http://paste.openstack.org/show/612629/ | 02:12 |
| rm_work | which looks right | 02:12 |
| rm_work | but the boxes at ip1/ip2 don't show connections from the lb ever | 02:12 |
| rm_work | and we change the healthcheck page to status 500 and the members are still up | 02:12 |
| rm_work | but if we take down the box, it does go away | 02:12 |
| rm_work | <_< | 02:12 |
| rm_work | which functional tests? | 02:13 |
| johnsom | API functional tests, they are setting the statuses before they check the statuses | 02:15 |
| rm_work | yeah i fixed a ton of those | 02:15 |
| rm_work | they were essentially testing nothing | 02:15 |
| rm_work | you'll see a lot of the files that i heavily edited, those went down to ... not bothering | 02:15 |
| rm_work | because checking statuses isn't useful | 02:15 |
| rm_work | so i just clipped it out | 02:15 |
| rm_work | except where i was checking specifically if things went to pending | 02:16 |
| rm_work | but nothing can possibly go to ACTIVE/ONLINE legitimately in functional tests because there's no worker, lol | 02:16 |
| johnsom | Right, exactly, they should all be checking for pending | 02:17 |
| *** aojea has quit IRC | 02:25 | |
| johnsom | rm_work SoB, yep, it's broken. Just confirmed on my stack | 02:26 |
| *** aojea has joined #openstack-lbaas | 02:26 | |
| *** aojea has quit IRC | 02:26 | |
| rm_work | T_T | 02:26 |
| rm_work | johnsom: i wonder what happened... are we typoing the checktype and haproxy still accepts the config? | 02:27 |
| rm_work | should it be "httpchk" or "httpcheck"? | 02:28 |
| johnsom | Wait, no, it is working | 02:28 |
| rm_work | nope, httpchk is right... | 02:28 |
| rm_work | hmmm | 02:28 |
| johnsom | Yeah, just confirmed, it is working for me, I see HTTP hits | 02:29 |
| johnsom | https://www.irccloud.com/pastebin/Eccu7t7d/ | 02:30 |
| rm_work | hmm | 02:31 |
| rm_work | ubuntu amps? | 02:32 |
| rm_work | i'm on centos amps? different haproxy? | 02:32 |
| johnsom | Yes | 02:32 |
| rm_work | HA-Proxy version 1.5.18 2016/05/10 | 02:33 |
| rm_work | <_< | 02:33 |
| rm_work | >_> | 02:33 |
| rm_work | <_< | 02:33 |
| rm_work | that doesn't look right | 02:33 |
| johnsom | Well, that is old, but | 02:33 |
| rm_work | yeah i mean | 02:33 |
| rm_work | it shouldn't break this | 02:33 |
| rm_work | but it *is* wrong | 02:33 |
| rm_work | centos doesn't have 1.6 i guess? | 02:34 |
| rm_work | need to make sure we have the ability to run a custom haproxy version I suppose... :/ | 02:34 |
| johnsom | ii haproxy 1.6.3-1ubuntu0.1 | 02:34 |
| johnsom | Which is still old, but better | 02:34 |
| rm_work | yeah... | 02:35 |
| johnsom | Ok, I am going to give up on having functional tests for the LB policy stuff done today (they work, though the tests are bogus) | 02:38 |
| johnsom | I was just sitting here puzzled why the heck you would have "set" methods in assert wrappers.... | 02:39 |
| johnsom | Oh.... | 02:39 |
| rm_work | rofl | 02:39 |
| rm_work | yeah it's dumb | 02:39 |
| rm_work | i removed them as I saw them | 02:39 |
| rm_work | but there's a ton | 02:39 |
| rm_work | because it all got copy/pasted | 02:39 |
| rm_work | literally 7 times | 02:39 |
| johnsom | Yeah, you aren't kidding. I fixed one and now 14 tests blow up. I'm sure that is just the start. | 02:40 |
| johnsom | Tomorrow for that fun | 02:40 |
| rm_work | whelp.... https://github.com/DBezemer/rpm-haproxy | 02:43 |
| rm_work | maybe that | 02:43 |
| rm_work | lol | 02:43 |
| rm_work | AFAICT centos is dead | 02:43 |
| rm_work | i don't understand why people use it | 02:43 |
| rm_work | but i'm being forced to | 02:43 |
| rm_work | (not for any good reason) | 02:43 |
| rm_work | at least fedora has 1.7 | 02:44 |
| rm_work | https://haproxy.debian.net/#?distribution=Ubuntu&release=xenial&version=1.7 | 02:44 |
| rm_work | so does debuntu | 02:44 |
| rm_work | @*($&@* Debian stable has 1.7 in its base repos rofl | 02:45 |
| rm_work | if *debian stable* has significantly newer packages that your distro, your distro is dead and buried | 02:45 |
| *** sanfern has joined #openstack-lbaas | 02:50 | |
| *** gcheresh has joined #openstack-lbaas | 03:12 | |
| *** catintheroof has joined #openstack-lbaas | 03:15 | |
| *** gans has joined #openstack-lbaas | 03:29 | |
| *** catintheroof has quit IRC | 03:47 | |
| *** cody-somerville has joined #openstack-lbaas | 04:32 | |
| *** csomerville has quit IRC | 04:35 | |
| *** gcheresh has quit IRC | 05:02 | |
| *** dementor has joined #openstack-lbaas | 05:06 | |
| dementor | hello can somebody help me to using load balancer v2 octavia, i create so difficult | 05:07 |
| dementor | i can't find the practice or example load balancer in dashboard | 05:08 |
| gans | dementor, i am not sure from dashboard, but you can see this to create LB via CLI , https://docs.openstack.org/developer/devstack/guides/devstack-with-lbaas-v2.html | 05:25 |
| *** blogan_ has joined #openstack-lbaas | 05:31 | |
| dementor | that same for LB v2??? | 05:31 |
| dementor | oke I'll try | 05:32 |
| dementor | thx you gans | 05:32 |
| gans | dementor, yeah for lb v2 | 05:33 |
| *** blogan has quit IRC | 05:33 | |
| *** krypto has joined #openstack-lbaas | 05:45 | |
| *** blogan_ has quit IRC | 05:48 | |
| *** rcernin has quit IRC | 06:11 | |
| *** diltram has quit IRC | 06:13 | |
| *** diltram has joined #openstack-lbaas | 06:18 | |
| *** rcernin has joined #openstack-lbaas | 06:37 | |
| *** JudeC has joined #openstack-lbaas | 06:41 | |
| *** kobis has joined #openstack-lbaas | 06:42 | |
| *** tesseract has joined #openstack-lbaas | 06:43 | |
| *** diltram has quit IRC | 06:47 | |
| *** armax has quit IRC | 06:49 | |
| *** kobis has quit IRC | 06:53 | |
| *** diltram has joined #openstack-lbaas | 06:53 | |
| *** diltram has quit IRC | 06:57 | |
| *** pcaruana has joined #openstack-lbaas | 06:58 | |
| *** krypto has quit IRC | 07:03 | |
| *** diltram has joined #openstack-lbaas | 07:09 | |
| *** armax has joined #openstack-lbaas | 07:10 | |
| *** diltram has quit IRC | 07:13 | |
| *** armax has quit IRC | 07:15 | |
| *** diltram has joined #openstack-lbaas | 07:17 | |
| *** kobis has joined #openstack-lbaas | 07:36 | |
| *** kobis1 has joined #openstack-lbaas | 07:39 | |
| *** kobis has quit IRC | 07:40 | |
| dementor | hello why cannot delete the load balance on dashboard | 07:51 |
| *** aojea has joined #openstack-lbaas | 07:54 | |
| *** aojea has quit IRC | 08:01 | |
| *** JudeC has quit IRC | 08:02 | |
| *** armax has joined #openstack-lbaas | 08:11 | |
| *** dosaboy has quit IRC | 08:11 | |
| *** dementor has quit IRC | 08:14 | |
| *** armax has quit IRC | 08:16 | |
| *** dosaboy has joined #openstack-lbaas | 08:19 | |
| *** rstarmer_ has joined #openstack-lbaas | 08:34 | |
| *** rstarmer_ has quit IRC | 08:41 | |
| *** rstarmer_ has joined #openstack-lbaas | 08:42 | |
| *** dosaboy has quit IRC | 08:50 | |
| *** dosaboy has joined #openstack-lbaas | 08:51 | |
| *** dosaboy has quit IRC | 08:53 | |
| *** dosaboy has joined #openstack-lbaas | 09:08 | |
| nmagnezi | rm_work, I wonder how many more rechecks.. | 09:11 |
| nmagnezi | O_o | 09:11 |
| *** armax has joined #openstack-lbaas | 09:12 | |
| *** armax has quit IRC | 09:17 | |
| *** rstarmer_ has quit IRC | 09:19 | |
| *** rstarmer_ has joined #openstack-lbaas | 09:21 | |
| *** rstarmer_ has quit IRC | 10:12 | |
| *** armax has joined #openstack-lbaas | 10:12 | |
| *** rstarmer_ has joined #openstack-lbaas | 10:16 | |
| *** armax has quit IRC | 10:17 | |
| *** sanfern has quit IRC | 10:53 | |
| *** gans has quit IRC | 10:53 | |
| *** rstarmer_ has quit IRC | 10:55 | |
| *** armax has joined #openstack-lbaas | 11:13 | |
| *** Dave____ is now known as Dave | 11:14 | |
| *** armax has quit IRC | 11:18 | |
| *** chlong has joined #openstack-lbaas | 11:32 | |
| *** armax has joined #openstack-lbaas | 12:14 | |
| *** armax has quit IRC | 12:19 | |
| *** atoth has joined #openstack-lbaas | 12:40 | |
| *** sanfern has joined #openstack-lbaas | 12:47 | |
| *** leitan has joined #openstack-lbaas | 12:59 | |
| *** catintheroof has joined #openstack-lbaas | 13:07 | |
| *** armax has joined #openstack-lbaas | 13:13 | |
| *** armax has quit IRC | 13:17 | |
| *** catinthe_ has joined #openstack-lbaas | 13:21 | |
| *** catintheroof has quit IRC | 13:24 | |
| *** cpuga has joined #openstack-lbaas | 13:25 | |
| *** catintheroof has joined #openstack-lbaas | 13:25 | |
| *** catinthe_ has quit IRC | 13:25 | |
| *** catinthe_ has joined #openstack-lbaas | 13:26 | |
| *** catinth__ has joined #openstack-lbaas | 13:26 | |
| *** catintheroof has quit IRC | 13:26 | |
| *** cpuga has quit IRC | 13:26 | |
| *** cpuga has joined #openstack-lbaas | 13:27 | |
| *** catintheroof has joined #openstack-lbaas | 13:29 | |
| *** catinthe_ has quit IRC | 13:30 | |
| *** catinth__ has quit IRC | 13:33 | |
| *** mariusv has joined #openstack-lbaas | 13:33 | |
| *** mariusv has quit IRC | 13:55 | |
| xgerman_ | o/ | 13:56 |
| *** aojea has joined #openstack-lbaas | 14:06 | |
| korean101 | johnsom: are you there? | 14:34 |
| korean101 | you probably 14:34 time | 14:35 |
| *** kobis1 has quit IRC | 14:51 | |
| *** reedip_ has joined #openstack-lbaas | 14:54 | |
| *** fnaval has quit IRC | 14:55 | |
| *** fnaval has joined #openstack-lbaas | 14:55 | |
| *** rstarmer has joined #openstack-lbaas | 14:59 | |
| *** fnaval has quit IRC | 15:00 | |
| openstackgerrit | Merged openstack/neutron-lbaas master: Remove usage of parameter enforce_type https://review.openstack.org/474062 | 15:07 |
| *** rstarmer has quit IRC | 15:09 | |
| *** rcernin has quit IRC | 15:09 | |
| johnsom | o/ | 15:10 |
| *** fnaval has joined #openstack-lbaas | 15:19 | |
| *** fnaval has quit IRC | 15:25 | |
| *** fnaval has joined #openstack-lbaas | 15:26 | |
| *** rcernin has joined #openstack-lbaas | 15:44 | |
| *** dayou has quit IRC | 15:51 | |
| *** blogan has joined #openstack-lbaas | 15:55 | |
| *** aojea has quit IRC | 15:56 | |
| *** reedip_ has quit IRC | 15:56 | |
| *** rcernin has quit IRC | 16:12 | |
| *** SumitNaiksatam has joined #openstack-lbaas | 16:15 | |
| *** diltram has quit IRC | 16:18 | |
| *** diltram has joined #openstack-lbaas | 16:21 | |
| *** diltram has quit IRC | 16:26 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/neutron-lbaas master: Updated from global requirements https://review.openstack.org/474671 | 16:28 |
| *** diltram has joined #openstack-lbaas | 16:30 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/octavia master: Updated from global requirements https://review.openstack.org/474674 | 16:30 |
| *** cody-somerville has quit IRC | 16:31 | |
| *** cody-somerville has joined #openstack-lbaas | 16:31 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Add RBAC enforcement to Octavia v2 API https://review.openstack.org/472872 | 16:39 |
| johnsom | ^^^ WIP - I will probably be check-pointing a few times today. That one is LB basically done with functional tests. I need to incorporate the new policy idea I pitched yesterday and then start turning the crank for the rest of the API. | 16:44 |
| *** tesseract has quit IRC | 16:46 | |
| *** sshank has joined #openstack-lbaas | 16:55 | |
| rm_work | nmagnezi: our whole gate system is really bonkers | 17:06 |
| rm_work | nmagnezi: we have so many checks, and they're unreliable enough again for various reasons, that 75% of the time or so at least one will fail >_> | 17:06 |
| *** csomerville has joined #openstack-lbaas | 17:10 | |
| *** JudeC has joined #openstack-lbaas | 17:11 | |
| *** cody-somerville has quit IRC | 17:13 | |
| johnsom | rm_work the only intermittent test failure I am aware of is the gunicorn 404 issue. Are you aware of others? | 17:15 |
| *** pcaruana has quit IRC | 17:16 | |
| *** greghaynes is now known as greghayn1 | 17:18 | |
| *** greghaynes has joined #openstack-lbaas | 17:18 | |
| *** greghaynes has quit IRC | 17:22 | |
| *** greghaynes has joined #openstack-lbaas | 17:23 | |
| *** SumitNaiksatam has quit IRC | 17:23 | |
| rm_work | johnsom: i was going to look today | 17:24 |
| rm_work | some stuff appears to be failing during devstack setup... | 17:24 |
| *** greghayn1 has quit IRC | 17:25 | |
| *** SumitNaiksatam has joined #openstack-lbaas | 17:25 | |
| rm_work | http://logs.openstack.org/18/474318/2/gate/gate-octavia-v1-dsvm-scenario-ubuntu-xenial/9c55621/logs/devstacklog.txt.gz#_2017-06-15_00_48_45_130 | 17:26 |
| rm_work | wtf happened there | 17:26 |
| johnsom | My two first thoughts, the test host is out of disk or I know DIB had recently made changes to the block device stuff. | 17:28 |
| rm_work | >_> | 17:28 |
| johnsom | Well, the host looks fine for disk | 17:29 |
| johnsom | Would be nice if DIB captured the error stream. It probably would if we turning on it's debug, but ouch, talk about a flood of log messages | 17:30 |
| rm_work | yeah :/ | 17:30 |
| *** JudeC has quit IRC | 17:30 | |
| rm_work | ugh why can't i run coverage tests inside pycharm | 17:30 |
| rm_work | blegh | 17:30 |
| *** SumitNaiksatam has quit IRC | 17:31 | |
| *** SumitNaiksatam has joined #openstack-lbaas | 17:35 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Small refactor for load_balancer v2 vip validation https://review.openstack.org/474711 | 17:35 |
| rm_work | johnsom: ^^ that is tiny and probably meaningless in general but would appreciate merging it quickly, will save me a lot of work | 17:35 |
| johnsom | rm_work You owe me if I merge that. It's going to likely merge conflict with the RBAC patch. | 17:38 |
| rm_work | is it? lol | 17:39 |
| rm_work | hmmmm | 17:39 |
| rm_work | what if i rebase for you? :P | 17:39 |
| johnsom | No worries, just giving you a hard time. | 17:39 |
| johnsom | Oh, geez, is that mkfs issue what blocked the GR update? | 17:41 |
| johnsom | No, broken mirrors: | 17:42 |
| johnsom | is_admin:True | 17:42 |
| johnsom | E: Unable to locate package python-pip | 17:43 |
| johnsom | 2017-06-15 17:04:30.505 | E: Package 'python3-pip' has no installation candidate | 17:43 |
| johnsom | 2017-06-15 17:04:30.505 | E: Package 'python-virtualenv' has no installation candidate | 17:43 |
| johnsom | 2017-06-15 17:04:30.505 | E: Unable to locate package python3-virtualenv | 17:43 |
| johnsom | Oh, and plus: | 17:44 |
| johnsom | Connected to amphora. Response: <Response [404]> request /opt/stack/new/octavia/octavia/amphorae/drivers/haproxy/rest_api_driver.py:280 | 17:44 |
| rm_work | >_< | 17:44 |
| johnsom | Wow, and mkfs. There gate tests, all failed in different ways.... Though only one is our own code | 17:45 |
| rm_work | this is what i'm talking about T_T | 17:48 |
| rm_work | ugh | 17:48 |
| *** JudeC has joined #openstack-lbaas | 17:48 | |
| *** JudeC has quit IRC | 17:53 | |
| johnsom | The 404 is the one that kills me, pretty random | 17:55 |
| *** JudeC has joined #openstack-lbaas | 18:15 | |
| *** JudeC has quit IRC | 18:19 | |
| *** rstarmer has joined #openstack-lbaas | 18:25 | |
| *** fnaval has quit IRC | 18:30 | |
| rm_work | yeah i still don't understand that | 18:30 |
| rm_work | might need to track down an expert or something | 18:30 |
| rm_work | gunicorn expert? | 18:30 |
| rm_work | wsgi expert? i dunno | 18:30 |
| rm_work | I'm asking in #gunicorn | 18:33 |
| *** rstarmer has quit IRC | 18:36 | |
| *** JudeC has joined #openstack-lbaas | 18:36 | |
| *** rstarmer has joined #openstack-lbaas | 18:40 | |
| johnsom | I am almost thinking we need to put some ugly debug code in that when this situation triggers we scp the amp logs off the amp before it gets deleted in the revert. | 18:42 |
| rm_work | i'm considering other stupidly hacky workarounds as well | 18:42 |
| rm_work | change the default 404 page in the agent, so once the agent is loaded up, a real 404 is distinct via the body? | 18:44 |
| johnsom | Currently that code to connect is shared across a lot of the functions. That is why I have been reluctant to up the retry count. | 18:46 |
| rm_work | right | 18:46 |
| rm_work | but if we made the 404 distinct | 18:46 |
| rm_work | we could tell if it's a real 404, or a "not yet loaded 404" | 18:46 |
| johnsom | If we are going super hacky fix style, I guess we could create a "special" code path just for that initial connect. | 18:46 |
| rm_work | because it's more than just a HTTP code | 18:47 |
| rm_work | 404s have a body | 18:47 |
| rm_work | we can scan that | 18:47 |
| johnsom | It just seems like that would impact the other API calls and points in time.... | 18:47 |
| rm_work | why | 18:47 |
| johnsom | Hmm, so agent custom 404 means it's really not there for all API calls, non custom 404 means not started? | 18:48 |
| rm_work | yes | 18:48 |
| rm_work | conveniently using the failure to load to our advantage :P | 18:49 |
| johnsom | This is making me ill right before lunch. I think I might be willing to add the hacky debug code first and try to figure out the "why" part | 18:49 |
| rm_work | lol | 18:49 |
| rm_work | i might ... do the custom 404 | 18:49 |
| johnsom | For all I know this is some stupid gate environment issue where our connections are being routed wrong (proxy server or some such mess) or there is something else listening on the IP. | 18:50 |
| johnsom | I have never seen it local | 18:50 |
| johnsom | Though I haven't done a CHO local in a while. | 18:50 |
| rm_work | well | 18:51 |
| rm_work | yeah me either lol | 18:51 |
| rm_work | but my solution would still work :P | 18:51 |
| rm_work | though also i wish i could get rid of Flask completely | 18:51 |
| rm_work | i had some progress on that | 18:51 |
| johnsom | This is one of these things that 6 months later you are going to look at and go "Ugh, why????" and I will remind you it was your idea. Grin | 18:51 |
| johnsom | Ok, I need to run an errand over lunch, back in around an hour. | 18:52 |
| *** sanfern has quit IRC | 18:53 | |
| *** sanfern has joined #openstack-lbaas | 18:53 | |
| rm_work | johnsom: lol BTW, we already DO have a "get_haproxy_config", I thought that was the case | 19:00 |
| rm_work | on the agent | 19:00 |
| johnsom | Yeah, I could not remember if it got implemented or not. | 19:02 |
| *** armax has joined #openstack-lbaas | 19:03 | |
| *** sshank has quit IRC | 19:04 | |
| xgerman_ | mmh, if it’s not up it should do something like 503 but… | 19:05 |
| *** fnaval has joined #openstack-lbaas | 19:06 | |
| *** catinthe_ has joined #openstack-lbaas | 19:07 | |
| rm_work | johnsom: ummmmmm | 19:08 |
| rm_work | johnsom: i think we might already be doing that rofl | 19:08 |
| rm_work | i think we have custom error pages all around | 19:08 |
| rm_work | rofl | 19:09 |
| *** catintheroof has quit IRC | 19:11 | |
| *** catintheroof has joined #openstack-lbaas | 19:22 | |
| *** catinthe_ has quit IRC | 19:25 | |
| *** leitan has quit IRC | 20:06 | |
| *** leitan has joined #openstack-lbaas | 20:06 | |
| *** leitan has quit IRC | 20:11 | |
| *** rstarmer has quit IRC | 20:31 | |
| *** JudeC has quit IRC | 20:47 | |
| *** JudeC has joined #openstack-lbaas | 20:49 | |
| *** JudeC has quit IRC | 20:53 | |
| *** catinthe_ has joined #openstack-lbaas | 21:06 | |
| *** catintheroof has quit IRC | 21:09 | |
| *** dayou has joined #openstack-lbaas | 21:14 | |
| *** fnaval has quit IRC | 21:23 | |
| *** catinthe_ has quit IRC | 21:28 | |
| rm_work | johnsom: ok i'm going to post a workaround because this is stupid | 21:42 |
| rm_work | working out the mock issues for testing tho | 21:42 |
| johnsom | Ok, I will have a look. | 21:42 |
| rm_work | k got it i think, it's hard to test for sure since, well, can't repro | 21:50 |
| rm_work | but | 21:50 |
| rm_work | i can recheck this a few times and hopefully not get any fails | 21:50 |
| rm_work | unfortunately it's all bundled in with the webob work i was doing, but i can pull it out if we need to | 21:51 |
| *** cpuga has quit IRC | 21:53 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Agent: swap flask responses to webob, handle 404 retries better https://review.openstack.org/474790 | 21:54 |
| rm_work | johnsom: ^^ so the real change is https://review.openstack.org/#/c/474790/1/octavia/amphorae/drivers/haproxy/rest_api_driver.py@283 | 21:54 |
| johnsom | Yeah, already looking at it. | 21:54 |
| johnsom | How do you know the fake 404 isn't returning json? | 21:55 |
| rm_work | because that's insane | 21:55 |
| rm_work | nothing does that by default | 21:55 |
| johnsom | Hahaa | 21:55 |
| rm_work | we have to override to make everything json specifically | 21:55 |
| rm_work | we do it here: | 21:55 |
| johnsom | Leave it to us to find the counter example..... | 21:55 |
| rm_work | https://github.com/openstack/octavia/blob/master/octavia/amphorae/backends/agent/api_server/server.py#L32-L35 | 21:56 |
| rm_work | :P | 21:56 |
| rm_work | the work was already done for me | 21:56 |
| rm_work | lol | 21:56 |
| rm_work | but to be explicit: https://review.openstack.org/#/c/474790/1/octavia/amphorae/backends/agent/api_server/server.py@36 | 21:56 |
| rm_work | actually i don't need one of those lines, sec | 21:57 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Agent: swap flask responses to webob, handle 404 retries better https://review.openstack.org/474790 | 21:57 |
| rm_work | i didn't have json= when i was testing | 21:57 |
| rm_work | but it does the content-type automatically with that | 21:58 |
| rm_work | so this does a ton more towards removing flask | 21:59 |
| rm_work | then I just need to figure out what to use to do routing | 21:59 |
| rm_work | though maybe we can just leave Flask for that <_< | 21:59 |
| rm_work | very minimal | 21:59 |
| rm_work | JUST use its routing engine | 21:59 |
| rm_work | though I have a feeling the issue is related | 21:59 |
| rm_work | i think Flask is where the bug is | 21:59 |
| johnsom | So, this will retry ten times for any amp API call that comes back 404. | 22:00 |
| rm_work | ONLY if it's non-json | 22:00 |
| rm_work | which will ONLY be if it isn't spun up yet | 22:01 |
| rm_work | because once our API is actually loaded and responding, 404s will be json | 22:01 |
| johnsom | Only if it is json... | 22:01 |
| rm_work | errr | 22:01 |
| rm_work | i did it backwards lol one sec | 22:02 |
| rm_work | good catch :P | 22:02 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Agent: swap flask responses to webob, handle 404 retries better https://review.openstack.org/474790 | 22:02 |
| rm_work | forgot a "not" | 22:02 |
| johnsom | If we are putting hacks in I'm going to look at it closely.... | 22:02 |
| johnsom | grin | 22:02 |
| rm_work | yes :P | 22:02 |
| rm_work | feel free | 22:02 |
| rm_work | to pick this apart | 22:02 |
| rm_work | but I THINK it'll work | 22:02 |
| rm_work | also I think removing the stupid flask jsonify stuff and moving to webob is ++ | 22:03 |
| rm_work | ugh forgot to put it in requirements tho >_< one more patch | 22:03 |
| rm_work | oh nm it's there | 22:04 |
| rm_work | i guess we already had it for some reason | 22:04 |
| johnsom | Yeah, so no test that fakes up a non-json 404 or two? | 22:08 |
| rm_work | ugh | 22:08 |
| rm_work | sometimes .... i just... >_> | 22:09 |
| * rm_work sighs | 22:09 | |
| rm_work | also the other tests are wrong now rofl | 22:11 |
| johnsom | Otherwise I think it looks ok. Just curious about the unit test coverage and if we can put in a functional that tries that out. | 22:12 |
| rm_work | ah i did it wrong anyway | 22:13 |
| rm_work | find is -1 / 0 | 22:13 |
| rm_work | which ... | 22:13 |
| rm_work | both are false | 22:13 |
| rm_work | lol | 22:13 |
| johnsom | Just to capture my comment | 22:14 |
| rm_work | ok there you go | 22:15 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Agent: swap flask responses to webob, handle 404 retries better https://review.openstack.org/474790 | 22:15 |
| rm_work | A) fixed it | 22:15 |
| rm_work | B) added a negative | 22:15 |
| rm_work | really i just want to see if it passes | 22:16 |
| *** KeithMnemonic has quit IRC | 22:16 | |
| rm_work | johnsom: we seriously can't get a check to pass today | 22:18 |
| rm_work | sorry, ONE of 10 did | 22:18 |
| rm_work | <_< | 22:18 |
| johnsom | Yeah, I'm working on the mkfs thing in the DIB channel | 22:22 |
| rm_work | k | 22:23 |
| johnsom | My WIP patch passed though: https://review.openstack.org/#/c/472872/ LOL | 22:23 |
| rm_work | but that's also intermittent | 22:23 |
| rm_work | lol. great. | 22:23 |
| johnsom | Yeah I think Ian is on the right track with the mkfs thing | 22:23 |
| rm_work | wait what is the dib channel | 22:24 |
| rm_work | am i somehow not in that | 22:24 |
| rm_work | johnsom: ^^ | 22:27 |
| johnsom | openstack-dib | 22:29 |
| *** fnaval has joined #openstack-lbaas | 22:29 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Enable DIB trace logging https://review.openstack.org/474800 | 22:43 |
| rm_work | johnsom: that? | 22:43 |
| johnsom | Yes, I think so | 22:44 |
| johnsom | Though, it might have to be a: export DEVSTACK_LOCAL_CONFIG+=$'\n'"DISABLE_AMP_IMAGE_BUILD=True"$'\n' | 22:44 |
| rm_work | lol | 22:45 |
| rm_work | whelp | 22:45 |
| johnsom | I think it does actually have to get shoved in the local.conf | 22:45 |
| rm_work | k | 22:45 |
| johnsom | The scoping is nasty for this stuff. | 22:45 |
| johnsom | Still OCTAVIA_DIB_TRACING=1 obviously | 22:46 |
| rm_work | well | 22:47 |
| rm_work | True works | 22:47 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Enable DIB trace logging https://review.openstack.org/474800 | 22:47 |
| rm_work | the plugin looks for !=0 | 22:47 |
| *** fnaval has quit IRC | 22:59 | |
| rm_work | lol we have so many broken tests | 23:55 |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!