| *** mithilarun has joined #openstack-lbaas | 00:56 | |
| *** mithilarun has quit IRC | 01:01 | |
| rm_work | sorrison: I think I got the main two, test out my two patches in your env if you would | 01:11 |
|---|---|---|
| *** Dinesh_Bhor has quit IRC | 01:28 | |
| *** Dinesh_Bhor has joined #openstack-lbaas | 01:34 | |
| *** mithilarun has joined #openstack-lbaas | 01:50 | |
| *** vishalmanchanda has joined #openstack-lbaas | 01:56 | |
| *** ramishra has joined #openstack-lbaas | 02:05 | |
| *** mithilarun has quit IRC | 02:09 | |
| *** mithilarun has joined #openstack-lbaas | 02:16 | |
| *** mithilarun has quit IRC | 02:23 | |
| *** mithilarun has joined #openstack-lbaas | 02:24 | |
| *** rcernin has joined #openstack-lbaas | 02:27 | |
| *** mithilarun has quit IRC | 02:31 | |
| *** mithilarun has joined #openstack-lbaas | 03:24 | |
| *** mithilarun has quit IRC | 03:29 | |
| *** rcernin has quit IRC | 03:31 | |
| *** mithilarun has joined #openstack-lbaas | 03:37 | |
| *** mithilarun has quit IRC | 03:41 | |
| *** psachin has joined #openstack-lbaas | 04:29 | |
| *** hongbin has joined #openstack-lbaas | 05:23 | |
| *** hongbin has quit IRC | 06:16 | |
| *** gcheresh_ has joined #openstack-lbaas | 06:33 | |
| *** ccamposr has joined #openstack-lbaas | 06:51 | |
| *** numans has joined #openstack-lbaas | 07:20 | |
| *** AlexStaf has joined #openstack-lbaas | 07:26 | |
| *** tkajinam has quit IRC | 07:36 | |
| *** tkajinam has joined #openstack-lbaas | 07:38 | |
| *** pcaruana has joined #openstack-lbaas | 07:43 | |
| *** mithilarun has joined #openstack-lbaas | 07:45 | |
| *** mithilarun has quit IRC | 07:50 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 07:59 | |
| *** tkajinam_ has joined #openstack-lbaas | 08:01 | |
| *** tkajinam has quit IRC | 08:04 | |
| *** rpittau|afk is now known as rpittau | 08:05 | |
| *** gcheresh_ has quit IRC | 08:07 | |
| *** gcheresh has joined #openstack-lbaas | 08:07 | |
| *** luksky has joined #openstack-lbaas | 08:09 | |
| *** tesseract has joined #openstack-lbaas | 08:14 | |
| *** maciejjozefczyk_ has joined #openstack-lbaas | 08:14 | |
| *** devfaz has quit IRC | 08:14 | |
| *** devfaz has joined #openstack-lbaas | 08:15 | |
| *** maciejjozefczyk has quit IRC | 08:17 | |
| *** yboaron has joined #openstack-lbaas | 08:25 | |
| *** yboaron_ has joined #openstack-lbaas | 08:30 | |
| *** yboaron has quit IRC | 08:33 | |
| *** yboaron_ has quit IRC | 08:37 | |
| *** yboaron_ has joined #openstack-lbaas | 08:37 | |
| *** maciejjozefczyk_ has quit IRC | 08:37 | |
| *** tkajinam_ has quit IRC | 08:38 | |
| *** maciejjozefczyk_ has joined #openstack-lbaas | 09:14 | |
| *** gcheresh has quit IRC | 09:17 | |
| *** gcheresh has joined #openstack-lbaas | 09:19 | |
| *** maciejjozefczyk_ has quit IRC | 09:42 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 09:43 | |
| *** mithilarun has joined #openstack-lbaas | 10:02 | |
| *** mithilarun has quit IRC | 10:08 | |
| *** gcheresh has quit IRC | 10:23 | |
| *** gcheresh has joined #openstack-lbaas | 10:24 | |
| *** gcheresh_ has joined #openstack-lbaas | 10:35 | |
| *** gcheresh has quit IRC | 10:35 | |
| *** tkajinam has joined #openstack-lbaas | 11:28 | |
| *** yboaron_ has quit IRC | 11:35 | |
| *** rpittau is now known as rpittau|bbl | 11:42 | |
| *** luksky has quit IRC | 11:52 | |
| *** yboaron_ has joined #openstack-lbaas | 11:59 | |
| *** vishalmanchanda has quit IRC | 12:12 | |
| *** luksky has joined #openstack-lbaas | 12:17 | |
| *** ramishra has quit IRC | 12:51 | |
| *** mithilarun has joined #openstack-lbaas | 12:54 | |
| *** mithilarun has quit IRC | 12:58 | |
| *** ramishra has joined #openstack-lbaas | 13:10 | |
| *** psachin has quit IRC | 13:16 | |
| *** yboaron_ has quit IRC | 13:19 | |
| *** rpittau|bbl is now known as rpittau | 13:34 | |
| *** vishalmanchanda has joined #openstack-lbaas | 13:43 | |
| *** tkajinam has quit IRC | 14:16 | |
| *** TrevorV has joined #openstack-lbaas | 14:33 | |
| *** yboaron_ has joined #openstack-lbaas | 14:34 | |
| *** AlexStaf has quit IRC | 15:04 | |
| brtknr | When is octavia going to get cli option to reset loadbalancer state | 15:17 |
| brtknr | at the moment, if the loadbalancer gets stuck in pending_update state, the only way to delete the entry is via manual db hacking | 15:17 |
| johnsom | Never, state is managed by the controllers | 15:17 |
| brtknr | how do i get the controllers to get a loadbalancer out of pending_update state? | 15:18 |
| brtknr | johnsom: | 15:18 |
| johnsom | PENDING_* means one of the controllers has taken ownership of that object and is actively working on the object. This can include retrying actions against other cloud resources that have failed. | 15:18 |
| brtknr | is there a way to intervene? abort? | 15:19 |
| johnsom | At the end of these retries, either the object will go ACTIVE or the controller will move it to the ERROR state. | 15:19 |
| *** mithilarun has joined #openstack-lbaas | 15:19 | |
| brtknr | well, mine has been stuck in PENDING_UPDATE for several weeks | 15:19 |
| johnsom | You may want to tune your retry timeouts to me something shorter in your environment. | 15:20 |
| brtknr | where is the controller running? | 15:20 |
| johnsom | [haproxy_amphora] connection_max_retries = 120 and [haproxy_amphora] build_active_retries = 120 and [networking] port_detach_timeout = 1800 for example. | 15:20 |
| johnsom | Also, be sure to use graceful shutdown, not kill -9. There is an open bug we are working on at the moment if you kill -9 a controller. | 15:21 |
| brtknr | ah so controller is the service running inside the amphorae? | 15:22 |
| johnsom | This includes any systemd scripts you may have created that might have short timeouts and kill -9's things | 15:22 |
| johnsom | No, not inside the amphorae | 15:22 |
| johnsom | The controllers are the processes running on your control plane. The worker, health manager, and housekeeping processes | 15:23 |
| *** mithilarun has quit IRC | 15:24 | |
| *** mloza has joined #openstack-lbaas | 15:25 | |
| brtknr | ah okay | 15:26 |
| brtknr | johnsom: thanks! | 15:27 |
| johnsom | NP | 15:27 |
| *** armax has joined #openstack-lbaas | 15:31 | |
| *** mithilarun has joined #openstack-lbaas | 15:33 | |
| *** mithilarun has quit IRC | 15:43 | |
| *** gcheresh_ has quit IRC | 15:43 | |
| *** yboaron_ has quit IRC | 15:52 | |
| *** gcheresh_ has joined #openstack-lbaas | 15:53 | |
| *** mithilarun has joined #openstack-lbaas | 15:55 | |
| *** mithilarun has quit IRC | 16:00 | |
| *** mithilarun has joined #openstack-lbaas | 16:01 | |
| *** yboaron_ has joined #openstack-lbaas | 16:07 | |
| *** nicolasbock has joined #openstack-lbaas | 16:10 | |
| dulek | Hi! Any issue with Amphora building in stable/train? We're observing them in kuryr-kubernetes gates. | 16:14 |
| dulek | Looks like the culprit is this error: "ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output." - happens after /opt/amphora-agent is tried to be installed. | 16:15 |
| dulek | Failing build: https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_2c0/703505/1/check/kuryr-kubernetes-tempest/2c0bb93/controller/logs/dib-build/amphora-x64-haproxy.qcow2_log.txt. | 16:15 |
| johnsom | dulek Not that I know of. We just merged a patch on stable/train Friday, so image builds were working fine then. | 16:22 |
| johnsom | The nightly builds against master are all working ok too. | 16:23 |
| johnsom | Looking at the log now | 16:23 |
| johnsom | dulek It could be an infra mirror failure at limestone, did you try a recheck? | 16:32 |
| *** luksky has quit IRC | 16:32 | |
| *** rpittau is now known as rpittau|afk | 16:33 | |
| *** gcheresh_ has quit IRC | 16:35 | |
| *** vishalmanchanda has quit IRC | 16:42 | |
| *** mithilarun has quit IRC | 16:45 | |
| *** ramishra has quit IRC | 16:48 | |
| *** ccamposr has quit IRC | 16:49 | |
| *** spatel has joined #openstack-lbaas | 17:06 | |
| *** yboaron_ has quit IRC | 17:07 | |
| *** yboaron_ has joined #openstack-lbaas | 17:08 | |
| openstackgerrit | Gregory Thiemonge proposed openstack/octavia stable/queens: Fix house keeping graceful shutdown https://review.opendev.org/705223 | 17:09 |
| *** spatel has quit IRC | 17:12 | |
| *** yboaron_ has quit IRC | 17:13 | |
| rm_work | brtknr: yeah given "several weeks", almost assuredly it was a controller that died mid-workflow, and it'll be stuck forever at this point. You'll need to change the provisioning_status to ERROR in the DB. We are nearly done merging code that will allow jobs like that to resume if the controller working them dies -- should be done inside this cycle for the U release | 17:17 |
| *** mithilarun has joined #openstack-lbaas | 17:20 | |
| *** mithilarun has quit IRC | 17:35 | |
| *** yboaron_ has joined #openstack-lbaas | 17:35 | |
| *** yboaron_ has quit IRC | 17:39 | |
| *** gcheresh has joined #openstack-lbaas | 17:42 | |
| *** gcheresh has quit IRC | 17:59 | |
| *** gcheresh has joined #openstack-lbaas | 18:03 | |
| *** yboaron_ has joined #openstack-lbaas | 18:08 | |
| rm_work | johnsom / cgoncalves: https://review.opendev.org/#/c/698082/ eh? eh? eh? :D | 18:15 |
| cgoncalves | rm_work, remind me why this is needed? | 18:16 |
| rm_work | it's just dumb presently | 18:17 |
| rm_work | there's a hole in the amp boot process where amps that are *being booted for a specific LB* don't show up as being tied to that LB | 18:17 |
| rm_work | it's nonsensical | 18:17 |
| rm_work | especially when certain stuff breaks in the provisioning process, it makes things more difficult to troubleshoot | 18:18 |
| cgoncalves | ok, cosmetic change | 18:20 |
| rm_work | yeah i think all the stuff in the flow already pulls all its data from the required Loadbalancer object, not from the DB entry | 18:20 |
| rm_work | because previously it wasn't even linked in the DB :D | 18:21 |
| rm_work | maybe ataraday could remember if there was any other thing this solved, but i don't think so | 18:21 |
| rm_work | she wrote a patch to do this before I did, i just didn't notice and wrote it again a slightly different way that we decided to settle on for some reason | 18:22 |
| * rm_work shrugs | 18:22 | |
| rm_work | yeah i think her original change explained the reasoning better :D | 18:22 |
| cgoncalves | considering if it can and should be backported | 18:23 |
| rm_work | i think it can | 18:23 |
| rm_work | but whether it *matters*? >_> | 18:23 |
| rm_work | depending on other changes to flows, the tests might be annoying to cherry-pick back | 18:23 |
| cgoncalves | you said it's useful for troubleshooting so... | 18:23 |
| rm_work | but they might also *just work* too | 18:23 |
| rm_work | yes | 18:24 |
| rm_work | i've run into this a lot lately, when stuff dies before recording the LB-id on the amp in the DB T_T | 18:24 |
| *** gcheresh has quit IRC | 18:25 | |
| *** gcheresh has joined #openstack-lbaas | 18:29 | |
| cgoncalves | seems like it can be backported and given you such change is useful for troubleshooting, it should IMO which means it should have a release note and/or story. hmmmm | 18:30 |
| rm_work | augh | 18:30 |
| rm_work | ok | 18:31 |
| johnsom | I think I had a concern about that patch. Will look at it again | 18:31 |
| rm_work | while I'm doing that, you want to review https://review.opendev.org/699521 ? :D | 18:32 |
| rm_work | cgoncalves: "fixes" or "other"? :D | 18:34 |
| cgoncalves | rm_work, whatever as long as it states its valuable for troubleshooting? | 18:35 |
| rm_work | k | 18:35 |
| *** luksky has joined #openstack-lbaas | 18:37 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Update the lb_id on an amp earlier if we know it https://review.opendev.org/698082 | 18:41 |
| rm_work | added a release note and copied over ann's commit summary (and remembered to add her as a co-author this time) | 18:42 |
| rm_work | johnsom: adding tests. found a bug :D | 18:52 |
| *** gcheresh has quit IRC | 18:55 | |
| *** gcheresh has joined #openstack-lbaas | 18:59 | |
| rm_work | johnsom: ok well while i'm working on THAT... what about this one? https://review.opendev.org/#/c/702535/ | 19:01 |
| rm_work | I think I was more thorough with tests and doc | 19:01 |
| johnsom | I'm busy -1'ing 698082 | 19:01 |
| rm_work | lol | 19:01 |
| rm_work | y u do dis | 19:02 |
| johnsom | http://www.quickmeme.com/img/26/26b895dc7afc199f420cca84c67d11bbca418d9e3ba85fe90287fa0a07be4011.jpg | 19:08 |
| *** mithilarun has joined #openstack-lbaas | 19:11 | |
| johnsom | Ok, VIP as source is going to take some testing. You are out of review credits for today. Please come back tomorrow for another round of review roulette. | 19:14 |
| rm_work | lol k | 19:18 |
| rm_work | well i responded on your other one | 19:18 |
| rm_work | and am posting an update for another one | 19:18 |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Allow AZ to override valid_vip_networks config https://review.opendev.org/699521 | 19:22 |
| rm_work | update posted. added test, fixed bug, added releasenote, updated docs | 19:23 |
| *** ltomasbo has quit IRC | 19:50 | |
| *** tesseract has quit IRC | 20:01 | |
| *** andy_ has quit IRC | 20:02 | |
| *** irclogbot_3 has quit IRC | 20:05 | |
| *** irclogbot_2 has joined #openstack-lbaas | 20:06 | |
| *** andy_ has joined #openstack-lbaas | 20:23 | |
| *** mithilarun has quit IRC | 20:34 | |
| *** mithilarun has joined #openstack-lbaas | 20:34 | |
| *** mithilarun has quit IRC | 20:36 | |
| *** mithilarun has joined #openstack-lbaas | 20:37 | |
| *** mithilarun has quit IRC | 20:38 | |
| *** mithilarun has joined #openstack-lbaas | 20:39 | |
| *** maciejjozefczyk has quit IRC | 20:40 | |
| *** yboaron_ has quit IRC | 20:48 | |
| *** andy_ has quit IRC | 21:03 | |
| *** gcheresh has quit IRC | 21:08 | |
| *** mithilarun has quit IRC | 21:10 | |
| *** mithilarun has joined #openstack-lbaas | 21:11 | |
| *** andy_ has joined #openstack-lbaas | 21:21 | |
| *** nicolasbock has quit IRC | 21:29 | |
| *** mithilarun has quit IRC | 21:35 | |
| *** mithilarun has joined #openstack-lbaas | 21:36 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia stable/stein: Fix multi-listener LB with missing certificate https://review.opendev.org/698597 | 21:40 |
| *** mithilarun has quit IRC | 21:41 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia stable/stein: Fix multi-listener LB client auth/re-encryption https://review.opendev.org/698608 | 21:45 |
| *** mithilarun has joined #openstack-lbaas | 21:45 | |
| *** mithilarun has quit IRC | 22:02 | |
| *** mithilarun has joined #openstack-lbaas | 22:03 | |
| *** mithilarun has quit IRC | 22:07 | |
| *** mithilarun has joined #openstack-lbaas | 22:07 | |
| *** TrevorV has quit IRC | 22:53 | |
| *** tkajinam has joined #openstack-lbaas | 22:53 | |
| *** mithilarun has quit IRC | 22:58 | |
| *** mithilarun has joined #openstack-lbaas | 22:59 | |
| *** mithilarun has quit IRC | 23:03 | |
| *** mithilarun has joined #openstack-lbaas | 23:37 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!
