| *** sapd1_x has joined #openstack-lbaas | 00:01 | |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia master: Add Python 3.7 support https://review.openstack.org/635236 | 00:16 |
|---|---|---|
| openstackgerrit | Carlos Goncalves proposed openstack/octavia master: Add Python 3.7 support https://review.openstack.org/635236 | 00:22 |
| *** sapd1_x has quit IRC | 00:32 | |
| *** lxkong_ has joined #openstack-lbaas | 00:54 | |
| *** NobodyCam_ has joined #openstack-lbaas | 00:54 | |
| *** coreycb_ has joined #openstack-lbaas | 00:54 | |
| *** fyx_ has joined #openstack-lbaas | 00:54 | |
| *** OGtrilliams has joined #openstack-lbaas | 01:01 | |
| *** NobodyCam has quit IRC | 01:02 | |
| *** Dinesh_Bhor has quit IRC | 01:02 | |
| *** coreycb has quit IRC | 01:02 | |
| *** fyx has quit IRC | 01:02 | |
| *** lxkong has quit IRC | 01:02 | |
| *** PagliaccisCloud has quit IRC | 01:02 | |
| *** NobodyCam_ is now known as NobodyCam | 01:02 | |
| *** lxkong_ is now known as lxkong | 01:02 | |
| *** coreycb_ is now known as coreycb | 01:02 | |
| *** fyx_ is now known as fyx | 01:02 | |
| *** OGtrilliams is now known as PagliaccisCloud | 01:03 | |
| *** irclogbot_3 has quit IRC | 01:05 | |
| *** ricolin has joined #openstack-lbaas | 01:05 | |
| *** abaindur has quit IRC | 01:06 | |
| *** irclogbot_3 has joined #openstack-lbaas | 01:06 | |
| *** Dinesh_Bhor has joined #openstack-lbaas | 01:08 | |
| *** sapd1 has joined #openstack-lbaas | 01:18 | |
| *** psachin has joined #openstack-lbaas | 03:08 | |
| *** ramishra has joined #openstack-lbaas | 04:43 | |
| *** yboaron_ has joined #openstack-lbaas | 04:55 | |
| *** ivve has joined #openstack-lbaas | 05:43 | |
| *** gcheresh has joined #openstack-lbaas | 05:43 | |
| *** sapd1 has quit IRC | 05:51 | |
| *** gcheresh has quit IRC | 05:55 | |
| *** rpittau|afk is now known as rpittau | 07:17 | |
| ivve | johnsom: if some old amphora row is left in octavia.amphora_health table really wierd stuff happens (even if it's busy column is set to 1), eventually health_manager breaks the haproxy to the controller nodes | 07:17 |
| ivve | probably by spamming commands.. it shouts out this in non-debug: Amphora 725572bf-ab0f-4218-a9f1-470c1e94f3b2 health message was processed too sl | 07:18 |
| ivve | owly: 55.8338739872s! The system may be overloaded or otherwise malfunctioning. This heartbeat has been ignored and no update was made to the amphora health entry. THIS IS NOT GOOD. | 07:18 |
| ivve | i guess a lot of things have to go wrong for this to happen | 07:19 |
| ivve | but the result is very unpleasant and probably impossible for someone to troubleshoot properly since haproxy to controller-nodes explode :) | 07:19 |
| *** yboaron_ has quit IRC | 07:27 | |
| *** yboaron_ has joined #openstack-lbaas | 07:28 | |
| *** gcheresh has joined #openstack-lbaas | 07:28 | |
| ivve | well i was wrong, it can happen quite easily it seems. spawning a k8s cluster with default settings | 07:28 |
| *** luksky has joined #openstack-lbaas | 07:28 | |
| *** pcaruana has joined #openstack-lbaas | 07:32 | |
| *** gcheresh has quit IRC | 07:43 | |
| *** gcheresh has joined #openstack-lbaas | 07:59 | |
| *** yboaron_ has quit IRC | 08:53 | |
| *** salmankhan has joined #openstack-lbaas | 08:58 | |
| *** luksky has quit IRC | 08:59 | |
| *** ramishra has quit IRC | 09:35 | |
| *** ramishra has joined #openstack-lbaas | 09:40 | |
| *** luksky has joined #openstack-lbaas | 09:41 | |
| *** ramishra has quit IRC | 09:51 | |
| *** ramishra has joined #openstack-lbaas | 09:51 | |
| *** yboaron_ has joined #openstack-lbaas | 09:56 | |
| *** luksky has quit IRC | 09:57 | |
| *** pcaruana has quit IRC | 10:36 | |
| *** mkuf has quit IRC | 10:38 | |
| *** luksky has joined #openstack-lbaas | 10:42 | |
| *** pcaruana has joined #openstack-lbaas | 10:58 | |
| *** mkuf has joined #openstack-lbaas | 11:01 | |
| *** zigo has joined #openstack-lbaas | 11:07 | |
| *** sapd1_x has joined #openstack-lbaas | 11:27 | |
| *** ramishra has quit IRC | 11:52 | |
| *** ramishra has joined #openstack-lbaas | 12:11 | |
| *** boden has joined #openstack-lbaas | 12:26 | |
| *** goldyfruit has joined #openstack-lbaas | 12:36 | |
| johnsom | ivve Those log messages mean yout database is failing. Also make sure you have a current version of Octavia installed, as we spent a bunch of time optimizing the database calls to reduce the load on it and help the controllers scale. | 12:48 |
| johnsom | Not sure what you mean by controller haproxy, if the API is not responding well, maybe your host is overloaded from other applications/services? | 12:50 |
| *** yamamoto has joined #openstack-lbaas | 12:52 | |
| *** sapd1_x has quit IRC | 12:57 | |
| ivve | johnsom: i located the problem. it was missing some health_manager settings. since one of the options is deprecated all default to "None" | 13:10 |
| ivve | so they have to be configured.. namely: health_update_threads = 5 & stats_update_threads = 5 | 13:10 |
| *** pcaruana has quit IRC | 13:10 | |
| ivve | these are also "None" by default | 13:10 |
| ivve | setting them resolved the issue completely | 13:10 |
| ivve | or it was just overloaded if default is lower | 13:11 |
| ivve | by controller haproxy i mean the loadbalancer that loadbalances api-calls for openstack in general (haproxy+keepalived for admin,public and internal api's) | 13:12 |
| johnsom | Those default to the number of processors on the host, so yes, may be too low in some cases | 13:13 |
| *** yamamoto has quit IRC | 13:13 | |
| ivve | johnsom: wierd because i have quite a few processors.. maybe too many is also bad | 13:16 |
| ivve | 112 | 13:17 |
| *** pcaruana has joined #openstack-lbaas | 13:32 | |
| *** Emine has joined #openstack-lbaas | 13:50 | |
| *** yamamoto has joined #openstack-lbaas | 13:52 | |
| *** yamamoto has quit IRC | 13:55 | |
| *** yamamoto has joined #openstack-lbaas | 13:55 | |
| *** sapd1 has joined #openstack-lbaas | 13:56 | |
| *** Emine has quit IRC | 13:58 | |
| *** Emine has joined #openstack-lbaas | 13:58 | |
| *** yboaron_ has quit IRC | 14:01 | |
| *** yboaron_ has joined #openstack-lbaas | 14:01 | |
| *** fnaval has joined #openstack-lbaas | 14:04 | |
| *** gcheresh has quit IRC | 14:25 | |
| *** yamamoto has quit IRC | 14:28 | |
| openstackgerrit | Trevor Vardeman proposed openstack/octavia master: Adding changes to support ppc64le amphora builds with CentOS https://review.openstack.org/653488 | 14:30 |
| *** Vorrtex has joined #openstack-lbaas | 14:31 | |
| *** ricolin has quit IRC | 14:32 | |
| *** mkuf has quit IRC | 14:34 | |
| *** gcheresh has joined #openstack-lbaas | 14:51 | |
| *** gcheresh has quit IRC | 15:01 | |
| *** yamamoto has joined #openstack-lbaas | 15:03 | |
| *** yamamoto has quit IRC | 15:11 | |
| openstackgerrit | Merged openstack/octavia stable/queens: Fix a lifecycle bug with child objects https://review.openstack.org/652989 | 15:22 |
| *** pcaruana has quit IRC | 15:31 | |
| *** luksky has quit IRC | 15:46 | |
| *** ivve has quit IRC | 15:48 | |
| openstackgerrit | Trevor Vardeman proposed openstack/octavia master: Adding changes to support ppc64le amphora builds with CentOS https://review.openstack.org/653488 | 16:11 |
| *** Vorrtex has quit IRC | 16:12 | |
| *** ramishra has quit IRC | 16:22 | |
| *** sapd1 has quit IRC | 16:31 | |
| *** rcernin has quit IRC | 16:32 | |
| *** rcernin has joined #openstack-lbaas | 16:45 | |
| *** pcaruana has joined #openstack-lbaas | 17:02 | |
| *** mkuf has joined #openstack-lbaas | 17:02 | |
| *** mkuf has quit IRC | 17:10 | |
| *** psachin has quit IRC | 17:27 | |
| *** yboaron_ has quit IRC | 17:29 | |
| *** yboaron_ has joined #openstack-lbaas | 17:29 | |
| *** rpittau is now known as rpittau|afk | 17:49 | |
| *** AlexStaf has joined #openstack-lbaas | 18:23 | |
| *** AlexStaf has quit IRC | 18:36 | |
| *** abaindur has joined #openstack-lbaas | 18:52 | |
| *** abaindur has quit IRC | 18:53 | |
| *** abaindur has joined #openstack-lbaas | 18:53 | |
| *** salmankhan has quit IRC | 19:15 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Adds provider flavor capabilities API tests https://review.openstack.org/631113 | 19:15 |
| *** salmankhan has joined #openstack-lbaas | 19:15 | |
| *** salmankhan has quit IRC | 19:55 | |
| *** pcaruana has quit IRC | 20:47 | |
| *** abaindur_ has joined #openstack-lbaas | 20:50 | |
| *** luksky has joined #openstack-lbaas | 20:51 | |
| *** abaindur has quit IRC | 20:52 | |
| *** abaindur has joined #openstack-lbaas | 20:56 | |
| *** abaindur_ has quit IRC | 20:57 | |
| *** abaindur_ has joined #openstack-lbaas | 21:00 | |
| *** abaindur has quit IRC | 21:02 | |
| *** abaindur has joined #openstack-lbaas | 21:06 | |
| *** abaindur_ has quit IRC | 21:07 | |
| *** yamamoto has joined #openstack-lbaas | 21:09 | |
| *** abaindur_ has joined #openstack-lbaas | 21:10 | |
| *** abaindur has quit IRC | 21:12 | |
| *** yamamoto has quit IRC | 21:14 | |
| *** abaindur has joined #openstack-lbaas | 21:16 | |
| *** abaindu__ has joined #openstack-lbaas | 21:19 | |
| *** abaindur_ has quit IRC | 21:19 | |
| *** abaindu__ is now known as abaindur__ | 21:19 | |
| *** abaindur has quit IRC | 21:21 | |
| *** abaindur has joined #openstack-lbaas | 21:26 | |
| *** abaindur__ has quit IRC | 21:26 | |
| *** abaindur_ has joined #openstack-lbaas | 21:29 | |
| *** abaindu__ has joined #openstack-lbaas | 21:31 | |
| *** abaindur has quit IRC | 21:31 | |
| *** abaindu__ is now known as abaindur__ | 21:32 | |
| *** abaindur_ has quit IRC | 21:34 | |
| *** abaindur has joined #openstack-lbaas | 21:34 | |
| *** abaindur__ has quit IRC | 21:36 | |
| *** abaindur_ has joined #openstack-lbaas | 21:37 | |
| *** abaindur has quit IRC | 21:39 | |
| *** abaindur has joined #openstack-lbaas | 21:39 | |
| *** abaindur_ has quit IRC | 21:41 | |
| *** abaindur_ has joined #openstack-lbaas | 21:42 | |
| *** abaindu__ has joined #openstack-lbaas | 21:44 | |
| *** abaindu__ is now known as abaindur__ | 21:44 | |
| *** abaindur has quit IRC | 21:44 | |
| *** abaindur_ has quit IRC | 21:47 | |
| *** boden has quit IRC | 21:48 | |
| openstackgerrit | Merged openstack/octavia master: Fix missing REDIRECT_PREFIX in the api-ref https://review.openstack.org/652795 | 22:04 |
| openstackgerrit | Michael Johnson proposed openstack/octavia master: Handle non-graceful shutdown https://review.openstack.org/653872 | 22:28 |
| *** yamamoto has joined #openstack-lbaas | 22:39 | |
| *** yamamoto has quit IRC | 22:41 | |
| cgoncalves | this ^ is gonna be interesting reviewing :) | 22:43 |
| cgoncalves | db migration :( | 22:44 |
| *** fnaval has quit IRC | 22:47 | |
| *** yamamoto has joined #openstack-lbaas | 22:49 | |
| *** luksky has quit IRC | 22:59 | |
| *** yamamoto has quit IRC | 23:05 | |
| *** yamamoto has joined #openstack-lbaas | 23:10 | |
| *** yamamoto has quit IRC | 23:25 | |
| *** abaindur has joined #openstack-lbaas | 23:55 | |
| *** abaindur_ has joined #openstack-lbaas | 23:57 | |
| *** abaindur__ has quit IRC | 23:58 | |
| *** abaindur has quit IRC | 23:59 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!