| *** JudeCross has quit IRC | 00:39 | |
| *** Emine has quit IRC | 00:55 | |
| *** hongbin has joined #openstack-lbaas | 01:19 | |
| *** aojea has joined #openstack-lbaas | 01:23 | |
| *** aojea has quit IRC | 01:27 | |
| *** numans has quit IRC | 01:34 | |
| *** JudeCross has joined #openstack-lbaas | 01:44 | |
| *** JudeCross has quit IRC | 01:49 | |
| *** abaindur has quit IRC | 02:18 | |
| *** abaindur has joined #openstack-lbaas | 02:22 | |
| *** abaindur has quit IRC | 02:32 | |
| *** ramishra has joined #openstack-lbaas | 03:27 | |
| *** abaindur has joined #openstack-lbaas | 03:35 | |
| *** rcernin_ has quit IRC | 03:42 | |
| *** rcernin has joined #openstack-lbaas | 03:43 | |
| *** hongbin has quit IRC | 03:47 | |
| *** sapd1 has quit IRC | 03:55 | |
| *** sapd1 has joined #openstack-lbaas | 04:00 | |
| *** blake has quit IRC | 04:04 | |
| *** blake has joined #openstack-lbaas | 04:06 | |
| *** blake has quit IRC | 04:11 | |
| *** abaindur has quit IRC | 04:31 | |
| *** blake has joined #openstack-lbaas | 04:36 | |
| *** abaindur has joined #openstack-lbaas | 04:37 | |
| *** blake has quit IRC | 05:06 | |
| *** pcaruana has joined #openstack-lbaas | 06:33 | |
| *** numans has joined #openstack-lbaas | 07:01 | |
| *** abaindur has quit IRC | 07:03 | |
| *** abaindur has joined #openstack-lbaas | 07:03 | |
| *** rcernin has quit IRC | 07:12 | |
| *** openstackgerrit has joined #openstack-lbaas | 07:16 | |
| openstackgerrit | Merged openstack/neutron-lbaas-dashboard master: Imported Translations from Zanata https://review.openstack.org/604579 | 07:16 |
|---|---|---|
| *** celebdor has joined #openstack-lbaas | 07:32 | |
| *** JudeCross has joined #openstack-lbaas | 07:32 | |
| *** abaindur has quit IRC | 07:49 | |
| cgoncalves | o/ | 07:53 |
| *** abaindur has joined #openstack-lbaas | 07:58 | |
| *** abaindur has quit IRC | 08:05 | |
| *** velizarx has joined #openstack-lbaas | 08:21 | |
| *** JudeCross has quit IRC | 08:35 | |
| *** numans has quit IRC | 08:49 | |
| *** velizarx has quit IRC | 08:52 | |
| *** velizarx has joined #openstack-lbaas | 08:54 | |
| *** salmankhan has joined #openstack-lbaas | 09:16 | |
| *** Emine has joined #openstack-lbaas | 09:24 | |
| *** yamamoto has quit IRC | 10:17 | |
| *** yamamoto has joined #openstack-lbaas | 10:25 | |
| *** yamamoto has quit IRC | 10:46 | |
| *** yamamoto has joined #openstack-lbaas | 10:48 | |
| *** yamamoto has quit IRC | 10:49 | |
| *** yamamoto has joined #openstack-lbaas | 11:23 | |
| *** pcaruana has quit IRC | 11:50 | |
| *** salmankhan has quit IRC | 12:08 | |
| *** yamamoto has quit IRC | 12:14 | |
| *** yamamoto has joined #openstack-lbaas | 12:16 | |
| *** pcaruana has joined #openstack-lbaas | 12:39 | |
| *** ramishra has quit IRC | 12:54 | |
| *** amuller has joined #openstack-lbaas | 13:02 | |
| *** ramishra has joined #openstack-lbaas | 13:22 | |
| *** yamamoto has quit IRC | 13:31 | |
| *** yamamoto has joined #openstack-lbaas | 13:31 | |
| *** salmankhan has joined #openstack-lbaas | 13:45 | |
| *** evgenyf has joined #openstack-lbaas | 13:47 | |
| *** yamamoto has quit IRC | 14:32 | |
| *** fnaval has joined #openstack-lbaas | 14:33 | |
| *** yamamoto has joined #openstack-lbaas | 14:33 | |
| *** yamamoto has quit IRC | 14:33 | |
| *** yamamoto has joined #openstack-lbaas | 14:34 | |
| *** yamamoto has quit IRC | 14:38 | |
| *** yamamoto has joined #openstack-lbaas | 14:43 | |
| *** yamamoto has quit IRC | 14:52 | |
| *** evgenyf has quit IRC | 14:55 | |
| *** Swami has joined #openstack-lbaas | 14:59 | |
| xgerman_ | cgoncalves: o/ | 15:15 |
| cgoncalves | xgerman_, o/ | 15:20 |
| cgoncalves | xgerman_, in which hotel are you going to stay in Berlin? | 15:20 |
| xgerman_ | Hotel Indigo Centre Alexanderplatz | 15:26 |
| xgerman_ | cgoncalves: had to look in Concur — why didn’t they send an e-mail? | 15:26 |
| xgerman_ | overall, that was the cheapest + a train station with a direct rail to the conference center is near by ;-) | 15:26 |
| *** yamamoto has joined #openstack-lbaas | 15:28 | |
| cgoncalves | hmm matches with discounted group rate for Crowne Plaza also with direct rail | 15:31 |
| cgoncalves | plus includes breakfast | 15:31 |
| xgerman_ | well, our concur might have different prices then yours ;-) | 15:35 |
| cgoncalves | I concur | 15:38 |
| *** rpittau has quit IRC | 15:54 | |
| *** evgenyf has joined #openstack-lbaas | 15:55 | |
| *** JudeCross has joined #openstack-lbaas | 15:59 | |
| *** yamamoto has quit IRC | 16:02 | |
| *** velizarx has quit IRC | 16:04 | |
| *** evgenyf has quit IRC | 16:31 | |
| *** sapd1_ has joined #openstack-lbaas | 16:40 | |
| *** Swami has quit IRC | 16:46 | |
| *** JudeCross has quit IRC | 16:49 | |
| *** salmankhan has quit IRC | 17:06 | |
| *** ramishra has quit IRC | 17:13 | |
| *** Emine has quit IRC | 17:21 | |
| *** JudeCross has joined #openstack-lbaas | 17:30 | |
| *** Emine has joined #openstack-lbaas | 17:34 | |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add v2 two-node scenario test https://review.openstack.org/605163 | 17:58 |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add v2 two-node scenario test https://review.openstack.org/605163 | 17:59 |
| openstackgerrit | Michael Johnson proposed openstack/octavia-tempest-plugin master: Add v2 two-node scenario test https://review.openstack.org/605163 | 18:00 |
| *** yamamoto has joined #openstack-lbaas | 18:00 | |
| *** dims_ is now known as dims | 18:05 | |
| *** hyang has joined #openstack-lbaas | 18:08 | |
| *** Swami has joined #openstack-lbaas | 18:10 | |
| *** sapd1_ has quit IRC | 18:12 | |
| Swami | I have a question regarding the missing Amphora VM after a reboot of the Compute immediately after an upgrade from Newton to Pike. The Provisioning_status of the Loadbalancer changed to ERROR state and also the Amphora provisioning_status. How do I recreate the Amphora VM? Is there a way by changing manually in the DB and restarting the service. More information can be seen here. http://paste.openstack.org/show/731040/. Also when the health manager | 18:15 |
| Swami | was trying to recreate the VM it failed with a LOG information saying that the "interface detach wait timeout". | 18:15 |
| Swami | octavia/octavia-health-manager.log: |__Flow 'octavia-failover-amphora-flow': TimeoutException: Port acd30c98-32b9-4564-be63-97ffce731f40 failed to detach (device_id e4796efc-0d41-47b0-a0cb-1c9a6bc68f90) within the required time (300 s). | 18:18 |
| hyang | Hi there, I have a question for LB healthmonitor, I'm using stable/queens Octavia and followed this doc https://docs.openstack.org/octavia/queens/user/guides/basic-cookbook.html#http-heath-monitors to create a basic HTTP load balancer with a PING type health monitor. But I found the node member will be in ERROR operating_status until I run a basic web server on port 80 in the node then it become ONLINE. | 18:21 |
| hyang | Since my healthmonitor is PING type, I expect it will show the node ONLINE as long as it is pingable. Did I misunderstand something? | 18:22 |
| johnsom | Swami This is an issue we recently became aware of. When the compute host is down, Octavia notices the failed amphora and starts a failover. The problem comes when we call nova delete. It accepts it, but doesn't actually delete the instance. More importantly it doesn't detach the neutron port(s). So we get stuck retrying to get the ports (VIP is important), and end up having to stop and mark things in ERROR. | 18:23 |
| johnsom | Swami You should be able to use the amphora failover API to rebuild it once nova comes to it's senses and lets the resources go. We are working on a different approach to work around this issue with nova now. | 18:24 |
| Swami | johnsom: thanks for your information. | 18:25 |
| Swami | johnsom: Can that be run from the cli | 18:25 |
| Swami | johnsom: failover api and is it available in Pike. | 18:25 |
| johnsom | Swami I'm not sure that has merged yet | 18:25 |
| johnsom | Swami https://review.openstack.org/#/c/532424/ Nope | 18:25 |
| Swami | johnsom: thanks | 18:26 |
| johnsom | hyang There was a bug fix for that issue. The release notes state that CentOS is still not fixed, are you using a CentOS amphora? CentOS wasn't fixed as the HAProxy version is too old in CentOS. | 18:27 |
| Swami | johnsom: Generally what is the trigger in the DB to recreate an Amphora for a configured LBaaS | 18:27 |
| johnsom | hyang See the release notes here: https://docs.openstack.org/releasenotes/octavia/queens.html | 18:27 |
| johnsom | Swami You could try making it ACTIVE again. Also remove the "busy" flag in the amphora_health table if it's set. | 18:28 |
| johnsom | Swami Assuming you know for a fact that the amphora is not being worked on by a health manager process | 18:28 |
| hyang | johnsom Yes, I'm using the CentOS amphora, so there is no workaround for this issue? except for change the base amphora image? | 18:29 |
| johnsom | hyang Correct. CentOS has too old of code and doesn't support the ping capability. | 18:30 |
| Swami | johnsom: ok, I can try that. | 18:31 |
| hyang | johnsom ok, thanks for your information | 18:31 |
| Swami | Also in the first case where the failure happened in detach state, if I have a ACTIVE_STANDBY configuration and if I have two Amphora running on two different nodes, then if the reboot happens, will it still end up in ERROR state or can it recover because of ACTIVE_STANDBY configuration. | 18:32 |
| Swami | I meant rebooting just one compute at a time. | 18:33 |
| johnsom | Swami Well, the load balancer will still be functioning, but it will be in ERROR because one of the amphora failed all of it's retries to rebuild, etc. | 18:33 |
| johnsom | You can also change the configuration settings to wait longer before failing over, or to wait longer for nova to actually delete the resources. | 18:34 |
| Swami | Ok, I can adjust the config option to raise the timeout beyong 300sec then. | 18:35 |
| Swami | s/beyong/beyond | 18:35 |
| johnsom | Normally a simple instance reboot completes before we give up on the amphora, but a host reboot, some of those servers are super slow to come back up. | 18:35 |
| Swami | agreed | 18:36 |
| *** salmankhan has joined #openstack-lbaas | 18:37 | |
| cgoncalves | hyang, johnsom: starting from Rocky, CentOS-based amp image include newer haproxy (v1.8) | 18:39 |
| cgoncalves | hyang, https://review.openstack.org/#/c/548420/ | 18:39 |
| hyang | cgoncalves oh, can we backport it to queens? | 18:40 |
| *** salmankhan has quit IRC | 18:41 | |
| hyang | cgoncalves, johnsom I just requested the cherry-pick https://review.openstack.org/#/c/605835/ | 18:47 |
| johnsom | hyang Why not just use the Rocky amphora image? | 18:48 |
| hyang | johnsom Our Octavia service is in Queens, is it ok to use amphora in Rocky but controller in Queens? | 18:50 |
| johnsom | Yes, should work fine | 18:50 |
| hyang | ok, I'll consider that if the cherry pick got rejected :) But would it be better to back port and fix the issue in stable/queens as well? | 18:52 |
| *** abaindur has joined #openstack-lbaas | 18:56 | |
| johnsom | That patch introduces an alternate repo, so really should not be backported. | 18:59 |
| *** yamamoto has quit IRC | 19:01 | |
| hyang | johnsom You mean the centos repo? I don't understand why is that not a problem for rocky version then? | 19:06 |
| johnsom | The backport adds a repo source that was not used in the Queens release, it may pull in other changes, etc. It was however added during the Rocky development cycle and not backported to a stable branch. | 19:07 |
| *** dayou has quit IRC | 19:10 | |
| hyang | johnsom ok, I see your concerns. I don't know will the CI test cover the amphora build tests but I'll give a try locally to see will it work | 19:11 |
| *** hyang has quit IRC | 19:18 | |
| *** dayou has joined #openstack-lbaas | 19:37 | |
| *** pcaruana has quit IRC | 19:54 | |
| *** aojea has joined #openstack-lbaas | 19:57 | |
| xgerman_ | cgoncalves: https://review.openstack.org/#/c/587505/ — rewrote it but no traction… | 20:10 |
| *** amuller has quit IRC | 20:13 | |
| *** Emine has quit IRC | 20:27 | |
| *** Emine has joined #openstack-lbaas | 20:28 | |
| *** aojea has quit IRC | 20:32 | |
| *** aojea has joined #openstack-lbaas | 20:33 | |
| *** dayou has quit IRC | 20:38 | |
| Swami | johnsom: Another question if the amphora entry from the amphora table is missing and only the load_balancer table has an entry for the lbaas, is there a way to trigger an Amphora create. | 20:46 |
| johnsom | Swami The failover API | 20:46 |
| Swami | johnsom: ok | 20:46 |
| Swami | johnsom: but there is no way without the failover API. | 20:47 |
| johnsom | I'm not sure on your version. If the amp record has already purged, it might be hard | 20:48 |
| Swami | johnsom: Ok, that helps. | 20:49 |
| cgoncalves | xgerman_, reviewed | 20:54 |
| xgerman_ | thanks | 20:54 |
| xgerman_ | should have kept the old version - at least someone gave me +1 | 20:55 |
| cgoncalves | xgerman_, zuul? :P | 20:56 |
| *** yamamoto has joined #openstack-lbaas | 20:59 | |
| xgerman_ | cgoncalves: just rewrite it the way you want it and I will +2… | 21:04 |
| xgerman_ | at that point I will just file a RPC-R support ticket when I see zombies and call it. a day :-) | 21:05 |
| xgerman_ | abandoned | 21:05 |
| *** dayou has joined #openstack-lbaas | 21:06 | |
| openstackgerrit | German Eichberger proposed openstack/octavia master: [WIP] Refactor the pluggin of the VIP https://review.openstack.org/604479 | 21:25 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia master: Delete zombie amphora when detected https://review.openstack.org/587505 | 21:42 |
| cgoncalves | xgerman_, ^ | 21:43 |
| xgerman_ | yeah, patches are coming too quickly for +2 | 21:46 |
| *** yamamoto has quit IRC | 21:48 | |
| xgerman_ | cgoncalves: also when you are bored please go through the reviews and change your +1 to +2 - since you have that power now ;-) | 21:49 |
| cgoncalves | xgerman_, k. will do it tomorrow | 21:50 |
| xgerman_ | isn’t it tomorrow already in germany :-) | 21:50 |
| cgoncalves | 23:53 :P | 21:53 |
| *** abaindur has quit IRC | 22:25 | |
| *** abaindur has joined #openstack-lbaas | 22:26 | |
| *** rcernin has joined #openstack-lbaas | 22:29 | |
| *** abaindur has quit IRC | 22:30 | |
| *** abaindur has joined #openstack-lbaas | 22:30 | |
| *** KeithMnemonic has quit IRC | 22:33 | |
| *** KeithMnemonic has joined #openstack-lbaas | 22:33 | |
| *** fnaval has quit IRC | 22:38 | |
| *** aojea has quit IRC | 22:38 | |
| *** celebdor has quit IRC | 23:10 | |
| *** abaindur has quit IRC | 23:32 | |
| *** Swami has quit IRC | 23:45 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!