| openstackgerrit | Vishal Manchanda proposed openstack/octavia-dashboard master: Remove six usage https://review.opendev.org/702336 | 00:14 |
|---|---|---|
| openstackgerrit | Merged openstack/octavia-dashboard stable/train: Fix Django version in lower-constraints https://review.opendev.org/700720 | 00:28 |
| *** hongbin has joined #openstack-lbaas | 02:02 | |
| *** hongbin has joined #openstack-lbaas | 02:03 | |
| *** hongbin has joined #openstack-lbaas | 02:04 | |
| *** hongbin has quit IRC | 02:04 | |
| *** hongbin has joined #openstack-lbaas | 02:06 | |
| *** mithilarun has joined #openstack-lbaas | 02:29 | |
| *** mithilar_ has joined #openstack-lbaas | 02:36 | |
| *** mithilarun has quit IRC | 02:39 | |
| *** mithilar_ has quit IRC | 02:41 | |
| *** tkajinam has quit IRC | 03:48 | |
| *** hongbin_ has joined #openstack-lbaas | 04:12 | |
| *** hongbin has quit IRC | 04:15 | |
| *** tkajinam has joined #openstack-lbaas | 04:28 | |
| *** hongbin_ has quit IRC | 04:37 | |
| *** sapd1_x has joined #openstack-lbaas | 04:40 | |
| openstackgerrit | Merged openstack/octavia-dashboard master: Remove six usage https://review.opendev.org/702336 | 04:57 |
| *** sapd1_x has quit IRC | 05:44 | |
| *** mugsie has quit IRC | 06:11 | |
| *** mugsie has joined #openstack-lbaas | 06:14 | |
| openstackgerrit | Akihiro Motoki proposed openstack/octavia-dashboard stable/stein: Fix Django version in lower-constraints https://review.opendev.org/700721 | 06:30 |
| *** gcheresh has joined #openstack-lbaas | 06:32 | |
| *** KeithMnemonic1 has joined #openstack-lbaas | 07:15 | |
| *** KeithMnemonic has quit IRC | 07:19 | |
| *** gcheresh has quit IRC | 07:38 | |
| *** gcheresh has joined #openstack-lbaas | 07:40 | |
| *** tesseract has joined #openstack-lbaas | 07:58 | |
| *** tesseract has quit IRC | 07:58 | |
| *** tesseract has joined #openstack-lbaas | 07:59 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 08:06 | |
| *** lxkong has quit IRC | 08:06 | |
| *** rpittau|afk is now known as rpittau | 08:22 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/octavia-dashboard stable/train: Imported Translations from Zanata https://review.opendev.org/698667 | 08:30 |
| *** ataraday_ has joined #openstack-lbaas | 08:40 | |
| *** tkajinam has quit IRC | 08:48 | |
| *** dmellado has quit IRC | 08:51 | |
| *** dmellado has joined #openstack-lbaas | 08:55 | |
| *** ccamposr has joined #openstack-lbaas | 09:01 | |
| openstackgerrit | Ann Taraday proposed openstack/octavia master: Add an ability to retry failover process automatically https://review.opendev.org/702599 | 09:08 |
| *** ccamposr__ has joined #openstack-lbaas | 09:42 | |
| *** pcaruana has joined #openstack-lbaas | 09:42 | |
| *** ccamposr has quit IRC | 09:44 | |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia-tempest-plugin master: Add tests for allowed CIDRs in listeners https://review.opendev.org/702629 | 10:11 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia-tempest-plugin master: Add tests for allowed CIDRs in listeners https://review.opendev.org/702629 | 10:54 |
| *** ccamposr__ has quit IRC | 10:55 | |
| openstackgerrit | Adam Harwell proposed openstack/octavia master: Conf option to use VIP ip as source ip for backend https://review.opendev.org/702535 | 11:11 |
| rm_work | doubt i will make the meeting in 5 hours, as it will fall right in the middle of my 8 hours of sleep <_< | 11:12 |
| *** pcaruana has quit IRC | 11:12 | |
| rm_work | still need to get around to making another meeting time poll | 11:13 |
| openstackgerrit | Ann Taraday proposed openstack/octavia master: Jobboard based controller https://review.opendev.org/647406 | 11:22 |
| openstackgerrit | Carlos Goncalves proposed openstack/octavia-tempest-plugin master: Add scenario test for IPv6 VIP and IPv6-only members https://review.opendev.org/702650 | 11:24 |
| *** ccamposr has joined #openstack-lbaas | 11:27 | |
| *** AlexStaf has joined #openstack-lbaas | 11:30 | |
| *** AlexStaf has quit IRC | 11:34 | |
| *** ccamposr has quit IRC | 11:52 | |
| *** ccamposr has joined #openstack-lbaas | 11:53 | |
| *** ccamposr has quit IRC | 12:31 | |
| *** pcaruana has joined #openstack-lbaas | 12:49 | |
| *** cgoncalves has quit IRC | 12:58 | |
| *** cgoncalves has joined #openstack-lbaas | 13:00 | |
| *** cgoncalves has quit IRC | 13:04 | |
| *** cgoncalves has joined #openstack-lbaas | 13:10 | |
| *** ccamposr has joined #openstack-lbaas | 13:24 | |
| *** ppeere_os has joined #openstack-lbaas | 13:28 | |
| ppeere_os | Hi All! Does any core reviewer has time to review: https://review.opendev.org/#/c/700513/ ? | 13:28 |
| *** servagem has joined #openstack-lbaas | 13:33 | |
| ppeere_os | Ah i see it's already in the ussuri priority reviews nvm | 13:47 |
| *** mnaser has quit IRC | 13:54 | |
| *** mnaser has joined #openstack-lbaas | 13:55 | |
| *** rcernin has quit IRC | 14:04 | |
| *** ccamposr has quit IRC | 14:09 | |
| *** TrevorV has joined #openstack-lbaas | 15:08 | |
| *** ccamposr has joined #openstack-lbaas | 15:13 | |
| *** ataraday_ has quit IRC | 15:22 | |
| *** ataraday_ has joined #openstack-lbaas | 15:26 | |
| johnsom | gthiemonge So, if you have time and the environment up, hack the systemd unit for keepalived and add the -D and maybe even a --debug level and try to reproduce it. To my knowledge, the keepalived config should never know about the "ens9" interface name, so I'm not sure that is related. | 15:56 |
| johnsom | I mean obviously it knows from the netlink stream, but our config file should never know about that name | 15:57 |
| gthiemonge | johnsom: I tried the "debug 4" in the conf, it didn't add many useful information | 15:57 |
| johnsom | Yeah, keepalived is conservative on what it puts in each debug level and then you hit a certain number and it's a flood | 15:58 |
| gthiemonge | johnsom: I think this is the issue with keepalived 2.0.10, it does some stuff when moving/updating unrelated interfaces | 15:58 |
| gthiemonge | johnsom: I need to check with 2.0.19 or master | 15:59 |
| johnsom | Ok. Yeah. It also could be faulting prior to us configuring it for our interfaces, in which case we don't care. | 15:59 |
| johnsom | #startmeeting Octavia | 16:00 |
| openstack | Meeting started Wed Jan 15 16:00:38 2020 UTC and is due to finish in 60 minutes. The chair is johnsom. Information about MeetBot at http://wiki.debian.org/MeetBot. | 16:00 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 16:00 |
| *** openstack changes topic to " (Meeting topic: Octavia)" | 16:00 | |
| openstack | The meeting name has been set to 'octavia' | 16:00 |
| johnsom | Hi everyone | 16:00 |
| cgoncalves | hey | 16:01 |
| gthiemonge | Hi | 16:01 |
| johnsom | Pretty light agenda today as I didn't have a lot of time to prepare in the absence of our PTL.... | 16:02 |
| *** gcheresh has quit IRC | 16:02 | |
| johnsom | #topic Announcements | 16:02 |
| *** openstack changes topic to "Announcements (Meeting topic: Octavia)" | 16:02 | |
| johnsom | The only item I have here is the North Dakota State University (NDSU) students are selecting projects this week. We will hear in the next week or two if there is a team that will work on Octavia. | 16:03 |
| johnsom | Background is that OpenStack has worked with NDSU in the past for student "capstone" projects. Adam was offered for us to put in a project idea. We put in the TLS cipher and protocol stories. | 16:04 |
| johnsom | If we get students, they will work on completing those two stories for us. | 16:04 |
| johnsom | Kendall, rm_work, and I will act as mentors | 16:05 |
| cgoncalves | very good news :) | 16:05 |
| ataraday_ | hi | 16:05 |
| cgoncalves | The "V" release will be named Victoria | 16:05 |
| cgoncalves | #link http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011947.html | 16:05 |
| johnsom | More info on the capstone projects past/present is here: | 16:06 |
| johnsom | #link https://csprojects.cs.ndsu.nodak.edu/capstone/PreviousProjects/about/ | 16:06 |
| johnsom | Ah yeah, Victoria! | 16:06 |
| *** armax has joined #openstack-lbaas | 16:07 | |
| cgoncalves | back to pronounceable names :) | 16:07 |
| johnsom | Ha, yes | 16:07 |
| johnsom | Any other announcements today? | 16:07 |
| johnsom | #topic Brief progress reports / bugs needing review | 16:08 |
| *** openstack changes topic to "Brief progress reports / bugs needing review (Meeting topic: Octavia)" | 16:08 | |
| johnsom | I have been working on internal planning activities, so a bit distracted. | 16:08 |
| johnsom | I did get some time in on failover, which is good, but could use some more.... | 16:08 |
| johnsom | I have also carved out some time doing reviews as I can. | 16:09 |
| ataraday_ | Jobboard patches are waiting for reviews :) All discovered issues seems to be resolved, I proposed https://review.opendev.org/#/c/700553/ to mask retry connecntion traces | 16:10 |
| johnsom | Also, I started work on tags support in our CLI a long time ago. That is finally moving forward in OSC/OSC-lib so that might be ready to work on again in the next few weeks. Waiting on the osc-lib Upper Constraints (UC) update to merge. | 16:10 |
| cgoncalves | stable branch gates got broken with the release of hacking 2.0.0. this has been fixed, thanks for the quick reviews | 16:10 |
| johnsom | ataraday_ I saw that patch go by, thank you! | 16:10 |
| johnsom | ataraday_ With my current work, it will likely be next week before I can start reviewing that chain again. | 16:11 |
| ataraday_ | cgoncalves, I want to ask you to review devstack part of https://review.opendev.org/#/c/647406/ if you will some time | 16:11 |
| cgoncalves | also spent time doing reviews | 16:12 |
| cgoncalves | sadly I did not review jobboard patches yet, sorry | 16:12 |
| ataraday_ | johnsom, sure! thanks! | 16:12 |
| cgoncalves | ataraday_, I'm stacking with jobboard now :) | 16:13 |
| johnsom | Any other updates today? | 16:14 |
| cgoncalves | any chance more people could help review https://review.opendev.org/#/c/668619/ would be appreciated | 16:15 |
| johnsom | Ah, yes, I can take a look later. | 16:16 |
| cgoncalves | thanks | 16:16 |
| johnsom | I guess it was November the last time I reviewed that... sigh | 16:17 |
| johnsom | #topic Open Discussion | 16:17 |
| *** openstack changes topic to "Open Discussion (Meeting topic: Octavia)" | 16:17 | |
| johnsom | Any other topics for today? | 16:18 |
| ataraday_ | I've customer request about having retry for amphora failovers and pushed https://review.opendev.org/#/c/702599/ on the review, was this discussed in past days? | 16:20 |
| johnsom | No, I haven't seen this. However, my failover re-work code does enable retries. The current code is not idempotent and will blow up. This is part of what is taking so long, I'm going through the tasks and fixing them. | 16:21 |
| *** ppeere_os has quit IRC | 16:22 | |
| johnsom | I hope I can post an early version in the next week, but it won't be complete by then. | 16:22 |
| ataraday_ | I pushed this today as I feel having changes on review is the best way to start discussions :) Will mark it WIP for now then | 16:24 |
| johnsom | Yeah, I agree, I'm a bit embarrassed I haven't been able to push anything yet. I'm just getting pulled in too many directions. | 16:24 |
| johnsom | Right now 100% of the tests will fail, but a standalone failover will succeed. I'm working on cleaning up the standalone code and creating subflows that can be used for the act/stdby. Then I need to fix all of the tests I broke and then port the code over to the v2 (jobboard) paths. Somewhere in there a rebase is necessary as well. | 16:26 |
| johnsom | Key fixes in: | 16:26 |
| johnsom | The tasks are idempotent, so retries now work. | 16:27 |
| johnsom | There are retries in the code. | 16:27 |
| johnsom | It handles missing amphora correctly. I.e. an LB with no associated amphora now repairs itself. | 16:27 |
| johnsom | It cleans up extra resources and amphora that may be attached to an LB | 16:28 |
| johnsom | It checks and repairs the VIP if it is corrupted somehow | 16:28 |
| johnsom | It will succeed in failing over even if Nova will not release resources. | 16:28 |
| johnsom | It slices, dices, and juliennes | 16:29 |
| ataraday_ | wow, sounds great :) | 16:30 |
| johnsom | https://youtu.be/Ohidv69WfNQ | 16:30 |
| johnsom | lol | 16:30 |
| cgoncalves | hah :) | 16:30 |
| johnsom | Anyway, that is the beast I have been working on. | 16:31 |
| johnsom | Oh, and it parallelizes more of the flow, so should be slightly faster | 16:31 |
| cgoncalves | 1-800-EXPLODE-MY-HEAD xD | 16:32 |
| johnsom | ha | 16:32 |
| johnsom | Yeah, some days that is how I feel... | 16:32 |
| johnsom | Anyway, it's coming. Work is being done to make failover function as it was originally intended | 16:33 |
| *** gcheresh has joined #openstack-lbaas | 16:33 | |
| johnsom | Any other topics? | 16:34 |
| johnsom | questions/comments? | 16:34 |
| johnsom | Ok then, have a good week everyone! | 16:37 |
| johnsom | #endmeeting | 16:37 |
| *** openstack changes topic to "Discussions for OpenStack Octavia | Priority bug review list: https://etherpad.openstack.org/p/octavia-priority-reviews" | 16:37 | |
| openstack | Meeting ended Wed Jan 15 16:37:31 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 16:37 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-01-15-16.00.html | 16:37 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-01-15-16.00.txt | 16:37 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/octavia/2020/octavia.2020-01-15-16.00.log.html | 16:37 |
| cgoncalves | thanks for running the meeting :) | 16:37 |
| johnsom | +1 | 16:38 |
| gthiemonge | johnsom: FYI here is a reproducer for the keepalived bug with simple commands: http://paste.openstack.org/show/788421/ | 16:40 |
| johnsom | gthiemonge Hmmm, interesting for sure | 16:41 |
| johnsom | gthiemonge Do we have a story open for this? | 16:41 |
| *** openstackgerrit has quit IRC | 16:43 | |
| johnsom | gthiemonge FYI, the Ubuntu bionic we have running the gates is using 1.3.9. So, currently this is limited to centos 8, though it looks like the next LTS for Ubuntu will also have 2.0.10 | 16:43 |
| gthiemonge | nop, I'm opening one | 16:44 |
| *** gcheresh has quit IRC | 16:44 | |
| johnsom | Cool, thanks | 16:44 |
| *** pcaruana has quit IRC | 16:45 | |
| *** beisner has quit IRC | 16:50 | |
| *** beisner has joined #openstack-lbaas | 16:50 | |
| cgoncalves | centos just can't pick the right versions, uhm? :P | 17:00 |
| johnsom | Maybe they jumped too fast on that one. (can't believe I'm saying that) | 17:01 |
| gthiemonge | johnsom: https://storyboard.openstack.org/#!/story/2007095 | 17:02 |
| *** maciejjozefczyk has quit IRC | 17:04 | |
| *** maciejjozefczyk has joined #openstack-lbaas | 17:04 | |
| johnsom | gthiemonge That looks great, thanks! | 17:08 |
| *** gmann has quit IRC | 17:09 | |
| johnsom | Wow, that is a sizable changelog between 2.0.10 and 2.0.19 keepalived: https://www.keepalived.org/changelog.html | 17:09 |
| *** gmann has joined #openstack-lbaas | 17:10 | |
| gthiemonge | johnsom: adding no_track options in keepalived conf is a possible workaround (see https://www.keepalived.org/manpage.html) | 17:14 |
| *** rpittau is now known as rpittau|afk | 17:15 | |
| johnsom | gthiemonge Doesn't that mean that if the VIP base port goes down it won't transition to the standby amp? | 17:16 |
| johnsom | We would have to no_track a specific interface, which we don't know I think. | 17:17 |
| gthiemonge | I haven't verified if no_track affects the VIP port :/ | 17:18 |
| gthiemonge | but you're probably right | 17:18 |
| gthiemonge | I may have another workaround that I will try when centos mirrors will be fixed :D | 17:19 |
| johnsom | lol, ok | 17:19 |
| cgoncalves | gthiemonge, does keepalived crash/get respawn? | 17:24 |
| johnsom | Fault state would not be a crash, so I doubt it | 17:25 |
| gthiemonge | cgoncalves: no, it stays in FAULT state | 17:25 |
| gthiemonge | and if I restart keepalived service, everything works as expected | 17:26 |
| *** tesseract has quit IRC | 17:26 | |
| johnsom | FAULT just means something it is monitoring failed, so this instance cannot be in MASTER or BACKUP state | 17:26 |
| cgoncalves | ok. asked because I was looking at https://bugzilla.redhat.com/show_bug.cgi?id=1692788, also 2.0.10 and not reproducible in 2.0.7 | 17:27 |
| openstack | bugzilla.redhat.com bug 1692788 in keepalived "keepalived crashes in a loop when the vrrp interface does not exist" [Urgent,Closed: duplicate] - Assigned to rohara | 17:27 |
| johnsom | Looks like Ryan says that is due to smtp config, which we don't use | 17:29 |
| *** guilhermesp has quit IRC | 17:31 | |
| *** guilhermesp has joined #openstack-lbaas | 17:32 | |
| xgerman | Ryan? The guy who kept us out of all Redhat parties? | 17:41 |
| johnsom | Yes, one and the same | 17:42 |
| *** ataraday_ has quit IRC | 17:56 | |
| *** mithilarun has joined #openstack-lbaas | 17:58 | |
| *** servagem has quit IRC | 17:59 | |
| *** born2bake has joined #openstack-lbaas | 18:02 | |
| *** rpittau|afk has quit IRC | 18:30 | |
| *** andrein has quit IRC | 18:30 | |
| *** coreycb has quit IRC | 18:30 | |
| *** mnaser has quit IRC | 18:31 | |
| *** openstackstatus has quit IRC | 18:32 | |
| *** coreycb has joined #openstack-lbaas | 18:33 | |
| *** andrein has joined #openstack-lbaas | 18:33 | |
| *** johnsom has quit IRC | 18:33 | |
| *** mnaser has joined #openstack-lbaas | 18:33 | |
| *** johnsom has joined #openstack-lbaas | 18:35 | |
| *** rcernin has joined #openstack-lbaas | 18:40 | |
| *** sorrison has quit IRC | 18:42 | |
| *** sorrison has joined #openstack-lbaas | 18:46 | |
| *** ccamposr has quit IRC | 18:52 | |
| *** gcheresh has joined #openstack-lbaas | 18:57 | |
| *** openstackstatus has joined #openstack-lbaas | 19:22 | |
| *** ChanServ sets mode: +v openstackstatus | 19:22 | |
| *** gcheresh has quit IRC | 19:31 | |
| *** mithilarun has quit IRC | 19:36 | |
| *** mithilarun has joined #openstack-lbaas | 19:46 | |
| *** mithilarun has quit IRC | 20:25 | |
| *** mithilarun has joined #openstack-lbaas | 20:27 | |
| *** mithilarun has quit IRC | 20:31 | |
| *** mithilarun has joined #openstack-lbaas | 20:43 | |
| *** vishalmanchanda has quit IRC | 20:44 | |
| *** mithilarun has quit IRC | 20:49 | |
| *** mithilarun has joined #openstack-lbaas | 21:09 | |
| *** mithilarun has quit IRC | 21:13 | |
| *** gcheresh has joined #openstack-lbaas | 21:16 | |
| *** mithilarun has joined #openstack-lbaas | 21:32 | |
| *** rcernin has quit IRC | 21:42 | |
| *** gcheresh has quit IRC | 22:00 | |
| *** ccamposr has joined #openstack-lbaas | 22:19 | |
| *** maciejjozefczyk has quit IRC | 22:35 | |
| *** dosaboy has quit IRC | 22:36 | |
| *** TrevorV has quit IRC | 22:52 | |
| *** dosaboy has joined #openstack-lbaas | 22:53 | |
| *** rcernin has joined #openstack-lbaas | 22:55 | |
| *** tkajinam has joined #openstack-lbaas | 23:03 | |
| *** born2bake has quit IRC | 23:23 | |
| *** ccamposr has quit IRC | 23:34 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!