15:01:25 <haleyb> #startmeeting neutron_l3
15:01:26 <openstack> Meeting started Thu Dec 20 15:01:25 2018 UTC and is due to finish in 60 minutes.  The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:30 <openstack> The meeting name has been set to 'neutron_l3'
15:02:21 * haleyb will wait knew some would be late
15:03:02 <Swami> hi
15:03:13 <haleyb> hi Swami
15:03:44 <haleyb> i guess we can start then
15:03:56 <haleyb> #topic Announcements
15:04:03 <Swami> I think the turnout might be low, because of the holiday week.
15:04:33 <haleyb> There will be no meeting next week as I'm assuming everyone in US will be on break
15:04:53 <Swami> haleyb: makes sense.
15:05:54 <Swami> haleyb: we discussed about the 6a.m timeslot a while back, what is that for. Is it for the L3 meeting or is it for the other meeting.
15:06:03 <haleyb> Only other announcement is Stein-2 is January 7th, so we're in the last couple of weeks to merge code
15:06:34 <haleyb> Swami: miguel wanted to see about making this meeting one hour earlier, which would be 6am your time
15:06:44 <haleyb> he has a conflict at this time
15:07:10 <Swami> haleyb: Ok. That's what I thought.
15:07:41 <Swami> haleyb: Do you have any big plans for the break.
15:08:22 <haleyb> Swami: nothing big, skiing if the weather cooperates, how about you?
15:08:54 <Swami> haleyb: I have taken off next week, but need to check with my kids where they wanted to go.
15:10:29 <haleyb> Disney Land Dad! :)
15:11:06 <Swami> haleyb: True, going to Southern California would be a nice choice based on the weather conditions.
15:12:35 <haleyb> alright, let's continue...
15:12:43 <haleyb> #topic Bugs
15:13:38 <Swami> There is a new bug that was filed this week for DVR. #link https://bugs.launchpad.net/neutron/+bug/1807153
15:13:39 <openstack> Launchpad bug 1807153 in neutron "Race condition in metering agent when creating iptable managers for router namespaces" [Undecided,New]
15:13:56 * haleyb was supposed to triage that one
15:14:13 <Swami> There is a patch under review for this bug. #link https://review.openstack.org/#/c/621165/
15:15:00 <Swami> I have not yet triaged this, but will take a look at it. It seems this happens when the namespace is not ready for the router and when the metering rules are getting added.
15:15:20 <haleyb> i think it can happen based on other bugs we've fixed in the metering agent
15:15:50 <Swami> haleyb: agreed.
15:16:10 <haleyb> i've update it
15:16:23 <Swami> ok.
15:16:35 <Swami> The next one in the list is #link https://bugs.launchpad.net/neutron/+bug/1794991
15:16:37 <openstack> Launchpad bug 1794991 in neutron "Inconsistent flows with DVR l2pop VxLAN on br-tun" [Undecided,New]
15:17:04 <Swami> haleyb: I think we got some more information about the bug.
15:17:57 <Swami> This one as such could not be reproduced and i am not sure if only load on the compute will cause this or constant restart of the openvswitch agent would cause this. The table 22 seems to be the odd one right now.
15:18:13 <Swami> Let me focus on that table to see why the l2pop rules are getting messed up.
15:18:41 <haleyb> ok, thanks
15:18:53 * slaweq is back, sorry for being late
15:19:12 <Swami> slaweq: hi
15:19:18 <slaweq> hi Swami
15:19:20 <Swami> The next one in the list is #link https://bugs.launchpad.net/neutron/+bug/1774459
15:19:22 <openstack> Launchpad bug 1774459 in neutron "Update permanent ARP entries for allowed_address_pair IPs in DVR Routers" [High,Confirmed] - Assigned to Swaminathan Vasudevan (swaminathan-vasudevan)
15:20:31 <Swami> haleyb: slaweq: I need someone to take a look at this patch. Since it is getting more complex, we don't want to mess up the flow rules. haleyb I did read the comments from last week that you are in the process of reviewing this patch.
15:20:59 <Swami> #link https://review.openstack.org/#/c/601336/
15:21:14 <haleyb> Swami: yes, have not finished yet
15:21:45 <Swami> haleyb: I have been addressing some refactor comments from Ryan so far.
15:22:00 <Swami> But really we need L2 experts to take a look at it.
15:22:37 <slaweq> Swami: I'm not an expert but I will take a look
15:22:48 <Swami> The concern i have is on the exposing the DVR MAC on the local switching table in br-int.
15:23:00 <Swami> slaweq: Thanks, I would appreciate.
15:24:00 <Swami> haleyb: I think that's all I had for the DVR bugs. Back to you
15:24:18 <Swami> I may have to drop in another 5 mins.
15:24:30 <Swami> Happy Holidays to all.
15:24:34 <haleyb> ok, thanks Swami.
15:24:54 <slaweq> thx Swami, happy holidays to You too :)
15:25:51 <haleyb> i had added a bug yesterday
15:25:54 <haleyb> https://bugs.launchpad.net/neutron/+bug/1809134
15:25:55 <openstack> Launchpad bug 1809134 in neutron "TypeError in QoS gateway_ip code in l3-agent logs" [High,In progress] - Assigned to Brian Haley (brian-haley)
15:26:19 <haleyb> slaweq: that was one from the CI meeting, have to file for the second one was just trying to triage a little
15:26:32 <haleyb> https://review.openstack.org/#/c/626401
15:26:40 <haleyb> there are some comments i need to address
15:26:45 <slaweq> haleyb: thx
15:28:20 <haleyb> let me see if there were other bugs from last week
15:28:53 <haleyb> https://bugs.launchpad.net/neutron/+bug/1806770
15:28:54 <openstack> Launchpad bug 1806770 in neutron "DHCP Agent should not release DHCP lease when client ID is not set on port" [Medium,In progress] - Assigned to Arjun Baindur (abaindur)
15:29:01 <haleyb> https://review.openstack.org/#/c/623066/
15:29:44 <haleyb> has not been updated in a couple of weeks, i will take over next meeting if it's still the same
15:30:04 <haleyb> https://bugs.launchpad.net/neutron/+bug/1802006
15:30:06 <openstack> Launchpad bug 1802006 in neutron "Floating IP attach/detach fails for non-admin user and unbound port with router in different tenant" [Medium,In progress] - Assigned to Brian Haley (brian-haley)
15:30:11 <haleyb> https://review.openstack.org/#/c/622623/
15:30:24 <haleyb> this just refuses to merge
15:31:24 <haleyb> recheck again
15:31:42 <haleyb> https://bugs.launchpad.net/neutron/+bug/1804327
15:31:43 <openstack> Launchpad bug 1804327 in neutron "occasional connection reset on SNATed after tcp retries" [Medium,In progress] - Assigned to Dirk Mueller (dmllr)
15:31:50 <haleyb> https://review.openstack.org/#/c/618208/
15:33:31 <haleyb> i will review this one too
15:34:09 <haleyb> last one is https://bugs.launchpad.net/neutron/+bug/1798475
15:34:10 <openstack> Launchpad bug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,In progress] - Assigned to LIU Yulong (dragon889)
15:34:27 <haleyb> https://review.openstack.org/#/c/625054/
15:35:23 <haleyb> i have not looked at review yet
15:35:29 <liuyulong> Yes, I'm still working on this.
15:35:35 <slaweq> there is WIP patch for this one but it looks that this test is still failing on it :/
15:36:23 <liuyulong> All the failed LOG has `qg-port` not present in bridge br-int info.
15:37:21 <liuyulong> But I'm not quite sure if this is the root cause.
15:38:19 <haleyb> liuyulong: the l3-agent log has that or ovs agent?
15:38:30 <liuyulong> ovs-agent
15:39:17 <liuyulong> The LOG is basiclly near the ping loss time.
15:40:55 <slaweq> liuyulong: that is interesting
15:40:56 <liuyulong> I cannot reproduce the issue in my devstack env. Create a router, set gateway, ping gateway IP, kill -9 l3 agent. Everything works fine then.
15:41:09 <haleyb> i didn't see that in the latest logs, maybe i'm missing it
15:42:29 <slaweq> liuyulong: does it happen on "host" which should be master or backup?
15:42:32 <slaweq> or both?
15:44:29 <liuyulong> master for now, since the gateway IP set there. Or maybe I should kill the backup l3-agent and see again.
15:44:59 <liuyulong> haleyb, http://logs.openstack.org/09/608909/20/check/neutron-fullstack/c7b6401/logs/dsvm-fullstack-logs/TestHAL3Agent.test_ha_router_restart_agents_no_packet_lost/neutron-openvswitch-agent--2018-11-30--03-37-05-254669.txt.gz#_2018-11-30_03_37_34_555
15:45:20 <liuyulong> this is the LOG from slaweq's comment https://bugs.launchpad.net/neutron/+bug/1798475/comments/4
15:45:22 <openstack> Launchpad bug 1798475 in neutron "Fullstack test test_ha_router_restart_agents_no_packet_lost failing" [High,In progress] - Assigned to LIU Yulong (dragon889)
15:46:27 <haleyb> ah, i was looking for the wrong string, it's in the latest log too
15:47:26 <slaweq> I'm not sure if that is real issue - we are using same ovs for all tests/agents so maybe it's just side effect of adding port to some other bridge
15:50:01 <liuyulong> This is really a tough one, but IMO the l3 agent may remove the qg-device unexpectedly
15:50:45 <slaweq> liuyulong: if it is like You are saying, then it's real issue, not only tests issue :/
15:51:28 <liuyulong> L3 agent may get a router info without gateway info due to the re-consuming notification.
15:52:20 <liuyulong> But seems not so much right, because l3 agent will try to retrieve router info from neutron-server everytime.
15:54:17 <liuyulong> If you guys have any information please directly add it to the gerrit. : )
15:54:38 <haleyb> yes, i thought we just changed that code for the 2 dvr routers case, i'll try to look at the review again
15:55:05 <slaweq> haleyb: yep, but IIRC it was failing before this big patch too
15:55:12 <haleyb> :(
15:55:55 <haleyb> we're running out of time...
15:55:59 <haleyb> #topic Open discussion
15:56:11 <haleyb> anything else someone wants to discuss?
15:58:54 <haleyb> alright, have a good time off everyone and see you in the new year!
15:59:02 <haleyb> #endmeeting