14:01:34 <mlavalle> #startmeeting neutron_l3
14:01:35 <openstack> Meeting started Wed Mar 27 14:01:34 2019 UTC and is due to finish in 60 minutes.  The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:01:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:01:38 <openstack> The meeting name has been set to 'neutron_l3'
14:01:47 <njohnston> o/
14:01:51 <ralonsoh> hi
14:02:48 <mlavalle> how are you all?
14:03:01 <liuyulong> hi
14:03:07 <ralonsoh> Very well!
14:03:42 <igordc> hi
14:03:54 <mlavalle> igordc: hi, welcome
14:04:01 <igordc> I'm alright, thanks
14:04:16 <mlavalle> #topic Announcements
14:05:29 <mlavalle> Two announcements:
14:06:08 <mlavalle> 1) We cut RC-1 over the weekend and yesterday during the Neutron weekly meeting we decided that there won't be another RC this cycle
14:06:24 <mlavalle> so we are merging code to master as normal again
14:07:12 <mlavalle> 2) We have an etherpad to propose topics for the PTG in Denver: https://etherpad.openstack.org/p/openstack-networking-train-ptg
14:08:17 <mlavalle> I can see that 13 people have already registered and 11 confirmed attendance to social event Friday night
14:08:21 <mlavalle> \o/
14:08:33 <mlavalle> davidsha is attending!
14:08:38 <slaweq> hi
14:08:48 <mlavalle> igordc: are you planning to be in Denver?
14:09:06 <njohnston> Looking forward to seeing davidsha again!
14:09:16 <igordc> mlavalle, no, manjeets will be covering for me
14:09:44 <ralonsoh> igordc, what a pity...
14:10:00 <igordc> ralonsoh, so close but so far... :)
14:11:12 <mlavalle> igordc: ok, please look at line 47 of the etherpad. We want to review DVR openflow. the idea is that I would like to merge the code as eary as possible in the cycle, ideally before T-1, so we can debug it for the rest of the cycle
14:11:33 <mlavalle> manjeets should be ready to discuss the status in Denver
14:11:40 <igordc> mlavalle, yep
14:11:48 <mlavalle> igordc: cool
14:12:08 <mlavalle> any other announcements?
14:12:19 <igordc> mlavalle, I'm also going to stay on etherpad and IRC during the PTG.. any plans for attempting live streaming like those few times before?
14:13:29 <mlavalle> igordc: we have done that before. I will try it again this time aorund. Worst case, we can have a hangout with you on the dvr topic
14:14:20 <igordc> mlavalle, awesome
14:14:21 <mlavalle> njohnston: nice topic: IPAM... I agree
14:15:07 <mlavalle> ok next topic...
14:15:12 <mlavalle> #topic Bugs
14:15:17 <njohnston> I'm interested in the DVR openflow discussion as well.  I'm especially interested in how we can manage the complexity of having so many kinds of DVR and supporting them all adequately.  I think the instability of the DVR+L3HA job shows us that there is real operational and cognitive overhead there.
14:16:23 <mlavalle> njohnston: also agree
14:16:39 <mlavalle> and that includes lack of documentation
14:17:05 <igordc> njohnston, I can see how dvr openflow can potentially also make things a bit more stable since there are less moving parts overall
14:17:29 <liuyulong> It may needs more dev cycle to make it stable. Please follow the team and trace the bug to make it steady.
14:17:37 <igordc> njohnston, but to get to feature parity with classic backend would still take a long while
14:17:53 <igordc> liuyulong, yep
14:18:22 <mlavalle> ok, first bug today:
14:18:24 <mlavalle> https://bugs.launchpad.net/neutron/+bug/1789434
14:18:25 <openstack> Launchpad bug 1789434 in neutron "neutron_tempest_plugin.scenario.test_migration.NetworkMigrationFromHA failing 100% times" [High,In progress] - Assigned to Miguel Lavalle (minsel)
14:18:54 <mlavalle> I think we can close this one with the merge of https://review.openstack.org/636710
14:19:18 <mlavalle> and done
14:19:53 <mlavalle> Next one is https://bugs.launchpad.net/neutron/+bug/1821912
14:19:54 <openstack> Launchpad bug 1821912 in neutron "intermittent ssh failures in various scenario tests" [High,Confirmed]
14:20:14 <mlavalle> This is a bug report that we talke about yesterday during the CI meeting
14:20:20 <slaweq> yes
14:20:25 <mlavalle> it is intended to be a collective effort
14:20:52 <mlavalle> since at this point we haven't found a unifying pattern
14:21:16 <mlavalle> so slaweq and I will be using the logstash query to find cases and then dig deeper in them
14:21:24 <mlavalle> others are welcome to do the same
14:21:41 <slaweq> there is a lot of such examples in logstash, so it's easy to find :)
14:22:13 <mlavalle> slaweq: thanks for filing it
14:22:22 <slaweq> yw mlavalle :)
14:22:57 <mlavalle> we also have https://bugs.launchpad.net/neutron/+bug/1774459
14:22:58 <openstack> Launchpad bug 1774459 in neutron "Update permanent ARP entries for allowed_address_pair IPs in DVR Routers" [High,Confirmed] - Assigned to Swaminathan Vasudevan (swaminathan-vasudevan)
14:23:18 <mlavalle> progress on this has stalled since early January
14:24:05 <mlavalle> I'll send a message to swami today and ask about it
14:24:24 <mlavalle> if anyone wants to tackle this, please feel free
14:25:03 <mlavalle> those are all the bugs I have today
14:25:10 <mlavalle> any other bugs we should discuss?
14:26:14 <liuyulong> I have
14:26:15 <liuyulong> https://launchpad.net/bugs/1819160
14:26:17 <openstack> Launchpad bug 1819160 in neutron "Functional tests for dvr ha routers are broken" [High,In progress] - Assigned to LIU Yulong (dragon889)
14:26:44 <liuyulong> For now, I think we have found the root cause of this failure.
14:26:46 <liuyulong> https://review.openstack.org/#/c/647784/
14:26:48 <igordc> oh I've been a victim of that one
14:27:20 <liuyulong> That HA failover bridges veth pair devices were not set UP during the test.
14:27:54 <liuyulong> This causes the VRRP advertisement packet can not pass to each HA port. So they set themselves all 'master'.
14:27:58 <slaweq> liuyulong: but it is needed together with https://review.openstack.org/#/c/642220/4 IIUC
14:28:12 <liuyulong> slaweq, Yes
14:28:15 <slaweq> ok
14:28:40 <liuyulong> For this patch https://review.openstack.org/#/c/642220/, we are going to give each HA case a specific VRRP id.
14:28:58 <liuyulong> And each HA port a independent IP address.
14:29:45 <liuyulong> https://review.openstack.org/#/c/642220/4/neutron/tests/functional/agent/l3/test_dvr_router.py There are some more cases need to add such IP, not only the failover cases.
14:30:46 <liuyulong> ralonsoh's patch may also be needed.
14:31:13 <liuyulong> https://review.openstack.org/#/c/645225/
14:31:21 <ralonsoh> so we need 1) the veth links up, 2) make the IP different and 3) check both routers status
14:31:54 <slaweq> ralonsoh++
14:31:55 <liuyulong> ralonsoh, yes
14:32:33 <liuyulong> ralonsoh's patch handled the situation is router2 can also be 'master' during the test.
14:32:33 <mlavalle> is this latter patch rebased on one of the former two?
14:32:53 <ralonsoh> I'll rebase my patch to yours, liuyulong
14:33:00 <mlavalle> ralonsoh: +++
14:33:00 <liuyulong> ralonsoh's patch can be independent.
14:33:23 <igordc> is it worth doing some manual testing of this? any recommendations to try and make it fail?
14:34:26 <liuyulong> igordc, I just run tox locally
14:35:41 <mlavalle> anything else on this bug?
14:36:39 <mlavalle> any other bugs?
14:36:52 <liuyulong> Not from me
14:37:10 <igordc> mlavalle, if I suspect a bug, is this a good place to mention it or just I first report it in launchpad always?
14:37:11 <mlavalle> ok, let's move on
14:37:31 <igordc> or should I first*
14:37:54 <mlavalle> igordc: just report it in launchpad at tag it l3-dvr-backlog
14:38:09 <mlavalle> and^^^
14:38:24 <igordc> mlavalle, what's the tag for non dvr?
14:38:43 <mlavalle> l3-ipam-dhcp
14:39:05 <igordc> mlavalle, thanks
14:39:14 <mlavalle> ok, moving on
14:39:22 <mlavalle> #topic openflow dvr
14:39:32 <mlavalle> igordc: you are up
14:39:43 <igordc> xubozhang also in the house
14:40:03 <igordc> anyway, we've been polishing what we have
14:40:33 <igordc> I've personally started the implemention of legacy for dvr, but am facing some issues and doubts regarding existing DNAT and SNAT in the openflow dvr patch
14:41:14 <igordc> I don't see any risk for the scope we discussed on the mailing list except the scenario job, for <train-1
14:41:53 <igordc> so I'd like to ask whether merging every needed patch without a scenario job is something acceptable before train-1.... and later work on the scenario job?
14:42:16 <mlavalle> I think that's ok
14:42:39 <mlavalle> let's just try to keep things close to T-1
14:43:08 <igordc> I'm also moving to other work at the moment, but will continue to hold the l3 agent refactor.. xubozhang will mostly hold the openflow dvr patch
14:43:21 <igordc> mlavalle, yep
14:43:53 <igordc> please review https://review.openstack.org/#/c/639605/ once I rebase it again
14:44:08 <igordc> I expect it will be controversial due to the existing agent changes, so time will be needed to address things
14:44:49 <mlavalle> yes, that is why I want to do this early
14:45:01 <mlavalle> are thse all the patches: https://review.openstack.org/#/q/topic:openflow-based-dvr+(status:open+OR+status:merged)
14:45:28 <igordc> mlavalle, that's right, that is the new topic per liuyulong's recommendation
14:45:46 <igordc> mlavalle, we will tag the commit with it too so it doesn't reset the topic
14:45:49 <mlavalle> ok I will add it to the blueprint in launchpad
14:45:58 <liuyulong> Yes, it is based on the BP name.
14:46:20 <mlavalle> that way it is easy to find the patches
14:47:18 <igordc> also something that might not be clear:
14:47:42 <igordc> the refactor patch actually depends on yang's router factory patch: https://review.openstack.org/#/c/620349
14:48:23 <mlavalle> ahh good point
14:50:05 <mlavalle> maybe we should put this patch under the samae topic
14:50:17 <mlavalle> gerrit topic^^^^
14:51:15 <mlavalle> anything else?
14:51:56 <igordc> haleyb, what's the state of neutron + ovn + dvr today?
14:53:00 <mlavalle> I don't think he is in the meeting today
14:53:17 <igordc> alright
14:53:21 <igordc> that's all from my side
14:53:36 <mlavalle> thanks for the update igordc
14:53:47 <mlavalle> #topic On demand agenda
14:54:04 <mlavalle> anything else we should discuss today?
14:55:01 <mlavalle> ok, thanks for attending
14:55:22 <mlavalle> especially you liuyulong and igordc. These are odd hours for you
14:55:29 <igordc> thank you!
14:55:37 <mlavalle> #endmeeting