14:00:07 <liuyulong> #startmeeting neutron_l3
14:00:08 <openstack> Meeting started Wed Jun 19 14:00:07 2019 UTC and is due to finish in 60 minutes.  The chair is liuyulong. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:12 <openstack> The meeting name has been set to 'neutron_l3'
14:00:16 <mlavalle> o/
14:00:21 <haleyb> o/
14:00:24 <ralonsoh> hi
14:00:41 <tidwellr> hi
14:02:05 <mlavalle> dang our chair person you fell off
14:02:14 <mlavalle> he came back
14:02:23 <liuyulong> I lost the connection
14:02:26 <liuyulong> ...
14:02:37 <liuyulong> The meeting is started or not?
14:02:42 <mlavalle> yes
14:02:52 <liuyulong> #chair mlavalle
14:02:53 <openstack> Current chairs: liuyulong mlavalle
14:02:57 <liuyulong> #chair haleyb
14:02:58 <openstack> Current chairs: haleyb liuyulong mlavalle
14:02:58 <slaweq> hi
14:03:12 <liuyulong> Hello everyone, it is my first time to host L3 meeting. : )
14:03:29 <slaweq> liuyulong: good luck :)
14:03:30 <mlavalle> Nice!
14:03:58 <liuyulong> #action liuyulong lost the connection at the very beginning.
14:04:20 <liuyulong> #topic Announcements
14:07:18 <liuyulong> Terribly sorry...
14:07:30 <mlavalle> don't worry about it
14:07:50 <liuyulong> We are in T-2 dev cycle now.
14:08:07 <liuyulong> #link https://launchpad.net/neutron/+milestone/train-2
14:08:07 <liuyulong> No bugs are targeted to this milestone.
14:08:24 <liuyulong> Only 4 BPs
14:08:35 <mlavalle> there are a few more to come
14:08:44 <mlavalle> they will be in the dashboard soon
14:09:25 <liuyulong> mlavalle, OK
14:09:43 <liuyulong> #link https://cfp.openstack.org/
14:09:57 <liuyulong> Welcome to China and welcome to Shanghai.
14:10:09 <liuyulong> Yes, call for speakers, please submit your presentation for Open Infrastructure Summit Shanghai, and hope to see all you guys there.
14:10:39 <mlavalle> yeah
14:10:48 <mlavalle> looking forward to it
14:11:47 <mlavalle> one thing to mention is that we all need business Chinese visa
14:11:50 <liuyulong> mlavalle and I will submit a port forwarding topic.
14:12:21 <mlavalle> You need an invitation letter from the OpenStack Foundation
14:12:37 <mlavalle> the Foundation will make it available over the next 2 or 3 weeks
14:13:03 <liuyulong> Yes, our community supports us.
14:13:15 <mlavalle> and with that you have to go to the nearest Chinese consulate
14:13:41 <mlavalle> in the USA, there are companies that help you get the visa. In my case, I used in the past cibt.com
14:14:16 <mlavalle> https://cibtvisas.com/china-visa
14:14:48 * mlavalle will go quiet now
14:15:18 <liuyulong> I don't need this, haha
14:15:25 <liuyulong> mlavalle, thanks for the tips
14:15:34 <brinzhang> The application for the visa card is very simple. As long as the conditions are met, it is ok to wait for the application to pass.
14:16:03 <liuyulong> brinzhang, great news, thanks.
14:16:06 <liuyulong> any other announcements?
14:16:42 <liuyulong> OK, let's move on.
14:16:47 <liuyulong> #topic Bugs
14:16:57 <liuyulong> mlavalle is our bug deputy this week
14:17:01 <liuyulong> #link http://lists.openstack.org/pipermail/openstack-discuss/2019-June/007189.html
14:17:09 <brinzhang> https://ccclub.cmbchina.com/CrdCardApply/LoginChannelSelect.aspx?cardsel=7602&WT.mc_id=N3700BD1158E585228CT
14:17:09 <mlavalle> no, I was last week
14:17:09 <liuyulong> s/last week
14:17:16 <mlavalle> yeap
14:17:28 <liuyulong> #link https://bugs.launchpad.net/neutron/+bug/1832307
14:17:29 <openstack> Launchpad bug 1832307 in neutron "Functional test neutron.tests.functional.agent.linux.test_ip_lib.IpMonitorTestCase. test_add_remove_ip_address_and_interface is failing" [Critical,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)
14:17:32 <brinzhang> liuyulong: this is the Zhao Shang Bank
14:17:43 <liuyulong> The fix is here: https://review.opendev.org/#/c/664889
14:17:55 <liuyulong> it is almost done
14:18:05 <liuyulong> I run the test 100 times locally, all passed.
14:19:10 <liuyulong> It's in the gate now (another recheck is needed.)
14:19:29 <mlavalle> ++
14:20:06 <liuyulong> #link https://bugs.launchpad.net/neutron/+bug/1821912
14:20:07 <openstack> Launchpad bug 1821912 in neutron "intermittent ssh failures in various scenario tests" [High,In progress] - Assigned to LIU Yulong (dragon889)
14:20:38 <liuyulong> I did not make much more progress on this issue recently, but I've noticed that seems we have a smaller chance to encounter it now.
14:21:10 <mlavalle> I am investigating failures with the connectivity test with 2 routers
14:21:41 <mlavalle> and some of those ssh failures must come from it
14:22:01 <mlavalle> so I think we are slowly nibbling at it
14:22:51 <mlavalle> is there a test case that produces the most failures?
14:22:57 <liuyulong> This link is pointing to all the patch sets we have: https://review.opendev.org/#/q/bug/1821912, most of them do not have much activities.
14:23:18 <liuyulong> mlavalle, yes, I read the neutron_ci meeting log.
14:23:36 <mlavalle> cool
14:24:04 <liuyulong> mlavalle, no,  I have no data about the most failure case
14:24:41 <liuyulong> totally random, IMOP
14:24:43 <liuyulong> IMO
14:24:48 <mlavalle> are we still using the kibana search query at the top of the bug?
14:24:59 <liuyulong> I have a SPEC https://review.opendev.org/#/c/662541/, the idea is coming out from this bug. Also, no much activities.
14:25:40 <mlavalle> I see you added a query in note #16
14:25:54 <mlavalle> I'll give it a try nd see if I can identify a pattern
14:26:23 <liuyulong> http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22line%20107%2C%20in%20_get_ssh_connection%5C%22%20AND%20build_status%3A%5C%22FAILURE%5C%22%20AND%20build_branch%3A%5C%22master%5C%22%20AND%20NOT%20build_name%3A%5C%22neutron-tempest-plugin-dvr-multinode-scenario%5C%22%20AND%20project%3A%5C%22openstack%2Fneutron%5C%22%20AND%20NOT%20build_name%3A%5C%22networking-ovn-tempest-dsvm-ovs-release%5C%22
14:26:50 <liuyulong> ^ this was a newer log query I left before.
14:26:54 <mlavalle> and I'll go back to the probe RFE
14:27:16 <mlavalle> good to know it is connected to this bug
14:28:12 <liuyulong> I will continue pay attention on this bug.
14:28:18 <mlavalle> thanks
14:28:29 <liuyulong> #link https://bugs.launchpad.net/neutron/+bug/1831534
14:28:30 <openstack> Launchpad bug 1831534 in neutron "[l3][dvr] with openflow security group east-west traffic between different vlan networks is broken" [High,In progress] - Assigned to yangjianfeng (yangjianfeng)
14:30:09 <liuyulong> mlavalle, cool
14:30:13 <liuyulong> The first version fix is here: https://review.opendev.org/#/c/663008/, after some tests locally, IMO, the potential impact of this change is too wide. Almost every L3 scenario traffic is affected.
14:30:27 <liuyulong> For instance, dvr router floating IP, dvr_no_external centrilized floating IP, vlan networks east-west traffic, vlan network to tunnel network east-west traffic, L3 dvr east-west traffic between dvr and dvr_no_external, and various combinations based on the state of ovs-agent security group: enable_security_group = False/True.
14:30:40 <liuyulong> And neutron upstream CI does not cover all these cases, IMO, at least vlan networks east-west traffic is not covered. So we may need to test these cases manually. It is not achievable, IMO, people always forget some details.
14:31:11 <liuyulong> So, I have an alternative fix for this now: https://review.opendev.org/#/c/665517/ Comparing with Jianfeng's fix, this now has the minimum influence. It only touches one scenario: the VLAN networks with enabled security group.
14:32:15 <liuyulong> Am I still here?
14:32:19 <mlavalle> yes
14:32:38 <mlavalle> so your point is trying to be the least disruptive, right?
14:35:15 <liuyulong_> #action I'm the backup.
14:35:54 <liuyulong> But I have a question about how the ovs-agent take care of those flows based on the physical VLAN id when update the network segmentation_id.
14:36:23 <liuyulong> And after some test, if restart the ovs-agent, the new vlan id will be used for these related flows.
14:36:41 <ralonsoh> there is no need to restart the agent
14:36:46 <liuyulong> So it's OK. Still the small fix for this bug.
14:36:47 <ralonsoh> that's the point of this feature
14:36:59 <ralonsoh> yes, slaweq has two patches upstream to fix it
14:38:12 <tidwellr> this issue is a strange one, I'm surprised to see people attempting to use DVR with VLAN networks. I was always under the impression it was known that DVR only works with vxlan networks, but there's no reason we shouldn't make this work
14:38:19 <liuyulong> Let me quickly go through the bugs.
14:38:24 <liuyulong> Connection is so unstable...
14:38:37 <mlavalle> tidwellr: +++
14:39:07 <liuyulong> ralonsoh, if ovs-agent restart is not needed, then those flows will not be updated.
14:39:27 <slaweq> liuyulong: they should be I think
14:39:38 <ralonsoh> liuyulong, which flows? in which bridge?
14:39:53 <liuyulong> #link https://bugs.launchpad.net/neutron/+bug/1831706
14:39:55 <openstack> Launchpad bug 1831706 in neutron "[DVR] Modify `in_port` field of packets which from remote qr-* port" [Undecided,In progress] - Assigned to yangjianfeng (yangjianfeng)
14:40:08 <liuyulong> Based on the former bug, Jianfeng wants to start a new approach which is trying to simplify the security group openflow flows.
14:40:17 <liuyulong> And the gerrit link is https://review.opendev.org/#/c/665647/.
14:40:32 <liuyulong> This may be significant for large scale deployment with countless user security group rules.
14:40:48 <liuyulong> #link https://bugs.launchpad.net/neutron/+bug/1832745
14:40:49 <openstack> Launchpad bug 1832745 in neutron "_update_network_segmentation_id KeyError: 'provider:network_type'" [Medium,New]
14:41:04 <liuyulong> During the recent local testing, I noticed everytime I restart the OVS-agent, I will get such KeyError: 'provider:network_type' log. And seems, as haleyb mentioned in LP, in neutron we have many place which is in such style network['provider:network_type']. Each one may need to take care.
14:41:56 <liuyulong> ralonsoh, physical bridges (e.g. br-vlan)
14:42:12 <liuyulong> That's all from me.
14:42:17 <liuyulong> So, are there any other bugs that need the team to pay attention?
14:42:32 <ralonsoh> liuyulong, please take a look at the function: I change the connection between br-int and br-phys
14:43:04 <haleyb> liuyulong: regarding that bug ^^^, is it only provider networks that are affected?
14:43:29 <haleyb> i.e. others don't have this segment list
14:43:42 <liuyulong> haleyb, for my test env, it is router HA networks.
14:44:32 <ralonsoh> liuyulong, I'll take care of this bug
14:44:46 <ralonsoh> this is the problem of not having a common API (or object) for agents
14:44:57 <liuyulong> ralonsoh, cool, thanks
14:45:09 <ralonsoh> everything passed from the server is an undefined dict
14:45:10 <liuyulong> No? Let's move on.
14:45:14 <liuyulong> #topic Routed Networks
14:45:19 <liuyulong> mlavalle, tidwellr, wwriverrat: if you guys need to continue your discussion, please go ahead.
14:45:43 <mlavalle> as mentioned yesterday during the neutron meeting
14:46:07 <mlavalle> 1) ralonsoh looking at sriov implications. There doesn't seem to be many
14:46:51 <mlavalle> 2) I'm deploying the patch in my ovs system, to gauge implications.
14:47:10 <mlavalle> with this^^^ I think we can help wwriverrat to complete the spec
14:47:53 <mlavalle> the patch is here: https://review.opendev.org/#/c/657170
14:48:09 <mlavalle> and that's all I have for today
14:48:36 <tidwellr> on the floating IP's front, I was thinking of pushing a WIP patch illustrating my idea around https://review.opendev.org/#/c/486450/
14:50:37 <mlavalle> that woyuld be great
14:50:40 <mlavalle> would
14:51:55 <tidwellr> I don't this approach is terribly invasive, I think we just need to agree on an approach
14:52:00 <tidwellr> *think
14:52:04 <mlavalle> yes
14:53:51 <liuyulong> again...sorry for that
14:54:05 <liuyulong> #topic On demand agenda
14:54:57 <liuyulong> I wasted some time in the re-connection.
14:55:19 <liuyulong> And we are running out of time.
14:55:52 <mlavalle> no additional topics from me
14:56:11 <liuyulong> Anything else to discuss here today?
14:56:41 <liuyulong> OK, a happy ending
14:56:49 <liuyulong> #endmeeting