14:00:14 <haleyb> #startmeeting neutron_drivers 14:00:14 <opendevmeet> Meeting started Fri Apr 25 14:00:14 2025 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:14 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:14 <opendevmeet> The meeting name has been set to 'neutron_drivers' 14:00:22 <haleyb> Ping list: ykarel, mlavalle, mtomaska, slaweq, obondarev, tobias-urdin, lajoskatona, haleyb, ralonsoh 14:00:27 <slaweq> o/ 14:00:28 <obondarev> o/ 14:00:30 <ralonsoh> hello 14:00:34 <yusufgungor_> o/ 14:01:43 <haleyb> hopefully one more driver will come, but we do have 4 so can get started 14:02:10 <haleyb> slaweq: did you want to start with https://bugs.launchpad.net/neutron/+bug/2106816 ? it was on the agenda for a previous week 14:02:24 <slaweq> sure 14:02:44 <mlavalle> \o 14:02:46 <slaweq> basically this is just an idea about improvement 14:02:59 <slaweq> this is already available in ovn since some time 14:03:39 <slaweq> so I was thinking on doing that in neutrn as new qos rule type 14:04:09 <slaweq> of course if there will be no hard "no" for this idea I will do spec first as this will require new APIs, etc. 14:05:15 <ralonsoh> just asking, what does this feature provides? 14:05:37 <haleyb> the description in the OVN change makes it sound like limiting the number of things in a CT zone, but it's more at QoS? 14:06:02 <ralonsoh> right (I was just asking to make it public here) 14:06:07 <slaweq> possiblity to set max conntrack entries per zone, not globally 14:06:38 <slaweq> currently number of connections is set on node globally and with that it could be changed per port or per network 14:07:51 <ralonsoh> I think the idea of using the QoS API is good, IMO 14:08:27 <ralonsoh> I would ask for a spec, because this is a complex feature, but I'm good with this 14:08:40 <mlavalle> in priciple it is a good idea, but I 14:08:49 <slaweq> for sure I will write spec for this before doing any code changes :) 14:08:50 <mlavalle> would really like to see a spec 14:09:38 <haleyb> agreed, since it took me a minute to figure out how QoS applied, but it's more just the API to get the info into neutron 14:10:09 <haleyb> if i'm understanding it correctly 14:10:21 <slaweq> yeap, you are 14:10:42 <mlavalle> since a spec will be provided, +1 from me 14:10:45 <ralonsoh> +1 14:10:54 <haleyb> +1 from me 14:11:01 <opendevreview> Rico Lin proposed openstack/neutron master: Fix ovn db sync with log resources https://review.opendev.org/c/openstack/neutron/+/948053 14:11:02 <opendevreview> Rico Lin proposed openstack/neutron master: Update instead of recreate acl in ovn sync https://review.opendev.org/c/openstack/neutron/+/948215 14:11:43 <haleyb> obondarev: vote? 14:11:56 <obondarev> +1 with the spec 14:12:26 <haleyb> great, that's three, i'll update the rfe flags, etc 14:13:08 <haleyb> next one i added and i see yusufgungor_ is here 14:13:10 <slaweq> thanks 14:13:12 <haleyb> #link https://bugs.launchpad.net/neutron/+bug/2107634 14:14:07 <yusufgungor_> can i get your comments? do you think it is doable? 14:14:19 <opendevreview> Rico Lin proposed openstack/neutron master: Update instead of recreate acl in ovn sync https://review.opendev.org/c/openstack/neutron/+/948215 14:15:42 <haleyb> yusufgungor_: can you give a short overview as it's a large descrtiption in the bug 14:16:46 <yusufgungor_> we aim to apply acl rules between the tenant networks so trying to send the different vxlan tenant networks traffic to the external firewall 14:17:08 <yusufgungor_> but using dvr and bgp, it is not possible for the instances which are located on the same host 14:17:33 <slaweq> can you maybe use neutron-fwaas to apply SG rules on the router's level? Would that be solution for you maybe? 14:18:20 <yusufgungor_> we have tried the security group rules but no chance 14:18:24 <yusufgungor_> We cannot use Security Groups because we cannot manage ACLs centrally and easily, and as discussed in a bug report we previously submitted, packet loss during live migration increases dramatically as the number of rules grows. Neutron developers informed us that there is no definitive solution for this, and it operates on a best-effort basis. (https://bugs.launchpad.net/neutron/+bug/1970606) Therefore, we need to 14:18:24 <yusufgungor_> route the traffic between tenant networks through a physical firewall. 14:19:10 <ralonsoh> so what you are basically proposing is a way to, via configuration, revert https://review.opendev.org/c/openstack/neutron/+/355062 14:19:17 <ralonsoh> is that a quick summary? 14:19:29 <slaweq> I'm not talking abotut SGs but fwaas: https://docs.openstack.org/neutron/latest/admin/fwaas.html which is a bit different thing 14:20:47 <yusufgungor_> @ralansoh yes 14:22:17 <yusufgungor_> @slaweq you are right but we have not tried fwaas because it was deprecated once before, so we had doubts about using it 14:22:33 <mlavalle> ralonsoh: wasn't that patch reverted here https://review.opendev.org/c/openstack/neutron/+/468075? 14:23:02 <ralonsoh> mlavalle, sorry, the other one: https://review.opendev.org/c/openstack/neutron/+/283757 14:23:35 <slaweq> mlavalle isn't https://review.opendev.org/c/openstack/neutron/+/474007 adding the same again? 14:24:36 <ralonsoh> in any case, is the code related to https://bugs.launchpad.net/neutron/+bug/1577488 14:26:14 <yusufgungor_> @ralansoh yes you are right. actually one can say that this behaviour is normal because these tenant networks are belong to same address scope 14:26:21 <slaweq> I am reading this LP once again and there is one thing which worries me. You said "instances in VXLAN tenant networks located in different routers within different projects can directly access each other if they are on the same compute host." - does it means that instances from 2 different tenant networks can talk to each other if they are on the same host? 14:26:31 <slaweq> or am I misunderstanding something there? 14:28:27 <yusufgungor_> @slaweq you are right, that is the case, the instances which are on the same host can talk the each other directly on the fip netns of the host 14:28:43 <mlavalle> I understood that from the LP 14:28:45 <haleyb> i'm assuming only if in the same address scope? 14:28:53 <ralonsoh> well, they can route the traffic, not talk directly 14:29:03 <yusufgungor_> because the external gw of the routers are the same vlan provider network. tenant networks and the provider networks also on the same address scope. (bgp requirements) 14:29:07 <slaweq> but how - do the have somehow L2 connectivity or L3 using SNAT/FIPs ? 14:30:00 <slaweq> ralonsoh ok, I think I understand now 14:30:21 <yusufgungor_> sorry, when saying directly i mean routing. (instead of the default gw of the provider networks) 14:30:31 <ralonsoh> but I have question (because I dont' remember now the architecture) 14:30:39 <ralonsoh> there is a FIP namespace per router, right? 14:30:50 <slaweq> so basically neutron works as expected here, the only problem is that there is no way for the operator to apply rule which would prevent such kind of communication between different tenant networks using External network 14:31:05 <yusufgungor_> actually not per router, per external provider network, we have seen in our tests 14:31:11 <slaweq> ralonsoh FIP namespace is one per external network IIRC 14:31:48 <yusufgungor_> @org.matrix:slaweq yes, we are looking for a way to prevent that routing 14:31:52 <ralonsoh> hmmmm yeah, that could be a problem if we just allow routing any port inside this namespace 14:32:15 <ralonsoh> another question: how do we create this address scopes? 14:32:29 <ralonsoh> because this is what is allowing the routing between IPs inside the FIP namespace 14:33:14 <haleyb> openstack address scope create ... 14:33:50 <ralonsoh> no, I mean how are we "connecting" these subnets inside the FIP namespace 14:34:36 <haleyb> i know we wrote a whole section in the docs for these cases so that users could allocate blocks, but think that patch is just doing it ? 14:35:03 <haleyb> i.e. dvr_local_router.py 14:35:47 <haleyb> it's been a while since i've looked at dvr as well 14:35:57 <ralonsoh> If a gateway is configured and if it has internal ports that belong 14:35:57 <ralonsoh> to the same address_scopes then no need to add the redirect rules. 14:35:57 <ralonsoh> At the same we should also add a static route in the fip namespace 14:35:57 <ralonsoh> for every interface that is connected to the router that belongs to 14:35:58 <ralonsoh> the same address scope. 14:36:02 <ralonsoh> from the patch 14:36:19 <ralonsoh> should we also check that the port belongs to the same router of the FIP? 14:36:44 <ralonsoh> because we can have several subnets, belonging to different routers, with the same address scope 14:36:49 <ralonsoh> that is something legit 14:38:43 <haleyb> we would need to loop-in someone like liushy since i think he still runs ml2/ovs/dvr in production 14:38:43 <yusufgungor_> actually the routes on the fip netns from different routers used to get traffic inside to the compute node when using bgp. so these routes are acceptable but they should only used for the incoming traffic to the agent gw port. not the traffic goes from qrouters to the fip netns 14:39:56 <ralonsoh> is that something that we can reproduce without bgp? 14:40:01 <yusufgungor_> you check the solution file which we have added to the bug report 14:40:01 <yusufgungor_> https://launchpadlibrarian.net/788646867/fip-netns-ip-route-rule.txt 14:40:15 <ralonsoh> just with two routers with dvr and several VMs in the same compute? 14:40:16 <slaweq> haleyb I don't know about liushy but for sure liuyulong is using it in their production 14:40:25 <haleyb> slaweq: sorry wrong nick 14:41:14 <mlavalle> yusufgungor_: you've deployed https://launchpadlibrarian.net/788646867/fip-netns-ip-route-rule.txt and achieved the result you describe above? 14:41:30 <yusufgungor_> @ralonsoh i am not sure, we can try to disable bgp, removing address scopes etc to see it. if you need that 14:42:06 <ralonsoh> not removing address scopes, that is a neutron feature 14:42:28 <ralonsoh> but seems that can be reproduced just with a couple of routers, one FIP and one single compute node with a dvr router 14:42:38 <yusufgungor_> we have created a new ip table and moved the routes from the routers to the in it (from main table to that table) then make an ip rule to use this table for only the exiting traffic from qrouters to the fip netns 14:42:39 <ralonsoh> if that is the case, that could be much easier to test 14:43:35 <yusufgungor_> * sorry, that would be incoming traffic 14:43:37 <haleyb> brb, doorbell 14:44:01 <yusufgungor_> @ralonsoh i hope 14:44:01 <ralonsoh> yusufgungor_, in any case, is this isolation breaking the previous feature? 14:44:11 <ralonsoh> https://review.opendev.org/c/openstack/neutron/+/474007 ^ 14:44:40 <ralonsoh> (I'm not saying this is bad if there is a bug) 14:45:13 <yusufgungor_> @ralonsoh as far as I understand, no 14:46:16 <yusufgungor_> actually that behaviour is not a problem for the most of the neutron users. Keeping the same tenant network traffic on the same host is acceptable. But in our case we have SOX cybersecurity compliance and find a way to disable this behaviour 14:46:52 <yusufgungor_> we can make the change using a new flag like isolate_tenant_networks etc and keep the old behaviour as default 14:47:41 <yusufgungor_> * same tenant network traffic = same address scoped tenant network 14:48:29 <ralonsoh> ok I think you should propose a patch for Neutron but with a good documentation and testing 14:49:26 <yusufgungor_> the probability of this patch coming from the community is almost zero, right? Just asking to be sure 😀 14:49:45 <mlavalle> unlikely 14:49:47 <ralonsoh> I don't think so but you can ask via mail 14:50:04 <ralonsoh> btw, this is not strictly a bug, as you asked in the LP 14:50:46 <yusufgungor_> I asked to forward this to my manager regarding the time spent on development, thanks 14:51:31 <yusufgungor_> yes it is not a bug when you look at from the address scope site, but using the address scopes is not a choice, it is must when using dynamic routing 14:52:28 <haleyb> ok, so it sounds like we need a spec? 14:52:56 <ralonsoh> I think so, at least to describe this new feature 14:53:32 <yusufgungor_> ok but i can't make promises, I need to talk to my manager about making time 14:53:44 <haleyb> sure, so then should we vote? it seems there is a need for this 14:53:49 <mlavalle> understood 14:53:57 <mlavalle> +1 14:54:09 <ralonsoh> +1 14:54:12 <obondarev> +1 14:54:15 <haleyb> +1 14:54:43 <haleyb> yusufgungor_: thanks for bringing it up 14:55:05 <slaweq> +1 14:55:16 <haleyb> since we don't have much time will move on to last rfe 14:55:17 <yusufgungor_> thanks you for your time guys 🙏 14:55:21 <haleyb> #link https://bugs.launchpad.net/neutron/+bug/2092271 14:55:26 <ralonsoh> very quick 14:55:33 <ralonsoh> I'm implenting that in https://review.opendev.org/q/topic:%22bug/2092271%22 14:55:50 <ralonsoh> but when doing the last patches (https://review.opendev.org/c/openstack/neutron/+/947906) 14:56:17 <ralonsoh> There was a report about ovn-octavia-provider 14:56:26 <ralonsoh> that is relying on lrp.gateway_chassis 14:56:31 <ralonsoh> the patch for this project: https://review.opendev.org/c/openstack/ovn-octavia-provider/+/943243 14:57:03 <ralonsoh> my question: can we backport it to 2025.1, send a mail to upgrade to this version and then implement the change in neutron in 2026.1? 14:57:17 <ralonsoh> I'm saying this because if we cannot do this (the backport) 14:57:24 <ralonsoh> we would need to wait for 2027.1 14:57:35 <ralonsoh> at this point, I'll be closer to my retirement! 14:58:11 <ralonsoh> I know I'm blending a bit the SLURP release cadence by asking for this backport (https://review.opendev.org/c/openstack/ovn-octavia-provider/+/943243) 14:58:12 <haleyb> we all will :) 14:58:54 <ralonsoh> if we all agree with this, I'll send the backport, the releases patch for a new release and a mail 14:59:36 <haleyb> ralonsoh: i don't think the backport to 2025.1 would break anything as it does check for ha_chassis_group 14:59:58 <ralonsoh> not at all, the backport won't break anything 15:00:16 <slaweq> if it won't break things I am fine with it 15:00:51 <haleyb> ralonsoh: the other thing i saw you mention is there is no neutron requirement there, does that need to be addressed? 15:01:20 <haleyb> separate issue i suppose 15:01:23 <ralonsoh> hmmmm I thought that was mandatory for any project. ovn-octavia-provider requires neutron 15:01:29 <ralonsoh> yes, separate issue 15:01:39 <ralonsoh> let me check that and I'll raise that topic next week 15:01:46 <ralonsoh> (or open a LP bug) 15:01:52 <haleyb> i only saw neutron-lib in requirements.txt just now 15:02:17 <haleyb> ack 15:02:50 <haleyb> +1 for backport 15:03:11 <mlavalle> +1 15:03:37 <ralonsoh> thank you folks 15:03:42 <haleyb> we can all retire before 2027.1 now :) 15:03:47 <ralonsoh> yeah! 15:03:56 <obondarev> +1 15:03:58 <ralonsoh> (sorry for the extra time) 15:04:07 <haleyb> np 15:04:23 <slaweq> haleyb yeah \o/ 15:04:33 <haleyb> that was the last item, anything else? if not have a nice weekend everyone! 15:04:54 <haleyb> #endmeeting