| *** elodilles is now known as elodilles_ooo | 05:45 | |
| opendevreview | Lajos Katona proposed openstack/networking-bagpipe master: Remove py39 job and reference in setup.cfg https://review.opendev.org/c/openstack/networking-bagpipe/+/950956 | 07:47 |
|---|---|---|
| opendevreview | Lajos Katona proposed openstack/networking-bgpvpn master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/networking-bgpvpn/+/950958 | 08:38 |
| opendevreview | Lajos Katona proposed openstack/networking-sfc master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/networking-sfc/+/950963 | 08:42 |
| opendevreview | Lajos Katona proposed openstack/neutron-fwaas master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/neutron-fwaas/+/950959 | 08:44 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas master: Remove py39 jobs and reference from setup.cfg https://review.opendev.org/c/openstack/neutron-vpnaas/+/950960 | 08:47 |
| opendevreview | Lajos Katona proposed openstack/neutron-dynamic-routing master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/950961 | 08:49 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron master: Consider logging options when using OVNdbsync https://review.opendev.org/c/openstack/neutron/+/948783 | 09:20 |
| opendevreview | Lajos Katona proposed openstack/neutron-dynamic-routing master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/neutron-dynamic-routing/+/950961 | 10:05 |
| opendevreview | Lajos Katona proposed openstack/neutron-vpnaas master: Remove py39 jobs and reference from setup.cfg https://review.opendev.org/c/openstack/neutron-vpnaas/+/950960 | 10:05 |
| opendevreview | Lajos Katona proposed openstack/neutron-fwaas master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/neutron-fwaas/+/950959 | 10:06 |
| opendevreview | Lajos Katona proposed openstack/networking-sfc master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/networking-sfc/+/950963 | 10:06 |
| opendevreview | Lajos Katona proposed openstack/networking-bgpvpn master: Remove py39 reference from setup.cfg https://review.opendev.org/c/openstack/networking-bgpvpn/+/950958 | 10:06 |
| opendevreview | Dai Dang Van proposed openstack/neutron master: [WIP] Add dns proxy l2 extension https://review.opendev.org/c/openstack/neutron/+/951390 | 10:40 |
| opendevreview | Merged openstack/neutron-tempest-plugin master: Use only "ubuntu-22.04-minimal" advance image https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/951149 | 11:19 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Ensure LB with HM rechecks member status after re-enable https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951274 | 12:30 |
| opendevreview | Elvira García Ruiz proposed openstack/neutron master: Consider logging options when using OVNdbsync https://review.opendev.org/c/openstack/neutron/+/948783 | 12:45 |
| mlavalle | haleyb: are we meeting today? | 13:03 |
| mlavalle | haleyb: ok, it seems we are not meeting today. I have some errands to run. Have a nice weekend | 13:11 |
| lajoskatona1 | mlavalle: :) Have a nice weekend | 13:11 |
| ralonsoh | we have the drivers meeting, but at 1400UTC | 13:14 |
| ralonsoh | mlavalle, ^ | 13:14 |
| ralonsoh | actually we have agenda and is pretty packed | 13:14 |
| opendevreview | Fernando Royo proposed openstack/ovn-octavia-provider master: Ensure LB with HM rechecks member status after re-enable https://review.opendev.org/c/openstack/ovn-octavia-provider/+/951274 | 13:22 |
| haleyb | mlavalle: yes, we are meeting, sorry did not send out agenda | 13:51 |
| haleyb | ralonsoh: had you seen my neutron release patches for 2024.1/2 and 2025.1? I only saw one backport in-flight for your provider network resources filter, if you wanted to wait for that | 13:55 |
| haleyb | eg https://review.opendev.org/c/openstack/releases/+/951191 | 13:55 |
| ralonsoh | haleyb, checking now | 13:57 |
| ralonsoh | haleyb, maybe we can add https://review.opendev.org/q/topic:%22bug/2109354%22 | 13:58 |
| ralonsoh | no, don't worry | 13:58 |
| ralonsoh | this is not an urgent patch (just for U/S testing) | 13:58 |
| ralonsoh | we can keep going | 13:58 |
| haleyb | https://review.opendev.org/c/openstack/neutron/+/950328 was the only one i saw for 2024.1 and others | 13:59 |
| ralonsoh | yes, this not an urgent patch, to be honest | 13:59 |
| ralonsoh | if it doesn't make this release, we can wait for the next one | 14:00 |
| haleyb | ralonsoh: ack, will ping you after meeting about it | 14:00 |
| haleyb | #startmeeting neutron_drivers | 14:00 |
| opendevmeet | Meeting started Fri May 30 14:00:54 2025 UTC and is due to finish in 60 minutes. The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot. | 14:00 |
| opendevmeet | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 14:00 |
| opendevmeet | The meeting name has been set to 'neutron_drivers' | 14:00 |
| fnordahl | o/ | 14:00 |
| haleyb | Ping list: ykarel, mlavalle, mtomaska, slaweq, obondarev, tobias-urdin, lajoskatona, haleyb, ralonsoh | 14:01 |
| ralonsoh | hello! | 14:01 |
| lajoskatona1 | o/ | 14:01 |
| tore | o/ | 14:01 |
| obondarev | o/ | 14:01 |
| mkalcok | o/ | 14:01 |
| haleyb | we can get started, others might show up late, but we have a number of topics | 14:02 |
| haleyb | jlibosva: are you coming? | 14:02 |
| jlibosva | yes :) hi :) | 14:02 |
| haleyb | ah, perfect | 14:02 |
| haleyb | first is this one | 14:02 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2111276 | 14:03 |
| haleyb | [RFE] Integrate OVN BGP capabilities into Neutron OVN driver | 14:03 |
| haleyb | i know you added it, but fnordahl also has an interest | 14:03 |
| slaweq | o/ | 14:03 |
| haleyb | i'll let either or both of you talk about it while i read the latest update | 14:03 |
| jlibosva | right, so basically the idea would be to replace the ovn-bgp-agent - either by implementing the same like right now I have some PoC where the public switches connecting to the provider network connect to some underlaying BGP routers instead of local port. That would require no Neutron API changes | 14:04 |
| jlibosva | or something smarter using existing API like networking-bgpvpn if there is interest and use case for it | 14:05 |
| lajoskatona1 | +1 for reusing existing APIs or extending them | 14:05 |
| lajoskatona1 | (or no new API :P) | 14:06 |
| jlibosva | the first approach would likely be a knob in Neutron conf and then Neutron would manage the BGP resources similarly to what ovn-bgp-agent does today | 14:06 |
| jlibosva | and FIPs, directly connected ports to provider networks and LBs would get automatically advertised their routes | 14:07 |
| tore | if EVPN is in scope (I certainly hope it would be), it would be necessary to be able to associate the provider network with L2VNI and/or L3VNI values | 14:07 |
| jlibosva | EVPN is in mind - but currently the work in core OVN is at very early stages but we would be interested in it too :) | 14:07 |
| fnordahl | tore: I linked the OVN Community A/V meeting notes which has link to spec on the bug | 14:08 |
| jlibosva | but for this particular RFE it is out of the scope but we definitely keep it in mind so once we get the support we can continue with the work | 14:08 |
| tore | fnordahl: I saw, thanks | 14:08 |
| jlibosva | and I assume a neutron-spec with details would be required :) | 14:09 |
| haleyb | fnordahl: i have not read through all the docs you linked yet, but does this match with what you were thinking? | 14:09 |
| fnordahl | +1 on spec and we have roadmap time to contribute to such a spec | 14:10 |
| ralonsoh | a couple of questions | 14:10 |
| lajoskatona1 | is there something than have to happen first in core OVN ? | 14:10 |
| jlibosva | thanks fnordahl for the comments on the LP - just to make it more clear - the BGP protocol itself would be managed by external tool - like FRR | 14:11 |
| jlibosva | lajoskatona1: I have a working PoC - manually configured running with Neutron and core OVN advertising FIPs | 14:11 |
| tore | I am not very familiar with the Neutron or OVN codebases, so I don't know how much I could contribude development-wise (would perhaps need a mentor), but I am very happy to provide an operator's perspective on how this feature could be implemented in a sensible and useful manner | 14:11 |
| jlibosva | so, so far it seems it has everything we need :) | 14:11 |
| lajoskatona1 | jlibosva: ack, thanks | 14:11 |
| ralonsoh | jlibosva, you said there is not API change. Do we need DB changes? Any agent (we do, I think)? | 14:11 |
| ralonsoh | and would this be implemented as a service plugin? So anyone can enable/disable it? | 14:12 |
| mkalcok | jlibosva: "yes" to your question about the external tool like FRR | 14:12 |
| jlibosva | ralonsoh: The changes will be mostly creating the underlaying topology and avoiding using localnet ports. Now MySQL DB changes or agents | 14:12 |
| slaweq | and what about migration from existing ovn-bgp-agent - will that require any extra steps or will be as easy as stop old agent and run new one with proper config/extension? | 14:12 |
| jlibosva | ralonsoh: that's a great question about where to put it - it would make sense to separate it from the mech driver | 14:13 |
| ralonsoh | cool | 14:13 |
| jlibosva | slaweq: probably a migration path would be needed if required. the agent configures the host networking so we would need some cleanup | 14:14 |
| fnordahl | There are multiple approaches to this, it could either be a light touch (from Neutron's POV) implementation where we only do the changes needed for it to work with the features provided by OVN, and then have downstream deployment tooling do the lifting of BGP agents and such, or Neutron could take more. I think this needs to be ironed out at spec creation time | 14:14 |
| slaweq | jlibosva will such migration tool be part of this RFE? | 14:15 |
| jlibosva | fnordahl: I think the BGP speaker configuration and such should be done by some deployment tooling | 14:16 |
| fnordahl | +1 | 14:16 |
| jlibosva | slaweq: I don't think it will in my opinion | 14:17 |
| haleyb | are there any other questions? should we vote on this? | 14:18 |
| * haleyb is assuming jlibosva and fnordahl will work together on the spec | 14:18 | |
| jlibosva | I would be happy to | 14:18 |
| fnordahl | I might include other people in my team too , and yes | 14:19 |
| lajoskatona1 | No questions from me, big +1 for spec | 14:19 |
| ralonsoh | +1 (with spec, of course). I would like to have this RFE | 14:19 |
| haleyb | great, thanks | 14:19 |
| tore | +1 | 14:19 |
| haleyb | +1 for me to move forward with spec | 14:19 |
| obondarev | +1 | 14:19 |
| haleyb | slaweq: ? | 14:20 |
| slaweq | ahh sorry | 14:20 |
| slaweq | +1 | 14:20 |
| slaweq | even if migration will be done separately | 14:20 |
| haleyb | great, that's consensus, i'll approve the rfe and looking forward to the spec | 14:21 |
| jlibosva | thanks for the questions and for the interest :) | 14:21 |
| fnordahl | \o/ | 14:21 |
| haleyb | we can move onto next rfe | 14:22 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2111899 | 14:22 |
| haleyb | [RFE] Use stateless NAT rules for FIPs | 14:22 |
| ralonsoh | yeah, this is a feature already implemented | 14:22 |
| ralonsoh | and the reverted | 14:22 |
| ralonsoh | it was reverted because the expected benefit with HW offladed envs didn't happen | 14:22 |
| ralonsoh | because the dnat rule flows where not offloaded | 14:23 |
| ralonsoh | but this feature can "bring joy" (and good performance) to DPDK envs | 14:23 |
| ralonsoh | because we skip the conntrack module (that is always a pain with DPDK) | 14:23 |
| ralonsoh | so I would like to re-propose it but this time configurable | 14:23 |
| ralonsoh | questions? | 14:24 |
| obondarev | all clear | 14:24 |
| haleyb | i see from the original change stateless fips were added to OVN 20.03 so that's good | 14:24 |
| tore | did the hw offloaded environments experience a performance degradation, or was it just same performance as before? | 14:24 |
| ralonsoh | yes, very old core OVN | 14:24 |
| ralonsoh | tore, performance degradation because these flows in particular were not offloaded | 14:25 |
| ralonsoh | (don't ask me for more details in this, I would need to request them) | 14:25 |
| tore | understood, then it makes sense to me why it was reverted | 14:25 |
| ralonsoh | but for DPDK, for sure, this is a performance boost, as long as the FIPs rules do not need conntrack | 14:25 |
| lajoskatona1 | +1 for configurable | 14:26 |
| haleyb | ralonsoh: so is the offloading of dnat particular to the nic involved? i.e. they didn't exist when this originally merged? but do now? | 14:27 |
| slaweq | I agree, we can add it back as configurable option | 14:27 |
| ralonsoh | haleyb, at least, as far a I know, the tests done with this feature were not good | 14:28 |
| tore | makes perfect sense for me to have 1:1 NAT rules be stateless, so +1 from me for being able to activate this | 14:28 |
| ralonsoh | and actually they initially implemented that for HW offloaded envs | 14:28 |
| haleyb | ralonsoh: ack. i think i'm fine with the change and config option, but just wanted to make sure we somehow explain where it's expected to work or not work, maybe the release note would be enough? | 14:30 |
| ralonsoh | for sure I'll add a release note explaining the rationale of this implementation | 14:30 |
| haleyb | since it is clear as mud to me how to get this improvement other than trying it out and doing perf testing | 14:30 |
| ralonsoh | and where it could be benefitial | 14:30 |
| ralonsoh | of course, anyone will be able to enable/disable (restaring the API) this feature and restest again | 14:31 |
| ralonsoh | I'll add a maintenance method to change the existing FIPs to the configured value | 14:31 |
| ralonsoh | so it will be just a matter of changing the parameter and test | 14:31 |
| haleyb | ralonsoh: and since you did mention a performance degradation initially should include that wording so it's not treated as a silver bullet | 14:34 |
| haleyb | anyways, let's vote | 14:34 |
| ralonsoh | sure, I'll add where this is actually a benefit (DPDK) and why | 14:34 |
| haleyb | perfect | 14:35 |
| obondarev | +1 | 14:35 |
| haleyb | +1 from me | 14:35 |
| lajoskatona1 | +1 | 14:35 |
| slaweq | +1 | 14:36 |
| haleyb | ok, i believe that is quorum (4) | 14:37 |
| haleyb | i will approve and add a link to discussion | 14:38 |
| ralonsoh | thanks folks | 14:38 |
| ralonsoh | (please, skip my next point in the agenda) | 14:38 |
| haleyb | ralonsoh: ack, can discuss in a future meeting | 14:38 |
| ralonsoh | sure | 14:39 |
| haleyb | next is a bug created by tore so glad he is here | 14:39 |
| haleyb | #link https://bugs.launchpad.net/neutron/+bug/2111891 | 14:39 |
| haleyb | IPv6 Subnet-Router anycast address inappropriately used for gateway_ip when specifying --subnet-range | 14:39 |
| tore | thanks for the invite, haleyb :) | 14:40 |
| haleyb | slaweq mentioned making this change is a change in the API behaviour | 14:40 |
| tore | so to my mind it makes no sense that the behaviour differs from when using --subnet-range or not, that seems very inconsistent, and just that alone is imho a bug | 14:41 |
| tore | for what it is worth, for ipv4 "address one" is always used, both with or without --subnet-range being specified | 14:42 |
| haleyb | tore: so my IPv6 knowledge in this area is a little rusty, but if that address is configured in the neutron router it cannot be used? not sure if i read that correctly | 14:42 |
| haleyb | i thought all the routers could use the anycast address | 14:43 |
| tore | haleyb: it cannot be used by VMs that are routers themselves, i.e., has enabled IPv6 forwarding. which I believe dockerd and such does automatically | 14:43 |
| haleyb | but i do agree it's odd this one case if different from the others | 14:43 |
| haleyb | ah, ok, vm deployed as a router | 14:43 |
| tore | haleyb: if the VM enables forwarding sysctl (is a router), it is required by the spec to consider the subnet-router "address zero" as a *local* address | 14:43 |
| tore | meaning it is unusable as an ipv6 default gateway, since it is a next-hop pointing to itself essentially | 14:44 |
| slaweq | for this this proposal sounds reasonable, I just wanted to have it discussed here as it will change API which we have now | 14:44 |
| haleyb | tore: understood, thanks for the explanation | 14:44 |
| tore | note it will not be a problem if using ipv6_ra_mode=something. because then the default gateway is a fe80::x link-local address learned from RA, and the gateway_ip address isn't used for anything | 14:45 |
| tore | but if ipv6_ra_mode is left at default "none" (another bad default imho, but that's a different issue), the VM would need to get the default route from metadata service or manual config, and in that case gateway_ip will be used as next-hop | 14:46 |
| tore | …and break the moment VM enables forwarding sysctl | 14:47 |
| haleyb | slaweq: i can understand the API change, but it seems like as it's implemented it creates a broken object | 14:48 |
| haleyb | i'm curious what the API says about the address used, if anything | 14:48 |
| haleyb | "If the gateway_ip is not specified, OpenStack Networking allocates an address from the CIDR for the gateway for the subnet by default" | 14:49 |
| ralonsoh | Ok, it took me a while to find it | 14:50 |
| ralonsoh | https://review.opendev.org/q/I3f2905c2c4fca02406dfa3c801c166c14389ba41 | 14:50 |
| haleyb | so we don't actually say, although we do typically pick .1/::1 | 14:50 |
| ralonsoh | where in the RFC is this issue commented? | 14:51 |
| tore | there is one exception to this problem, which is /127 p2p subnets where the subnet-router anycast address is disabled, cf. https://datatracker.ietf.org/doc/html/rfc6164. so for /127s "address zero" would be the appropriate choice | 14:51 |
| haleyb | ralonsoh: thanks for finding that, i even +2'd it | 14:51 |
| tore | (same thing for ipv4 /31s I guess) | 14:51 |
| haleyb | ralonsoh: it just says you can use it - "This anycast address is syntactically | 14:53 |
| haleyb | the same as a unicast address for an interface on the link with the | 14:53 |
| haleyb | interface identifier set to zero." | 14:53 |
| haleyb | it doesn't say you *should* use it, and maybe in practice it's not a good idea | 14:53 |
| tore | https://blog.apnic.net/2023/08/28/behavioural-differences-of-ipv6-subnet-router-anycast-address-implementations/ Ctrl-F strange behaviour | 14:54 |
| haleyb | but yes, we clearly single out IPv6 | 14:54 |
| haleyb | https://review.opendev.org/c/openstack/neutron/+/647484/10/neutron/db/ipam_backend_mixin.py | 14:54 |
| haleyb | tore: and obviously, you can specify the ::1 address and it all works | 14:55 |
| tore | yep, so this is about default behaviour (and the inconsistency between using --subnet-range and not) | 14:56 |
| tore | it also seems conceivable that on a multi-node network where >0 VMs have forwarding enabled, this could be problematic for other VMs even with forwarding disabled, because the router VM(s) might answer Neighbour Solicitations for that address, effectively hijacking traffic | 14:58 |
| tore | haven't tested this though. might be portsec will prevent it | 14:59 |
| haleyb | i don't know the best thing to do here, seems you've found a corner case, but it does change the API. I would be inclined to make it consistent saying you can still choose :: manually, but have reservations | 15:00 |
| obondarev | (sorry, have to logoff) | 15:01 |
| ralonsoh | I think we can document it | 15:01 |
| haleyb | i have a hard stop as well, any other opinions? | 15:01 |
| tore | agreed, would be fine to explicitly request the ::0 address, but it's not a good default imho, regardless of --subnet-range being used | 15:01 |
| ralonsoh | the user can always defined the gw-ip when creating the subnet | 15:01 |
| ralonsoh | at least let's document it | 15:02 |
| haleyb | ralonsoh: change and document? or just document it could be an issue? | 15:02 |
| ralonsoh | document that this could be an issue | 15:02 |
| ralonsoh | I think we have a section for ipv6 | 15:02 |
| ralonsoh | (somewhere...) | 15:02 |
| lajoskatona1 | https://docs.openstack.org/neutron/latest/admin/config-ipv6.html | 15:03 |
| lajoskatona1 | I beleive this one | 15:03 |
| ralonsoh | correct | 15:03 |
| haleyb | so i don't think we'll have consensus on changing to use ::1, so documenting would be next best thing | 15:05 |
| tore | I'm most concerned about the user behaviour here. users clicking through the web ui or using "openstack subnet create" shouldn't be handed a loaded gun pointing at themselves, imho | 15:05 |
| haleyb | i'm not a fan of creating a broken config by default though | 15:05 |
| tore | there's also the issue of the unexplained inconsistency between not using --subnet-range and using it. if you want to clean up that inconsistency, you'll need to change something anyway... | 15:06 |
| slaweq | IMHO we can fix it in master, maybe not backporting to stable branches though | 15:06 |
| haleyb | we can maybe continue discussion in the bug or next week, but i have to go to kids school presentation | 15:06 |
| lajoskatona1 | good idea | 15:06 |
| slaweq | and highlight in the release notes that default behaviour has changed since this release | 15:06 |
| haleyb | I would be fine with slaweq's suggestion | 15:07 |
| tore | +1 to slaweq | 15:07 |
| haleyb | i really have to go so will end meeting | 15:07 |
| haleyb | #endmeeting | 15:07 |
| opendevmeet | Meeting ended Fri May 30 15:07:40 2025 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 15:07 |
| opendevmeet | Minutes: https://meetings.opendev.org/meetings/neutron_drivers/2025/neutron_drivers.2025-05-30-14.00.html | 15:07 |
| opendevmeet | Minutes (text): https://meetings.opendev.org/meetings/neutron_drivers/2025/neutron_drivers.2025-05-30-14.00.txt | 15:07 |
| opendevmeet | Log: https://meetings.opendev.org/meetings/neutron_drivers/2025/neutron_drivers.2025-05-30-14.00.log.html | 15:07 |
| slaweq | ok, I have to go too, have a great weekend | 15:07 |
| slaweq | o/ | 15:07 |
| lajoskatona1 | o/ | 15:07 |
| haleyb | can discuss more in bug, etc | 15:07 |
| tore | have an nice weekend o/ | 15:07 |
| jlibosva | o/ | 15:08 |
| ralonsoh | bye | 15:09 |
| *** haleyb is now known as haleyb|out | 16:59 | |
Generated by irclog2html.py 4.0.0 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!