14:00:40 <ralonsoh> #startmeeting networking
14:00:40 <opendevmeet> Meeting started Tue Aug 29 14:00:40 2023 UTC and is due to finish in 60 minutes.  The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:40 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:40 <opendevmeet> The meeting name has been set to 'networking'
14:00:43 <ralonsoh> hello all
14:00:45 <obondarev> hi
14:00:50 <slaweq> o/
14:00:52 <lajoskatona> o/
14:00:55 <haleyb> o/
14:01:16 <ykarel> o/
14:01:38 <isabek> o/
14:01:57 <ralonsoh> let's start
14:02:05 <frickler> \o
14:02:11 <ralonsoh> #topic announcements
14:02:28 <ralonsoh> Bobcat release schedule
14:02:31 <ralonsoh> #link https://releases.openstack.org/bobcat/schedule.html
14:02:43 <ralonsoh> this week is quite important
14:03:00 <ralonsoh> we have the b-3 milestone, that will require a releases patch
14:03:06 <ralonsoh> we have the feature freeze
14:03:17 <ralonsoh> that means no new features should be merged at this point
14:03:27 <ralonsoh> we should accept, of course, bug fixes
14:03:30 <mtomaska> o/ ... late
14:03:57 <ralonsoh> this week is also the final release for client libraries
14:04:21 <ralonsoh> that will also require releases patches (I'm waiting for the release team to push them, as they always do)
14:04:30 <slaweq> I would like to ask for exception for https://review.opendev.org/c/openstack/neutron/+/884474
14:04:31 <ralonsoh> but we would need to review them, of course
14:04:42 <slaweq> but we can talk about it later if needed
14:04:52 <ralonsoh> in the on demand section
14:04:55 <ralonsoh> I'll raise this topic
14:04:59 <slaweq> ++
14:05:19 <ralonsoh> and we have the cycle highlights
14:05:35 <ralonsoh> I need to prepare the patch and then I'll ask you to review/comment it
14:05:42 <ralonsoh> (for sure I'll forget something!)
14:05:55 <ralonsoh> and finally, the election nomination
14:06:00 <lajoskatona> +1, thanks for it
14:06:29 <ralonsoh> we still have 1,5 day left to propose yourself as PTL candidate
14:06:39 <ralonsoh> so far we have haleyb candidacy
14:06:47 <ralonsoh> haleyb, thanks!!
14:06:49 <slaweq> and/or TC candidates also :)
14:06:59 <slaweq> if You are interested in such role maybe
14:07:02 <ralonsoh> and TC candidates, of course!
14:07:14 <mlavalle> thanks haleyb!
14:07:32 <lajoskatona> haleyb saved the cycle
14:07:39 <ralonsoh> you have the documentation in https://releases.openstack.org/bobcat/schedule.html#c-election-nominations, if you need it
14:07:39 <haleyb> ack, and none of you go running away :)
14:07:51 <ralonsoh> no this cycle, sorry
14:08:09 <lajoskatona> true, I meant the next one of course
14:08:41 <ralonsoh> Am I missing something in this topic?
14:09:41 <ralonsoh> ok, let's jump to the next topic
14:09:45 <ralonsoh> #topic bugs
14:10:00 <ralonsoh> last week report is from isabek
14:10:07 <ralonsoh> #link https://lists.openstack.org/pipermail/openstack-discuss/2023-August/034877.html
14:10:32 <ralonsoh> we still have some bugs not assigned
14:10:35 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2032817
14:10:39 <ralonsoh> OVN: Distributed FIP packet drops when provider network MTU exceeds tenant network MTU
14:11:05 <ralonsoh> I still need to check that one in a testing environment (but our CI was under development this week)
14:11:26 <ralonsoh> but is that is confirmed, that even with the DF flag disabled
14:11:38 <frickler> I would suggest do drop the "Distributed FIP" from the title
14:11:54 <ralonsoh> the packet is dropped, that maybe means soemthing is wrong in OVN
14:12:09 <frickler> in my testing the drops happen anywhere, just when the packet is larger than the destination network MTU
14:12:11 <ralonsoh> I'll update the title, right
14:12:27 <ralonsoh> so even from a private network to another private network
14:12:30 <ralonsoh> right?
14:12:30 <frickler> yes
14:12:39 <ralonsoh> ok, good to know this
14:12:45 <ralonsoh> I'll update the title and this info too
14:12:55 <slaweq> for me it looks like core OVN bug, not neutron issue really
14:12:56 <ralonsoh> and I hope I will be able to test this ASAP
14:13:18 <ralonsoh> exactly, but if I open a core OVN bug I would need to provide a way to reproduce it
14:13:26 <ralonsoh> I'll take this one for now
14:13:29 <haleyb> and even with ovn_emit_need_to_frag=True
14:13:43 <slaweq> yes, I think we need to prepare reproducer without Neutron involved and report it to OVN
14:13:52 <ralonsoh> yes, even with this flag
14:13:58 <ralonsoh> but there is something important
14:14:08 <ralonsoh> if you change the flag in the config and restart the Neutron API
14:14:17 <ralonsoh> that won't reconfigure the existing routers
14:14:50 <frickler> I tested on a fresh deployment and also saw no effect from that flag
14:15:08 <frickler> but of course maybe I did something wrong, so you should check for yourself
14:15:36 <ralonsoh> I'll try to test it this week ASAP, this is important
14:15:58 <ralonsoh> ok, the next bug is
14:16:02 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2033083
14:16:07 <ralonsoh> [OVN] The router GW is "restoring" the NAT rule that disables SNAT
14:16:19 <ralonsoh> again, I think this is a problem in core OVN handling sftp traffic
14:16:37 <ralonsoh> according to Julia, OVN is dropping this traffic when SNATing
14:16:49 <ralonsoh> (maybe related to MTU???)
14:17:11 <frickler> tftp?
14:17:26 <ralonsoh> sorry, tftp (no sftp)
14:17:30 <frickler> iirc it may need a special nat helper
14:17:46 <ralonsoh> which one?
14:18:08 <frickler> one that handles IP addresses within the protocol, similar to ftp
14:19:05 <frickler> there's an ip_conntrack_tftp module that does this, not sure how OVN does snat
14:19:18 <ralonsoh> but this is eventually udp traffic, I see no reason to handle that differently
14:19:28 <ralonsoh> right...
14:19:40 <frickler> because of IP addresses within the protocol data
14:19:50 <ralonsoh> we handle the traffic using the conntrack kernel modules
14:19:56 <ralonsoh> if that is not present, we won't handle it
14:20:17 <frickler> so maybe check whether that module is loaded
14:20:40 <ralonsoh> hmm, ok, I'll ping core OVN folks today (and open a bug) to handle this issue
14:20:54 <ralonsoh> but doesn't seem related to neutron so far
14:21:07 <ralonsoh> I'll update the LP with the core OVN bug
14:21:37 <ralonsoh> ok, the next one
14:21:41 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2033179
14:21:47 <ralonsoh> Using .local as TLD in dns testing is breaking some scenarios
14:22:01 <ralonsoh> I didn't have time to check it, sorry
14:22:30 <frickler> it's not urgent, just something I noticed during other testing
14:22:33 <ralonsoh> mlavalle, ^ is it possible for you to check this one?
14:22:48 <ralonsoh> I remember that this specific string
14:22:53 <ralonsoh> is treated differently
14:23:11 <frickler> we could change to some other domain, either example.org or something.devstack.org
14:23:15 <mlavalle> yes, I'll check it
14:23:34 <ralonsoh> mlavalle, thanks!
14:23:54 <mlavalle> towards the end of the week
14:23:54 <slaweq> the only different handling of it which I know is that if it's set to the default value https://github.com/openstack/neutron-lib/blob/master/neutron_lib/constants.py#L352 then dns integration is disabled
14:23:59 <mlavalle> if that's ok
14:24:11 <ralonsoh> for sure, should I assign it to you?
14:24:16 <mlavalle> yes please
14:24:19 <ralonsoh> perfect
14:24:41 <ralonsoh> and the last one
14:24:43 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2032765
14:24:47 <ralonsoh> [L3][DVR]disable ARP response for vlan networks on compute node
14:25:01 <ralonsoh> again, I didn't have time for this one
14:25:56 <ralonsoh> qq (if you know the answer): why should we prevent that?
14:26:08 <ralonsoh> if we are in the same broadcast domain...
14:27:30 <ralonsoh> ok, more context in the previous LP bug: https://bugs.launchpad.net/neutron/+bug/1791268
14:27:58 <ralonsoh> because the ARP request is flooding the switch
14:28:21 <haleyb> and the previous bug mentioned getting multiple ARP responses
14:28:43 <ralonsoh> right (that I don't know why this could be happening)
14:29:35 <haleyb> me either. liu usually follows-up with a patch for bugs, maybe we just ask if he has a solution?
14:29:38 <ralonsoh> I mean, why there are multiple ARP responses? are there ports with the same IP in the network?
14:30:10 <ralonsoh> ok, I'll ping him for more information and maybe a possible implementation of the solution, if any
14:30:16 <slaweq> it's about dvr and gateway is set on each compute node
14:30:26 <slaweq> so we have multiple interfaces with same IP address
14:30:30 <slaweq> I guess that this is an issue
14:30:40 <ralonsoh> right, the GW IP
14:31:17 <ralonsoh> ok, with this information I'll ping him to confirm if that is the case and, maybe, a solution
14:31:24 <slaweq> ++
14:31:40 <ralonsoh> and that's I'll I have for today
14:31:42 <ralonsoh> something else?
14:31:45 <frickler> I have two older bugs related to DNS handling in OVN that are still unassigned, https://bugs.launchpad.net/neutron/+bug/2030294 and 2030295
14:32:25 <ralonsoh> but this one seems to be fixed in 23.06, right?
14:32:42 <frickler> no, not sending a response is not a fix IMO
14:33:02 <frickler> it fixed the "garbled response" part, that does not make it a proper response
14:34:04 <haleyb> OVN doesn't support EDNS so should not have been responding imo
14:34:24 <frickler> but if you have a DNS service, you cannot just support the easy bits
14:35:15 <ralonsoh> I agree with that but let's keep talking to OVN folks to continue implementing new features
14:35:32 <frickler> if you say that this is a restriction in OVN that is not going to be fixed, then neutron should support using the old dhcp-agent setup for DNS instead
14:36:17 <slaweq> frickler You can use dhcp-agent with ML2/OVN backend
14:36:23 <slaweq> there shouldn't be problem with that
14:36:40 <frickler> slaweq: but OVN will still do dhcp?
14:36:50 <ralonsoh> I don't think so
14:36:59 <ralonsoh> we have it only for baremetal provisioning
14:37:04 <ralonsoh> not for OVN owned ports
14:37:11 <ykarel> for baremetal we have a option to disable ovn dhcp
14:37:14 <slaweq> ahh, ok
14:37:17 <ralonsoh> (but I would need to double check that)
14:37:19 <ykarel> but non baremetal i doubt we have that option
14:37:25 <ralonsoh> ^ right
14:37:27 <slaweq> yes, for BM we have option to disable dhcp in ovn
14:37:32 <slaweq> ok
14:37:33 <frickler> but would that be feasible for vms?
14:37:46 <ralonsoh> frickler, so for now I think we should document this gap (as you did)
14:37:49 <slaweq> so we would need to add such option for all ports probably
14:37:54 <ralonsoh> and then request it to core OVN
14:38:35 <frickler> yes, please review https://review.opendev.org/c/openstack/neutron/+/892578 for the gap documentation
14:38:39 <ralonsoh> sure
14:39:02 <ralonsoh> let's continue because we have a couple of issues to talk about later
14:39:18 <ralonsoh> so let's jump to the next topic
14:39:20 <ralonsoh> sorry
14:39:29 <ralonsoh> This week elvira is the deputy, next week will be ykarel.
14:39:36 <ykarel> ack
14:39:38 <ralonsoh> ack?
14:39:46 <ralonsoh> I'll ping elvira later
14:39:55 <elvira> ack! ralonsoh
14:40:02 <ralonsoh> #topic community_goals
14:40:10 <ralonsoh> 1) Add support for the service role in neutron API policies
14:40:23 <ralonsoh> slaweq, please, propose your exception here
14:40:25 <slaweq> this one we decided to move to the next cycle
14:40:38 <slaweq> I didn't want to ask for exception for that one
14:40:57 <slaweq> but for the https://review.opendev.org/c/openstack/neutron/+/884474/ which is Default SG rules templates
14:41:08 <ralonsoh> ahh yes, sorry, my bad
14:41:14 <slaweq> no problem :)
14:41:15 <ralonsoh> (I opened the wrong link)
14:41:23 <ralonsoh> any update in https://review.opendev.org/c/openstack/neutron/+/886724?
14:41:30 <ralonsoh> (I made some comments)
14:41:40 <slaweq> nothing
14:41:50 <ralonsoh> ok thanks!
14:41:52 <ralonsoh> next one
14:41:58 <slaweq> thx for comments, I didn't check them yet as I wanted to focus on more urgent things for now
14:42:05 <ralonsoh> no rush
14:42:13 <ralonsoh> 2) Neutron client deprecation
14:42:17 <ralonsoh> lajoskatona, any update?
14:42:33 <lajoskatona> yes, the SDK patches are merged, and waiting for release
14:42:53 <lajoskatona> but as we are close to the client lib freeze that will happen sooner or later :-)
14:43:00 <ralonsoh> right
14:43:22 <ralonsoh> (for everyone here) please check the releases patches during this week
14:43:29 <lajoskatona> if we have the sdk release we can see and review/merge the neutronclient patches for SFC and FWAAS
14:44:16 <lajoskatona> and VPNAAS (I just checked what is hanging out on gerrit)
14:44:39 <lajoskatona> the open patch list: https://review.opendev.org/q/topic:bug/1999774+status:open
14:45:09 <lajoskatona> the Horizon one and one small doc update for SDK are not that interesting
14:45:20 <lajoskatona> That's it for this topic from me
14:45:41 <ralonsoh> thanks a lot! you are doing a fantastic job here!
14:45:57 <ralonsoh> ok, so last topic today
14:46:01 <ralonsoh> #topic on_demand
14:46:07 <ralonsoh> slaweq, please
14:46:12 <slaweq> thx
14:46:21 <opendevreview> Merged openstack/neutron stable/2023.1: [OVN] Skip the port status UP update during a live migration  https://review.opendev.org/c/openstack/neutron/+/892855
14:46:25 <opendevreview> Merged openstack/neutron stable/zed: [OVN] Skip the port status UP update during a live migration  https://review.opendev.org/c/openstack/neutron/+/892883
14:46:29 <slaweq> so, I wanted to ask eventually for exception for https://review.opendev.org/c/openstack/neutron/+/884474/
14:46:57 <slaweq> this is last piece (except tests, release notes and OSC) needed for this default SG rules templates
14:47:03 <slaweq> API implementation is already merged
14:47:18 <slaweq> now, with this last patch I'm still struggling with CI (fullstack job)
14:47:32 <slaweq> but it's not that patch is somehow behaving in wrong or flaky way
14:48:12 <slaweq> it is working fine, the problem is that in fullstack tests we are doing "weird" things in DB for each test and for that reason in some tests default SG rules aren't in db so test is failing
14:48:22 <lajoskatona> are those fullstack failures related?
14:48:27 <slaweq> I think I have solution for it finally, I will try to push it today or tomorrow
14:48:45 <lajoskatona> ahh, ok, sound good if yo know what happens with them :-)
14:48:53 <slaweq> but other than that this patch should be good and I would like to still land it in the 2023.2 release
14:48:54 <frickler> is there documentation for this changeset?
14:49:24 <lajoskatona> frickler: this will be it: https://review.opendev.org/c/openstack/neutron/+/887664/4 I think
14:49:28 <slaweq> frickler there is api ref https://review.opendev.org/c/openstack/neutron-lib/+/884578
14:49:35 <ralonsoh> I have a procedure question: what paper work is needed for a feature exception?
14:49:35 <slaweq> what else documentation do You think is needed?
14:50:07 <frickler> ah, so documentation is already changed, o.k.
14:50:27 <mlavalle> nice that the doc is taken care of
14:50:52 <slaweq> the reason why I would like to still land it now is that it's almost ready, API is there already but without this last patch it will be useless and it's not something what we will be able to backport later to stable release
14:51:12 <slaweq> ralonsoh TBH I don't know. I will need to check
14:51:16 <mlavalle> that makes sense
14:51:16 <ralonsoh> the feature freeze is this week, so I "think" we still can merge feature patches
14:51:18 <slaweq> probably some email to send
14:51:24 <ralonsoh> until b-3 is released
14:51:32 <ralonsoh> ok, I'll check that today
14:51:37 <slaweq> ok, I will do my best to try to get it merged this week
14:51:47 <ralonsoh> but the b-3 patch has not been proposed yet
14:51:56 <ralonsoh> so I think you still have time without the request
14:51:57 <slaweq> if it will not make it, I will check how officially ask for exception and will do so next week
14:52:02 <slaweq> thx
14:52:02 <ralonsoh> of course, tests should pass
14:52:06 <frickler> releasing with documentation for a feature that is not implemented would also sound wrong, so better merge this than revert the other stuff
14:52:20 <frickler> IIUC this is just sending a mail and ralonsoh acking
14:52:23 <slaweq> frickler yes
14:52:32 <ralonsoh> thanks for the confirmation
14:52:51 <mlavalle> yes, the PTL grants the exception
14:52:51 <ralonsoh> ok, please fix the fullstack tests
14:52:56 <ralonsoh> ping me for help if needed
14:53:03 <slaweq> thx
14:53:13 <slaweq> so that's all from my
14:53:22 <ralonsoh> I have one topic I added to the agenda
14:53:24 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2030741/
14:53:28 <ralonsoh> [OVN] Lack of AZs awareness in L3 port scheduler
14:53:40 <mlavalle> slaweq: your destiny lies in the hands of our all powerful Fearless Leader
14:53:52 <ralonsoh> there is a patch proposed: https://review.opendev.org/c/openstack/neutron/+/892604
14:54:06 <ralonsoh> just is a nutshell
14:54:28 <ralonsoh> OVN schedulers *are" already AZ aware, but not distributing the ports among the AZs
14:54:49 <ralonsoh> the L3 scheduler has a specific class that is AZ aware and does it
14:55:15 <ralonsoh> so this patch is adding the same OVN schedulers but with this functionality, distributing the ports
14:55:56 <ralonsoh> my question is: should we create new schedulers or just implement this feature in the existing ones (with a release note of course, because we are changing the current behaviour)
14:56:10 <ralonsoh> because we don't have time and we have a patch in gerrit
14:56:34 <ralonsoh> I think the best place to talk about this is in the patch
14:56:43 <ralonsoh> that was only an introduction
14:56:47 <ralonsoh> so thank you all
14:56:58 <ralonsoh> any other topic??
14:57:03 <mlavalle> ok, that's for the context
14:57:10 <mlavalle> thanks^^^
14:57:15 <ralonsoh> yes, I'll add this context in the patch
14:57:26 <ralonsoh> but I considered important to comment it here
14:57:35 <mlavalle> appreciated
14:58:11 <ralonsoh> so if there are no more topic, I'll leave you 2 minutes before the CI meeting
14:58:14 <ralonsoh> thank you all
14:58:15 <lajoskatona> +1, thanks for bringing it here
14:58:23 <ralonsoh> #endmeeting