14:00:40 <ralonsoh> #startmeeting networking 14:00:40 <opendevmeet> Meeting started Tue Aug 29 14:00:40 2023 UTC and is due to finish in 60 minutes. The chair is ralonsoh. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:40 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:40 <opendevmeet> The meeting name has been set to 'networking' 14:00:43 <ralonsoh> hello all 14:00:45 <obondarev> hi 14:00:50 <slaweq> o/ 14:00:52 <lajoskatona> o/ 14:00:55 <haleyb> o/ 14:01:16 <ykarel> o/ 14:01:38 <isabek> o/ 14:01:57 <ralonsoh> let's start 14:02:05 <frickler> \o 14:02:11 <ralonsoh> #topic announcements 14:02:28 <ralonsoh> Bobcat release schedule 14:02:31 <ralonsoh> #link https://releases.openstack.org/bobcat/schedule.html 14:02:43 <ralonsoh> this week is quite important 14:03:00 <ralonsoh> we have the b-3 milestone, that will require a releases patch 14:03:06 <ralonsoh> we have the feature freeze 14:03:17 <ralonsoh> that means no new features should be merged at this point 14:03:27 <ralonsoh> we should accept, of course, bug fixes 14:03:30 <mtomaska> o/ ... late 14:03:57 <ralonsoh> this week is also the final release for client libraries 14:04:21 <ralonsoh> that will also require releases patches (I'm waiting for the release team to push them, as they always do) 14:04:30 <slaweq> I would like to ask for exception for https://review.opendev.org/c/openstack/neutron/+/884474 14:04:31 <ralonsoh> but we would need to review them, of course 14:04:42 <slaweq> but we can talk about it later if needed 14:04:52 <ralonsoh> in the on demand section 14:04:55 <ralonsoh> I'll raise this topic 14:04:59 <slaweq> ++ 14:05:19 <ralonsoh> and we have the cycle highlights 14:05:35 <ralonsoh> I need to prepare the patch and then I'll ask you to review/comment it 14:05:42 <ralonsoh> (for sure I'll forget something!) 14:05:55 <ralonsoh> and finally, the election nomination 14:06:00 <lajoskatona> +1, thanks for it 14:06:29 <ralonsoh> we still have 1,5 day left to propose yourself as PTL candidate 14:06:39 <ralonsoh> so far we have haleyb candidacy 14:06:47 <ralonsoh> haleyb, thanks!! 14:06:49 <slaweq> and/or TC candidates also :) 14:06:59 <slaweq> if You are interested in such role maybe 14:07:02 <ralonsoh> and TC candidates, of course! 14:07:14 <mlavalle> thanks haleyb! 14:07:32 <lajoskatona> haleyb saved the cycle 14:07:39 <ralonsoh> you have the documentation in https://releases.openstack.org/bobcat/schedule.html#c-election-nominations, if you need it 14:07:39 <haleyb> ack, and none of you go running away :) 14:07:51 <ralonsoh> no this cycle, sorry 14:08:09 <lajoskatona> true, I meant the next one of course 14:08:41 <ralonsoh> Am I missing something in this topic? 14:09:41 <ralonsoh> ok, let's jump to the next topic 14:09:45 <ralonsoh> #topic bugs 14:10:00 <ralonsoh> last week report is from isabek 14:10:07 <ralonsoh> #link https://lists.openstack.org/pipermail/openstack-discuss/2023-August/034877.html 14:10:32 <ralonsoh> we still have some bugs not assigned 14:10:35 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2032817 14:10:39 <ralonsoh> OVN: Distributed FIP packet drops when provider network MTU exceeds tenant network MTU 14:11:05 <ralonsoh> I still need to check that one in a testing environment (but our CI was under development this week) 14:11:26 <ralonsoh> but is that is confirmed, that even with the DF flag disabled 14:11:38 <frickler> I would suggest do drop the "Distributed FIP" from the title 14:11:54 <ralonsoh> the packet is dropped, that maybe means soemthing is wrong in OVN 14:12:09 <frickler> in my testing the drops happen anywhere, just when the packet is larger than the destination network MTU 14:12:11 <ralonsoh> I'll update the title, right 14:12:27 <ralonsoh> so even from a private network to another private network 14:12:30 <ralonsoh> right? 14:12:30 <frickler> yes 14:12:39 <ralonsoh> ok, good to know this 14:12:45 <ralonsoh> I'll update the title and this info too 14:12:55 <slaweq> for me it looks like core OVN bug, not neutron issue really 14:12:56 <ralonsoh> and I hope I will be able to test this ASAP 14:13:18 <ralonsoh> exactly, but if I open a core OVN bug I would need to provide a way to reproduce it 14:13:26 <ralonsoh> I'll take this one for now 14:13:29 <haleyb> and even with ovn_emit_need_to_frag=True 14:13:43 <slaweq> yes, I think we need to prepare reproducer without Neutron involved and report it to OVN 14:13:52 <ralonsoh> yes, even with this flag 14:13:58 <ralonsoh> but there is something important 14:14:08 <ralonsoh> if you change the flag in the config and restart the Neutron API 14:14:17 <ralonsoh> that won't reconfigure the existing routers 14:14:50 <frickler> I tested on a fresh deployment and also saw no effect from that flag 14:15:08 <frickler> but of course maybe I did something wrong, so you should check for yourself 14:15:36 <ralonsoh> I'll try to test it this week ASAP, this is important 14:15:58 <ralonsoh> ok, the next bug is 14:16:02 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2033083 14:16:07 <ralonsoh> [OVN] The router GW is "restoring" the NAT rule that disables SNAT 14:16:19 <ralonsoh> again, I think this is a problem in core OVN handling sftp traffic 14:16:37 <ralonsoh> according to Julia, OVN is dropping this traffic when SNATing 14:16:49 <ralonsoh> (maybe related to MTU???) 14:17:11 <frickler> tftp? 14:17:26 <ralonsoh> sorry, tftp (no sftp) 14:17:30 <frickler> iirc it may need a special nat helper 14:17:46 <ralonsoh> which one? 14:18:08 <frickler> one that handles IP addresses within the protocol, similar to ftp 14:19:05 <frickler> there's an ip_conntrack_tftp module that does this, not sure how OVN does snat 14:19:18 <ralonsoh> but this is eventually udp traffic, I see no reason to handle that differently 14:19:28 <ralonsoh> right... 14:19:40 <frickler> because of IP addresses within the protocol data 14:19:50 <ralonsoh> we handle the traffic using the conntrack kernel modules 14:19:56 <ralonsoh> if that is not present, we won't handle it 14:20:17 <frickler> so maybe check whether that module is loaded 14:20:40 <ralonsoh> hmm, ok, I'll ping core OVN folks today (and open a bug) to handle this issue 14:20:54 <ralonsoh> but doesn't seem related to neutron so far 14:21:07 <ralonsoh> I'll update the LP with the core OVN bug 14:21:37 <ralonsoh> ok, the next one 14:21:41 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2033179 14:21:47 <ralonsoh> Using .local as TLD in dns testing is breaking some scenarios 14:22:01 <ralonsoh> I didn't have time to check it, sorry 14:22:30 <frickler> it's not urgent, just something I noticed during other testing 14:22:33 <ralonsoh> mlavalle, ^ is it possible for you to check this one? 14:22:48 <ralonsoh> I remember that this specific string 14:22:53 <ralonsoh> is treated differently 14:23:11 <frickler> we could change to some other domain, either example.org or something.devstack.org 14:23:15 <mlavalle> yes, I'll check it 14:23:34 <ralonsoh> mlavalle, thanks! 14:23:54 <mlavalle> towards the end of the week 14:23:54 <slaweq> the only different handling of it which I know is that if it's set to the default value https://github.com/openstack/neutron-lib/blob/master/neutron_lib/constants.py#L352 then dns integration is disabled 14:23:59 <mlavalle> if that's ok 14:24:11 <ralonsoh> for sure, should I assign it to you? 14:24:16 <mlavalle> yes please 14:24:19 <ralonsoh> perfect 14:24:41 <ralonsoh> and the last one 14:24:43 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2032765 14:24:47 <ralonsoh> [L3][DVR]disable ARP response for vlan networks on compute node 14:25:01 <ralonsoh> again, I didn't have time for this one 14:25:56 <ralonsoh> qq (if you know the answer): why should we prevent that? 14:26:08 <ralonsoh> if we are in the same broadcast domain... 14:27:30 <ralonsoh> ok, more context in the previous LP bug: https://bugs.launchpad.net/neutron/+bug/1791268 14:27:58 <ralonsoh> because the ARP request is flooding the switch 14:28:21 <haleyb> and the previous bug mentioned getting multiple ARP responses 14:28:43 <ralonsoh> right (that I don't know why this could be happening) 14:29:35 <haleyb> me either. liu usually follows-up with a patch for bugs, maybe we just ask if he has a solution? 14:29:38 <ralonsoh> I mean, why there are multiple ARP responses? are there ports with the same IP in the network? 14:30:10 <ralonsoh> ok, I'll ping him for more information and maybe a possible implementation of the solution, if any 14:30:16 <slaweq> it's about dvr and gateway is set on each compute node 14:30:26 <slaweq> so we have multiple interfaces with same IP address 14:30:30 <slaweq> I guess that this is an issue 14:30:40 <ralonsoh> right, the GW IP 14:31:17 <ralonsoh> ok, with this information I'll ping him to confirm if that is the case and, maybe, a solution 14:31:24 <slaweq> ++ 14:31:40 <ralonsoh> and that's I'll I have for today 14:31:42 <ralonsoh> something else? 14:31:45 <frickler> I have two older bugs related to DNS handling in OVN that are still unassigned, https://bugs.launchpad.net/neutron/+bug/2030294 and 2030295 14:32:25 <ralonsoh> but this one seems to be fixed in 23.06, right? 14:32:42 <frickler> no, not sending a response is not a fix IMO 14:33:02 <frickler> it fixed the "garbled response" part, that does not make it a proper response 14:34:04 <haleyb> OVN doesn't support EDNS so should not have been responding imo 14:34:24 <frickler> but if you have a DNS service, you cannot just support the easy bits 14:35:15 <ralonsoh> I agree with that but let's keep talking to OVN folks to continue implementing new features 14:35:32 <frickler> if you say that this is a restriction in OVN that is not going to be fixed, then neutron should support using the old dhcp-agent setup for DNS instead 14:36:17 <slaweq> frickler You can use dhcp-agent with ML2/OVN backend 14:36:23 <slaweq> there shouldn't be problem with that 14:36:40 <frickler> slaweq: but OVN will still do dhcp? 14:36:50 <ralonsoh> I don't think so 14:36:59 <ralonsoh> we have it only for baremetal provisioning 14:37:04 <ralonsoh> not for OVN owned ports 14:37:11 <ykarel> for baremetal we have a option to disable ovn dhcp 14:37:14 <slaweq> ahh, ok 14:37:17 <ralonsoh> (but I would need to double check that) 14:37:19 <ykarel> but non baremetal i doubt we have that option 14:37:25 <ralonsoh> ^ right 14:37:27 <slaweq> yes, for BM we have option to disable dhcp in ovn 14:37:32 <slaweq> ok 14:37:33 <frickler> but would that be feasible for vms? 14:37:46 <ralonsoh> frickler, so for now I think we should document this gap (as you did) 14:37:49 <slaweq> so we would need to add such option for all ports probably 14:37:54 <ralonsoh> and then request it to core OVN 14:38:35 <frickler> yes, please review https://review.opendev.org/c/openstack/neutron/+/892578 for the gap documentation 14:38:39 <ralonsoh> sure 14:39:02 <ralonsoh> let's continue because we have a couple of issues to talk about later 14:39:18 <ralonsoh> so let's jump to the next topic 14:39:20 <ralonsoh> sorry 14:39:29 <ralonsoh> This week elvira is the deputy, next week will be ykarel. 14:39:36 <ykarel> ack 14:39:38 <ralonsoh> ack? 14:39:46 <ralonsoh> I'll ping elvira later 14:39:55 <elvira> ack! ralonsoh 14:40:02 <ralonsoh> #topic community_goals 14:40:10 <ralonsoh> 1) Add support for the service role in neutron API policies 14:40:23 <ralonsoh> slaweq, please, propose your exception here 14:40:25 <slaweq> this one we decided to move to the next cycle 14:40:38 <slaweq> I didn't want to ask for exception for that one 14:40:57 <slaweq> but for the https://review.opendev.org/c/openstack/neutron/+/884474/ which is Default SG rules templates 14:41:08 <ralonsoh> ahh yes, sorry, my bad 14:41:14 <slaweq> no problem :) 14:41:15 <ralonsoh> (I opened the wrong link) 14:41:23 <ralonsoh> any update in https://review.opendev.org/c/openstack/neutron/+/886724? 14:41:30 <ralonsoh> (I made some comments) 14:41:40 <slaweq> nothing 14:41:50 <ralonsoh> ok thanks! 14:41:52 <ralonsoh> next one 14:41:58 <slaweq> thx for comments, I didn't check them yet as I wanted to focus on more urgent things for now 14:42:05 <ralonsoh> no rush 14:42:13 <ralonsoh> 2) Neutron client deprecation 14:42:17 <ralonsoh> lajoskatona, any update? 14:42:33 <lajoskatona> yes, the SDK patches are merged, and waiting for release 14:42:53 <lajoskatona> but as we are close to the client lib freeze that will happen sooner or later :-) 14:43:00 <ralonsoh> right 14:43:22 <ralonsoh> (for everyone here) please check the releases patches during this week 14:43:29 <lajoskatona> if we have the sdk release we can see and review/merge the neutronclient patches for SFC and FWAAS 14:44:16 <lajoskatona> and VPNAAS (I just checked what is hanging out on gerrit) 14:44:39 <lajoskatona> the open patch list: https://review.opendev.org/q/topic:bug/1999774+status:open 14:45:09 <lajoskatona> the Horizon one and one small doc update for SDK are not that interesting 14:45:20 <lajoskatona> That's it for this topic from me 14:45:41 <ralonsoh> thanks a lot! you are doing a fantastic job here! 14:45:57 <ralonsoh> ok, so last topic today 14:46:01 <ralonsoh> #topic on_demand 14:46:07 <ralonsoh> slaweq, please 14:46:12 <slaweq> thx 14:46:21 <opendevreview> Merged openstack/neutron stable/2023.1: [OVN] Skip the port status UP update during a live migration https://review.opendev.org/c/openstack/neutron/+/892855 14:46:25 <opendevreview> Merged openstack/neutron stable/zed: [OVN] Skip the port status UP update during a live migration https://review.opendev.org/c/openstack/neutron/+/892883 14:46:29 <slaweq> so, I wanted to ask eventually for exception for https://review.opendev.org/c/openstack/neutron/+/884474/ 14:46:57 <slaweq> this is last piece (except tests, release notes and OSC) needed for this default SG rules templates 14:47:03 <slaweq> API implementation is already merged 14:47:18 <slaweq> now, with this last patch I'm still struggling with CI (fullstack job) 14:47:32 <slaweq> but it's not that patch is somehow behaving in wrong or flaky way 14:48:12 <slaweq> it is working fine, the problem is that in fullstack tests we are doing "weird" things in DB for each test and for that reason in some tests default SG rules aren't in db so test is failing 14:48:22 <lajoskatona> are those fullstack failures related? 14:48:27 <slaweq> I think I have solution for it finally, I will try to push it today or tomorrow 14:48:45 <lajoskatona> ahh, ok, sound good if yo know what happens with them :-) 14:48:53 <slaweq> but other than that this patch should be good and I would like to still land it in the 2023.2 release 14:48:54 <frickler> is there documentation for this changeset? 14:49:24 <lajoskatona> frickler: this will be it: https://review.opendev.org/c/openstack/neutron/+/887664/4 I think 14:49:28 <slaweq> frickler there is api ref https://review.opendev.org/c/openstack/neutron-lib/+/884578 14:49:35 <ralonsoh> I have a procedure question: what paper work is needed for a feature exception? 14:49:35 <slaweq> what else documentation do You think is needed? 14:50:07 <frickler> ah, so documentation is already changed, o.k. 14:50:27 <mlavalle> nice that the doc is taken care of 14:50:52 <slaweq> the reason why I would like to still land it now is that it's almost ready, API is there already but without this last patch it will be useless and it's not something what we will be able to backport later to stable release 14:51:12 <slaweq> ralonsoh TBH I don't know. I will need to check 14:51:16 <mlavalle> that makes sense 14:51:16 <ralonsoh> the feature freeze is this week, so I "think" we still can merge feature patches 14:51:18 <slaweq> probably some email to send 14:51:24 <ralonsoh> until b-3 is released 14:51:32 <ralonsoh> ok, I'll check that today 14:51:37 <slaweq> ok, I will do my best to try to get it merged this week 14:51:47 <ralonsoh> but the b-3 patch has not been proposed yet 14:51:56 <ralonsoh> so I think you still have time without the request 14:51:57 <slaweq> if it will not make it, I will check how officially ask for exception and will do so next week 14:52:02 <slaweq> thx 14:52:02 <ralonsoh> of course, tests should pass 14:52:06 <frickler> releasing with documentation for a feature that is not implemented would also sound wrong, so better merge this than revert the other stuff 14:52:20 <frickler> IIUC this is just sending a mail and ralonsoh acking 14:52:23 <slaweq> frickler yes 14:52:32 <ralonsoh> thanks for the confirmation 14:52:51 <mlavalle> yes, the PTL grants the exception 14:52:51 <ralonsoh> ok, please fix the fullstack tests 14:52:56 <ralonsoh> ping me for help if needed 14:53:03 <slaweq> thx 14:53:13 <slaweq> so that's all from my 14:53:22 <ralonsoh> I have one topic I added to the agenda 14:53:24 <ralonsoh> #link https://bugs.launchpad.net/neutron/+bug/2030741/ 14:53:28 <ralonsoh> [OVN] Lack of AZs awareness in L3 port scheduler 14:53:40 <mlavalle> slaweq: your destiny lies in the hands of our all powerful Fearless Leader 14:53:52 <ralonsoh> there is a patch proposed: https://review.opendev.org/c/openstack/neutron/+/892604 14:54:06 <ralonsoh> just is a nutshell 14:54:28 <ralonsoh> OVN schedulers *are" already AZ aware, but not distributing the ports among the AZs 14:54:49 <ralonsoh> the L3 scheduler has a specific class that is AZ aware and does it 14:55:15 <ralonsoh> so this patch is adding the same OVN schedulers but with this functionality, distributing the ports 14:55:56 <ralonsoh> my question is: should we create new schedulers or just implement this feature in the existing ones (with a release note of course, because we are changing the current behaviour) 14:56:10 <ralonsoh> because we don't have time and we have a patch in gerrit 14:56:34 <ralonsoh> I think the best place to talk about this is in the patch 14:56:43 <ralonsoh> that was only an introduction 14:56:47 <ralonsoh> so thank you all 14:56:58 <ralonsoh> any other topic?? 14:57:03 <mlavalle> ok, that's for the context 14:57:10 <mlavalle> thanks^^^ 14:57:15 <ralonsoh> yes, I'll add this context in the patch 14:57:26 <ralonsoh> but I considered important to comment it here 14:57:35 <mlavalle> appreciated 14:58:11 <ralonsoh> so if there are no more topic, I'll leave you 2 minutes before the CI meeting 14:58:14 <ralonsoh> thank you all 14:58:15 <lajoskatona> +1, thanks for bringing it here 14:58:23 <ralonsoh> #endmeeting