14:00:46 <lajoskatona> #startmeeting networking 14:00:46 <opendevmeet> Meeting started Tue Dec 14 14:00:46 2021 UTC and is due to finish in 60 minutes. The chair is lajoskatona. Information about MeetBot at http://wiki.debian.org/MeetBot. 14:00:46 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 14:00:46 <opendevmeet> The meeting name has been set to 'networking' 14:00:49 <mlavalle> o/ 14:00:52 <lajoskatona> o/ 14:00:54 <slaweq> hi 14:00:56 <frickler> \o 14:00:56 <amotoki> hi 14:00:57 <isabek> Hi 14:00:57 <njohnston> o/ 14:01:08 <manub> hi 14:01:12 <damiandabrowski[m]> hi! 14:01:55 <obondarev> hi 14:02:05 <lajoskatona> #topic Announcements 14:02:21 <lajoskatona> The usual reminder for Yoga cycle calendar https://releases.openstack.org/yoga/schedule.html 14:02:34 <rubasov> o/ 14:02:58 <bcafarel> o/ 14:02:59 <lajoskatona> Last week the TC organized meeting series was started for operator pain points 14:03:15 <lajoskatona> #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026169.html 14:03:29 <lajoskatona> summary of the meeting from Rico: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026245.html 14:03:53 <lajoskatona> and there is a nice etherpad: https://etherpad.opendev.org/p/pain-point-elimination 14:04:17 <ralonsoh> hi 14:04:52 <lajoskatona> hm, I can send the link to the line from where Neutron is mentioned: https://etherpad.opendev.org/p/pain-point-elimination#L260 14:05:30 <lajoskatona> 4 topics were mentioned for neutron: 14:05:39 <slaweq> I created one LP bug from that etherpad last week 14:05:51 <slaweq> I think that ykarel is working on it now 14:05:55 <lajoskatona> neutron (via python calls) and OVN (via C calls) can have different ideas about what the hostname is, particulary if a deployer (rightly or wrongly) sets their hostname to be an FQDN 14:06:03 <lajoskatona> slaweq: thanks 14:06:05 <amotoki> https://bugs.launchpad.net/neutron/+bug/1953716 this is a bug slaweq filed 14:06:57 <lajoskatona> yes and we have another for "full" solution for network cascade deletion: https://bugs.launchpad.net/neutron/+bug/1870319 14:07:11 <ykarel> o/ 14:08:00 <lajoskatona> The next one from the list is more an opinion: OpenVSwitch Agent excess logging at INFO level 14:08:21 <ralonsoh> this is the opposite to what we did 2 years ago 14:08:27 <slaweq> exactly 14:08:29 <ralonsoh> moving the debug messages to INFO 14:08:31 <lajoskatona> exactly :-) 14:08:36 <slaweq> I personally I think it's better how it's now 14:08:44 <ralonsoh> I have the same opinion 14:08:57 <ralonsoh> (maybe there is something that could improve) 14:09:10 <lajoskatona> perhaps we can do something which stops in the middle :-) 14:09:25 <lajoskatona> half info half debug :-) 14:09:34 <ralonsoh> that's an option, yes 14:09:45 <slaweq> for me personally it's very hard issue because when all works fine, You don't need almost any logs 14:09:58 <slaweq> but if something is going wrong, You basically need to have debug enabled always 14:10:17 <slaweq> so in case of the problem, our current INFO logging isn't even enough for me :) 14:11:20 <lajoskatona> yeah it's more, a feedback that the agent loop is ok 14:11:46 <amotoki> from my operator experience, it would be nice if INFO messages mention what goes good and what goes wrong. Detail is not needed. DEBUG should help us for such caes. 14:12:04 <amotoki> I am not sure the loop message in ovs-agent really helps 14:13:03 <frickler> would it be an option to move some logs to WARNING, so that deployers getting too much log can turn off INFO? 14:13:07 <lajoskatona> amotoki: thanks 14:15:53 <lajoskatona> from operator/deployer perspective that's an extra pain to have different (log)settings for just ovs-agent for example, and the esiest way from their perspective is to change the code :-) 14:16:22 <amotoki> I think we can start from the logging guideline https://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html 14:16:26 <amotoki> interpreation of the log levels might be different per person 14:17:02 <lajoskatona> amotoki: good that you mentioned, the guideline was mentioned also, but not sure if it is up-to-date 14:17:40 <lajoskatona> and another point for OVN: Spoofing of DNS reponses seems very wrong behavior 14:18:35 <frickler> amotoki: from that, a lot of errors should likely only be warnings 14:18:45 <lajoskatona> to tell the truth I don't know details of it, because the meeting finished, and not sure if there was anybody who has insight of this one 14:19:51 * slaweq will be back in 5 minutes 14:20:37 <ralonsoh> lajoskatona, is there a LP bug? 14:20:56 <ralonsoh> for the OVN DNS issue 14:21:22 <amotoki> we can add the labels defined at https://etherpad.opendev.org/p/pain-point-elimination#L10 14:21:30 <lajoskatona> ralonsoh: nothing: https://etherpad.opendev.org/p/pain-point-elimination#L279 14:21:36 <ralonsoh> ok 14:21:41 <amotoki> if we need more info, we can INFO NEEDED label 14:22:33 <lajoskatona> amotoki: I added now 14:22:38 <amotoki> thanks 14:23:31 <lajoskatona> for logging TC will work on it to have fresh logging guidelines, I just saw in the etherpad 14:24:09 <njohnston> I know a bit about OVN DNS spoofing, but I am not an expert. I think dalvarez knows more. 14:24:34 <dalvarez> hi, can i help somehow? o/ 14:25:10 <lajoskatona> njohnston, dalvarez: Hi, there's an "operator pain points" etherpad and related discussion 14:25:36 <lajoskatona> njohnston, dalvarez: and one point is: "OVN: Spoofing of DNS reponses seems very wrong behavior" without more details 14:25:59 <lajoskatona> #link https://etherpad.opendev.org/p/pain-point-elimination#L279 14:26:38 <lajoskatona> if you can think about it please check the etherpad if this issue is relevant at all 14:27:13 <njohnston> So when OVN sees a DNS request originating on a local net it captures it and compares it against it's local cache, answers it if it can and then allows it to proceed out via normal routing if there is no match 14:28:03 <dalvarez> lajoskatona: ack, i agree we need more info. DNS is not really spoofing anything but acting as a DNS server itself, not a full fledged server though so we may be missing some options or maybe it's not fully compliant with the standard... :? 14:28:03 <njohnston> that causes a change in functionality for private networks that don't have egress; in ML2/OVS they could resolve DNS via the local dnsmasq but in OVN it is not possible 14:28:38 <njohnston> ^ s/resolve DNS/resolve external DNS/ 14:28:49 <njohnston> where external DNS includes Designate 14:29:35 <ralonsoh> right, this is a feature gap, OVN still doesn't support external DNS servers 14:30:25 <ralonsoh> njohnston, is there a LP bug? 14:30:44 <lajoskatona> johnston, ralonsoh, dalvarez: thanks for the explanation 14:30:49 <njohnston> I'll have to go back and check, we last talked about it maybe 6-8 months ago 14:31:09 <frickler> IIUC the issue is more that the instance sends a query to e.g. 8.8.8.8, but the query never reaches that server, instead OVN creates a fake response that looks like 8.8.8.8 sent it 14:32:17 <njohnston> ralonsoh: https://bugs.launchpad.net/neutron/+bug/1902950 14:32:25 <ralonsoh> njohnston, thanks! 14:32:36 <lajoskatona> I will add these insights to the etherpad, check if we have a gap written for it, and we can continue the discussion under a LP if there's already one 14:33:08 <opendevreview> Maor Blaustein proposed openstack/neutron-tempest-plugin master: Add test_create_and_update_port_with_dns_name https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/821079 14:33:24 <lajoskatona> That's all from the meeting last week 14:34:03 <lajoskatona> #topic Community Goals 14:34:41 <lajoskatona> it was again something on the TC worked and discussed: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026280.html (look for "Community-wide goal updates") 14:34:52 <lajoskatona> mail from gmann: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026218.html 14:35:11 <lajoskatona> Current Selected (Active) Goals: 14:35:13 <lajoskatona> Migrate from oslo.rootwrap to oslo.privsep 14:35:21 <lajoskatona> Consistent and Secure Default RBAC 14:35:40 <slaweq> regarding Secure RBAC I proposed patch with update of the policies: https://review.opendev.org/c/openstack/neutron/+/821208 14:35:47 <slaweq> I forgot to reply to ralonsoh's comments there 14:36:02 <ralonsoh> regarding to the first one, privsep, I think we are done 14:36:05 <lajoskatona> with privsep migration we are good, I think we even finished stadium project migration 14:36:15 <ralonsoh> we still use rootrwap to spawn the privsep daemons 14:36:22 <ralonsoh> this like some kind of "inception" 14:36:26 <ralonsoh> I would remove this part 14:36:36 <ralonsoh> removing any dependency from rootwrap 14:36:41 <lajoskatona> ralonsoh, slaweq: thanks 14:39:49 <lajoskatona> This was more a headsup (as we are progressing well /finished these) that we have these 2 goals for this cycle again 14:40:28 <lajoskatona> #topic Bugs 14:40:36 <lajoskatona> amotoki was bug deputy last week: http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026335.html 14:41:27 <lajoskatona> As I checked there are a few unassigned bugs: 14:41:37 <lajoskatona> #link https://bugs.launchpad.net/neutron/+bug/1954384 14:42:21 <lajoskatona> this one again a DNS related bug 14:42:32 <ralonsoh> I think we had the same discussion last week 14:42:42 <ralonsoh> this is for designate 14:42:45 <frickler> I can look into that 14:42:57 <amotoki> but it is a case with linux bridge (not OVN) 14:43:03 <ralonsoh> if you don't use it, then this dns-domain name should be the same as the VM name 14:43:34 <ralonsoh> I'll find the LP bug I'm talking about 14:44:53 <lajoskatona> frickler, amotoki, ralonsoh: thanks 14:45:15 <njohnston> I wonder if this behavior change in Nova may affect things: https://specs.openstack.org/openstack/nova-specs/specs/xena/implemented/configurable-instance-hostnames.html 14:45:38 <njohnston> In any evenbt, it was not something I was aware of until recently, so good to boost exposure within the team 14:45:49 <opendevreview> Merged openstack/neutron-tempest-plugin master: Add test_create_port_with_dns_name https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/820456 14:46:50 <lajoskatona> amotoki: do you have other highlights from last week's bugs? 14:47:12 <amotoki> one thing is https://bugs.launchpad.net/neutron/+bug/1953478 14:47:24 <amotoki> it is realted to the timing of vif-plugged event. 14:48:09 <amotoki> we need to clarify when we send the event and what is expected from both nova and neutron perspective. 14:48:48 <amotoki> in addition, two bugs on OVN metadata agent. it would be nice if folks familiar with OVN can check them. 14:49:30 <ralonsoh> I'm checking the issues with the OVN metadata agent 14:49:43 <lajoskatona> https://bugs.launchpad.net/neutron/+bug/1953510 & https://bugs.launchpad.net/neutron/+bug/1953295 I think 14:50:01 <amotoki> lajoskatona: correct. thanks 14:50:05 <ralonsoh> at least the first one (I'll assign it to me) 14:50:23 <damiandabrowski[m]> I would really appreciate a helping hand here: https://bugs.launchpad.net/neutron/+bug/1952907 14:50:28 <damiandabrowski[m]> I have already spent several hours on troubleshooting and proposed a few possible solutions, but that's probably all I can do. 14:51:35 <amotoki> regarding the vif-plugged event bug, I can look it into detail, but more eyes would be appreciated so I let it unassigned. 14:52:29 <lajoskatona> damiandabrowski[m]: I will check your proposals, thanks for working on it 14:52:40 <damiandabrowski[m]> thank You! 14:53:34 <lajoskatona> amotoki: is it written somewhere when we send events like vif-plugged? 14:54:06 <amotoki> lajoskatona: I don't think so.... unfortunately 14:54:22 <ralonsoh> no and this is something I said I was going to write 14:54:32 <ralonsoh> (in the PTG) but I didn't have time yet 14:54:54 <lajoskatona> perhaps we can start with syncing it with nova and writing down, and check each backend 14:55:02 <amotoki> +1 14:55:09 <lajoskatona> ralonsoh: ok, thanks 14:57:01 <frickler> did anyone look at the discussion I had with mdbooth yesterday? tldr: trunk creation/deletion races with unplugging the baseport from a server 14:57:07 <frickler> no bug reported yet, I was waiting to ask mdbooth whether they would like to do that 14:58:04 <lajoskatona> frickler: that's a good idea to have a bug if they have issue 14:58:45 <lajoskatona> Ok, the CI meeting soon will start, so we have to finish now 14:58:49 <frickler> maybe it was known already or expected somehow, that's why I'm asking 14:59:40 <frickler> final question: anyone looking at the OVN/n-d-r issue yet? 15:00:10 <lajoskatona> #endmeeting