14:00:46 <lajoskatona> #startmeeting networking
14:00:46 <opendevmeet> Meeting started Tue Dec 14 14:00:46 2021 UTC and is due to finish in 60 minutes.  The chair is lajoskatona. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:46 <opendevmeet> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:46 <opendevmeet> The meeting name has been set to 'networking'
14:00:49 <mlavalle> o/
14:00:52 <lajoskatona> o/
14:00:54 <slaweq> hi
14:00:56 <frickler> \o
14:00:56 <amotoki> hi
14:00:57 <isabek> Hi
14:00:57 <njohnston> o/
14:01:08 <manub> hi
14:01:12 <damiandabrowski[m]> hi!
14:01:55 <obondarev> hi
14:02:05 <lajoskatona> #topic Announcements
14:02:21 <lajoskatona> The usual reminder for Yoga cycle calendar https://releases.openstack.org/yoga/schedule.html
14:02:34 <rubasov> o/
14:02:58 <bcafarel> o/
14:02:59 <lajoskatona> Last week the TC organized meeting series was started for operator pain points
14:03:15 <lajoskatona> #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026169.html
14:03:29 <lajoskatona> summary of the meeting from Rico: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026245.html
14:03:53 <lajoskatona> and there is a nice etherpad: https://etherpad.opendev.org/p/pain-point-elimination
14:04:17 <ralonsoh> hi
14:04:52 <lajoskatona> hm, I can send the link to the line from where Neutron is mentioned: https://etherpad.opendev.org/p/pain-point-elimination#L260
14:05:30 <lajoskatona> 4 topics were mentioned for neutron:
14:05:39 <slaweq> I created one LP bug from that etherpad last week
14:05:51 <slaweq> I think that ykarel is working on it now
14:05:55 <lajoskatona> neutron (via python calls) and OVN (via C calls) can have different ideas about what the hostname is, particulary if a deployer (rightly or wrongly) sets their hostname to be an FQDN
14:06:03 <lajoskatona> slaweq: thanks
14:06:05 <amotoki> https://bugs.launchpad.net/neutron/+bug/1953716 this is a bug slaweq filed
14:06:57 <lajoskatona> yes and we have another for "full" solution for network cascade deletion: https://bugs.launchpad.net/neutron/+bug/1870319
14:07:11 <ykarel> o/
14:08:00 <lajoskatona> The next one from the list is more an opinion: OpenVSwitch Agent excess logging at INFO level
14:08:21 <ralonsoh> this is the opposite to what we did 2 years ago
14:08:27 <slaweq> exactly
14:08:29 <ralonsoh> moving the debug messages to INFO
14:08:31 <lajoskatona> exactly :-)
14:08:36 <slaweq> I personally I think it's better how it's now
14:08:44 <ralonsoh> I have the same opinion
14:08:57 <ralonsoh> (maybe there is something that could improve)
14:09:10 <lajoskatona> perhaps we can do something which stops in the middle :-)
14:09:25 <lajoskatona> half info half debug :-)
14:09:34 <ralonsoh> that's an option, yes
14:09:45 <slaweq> for me personally it's very hard issue because when all works fine, You don't need almost any logs
14:09:58 <slaweq> but if something is going wrong, You basically need to have debug enabled always
14:10:17 <slaweq> so in case of the problem, our current INFO logging isn't even enough for me :)
14:11:20 <lajoskatona> yeah it's more, a feedback that the agent loop is ok
14:11:46 <amotoki> from my operator experience, it would be nice if INFO messages mention what goes good and what goes wrong. Detail is not needed. DEBUG should help us for such caes.
14:12:04 <amotoki> I am not sure the loop message in ovs-agent really helps
14:13:03 <frickler> would it be an option to move some logs to WARNING, so that deployers getting too much log can turn off INFO?
14:13:07 <lajoskatona> amotoki: thanks
14:15:53 <lajoskatona> from operator/deployer perspective that's an extra pain to have different (log)settings for just ovs-agent for example, and the esiest way from their perspective is to change the code :-)
14:16:22 <amotoki> I think we can start from the logging guideline https://specs.openstack.org/openstack/openstack-specs/specs/log-guidelines.html
14:16:26 <amotoki> interpreation of the log levels might be different per person
14:17:02 <lajoskatona> amotoki: good that you mentioned, the guideline was mentioned also, but not sure if it is up-to-date
14:17:40 <lajoskatona> and another point for OVN: Spoofing of DNS reponses seems very wrong behavior
14:18:35 <frickler> amotoki: from that, a lot of errors should likely only be warnings
14:18:45 <lajoskatona> to tell the truth I don't know details of it, because the meeting finished, and not sure if there was anybody who has insight of this one
14:19:51 * slaweq will be back in 5 minutes
14:20:37 <ralonsoh> lajoskatona, is there a LP bug?
14:20:56 <ralonsoh> for the OVN DNS issue
14:21:22 <amotoki> we can add the labels defined at https://etherpad.opendev.org/p/pain-point-elimination#L10
14:21:30 <lajoskatona> ralonsoh: nothing: https://etherpad.opendev.org/p/pain-point-elimination#L279
14:21:36 <ralonsoh> ok
14:21:41 <amotoki> if we need more info, we can INFO NEEDED label
14:22:33 <lajoskatona> amotoki: I added now
14:22:38 <amotoki> thanks
14:23:31 <lajoskatona> for logging TC will work on it to have fresh logging guidelines, I just saw in the etherpad
14:24:09 <njohnston> I know a bit about OVN DNS spoofing, but I am not an expert.  I think dalvarez knows more.
14:24:34 <dalvarez> hi, can i help somehow? o/
14:25:10 <lajoskatona> njohnston, dalvarez: Hi, there's an "operator pain points" etherpad and related discussion
14:25:36 <lajoskatona> njohnston, dalvarez:  and one point is: "OVN: Spoofing of DNS reponses seems very wrong behavior" without more details
14:25:59 <lajoskatona> #link https://etherpad.opendev.org/p/pain-point-elimination#L279
14:26:38 <lajoskatona> if you can think about it please check the etherpad if this issue is relevant at all
14:27:13 <njohnston> So when OVN sees a DNS request originating on a local net it captures it and compares it against it's local cache, answers it if it can and then allows it to proceed out via normal routing if there is no match
14:28:03 <dalvarez> lajoskatona: ack, i agree we need more info. DNS is not really spoofing anything but acting as a DNS server itself, not a full fledged server though so we may be missing some options or maybe it's not fully compliant with the standard... :?
14:28:03 <njohnston> that causes a change in functionality for private networks that don't have egress; in ML2/OVS they could resolve DNS via the local dnsmasq but in OVN it is not possible
14:28:38 <njohnston> ^ s/resolve DNS/resolve external DNS/
14:28:49 <njohnston> where external DNS includes Designate
14:29:35 <ralonsoh> right, this is a feature gap, OVN still doesn't support external DNS servers
14:30:25 <ralonsoh> njohnston, is there a LP bug?
14:30:44 <lajoskatona> johnston, ralonsoh, dalvarez: thanks for the explanation
14:30:49 <njohnston> I'll have to go back and check, we last talked about it maybe 6-8 months ago
14:31:09 <frickler> IIUC the issue is more that the instance sends a query to e.g. 8.8.8.8, but the query never reaches that server, instead OVN creates a fake response that looks like 8.8.8.8 sent it
14:32:17 <njohnston> ralonsoh: https://bugs.launchpad.net/neutron/+bug/1902950
14:32:25 <ralonsoh> njohnston, thanks!
14:32:36 <lajoskatona> I will add these insights to the etherpad, check if we have a gap written for it, and we can continue the discussion under a LP if there's already one
14:33:08 <opendevreview> Maor Blaustein proposed openstack/neutron-tempest-plugin master: Add test_create_and_update_port_with_dns_name  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/821079
14:33:24 <lajoskatona> That's all from the meeting last week
14:34:03 <lajoskatona> #topic Community Goals
14:34:41 <lajoskatona> it was again something on the TC worked and discussed: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026280.html (look for "Community-wide goal updates")
14:34:52 <lajoskatona> mail from gmann: #link http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026218.html
14:35:11 <lajoskatona> Current Selected (Active) Goals:
14:35:13 <lajoskatona> Migrate from oslo.rootwrap to oslo.privsep
14:35:21 <lajoskatona> Consistent and Secure Default RBAC
14:35:40 <slaweq> regarding Secure RBAC I proposed patch with update of the policies: https://review.opendev.org/c/openstack/neutron/+/821208
14:35:47 <slaweq> I forgot to reply to ralonsoh's comments there
14:36:02 <ralonsoh> regarding to the first one, privsep, I think we are done
14:36:05 <lajoskatona> with privsep migration we are good, I think we even finished stadium project migration
14:36:15 <ralonsoh> we still use rootrwap to spawn the privsep daemons
14:36:22 <ralonsoh> this like some kind of "inception"
14:36:26 <ralonsoh> I would remove this part
14:36:36 <ralonsoh> removing any dependency from rootwrap
14:36:41 <lajoskatona> ralonsoh, slaweq: thanks
14:39:49 <lajoskatona> This was more a headsup (as we are progressing well /finished these) that we have these 2 goals for this cycle again
14:40:28 <lajoskatona> #topic Bugs
14:40:36 <lajoskatona> amotoki was bug deputy last week: http://lists.openstack.org/pipermail/openstack-discuss/2021-December/026335.html
14:41:27 <lajoskatona> As I checked there are a few unassigned bugs:
14:41:37 <lajoskatona> #link https://bugs.launchpad.net/neutron/+bug/1954384
14:42:21 <lajoskatona> this one again a DNS related bug
14:42:32 <ralonsoh> I think we had the same discussion last week
14:42:42 <ralonsoh> this is for designate
14:42:45 <frickler> I can look into that
14:42:57 <amotoki> but it is a case with linux bridge (not OVN)
14:43:03 <ralonsoh> if you don't use it, then this dns-domain name should be the same as the VM name
14:43:34 <ralonsoh> I'll find the LP bug I'm talking about
14:44:53 <lajoskatona> frickler, amotoki, ralonsoh: thanks
14:45:15 <njohnston> I wonder if this behavior change in Nova may affect things: https://specs.openstack.org/openstack/nova-specs/specs/xena/implemented/configurable-instance-hostnames.html
14:45:38 <njohnston> In any evenbt, it was not something I was aware of until recently, so good to boost exposure within the team
14:45:49 <opendevreview> Merged openstack/neutron-tempest-plugin master: Add test_create_port_with_dns_name  https://review.opendev.org/c/openstack/neutron-tempest-plugin/+/820456
14:46:50 <lajoskatona> amotoki: do you have other highlights from last week's bugs?
14:47:12 <amotoki> one thing is https://bugs.launchpad.net/neutron/+bug/1953478
14:47:24 <amotoki> it is realted to the timing of vif-plugged event.
14:48:09 <amotoki> we need to clarify when we send the event and what is expected from both nova and neutron perspective.
14:48:48 <amotoki> in addition, two bugs on OVN metadata agent. it would be nice if folks familiar with OVN can check them.
14:49:30 <ralonsoh> I'm checking the issues with the OVN metadata agent
14:49:43 <lajoskatona> https://bugs.launchpad.net/neutron/+bug/1953510 & https://bugs.launchpad.net/neutron/+bug/1953295 I think
14:50:01 <amotoki> lajoskatona: correct. thanks
14:50:05 <ralonsoh> at least the first one (I'll assign it to me)
14:50:23 <damiandabrowski[m]> I would really appreciate a helping hand here: https://bugs.launchpad.net/neutron/+bug/1952907
14:50:28 <damiandabrowski[m]> I have already spent several hours on troubleshooting and proposed a few possible solutions, but that's probably all I can do.
14:51:35 <amotoki> regarding the vif-plugged event bug, I can look it into detail, but more eyes would be appreciated so I let it unassigned.
14:52:29 <lajoskatona> damiandabrowski[m]: I will check your proposals, thanks for working on it
14:52:40 <damiandabrowski[m]> thank You!
14:53:34 <lajoskatona> amotoki: is it written somewhere when we send events like vif-plugged?
14:54:06 <amotoki> lajoskatona: I don't think so.... unfortunately
14:54:22 <ralonsoh> no and this is something I said I was going to write
14:54:32 <ralonsoh> (in the PTG) but I didn't have time yet
14:54:54 <lajoskatona> perhaps we can start with syncing it with nova and writing down, and check each backend
14:55:02 <amotoki> +1
14:55:09 <lajoskatona> ralonsoh: ok, thanks
14:57:01 <frickler> did anyone look at the discussion I had with mdbooth yesterday? tldr: trunk creation/deletion races with unplugging the baseport from a server
14:57:07 <frickler> no bug reported yet, I was waiting to ask mdbooth whether they would like to do that
14:58:04 <lajoskatona> frickler: that's a good idea to have a bug if they have issue
14:58:45 <lajoskatona> Ok, the CI meeting soon will start, so we have to finish now
14:58:49 <frickler> maybe it was known already or expected somehow, that's why I'm asking
14:59:40 <frickler> final question: anyone looking at the OVN/n-d-r issue yet?
15:00:10 <lajoskatona> #endmeeting