18:14:55 <mmichelson> #startmeeting ovn_community_development_meeting
18:14:56 <openstack> Meeting started Thu Mar  4 18:14:55 2021 UTC and is due to finish in 60 minutes.  The chair is mmichelson. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:14:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:14:58 <_lore_> hi all
18:14:59 <openstack> The meeting name has been set to 'ovn_community_development_meeting'
18:15:15 <numans> Hello
18:15:22 <mmichelson> As a reminder, tomorrow is the expected release date of OVN 21.03
18:15:40 <mmichelson> Are there any urgent last minute bug fixes that need to be made before we can release?
18:16:07 <mmichelson> If so, then please post them in here at some point during the meeting
18:16:49 <numans> Yes. I have a 2 patch series - https://patchwork.ozlabs.org/project/ovn/list/?series=231930
18:16:51 <mmichelson> Most all of my work recently has been trying to fix an oddball network setup that OpenStack expects to work but that does not in OVN currently
18:16:59 <numans> in which the 2nd one applies to branch-21.03
18:17:04 <numans> sorry go ahead
18:17:11 <mmichelson> numans, it's fine, I'm done.
18:17:15 <imaximets> mmichelson, we need to shift OVS submodule, I guess. DO we have submodule on branch-21.03?
18:17:34 <mmichelson> imaximets, yes, we should. We can update it if needed.
18:17:59 <numans> mmichelson, for sure we need to update for master
18:18:09 <numans> otherwise north-ddlog compilation fails.
18:18:13 <mmichelson> got it
18:18:33 <mmichelson> If we can determine which specific OVS commit to update the submodule to, then I can put that change in.
18:19:32 <imaximets> mmichelson, there is a ovsdb-cs fix that we need.  So, at least: ac09cbfcb70ac6f443f039d5934448bd80f74493
18:19:38 <mmichelson> imaximets, ok
18:19:48 <numans> Also blp sent an email for this - https://mail.openvswitch.org/pipermail/ovs-dev/2021-February/380834.html
18:20:51 <blp> I wasn't quite sure what to do to update the submodule. Do I just do a commit for it and then send an email as usual?
18:21:03 <blp> I have not worked with submodules much.
18:21:27 <mmichelson> blp, this is new territory for us as a project. My thought was that we'd submit it as a patch just like anything else.
18:22:10 <imaximets> mmichelson, dceara also has idl fixes on a list, but I had no enough time to review them yet.  Pretty important, so we will have to move submodule one more time once they got in.
18:23:25 <mmichelson> imaximets, OK. Should this delay the release?
18:24:32 <numans> I'd say better to delay so that release ovn tag points to the required ovs commit
18:24:37 <imaximets> blp, mmichelson: for submodule updates... Submodule updates are just simple patches, so the process should not be different.
18:25:33 <imaximets> mmichelson, numans: idl fixes are important, but I don't know how much time it will take for me to review them.  It's a tracking code and it's not easy.
18:26:07 <numans> I'm planning to test them out tomorrow and hopefully review the code too. I don't know much of the tracking code.
18:26:13 <numans> may be zhouhan or blp can take a look ?
18:26:20 <numans> its blocking a customer deployment too.
18:26:29 <imaximets> If someone could help, that will be great.
18:26:46 <numans> This is the patchset - https://patchwork.ozlabs.org/project/openvswitch/list/?series=231872
18:27:30 <blp> Oh, yuck, this is the hardest part of the idl code.
18:27:58 <blp> Maintaining the graph structure is not fun.
18:28:07 <zhouhan> numans: ok. I will take a look. It looks to be a bug of my initial patch
18:28:27 <dceara> Sorry for joining late, thanks numans and imaximets for bringing up the IDL issue.  Yes, I'd be very grateful for reviews because it's quite complex what's going on there.
18:28:32 <numans> Also we have deployed this fix on our one of the internal deployment where the issue is seen and we haven't heard of any ovn-controller crashes after that.
18:28:49 <numans> So seems like its working :)
18:28:58 <blp> I do like that it has a thorough commit message.
18:29:17 <zhouhan> Does DDlog ovsdb wrapper use this part of code?
18:29:29 <blp> No, DDlog doesn't use anything in ovsdb-idl.*
18:30:15 <zhouhan> blp: ok, good to know. So it must have its own way to find the old/new data even for a deleted row, right?
18:30:28 <blp> Yes.
18:30:44 <blp> DDlog is very strong at dealing with changes, and with graph algorithms.
18:31:33 <blp> Algorithms like graph connectivity are basically one-liners.
18:32:06 <imaximets> blp, we need ovn-controller-ddlog ASAP. :D
18:32:30 <imaximets> jk
18:32:31 * zhouhan need to get some time to experience the power of DDlog
18:32:59 * numans started getting some nice feelers about ddlog. long way to go though.
18:33:14 <numans> I mean to learn it :)
18:33:15 <mmichelson> Yeah I need to take another week to really get my DDLog sea legs
18:33:40 <numans> Ok. Can I go real quick If no one is updating ?
18:34:02 <mmichelson> go for it numans
18:34:20 <numans> I tested blp's new ddlog improvement patches and provided my Acks. Also reported a couple of issues.
18:35:01 <blp> I see a remarkable number of races in our tests.
18:35:19 <numans> I submitted a 2 patch series to address some issues for the feature on supporting lb_force_snat_ip router ip option. The first patch is a ddlog patch.
18:35:59 <numans> zhouhan, If you could take a look at my replies for the ct.inv patch and provide your comments. That would be great.
18:36:09 <numans> That's it from me.
18:36:15 <zhouhan> numans: ok, checking now
18:36:22 <mmichelson> numans, I had a look at those patches, but my lack of ddlog certainty has made it difficult for me to ACK them with confidence
18:36:39 <mmichelson> (I understand the C part of patch 2 just fine though :) )
18:36:53 <numans> mmichelson, you could ack for the C part :).
18:37:01 <mmichelson> numans, yeah I guess that's true
18:37:25 <numans> blp, I also noticed some memory leaks with northd-ddlog.
18:37:43 <blp> numans: I'll take a look at the ddlog code in your patch "northd: Fix the missing force_snat_for_lb flows when router_ip is..."
18:37:57 <blp> numans: Memory leaks are usually an easy fix. I'll take care of them.
18:38:18 <numans> blp, thanks. I think there can be a better way to do the ddlog changes I did.
18:38:40 <numans> cool.
18:39:01 <numans> If someone wants to go next.
18:39:26 <imaximets> Very small update from my side.
18:40:05 <blp> numans: Your patch introduces a new ddlog function force_snat_for_lb(), but I don't think it calls it anywhere.
18:40:43 <imaximets> I pushed raft fixes and ovsdb-cs fix to OVS master and backported as necessary.  Will review idl patches from dceara once will find enough time.  That's it.
18:40:50 <numans> blp, I think I'm making use of it in northd.dl. May be I forgot to check in the code. I'll double check the patch now.
18:41:23 <blp> numans: Oh, gosh, I didn't see that change because there was C code in the middle. Sorry.
18:41:46 <numans> no worries. I was wondering if I checked in the code. I normally do that mistake.
18:42:09 <zhouhan> numans: getting rid of ct.inv seems to be a big behavior change (although maybe small in the code). Did we get enough feedback from users (e.g. customers of RedHat?). I am really curious how people rely on (or disregard) it.
18:43:22 <numans> zhouhan, From what I understand, with ovs datapath being liberal on tcp window validation we will never hit the ct.inv scenario.
18:44:05 <numans> zhouhan, I understand your concern. I'll work on v2 and make it's usage enabled by default.
18:44:14 <numans> and add a config option.
18:44:21 <zhouhan> numans: how about checksum errors, or receiving a packet without TCP connection established?
18:45:18 <numans> zhouhan, when you send a pkt to conntrack it will mark it as new if its not yet established.
18:45:26 <numans> I need to check on the checksum errors.
18:46:21 <zhouhan> numans: sorry that I don't have fresh memory about LB usage of CT. Does it rely on ct.inv, too? (in response to why users won't just use stateless ACLs if they don't care ct.inv)
18:47:32 <numans> zhouhan, right now we send all the pkts to conntrack if a logical switch has lb associated.
18:47:45 <numans> and hence we can't have stateless ACLs.
18:47:59 <zhouhan> numans: hmm, got it
18:48:48 <zhouhan> numans: thx for explaining. I will response in the email. (worse case the configurable option should work, I think)
18:48:57 <numans> zhouhan, thanks.
18:49:28 <numans> zhouhan, if you could also take a look at the other physical/logical flow engine patch that would be great.
18:49:41 <numans> I need to rebase though.
18:50:03 <zhouhan> numans: yes, really sorry that. I started the review but somehow got distracted. It is hanging in my head :)
18:50:19 <numans> no worries.
18:50:38 <zhouhan> numans: no worries about rebase. I can review based on a earlier commit.
18:50:41 <numans> there is another RFC patch I submitted - if you could take a real quick when you get time - https://patchwork.ozlabs.org/project/ovn/patch/20210225191950.3494656-1-numans@ovn.org/
18:50:48 <numans> thanks.
18:50:57 <numans> this RFC patch is not urgent.
18:51:15 <zhouhan> numans: sure. It is somehow related
18:52:00 * zhouhan sorry for hijacking the meeting. If someone is updating please continue
18:52:20 <mmichelson> Uh, I think we were on imaximets but I think he was done
18:52:32 <imaximets> mmichelson, yep.
18:52:33 <mmichelson> So whoever wants to go next, feelf ree
18:52:40 <mmichelson> s/feelf ree/feel free/
18:53:41 <mmichelson> And if nobody else wishes to report, then I will declare this meeting over.
18:54:05 <mmichelson> Based on the need to update the OVS submodule, and based on needing an unmerged fix, I think we should probably delay the release of 21.03 until we get that fix included.
18:54:22 <numans> +1
18:54:36 <mmichelson> In the meantime, I'll take a look at the OVN patches that numans linked and see if we can get those merged too.
18:54:44 <_lore_> can I go next?
18:54:47 <numans> thanks.
18:54:48 <mmichelson> oh sure thing _lore_
18:55:12 <_lore_> last week I mainly worked adding counters for ovn incremental processing
18:55:19 <_lore_> posted v6 upstream
18:55:29 <_lore_> acked by Mark (Gray)
18:56:02 <_lore_> then I posted a refactor of nat code in ovn-northd, no behaviour changes, just code movement :)
18:56:09 <_lore_> thx
18:57:02 <blp> I've got to go... talk to you guys next week
18:57:51 <mmichelson> _lore_, I'm interested in that refactor. I need to reserve some time to review it :)
18:57:58 <mmichelson> Anybody else?
18:59:04 <mmichelson> OK, thanks everyone!
18:59:05 <_lore_> mmichelson: ack, go for it :)
18:59:11 <mmichelson> Bye!
18:59:13 <mmichelson> #endmeeting