17:26:50 <imaximets> #startmeeting ovn_community_development_meeting 17:26:50 <openstack> Meeting started Thu Apr 22 17:26:50 2021 UTC and is due to finish in 60 minutes. The chair is imaximets. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:26:51 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:26:54 <openstack> The meeting name has been set to 'ovn_community_development_meeting' 17:26:55 <zhouhan> seems the eavesdrop link is not updating? 17:27:44 <imaximets> zhouhan, hi. 17:27:55 <zhouhan> the last meeting notes was 2021-01-07 in the link :) 17:27:56 <imaximets> hmm, seems the last log is form january... 17:28:23 <imaximets> Someone will need to figure this out, I'm not sure who manages it. 17:29:07 <zhouhan> someone from openstack ;D 17:29:27 <dceara> imaximets: zhouhan: The correct link for the logs is here: http://eavesdrop.openstack.org/meetings/ovn_community_development_meeting/2021/ 17:29:32 <imaximets> zhouhan, hmm, seems like the meeting name changed from ovn_community_development_discussion to ovn_community_development_meeting. 17:30:03 <zhouhan> Ahh. Thx dceara 17:30:40 <imaximets> so, we can conclude that it's mmichelson's fault. :) 17:30:43 <dceara> zhouhan: Np, now we need to figure out how to get it updated, IIRC dalvarez set it up :) 17:31:01 <zhouhan> so we are on the record now. we can start updating :) 17:31:15 <dceara> :) 17:31:23 <imaximets> we just need to use consistent names, I think. 17:31:34 <dceara> imaximets: Ah, true. 17:31:50 <imaximets> And, probably, ask dalvarez to move logs if possible to the old folder. 17:32:17 <imaximets> I just copied from the last one. 17:32:28 <imaximets> But let's start the meeting. :) 17:32:39 <imaximets> I'll start with a small update 17:33:07 <imaximets> For the last couple of weeks I worked on ovsdb 2-tier deployment. 17:34:05 <imaximets> i.e. to allow running a series of replication servers that will replicate a main raft cluster, so the actual clients can connect to replicas instead. 17:34:43 <imaximets> This should move a lot of load from the main raft cluster and increase scalability. 17:35:02 <zhouhan> imaximets awesome! I have been talking about this for a while but never got to implement it 17:35:31 <imaximets> one of the main problems is that current ovsdb replication works in read-only mode, so I implemented a transaction forwarding. 17:36:07 <imaximets> if enabled, replication server will forward incoming transaction to the replication source. 17:36:41 <zhouhan> imaximets Oh, I thought we will have to make SB replicas read-only by ovn-controller. So instead of that you made the replicas writable? 17:39:57 <zhouhan> imaximets seems you disconnected? 17:39:58 <imaximets_> opps. got disconnected for some reason. 17:40:18 <imaximets_> zhouhan, yep. If you asked something, please, repeat. 17:40:42 <zhouhan> Oh, I thought we will have to make SB replicas read-only by ovn-controller. So instead of that you made the replicas writable? 17:41:20 <imaximets_> I replied this (not sure if you received): replicas are not technically writable, because the do not execute transactions by their own, but forwards the transaction to the main cluster and forwards results back. 17:41:31 <imaximets_> s/the/they/ 17:41:47 <zhouhan> Oh, I didn't see this reply. 17:43:08 <imaximets_> ovn-controller doesn't seem to execute a lot of "write" transactions, so this should be fine. 17:43:11 <zhouhan> This is definitely more generic and less restricted by the use cases (e.g. when ovn-controller has to update the SB). It just seemed tricky to me to support it 17:43:34 <zhouhan> It would be great if it can be done. 17:43:44 <imaximets_> I think, my implementation is straightforward enough. :) 17:43:49 <zhouhan> Would the transaction ACID still be kept? 17:44:32 <imaximets_> zhouhan, all the same as with a usual ovsdb transaction. replica is just a proxy. 17:45:30 <zhouhan> hmm... Maybe I thought something unnecessarily complex. This is great :) 17:45:47 <imaximets_> I hope to post the patch-set somewhere soon. Will add you in CC. 17:45:59 <zhouhan> Cool! thx 17:47:05 <zhouhan> I have a quick update 17:47:07 <imaximets_> The second part is that we need to keep HA, i.e. replicas should reconnect to active cluster member or clinet need to know that current replica syncs from the non-cluster member. 17:47:28 <imaximets_> zhouhan, let me finish a real quick. :) 17:47:32 * zhouhan sorry, please continue 17:48:23 <imaximets_> I took the second approach. Implementation is to replicate a _Server database, so the client will receive the same information about the cluster as it is now. 17:49:57 <imaximets_> It's a bit tricky, but doable without loosing a _Server database of the replication server itself. 17:49:57 <imaximets_> Will post this part along with transaction forwarding patches. 17:49:57 <imaximets_> That's it from me. 17:49:57 <zhouhan> What's the problem of the first approach? 17:50:10 <zhouhan> replicas always connect to active RAFT cluster member? 17:50:56 <imaximets_> zhouhan, the problem is current implementation of the replication code. Currently, replicas doesn't care where to connect and has no support for multiple remotes. 17:51:48 <zhouhan> ok, so you are using the replication mode of OVSDB server instead of a standard RAFT client 17:52:37 <imaximets_> yes, replication server is not part of a raft cluster. it's a standard --sync-from replication. 17:52:57 <zhouhan> I was thinking a proxy working as a client to the RAFT cluster and sit in front of the "replica" server. 17:53:56 <imaximets_> I thought about something like a non-voting cluster member, but it seems like a more invasive and complicated thing. 17:53:59 <zhouhan> We can discuss more when you send out the patch. 17:54:03 <imaximets_> sure 17:54:33 <zhouhan> I worked on the "table 45 flow explostion" problem, and sent a patch series: #link https://patchwork.ozlabs.org/project/ovn/list/?series=240294 17:54:49 <zhouhan> Also sent out some minor fixes 17:55:12 <zhouhan> I forgot to add cover letter to the patch series so the title may be misleading. 17:55:44 <zhouhan> dceara Please take a look if you can, since you also discussed the problem 17:55:46 <dceara> zhouhan: Nice! I had a very brief look at the patch to fix the flow explosion, I hope to be able to have a closer look next week. 17:55:56 <_lore_> can I go next? quite fast 17:56:02 <zhouhan> thx dceara 17:56:08 <dceara> zhouhan: Np. 17:56:11 <zhouhan> _lore_ please go ahead 17:56:32 <_lore_> this week I resumed CoPP work from dceara to rate limit control plane 17:57:18 <_lore_> we have some spot we want to rate limit so this general approach can fix them at once doing it in the kernel 17:57:44 <_lore_> I rebased it ontop of ovn master and doing some tests, it works so far 17:58:04 <_lore_> I guess we can sync if this aprroach is fine for all 17:58:24 * zhouhan refreshing his memory of CoPP 17:58:37 <_lore_> I guess I can add support for some lefover controller actions and post v1 17:58:43 <_lore_> to continue the discussion 17:59:14 <dceara> _lore_: Thanks for taking over that work! We might want to simplify it a bit though, the configuration part was too cumbersome in my RFC. 17:59:15 <_lore_> zhouhan: actually dceara added a new column in logical flow table 17:59:31 <zhouhan> thanks _lore_ 17:59:36 <zhouhan> dceara any idea why was the CoPP work paused? 17:59:43 <_lore_> yw :) 17:59:54 <_lore_> I like the generality of the approach 18:00:04 <_lore_> but maybe we can decide to simplify it 18:00:04 <dceara> zhouhan: It seems that it just fell off my priority list in time :) 18:00:24 <zhouhan> ok 18:01:01 <_lore_> there is a new table called CoPP to a list of meter devided for type like arp, nd, ... 18:02:07 <_lore_> so a logical switch, logical router or logical switch port can refer to a given row in the CoPP table to filter like arp, controller_event and so on 18:02:20 <dceara> Not sure if it's useful now but this was the last CoPP RFC version I sent out: https://patchwork.ozlabs.org/project/openvswitch/list/?series=140778&state=* 18:02:38 <_lore_> dceara: I can share the updated one in my gh 18:03:06 <_lore_> https://github.com/LorenzoBianconi/ovn/tree/CoPP 18:04:54 <dceara> _lore_: Nice, looking forward to the patches on the mailing list! 18:05:34 <numans> I can go real quick if _lore_ is done. 18:06:46 <imaximets> numans, go ahead, I guess. :) 18:06:55 <numans> I did some reviews this week. 18:07:09 <numans> And submitted v4 of the ct inv patch series. 18:07:13 <numans> thanks mdgray for the review. 18:07:25 <_lore_> numans: sorry I am done :) 18:07:42 <numans> zhouhan, Can you please take a look at these patches ? https://patchwork.ozlabs.org/project/ovn/list/?series=240351 18:08:11 <numans> and also this one please - https://patchwork.ozlabs.org/project/ovn/patch/20210412132029.675432-1-numans@ovn.org/ 18:08:43 <numans> Right now I'm working on crash seen due to assert seen here - https://github.com/ovn-org/ovn/blob/master/controller/ofctrl.c#L1936 18:09:09 <zhouhan> numans sure, I will be in review mode next week 18:09:23 <numans> Any comments on this are welcome. From the logs I see that there are many time related warnings. 18:09:57 <numans> I am suspecting if a desired_flow->installed is NULL because ovn-controller couldn't send the flow to vswitchd. 18:10:09 <numans> zhouhan, thanks. 18:10:32 <numans> I am also suspecting this commit https://github.com/ovn-org/ovn/commit/c6c61b4e3462fb5201a61a226c2acaf6f4caf917 to be the cause of the regression 18:10:45 <numans> that's it from me. 18:11:47 <dceara> I have a one line update too: I've been chasing a bug in the Port.UP/ovn-installed "notification" mechanism (due to not retrying OVS transactions on failure), I have the patch ready, running some more tests, hopefully I can post it soon. 18:12:00 <dceara> That's it from me. 18:12:51 <imaximets> OK. Anyone else? Questions/Discussions? 18:14:22 <imaximets> So, I guess, that's it for today. 18:14:25 <imaximets> Thanks! 18:14:28 <imaximets> #endmeeting