17:26:50 <imaximets> #startmeeting ovn_community_development_meeting
17:26:50 <openstack> Meeting started Thu Apr 22 17:26:50 2021 UTC and is due to finish in 60 minutes.  The chair is imaximets. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:26:51 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:26:54 <openstack> The meeting name has been set to 'ovn_community_development_meeting'
17:26:55 <zhouhan> seems the eavesdrop link is not updating?
17:27:44 <imaximets> zhouhan, hi.
17:27:55 <zhouhan> the last meeting notes was 2021-01-07 in the link :)
17:27:56 <imaximets> hmm, seems the last log is form january...
17:28:23 <imaximets> Someone will need to figure this out, I'm not sure who manages it.
17:29:07 <zhouhan> someone from openstack ;D
17:29:27 <dceara> imaximets: zhouhan: The correct link for the logs is here: http://eavesdrop.openstack.org/meetings/ovn_community_development_meeting/2021/
17:29:32 <imaximets> zhouhan, hmm, seems like the meeting name changed from ovn_community_development_discussion to ovn_community_development_meeting.
17:30:03 <zhouhan> Ahh. Thx dceara
17:30:40 <imaximets> so, we can conclude that it's mmichelson's fault. :)
17:30:43 <dceara> zhouhan: Np, now we need to figure out how to get it updated, IIRC dalvarez set it up :)
17:31:01 <zhouhan> so we are on the record now. we can start updating :)
17:31:15 <dceara> :)
17:31:23 <imaximets> we just need to use consistent names, I think.
17:31:34 <dceara> imaximets: Ah, true.
17:31:50 <imaximets> And, probably, ask dalvarez to move logs if possible to the old folder.
17:32:17 <imaximets> I just copied from the last one.
17:32:28 <imaximets> But let's start the meeting. :)
17:32:39 <imaximets> I'll start with a small update
17:33:07 <imaximets> For the last couple of weeks I worked on ovsdb 2-tier deployment.
17:34:05 <imaximets> i.e. to allow running a series of replication servers that will replicate a main raft cluster, so the actual clients can connect to replicas instead.
17:34:43 <imaximets> This should move a lot of load from the main raft cluster and increase scalability.
17:35:02 <zhouhan> imaximets awesome! I have been talking about this for a while but never got to implement it
17:35:31 <imaximets> one of the main problems is that current ovsdb replication works in read-only mode, so I implemented a transaction forwarding.
17:36:07 <imaximets> if enabled, replication server will forward incoming transaction to the replication source.
17:36:41 <zhouhan> imaximets Oh, I thought we will have to make SB replicas read-only by ovn-controller. So instead of that you made the replicas writable?
17:39:57 <zhouhan> imaximets seems you disconnected?
17:39:58 <imaximets_> opps. got disconnected for some reason.
17:40:18 <imaximets_> zhouhan, yep.  If you asked something, please, repeat.
17:40:42 <zhouhan> Oh, I thought we will have to make SB replicas read-only by ovn-controller. So instead of that you made the replicas writable?
17:41:20 <imaximets_> I replied this (not sure if you received): replicas are not technically writable, because the do not execute transactions by their own, but forwards the transaction to the main cluster and forwards results back.
17:41:31 <imaximets_> s/the/they/
17:41:47 <zhouhan> Oh, I didn't see this reply.
17:43:08 <imaximets_> ovn-controller doesn't seem to execute a lot of "write" transactions, so this should be fine.
17:43:11 <zhouhan> This is definitely more generic and less restricted by the use cases (e.g. when ovn-controller has to update the SB). It just seemed tricky to me to support it
17:43:34 <zhouhan> It would be great if it can be done.
17:43:44 <imaximets_> I think, my implementation is straightforward enough. :)
17:43:49 <zhouhan> Would the transaction ACID still be kept?
17:44:32 <imaximets_> zhouhan, all the same as with a usual ovsdb transaction.  replica is just a proxy.
17:45:30 <zhouhan> hmm... Maybe I thought something unnecessarily complex. This is great :)
17:45:47 <imaximets_> I hope to post the patch-set somewhere soon.  Will add you in CC.
17:45:59 <zhouhan> Cool! thx
17:47:05 <zhouhan> I have a quick update
17:47:07 <imaximets_> The second part is that we need to keep HA, i.e. replicas should reconnect to active cluster member or clinet need to know that current replica syncs from the non-cluster member.
17:47:28 <imaximets_> zhouhan, let me finish a real quick. :)
17:47:32 * zhouhan sorry, please continue
17:48:23 <imaximets_> I took the second approach.  Implementation is to replicate a _Server database, so the client will receive the same information about the cluster as it is now.
17:49:57 <imaximets_> It's a bit tricky, but doable without loosing a _Server database of the replication server itself.
17:49:57 <imaximets_> Will post this part along with transaction forwarding patches.
17:49:57 <imaximets_> That's it from me.
17:49:57 <zhouhan> What's the problem of the first approach?
17:50:10 <zhouhan> replicas always connect to active RAFT cluster member?
17:50:56 <imaximets_> zhouhan, the problem is current implementation of the replication code.  Currently, replicas doesn't care where to connect and has no support for multiple remotes.
17:51:48 <zhouhan> ok, so you are using the replication mode of OVSDB server instead of a standard RAFT client
17:52:37 <imaximets_> yes, replication server is not part of a raft cluster.  it's a standard --sync-from replication.
17:52:57 <zhouhan> I was thinking a proxy working as a client to the RAFT cluster and sit in front of the "replica" server.
17:53:56 <imaximets_> I thought about something like a non-voting cluster member, but it seems like a more invasive and complicated thing.
17:53:59 <zhouhan> We can discuss more when you send out the patch.
17:54:03 <imaximets_> sure
17:54:33 <zhouhan> I worked on the "table 45 flow explostion" problem, and sent a patch series: #link https://patchwork.ozlabs.org/project/ovn/list/?series=240294
17:54:49 <zhouhan> Also sent out some minor fixes
17:55:12 <zhouhan> I forgot to add cover letter to the patch series so the title may be misleading.
17:55:44 <zhouhan> dceara Please take a look if you can, since you also discussed the problem
17:55:46 <dceara> zhouhan: Nice!  I had a very brief look at the patch to fix the flow explosion, I hope to be able to have a closer look next week.
17:55:56 <_lore_> can I go next? quite fast
17:56:02 <zhouhan> thx dceara
17:56:08 <dceara> zhouhan: Np.
17:56:11 <zhouhan> _lore_ please go ahead
17:56:32 <_lore_> this week I resumed CoPP work from dceara to rate limit control plane
17:57:18 <_lore_> we have some spot we want to rate limit so this general approach can fix them at once doing it in the kernel
17:57:44 <_lore_> I rebased it ontop of ovn master and doing some tests, it works so far
17:58:04 <_lore_> I guess we can sync if this aprroach is fine for all
17:58:24 * zhouhan refreshing his memory of CoPP
17:58:37 <_lore_> I guess I can add support for some lefover controller actions  and post v1
17:58:43 <_lore_> to continue the discussion
17:59:14 <dceara> _lore_: Thanks for taking over that work! We might want to simplify it a bit though, the configuration part was too cumbersome in my RFC.
17:59:15 <_lore_> zhouhan: actually dceara added a new column in logical flow table
17:59:31 <zhouhan> thanks _lore_
17:59:36 <zhouhan> dceara any idea why was the CoPP work paused?
17:59:43 <_lore_> yw :)
17:59:54 <_lore_> I like the generality of the approach
18:00:04 <_lore_> but maybe we can decide to simplify it
18:00:04 <dceara> zhouhan: It seems that it just fell off my priority list in time :)
18:00:24 <zhouhan> ok
18:01:01 <_lore_> there is a new table called CoPP to a list of meter devided for type like arp, nd, ...
18:02:07 <_lore_> so a logical switch, logical router or logical switch port can refer to a given row in the CoPP table to filter like arp, controller_event and so on
18:02:20 <dceara> Not sure if it's useful now but this was the last CoPP RFC version I sent out: https://patchwork.ozlabs.org/project/openvswitch/list/?series=140778&state=*
18:02:38 <_lore_> dceara: I can share the updated one in my gh
18:03:06 <_lore_> https://github.com/LorenzoBianconi/ovn/tree/CoPP
18:04:54 <dceara> _lore_: Nice, looking forward to the patches on the mailing list!
18:05:34 <numans> I can go real quick if _lore_ is done.
18:06:46 <imaximets> numans, go ahead, I guess. :)
18:06:55 <numans> I did some reviews this week.
18:07:09 <numans> And submitted v4 of the ct inv patch series.
18:07:13 <numans> thanks mdgray for the review.
18:07:25 <_lore_> numans: sorry I am done :)
18:07:42 <numans> zhouhan, Can you please take a look at these patches ? https://patchwork.ozlabs.org/project/ovn/list/?series=240351
18:08:11 <numans> and also this one please - https://patchwork.ozlabs.org/project/ovn/patch/20210412132029.675432-1-numans@ovn.org/
18:08:43 <numans> Right now I'm working on crash seen due to  assert seen here - https://github.com/ovn-org/ovn/blob/master/controller/ofctrl.c#L1936
18:09:09 <zhouhan> numans sure, I will be in review mode next week
18:09:23 <numans> Any comments on this are welcome. From the logs I see that there are many time related warnings.
18:09:57 <numans> I am suspecting if a desired_flow->installed is NULL because ovn-controller couldn't send the flow to vswitchd.
18:10:09 <numans> zhouhan, thanks.
18:10:32 <numans> I am also suspecting this commit https://github.com/ovn-org/ovn/commit/c6c61b4e3462fb5201a61a226c2acaf6f4caf917 to be the cause of the regression
18:10:45 <numans> that's it from me.
18:11:47 <dceara> I have a one line update too: I've been chasing a bug in the Port.UP/ovn-installed "notification" mechanism (due to not retrying OVS transactions on failure), I have the patch ready, running some more tests, hopefully I can post it soon.
18:12:00 <dceara> That's it from me.
18:12:51 <imaximets> OK.  Anyone else?  Questions/Discussions?
18:14:22 <imaximets> So, I guess, that's it for today.
18:14:25 <imaximets> Thanks!
18:14:28 <imaximets> #endmeeting