| *** yamamoto has joined #openvswitch | 00:00 | |
| *** yamamoto has quit IRC | 00:05 | |
| *** |subz3r0| has quit IRC | 00:12 | |
| *** webczat has quit IRC | 00:13 | |
| *** |subz3r0| has joined #openvswitch | 00:29 | |
| *** |subz3r0| has joined #openvswitch | 00:29 | |
| *** |subz3r0| has quit IRC | 00:34 | |
| *** |subz3r0| has joined #openvswitch | 00:35 | |
| *** |subz3r0| has joined #openvswitch | 00:35 | |
| *** rcernin has quit IRC | 01:11 | |
| *** rcernin has joined #openvswitch | 01:11 | |
| *** yamamoto has joined #openvswitch | 01:52 | |
| *** zhouhan_ has quit IRC | 02:50 | |
| *** zhouhan has joined #openvswitch | 02:51 | |
| *** rcernin has quit IRC | 03:06 | |
| *** armax has quit IRC | 03:08 | |
| *** dcbw has quit IRC | 03:33 | |
| *** rcernin has joined #openvswitch | 03:52 | |
| *** psahoo has joined #openvswitch | 03:52 | |
| *** yamamoto has quit IRC | 04:36 | |
| *** yamamoto has joined #openvswitch | 04:39 | |
| *** anilvenkata has joined #openvswitch | 05:23 | |
| *** JamesBenson has quit IRC | 05:25 | |
| *** JamesBenson has joined #openvswitch | 05:27 | |
| *** JamesBenson has quit IRC | 05:31 | |
| *** dholler has joined #openvswitch | 05:43 | |
| *** jaicaa has quit IRC | 06:07 | |
| *** jaicaa has joined #openvswitch | 06:10 | |
| *** Limech has quit IRC | 06:31 | |
| *** psahoo_ has joined #openvswitch | 06:33 | |
| *** psahoo_ has quit IRC | 06:40 | |
| *** slaweq has joined #openvswitch | 06:46 | |
| *** psahoo_ has joined #openvswitch | 06:47 | |
| *** psahoo has quit IRC | 06:48 | |
| *** JamesBenson has joined #openvswitch | 06:50 | |
| *** zhouhan has quit IRC | 06:54 | |
| *** zhouhan has joined #openvswitch | 06:54 | |
| *** JamesBenson has quit IRC | 06:55 | |
| *** dholler has quit IRC | 06:56 | |
| *** dholler has joined #openvswitch | 06:56 | |
| *** maciejjozefczyk has joined #openvswitch | 06:57 | |
| *** dholler has quit IRC | 06:58 | |
| *** dholler has joined #openvswitch | 06:59 | |
| *** psahoo has joined #openvswitch | 07:26 | |
| *** zhouhan_ has joined #openvswitch | 07:50 | |
| *** zhouhan has quit IRC | 07:53 | |
| *** jpwgarrison has quit IRC | 07:54 | |
| *** jpwgarrison has joined #openvswitch | 07:54 | |
| *** maciejjozefczyk_ has joined #openvswitch | 08:14 | |
| *** maciejjozefczyk has quit IRC | 08:15 | |
| *** maciejjozefczyk has joined #openvswitch | 08:15 | |
| *** maciejjozefczyk_ has quit IRC | 08:19 | |
| *** psahoo_ has quit IRC | 08:43 | |
| *** JamesBenson has joined #openvswitch | 08:51 | |
| *** JamesBenson has quit IRC | 08:56 | |
| *** links has joined #openvswitch | 09:03 | |
| *** rcernin has quit IRC | 09:05 | |
| *** psahoo_ has joined #openvswitch | 09:05 | |
| *** JamesBenson has joined #openvswitch | 09:26 | |
| *** JamesBenson has quit IRC | 09:30 | |
| *** rcernin has joined #openvswitch | 09:58 | |
| *** rcernin has quit IRC | 10:40 | |
| *** rcernin has joined #openvswitch | 10:48 | |
| *** rcernin has quit IRC | 11:00 | |
| *** yamamoto has quit IRC | 11:01 | |
| *** yamamoto has joined #openvswitch | 11:23 | |
| *** yamamoto has quit IRC | 11:29 | |
| *** bostondriver has joined #openvswitch | 11:59 | |
| *** yamamoto has joined #openvswitch | 12:03 | |
| *** webczat has joined #openvswitch | 12:05 | |
| *** yamamoto has quit IRC | 12:11 | |
| *** acidfu has joined #openvswitch | 12:40 | |
| *** riuzen has joined #openvswitch | 12:46 | |
| *** JamesBenson has joined #openvswitch | 12:46 | |
| *** JamesBenson has quit IRC | 12:51 | |
| riuzen | Hi, I wanna create ovn cluster for my 3nodes Openstack controller. Is it enough just using pacemaker to make my ovn db on active-stanby? Or, I also have to create ovsdb cluster replication? | 12:51 |
|---|---|---|
| *** zhouhan has joined #openvswitch | 12:57 | |
| *** zhouhan_ has quit IRC | 12:59 | |
| *** JamesBenson has joined #openvswitch | 13:01 | |
| *** fbl has quit IRC | 14:17 | |
| *** fbl has joined #openvswitch | 14:23 | |
| *** riuzen has quit IRC | 14:35 | |
| *** armax has joined #openvswitch | 14:40 | |
| *** psahoo has quit IRC | 14:44 | |
| *** JamesBen_ has joined #openvswitch | 15:16 | |
| *** JamesBenson has quit IRC | 15:17 | |
| *** psahoo_ has quit IRC | 16:15 | |
| *** acidfoo has joined #openvswitch | 16:20 | |
| *** acidfu has quit IRC | 16:23 | |
| *** dceara has joined #openvswitch | 16:34 | |
| *** dholler has quit IRC | 17:07 | |
| *** labelette has joined #openvswitch | 17:12 | |
| imaximets | Hi there. It seems like time for the OVN weekly meeting. | 17:19 |
| zhouhan | Hi | 17:19 |
| imaximets | OK. Let me just start it and other will join, I hope. :) | 17:20 |
| imaximets | #startmeeting ovn_community_development_discussion | 17:20 |
| openstack | Meeting started Thu Aug 13 17:20:33 2020 UTC and is due to finish in 60 minutes. The chair is imaximets. Information about MeetBot at http://wiki.debian.org/MeetBot. | 17:20 |
| openstack | Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. | 17:20 |
| openstack | The meeting name has been set to 'ovn_community_development_discussion' | 17:20 |
| imaximets | AFAIK, mmichelson and dceara will not be here today. | 17:21 |
| imaximets | numans, are you here? | 17:21 |
| *** zhouhan_ has joined #openvswitch | 17:21 | |
| *** zhouhan_ has quit IRC | 17:21 | |
| *** zhouhan_ has joined #openvswitch | 17:21 | |
| imaximets | OK. I could start with a quick update. | 17:22 |
| imaximets | This week I looked at issues with DB sizes. Mostly SB DB size. | 17:23 |
| imaximets | It seems like we creating lots of identical lflows for each logical datapath. | 17:24 |
| *** zhouhan has quit IRC | 17:24 | |
| imaximets | And that might be optimized by only having one lflow referencing all logical datapaths it should be applicable to. | 17:24 |
| numans | imaximets, zhouhan_ Hi | 17:25 |
| imaximets | I don't know how exactly and under which conditions this could be done, but I'm looking into that. | 17:25 |
| numans | I'm late | 17:25 |
| imaximets | That's it from my side. | 17:26 |
| *** zhouhan_ has quit IRC | 17:26 | |
| numans | imaximets, Thanks for looking into that. | 17:26 |
| *** zhouhan has joined #openvswitch | 17:26 | |
| imaximets | zhouhan seems to have connection issues. | 17:26 |
| numans | imaximets, yeah there are many flows which are repetitive | 17:26 |
| numans | I can go real quick. | 17:27 |
| imaximets | numans, It all yours. :) | 17:27 |
| numans | I did some reviews. | 17:27 |
| zhouhan | imaximets: sorry I was in readonly mode. I was asking if there is any example? | 17:27 |
| imaximets | zhouhan, for example we had a lot of reject ACL flows. | 17:28 |
| * numans will contnue after this discussion. | 17:28 | |
| zhouhan | imaximets: with the optimization, do we still need datapath information in the flow? | 17:28 |
| numans | zhouhan, what do you mean by "we still need datapath information ..." | 17:29 |
| imaximets | zhouhan, I thought to have a new table, e.g. Datapath_Group with sets of logical datapaths and have a single reference to a set from the logical flow. | 17:29 |
| zhouhan | numans: I mean, if the flows are common for all datapaths, then we can replace them with just one flow, removing the datapath match. | 17:30 |
| *** links has quit IRC | 17:30 | |
| numans | zhouhan, ok. That makes sense too. | 17:31 |
| imaximets | zhouhan, I see, but we have switches and routers and flow might be only applicable for switches, but not routers. | 17:31 |
| imaximets | at least. | 17:32 |
| numans | imaximets, may be a new column option which says its for logical switches or for routers. | 17:32 |
| zhouhan | imaximets: if that's the case, would it be better to add a datapath type, instead of creating groups? | 17:32 |
| imaximets | zhouhan, that make sense. Good point. Need to explore usecases deeply to understand if it's possible/feasible to have smaller groups. | 17:33 |
| imaximets | zhouhan, one more case is lfows for port group. e.g. port group specific ACLs. | 17:34 |
| zhouhan | I think it is good to optimize such cases if it is low hanging fruit, but I would avoid heavy changes for that, because I think the number of datapaths is much smaller than the number of ports. The size of the flow table mainly determined by number of ports. | 17:35 |
| zhouhan | imaximets: oh, I didn't notice that. If it is for all port groups, it may be straightforward to optimize, too. | 17:36 |
| numans | zhouhan, in the case of ovn-k8s, where its switch per node, there could be significat flows if say number of computes is 100 | 17:37 |
| numans | ok. | 17:37 |
| zhouhan | numans: even though, it is normal to have 10x more ports than number of computes, right? | 17:38 |
| zhouhan | sometimes, even 100x | 17:38 |
| numans | yeah. | 17:38 |
| imaximets | zhouhan, ok. we definitely still need to explore some usecases and see if it will have real benefits in real-world cases. Work in progress. :) | 17:39 |
| zhouhan | imaximets: sure, thanks! | 17:40 |
| zhouhan | numans: please continue. I will update after you. | 17:40 |
| numans | zhouhan, thanks. | 17:41 |
| numans | So I did some reviews. | 17:42 |
| numans | and a couple of small bug fix patches. | 17:42 |
| numans | There is one issue reported by openshift on openstack scenario | 17:42 |
| numans | the etcd cluster is having a downtime and a new leader is elected when some tests are run | 17:43 |
| numans | and the leader change happens when ovn-controller program flows and it updates the conjunction ids of existing flows | 17:43 |
| numans | it is for ACLs which results in conjunction | 17:43 |
| numans | So I'm working on making conjunction ids persistent | 17:44 |
| numans | so that when a port is added to a port group or during a recompute, we use the same conjunction id. | 17:44 |
| numans | It is not a big issue. But there is a very very small window for packet drops. | 17:44 |
| zhouhan | numans: sorry, how does etcd cluster impact ovn-controller? | 17:45 |
| numans | zhouhan, etcd cluster is running as application pods | 17:45 |
| numans | so the etcd traffic gets disrupted when ovn-controller changes the conjunction id for ACL flows which allow traffic for these etcd ports | 17:46 |
| numans | zhouhan, the CI test creates other pods and other ACLs and while processing those, ovn-controller is updating the existing flows | 17:46 |
| zhouhan | I see, does it happen only during flow recompute | 17:46 |
| zhouhan | ? | 17:46 |
| numans | zhouhan, it also happens when a port is added to the port group | 17:47 |
| zhouhan | ok | 17:47 |
| numans | zhouhan, with the new I-P patches, the issue is not seen often | 17:47 |
| zhouhan | I wonder is it a generic problem even without conjunction flows. | 17:47 |
| numans | as we don't recompute enough now | 17:47 |
| numans | but the issue is still seen when the CI is run with parallel=2 itseems. | 17:48 |
| numans | I'm not sure what parallel=2 exactly mean, I assume more tests are run in parallle | 17:48 |
| numans | parallel. | 17:48 |
| zhouhan | i.e. problem when there are updates in OVS flows, groups, meters, is it possible to see transient traffic broke? | 17:49 |
| numans | zhouhan, that's what I observed. | 17:49 |
| zhouhan | or, is conjunction ID recompute is the only thing we worry about? | 17:49 |
| numans | zhouhan, the issue is seen when the ACL is added like this -- "ip && .. inport == @pg1 && tcp.dst >=900 && tcp.dst <=901" | 17:50 |
| numans | zhouhan, and the issue is not seen when 2 separate ACLs are added for these 2 tcp dst ports | 17:50 |
| numans | so I think its happening when conjunction is involved | 17:50 |
| zhouhan | numans: I understand that the problem you saw is related to conjunction. I was just thinking is there similar issue even without using conjunction. | 17:51 |
| numans | zhouhan, I don't think so. | 17:51 |
| zhouhan | numans: that's great. | 17:51 |
| numans | zhouhan, in the case of conjunction, we do FLOW_MOD. | 17:51 |
| numans | zhouhan, in other cases, we don't do FLOW_MOD right ? | 17:51 |
| numans | I think either the OF flow will be deleted and added again | 17:52 |
| zhouhan | numans: I don't remember. Maybe we can check offline. I don't remember either is there any chance ct-zone-id, etc. could have similar issue. | 17:52 |
| numans | or nothing happens. | 17:52 |
| numans | zhouhan, ok | 17:52 |
| numans | sounds good | 17:52 |
| numans | may be we can discuss further when I submit the patch | 17:52 |
| zhouhan | numans: yeah, sounds good | 17:53 |
| numans | one point though - I'm planning to revisit the lflow expr patches which we had revertd earlier. | 17:53 |
| numans | to solve this issue. | 17:53 |
| numans | That's it from me. | 17:53 |
| zhouhan | I can go quickly | 17:53 |
| imaximets | zhouhan, sure. | 17:54 |
| zhouhan | I found the root cause of the scale-test regression in 20.03 compared with 2.12. It has nothing to do with 20.03 OVN, but related to the upstream OVS. | 17:54 |
| numans | great finding. I didn't see the patch closely though. | 17:55 |
| zhouhan | It is a change in ovsdb IDL code that caused the problem. I reverted the patch and the performance is comparable with 2.12 now. The revert is merged by imaximets. Thanks imaximets for the review. | 17:55 |
| imaximets | zhouhan, thanks for finding this! | 17:56 |
| numans | zhouhan, its on the client IDL side right ? | 17:56 |
| zhouhan | numans: yes | 17:56 |
| numans | zhouhan, ok. cool. | 17:56 |
| zhouhan | Now with this solved, I can compare 20.03 v.s. 20.06 more fairly, because northd doesn't appear to be the main bottleneck now. | 17:57 |
| zhouhan | There is obvious latency reduce in 20.06, thanks to numans's I-P improvement for handling changes in local chassis. | 17:58 |
| zhouhan | Now there are still bottlenecks in ovn-controller ofctrl_put() when number of ports is big enough. I am working on incrementally installing flows. | 17:59 |
| numans | zhouhan, that's cool. | 17:59 |
| numans | zhouhan, anilvenkata had a WIP patch to improve the ofctl_put. But he switched his focus else where. I'll just share the commit in his private branch. | 18:00 |
| numans | please take a look in case if that interests you. | 18:00 |
| zhouhan | numans: cool. Thanks a lot! | 18:01 |
| zhouhan | While working on this, I also found a bug related to conjunction when I-P is involved. It was introduced by the patch that handling merging conjunction flows from different logical flows. | 18:01 |
| numans | ok. | 18:01 |
| zhouhan | I fixed this first, and working on the tests, hope to send a patch soon. | 18:01 |
| numans | that's great. Looking forward to it. | 18:01 |
| zhouhan | that's my update | 18:02 |
| imaximets | zhouhan, Thanks! | 18:02 |
| imaximets | Anyone else here? | 18:02 |
| numans | zhouhan, I just need another minute to share the commit | 18:04 |
| zhouhan | numans: no worries. We can share offline. | 18:05 |
| numans | imaximets, I think we are done. | 18:05 |
| numans | zhouhan, sounds good. | 18:05 |
| imaximets | OK. Let's call it. :) | 18:05 |
| zhouhan | Let's end the meeting and discuss secrets to punish the people who don't join :) | 18:05 |
| imaximets | Thanks everyone! | 18:05 |
| imaximets | zhouhan, :) | 18:05 |
| imaximets | #endmeeting | 18:05 |
| openstack | Meeting ended Thu Aug 13 18:05:48 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4) | 18:05 |
| openstack | Minutes: http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-08-13-17.20.html | 18:05 |
| openstack | Minutes (text): http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-08-13-17.20.txt | 18:05 |
| openstack | Log: http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-08-13-17.20.log.html | 18:05 |
| numans | bYE | 18:07 |
| numans | Bye | 18:07 |
| imaximets | Bye. | 18:07 |
| flaviof | zhouhan: er: secrets to punish the people who don't join? 8$ | 18:20 |
| flaviof | sorry I'm stuck on things and never com in time. | 18:20 |
| flaviof | but I'm around. Nothikng new to say, so I keep my mouth shut too. ;) | 18:20 |
| flaviof | it is awesome hearing about all the activities y'all into! rock on!!! | 18:21 |
| zhouhan | flaviof: haha. Please keep the secret :D | 18:21 |
| flaviof | zhouhan++ <3 | 18:21 |
| *** jon_d___ has quit IRC | 18:38 | |
| *** jon_d___ has joined #openvswitch | 18:38 | |
| *** dceara has quit IRC | 18:46 | |
| *** maciejjozefczyk has quit IRC | 18:47 | |
| *** zhouhan has quit IRC | 18:58 | |
| *** zhouhan has joined #openvswitch | 18:58 | |
| *** acidfoo has quit IRC | 19:45 | |
| *** acidfoo has joined #openvswitch | 19:56 | |
| *** acidfoo has quit IRC | 20:01 | |
| *** yamamoto has joined #openvswitch | 20:13 | |
| *** yamamoto has quit IRC | 20:18 | |
| *** jtollet has joined #openvswitch | 20:33 | |
| *** slaweq has quit IRC | 20:37 | |
| *** zhouhan has quit IRC | 21:03 | |
| *** zhouhan has joined #openvswitch | 21:04 | |
| *** jtollet has quit IRC | 21:15 | |
| *** jtollet has joined #openvswitch | 21:16 | |
| *** jtollet_ has joined #openvswitch | 21:17 | |
| *** jtollet has quit IRC | 21:17 | |
| *** dceara has joined #openvswitch | 21:18 | |
| *** bostondriver has quit IRC | 21:19 | |
| *** yamamoto has joined #openvswitch | 21:55 | |
| *** dceara has quit IRC | 22:00 | |
| *** rcernin has joined #openvswitch | 22:00 | |
| *** armax has quit IRC | 22:07 | |
| *** bostondriver has joined #openvswitch | 22:24 | |
| *** bostondriver has quit IRC | 22:26 | |
| *** yamamoto has quit IRC | 22:31 | |
| *** jtollet_ has quit IRC | 22:41 | |
| *** armax has joined #openvswitch | 22:49 | |
| *** zhouhan has quit IRC | 22:52 | |
| *** zhouhan has joined #openvswitch | 22:52 | |
| *** fbl_ has joined #openvswitch | 23:52 | |
| *** fbl has quit IRC | 23:53 | |
Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!