Thursday, 2020-05-28

*** tbachman has quit IRC00:26
*** oanson has quit IRC00:28
*** oanson has joined #openvswitch00:29
*** troulouliou_div2 has quit IRC00:36
*** troulouliou_div2 has joined #openvswitch00:48
*** rcernin has quit IRC00:58
*** rcernin has joined #openvswitch01:04
*** fbl_ has joined #openvswitch01:21
*** fbl has quit IRC01:22
*** tbachman has joined #openvswitch01:44
*** donhw_ has joined #openvswitch02:10
*** donhw has quit IRC02:13
*** JamesBenson has quit IRC02:17
*** dcbw1 has quit IRC02:50
*** rcernin has quit IRC03:01
*** troulouliou_div2 has quit IRC03:08
*** rcernin has joined #openvswitch03:15
*** rcernin has quit IRC03:19
*** rcernin has joined #openvswitch03:19
*** rcernin has quit IRC03:57
*** troulouliou_div2 has joined #openvswitch04:14
*** rcernin has joined #openvswitch04:16
*** troulouliou_div2 has quit IRC04:41
*** yamamoto has joined #openvswitch04:43
*** rcernin has quit IRC04:48
*** rcernin has joined #openvswitch04:49
*** psahoo has joined #openvswitch05:00
*** dholler has joined #openvswitch05:31
*** anilvenkata has joined #openvswitch05:35
*** links has joined #openvswitch05:41
*** links has quit IRC06:02
*** mmirecki has joined #openvswitch06:03
*** eelco has joined #openvswitch06:03
*** Limech_ has joined #openvswitch06:11
*** Limech has quit IRC06:12
*** links has joined #openvswitch06:22
*** mmirecki has quit IRC06:25
*** jaicaa has quit IRC06:42
*** jaicaa has joined #openvswitch06:45
*** rcernin has quit IRC06:46
*** zhouhan has quit IRC07:08
*** anilvenkata has quit IRC07:22
*** anilvenkata has joined #openvswitch07:22
*** slaweq_ has joined #openvswitch08:06
*** slaweq has quit IRC08:07
*** rcernin has joined #openvswitch08:10
*** icarusfactor has quit IRC08:14
*** rcernin has quit IRC08:15
*** dceara has joined #openvswitch08:17
*** mmirecki has joined #openvswitch08:26
*** timothy has joined #openvswitch08:34
*** dholler has quit IRC08:54
*** dholler has joined #openvswitch09:07
*** rcernin has joined #openvswitch09:11
*** rcernin has quit IRC09:24
*** rcernin has joined #openvswitch09:49
*** rcernin has quit IRC09:54
*** slaweq_ is now known as slaweq10:15
*** dholler has quit IRC10:39
*** dholler has joined #openvswitch10:51
*** yamamoto has quit IRC11:31
*** yamamoto has joined #openvswitch11:32
*** anilvenkata has quit IRC11:36
*** anilvenkata has joined #openvswitch11:36
*** yamamoto has quit IRC11:37
*** JamesBenson has joined #openvswitch11:41
*** eelco has quit IRC11:43
*** eelco has joined #openvswitch11:44
*** eelco has quit IRC11:46
*** eelco has joined #openvswitch11:47
*** yamamoto has joined #openvswitch11:49
*** eelco has quit IRC11:51
*** eelco has joined #openvswitch11:51
*** ghormoon has quit IRC11:53
*** ghormoon has joined #openvswitch11:53
*** _lore_ has quit IRC11:57
*** _lore_ has joined #openvswitch12:10
*** mmirecki has quit IRC12:22
*** bostondriver has joined #openvswitch12:29
*** psahoo has quit IRC12:39
*** psahoo has joined #openvswitch12:59
*** dcbw has joined #openvswitch13:07
*** psahoo has quit IRC13:11
*** yamamoto has quit IRC13:22
*** psahoo has joined #openvswitch13:23
*** Limech_ has quit IRC13:30
*** ihrachys has joined #openvswitch13:32
*** anilvenkata_ has joined #openvswitch13:45
*** anilvenkata has quit IRC13:48
*** yamamoto has joined #openvswitch13:53
*** a5m0 has quit IRC14:11
*** yamamoto has quit IRC14:15
*** a5m0 has joined #openvswitch14:17
*** yamamoto has joined #openvswitch14:18
*** a5m0 has quit IRC14:23
*** a5m0 has joined #openvswitch14:24
*** eelco has quit IRC14:27
*** eelco has joined #openvswitch14:27
*** troulouliou_div2 has joined #openvswitch14:47
*** mmirecki has joined #openvswitch15:17
*** troulouliou_div2 has quit IRC15:25
*** mmirecki has quit IRC15:25
*** eelco has quit IRC15:26
*** links has quit IRC15:35
*** troulouliou_div2 has joined #openvswitch15:38
*** yamamoto has quit IRC15:39
*** eelco has joined #openvswitch15:47
*** eelco has quit IRC15:53
*** donhw has joined #openvswitch15:58
*** donhw_ has quit IRC16:00
*** yamamoto has joined #openvswitch16:10
*** yamamoto has quit IRC16:21
*** anilvenkata_ has quit IRC17:03
*** zhouhan has joined #openvswitch17:04
*** timothy has quit IRC17:05
*** mmirecki has joined #openvswitch17:13
numansHello17:15
numansI guess time for the OVN meeting.17:15
dcearaHi17:15
panda|offo/17:15
mmichelson#startmeeting ovn-community-development-discussion17:16
openstackMeeting started Thu May 28 17:16:16 2020 UTC and is due to finish in 60 minutes.  The chair is mmichelson. Information about MeetBot at http://wiki.debian.org/MeetBot.17:16
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:16
openstackThe meeting name has been set to 'ovn_community_development_discussion'17:16
mmichelsonHow's everyone?17:16
zhouhanHi17:16
numansGood17:16
numansHi17:16
numanspanda|off, Hi17:16
panda|offnumans: hello.17:16
flaviofhi all17:17
mmichelsonCool. I got an ack for branch creation patches, so I'm planning to get the branch created. Once we have an ACK on numans's I-P patch series then we can backport it to the branch. From there, we can take some time to test/bug fix and release 20.06.1 in early June17:17
_lore_hi all17:17
mmichelsons/20.06.1/20.06.0/17:17
numanssounds good17:18
mmichelsonWhat are people's opinions about making a 20.03.1 release?17:18
mmichelsonThere are some important bug fixes in there.17:18
zhouhansounds good to me17:18
numansI agree with that. There are bug fixes which went into 20.03 branch17:19
dceara++ for 20.03.117:19
numans*many*17:19
mmichelsonOK, cool.17:19
mmichelsonAside from release-related stuff, I don't have much to talk about so that's it for me.17:19
numansI can go real quick17:19
numansI've been working on addressing the I-P patches.17:20
numansI submitted v9 a while back.17:20
numanszhouhan and dceara FYI17:20
numansAnd thanks a lot for showing patience with me during the reviews.17:20
numansI also did some reviews a bit today.17:21
zhouhannumans: sure, will take a look asap17:21
dcearanumans, ack, sorry, I'm quite slow in reviewing them, I only focused on the first patches of the previous versions of the series17:21
numansthanks.17:21
numansdceara, no worries.17:21
numansThat's it from me.17:21
numansUnless there are any questions.17:21
zhouhannumans: it is really big change and great improvement. thanks numans!17:21
numansThanks.17:21
numanszhouhan, Got any chance to test these patches with your scale setup ?17:22
dcearanumans, zhouhan I had a question about the tracked data but I can share it on the ML if that's preferable17:22
numansIf so, I hope all went fine. If not, no worries :)17:22
zhouhannumans: no, we are still working on setting up a new scale env since the old one was decommed17:22
numansOk.17:23
numansdceara, I'm fine anything. If you want we can discuss here17:23
numansor if it would consume more time, we can take it in ML too.17:24
dcearanumans, i was just wondering if we really need a new field in the I-P engine nodes for "changed data". Isn't it enough to use the existing "data" field17:24
dcearanumans, but we can discuss the implications on the ML, it's probably better :)17:24
numansdceara, ack.17:24
numansor at the end when everyone is done.17:24
dcearanumans, ack17:24
numansThat's it from me. If some one wants to go next.17:25
panda|offnumans: I'm new, so I would be curious to know how did you sart proposing this big topic. If there's a change overview document somewhere. Sorry for the trivial question, you don't need to answer me now.17:25
numanspanda|off, You mean if we have like feature proposal thing similar to what openstack does ?17:26
numansbefore working on a feature ?17:26
panda|offnumans: yes17:26
numansWe usually do everything in the mailing list. And if someone wants to work on a feature/refactoring, he/she can chose to discuss before17:26
zhouhanI can go next17:27
numansor submit the patches as RFC to discuss further in ML17:27
numanszhouhan, sure.17:27
*** ryzhyk has joined #openvswitch17:27
* zhouhan sorry my screen was frozen and didn't see discussion going on. I can wait ...17:27
panda|offnumans: thanks17:27
numanspanda|off, let me know if any further questions.17:28
numansIf not, zhouhan all yours.17:28
flaviofpanda|off: ml info: https://mail.openvswitch.org/mailman/listinfo17:28
zhouhanok, I was mostly on code reviews and discussions.17:29
panda|offnumans: flaviof ok, thanks, I subcribed, but only recently so proably I missed the conversation. No other questions for now17:29
zhouhanI am also waiting for review of the OVS patch: #link https://patchwork.ozlabs.org/project/openvswitch/patch/1589527067-91901-1-git-send-email-hzhou@ovn.org/17:30
zhouhanHope blp or Ilya could take a look there.17:30
zhouhanThat's all17:30
*** imaximets_ is now known as imaximets17:31
imaximetszhouhan, I'll take a look, but most likely on next week.17:31
zhouhanimaximets: thanks a lot. Glad you are here :)17:31
dcearaI can go next if that's ok17:32
zhouhandceara: if we have time, we can discuss the flow explosion problem later17:32
dcearazhouhan, ah, exactly what I wanted to mention :)17:32
dcearaI ported your RFC v2 patch downstream for dalvarez to test in their scaled OSP setup. To see how big of an impact it is on openstack17:33
zhouhandceara: that patch alone doesn't solve the problem because the ARP requests are broadcasted and learned by other routers17:34
dcearazhouhan, ack, the question was more regarding the main culprit: i.e., high number of logical flows and/or mac_bindings17:34
dcearaexcept for the flow explosion problem, I did a few reviews this week and sent a new version of the monitor_cond_change series (thanks zhouhan for acks!)17:35
dcearathat's it on my side17:35
flaviofdceara: do you see any value/possibility in scaling better if the mac binding table was a per chassis thing?17:37
zhouhandceara: well, move lflow to mac-bindings should have some improvement, but it is still O(N^2) problem. But I think that RFC patch together with the proposed learn_from_arp_request=false approach should solve the problem completely.17:37
numansflaviof, the idea of the mac_Bidning table is so that other ovn-controllers can learn and program.17:38
dcearazhouhan, I agree. Did you already start looking at how we could implement learn_from_arp_request?17:38
zhouhandceara: yes, I am working on it17:39
dcearazhouhan, nice!17:39
flaviofnumans: ack. but onlt the gw chassis that own the fip is the one that needs it? I may be missing something here, so my apologies if I'm somking something with this idea.17:39
flaviof*only17:39
numansI'm way behind on this issue due to PTO :)17:40
numansflaviof, if we have per chassis, we can as well just remove mac_binding table and maintain the learnt mac bindings in-memory.17:40
numansflaviof, thats fine :)17:40
dcearazhouhan, back to flow explosion, to close the loop, for the self originated ARP packets (i.e., src.mac == router-mac) we keep the same behavior: flood in the L2 domain, right?17:41
zhouhanflaviof: the mac-binding is equivalent to the dynamic ARP cache table in a physical router. Since OVN routers are distributed (except the gateway routers), it should be in SB DB so that all chassis with a leg in the LR can use it. I am not sure how to make it per-chassis.17:42
zhouhandceara: yes I think we can keep it unchanged, unless we really see a scale issue in the dataplane for the ARP messages.17:43
dcearazhouhan, sounds good to me17:44
flaviofzhouhan: ack. I was thinking along the lines of a sb db table that only gets populated on the chassis that need it, instead of all the gateway chassis.17:44
flaviofso, thinking specifically about the gw routers17:45
dcearaflaviof, I think that could work too, but has the disadvantage that all/a lot of chassis might need to learn the same ARP in specific scenarios.17:47
numansdceara, which could be ok, since its a onetime thing.17:48
numanss/a/an17:48
flaviofack. thanks for entertaining the idea. I have lots to learn before talking more nonsense17:48
zhouhanflaviof: in this case, for the gw routers, all the gateway chassis need (they thought) 99% of the entries (if there are 100 GRs). The proposal of learn_from_arp_request=false is to let the admin tell the GRs that they don't need them.17:48
dcearanumans, until we add mac_binding timeout support. If that will ever be needed.17:49
flaviofzhouhan: ah, ack. I think we are on the same page then. just about how we could be smart about distributing the table so not all the entries need to be in a single gw somehow.17:49
flaviofdceara: ack17:50
numansdceara, at the end we can discuss about the tracking data if you're fine :)17:51
dcearanumans, sure, sounds good to me.17:51
_lore_can I go next? very fast17:51
_lore_this week I mainly this week I finalized the work for DVR ARP issue upstream and downstream17:52
_lore_:s/this week I mainly//17:52
_lore_thx to zhouhan, dceara and numans for the support17:53
zhouhan_lore_: thank you17:53
_lore_now I am switching to finalize the work to run perf on ovn-scale-test17:53
dceara_lore_, np :)17:53
_lore_I need to add the capability to stop it a given iteration otherwise the output file will be huge17:54
_lore_that's all from my side17:54
* zhouhan behind the reviews on ovn-scale-test17:55
_lore_zhouhan: I have not sent it yet :)17:55
mmichelsonDoes anyone else have anything to share?17:55
imaximetsI could17:56
imaximetsI worked with the ovsdb memory consumption issue for a last couple of weeks.17:56
imaximetsIt turned out to be (at least partially) due to huge jsnonrpc backlogs on raft connections17:57
imaximets2 patches prepared.  Thanks zhouhan for reviews!17:57
zhouhanimaximets: np :)17:58
imaximetsBoth applied today on master.17:58
imaximetsAnd there was also one small memory leak while reading db.  Applied and backported too.17:58
imaximetsI also applied today the patch from zhouhan that backports hash passing via upcall.17:59
imaximetsfor the out-of-tree kernel module17:59
imaximetsThat's it.17:59
zhouhanthanks a lot imaximets18:00
imaximetszhouhan, np.18:00
imaximetsAnd I have one small fix for the "timeout" filed in "wait" command18:01
imaximetshttps://patchwork.ozlabs.org/project/openvswitch/patch/20200525170702.761482-1-i.maximets@ovn.org/18:01
imaximetswhould be nice to have some reviews.  It blocks Cirrus CI on FreeBSD.  clang build failure.18:01
imaximetsNow that's it. :)18:02
numansimaximets, I saw the same issue with my fedora 32 and clang18:02
numansimaximets, I'll take a look tomorrow.18:02
imaximetsnumans, thanks!18:02
numansdceara, you want to ? if no is going next ?18:03
dcearanumans, we can also discuss after the meeting officially ends. I'm not sure what the best way is. I'm fine either way.18:04
numansdceara, regarding the tracked data, can you plz take a  look at the patch 5 of the series and see if it can be done without the tracked data ?18:04
zhouhanimaximets: looks like a simple fix, but since numans encountered the same issue I will let him try and ack :)18:04
numanszhouhan, although its a while I've looked into the ovsdb code :).18:05
numansI'll give test it out tomorrow18:05
dcearanumans, I'm not suggesting to not track data. I'm just asking, by looking at patch 4, what is the difference (from an I-P engine node perspective) between "data" and "tracked_data"?18:05
numansdceara, tracked data gets cleared after every run18:06
dcearanumans, seems to me like the only difference is that tracked data needs to be cleared18:06
numansdceara, I tried to model just like how we have IDL track for loops.18:06
dcearanumans, so then isn't it easier to just add a clear_data handler that can does that?18:06
dcearas/can does/can do18:06
numansdceara, we could probably do, but do you see any issues ? design concerns with this approach ?18:07
numansI think this is clear. It also has track clear handler.18:07
numansBut I'm definitely biased :)18:07
dcearanumans, the only issue I had with it was that looking at the APIs we expose for tracked_data they do exactly the same thing as the ones we expose for "data". I was wondering if that would be confusing for people trying to understand the I-P engine.18:08
mmichelsondceara, to offer my perspective, yes :)18:08
dcearanumans, but I don't have a really strong opinion about it.18:08
mmichelsonI had to look real close to understand the meaning of "tracked_data"18:09
numansmmichelson, Ok.18:09
mmichelsonAnd it turns out it just is cleared on every run. Not sure why that's called "tracked"18:09
numansmmichelson, I thought of the name from this macro for example - SBREC_PORT_BINDING_TABLE_FOR_EACH_TRACKED18:10
dcearammichelson, numans, it's quite clear when you look at how it's used in follow up patches but just looking at the inc-proc-eng.[ch] I was a bit confused18:10
numansmmichelson, if you see, the idl code, it gets cleared after every idl_run18:10
zhouhanI kind of agree with dceara. The cleanup in every engine iteration is useful, and we can abstract it just as cleanup. We can document it and point out it is particularly useful for cleanup tracked changes when necessary.18:10
numansdceara, may be it needs more comments ?18:11
numansOk.18:11
numansdceara, zhouhan mmichelson I'm fine removing the tracked_data, which means we store the same in the engine data ?18:11
dcearanumans, I'll share my thoughts on the ML and we can decide there. As I said, I just wanted to make sure I understand the differences between the two.18:11
numansOk.18:12
dcearanumans, btw, great work tackling the I-P enhancement. That's a LOT of work18:12
zhouhannumans: the problem in I-P is that there is no way to abstract tracking because each node has its own ways of data orgnization with freedom, so the nodes depending on them have to explain (convert) the data anyway.18:12
numansThanks.18:12
numanszhouhan, Ok. Right now the flowoutput handler for runtime data checks if there is tracked data. If we remove this tracked_data, then it has to see the runtime data->is_tracked for example.18:13
zhouhannumans: but I didn't see it particularly harmful so I didn't comment on it earlier. It is merely add more burden to the I-P engine to maintain. But not a big deal.18:14
numansMy intention was to model around the idl tracking.18:14
numanszhouhan, Ok. We can discuss it in ML, and happy to remove it if it doesn't add much value/or complicate the code.18:15
*** mmirecki has quit IRC18:15
dcearanumans, zhouhan thanks for the time on this. It's more clear for me now.18:15
numansYou're welcome and thanks for brining it up here.18:16
mmichelsonI'm going to end the meeting here, but don't let that stop you from continuing to talk here if you wish.18:16
numansI think it is also time18:16
mmichelson#endmeeting18:16
openstackMeeting ended Thu May 28 18:16:37 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:16
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-28-17.16.html18:16
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-28-17.16.txt18:16
openstackLog:            http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-28-17.16.log.html18:16
* panda|off waves. Thanks all.18:16
dcearaBy everyone!18:16
numanspanda|off, Bye.18:16
numansBye everyone.18:16
dceara*Bye18:16
flaviofbye all.18:17
imaximetsBye18:17
zhouhanbye all18:17
*** zhouhan_ has joined #openvswitch18:22
*** zhouhan has quit IRC18:26
*** ryzhyk has quit IRC18:32
*** acidfu has joined #openvswitch18:38
*** acidfoo_ has quit IRC18:39
*** psahoo has quit IRC18:50
*** zhouhan_ has quit IRC18:54
*** zhouhan_ has joined #openvswitch18:55
*** mmichelson has quit IRC19:24
*** zhouhan_ has quit IRC19:32
*** zhouhan has joined #openvswitch19:33
*** acidfu has quit IRC19:33
*** acidfu has joined #openvswitch19:34
*** dceara has quit IRC19:46
*** yamamoto has joined #openvswitch20:19
*** rebrec has quit IRC20:23
*** yamamoto has quit IRC20:24
*** slaweq has quit IRC20:33
*** slaweq has joined #openvswitch20:59
*** troulouliou_div2 has quit IRC21:49
*** slaweq has quit IRC21:52
*** JamesBenson has quit IRC22:05
*** JamesBenson has joined #openvswitch22:08
*** JamesBenson has quit IRC22:12
*** dcbw has quit IRC22:16
*** donhw has quit IRC22:28
*** bostondriver has quit IRC22:33
*** zhouhan_ has joined #openvswitch22:36
*** zhouhan has quit IRC22:39
*** factor has joined #openvswitch23:06
*** rcernin has joined #openvswitch23:07
*** donhw has joined #openvswitch23:14
*** dholler has quit IRC23:32
*** dholler has joined #openvswitch23:45
*** JamesBenson has joined #openvswitch23:47
*** JamesBenson has quit IRC23:51

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!