Thursday, 2020-05-07

*** troulouliou_div2 has quit IRC00:26
*** armax has quit IRC00:39
*** dceara has quit IRC00:59
*** acidfoo has quit IRC00:59
*** acidfoo has joined #openvswitch01:05
*** acidfoo has quit IRC02:00
*** acidfoo has joined #openvswitch02:06
*** dholler has quit IRC02:15
*** acidfoo has quit IRC02:16
*** acidfoo has joined #openvswitch02:20
*** dholler has joined #openvswitch02:29
*** psahoo has joined #openvswitch02:33
*** acidfoo has quit IRC03:15
*** dcbw has quit IRC04:07
*** duso has quit IRC06:05
*** duso has joined #openvswitch06:06
*** mmirecki has joined #openvswitch06:08
*** FH_thecat has quit IRC06:44
*** slaweq has joined #openvswitch06:48
*** eelco has joined #openvswitch06:51
*** tbachman_ has joined #openvswitch06:51
*** tbachman has quit IRC06:52
*** tbachman_ is now known as tbachman06:52
*** yogananth has quit IRC07:10
*** yogananth has joined #openvswitch07:10
*** yogananth has quit IRC07:24
*** yogananth has joined #openvswitch07:24
*** jaicaa has quit IRC07:31
*** jaicaa has joined #openvswitch07:33
*** cpaelzer has quit IRC07:56
*** jbenet has quit IRC07:56
*** flaviof has quit IRC07:56
*** mnasiadka has quit IRC07:56
*** Anticimex has quit IRC07:56
*** mepholic has quit IRC07:56
*** Nirkus has quit IRC07:56
*** cpaelzer has joined #openvswitch08:01
*** jbenet has joined #openvswitch08:01
*** flaviof has joined #openvswitch08:01
*** mnasiadka has joined #openvswitch08:01
*** Anticimex has joined #openvswitch08:01
*** mepholic has joined #openvswitch08:01
*** Nirkus has joined #openvswitch08:01
*** dceara has joined #openvswitch08:29
*** ktraynor has joined #openvswitch08:29
*** dceara has joined #openvswitch08:32
*** livelace has joined #openvswitch08:53
*** zhouhan_ has quit IRC09:07
*** zhouhan has joined #openvswitch09:07
*** livelace has quit IRC09:16
*** theodotos[m] has quit IRC09:40
*** theodotos[m] has joined #openvswitch09:56
*** thaller has quit IRC09:57
*** thaller has joined #openvswitch09:57
*** duso has quit IRC10:16
*** duso has joined #openvswitch10:17
*** timothy has joined #openvswitch10:25
*** bern has joined #openvswitch10:34
*** darkemon has quit IRC10:50
*** darkemon has joined #openvswitch10:53
*** eelco has quit IRC11:08
*** eelco has joined #openvswitch11:08
*** yamamoto has joined #openvswitch11:26
*** livelace has joined #openvswitch11:38
*** thaller has quit IRC12:27
*** thaller has joined #openvswitch12:27
*** thaller has quit IRC12:27
*** thaller has joined #openvswitch12:27
*** thaller has quit IRC12:28
*** thaller has joined #openvswitch12:28
*** thaller has quit IRC12:28
*** thaller has joined #openvswitch12:29
*** armax has joined #openvswitch12:33
*** bostondriver has joined #openvswitch12:39
*** yamamoto has quit IRC12:54
*** yamamoto has joined #openvswitch12:54
*** yamamoto has quit IRC14:02
*** yamamoto has joined #openvswitch14:02
*** acidfoo has joined #openvswitch14:05
*** yamamoto has quit IRC14:07
*** slaweq_ has joined #openvswitch14:14
*** slaweq has quit IRC14:14
*** yamamoto has joined #openvswitch14:18
*** slaweq_ has quit IRC14:32
*** dcbw has joined #openvswitch14:34
*** slaweq has joined #openvswitch14:34
*** eelco has quit IRC15:22
*** livelace has quit IRC15:26
*** livelace has joined #openvswitch15:34
*** mmirecki has quit IRC16:04
*** troulouliou_div2 has joined #openvswitch16:19
*** mmirecki has joined #openvswitch16:30
*** troulouliou_div2 has quit IRC16:46
*** mmirecki has quit IRC16:52
*** troulouliou_div2 has joined #openvswitch16:59
*** mmirecki has joined #openvswitch17:01
*** mmirecki has quit IRC17:08
*** blp has joined #openvswitch17:13
blpgood day all17:13
*** mmirecki has joined #openvswitch17:14
*** ryzhyk has joined #openvswitch17:15
blpAre we all assembled?17:15
blpHaven't seen much from anyone yet.17:15
*** mmichelson_ has quit IRC17:15
zhouhanHi17:16
dcearaHi17:16
*** mmichelson has joined #openvswitch17:16
*** aginwala has joined #openvswitch17:16
blpI have good news, I have all the tests passing with the ddlog implementaiton of ovn-northd.17:16
ryzhykhi17:16
mmichelsonI had to drop, did someone #startmeeting?17:16
blpno17:16
numansHi17:16
blpgo ahead17:16
numansmmichelson, no17:16
blpI forgot17:16
mmichelson#startmeeting ovn-community-development-discussion17:16
openstackMeeting started Thu May  7 17:16:42 2020 UTC and is due to finish in 60 minutes.  The chair is mmichelson. Information about MeetBot at http://wiki.debian.org/MeetBot.17:16
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.17:16
openstackThe meeting name has been set to 'ovn_community_development_discussion'17:16
blpAh, let me restart the one thing I said.17:16
blpI have good news, I have all the tests passing with the ddlog implementaiton of ovn-northd.17:16
numansthat's great.17:16
zhouhanblp: wonderful17:17
mmichelsonblp, Awesome!17:17
blpI also rebased against OVN master yesterday,17:17
blpwhich made some tests fail because there are new features,17:17
blpbut it shouldn't take long to impelemnt them.17:17
_lore_hi all, sorry to be late17:17
numansblp++17:17
blpThen, we'll work on performance. ryzhyk will probably be the real source of power there.17:17
flaviofhi all17:18
mmichelsonblp, OK, with regards to performance, do you already have some known problems or is it a matter of running tests and tweaking based on how they go?17:19
blpmmichelson: Haven't even tried anything performance wise yet.17:19
blpI just assume there'll need some work.17:19
mmichelsonGot it17:20
blpAt the very least we need to run benchmarks to compare performance.17:20
*** mmirecki has quit IRC17:21
numansblp, We have a test setup for performance testing. I think we can trigger a run with your branch. I think dceara has plans to do that.17:22
*** anilvenkata has quit IRC17:22
mmichelsonYeah, and with an up-to-date ovn-northd-ddlog implementation, this should be real interesting.17:23
blpI'll keep everyone up-to-date.17:23
blpThat's all I've got.17:23
_lore_can I go next?17:23
mmichelsongo for it!17:24
_lore_this week I worked on locking feature of ovn-northd and I spotted this issue17:24
_lore_I have a cluster with 3 ovn-northd, one active and the other in standby17:24
_lore_if I put the interface on active northd down the cluster does not recover17:25
_lore_I mean, the other 2 remain in standby17:25
_lore_I think because the active does not relase the lock17:26
_lore_is there any feature on server side to take care of it?17:26
zhouhanovsdb server named lock should release when the connection lost is detected17:27
_lore_zhouhan: is the detection done on client side?17:27
zhouhan_lore_: it is on the server side17:27
blpWhen the server detects that a connection has broken, that releases any locks that the connection held.17:28
_lore_ok, so for some reason it does not work in my case17:28
zhouhan_lore_: is the probe disabled?17:28
_lore_need to check, it is default in ovn-fake-multinode17:28
_lore_numans: ..^17:28
_lore_I will double check17:30
_lore_anyway I added some unixctl commands to check the connection status on ovn-northd17:30
_lore_I will post the patch later today17:31
numans_lore_, I think it configures to 180 seconds.17:31
_lore_ack, thx17:31
_lore_one question, what happen if the connection is flapping?17:32
blpFrom the point of view of the probe, either it succeeds or fails.17:33
mmichelsonSounds like it's at the mercy of the probe interval. So if the probe interval is short, then flapping may cause a premature release of the lock17:33
blpIf one particular database connection is flapping, and the probe sees a failure, then it means that a backup will get the connection.17:33
blpthe lock, i mean17:33
blpIf that backup is more reliable, then that's a good thing.17:34
_lore_ok, thx17:34
_lore_I will try17:34
_lore_that's all from my side17:34
zhouhanMay I go next17:35
zhouhanI continued working on the dp_hash fastpath v.s. slowpath inconsistency problem. I back ported the upstream kernel fix to OVS tree and it solved the inconsistency problem for same flow.17:35
zhouhanHowever, it doesn’t solve the "same 5-tuple same bucket” requirement, because the dp_hash generated by socket is random.17:35
mmichelsonWould numan's patch for specifying hash fields help with that sort of consistency?17:36
mmichelsonOr is this related to that?17:36
zhouhanHere is the backport patch: https://patchwork.ozlabs.org/project/openvswitch/patch/1588554154-30608-1-git-send-email-hzhou@ovn.org/17:36
numanszhouhan, Yesterday I tested and with Fedora 32 with 5.6.8+ kernel and with ovs master, I noticed that the same bucket is selected.17:37
zhouhanmmichelson: yes, numan's patch enables changing to "hash" instead of "dp_hash", which solves the requirement. However, it would be a performance degrade compares to dp_hash.17:37
numanswith distro kernel datapath module17:37
numanszhouhan, I'll retest again to be sure.17:38
zhouhannumans: yes, in some environment it is not reproduced, even with old kernel, I noticed that as well.17:38
numansOk.17:38
zhouhanblp: do you know what's the general practice of backporting kernel patch? I thought the author would usually backport to OVS tree, but it seems that doesn;t always happend17:39
zhouhanFor the dp_hash consistency problem, here is more discussion: https://mail.openvswitch.org/pipermail/ovs-discuss/2020-May/050001.html17:39
blpSometimes authors backport, but when they don't, someone else has to do it.17:39
numansblp, Will using selection method=hash in group flows result in performance degradation when compared to dp_hash ?17:40
zhouhanblp: ok, could you help review the backport patch?17:40
*** atpa8a has quit IRC17:40
blpThe VMware team eventually catches up with these things, but they don't necessarily do it quickly, so if there's any particular patch you need then you have to either ask someone politely or do it yourself.17:40
zhouhanblp: I see, thanks.17:40
mmichelsonblp, who would a good "someone" be for these sorts of things?17:40
blpGreg17:41
blpGreg Rose17:41
flavioflet me drop a link format for that here17:42
flaviof#link https://mail.openvswitch.org/pipermail/ovs-discuss/2020-May/050001.html discussion on dp_hash consistency problem17:42
mmichelsonflaviof, thanks!17:42
flaviofis this is too disruptive, lmk and I will not offe it again ;)17:42
zhouhanflaviof: thanks!17:42
flaviofnp17:42
zhouhanI noticed another constraint of dp_hash, which is the 64 hash value limit17:43
*** atpa8a has joined #openvswitch17:43
blpzhouhan: It looks like only dp_hash uses recirculation. I think that method=hash is going to send a lot of packets to userspace, hurting performance.17:43
mmichelsonI thought it was 256?17:43
zhouhanSo in LB case, if the backend number is high, it automatically fall back to "hash"17:43
zhouhanblp: exactly, that's also my concern.17:43
zhouhanmmichelson: it is 64 and I verified that :)17:44
mmichelsonzhouhan, ok I guess I was mistaken17:44
zhouhanIn ECMP use case, it may be ok, since usually there won't be so many nexthops.17:44
blpThe constnats I see in the source imply 256 bucket max.17:44
blp#define MAX_SELECT_GROUP_HASH_VALUES 25617:45
mmichelsonThat's what I was basing my question on, too.17:45
zhouhanI am pretty sure, let me find the code I saw17:45
zhouhanI will send it later, let me continue17:46
mmichelsonI'm glad we're having this talk because when my turn comes around, I have more stuff regarding groups and buckets to discuss.17:46
numansovn-k8s adds a lot of backends to the load balancer. Some times > 64.17:46
_lore_zhouhan: sorry to jump in, what probe interval are you refering to since after 300s it does not recover yet17:46
zhouhanI found another more problem of dp_hash in ECMP testing, when pinging a LRP's IP. The packet is dropped because of "deferred action limit reached, drop recirc action"17:47
zhouhanThis suggests endless recirc in the datapath. I have no idea yet why would this happen.17:47
zhouhanThere is datapath flow like: recirc_id(0x1ef492),tunnel(tun_id=0xa0011ff0002,src=10.172.66.12,dst=10.78.211.43,flags(-df+csum+key)),in_port(2),eth(),eth_type(0x0800),ipv4(frag=no), packets:460, bytes:45080, used:6.278s, actions:hash(l4(0)),recirc(0x1ef492)17:48
blpThe probe interval is the idle time before a probe is sent. An additional interval is allowed without a reply before disconnecting. If the interval is 180 s then 360 s are required to disconnect.17:48
zhouhanThe recirc_id is same between match and action, so it is recirced back to same entry.17:49
zhouhanblp: any suggestion how to debug this?17:49
_lore_blp: ack, thx17:49
blpzhouhan: My suggestion is always ofproto/trace ;-)17:50
*** ktraynor has quit IRC17:51
zhouhanblp: with dp_hash, if I give the hash input (any value), the trace would should expected actions without dropping anything.17:51
zhouhanblp: without hash provided, it always goes to "no live bucket". I thought there might be some extra tricks needed here to debug.17:52
zhouhanI wonder if it is related to how the value of "dp_hash" is generated by the datapath.17:52
zhouhanmaybe we can discuss this offline, if no immediate hint17:53
*** timothy has quit IRC17:53
zhouhanApart from this, I did some code review, and also some minor patches.17:53
zhouhanOne of them is a fix to the manpage generation. #link: https://patchwork.ozlabs.org/project/openvswitch/patch/1588354161-17113-1-git-send-email-hzhou@ovn.org/17:54
blpHmm.17:54
zhouhanblp: could you take a look at this small change? Without this the ovn-architecture manpage is not generated properly.17:55
numanszhouhan, thanks for looking into the I-P patches.17:55
numansand dceara too17:55
zhouhannumans: I will continue reviewing this week.17:55
numansthanks.17:55
dcearanumans, me too, i just looked at the first patch until now17:55
zhouhanOne of the patches I reviewed worth discussing is the multi-localnet ports on same LS.17:56
zhouhanI finally got the whole point after discussion in the review. I think I am fine with the approach, although there might be some modeling incompleteness in OVN by using a single LS to model multiple L2 networks.17:57
zhouhanThat's all that I want to report :)17:57
numansI saw the discussion between you and ihrachys. I didn't get the chance to look into the patches yet.17:57
numansI can go real real quick.17:57
numansI reviewed few patches.17:58
numansAnd I worked on the load balancer selection fields support. Thanks to Han, Mark and Maciej for the reviews.17:58
numansI applied that patch to master.17:58
blpzhouhan: The question in my mind is, why would it recirc to the same bucket? It's very odd. Can you get that out of the trace?17:58
numansI also worked on a small feature to mark a packet from each VIF of the logical port if the mark is set by CMS.17:59
numans#link  https://patchwork.ozlabs.org/project/openvswitch/patch/20200501184810.1082602-1-numans@ovn.org/17:59
numansThis feature is required by ovn-k8s.17:59
numansThat's it from me.17:59
mmichelsonI'll jump in real quick17:59
mmichelsonI was working on an issue reported by Girish ages ago where he was attempting to set up a large load balancer. The problem is the GROUP_MOD request sent by ovn-controller exceeded the max size of an openflow message.18:00
zhouhanblp: the trace never showed recirc back to same ID. The trace shows either "no live bucket" and stop, or (if I provide a hash value like dp_hash(0x1/0xf)), it will just finish normally and finally output the packet.18:00
zhouhanblp: or is there anything I missed for how to do the trace in this case?18:01
mmichelsonI wrote up a patch to split the request into an ADD, followed by one or more INSERT_BUCKETS commands. This way the entire group can be communicated to ovs-vswitchd18:01
mmichelsonThis introduces two problems.18:02
mmichelson 1) If you try to run `ovs-ofctl dump-groups br-int` you get an openflow error, presumably because it's attempting to communicate the entire group in a single OF response instead of using multipart.18:02
mmichelsonAnd more importantly 2) There are messages in the ovs-vswitchd log saying that too many hash values are required, so it falls back to "default hash method" from dp_hash18:02
*** psahoo has quit IRC18:02
mmichelsonI'm curious about what this "default hash method" is, since it's not documented in the group documentation in ovs-ofctl18:03
mmichelsonMy assumption is that this means a super slow userspace hashing method, but it's not clear what the hash is based on.18:03
*** ryzhyk has quit IRC18:03
numansmmichelson, I think default hash method is selection_method=hash with no hash fields.18:04
numans#link https://github.com/openvswitch/ovs/blob/master/ofproto/ofproto-dpif-xlate.c#L455318:04
numanssee this function.18:04
numansI mean no hash fields specified by the controller18:05
zhouhanblp: mmichelson: My bad for the MAX dp_hash values. It is 256 instead of 64. So I guess what I tested was 256. :(18:05
mmichelsonnumans, I had seen that, but I hadn't followed what method that actually used for hashing. I guess I can dig a little deeper18:05
numansOk. I think it uses 5-tuple hash.18:06
mmichelsonall right18:06
mmichelsonOK, I'm about 75% through writing a test for it in the test suite. I'll go ahead and post the patch when I'm done with it. I suspect it won't make 20.0618:07
mmichelsonThat's fine though.18:07
mmichelsonSpeaking of, I haven't seen any responses to my e-mail from Friday about which patches people want reviewed for inclusion in 20.06.18:07
numansmmichelson, I've few patches. I reply to your email.18:07
mmichelsonnumans, thanks18:08
mmichelsonAnd that's all from me.18:08
dcearaI have a short update too18:08
dcearaI sent a v5 of the IDL fix for monitor_cond_since when disconnects happen. zhouhan, I don't know how easy it is to scale test this in your scenario but I was wondering if you could try it out as you saw the issue on a lot of nodes18:09
dceara#link https://patchwork.ozlabs.org/project/openvswitch/list/?series=175319 monitor_cond_since fix18:09
dcearazhouhan, also a question, re _lore_'s ovn-scale-test PR, I can merge it but I wasn't sure if you're ok with reworking the ACL/PG parts as a follow up PR18:10
dceara#link https://github.com/ovn-org/ovn-scale-test/pull/193 network_policy PR in ovn-scale-test18:11
zhouhanblp: mmichelson: I take it back. I found what I tested is due to this line: "if (group_setup_dp_hash_table(group, 64))"18:11
mmichelsonzhouhan, what a roller coaster ride!18:11
dcearaThat's it from me, thanks18:11
zhouhanIt was hardcoded to 64 when no selection method is specified. So with numan's fix, it should use 256 because dp_hash is specified with the OF1.5 :)18:12
zhouhandceara: sure, I can take a second look for the PR.18:12
zhouhandceara: and thanks a lot for the monitor_cond_since fix! I will review it.18:13
dcearazhouhan, np, now I managed to add a test for it too so that should give more confidence18:13
zhouhandceara: for the testing, I am not sure. We only saw it once, but we will keep watching.18:13
dcearazhouhan, ok, with our scale test setup I see the issue often but just on a few nodes18:14
dceara<10 out of 200 usually18:14
zhouhandceara: that's good, then it will be easy to verify the fix at least :)18:15
dcearazhouhan, with the fix, we don't hit the bug anymore, just saying :)18:15
flaviofMay I go next?18:16
flaviofzhouhan your comment on man page made me recall a modest progress I got working for ovn (with Russell's help):18:17
flaviof#link https://docs.ovn.org/en/latest/ref/index.html OVN readthedocs: reference guide18:17
flaviofThat is rebuilt out of a webhook from ovn repo, so it should always be up to date.18:17
flaviofalso...18:19
flaviofI noticed a change in behavior for acls. The acls for allow-related action do not conntrack packets to the logical router.18:19
numansflaviof++18:20
zhouhanflaviof: Actually I checked that one, too :) However it is not up to date.18:20
zhouhanflaviof: maybe it is from a stable branch?18:20
flaviofoh, it should now... the webhook was not working until ~1week ago18:21
blpmmichelson: The default hash method is a symmetric L4 hash.18:21
flaviofdid you check after that?18:21
zhouhanThe recent change made by blp in ovn-architecture doc is not there.18:21
zhouhanI checked just now :)18:21
flaviofoh, ok. sorry, that part needs to be address still. my bad.18:21
flaviofthe webhook does not yet do the "make docs" target.18:22
zhouhanflaviof: ok, I see.18:23
flaviofwill work on that next. ;)18:23
flaviofanyways, on the acl behavior subject.18:23
flaviofA few months back, I wrote a little blog covering ovsdbapp as an alternative18:24
flaviofto do what russellb wrote in an old gist with shell commands:18:24
flaviof#link https://gist.github.com/russellb/4ab0a9641f12f8ac66fdd6822ee7789e russellb/ovn-test-icmp-reproducer.sh18:24
flaviof#link https://github.com/flavio-fernandes/ovsdbapp_playground/blob/a9e780ce7ad57215b2200eba14c515482be84d63/scripts/step2_create_logical_ports.py russellb's equivalent in ovsdbapp18:24
zhouhanblp: I think the default for userspace datapath is "symmetric L4 hash", but for kernel datapath it is the hash from skb.18:24
flaviofThe acls for allow-related action do not conntrack packets to the logical router.18:25
flaviofIn order to make it work, I needed to add new explicit rules:18:25
flaviof#link https://github.com/flavio-fernandes/ovsdbapp_playground/commit/a9e780ce7ad57215b2200eba14c515482be84d63 acl rules changes to make18:25
flaviofIs this a known and expected behavior? No worries if you do not know; just thought of18:25
flaviofbringing it up here in case any of you have made changes in the last month or so that could have changed this behavior.18:25
flaviofthat is all. sorry for the overlapp.18:25
numansflaviof, you mean you're not able to ping the router ip unless an ACL is added ?18:25
blpzhouhan: I don't see anything datapath dependent in ofproto-dpif-xlate.c for this. Look at pick_select_group() and pick_default_select_group().18:25
flaviofnumans: no, sorry. I can ping the rtr just fine, using just the 'allow-related' rule.18:26
numansflaviof, ok18:26
flaviofnumans: but now it seems that the conntrack logic does not take packets to rtr into account. This may be a fad18:27
numansok.18:27
flaviofargh... sorry. I _used_ to be able to ping router by having the  'allow-related' rule.  Now that is no longer working.18:28
numansflaviof,  can you report this in discuss ML so tht we can discuss there ?18:28
flaviofnumans: will do. thanks.18:28
* numans signs off.18:28
numansBye everryone18:28
mmichelsonBye!18:28
mmichelson#endmeeting18:28
flaviofbue all18:28
openstackMeeting ended Thu May  7 18:28:53 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)18:28
openstackMinutes:        http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-07-17.16.html18:28
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-07-17.16.txt18:28
openstackLog:            http://eavesdrop.openstack.org/meetings/ovn_community_development_discussion/2020/ovn_community_development_discussion.2020-05-07-17.16.log.html18:28
zhouhanblp: for dp_hash, it should be pick_select_group() -> pick_dp_hash_select_group(), right?18:28
zhouhanand there it triggers ctx_trigger_recirculate_with_hash()18:29
dcearaBye everyone18:29
_lore_bye all18:30
zhouhanThis problem didn't happen at the source HV, where I believe the hash is generated by linux socket. It always happens on gateway nodes, where the packet is received from tunnel18:31
zhouhanblp: ^18:31
zhouhanNot sure if this is related18:31
*** acidfoo has quit IRC18:33
*** acidfoo has joined #openvswitch18:34
zhouhanblp: Oh I just realized your earlier post about the default is not about dp_hash default. Forgot what I said. It is not relevant :)18:36
*** mmirecki has joined #openvswitch18:38
*** aginwala has quit IRC18:38
*** ryzhyk has joined #openvswitch18:45
*** ryzhyk has quit IRC18:47
*** mmirecki has quit IRC19:02
*** SarahStep has joined #openvswitch19:08
*** SarahStep has quit IRC19:11
*** dceara has quit IRC19:24
*** livelace has quit IRC19:48
*** mmirecki has joined #openvswitch19:52
*** dceara has joined #openvswitch19:57
*** dceara has quit IRC20:13
*** mmirecki has quit IRC20:23
*** acidfu_ has joined #openvswitch20:29
*** acidfoo has quit IRC20:31
*** livelace has joined #openvswitch20:39
*** rebrec79 is now known as rebrec21:03
*** ChantalZale has joined #openvswitch21:11
*** ChantalZale has quit IRC21:14
*** livelace has quit IRC21:14
*** ohama has quit IRC21:35
*** ohama has joined #openvswitch21:35
*** blp has left #openvswitch21:37
*** dcbw has quit IRC22:01
*** slaweq has quit IRC22:14
*** duso has quit IRC22:15
*** duso has joined #openvswitch22:17
*** slaweq has joined #openvswitch22:20
*** troulouliou_div2 has quit IRC22:24
*** slaweq has quit IRC22:24
*** bostondriver has quit IRC22:56
*** zhouhan_ has joined #openvswitch23:21
*** zhouhan has quit IRC23:25

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!