Wednesday, 2020-05-13

*** moguimar has quit IRC00:03
*** b1airo has quit IRC00:15
*** martial has quit IRC00:16
*** dmacpher_ has quit IRC00:23
*** rfolco|rover|off has quit IRC00:25
*** dmacpher has joined #openstack-meeting00:27
*** yasufum has joined #openstack-meeting00:29
*** tetsuro has joined #openstack-meeting00:30
*** yasufum_ has joined #openstack-meeting00:33
*** yasufum has quit IRC00:33
*** yasufum_ is now known as yasufum00:33
*** brinzhang has joined #openstack-meeting00:36
*** ociuhandu has joined #openstack-meeting00:36
*** tetsuro_ has joined #openstack-meeting00:40
*** tetsuro has quit IRC00:43
*** ociuhandu has quit IRC00:44
*** dmacpher has quit IRC00:55
*** Liang__ has joined #openstack-meeting00:59
*** dmacpher has joined #openstack-meeting01:01
*** dmacpher has quit IRC01:16
*** dmacpher has joined #openstack-meeting01:21
*** dmacpher has quit IRC01:25
*** dmacpher has joined #openstack-meeting01:30
*** redrobot has quit IRC01:34
*** brinzhang_ has joined #openstack-meeting01:35
*** dmacpher_ has joined #openstack-meeting01:36
*** brinzhang has quit IRC01:39
*** dmacpher has quit IRC01:39
*** masahito has joined #openstack-meeting01:39
*** JangwonLee_ has joined #openstack-meeting01:41
*** brinzhang_ has quit IRC01:41
*** brinzhang_ has joined #openstack-meeting01:41
*** tetsuro has joined #openstack-meeting01:44
*** jmasud has quit IRC01:44
*** JangwonLee__ has quit IRC01:44
*** tetsuro_ has quit IRC01:47
*** JangwonLee__ has joined #openstack-meeting01:48
*** viks____ has joined #openstack-meeting01:49
*** JangwonLee has joined #openstack-meeting01:50
*** JangwonLee_ has quit IRC01:52
*** JangwonLee__ has quit IRC01:52
*** dmacpher has joined #openstack-meeting02:05
*** dmacpher_ has quit IRC02:06
*** JangwonLee_ has joined #openstack-meeting02:09
*** tetsuro_ has joined #openstack-meeting02:12
*** JangwonLee has quit IRC02:12
*** dmacpher_ has joined #openstack-meeting02:13
*** dmacpher has quit IRC02:15
*** tetsuro has quit IRC02:15
*** dmacpher_ has quit IRC02:18
*** dmacpher has joined #openstack-meeting02:19
*** andrebeltrami has quit IRC02:28
*** kevinz has joined #openstack-meeting02:35
*** dmacpher has quit IRC02:49
*** dmacpher has joined #openstack-meeting02:53
*** jmasud has joined #openstack-meeting02:56
*** tetsuro has joined #openstack-meeting03:06
*** tetsuro_ has quit IRC03:09
*** gyee has quit IRC03:18
*** masahito has quit IRC03:22
*** dustinc has quit IRC03:34
*** psachin has joined #openstack-meeting03:37
*** dmacpher_ has joined #openstack-meeting03:48
*** dmacpher has quit IRC03:51
*** dmacpher_ has quit IRC03:53
*** dmacpher has joined #openstack-meeting03:57
*** tetsuro_ has joined #openstack-meeting04:06
*** tetsuro has quit IRC04:09
*** jmasud has quit IRC04:11
*** dmacpher_ has joined #openstack-meeting04:15
*** dmacpher has quit IRC04:17
*** jmasud has joined #openstack-meeting04:22
*** evrardjp has quit IRC04:36
*** evrardjp has joined #openstack-meeting04:36
*** ociuhandu has joined #openstack-meeting04:42
*** diablo_rojo has quit IRC04:44
*** dmellado has quit IRC04:45
*** ociuhandu has quit IRC04:49
*** jmasud has quit IRC04:49
*** jmasud has joined #openstack-meeting04:54
*** psachin has quit IRC04:56
*** dklyle has quit IRC05:01
*** rbudden has quit IRC05:03
*** rbudden has joined #openstack-meeting05:03
*** rbudden has quit IRC05:04
*** rbudden has joined #openstack-meeting05:04
*** rbudden has quit IRC05:04
*** links has joined #openstack-meeting05:34
*** tetsuro has joined #openstack-meeting05:43
*** jmasud has quit IRC05:46
*** tetsuro_ has quit IRC05:47
*** manuvakery has joined #openstack-meeting05:52
*** jmasud has joined #openstack-meeting06:01
*** njohnston has quit IRC06:05
*** jmasud has quit IRC06:18
*** jmasud has joined #openstack-meeting06:19
*** tetsuro_ has joined #openstack-meeting06:23
*** belmoreira has joined #openstack-meeting06:24
*** tetsuro has quit IRC06:26
*** tetsuro has joined #openstack-meeting06:29
*** ociuhandu has joined #openstack-meeting06:30
*** ttsiouts has joined #openstack-meeting06:30
*** tetsuro_ has quit IRC06:32
*** links has quit IRC06:37
*** ttsiouts has quit IRC06:39
*** JangwonLee__ has joined #openstack-meeting06:39
*** links has joined #openstack-meeting06:40
*** ttsiouts has joined #openstack-meeting06:41
*** lpetrut has joined #openstack-meeting06:42
*** JangwonLee_ has quit IRC06:42
*** jmasud has quit IRC06:49
*** JangwonLee_ has joined #openstack-meeting06:50
*** slaweq has joined #openstack-meeting06:51
*** dmacpher_ has quit IRC06:52
*** JangwonLee__ has quit IRC06:53
*** dmacpher has joined #openstack-meeting06:55
*** witek_ has joined #openstack-meeting06:59
*** vishalmanchanda has joined #openstack-meeting07:03
*** Lucas_Gray has joined #openstack-meeting07:03
*** jmasud has joined #openstack-meeting07:03
*** apetrich has quit IRC07:11
*** Lucas_Gray has quit IRC07:22
*** ykatabam has quit IRC07:23
*** tetsuro_ has joined #openstack-meeting07:24
*** tetsuro has quit IRC07:27
*** ociuhandu has quit IRC07:27
*** ociuhandu_ has joined #openstack-meeting07:27
*** rpittau|afk is now known as rpittau07:29
*** ralonsoh has joined #openstack-meeting07:29
*** jmasud has quit IRC07:30
*** dmellado has joined #openstack-meeting07:37
*** apetrich has joined #openstack-meeting07:45
*** apetrich has quit IRC07:45
*** apetrich has joined #openstack-meeting07:46
*** e0ne has joined #openstack-meeting08:05
*** ttsiouts has quit IRC08:34
*** tetsuro_ has quit IRC08:47
*** ykatabam has joined #openstack-meeting08:49
*** ttsiouts has joined #openstack-meeting08:53
*** tetsuro has joined #openstack-meeting09:07
*** tetsuro has quit IRC09:23
*** jraju__ has joined #openstack-meeting09:27
*** links has quit IRC09:28
*** Liang__ has quit IRC09:32
*** links has joined #openstack-meeting09:47
*** jraju__ has quit IRC09:47
*** ociuhandu_ has quit IRC10:01
*** ociuhandu has joined #openstack-meeting10:02
*** manuvakery has quit IRC10:12
*** yasufum has quit IRC10:16
*** njohnston has joined #openstack-meeting11:01
*** rpittau is now known as rpittau|bbl11:04
*** ociuhandu has quit IRC11:23
*** ykatabam has quit IRC11:38
*** yamamoto has joined #openstack-meeting11:43
*** yamamoto has quit IRC11:43
*** brinzhang_ has quit IRC11:48
*** ykatabam has joined #openstack-meeting11:55
*** ociuhandu has joined #openstack-meeting12:04
*** rfolco has joined #openstack-meeting12:04
*** rfolco is now known as rfolco|rover12:05
*** ykatabam has quit IRC12:05
*** raildo has joined #openstack-meeting12:05
*** ociuhandu has quit IRC12:10
*** ociuhandu has joined #openstack-meeting12:10
*** rpittau|bbl is now known as rpittau12:19
*** ociuhandu has quit IRC12:27
*** ociuhandu has joined #openstack-meeting12:28
*** yasufum has joined #openstack-meeting12:36
*** yasufum_ has joined #openstack-meeting12:39
*** yasufum has quit IRC12:40
*** yasufum_ is now known as yasufum12:40
*** lpetrut has quit IRC12:46
*** ianychoi has quit IRC12:55
*** shintaro has joined #openstack-meeting13:00
*** ttsiouts has quit IRC13:04
*** ttsiouts has joined #openstack-meeting13:13
*** shintaro has quit IRC13:21
*** ociuhandu has quit IRC13:24
*** ttsiouts has quit IRC13:32
*** redrobot has joined #openstack-meeting13:34
*** TrevorV has joined #openstack-meeting13:42
*** liuyulong has joined #openstack-meeting13:49
*** yasufum has quit IRC13:50
liuyulongtest13:59
liuyulong#link https://bugs.launchpad.net/neutron/+bug/187729614:00
openstackLaunchpad bug 1877296 in neutron "Tunnel ports are not cleaned up with several ovs agents restart" [Medium,New]14:00
liuyulong#undo14:00
*** jmlowe has left #openstack-meeting14:00
liuyulong#startmeeting neutron_l314:00
openstackMeeting started Wed May 13 14:00:32 2020 UTC and is due to finish in 60 minutes.  The chair is liuyulong. Information about MeetBot at http://wiki.debian.org/MeetBot.14:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.14:00
*** openstack changes topic to " (Meeting topic: neutron_l3)"14:00
openstackThe meeting name has been set to 'neutron_l3'14:00
liuyulong#topic Announcements14:01
*** openstack changes topic to "Announcements (Meeting topic: neutron_l3)"14:01
liuyulong#link https://releases.openstack.org/ussuri/index.html14:01
liuyulongUssuri Released today, congratulations!14:01
*** Nizars has joined #openstack-meeting14:03
*** jamesdenton has quit IRC14:03
liuyulong#link https://launchpad.net/neutron/ussuri14:05
*** ociuhandu has joined #openstack-meeting14:05
liuyulongJust confirmed the LP ussuri series14:05
liuyulongMost of the are fully fixed/implemented.14:06
liuyulong#link https://docs.openstack.org/releasenotes/neutron/ussuri.html14:06
liuyulongThis is the neutron project release note for Ussuri.14:07
liuyulongHere are something I'd like to highlight:14:07
liuyulong1) Python 2 is no longer supported, this is the community goal.14:08
haleybhi, sorry i'm late14:08
liuyulong2) The networking-ovn mechanism driver is built-in now in Neutron repo.14:08
liuyulong3) Some long-term bug are fixed now, such as the flooding issue in ovs intergration bridge.14:10
liuyulonghaleyb, hi14:10
*** ociuhandu has quit IRC14:11
liuyulongThat's all from me about the information/announcement of the project, thank you all for your contribution to the community!14:11
*** jamesdenton has joined #openstack-meeting14:11
liuyulongI have a long bug list since we have two weeks lack of discussion. So let's move on.14:12
*** ttsiouts has joined #openstack-meeting14:12
liuyulong#topic Bugs14:12
*** openstack changes topic to "Bugs (Meeting topic: neutron_l3)"14:12
*** ociuhandu has joined #openstack-meeting14:12
liuyulong#link http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014660.html14:12
liuyulong#link http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014794.html14:12
liuyulongThese are the list from bug deputy in last two weeks.14:13
liuyulongThanks to Akihiro and Miguel.14:13
liuyulongFirst one:14:13
liuyulong#link  https://bugs.launchpad.net/neutron/+bug/187609214:14
openstackLaunchpad bug 1876092 in neutron "DVR with Flat network result in ICMP reply DUP" [Undecided,In progress] - Assigned to yangjianfeng (yangjianfeng)14:14
liuyulongThis should be a nice enhancement since yangjianfeng has submitted the implementation of the Flat network support for DVR.14:15
liuyulong#link https://review.opendev.org/#/c/726044/14:16
patchbotpatch 726044 - neutron - Make DVR router support FLAT network for ovs-agent - 7 patch sets14:16
liuyulongThis is the code.14:16
liuyulongI have not test the code yet, but I'm OK with the approach.14:17
*** yasufum has joined #openstack-meeting14:17
*** ttsiouts has quit IRC14:17
liuyulongSo maybe this can be raised to the drivers meeting for more discussion because this is more like a new feature for neutron.14:18
*** ociuhandu has quit IRC14:18
liuyulongAnd another thing should be considered this may introduce a potential gap for OVN and OVS. (I don't know the status of flat network supporting for DVR in OVN deployment)14:19
liuyulongMaybe such support can be implemented for OVN as well.14:19
liuyulongOK, next one14:20
liuyulong#link https://bugs.launchpad.net/neutron/+bug/187740414:20
openstackLaunchpad bug 1877404 in neutron "Add "qos_policy_id" field to "FloatingIP" OVO" [Medium,Fix released] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)14:20
haleybyes, we could add it to the known gaps, think there's an in-tree doc14:20
liuyulonghaleyb, pong, +1, np14:20
liuyulongThis attribute have been added to floating IP。14:21
liuyulong#link https://review.opendev.org/#/c/726208/14:21
patchbotpatch 726208 - neutron - Add "qos_policy_id" field to "FloatingIP" OVO (MERGED) - 4 patch sets14:21
liuyulongWe are in a efficient team. : )14:22
haleybyes, there is a lot of work getting done :)14:22
liuyulongAnd, mostly, this attribute is for the next bug(rfe):14:22
liuyulong#link https://bugs.launchpad.net/neutron/+bug/187740814:23
openstackLaunchpad bug 1877408 in neutron "Implement FIP QoS in OVN backend" [Medium,In progress] - Assigned to Rodolfo Alonso (rodolfo-alonso-hernandez)14:23
ralonsohI need to address some comments there14:23
ralonsoh(sorry for being late)14:23
ralonsohbut seems to work14:23
ralonsoh# https://review.opendev.org/#/c/722415/14:24
patchbotpatch 722415 - neutron - [OVN] Implement floating IP QoS in OVN backend - 9 patch sets14:24
liuyulongFor the qos support for floating IP in OVN, honestly, I've seen at least 3 different implementations. : )14:24
ralonsohI know14:25
ralonsohand, IMO, the other ones do not cover all cases14:25
ralonsoh(policy update, port changes, etc)14:25
liuyulong#link https://review.opendev.org/#/c/712239/14:25
patchbotpatch 712239 - neutron - [OVN]floating IP QoS. - 5 patch sets14:25
liuyulongralonsoh, sure, I'm adding myself to the reviewers list14:26
liuyulongralonsoh, so maybe you should inform the others that there are a more proper implementation for them to stop the rotation of their works.14:27
ralonsohliuyulong, I'll do14:27
ralonsohactually I'll tell them to join one single implementation14:28
ralonsohI don't mind which one (I prefer mine)14:28
ralonsohbut just one14:28
ralonsoh(adding all of them to the coders list)14:28
*** lpetrut has joined #openstack-meeting14:29
liuyulongOK, cool14:29
liuyulongNext14:29
liuyulong#link https://bugs.launchpad.net/neutron/+bug/187744714:29
openstackLaunchpad bug 1877447 in neutron "Add TCP/UDP port forwarding extension to OVN" [Wishlist,In progress] - Assigned to Flavio Fernandes (ffernand)14:29
liuyulongFilling gaps, so +1 to me14:29
liuyulong#link https://review.opendev.org/#/c/723863/14:30
patchbotpatch 723863 - neutron - WIP (DNM): [ovn]: port forwarding functionality - 15 patch sets14:30
liuyulongThis is the patch set.14:30
liuyulongSo I don't this need to be discuss in drivers meeting, we can let it go forward simply and quickly. Because we are efficient. : )14:31
liuyulongOK, next14:32
liuyulong#link https://bugs.launchpad.net/neutron/+bug/187689814:32
openstackLaunchpad bug 1876898 in neutron "L3 agent should reports enabled extensions to the server" [Low,Fix released] - Assigned to Slawek Kaplonski (slaweq)14:32
liuyulongIt is fixed and in all stable branches now.14:33
*** ociuhandu has joined #openstack-meeting14:33
liuyulongThanks to Slawek.14:33
liuyulongLast one from the deputy's list:14:34
liuyulong#link https://bugs.launchpad.net/neutron/+bug/187730114:34
openstackLaunchpad bug 1877301 in neutron "[RFE] L3 Router support ndp proxy" [Wishlist,Confirmed]14:34
liuyulongI may say, few months ago, I bring up the IPv6 topic in the L3 meeting. And this should be talked once.14:35
haleybbefore we had prefix delegation there was a POC using proxy NDP, I wonder if I can find those patches if it matters14:36
liuyulongFound it14:37
liuyulong#link http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-09-04-14.00.log.html#l-8414:37
liuyulongand this14:38
liuyulong#link http://eavesdrop.openstack.org/meetings/neutron_l3/2019/neutron_l3.2019-09-11-14.00.log.html#l-8714:38
*** ociuhandu has quit IRC14:39
haleybthat wasn't the patch i was thinking of, i'll have to search14:40
liuyulongSo basically, this is something mentioned in this spec:14:40
liuyulong#link https://review.opendev.org/#/c/139731/2/specs/kilo/ipv6-floatingip.rst14:40
patchbotpatch 139731 - neutron-specs - Support Floatingip for IPv6 (ABANDONED) - 2 patch sets14:40
haleybyes, we're getting closer, it was Robert that sent the neutron patches14:41
*** mlavalle has joined #openstack-meeting14:42
*** dklyle has joined #openstack-meeting14:42
haleybhttps://review.opendev.org/#/c/143917/14:42
patchbotpatch 143917 - neutron - A conceptual implementation of neutron DVR (ABANDONED) - 5 patch sets14:42
liuyulongYep14:42
haleybthere is some proxy_ndp code in there.  more of a reference14:43
liuyulongSo according to the use case, this is somewhat like the floating IPv6.14:46
haleyband we've been trying to stay away from IPv6 NAT14:46
liuyulong"openstack router add ndp proxy <router_id> --address <ipv6 address>" this is command from the bug.14:47
liuyulongIt has a some similar input like floating ip.14:47
haleybright, but there is no NAT or conntrack14:49
liuyulongFloating IP is set on router's leg, and NAT. This new IPv6 address ndp_proxy is something like, IPv6 address is reachable on the router interface (Neigh Proxy).14:49
haleybyes.  and the assumption is that the upstream router will send a ND for this IPv6 address on the link where this neutron router exists14:51
haleybeg it as a /52 route going to this network, each neutron router using a /64 in it14:52
liuyulongYes, it should have a large IPv6 network plane.14:53
haleybthe one problem with this is neighbor table explosion, for example each instance could cause an entry on this upstream router, where with PD there is only one per /6414:53
haleybi know with the bgp way it would be one route per instance, but i thought routers could deal with lots of routes14:55
liuyulongI guess the bug may miss a condition which is for those unauthorized IPv6 Address, the external world should be unreachable.14:55
liuyulongSo, who have this ndp proxy, who will have access from/to the public internet.14:56
liuyulongYes, more like a floating IPv6 again.14:56
haleybtoday, without proxy NDP or PD, an instance can send IPv6 traffic to the Internet, it just might not receive it.  I don't think we should change that14:57
haleybthis is assuming the address scopes, etc are valid14:57
liuyulongBut from other perspective, for the easy use, this approach could save the operation of the physical router Prefix Delegation/ bgp. It could be simple to use for small cloud deployment.14:57
haleybyes, i think it has a use case there14:58
liuyulongSo, maybe this can be raised to drivers meeting for further discussion.14:59
liuyulongI'm neutral. : )14:59
haleybi'll add more notes to the bug15:00
liuyulongOK, we are out of time.15:00
liuyulongThank you guys for attending.15:00
liuyulong#endmeeting15:00
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"15:00
openstackMeeting ended Wed May 13 15:00:53 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)15:00
openstackMinutes:        http://eavesdrop.openstack.org/meetings/neutron_l3/2020/neutron_l3.2020-05-13-14.00.html15:00
liuyulongBye15:00
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/neutron_l3/2020/neutron_l3.2020-05-13-14.00.txt15:00
ralonsohbye15:00
openstackLog:            http://eavesdrop.openstack.org/meetings/neutron_l3/2020/neutron_l3.2020-05-13-14.00.log.html15:00
*** ociuhandu has joined #openstack-meeting15:01
*** priteau has joined #openstack-meeting15:01
*** links has quit IRC15:02
*** maciejjozefczyk has joined #openstack-meeting15:02
*** lpetrut has quit IRC15:08
*** jmasud has joined #openstack-meeting15:17
*** moguimar has joined #openstack-meeting15:22
*** belmoreira has quit IRC15:24
*** yasufum has quit IRC15:40
*** jamesdenton has quit IRC15:42
*** gyee has joined #openstack-meeting15:43
*** jamesdenton has joined #openstack-meeting15:46
*** manuvakery has joined #openstack-meeting15:56
*** maciejjozefczyk has quit IRC16:04
*** jmasud has quit IRC16:06
*** andrebeltrami has joined #openstack-meeting16:14
*** witek_ has quit IRC16:15
*** jamesdenton has quit IRC16:16
*** liuyulong has quit IRC16:17
*** rf0lc0 has joined #openstack-meeting16:20
*** rpittau is now known as rpittau|afk16:21
*** gary_perkins has quit IRC16:23
*** rfolco|rover has quit IRC16:23
*** gary_perkins has joined #openstack-meeting16:24
*** witek_ has joined #openstack-meeting16:26
*** jmasud has joined #openstack-meeting16:31
*** moguimar has quit IRC16:35
*** evrardjp has quit IRC16:36
*** evrardjp has joined #openstack-meeting16:36
*** jamesdenton has joined #openstack-meeting16:38
*** jmasud has quit IRC16:42
*** ociuhandu has quit IRC16:42
*** ociuhandu has joined #openstack-meeting16:43
*** ociuhandu has quit IRC16:43
*** ociuhandu has joined #openstack-meeting16:43
*** moguimar has joined #openstack-meeting16:50
*** rf0lc0 is now known as rfolco|rover16:56
*** moguimar has quit IRC16:58
*** jmasud has joined #openstack-meeting17:03
*** jamesdenton has quit IRC17:15
*** ociuhandu has quit IRC17:21
*** jmasud has quit IRC17:22
*** ociuhandu has joined #openstack-meeting17:23
*** jmasud has joined #openstack-meeting17:25
*** hemna_ has quit IRC17:29
*** hemna has joined #openstack-meeting17:29
*** ociuhandu has quit IRC17:33
*** ociuhandu has joined #openstack-meeting17:34
*** viks____ has quit IRC17:37
*** priteau has quit IRC17:38
*** ociuhandu has quit IRC17:40
*** ralonsoh has quit IRC17:44
*** e0ne has quit IRC17:55
*** yasufum has joined #openstack-meeting17:55
*** moguimar has joined #openstack-meeting17:56
*** jamesdenton has joined #openstack-meeting17:57
*** jmasud has quit IRC17:59
*** jamesdenton has quit IRC18:04
*** jamesdenton has joined #openstack-meeting18:09
*** jmasud has joined #openstack-meeting18:15
*** ociuhandu has joined #openstack-meeting18:32
*** ociuhandu has quit IRC18:40
*** ociuhandu has joined #openstack-meeting18:40
*** jamesdenton has quit IRC18:53
*** jmasud has quit IRC18:53
*** jamesdenton has joined #openstack-meeting18:57
*** belmoreira has joined #openstack-meeting19:05
*** maciejjozefczyk has joined #openstack-meeting19:08
*** jmasud has joined #openstack-meeting19:13
*** maciejjozefczyk has quit IRC19:25
*** ociuhandu has quit IRC19:27
*** ociuhandu has joined #openstack-meeting19:28
*** jmasud has quit IRC19:35
*** manuvakery has quit IRC19:36
*** dklyle has quit IRC19:38
*** ociuhandu has quit IRC19:39
*** ociuhandu has joined #openstack-meeting19:40
*** ociuhandu has joined #openstack-meeting19:40
*** dklyle has joined #openstack-meeting19:46
*** jmasud has joined #openstack-meeting19:47
*** ociuhandu has quit IRC19:51
*** ociuhandu has joined #openstack-meeting19:52
*** ociuhandu has quit IRC19:52
*** ociuhandu has joined #openstack-meeting19:52
*** yasufum has quit IRC19:53
*** ociuhandu has quit IRC20:02
*** ociuhandu has joined #openstack-meeting20:03
*** ociuhandu has quit IRC20:03
*** ociuhandu has joined #openstack-meeting20:03
*** jmasud has quit IRC20:07
*** belmoreira has quit IRC20:07
*** ttsiouts has joined #openstack-meeting20:13
*** ociuhandu has quit IRC20:14
*** ociuhandu has joined #openstack-meeting20:15
*** jamesdenton has quit IRC20:16
*** ociuhandu has quit IRC20:26
*** ociuhandu has joined #openstack-meeting20:27
*** ociuhandu has joined #openstack-meeting20:27
*** ttsiouts has quit IRC20:30
*** jamesdenton has joined #openstack-meeting20:30
*** vishalmanchanda has quit IRC20:31
*** ttsiouts has joined #openstack-meeting20:32
*** ociuhandu has quit IRC20:37
*** ociuhandu has joined #openstack-meeting20:38
*** ociuhandu has joined #openstack-meeting20:39
*** ociuhandu has quit IRC20:49
*** ociuhandu has joined #openstack-meeting20:50
*** ociuhandu has quit IRC20:50
*** ociuhandu has joined #openstack-meeting20:50
timburke#startmeeting swift21:00
openstackMeeting started Wed May 13 21:00:00 2020 UTC and is due to finish in 60 minutes.  The chair is timburke. Information about MeetBot at http://wiki.debian.org/MeetBot.21:00
openstackUseful Commands: #action #agreed #help #info #idea #link #topic #startvote.21:00
*** openstack changes topic to " (Meeting topic: swift)"21:00
openstackThe meeting name has been set to 'swift'21:00
timburkewho's here for the swift meeting?21:00
seongsoochoo/21:00
kota_hi21:00
tdasilvahello21:00
rledisezo/21:01
*** ociuhandu has quit IRC21:01
*** ociuhandu has joined #openstack-meeting21:02
timburkeclayg, mattoliverau, alecuyer?21:02
claygo/21:02
timburkeagenda's at https://wiki.openstack.org/wiki/Meetings/Swift21:02
timburke#topic PTG21:02
*** openstack changes topic to "PTG (Meeting topic: swift)"21:02
timburkefirst up, the usual reminder that we'll have our virtual PTG in just a few weeks21:03
timburkemattoliverau has been doing a great job of working to organize it21:03
timburkeand you guys have been great about adding topics to the etherpad21:04
timburke#link https://etherpad.opendev.org/p/swift-ptg-victoria21:04
timburkemattoliverau added some times for us to https://ethercalc.openstack.org/126u8ek25noy21:05
tdasilvais there any info anywhere in how it will actually work? I read somewhere about reserving "rooms", but I'm not sure I understand what that means21:05
*** zaitcev has joined #openstack-meeting21:05
timburkei'll make sure they get added to the etherpad21:06
timburketdasilva, that's a great question! unfortunately i've been a bit distracted, so i'm not entirely sure myself. i'll find out more and drop what i find out (and probably links to mailing list messages) in -swift about it21:07
*** ociuhandu has quit IRC21:07
tdasilvatimburke: no worries, I can go dig for it too21:07
timburkeare there any other questions about the PTG? i'm not sure i'll be able to answer them right away, but i should be able to do some research21:08
timburkeall right, on to my most recent distraction, then :-)21:10
timburke#topic object updater21:10
*** openstack changes topic to "object updater (Meeting topic: swift)"21:10
timburkeso last week i shared that one of our clusters had a *lot* of async pendings21:10
timburkegood news! we're doing much better now :-)21:11
timburkebetter news: we've written up a bunch of bugs describing things we saw going wrong21:11
claygbest news (?): patches are starting to come up -> https://review.opendev.org/#/c/727876/121:12
timburkeclayg did the lion's share of the investigation -- the core of it was21:12
patchbotpatch 727876 - swift - Breakup reclaim into batches - 1 patch set21:12
timburke#link https://bugs.launchpad.net/swift/+bug/187765121:12
openstackLaunchpad bug 1877651 in OpenStack Object Storage (swift) "Reclaim of tombstone rows is unbounded and causes LockTimeout (10s)" [Medium,In progress] - Assigned to clayg (clay-gerrard)21:12
timburkeyeah, that's the stuff :-)21:12
claygrledisez: i don't know yet how this effects my thinking about the on-disk layout for the per-container stuff21:15
timburketl;dr: after pushing our container-replicator cycle time up by about a hundred-fold, we could get our updaters from not keeping up with incoming asyncs to net-reducing them by about 11M/hr21:15
rledisezclayg: me neither. i'll make sure to go look at these patches, i'll see then if it still make sense21:16
claygwe had a node offline for few days to get a nic swapped and we have a bunch of async pendings all of the cluster spread across all the containers on that node...21:16
clayg... which is kind of different than the situation you're optimizing for21:16
timburkealong the way we noticed some workers dying off due to https://bugs.launchpad.net/swift/+bug/187792421:16
openstackLaunchpad bug 1877924 in OpenStack Object Storage (swift) "object-updater should be more tolerant of already-removed async pendings" [Undecided,In progress]21:16
*** rfolco|rover has quit IRC21:16
timburkewe did some sharding (which helped, but it could be better; see https://bugs.launchpad.net/swift/+bug/1878090)21:17
openstackLaunchpad bug 1878090 in OpenStack Object Storage (swift) "object-updater should remember redirects and proactively check whether an update should be pointing at a shard instead" [Undecided,Confirmed]21:17
timburkedid some config changes that restarted updaters cluster-wide and caused some minor heart attacks due to https://bugs.launchpad.net/swift/+bug/187805621:18
openstackLaunchpad bug 1878056 in OpenStack Object Storage (swift) "object-updater should shuffle work before making requests" [Undecided,In progress]21:18
timburkebut at the end of the day, we've got a really great system!21:18
clayg💪 ten years of sweat and tears has to be good for something - it's resilient and flexible if nothing else21:19
timburke(speaking of -- swift's birthday is in four days! 10 years running in prod!)21:20
claygi also filed a couple more https://bugs.launchpad.net/bugs/1877662 https://bugs.launchpad.net/bugs/1877663 https://bugs.launchpad.net/bugs/187766521:21
openstackLaunchpad bug 1877662 in OpenStack Object Storage (swift) "Magic number for per_diff in rsync_then_merge " [Low,New]21:21
openstackLaunchpad bug 1877663 in OpenStack Object Storage (swift) "Default db_replicator per_diff is the degenerate configuration " [Low,New]21:21
openstackLaunchpad bug 1877665 in OpenStack Object Storage (swift) "Database WAL PENDING_CAP should be configurable" [Medium,New]21:21
claygi'm not sure how many I'll get before I go back to waterfall-ec21:21
kota_congrats for the birthday21:21
rledisez10 years, awesome! is swift one of the oldest object storage software?21:21
claygi guess none of them really have anything to do with big containers just eating PUTs slowly - but we're 100% on the sharding train and trying to do more and go faster21:21
claygso the whole "500M row database" might not be a thing we keep caring about 🤔21:22
timburkerledisez, i'm not sure, good question...21:23
zaitcevSage defended the dissertation on Ceph in 2007, and it was running on Bluehost, so that's longer than Swift.21:23
timburkeclayg, we need to help out mattoliverau with getting some *real* auto-sharding going :D21:23
clayg😬21:24
rlediseztimburke: (operator-speaking) it's clearly the next big thing21:24
timburkezaitcev, otoh, https://en.wikipedia.org/wiki/Ceph_(software) lists their initial release in 201221:24
timburke*shrug*21:24
mattoliverauo/ sorry I'm very late, kinda slept in.21:24
timburkeno worries!21:25
timburkeanyway, i think that's about all i've got by way of post-mortem -- i've been playing at putting a story together with graphs and everything, i'll see what comes out of that (and what i can share :-/ this working-for-a-Big-Corp is kinda new to me)21:27
timburkeonly other thing i've got for the agenda is an update for LOSF21:27
timburkebut i think alecuyer said he wasn't going to be able to make it to discuss?21:28
rledisezso, no update this week, alecuyer was busy on an other project21:28
timburke👍21:28
timburkei know how it goes ;-)21:28
timburkeoh, last minute topic! i think i'd like to get a swiftclient release out soon. i missed the window to get versioning support out in ussuri, but it's sitting there on master -- we should publish it!21:29
*** jamesdenton has quit IRC21:30
timburkeit'll also bring some recently-landed application credential support, and i'll see what other client patches might make sense to get merged in the next week21:30
timburkeif anyone has any they want to get in, please let me know (or update the priority reviews page)21:31
timburke#topic open discussion21:31
*** openstack changes topic to "open discussion (Meeting topic: swift)"21:31
timburkeanything else we should talk about?21:31
zaitcevNothing here.21:33
mattoliverauA note from earlier, the time booking is also booking the room. So I booked the same location (different times) for the other. We're in Liberty... Now not sure what means assume it's some virtual room21:33
mattoliverau*PTG21:33
mattoliverauDamn autocorrect21:33
zaitcevWell, the Dark Data is stuck. I didn't work on it in a while. I think I addressed Romain's concerns.21:33
rledisezI'll try to go look at it while I'm on updater patches21:35
claygI don't think Romain is a great candidate for the dark data auditors - it's like part power adjustment or container sync - it won't work for everyone to their satisfaction; still useful21:36
timburkecrap, i'd meant to look at that hadn't i...21:36
zaitcevHe's the only one who might have some for this to actually find. My test cluster seems to have none at all.21:36
claygIs there a question if it should be maintained upstream?21:37
claygzaitcev: no dark data is good!21:37
zaitcevAnything that's not upstream rots. Just look how well it worked for swift3 and swauth.21:37
claygit normally takes something to fall over pretty good and be missed for awhile21:37
timburkecounter-point: container-sync21:38
claygtimburke: but rledisez is gunna look at that again, and... Gil before that?  Alistair was a container sync fan.  1space container crawler is based off container-sycn!21:39
timburke(just playing devil's advocate -- i'm all for having audit-watchers upstream, and the feature makes a lot more sense when there's at least one user)21:39
claygwell, unless it's broken; in principle I'm in favor of merging it - with docs21:40
claygsome iterative review would be a great pre-ptg project21:40
zaitcevFor me, the feature itself has value, but Sam wanted a general API. Does that need still exit, and do we have people modding auditor with additional functions?21:41
clayglord knows I'm going to need some help/review https://review.opendev.org/#/c/727876/1/test/unit/account/test_backend.py@20921:42
patchbotpatch 727876 - swift - Breakup reclaim into batches - 1 patch set21:42
timburkei seem to recall that we have at least one audit-watcher-like thing that was implemented as a whole extra auditor (alongside ZBF and ALL) -- i forget what it was for offhand, though21:43
zaitcevMath sounds like an interesting challenge, although this probably can just be tested by running a few boundary conditions through.21:44
claygzaitcev: can you add dark data and auditor watchers to https://etherpad.opendev.org/p/swift-ptg-victoria21:44
timburkehow about this as an idea, too -- i'm assigning homework! everyone find 1-5 patches they'd like to see progress on over the course of PTG week. add them to the priority reviews wiki21:46
*** raildo has quit IRC21:46
claygtimburke: can they all be patches I write or review between now and ptg week?21:47
timburkei'll put a new section in for it. list of patches, then maybe a sub-bullet for irc nicks interested21:47
mattoliverauI wonder if we could use audit watchers to let the auditors walk the filesystem and trigger events like replication, sharding, etc. And that would mean much less filesystem walking by almost all the daemons (just thinking out loud and pre coffee).21:47
rledisezmattoliverau: i like that, I kinda of think about something similar last week21:48
timburkekinda like out old in-person sticky notes-and-dots system at swift hackathons21:48
rledisezsplit the "job producer" from the "job executor"21:48
timburkemattoliverau, rledisez yes! i've thought about that too -- gets us closer to a central point for all i/o scheduling21:49
rledisezwe are taking this approach to scale container-sync => we split container-sync, one crawl the DB, one execute what need to be synchronised21:49
mattoliverauyeah, might screw up our tunings on different daemon internvals, but less io is less io21:49
claygi want this even more with generic tasks queues21:52
clayglet's do it - let's build generic task queues - single producer multi executor for all daemons - and s3 bucket policies for multi-site replication and expiry21:52
mattoliverausounds like another PTG topic :)21:53
claygor two21:54
mattoliverauI wish we had this discussion before I had to book times. I could have tried to find some more slots.21:54
clayghahaha21:54
claygwe'll make do21:54
timburkeworst case, we keep talking in irc :D21:54
timburkei'll just need to make sure i have enough coffee at home21:55
timburkeall right, let's let mattoliverau, seongsoocho, and kota_ go have breakfast21:55
timburkethank you all for coming, and thank you for working on swift!21:56
timburke#endmeeting21:56
*** openstack changes topic to "OpenStack Meetings || https://wiki.openstack.org/wiki/Meetings/"21:56
openstackMeeting ended Wed May 13 21:56:15 2020 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)21:56
openstackMinutes:        http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-05-13-21.00.html21:56
openstackMinutes (text): http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-05-13-21.00.txt21:56
openstackLog:            http://eavesdrop.openstack.org/meetings/swift/2020/swift.2020-05-13-21.00.log.html21:56
*** patchbot has left #openstack-meeting21:56
*** zaitcev has left #openstack-meeting21:56
*** slaweq has quit IRC21:57
*** slaweq has joined #openstack-meeting22:08
*** slaweq has quit IRC22:13
*** slaweq has joined #openstack-meeting22:23
*** slaweq has quit IRC22:28
*** ykatabam has joined #openstack-meeting22:38
*** TrevorV has quit IRC22:42
*** ttsiouts has quit IRC22:43
*** ttsiouts has joined #openstack-meeting22:44
*** jmasud has joined #openstack-meeting22:48
*** jmasud has quit IRC22:52
*** moguimar has quit IRC22:54
*** hongbin has joined #openstack-meeting23:05
*** Liang__ has joined #openstack-meeting23:29
*** rcernin has quit IRC23:31
*** rcernin has joined #openstack-meeting23:32
*** ykatabam has quit IRC23:34
*** ociuhandu has joined #openstack-meeting23:36
*** dmacpher has quit IRC23:47
*** ociuhandu has quit IRC23:52
*** dmacpher has joined #openstack-meeting23:56

Generated by irclog2html.py 2.17.2 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!