Monday, 2017-06-12

*** zhengyin has joined #openstack-swift01:00
*** tovin07_ has joined #openstack-swift01:02
*** chlong has quit IRC01:04
kota_good morning01:12
*** chlong has joined #openstack-swift01:19
*** zhurong has joined #openstack-swift01:57
*** mingyu_ has joined #openstack-swift02:21
*** itlinux has joined #openstack-swift02:25
*** itlinux has quit IRC02:37
*** mingyu has joined #openstack-swift02:50
*** mingyu_ has quit IRC02:51
*** tonanhngo has joined #openstack-swift03:05
*** tonanhngo has quit IRC03:06
charzHey guys, just a question for `X-delete-at`/`X-delete-after` on Swift versioning. Do you think that'll work? If not, do we have a doc to tell user for that?03:21
*** klrmn1 has quit IRC03:26
*** klrmn has joined #openstack-swift03:47
*** psachin has joined #openstack-swift03:52
*** gkadam has joined #openstack-swift04:00
*** gkadam has quit IRC04:01
*** gkadam has joined #openstack-swift04:03
*** Dinesh_Bhor has joined #openstack-swift04:10
*** mingyu has quit IRC04:20
*** Dinesh_Bhor has quit IRC04:24
*** Dinesh_Bhor has joined #openstack-swift04:25
*** zhurong has quit IRC04:26
*** zhurong has joined #openstack-swift04:35
*** catintheroof has joined #openstack-swift04:36
*** catintheroof has quit IRC04:42
*** bkopilov has joined #openstack-swift04:45
*** zhengyin has quit IRC05:04
*** zhurong has quit IRC05:06
*** zhurong has joined #openstack-swift05:07
*** chsc has joined #openstack-swift05:08
*** chsc has joined #openstack-swift05:08
*** zhengyin has joined #openstack-swift05:11
*** chsc has quit IRC05:15
*** mingyu has joined #openstack-swift05:23
*** mingyu_ has joined #openstack-swift05:32
*** mingyu has quit IRC05:33
*** ChubYann has quit IRC05:36
*** mingyu_ has quit IRC05:38
*** cshastri has joined #openstack-swift05:46
*** mingyu has joined #openstack-swift05:47
*** mingyu has quit IRC05:58
*** mingyu has joined #openstack-swift06:08
*** mingyu has quit IRC06:17
*** mingyu has joined #openstack-swift06:19
*** rcernin has joined #openstack-swift06:19
*** rcernin has quit IRC06:36
*** rcernin has joined #openstack-swift06:40
*** hseipp has joined #openstack-swift06:41
*** mingyu has quit IRC06:42
*** mingyu has joined #openstack-swift06:47
*** pcaruana has joined #openstack-swift06:52
*** klrmn has quit IRC06:52
openstackgerritChristian Schwede proposed openstack/swift master: Add commands to ring-builder to change region / zone  https://review.openstack.org/29199007:03
*** mingyu has quit IRC07:06
*** zhengyin has quit IRC07:08
*** mingyu has joined #openstack-swift07:12
*** tesseract has joined #openstack-swift07:19
*** mingyu has quit IRC07:29
*** mingyu has joined #openstack-swift07:40
*** cbartz has joined #openstack-swift07:53
*** zhengyin has joined #openstack-swift07:56
*** jaosorior has joined #openstack-swift08:16
acolesgood morning08:46
acolesrledisez: re patch 472659, I have it starred, thanks for that08:49
patchbothttps://review.openstack.org/#/c/472659/ - swift - Allow to rebuild a fragment of an expired object08:49
*** oshritf has joined #openstack-swift09:01
kota_acoles: good morning09:15
acoleskota_: o/09:15
*** winggundamth has joined #openstack-swift09:38
*** skudlik has joined #openstack-swift09:51
openstackgerritKota Tsuyuzaki proposed openstack/swift master: Follow up for affinity config per policy  https://review.openstack.org/46792009:55
kota_acoles: ^^09:56
kota_sorry, late for pushing it09:56
acoleskota_: ok thanks, I'll take a look09:58
*** geaaru has joined #openstack-swift10:03
*** mvk has quit IRC10:11
*** zhurong has quit IRC10:12
*** mariusv has joined #openstack-swift10:17
*** tovin07_ has quit IRC10:20
openstackgerritTovin Seven proposed openstack/swift master: OSprofiler in OpenStack Swift  https://review.openstack.org/46831610:24
*** kei_yama has quit IRC10:36
*** mvk has joined #openstack-swift10:39
*** zhurong has joined #openstack-swift10:39
*** zhengyin has quit IRC10:43
*** sanchitmalhotra has quit IRC10:59
*** sanchitmalhotra1 has joined #openstack-swift10:59
*** sanchitmalhotra1 is now known as sanchitmalhotra11:01
*** PavelK has joined #openstack-swift11:03
*** PavelK has quit IRC11:07
*** mingyu has quit IRC11:14
*** hseipp has quit IRC11:15
*** mingyu has joined #openstack-swift11:15
*** oshritf has quit IRC12:02
*** oshritf has joined #openstack-swift12:03
*** winggundamth has quit IRC12:07
*** oshritf has quit IRC12:13
*** zhurong has quit IRC12:16
*** oshritf has joined #openstack-swift12:18
*** winggundamth has joined #openstack-swift12:23
*** oshritf has quit IRC12:25
*** catintheroof has joined #openstack-swift12:27
*** catintheroof has quit IRC12:28
*** oshritf has joined #openstack-swift12:36
*** oshritf has quit IRC12:50
*** oshritf has joined #openstack-swift13:05
*** oshritf has quit IRC13:14
*** oshritf has joined #openstack-swift13:15
*** lucasxu has joined #openstack-swift13:29
*** oshritf has quit IRC13:33
*** klamath has joined #openstack-swift13:34
*** klamath has quit IRC13:37
*** klamath has joined #openstack-swift13:38
*** oshritf has joined #openstack-swift13:38
*** oshritf has quit IRC13:48
*** oshritf has joined #openstack-swift13:52
*** SkyRocknRoll has joined #openstack-swift13:52
*** catintheroof has joined #openstack-swift13:52
*** bob_cheesey has joined #openstack-swift14:00
*** psachin has quit IRC14:14
*** cshastri has quit IRC14:30
*** silor has joined #openstack-swift14:31
*** gkadam has quit IRC14:35
*** jlvacation is now known as jlvillal14:41
*** winggundamth has quit IRC14:42
*** chsc has joined #openstack-swift15:05
*** chsc has joined #openstack-swift15:05
*** klrmn has joined #openstack-swift15:08
*** oshritf has quit IRC15:10
*** aselius has joined #openstack-swift15:11
*** thurloat has quit IRC15:14
*** thurloat has joined #openstack-swift15:14
*** thurloat has quit IRC15:15
*** thurloat has joined #openstack-swift15:17
rledisezclayg: good morning. what do you think of setting do_listdir to False most of the time in https://github.com/openstack/swift/blob/master/swift/obj/reconstructor.py#L776-L777 like in object-replicator. right now, at every passes, everything is listed, it burns so much IO. if there is a difference between nodes, it would get recalculated later in _get_suffixes_to_sync(). i'm not sure if there could be side effect i don't think of15:27
clayg*so* much i/o!  comment on patchset 5 on patch 42840815:30
patchbothttps://review.openstack.org/#/c/428408/ - swift - Don't rehash primaries in reconstructor handoffs_o...15:30
*** pcaruana has quit IRC15:31
rledisezclayg: not the same thing. i'm in SYNC mode. before comparing the hashes, the reconstructor list all suffixes in each partitions. I tested setting to False for one run, it really makes a difference15:35
claygin the comment (not the patch), the inline diff is the same line # - do_listdir=False - anyway - yes good idea15:36
claygyes, doubled rate for me15:36
claygit's a good idea15:36
claygyou should do more reviews - we leave this low hanging fruit stuff around all the time15:37
clayglet me know when you have the patch up - i'll totally +2 it15:37
*** SkyRocknRoll has quit IRC15:45
*** rcernin has quit IRC15:45
*** honga has joined #openstack-swift15:48
*** itlinux_ has joined #openstack-swift15:49
*** skudlik has quit IRC15:52
openstackgerritGábor Antal proposed openstack/swift master: Use more specific asserts in test/unit/common/middleware  https://review.openstack.org/34277015:52
hongaHi all, I've been investigating DLO and am really confused about the Etags. I have verified that the md5s do match on the original file and, after uploading, the downloaded file as well. When I stat the manifest file, the etag is different. Where is the original MD5 stored and Why is this the case?15:55
*** SkyRocknRoll has joined #openstack-swift15:58
zaitcevWhat manifest file? DLO doesn't have one, does it now?16:02
*** gyee has joined #openstack-swift16:04
zaitcevIn any case, segmented objects do not store the Etag of the whole object. They store an Etag of a catenation of segment Etags. It's basically impossible to calculate the summary Etag. Firstly you have to do it in sequence (to maintain MD5 state), and uploads can be parallel. And you don't want to re-read a large object that was just uploaded.16:05
hongazaitcev: the manifest file containing all the individual segment info? segment sizes, hashes, etc. or am I even more confused16:05
zaitcevhonga: SLO has a manifest file and DLO just fakes it from container listings, doesn't it? I may be misremembering.16:05
*** cbartz has left #openstack-swift16:07
*** itlinux_ has quit IRC16:07
*** klrmn has quit IRC16:07
*** itlinux_ has joined #openstack-swift16:08
hongaah, you're right, DLO does not use a manifest file, the information is stored in the headers16:09
rledisezclayg: i didn't see that comment. thx for pointing it. i'll propose a patch for that, definitively16:10
hongathank you for the explanation zaitcev, makes more sense now16:10
zaitcevSo, in order to have some kind of end-to-end integrity, we put the Etags together, then bracket it with its own Etag. This is something that can be verified, although auditors only verify etags of segments, I think.16:11
zaitcevFor a long time, both DLO and SLO supplied a Etag of an empty file, d41d8cd98f00b204e9800998ecf8427e.16:12
zaitcevThen, someone came up with this idea of a summary Etag16:12
hongai see16:14
hongaso to be clear, theres not a way to see the original md5 unless the user downloads back the file? or do the possibly lengthy compute16:15
hongawe wanted a way for users to upload their files then verify md5s matched easily16:17
*** mingyu has quit IRC16:20
zaitcevThey have to calculate MD5 of segments, concatenate the digest (as text, I think), then compute MD5 of that and compare it with what PUT returns, at least in case of SLO. For the DLO, I think, the only thing you can do is to compare the returned Etags of segments with ones computed locally.16:25
hongaperfect, thanks zaitcev16:31
*** charz has quit IRC16:34
*** charz has joined #openstack-swift16:36
notmynamegood morning16:48
*** SkyRocknRoll_ has joined #openstack-swift16:49
*** SkyRocknRoll has quit IRC16:51
*** ^andrea^ has quit IRC16:58
*** FarmerJoe has joined #openstack-swift16:58
*** mingyu has joined #openstack-swift17:06
*** dja has quit IRC17:08
*** klrmn has joined #openstack-swift17:08
*** mvk has quit IRC17:09
*** dja has joined #openstack-swift17:10
*** ChubYann has joined #openstack-swift17:14
*** ^andrea^ has joined #openstack-swift17:15
*** hseipp has joined #openstack-swift17:16
*** honga has quit IRC17:18
*** honga has joined #openstack-swift17:18
*** FarmerJoe has quit IRC17:22
*** lucasxu has quit IRC17:24
*** honga has quit IRC17:25
*** oshritf has joined #openstack-swift17:27
*** hseipp has quit IRC17:29
*** itlinux_ has quit IRC17:33
*** honga has joined #openstack-swift17:34
*** chsc has quit IRC17:36
*** itlinux_ has joined #openstack-swift17:38
*** jaosorior is now known as jaosorior_away17:41
*** tesseract has quit IRC17:54
*** thurloat has quit IRC17:56
*** tonanhngo has joined #openstack-swift18:05
*** thurloat has joined #openstack-swift18:05
*** tonanhngo has joined #openstack-swift18:06
*** lucasxu has joined #openstack-swift18:08
*** oshritf has quit IRC18:16
*** geaaru has quit IRC18:20
*** thurloat has quit IRC18:21
*** chsc has joined #openstack-swift18:22
*** chsc has joined #openstack-swift18:22
notmynamehttp://lists.openstack.org/pipermail/openstack-dev/2017-June/118242.html  <-- email to -dev and -ops about the EC bad configuration change18:28
*** catinthe_ has joined #openstack-swift18:31
*** catintheroof has quit IRC18:33
redboAre we still doing the monday meetings?  I don't remember what happened.18:42
zaitcevSwift community meeting is on Wednesday, IIRC 4 p.m. your time.18:44
zaitcevEr18:44
zaitcevNo, you're in Texas. 2 p.m. then18:44
redbooh wrong channel.18:44
redbohaha18:44
*** thurloat has joined #openstack-swift18:47
tdasilvaredbo: hi! just last week I started reading a bit about golang webservices and read about routers. seems like there are a few options. using the native one or what's seems to be most commonly used is gorilla/mux18:48
tdasilvaredbo: it seems like hummingbird is using the native one, do you remember what made you guys go that direction? do you have any suggestion/tips on one over the ohter?18:48
redbotdasilva: We actually have our own little router implementation, though that might be a bit of an over-optimization.  It originally used gorilla mux - it's good but a little slow because it uses regexes for everything.  There are a ton of router implementations, but most of them can't disambiguate all of our routes.18:54
redboThe standard library router is of very limited use because it can't do variable paths.18:55
redboLike you can route to "/hello", but not "/:arbitrary"18:56
tdasilvaredbo: ah got it, https://github.com/troubling/hummingbird/blob/master/proxyserver/main.go#L4418:59
*** mingyu has quit IRC19:06
*** mingyu has joined #openstack-swift19:08
*** itlinux_ has quit IRC19:09
*** mingyu has quit IRC19:12
openstackgerritMerged openstack/swift master: remove remote qualifier from release notes branch scanning  https://review.openstack.org/46988019:18
*** itlinux has joined #openstack-swift19:21
*** itlinux has quit IRC19:22
openstackgerritMerged openstack/swift master: Fix unit test failing when swift.conf has default policy index >10  https://review.openstack.org/46106219:22
openstackgerritMerged openstack/python-swiftclient master: Fix MockHttpResponse to be more like the Real  https://review.openstack.org/44288119:25
*** itlinux_ has joined #openstack-swift19:37
*** catintheroof has joined #openstack-swift19:39
*** catinthe_ has quit IRC19:42
openstackgerritMerged openstack/swift master: Allow to configure the nameservers in cname_lookup  https://review.openstack.org/43576819:48
openstackgerritOpenStack Proposal Bot proposed openstack/python-swiftclient master: Updated from global requirements  https://review.openstack.org/8925019:50
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873619:51
*** silor has quit IRC19:52
*** thurloat has quit IRC19:58
*** itlinux_ has quit IRC21:08
*** lucasxu has quit IRC21:12
openstackgerritMerged openstack/swift master: Cleanup db replicator probetest  https://review.openstack.org/47195721:13
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements  https://review.openstack.org/8873621:20
*** itlinux has joined #openstack-swift21:44
konghi, guys, could you please review https://review.openstack.org/#/c/470158/ which could get rid of our customers' confusion when deleting objects in multi-region swift deployment.21:50
patchbotpatch 470158 - swift - Write-affinity aware object deletion21:50
notmynamekong: yeah, you're not the only one :-)21:53
kongnotmyname: heh, our customers just keep asking why they met with 404 when deleting an existing object, which leads to their own job/script failing21:55
notmynamekong: thanks for reminding us, though. I too would like to see it land soon!21:55
kongnotmyname: cheers21:55
*** mingyu has joined #openstack-swift22:13
*** mingyu has quit IRC22:17
*** thurloat has joined #openstack-swift22:30
*** klamath has quit IRC22:42
*** thurloat has quit IRC22:43
*** catintheroof has quit IRC22:47
*** thurloat has joined #openstack-swift22:49
*** itlinux has quit IRC22:50
*** kei_yama has joined #openstack-swift23:28
*** chsc has quit IRC23:40
*** catintheroof has joined #openstack-swift23:45
openstackgerritTim Burke proposed openstack/swift master: Update Global EC docs with reference to composite rings  https://review.openstack.org/46801123:58

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!