Friday, 2017-02-03

*** catintheroof has joined #openstack-swift00:04
*** notmyname has quit IRC00:11
*** notmyname has joined #openstack-swift00:13
*** ChanServ sets mode: +v notmyname00:13
claygtimburke: why is it so obvious to you that we re-order the part_info lists to put non-primary parts first - I mean if we're going to rehash all the parts anyway?  Wouldn't it make more sense to break as soon as we encountered a non-primary part if there and .handoffs_remaining?00:23
claygand at that point - should we do the sync jobs?00:23
claygidk, i'll think about it some more or maybe we can discuss it later00:24
timburkei feel like we still wanna check primary parts for misplaced frags00:24
claygi've run patch 425493 and it works great00:24
patchbot - swift - Make the reconstructor handoffs_first work (and us...00:24
openstackgerritOpenStack Proposal Bot proposed openstack/swift master: Updated from global requirements
*** catintheroof has quit IRC00:30
*** vint_bra has joined #openstack-swift00:30
timburkeclayg: now, you might be able to convince me that we should just bail and not hit any primary parts when doing handoffs_first, er, *only*. but i'm a little nervous that if you're multiple rebalances behind, you might have a lot of misplaced frags that happen to hit primary partitions (just not for that frag index)00:32
*** bkopilov_ has quit IRC00:33
claygtimburke: yeah i'm not sure on that last point - i think it's very questionable - I like it how i wrote it - i'm not sure if the follow up is needed/useful00:33
timburkemoving the stuff we *know* we need to move => good. rehashing stuff that may or may not need to move so we can find out which it is => not bad? i think?00:33
claygtimburke: yes i think we agree on the value/risk tradeoff00:33
timburkei think the follow-up is useful! do the stuff we know we need to do first!00:33
claygyou might be right00:34
claygif it's *just* like that - same work different order - it's probably fine00:34
timburkethat's my thinking -- and i feel like it's pretty low-risk00:35
claygi'm not sure how much helps to spike on handoffs then do a bunch of primary part rehashing that all get skipped as opposed to just letting it all spin00:35
clayglike I'm thinking with a concurrency of say.. 4 - with the sorting they will *all* be doing reverts - then they will *all* be doing local rehash (no-value)00:35
claygif you let it fan out you have one or two shipping parts and one or two spinning on local rehash - but then they pick up a revert - and another revert finishes - and spins on local rehash for a bit00:36
clayglike having the sync/rehash mixed in I think is quick and doesn't really cost much - you mostly end up doing reverts00:36
claygeverytime a thread of execution hits a revert it'll essentially stall compared to the quick little local rehash/noops00:37
claygso the majority of the time it might be like 3 doing revert and one is "between" - then all four are doing reverts00:37
claygidk - i'm hesitant to say front loading is good until I test it00:37
claygi'd be really curious about the clif when it finishes the first half of the list - it could be weird/strange00:38
*** tqtran has quit IRC00:59
kota_good morning01:09
claygkota_: good moring!01:10
kota_clayg: :D01:10
kota_clayg: with quick look back to irc log, something progresses for handoffs_first patch?01:10
claygkota_: i've been complaining to timburke all afternoon that I'm trying to work on patch 219165 - but he's heartless ;)01:11
kota_patch 428408 and patch 42549301:11
patchbot - swift - EC Fragment Duplication - Foundational Global EC C...01:11
patchbot - swift - Optimize reconstructor handoffs_first01:11
patchbot - swift - Make the reconstructor handoffs_first work (and us...01:11
kota_clayg:  oh, ok01:11
claygtimburke: has been contemplating the finer points of orchestrating ec rebalance ant the nature of eventlet concurrency01:11
claygit's interesting stuff - easy to get caught up in it :)01:13
kota_clayg: k, so it sounds like patch 219165 is still under reviews?01:15
patchbot - swift - EC Fragment Duplication - Foundational Global EC C...01:15
kota_could we start to adress some comments?01:15
kota_clayg: and looks still WIP01:15
patchbotpatch 427994 - swift - WIP: trying to understand global-ec01:15
*** JimCheung has joined #openstack-swift01:16
claygyes, i've made very little progress on patch 219165 today and am quite frustrated about that grrrrrr....01:18
patchbot - swift - EC Fragment Duplication - Foundational Global EC C...01:18
claygpatch 427994 is nothing - i'm trying to figure out how to offer resolutions to the issues I see as blockers for patch 21916501:18
patchbot - swift - WIP: trying to understand global-ec01:18
patchbot - swift - EC Fragment Duplication - Foundational Global EC C...01:18
kota_what's point for you? I looked quickly your comments yesterday, it seems that sorting is guilty for you?01:19
claygwhich... is really just docs - so I should probably start there - but if you were able to go through my comments you probably gleaned that I had some other tactical concerns as well01:19
kota_oh, docs too01:19
claygmainly just docs - specifically about rasing the point that an operator can use a non-default ec_duplication_factor - it's really only for benchmarks w/o the corresponding ring change01:20
kota_i see01:21
claygI think there is risk of describing how ec_duplication_factor will be used with multi-region rings leading to a deployment with very unforunate data placement - so I feel messaging is important01:21
claygi think I should start there01:21
claygthe other stuff I can probably get over myself01:21
kota_will take to consider to add soon01:22
kota_in this morning, I am going to make to get aproval for boston CFP01:22
claygI am curious if you had another reason for the sort_key besides making TestObjectControllerECDuplication(ECTestMixin work?01:23
claygor maybe more importantly how sort_key on the storage policy evolves to support affinity01:26
kota_the reason I did on TestObjectControllerECDuplication was to make sure anything works well transpatently01:27
kota_and about sort_key01:27
kota_basically, I tend to get unique n fragments for the first pile01:27
kota_get_fragment (correct method name) with ec_k pile spawn01:28
kota_that is because, w/o ec_duplication, the candidate node is just shuffled (in default, or basis on read_affinity)01:29
kota_and no care about duplication01:29
claygok, so I understood correctly that part - but I think the side-effect is not worth the trade off
patchbotpatch 219165 - swift - EC Fragment Duplication - Foundational Global EC C...01:29
kota_so that, the proxy controller can get duplicated fragments even though it knows they're duplicated01:29
claygyes, you reduce some requests - but not a lot - and the cost is all of the requests go to the same (arbitrary) nodes?01:30
claygright, proxy should be robust to duplicate fragmnts already01:30
kota_might be01:31
claygperhaps I need to rerun my experiment with affinity enabled01:31
kota_so i think you concerned the dispersion for the requests01:31
claygkota_: it *is* - acoles_ worked for months on that - last time I reviewed the change I reverted *all* the changes to proxy.controller.obj and GET still worked01:31
kota_in the last night i was thinking on I could make it something like pesion hole slots01:31
kota_with shuffled nodes01:32
claygunittests don't pass tho - because the (annoying) TestObjectControllerECDuplication(ECTestMixin expects a duplicated policy to make the *exact same* requests as a non-duplicated policy - which seems like a terrible burden to add to tests!01:32
kota_oh, really01:32
claygI think so?  I expect that in *some* conditions we expect duplicated ec frags to make different requests?01:33
claygI mean there's still a lot to learn about that01:33
kota_ya, ya01:34
claygidk, i guess I need to test it with affinity and try to write some tests01:34
claygat least in the proxy tests the Mixin is a common base from which EC and ECDuplication derive01:36
claygif we *want* to add a test that *only* runs on EC or ECDuplication there's a place to do that - if we want to add a test that runs on both - there's a place to do that01:37
claygin the contoller tests it just goes TestECDuplicationObjController(TestECObjController)01:37
claygso if you try to add a test to TestECObjController it *has* to work with the duplicated policy as well01:38
kota_Oh, oh, gotcha01:39
claygthe fact that I can remove sort_key and it still works functionally but *all* of the tests break smells like undesirable coupling of unittest to implementation (instead of test to *behavior*)01:39
kota_so let's make sort of ECMixin to define common test state and01:40
claygbut... it really is all sorta meta stuff - and could be fixed up later - I was just pointing it out01:40
kota_specific tests should exist only in each TestCase classes01:40
claygI think the *diff* would be a lot smaller without some of the Mixin refactoring - and I think we can be just as confident in the change with functional testing01:40
clayg(e.g. a inprocess functional test configuration that worked with duplicated policies would be *much* more attractive to me than the unittest Mixins)01:41
claygI would be 100% satisfied with 0% overlap between the TestECDuplicationObjController and TestECObjController01:41
claygeven if there's only 3-4 tests in TestECDuplicationObjController - there's not *that* much that's different between TestECDuplicationObjController and TestECObjController - I don't we need as many tests to cover the behavior of of the ECObjController that's acctually different between duplicated EC and not duplicated EC01:42
claygbut I could be wrong - I need to split up the tests and see01:42
claygso... lot of work for me01:42
clayggunna be a late night :D01:43
clayg... but I'll start with docs ... :P01:43
kota_clayg: kk, thanks for you inputs, I'll re-consider on that01:46
claygcool - i'll be around and working on this - so hit me up if you think of something01:46
clayggood luck!01:46
*** sams-gleb has joined #openstack-swift01:49
*** sams-gleb has quit IRC01:54
*** _JZ_ has quit IRC01:55
*** Renich has joined #openstack-swift01:57
claygit's so hard to make heads or tails with the data placement messed up02:02
claygI thought about making a "ring-zipper" that would take to rings and write out a new .ring.gz with the device list concatinated and the replica2part2dev glued together so I can rebalance region1.builder and region2.builder with 6 replicas then zip them into object-1.ring.gz02:03
clayg... just to get a ring with reasonable placement for global-ec02:04
claygso with my current bad placement and sorting_method affinity the sort_key avoids a bunch of requests02:04
*** vint_bra has quit IRC02:06
claygshoot - i got some sick kids at home - i need to commute back - i'll be on later02:07
kota_clayg: take your time02:10
claygi updated the gist with the acctual ondisk locations (and frag index #) of the object i'm testing with02:11
claygit just a worse case scenario - I have a 4+2x2 and only 3 uniq frag per region :'(02:11
kota_ok, and the # at the left is the region number?02:11
kota_lemme check02:12
claygso this adds some weight behind sort_key I think - i was testing previously with shuffle and the extra requests (because proxy hit duplicates) were pretty few02:12
clayg... and I *liked* that they got spread around to *all* the disks!02:13
claygwith affinity sorting + pathologically bad placement - sort_key the backend request count is not so good :\02:13
claygbut I don't wanna add code - just to rip it out when we have compostie rings & per-policy sorting_method02:13
*** vint_bra has joined #openstack-swift02:14
claygupdated the gist again with some counts on the totals02:21
claygwith shuffle (default) it's a bad tradeoff - with affinity it's a pretty significant win02:22
claygI'm having trouble making a jugement call wrt affinity given our situation with placement :\02:22
claygand if I had a ring-zipper (i.e. good placement) I think no sort key would still perform just as good02:23
claygso I lean toward not adding code we don't need02:23
*** adu has joined #openstack-swift02:38
*** adu has quit IRC02:42
*** vint_bra has quit IRC02:54
*** vint_bra has joined #openstack-swift02:59
*** vint_bra has quit IRC03:14
*** adu has joined #openstack-swift03:18
*** dmorita has quit IRC03:30
*** dmorita has joined #openstack-swift03:51
*** sams-gleb has joined #openstack-swift03:52
*** dmorita has quit IRC03:55
*** sams-gleb has quit IRC03:56
*** adu has quit IRC04:01
*** remix_tj has quit IRC04:14
*** klrmn has quit IRC04:16
*** remix_tj has joined #openstack-swift04:17
*** psachin has joined #openstack-swift04:20
*** chsc has joined #openstack-swift04:25
*** chsc has quit IRC04:25
*** chsc has joined #openstack-swift04:25
*** furlongm has quit IRC04:27
*** furlongm has joined #openstack-swift04:32
*** chsc has quit IRC04:43
*** adu has joined #openstack-swift04:47
*** dmorita has joined #openstack-swift04:53
*** dmorita has quit IRC04:57
*** ppai has joined #openstack-swift04:59
*** adu has quit IRC05:14
*** JimCheung has quit IRC05:29
*** sams-gleb has joined #openstack-swift05:53
*** dmorita has joined #openstack-swift05:54
*** sams-gleb has quit IRC05:58
*** dmorita has quit IRC05:58
*** cschwede has quit IRC06:03
*** cschwede has joined #openstack-swift06:05
*** geaaru has joined #openstack-swift06:08
*** gabor_antal_ has quit IRC06:10
*** mvk has quit IRC06:37
*** mvk has joined #openstack-swift06:39
*** sams-gleb has joined #openstack-swift06:54
*** sams-gleb has quit IRC06:59
*** sams-gleb has joined #openstack-swift07:09
*** tesseract has joined #openstack-swift07:20
*** rcernin has joined #openstack-swift07:26
*** dmorita has joined #openstack-swift07:27
*** silor has joined #openstack-swift07:28
claygi'm back07:31
*** dmorita has quit IRC07:32
*** silor1 has joined #openstack-swift07:34
*** silor has quit IRC07:34
*** silor1 is now known as silor07:34
*** ChubYann has quit IRC07:44
*** rledisez has joined #openstack-swift08:18
*** dmorita has joined #openstack-swift08:39
*** cbartz has joined #openstack-swift08:42
*** cbartz has quit IRC08:43
*** dmorita has quit IRC08:44
*** oshritf has joined #openstack-swift08:49
*** cbartz has joined #openstack-swift08:50
*** oshritf has quit IRC09:00
claygjrichli: maybe I could have added the other interesting condition where process C conslidate_hashes does not get called - in that case the invalidation just sits in the invalidations file and all is right in the world after the next rehash - it's really more about consliate hashes being called while rehasing suffixes09:01
*** glb1 has quit IRC09:04
*** glb1 has joined #openstack-swift09:08
*** oshritf has joined #openstack-swift09:18
*** glb1 has quit IRC09:25
*** Shashikant86 has joined #openstack-swift09:26
*** glb1 has joined #openstack-swift09:29
*** dmorita has joined #openstack-swift09:32
*** dmorita has quit IRC09:36
*** chlong has quit IRC09:38
openstackgerritChristian Schwede proposed openstack/swift master: Add support to increase object ring partition power
*** oshritf has quit IRC09:44
*** oshritf has joined #openstack-swift09:47
claygcschwede: if you're online and wrapped up on ppi - any braincells you have left would be much appreciated on patch 41978709:48
patchbot - swift - Better optimistic lock in get_hashes09:48
claygi'm sorry it grew so big - i tried to keep it small - but alister and pavel kept wanting to fix "just *one* more race condition"09:48
*** acoles has joined #openstack-swift09:55
*** ChanServ sets mode: +v acoles09:55
*** acoles has left #openstack-swift09:55
claygacoles_: ok we have to migrate all the ctype_ts stuff in the container db's to this now ->
*** oshritf has quit IRC10:00
*** Shashikant86 has quit IRC10:03
*** Shashikant86 has joined #openstack-swift10:04
*** NM has joined #openstack-swift10:09
*** NM has quit IRC10:14
*** kei_yama has quit IRC10:15
*** sams-gle_ has joined #openstack-swift10:26
*** acoles has joined #openstack-swift10:27
*** ChanServ sets mode: +v acoles10:27
*** sams-gleb has quit IRC10:28
acolesnotmyname: clayg anyone else FYI my bouncer is broken :/10:28
*** glb1 has quit IRC10:38
*** bauruine has joined #openstack-swift10:41
*** bauruine has left #openstack-swift10:41
*** dmorita has joined #openstack-swift10:43
kota_acoles: too bad :/10:45
kota_clayg: I just now finished up to read all your worth comments.10:46
kota_clayg: in the rest of time i can stay at office today, I am going to try your WIP patch on global ec follow up. basically I agree the changes but it looks still get gate failure so that I'd try to resolve10:47
kota_sorry, still am delay to circle back to your reconstructor change.10:47
*** dmorita has quit IRC10:48
kota_and timburke, sorry, probably, i cannot reach out the swift3 change for one keystone request in this week.10:48
*** sams-gle_ has quit IRC10:49
*** sams-gleb has joined #openstack-swift10:49
*** glb1 has joined #openstack-swift10:52
*** sams-gleb has quit IRC10:54
*** vint_bra has joined #openstack-swift10:54
*** Shashikant86 has quit IRC10:56
*** glb1 has quit IRC11:00
*** Shashikant86 has joined #openstack-swift11:01
*** glb1 has joined #openstack-swift11:02
*** NM has joined #openstack-swift11:02
*** vint_bra has quit IRC11:09
*** Shashikant86 has quit IRC11:18
*** Shashikant86 has joined #openstack-swift11:19
*** Shashikant86 has quit IRC11:30
*** timss has quit IRC11:30
*** timss has joined #openstack-swift11:30
*** sams-gleb has joined #openstack-swift11:45
*** acoles has left #openstack-swift11:50
*** Shashikant86 has joined #openstack-swift12:01
*** caiobrentano has joined #openstack-swift12:18
*** catintheroof has joined #openstack-swift12:20
*** catintheroof has quit IRC12:20
*** catintheroof has joined #openstack-swift12:21
*** vint_bra has joined #openstack-swift12:27
*** ppai has quit IRC12:29
*** rcernin has quit IRC12:31
*** dmorita has joined #openstack-swift12:31
acoles_cschwede: thanks for reviewing the suffix hashing patch!12:32
cschwedeacoles_: you're welcome! though most of the work was already done by you, clayg and Pavel. good progress and lots of helpful discussions in that review12:33
cschwedeso kudos for fixing all of that!12:34
openstackgerritMerged openstack/swift master: Better optimistic lock in get_hashes
cschwedeacoles_, clayg: Yay \o/ ^^12:36
*** dmorita has quit IRC12:36
acoles_cschwede: Pavel is the mastermind :)12:36
openstackgerritKota Tsuyuzaki proposed openstack/swift master: WIP: trying to understand global-ec
*** Shashikant86 has quit IRC13:06
*** PavelK has joined #openstack-swift13:21
*** hoonetorg has quit IRC13:35
*** Shashikant86 has joined #openstack-swift13:40
*** hoonetorg has joined #openstack-swift13:48
*** PavelK has quit IRC13:51
jrichliclayg: Oh, i see. So it's all about needing the updated time, regardless. Thank you for the explanation and visual.14:06
*** mvk has quit IRC14:15
*** dmorita has joined #openstack-swift14:19
*** McMurlock1 has joined #openstack-swift14:19
*** dmorita has quit IRC14:23
*** sams-gleb has quit IRC14:33
*** sams-gleb has joined #openstack-swift14:34
*** sams-gleb has quit IRC14:38
*** mvk has joined #openstack-swift14:44
*** sams-gleb has joined #openstack-swift14:52
*** mvk has quit IRC14:58
*** klamath has joined #openstack-swift15:02
*** klamath has quit IRC15:04
*** klamath has joined #openstack-swift15:04
*** gabor_antal has joined #openstack-swift15:11
*** pcaruana has quit IRC15:11
*** Shashikant86 has quit IRC15:13
*** Shashikant86 has joined #openstack-swift15:13
*** pcaruana has joined #openstack-swift15:17
*** chlong has joined #openstack-swift15:23
*** JimCheung has joined #openstack-swift15:31
*** dmorita has joined #openstack-swift15:31
*** klrmn has joined #openstack-swift15:35
*** dmorita has quit IRC15:35
*** JimCheung has quit IRC15:35
*** psachin has quit IRC15:37
clayggood morning15:42
claygkota_: thanks for everything!  I'm going to keep working on global-ec - mainly docs15:44
claygjrichli: we had so many diagrams like that back and forth between Pavel & acoles :)15:45
claygcschwede: YOU DID IT!15:45
claygall praise to Pavel!15:46
*** Shashikant86 has quit IRC15:50
*** caiobrentano has quit IRC16:03
*** caiobrentano has joined #openstack-swift16:04
*** mvk has joined #openstack-swift16:07
*** simon-AS559 has joined #openstack-swift16:07
*** chsc has joined #openstack-swift16:12
*** chsc has joined #openstack-swift16:12
*** cbartz has left #openstack-swift16:23
*** Shashikant86 has joined #openstack-swift16:24
tdasilvatimburke and others, what's your opinion on adding a dependency on iso8601 for patch 423377 ?16:30
patchbot - python-swiftclient - ISO 8601 timestamps for tempurl16:30
notmynamegood morning16:33
*** chlong has quit IRC16:36
*** klrmn has quit IRC16:46
*** chlong has joined #openstack-swift16:51
*** d0ugal is now known as d0ugal|skiing16:54
*** McMurlock1 has quit IRC16:56
*** pumaranikar has joined #openstack-swift17:00
claygtdasilva: how much effort is it to write what we need ourselves - how likely is that code to change/require-maintaince over the next five years vs. how well is the project maintained - how many open issues - open changes?17:01
clayglooks like about 200 lines of code - with an annoying multi line regex - i see some openstack people in the commit history ~1yr ago (probably py3) - ~12 commits a year and slowing down - migrated from google-code to bitbucket - author has issues open ~4yrs17:05
claygI'd say it's probably not the best library - but I like that it's small - but perhaps non-trivial enough we're not going to discover some hugely simpler way to do it ourselves?17:06
*** sams-gleb has quit IRC17:06
*** sams-gleb has joined #openstack-swift17:06
claygdo we *have* to use iso timestamps - what about supporting some other standard that python stdlib can already parse?  Don't we have other examples of turning formated data strings into timestamps?  Maybe in container listings - form post - the reconciler - I'm sure stftime has *something* that can readily support a human readable datetime to timestamp17:07
*** sams-gleb has quit IRC17:10
*** chlong has quit IRC17:16
*** mvk has quit IRC17:17
*** Shashikant86 has quit IRC17:18
*** rledisez has quit IRC17:25
*** tqtran has joined #openstack-swift17:25
notmynameI thought at one time we'd inlined/vendored some iso8601 library17:25
notmynamebut I don't see it now, so I don't know17:25
*** jcaron has left #openstack-swift17:30
*** JimCheung has joined #openstack-swift17:30
*** simon-AS559 has left #openstack-swift17:31
*** glb1 has quit IRC17:33
*** tqtran_ has joined #openstack-swift17:34
*** tqtran has quit IRC17:34
*** garyj has joined #openstack-swift17:39
*** d34dh0r53 has joined #openstack-swift17:40
*** mkrai has left #openstack-swift17:40
d34dh0r53nadeem: I'll ping you when the environment is up, will take ~3 hours17:40
*** dmorita has joined #openstack-swift17:41
timburkekota_: no worries there -- i'd put off dealing with it for so long, what difference does an extra week or two make? sorry that i've not revisited ec duplication properly yet :-(17:43
notmynameis there a specific metric that can be used to measure/detect hub starvation, or is it only a symptomatic thing (+ experience)?17:43
timburketdasilva: i'm partial toward requiring that the user be fairly exact -- who are we to decide whether they meant at the very start of the day or the very end? it gets worse as we look at larger and larger time deltas -- if we support 2017-02 is it already expired, or will it expire in a few weeks?17:46
timburkeplus, if we're strict in what we allow now, we can always loosen it later, as people actually request that it be loosened17:47
*** oshritf has joined #openstack-swift17:47
timburkeon the whole, though, i'm not in a big rush to land that -- the majority of clouds won't accept such a url parameter. if we still haven't landed it in six months, *then* i'll get antsy17:48
*** garyj has quit IRC17:55
*** EmilienM has quit IRC17:56
*** tonyb has quit IRC17:56
*** chmouel has quit IRC17:56
*** Tahvok has quit IRC17:56
*** aj701 has quit IRC17:56
*** patchbot has quit IRC17:56
*** d34dh0r53 has quit IRC17:56
*** Anticimex has quit IRC17:56
*** verto has quit IRC17:56
*** hurricanerix has quit IRC17:56
*** tonyb has joined #openstack-swift17:56
*** aj701_ has joined #openstack-swift17:56
*** Tahvok has joined #openstack-swift17:56
*** ahale has quit IRC17:56
*** agarner has quit IRC17:56
*** topol has quit IRC17:56
*** nadeem has quit IRC17:56
*** d34dh0r53 has joined #openstack-swift17:56
*** verto has joined #openstack-swift17:56
*** Anticimex has joined #openstack-swift17:56
*** chmouel has joined #openstack-swift17:56
*** neonpastor has quit IRC17:56
*** pdardeau has quit IRC17:56
*** hurricanerix has joined #openstack-swift17:56
*** topol_ has joined #openstack-swift17:56
*** nadeem has joined #openstack-swift17:56
*** pdardeau has joined #openstack-swift17:56
*** ahale has joined #openstack-swift17:57
*** EmilienM has joined #openstack-swift17:57
*** neonpastor has joined #openstack-swift17:58
*** patchbot has joined #openstack-swift17:58
*** aleph1 has joined #openstack-swift17:59
*** DuncanT has quit IRC17:59
*** DuncanT has joined #openstack-swift18:00
*** dmorita has quit IRC18:01
*** dmorita has joined #openstack-swift18:02
*** silor has quit IRC18:05
*** klrmn has joined #openstack-swift18:09
*** oshritf has quit IRC18:09
*** oshritf has joined #openstack-swift18:12
caiobrentanohi all... Why python-swiftclient always do a "PUT container" in the upload command (swift upload container object). Is it cheaper than a head method?18:15
*** McMurlock1 has joined #openstack-swift18:15
timburkecaiobrentano: it's about the same. the real trick is that it avoids HEAD-then-PUT. i might be talked into just skipping it all together, but that gets messier with large objects...18:17
clayglet's do this18:19
*** oshritf has quit IRC18:22
claygnotmyname: there's an eventlet debug option that will like raise exceptions if the hub doesn't run every so often18:22
*** _JZ_ has joined #openstack-swift18:23
claygnotmyname: it doesn't do a good job of showing the death by a thousand papercuts problem - but it can definately prove to you something is a miss18:23
caiobrentanotimburke got it. I was checking some logs, and found *A LOT* of "PUT container" requests from a client. that's why I was wondering if a HEAD req wouldn't be better18:23
caiobrentanotimburke thanks, btw! :)18:23
timburkecaiobrentano: i *certainly* wouldn't be opposed to some --skip-container-put option or something for when you *know* you've already got the container. whether we can ever make it the default behavior is tricky, though, as we don't want to break existing workflows18:24
claygtimburke: I recently found email.utils.parsedate and with patch 331369 it works better than it ever has before!18:25
patchbot - swift - Always set swift processes to use UTC18:25
claygsorry ^ tdasilva RE isoutils18:25
caiobrentanotimburke sure! an option would be better, in this case! thanks!18:29
*** chlong has joined #openstack-swift18:38
*** tesseract has quit IRC18:38
*** sams-gleb has joined #openstack-swift18:52
*** silor has joined #openstack-swift18:56
*** klamath has quit IRC18:58
*** silor1 has joined #openstack-swift19:00
*** silor has quit IRC19:02
*** silor1 is now known as silor19:02
tdasilvaclayg, timburke: hi, i'm back, reading scrollback now19:07
*** NM has quit IRC19:09
*** NM has joined #openstack-swift19:10
*** chlong has quit IRC19:11
*** caiobrentano_ has joined #openstack-swift19:13
*** vint_bra has quit IRC19:14
tdasilvatimburke: yeah, I can see the argument of starting with a more strict version of what's supported for dates (which would not require any new dependency) and then opening up more as the need arises19:14
tdasilvaclayg: if we support just a couple of date formats then it is no trouble at all, I think we just need to document well enough so that users know what is supported and what not19:15
*** caiobrentano has quit IRC19:16
tdasilvaI think I was more concerned to the fact that the patch said it supported iso8601 (at that could means a lot of different formats), so we just need to explicitly say what's expected ???19:17
*** caiobrentano_ has quit IRC19:18
*** NM has quit IRC19:18
*** vint_bra has joined #openstack-swift19:18
*** ChubYann has joined #openstack-swift19:19
*** chlong has joined #openstack-swift19:21
*** chlong has quit IRC19:27
*** NM has joined #openstack-swift19:32
*** NM has quit IRC19:32
*** oshritf has joined #openstack-swift19:33
*** NM has joined #openstack-swift19:33
*** silor1 has joined #openstack-swift19:34
*** silor has quit IRC19:36
*** silor1 is now known as silor19:36
timburketdasilva: yeah, i'm all for being explicit about what we accept. and i'm not *opposed* to the dependency -- we already pick it up by way of keystoneauth if you want anything besides v1 auth, and i really want to make that a harder dep (just haven't found the time to move everything over to Sessions)19:40
*** chlong has joined #openstack-swift19:40
timburkei think we just want to say, we support *this specific subset* of ISO 860119:40
*** NM has joined #openstack-swift19:42
tdasilvatimburke: sounds good to me19:44
openstackgerritTim Burke proposed openstack/swift master: Add Timeouts to spawn()s in replicator/reconstructor
*** NM has quit IRC19:53
timburkeclayg: ^^ follow-up on those thoughts about the lockup_timeout19:53
*** Renich has quit IRC19:55
*** McMurlock1 has quit IRC19:56
*** silor has quit IRC20:04
*** vint_bra has quit IRC20:18
*** caiobrentano has joined #openstack-swift20:21
*** NM has joined #openstack-swift20:23
*** caiobrentano has quit IRC20:24
*** NM has quit IRC20:25
*** NM has joined #openstack-swift20:26
*** NM has quit IRC20:33
*** vint_bra has joined #openstack-swift20:34
claygtimburke: I know it's only a silly simulation - but he helped a little bit to confirm my intution - i updated the gist with the punchline (some workloads can take longer depending on the resource constraints if you front-load instead of interleave jobs which bottle-neck on different fixed limits)
*** mkaminski has joined #openstack-swift21:01
*** garyj has joined #openstack-swift21:01
*** sams-gleb has quit IRC21:09
*** catintheroof has quit IRC21:11
*** vint_bra has quit IRC21:25
*** Jeffrey4l has quit IRC21:35
*** Jeffrey4l has joined #openstack-swift21:35
*** mkaminski has quit IRC21:38
claygtdasilva: wait... patch 422679 already landed - don't we already support iso timestamp parsing!?21:39
patchbot - swift - ISO 8601 timestamps for tempurl (MERGED)21:39
tdasilvaclayg: sorry, that discussion was for swift client21:39
patchbotpatch 423377 - python-swiftclient - ISO 8601 timestamps for tempurl21:39
*** oshritf has quit IRC21:42
*** chlong has quit IRC21:44
*** NM has joined #openstack-swift21:46
claygwahcphapcha!?  why do we need new depends for the client and not the server (!?)  weird21:50
*** NM has quit IRC21:51
claygEXPIRES_ISO8601_FORMAT = '%Y-%m-%dT%H:%M:%SZ' <- that's what I'm talking about!21:51
*** NM has joined #openstack-swift21:51
*** NM has quit IRC21:52
timburkeclayg: server *is* strict. client (maybe?) wants to be able to accept inputs like 2017-02-03, or even just 2017020321:52
timburkei'm skeptical -- seems like the ambiguity will necessarily lead to confusion21:52
claygtimburke: unix timestamp or go home!21:53
timburkethen why'd we do this in the first place?? no, this definitely seems like a good thing; we just need reasonable constraints on what's accepted21:54
*** topol_ is now known as topol21:54
claygmartin thought it was a good thing - I think if you're too user friendly eventually lots of people will end up using it - where it the hipser elitism in that!?  Swift is *so* mainstream.21:56
tdasilvatimburke: "clayg> timburke: unix timestamp or go home!" << I can never tell if he is being sarcastic or not ;)21:57
tdasilvatimburke: you can probably tell better than me21:58
timburketdasilva: some days, i'm not so sure...21:58
tdasilvahehehe, the guy has a window, i'm in the basement!22:03
notmynametdasilva: it's ok. we'll get through this. do you see some stairs? walk up them. slowly. open the door and find some light22:04
notmynametdasilva: meanwhile mattoliverau is thinking, "what? I don't understand. I'm sitting here on the beach"22:05
tdasilvanotmyname: that's mattoliverau:
openstackgerritTim Burke proposed openstack/swift master: Add Timeouts to spawn()s in replicator/reconstructor
*** sams-gleb has joined #openstack-swift22:10
*** sams-gleb has quit IRC22:14
*** sanchitmalhotra has quit IRC22:17
*** sanchitmalhotra has joined #openstack-swift22:17
tdasilvaok, gotta run, have a good weekend everyone22:20
tdasilva#onemore  #gopats ;)22:20
*** oshritf has joined #openstack-swift22:22
timburkeno, tdasilva! you were supposed to go +2 !22:23
patchbotpatch 337297 - swift - Add support to increase object ring partition power22:23
*** oshritf_ has joined #openstack-swift22:27
*** geaaru has quit IRC22:28
*** oshritf has quit IRC22:30
*** klamath has joined #openstack-swift22:32
*** calebb has quit IRC22:34
*** garyj has quit IRC22:36
*** mvk has joined #openstack-swift22:38
*** klamath has quit IRC22:41
*** chlong has joined #openstack-swift22:42
*** calebb has joined #openstack-swift22:47
*** dmorita has quit IRC23:01
*** dmorita has joined #openstack-swift23:11
*** oshritf_ has quit IRC23:13
*** adu has joined #openstack-swift23:46
*** adu has quit IRC23:55

Generated by 2.14.0 by Marius Gedminas - find it at!