| *** tsg has joined #openstack-swift | 00:02 | |
| *** aschhabr has joined #openstack-swift | 00:19 | |
| *** tsg has quit IRC | 00:23 | |
| *** aschhabr is now known as achhabra | 00:24 | |
| *** aix has quit IRC | 00:26 | |
| *** aix has joined #openstack-swift | 00:30 | |
| *** kota_ has joined #openstack-swift | 00:35 | |
| openstackgerrit | Kota Tsuyuzaki proposed openstack/swift: Fix the prefix of messages caputured from stderr https://review.openstack.org/164552 | 01:21 |
|---|---|---|
| *** lcurtis has joined #openstack-swift | 01:30 | |
| *** achhabra has quit IRC | 01:45 | |
| *** ktsuyuzaki has joined #openstack-swift | 01:59 | |
| *** kota_ has quit IRC | 02:01 | |
| *** kota_ has joined #openstack-swift | 02:02 | |
| *** ho has joined #openstack-swift | 02:02 | |
| *** ktsuyuzaki has quit IRC | 02:04 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:04 | |
| *** kota_ has quit IRC | 02:06 | |
| *** kota_ has joined #openstack-swift | 02:07 | |
| *** ktsuyuzaki has quit IRC | 02:08 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:09 | |
| *** kota_ has quit IRC | 02:11 | |
| *** kota_ has joined #openstack-swift | 02:14 | |
| *** ktsuyuzaki has quit IRC | 02:15 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:18 | |
| *** kota_ has quit IRC | 02:20 | |
| *** kota_ has joined #openstack-swift | 02:21 | |
| *** ktsuyuzaki has quit IRC | 02:22 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:23 | |
| *** kota_ has quit IRC | 02:25 | |
| *** kota_ has joined #openstack-swift | 02:26 | |
| *** ktsuyuzaki has quit IRC | 02:28 | |
| *** ktsuyuza_ has joined #openstack-swift | 02:28 | |
| *** kota_ has quit IRC | 02:31 | |
| *** kota_ has joined #openstack-swift | 02:31 | |
| *** ktsuyuza_ has quit IRC | 02:33 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:33 | |
| *** kota_ has quit IRC | 02:36 | |
| *** kota_ has joined #openstack-swift | 02:40 | |
| *** ktsuyuzaki has quit IRC | 02:42 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:44 | |
| *** kota_ has quit IRC | 02:46 | |
| *** kota_ has joined #openstack-swift | 02:47 | |
| *** ktsuyuzaki has quit IRC | 02:49 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:51 | |
| openstackgerrit | Thiago da Silva proposed openstack/swift: refactor PUT method https://review.openstack.org/164561 | 02:53 |
| *** kota_ has quit IRC | 02:53 | |
| *** kota_ has joined #openstack-swift | 02:54 | |
| *** ktsuyuzaki has quit IRC | 02:56 | |
| *** ktsuyuzaki has joined #openstack-swift | 02:57 | |
| *** kota_ has quit IRC | 02:58 | |
| *** kota_ has joined #openstack-swift | 02:59 | |
| *** ktsuyuzaki has quit IRC | 03:01 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:01 | |
| *** kota_ has quit IRC | 03:03 | |
| *** kota_ has joined #openstack-swift | 03:04 | |
| *** ktsuyuzaki has quit IRC | 03:06 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:06 | |
| *** kota_ has quit IRC | 03:08 | |
| *** kota_ has joined #openstack-swift | 03:09 | |
| *** ktsuyuzaki has quit IRC | 03:11 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:11 | |
| *** kota_ has quit IRC | 03:14 | |
| *** kota_ has joined #openstack-swift | 03:14 | |
| *** ktsuyuzaki has quit IRC | 03:16 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:16 | |
| *** kota_ has quit IRC | 03:18 | |
| *** kota_ has joined #openstack-swift | 03:21 | |
| *** ktsuyuzaki has quit IRC | 03:23 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:24 | |
| *** kota_ has quit IRC | 03:25 | |
| *** lcurtis has quit IRC | 03:25 | |
| *** kota_ has joined #openstack-swift | 03:26 | |
| *** ktsuyuzaki has quit IRC | 03:28 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:29 | |
| *** kota_ has quit IRC | 03:30 | |
| *** kota_ has joined #openstack-swift | 03:31 | |
| *** ktsuyuzaki has quit IRC | 03:33 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:36 | |
| *** kota_ has quit IRC | 03:37 | |
| *** kota_ has joined #openstack-swift | 03:38 | |
| *** ktsuyuzaki has quit IRC | 03:40 | |
| *** ppai has joined #openstack-swift | 03:40 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:43 | |
| *** kota_ has quit IRC | 03:45 | |
| *** kota_ has joined #openstack-swift | 03:45 | |
| openstackgerrit | Janie Richling proposed openstack/swift: WIP - Provides a simple skeleton of middleware for encryption feature. - encrypt/decrypt functions and keymaster still stubbed-out - functests fail now because some metadata is being encoded, and the WSGIContext in the Decrypter is causing issues - mass https://review.openstack.org/157907 | 03:47 |
| *** ktsuyuzaki has quit IRC | 03:47 | |
| *** ktsuyuzaki has joined #openstack-swift | 03:48 | |
| *** haomaiwang has joined #openstack-swift | 03:49 | |
| *** kota_ has quit IRC | 03:50 | |
| *** kota_ has joined #openstack-swift | 03:51 | |
| *** ktsuyuzaki has quit IRC | 03:52 | |
| *** bkopilov has quit IRC | 04:39 | |
| mattoliverau | tdasilva: I have had a go at rebasing your versions middleware on the EC branch.. I noticed your new refactor put change, so now rebased it off that. I haven't pushed it over your change (yet) and it might be better if you take a look. https://github.com/matthewoliver/swift/tree/feature/ec | 04:39 |
| mattoliverau | my EC SAIO is failing to run tests on the current HEAD of the ec branch.. once I figure out whats broken on my sysyem I'll run the unit tests on it. | 04:41 |
| *** dfg has quit IRC | 04:48 | |
| *** dfg has joined #openstack-swift | 04:48 | |
| *** bkopilov has joined #openstack-swift | 04:55 | |
| *** dfg has quit IRC | 04:57 | |
| *** dfg has joined #openstack-swift | 04:57 | |
| *** dfg has quit IRC | 05:06 | |
| *** dfg has joined #openstack-swift | 05:06 | |
| *** dfg has quit IRC | 05:15 | |
| *** aswadr has joined #openstack-swift | 05:15 | |
| *** dfg has joined #openstack-swift | 05:15 | |
| *** dfg has quit IRC | 05:24 | |
| *** dfg has joined #openstack-swift | 05:24 | |
| *** SkyRocknRoll has joined #openstack-swift | 05:33 | |
| ho | mattoliverau: hello, could you review this (https://review.openstack.org/#/c/138342/10)? If there is no conflict with ec work. | 05:35 |
| ho | mattoliverau: I have kept a test env for this but it become difficult in near future. | 05:35 |
| *** sluo_wfh has joined #openstack-swift | 05:46 | |
| openstackgerrit | Prashanth Pai proposed openstack/swift: Make object creation more atomic in Linux https://review.openstack.org/162243 | 05:52 |
| *** sluo_wfh has quit IRC | 05:55 | |
| *** kota_ has quit IRC | 06:05 | |
| *** sluo_wfh has joined #openstack-swift | 06:08 | |
| *** dfg has quit IRC | 06:09 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Imported Translations from Transifex https://review.openstack.org/164086 | 06:10 |
| *** dfg has joined #openstack-swift | 06:10 | |
| *** ppai has quit IRC | 06:16 | |
| *** nshaikh has joined #openstack-swift | 06:17 | |
| *** dfg has quit IRC | 06:18 | |
| *** dfg has joined #openstack-swift | 06:18 | |
| *** dfg has quit IRC | 06:26 | |
| *** dfg has joined #openstack-swift | 06:28 | |
| *** ppai has joined #openstack-swift | 06:30 | |
| mattoliverau | ho: I'll take a look, and good afternoon :) | 06:47 |
| *** pcaruana has quit IRC | 07:14 | |
| *** ppai has quit IRC | 07:31 | |
| *** ppai has joined #openstack-swift | 07:44 | |
| ho | mattoliverau: thanks a lot! | 08:06 |
| *** rledisez has joined #openstack-swift | 08:07 | |
| *** jordanP has joined #openstack-swift | 08:09 | |
| *** joeljwright has joined #openstack-swift | 08:18 | |
| *** geaaru has joined #openstack-swift | 08:37 | |
| *** ppai has quit IRC | 08:38 | |
| *** nellysmitt has joined #openstack-swift | 08:40 | |
| *** joeljwright has quit IRC | 08:43 | |
| *** joeljwright has joined #openstack-swift | 08:44 | |
| *** jd__ has quit IRC | 08:50 | |
| *** jd__` has joined #openstack-swift | 08:50 | |
| *** ppai has joined #openstack-swift | 08:50 | |
| *** jd__` is now known as jd__ | 08:51 | |
| *** NM has joined #openstack-swift | 09:03 | |
| *** NM has quit IRC | 09:11 | |
| *** jistr has joined #openstack-swift | 09:11 | |
| *** NM has joined #openstack-swift | 09:25 | |
| *** NM has quit IRC | 09:37 | |
| *** zackmdavis has quit IRC | 09:57 | |
| *** timburke has quit IRC | 09:57 | |
| *** timburke has joined #openstack-swift | 09:59 | |
| *** zackmdavis has joined #openstack-swift | 09:59 | |
| *** joeljwright1 has joined #openstack-swift | 10:07 | |
| *** joeljwright has quit IRC | 10:09 | |
| *** ho has quit IRC | 10:24 | |
| *** SkyRocknRoll has quit IRC | 10:34 | |
| *** ppai has quit IRC | 10:41 | |
| *** ppai has joined #openstack-swift | 10:55 | |
| *** haomaiwang has quit IRC | 11:06 | |
| *** acoles_away is now known as acoles | 11:15 | |
| *** acoles is now known as acoles_away | 11:17 | |
| *** evanjfraser_ has joined #openstack-swift | 11:21 | |
| *** aix has quit IRC | 11:21 | |
| *** evanjfraser has quit IRC | 11:22 | |
| *** me has joined #openstack-swift | 11:24 | |
| *** me is now known as Guest5916 | 11:24 | |
| *** acoles_away is now known as acoles | 11:32 | |
| *** NM has joined #openstack-swift | 11:41 | |
| *** joeljwright1 has quit IRC | 11:54 | |
| *** ppai has quit IRC | 12:00 | |
| *** ppai has joined #openstack-swift | 12:04 | |
| *** panbalag has joined #openstack-swift | 12:11 | |
| openstackgerrit | paul luse proposed openstack/swift: Merge master to feature/ec https://review.openstack.org/164667 | 12:18 |
| acoles | peluse: you around this early? ^^ | 12:19 |
| peluse | acoles, yup, howdy | 12:19 |
| *** ppai has quit IRC | 12:19 | |
| acoles | peluse: good morning - i am just fixing the merge conflict on multi frag index patch | 12:19 |
| peluse | acoles, I thought I had that fixed up for you already, no? | 12:19 |
| acoles | peluse: it failed to merge over weekend - conflicts with the ruggedize to ssync rename | 12:20 |
| peluse | acoles, ahh, OK. That shouldn't be too bad I don't think. I didn't look after that one | 12:21 |
| peluse | acoles, clayg got a probe test started for the reconstructor so I'm about to fix an issue there and start building those up | 12:21 |
| acoles | peluse: hmmm, quite a few conflict sites but i'm done | 12:22 |
| peluse | acoles, sorry about that. Didn't think to circle back since it was really just string replacement | 12:23 |
| acoles | peluse: np. just one of those ones where two lines in a dict changed alongisde each other | 12:24 |
| acoles | peluse: so about the yield_hashes change i put in that gist on Friday... | 12:25 |
| acoles | peluse: i'll stare at it again, but i thought the issue there was that HCL clears out obsolete files but with EC we may have files .data *newer* than .durable that are nt lceaned out but do need to be ignored by yield_hashes | 12:26 |
| acoles | s/lceaned/cleaned/ | 12:27 |
| * peluse looking | 12:27 | |
| peluse | acoles, ahhh, OK. Not sure its worth the logic though because it should be very rare when that happens, well at least rare, and when it does... | 12:29 |
| acoles | peluse: ? when it does ssync will puch a newer timestamp to receiver, no? | 12:30 |
| peluse | I don't think its a huge problem but need to think about that scenario just a bit | 12:30 |
| peluse | yes but its not being replciated, its being reconstructed and I think pyeclib will fail | 12:30 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 12:31 |
| peluse | there swift opaque metadata in the FA and I just kinda assume if we give it a set of FA's and one doesn't belong that it wont build corrupt data and give it back to us but need to check | 12:32 |
| acoles | peluse: hmmm, ic. I guess i imagined a scenario like (quorum -1) .data's get written, so no durable gets written, but enough .data's out there to reconstruct?? | 12:33 |
| acoles | or is that by definition a contradiction? | 12:33 |
| peluse | durable doesn't get written unless there's enough data to reconstruct | 12:34 |
| acoles | peluse: ok. | 12:34 |
| *** rdaly2 has joined #openstack-swift | 12:34 | |
| peluse | but that will be a good probe test for my todo list! | 12:34 |
| peluse | cuz I'm not sure, maybe it will just explode :) | 12:35 |
| peluse | I've got one fix in the get path on the reconstructor to do and then I'll see how easyily I can add that. Well, have one other scenario that we drew up in CA that I need to do first but then | 12:36 |
| acoles | peluse: anyway, i really don't like that logic being in yield_hashes !) and i think HCL/GOF is kinda goofy - GOF has *exactly* the info that yield_hashes wants, but HCL then obscures it, and yield_hashes iterates the files again!! I'm pretty confident I can clean all that up without needing the crappy code in that gist. | 12:38 |
| acoles | peluse: heading to lunch. catch u later. | 12:38 |
| peluse | OK, thanks | 12:44 |
| *** dencaval has joined #openstack-swift | 12:48 | |
| *** joeljwright has joined #openstack-swift | 12:49 | |
| *** openstackgerrit has quit IRC | 12:50 | |
| *** openstackgerrit has joined #openstack-swift | 12:50 | |
| *** fifieldt has joined #openstack-swift | 12:51 | |
| *** aix has joined #openstack-swift | 13:10 | |
| *** silor has joined #openstack-swift | 13:12 | |
| tdasilva | mattoliverau: I think your change looks good. You just missed a couple of methods in proxy/controllers/obj.py that can be removed: _listing_iter and _listing_pages_iter | 13:23 |
| *** mahatic has joined #openstack-swift | 13:25 | |
| *** sandywalsh has joined #openstack-swift | 13:31 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Merge 'remotes/origin/master' into feature/crypto https://review.openstack.org/164700 | 13:45 |
| *** lpabon has joined #openstack-swift | 13:50 | |
| *** o5k has joined #openstack-swift | 13:51 | |
| *** silor has quit IRC | 13:58 | |
| *** jrichli_ has joined #openstack-swift | 14:12 | |
| *** jrichli_ has quit IRC | 14:17 | |
| *** jrichli_ has joined #openstack-swift | 14:17 | |
| *** jrichli_ has quit IRC | 14:18 | |
| *** jrichli has joined #openstack-swift | 14:18 | |
| *** bkopilov has quit IRC | 14:22 | |
| *** bkopilov has joined #openstack-swift | 14:33 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Diskfile decides if durable is written based on policy https://review.openstack.org/162717 | 14:54 |
| *** mahatic has quit IRC | 14:57 | |
| *** nshaikh has quit IRC | 14:58 | |
| *** o5k has quit IRC | 15:11 | |
| *** o5k_ has joined #openstack-swift | 15:11 | |
| *** aswadr has quit IRC | 15:14 | |
| *** david-lyle_afk is now known as david-lyle | 15:15 | |
| *** mahatic has joined #openstack-swift | 15:24 | |
| *** reed has joined #openstack-swift | 15:26 | |
| openstackgerrit | Jiangmiao Gao proposed openstack/swift: Add the missing "file" keyword for print https://review.openstack.org/164740 | 15:33 |
| *** silor has joined #openstack-swift | 15:35 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: DiskFile refactoring to per-policy classes https://review.openstack.org/162249 | 15:39 |
| *** tacotuesday has joined #openstack-swift | 15:40 | |
| tacotuesday | if I have staticweb configured, can I hit the files in my swift container directly from my browser? | 15:41 |
| acoles | peluse: clayg: torgomatic : Paul updated me on discussions you had re diskfile. I had worked this (^^ 162249) up in parallel (just couldn't rebase to feature/ec on Friday) so pushing anyway fwiw | 15:42 |
| *** o5k_ is now known as o5k | 15:49 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: DiskFile refactoring to per-policy classes https://review.openstack.org/162249 | 15:54 |
| *** gyee has joined #openstack-swift | 15:59 | |
| *** openstackgerrit has quit IRC | 16:11 | |
| *** openstackgerrit has joined #openstack-swift | 16:12 | |
| *** aix has quit IRC | 16:16 | |
| openstackgerrit | Merged openstack/swift: Imported Translations from Transifex https://review.openstack.org/164086 | 16:23 |
| notmyname | good morning | 16:24 |
| *** nellysmitt has quit IRC | 16:30 | |
| *** Fin1te has joined #openstack-swift | 16:35 | |
| clayg | tdasilva: nice, how were you able to get https://review.openstack.org/#/c/164561/1 going w/o the versioned writes work!? | 16:35 |
| *** sandywalsh has quit IRC | 16:36 | |
| tdasilva | clayg: I just left the old object versioning code there. mattoliverau said he was taking a stab at rebasing obj. versioning patch to ec branch too | 16:36 |
| clayg | whoa! thanks mattoliverau | 16:37 |
| *** tsg has joined #openstack-swift | 16:37 | |
| tdasilva | notmyname, clayg: now, I did not attempt to rebase patch 159610 yet. I noticed that in the EC branch there are two other controller classes already, so we would need to do work to separate ec code from replica code | 16:38 |
| patchbot | tdasilva: https://review.openstack.org/#/c/159610/ | 16:38 |
| tdasilva | or just leave it in the Base class as it is now | 16:39 |
| *** Nadeem has joined #openstack-swift | 16:42 | |
| clayg | acoles: i think we're about to run into converging patches again -> https://review.openstack.org/#/c/164380/ | 16:46 |
| acoles | clayg: :( | 16:47 |
| clayg | acoles: I wanted to see what how little the existing stuff would have to change to allow for a per policy diskfile to get swapped out | 16:47 |
| clayg | acoles: no it's good! | 16:47 |
| clayg | acoles: last time when we did the policies everywhere stuff it totally worked out! | 16:48 |
| acoles | clayg: oh, so you did the routing part but not yet split out an ECDiskFile | 16:50 |
| acoles | ? | 16:50 |
| clayg | yeah! | 16:50 |
| clayg | I mean I registered it - but haven't implemented any code in there | 16:50 |
| acoles | clayg: oh, ok, i'll upgrade to a :) then | 16:50 |
| clayg | although I was probably going to have a run at *not* changing any of the existing implementation | 16:51 |
| clayg | acoles: like try and what it would look at to leave the existing stuff totally safe and sound and just *add* ec changes under the router | 16:52 |
| acoles | clayg: yeah, i realised over weekend that having an *unchanged* ReplDiskFile would be really attractive to reviewers | 16:52 |
| clayg | acoles: mostly just to see where we'd have to split things up - then a real refactor into a common base could even happen after the ec merge | 16:52 |
| clayg | acoles: that's my line of thinking | 16:52 |
| clayg | like even if it' menas things aren't as dry as they could be | 16:53 |
| clayg | maybe | 16:53 |
| clayg | idk | 16:53 |
| acoles | clayg: i just like OO :| | 16:53 |
| clayg | don't be ashamed | 16:53 |
| clayg | i like me a good function here and there - but if the only caller of the function is a method inside the reference implemeantation of a class that you know is an iterface - *maybe* that function might wanna be over-ridden at some point :P | 16:54 |
| *** openstackgerrit has quit IRC | 16:54 | |
| acoles | clayg: tbh i was surpised how simply it all dropped out, and a lot of my line count is commentary | 16:54 |
| *** openstackgerrit has joined #openstack-swift | 16:55 | |
| clayg | acoles: ok, so you're probably 'bout done for today - you want me to try to rebase you per-policy diskfile on the router? | 16:55 |
| clayg | acoles: or should i leave it to you and work on proxy stuff? | 16:55 |
| clayg | acoles: i'm not even sure how long it takes to look at the router and see what it's doing - it seemed strightforward when I hacked it out | 16:55 |
| acoles | clayg: i have an hour or so, just finishing up a tweak on the multi frag patch to keep mattoliverau happy (hi mattoliverau :) | 16:56 |
| clayg | acoles: yeah i'm worried about that patch | 16:56 |
| clayg | acoles: obviously all that functionality we want | 16:56 |
| clayg | acoles: but the implementation was sort of what inspired all the per-policy diskfile discussion - and i'm not sure how much of it survives that refactoring | 16:56 |
| clayg | acoles: e.g. is the suffix hash classes | 16:57 |
| acoles | clayg: uhuh. well after peluse got me up to speed earlier i thought we were sticking with the multifrag patch for now and the per-policy-diskfile ideas were on ince | 16:58 |
| acoles | s/ince/ice/ | 16:58 |
| clayg | acoles: I think the hashes/suffix implemenation is a good candiate for OO (it has a state and an interface, which behaviors that are seperate from implemenation) - but again doing that for the EC diskfile doesn't mean the replicated diksfile *has* to change - althought it might *want* to after it sees how obvious it is! | 16:58 |
| *** nellysmitt has joined #openstack-swift | 16:58 | |
| acoles | clayg: have to confess i haven't paid too much attention to the hashes/suffix part yet (sorry) | 16:59 |
| clayg | acoles: well... yeah - he needs to make progress on the reconstructor - i'm just saying most of that change get's gobbled by an ECDiskfile | 16:59 |
| *** Fin1te has quit IRC | 16:59 | |
| acoles | clayg: yup | 16:59 |
| clayg | oh, i thought that was most of mattoliverau's complaints | 16:59 |
| acoles | clayg: mattoliverau pointed out a possible ValueError I ought to catch when resolving frag index | 16:59 |
| clayg | so we can merge it "wrong" and the clean it back up as the policy stuff goes in - or we can leave it as is and try to let paul iterate on the dependent change | 17:00 |
| acoles | str(fi) could ValueError if there's garbage in the fir | 17:00 |
| acoles | fir??? s/fir/dir/ !! | 17:00 |
| clayg | acoles: oh, he also said some stuff about how a bunch of public diskfile functions that manipulate the suffix state should be instance methods - and he's was spot on about that (at least I agreed after having hacked on it) | 17:01 |
| *** ChanServ changes topic to "Review Dashboard: http://goo.gl/uRzLBX | Overview Dashboard: http://goo.gl/2By1qv | Priority Reviews: https://wiki.openstack.org/wiki/Swift/PriorityReviews | Ideas: https://wiki.openstack.org/wiki/Swift/ideas | Logs: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/" | 17:01 | |
| acoles | clayg: so far i have kept both up to date i.e. the multi-frag patch works (i believe) and the per-policy stuff is there waitng in the wings | 17:01 |
| clayg | but ultimately I decided no matter how I'd like to clean up the suffix hashes interface I'd rather *review* ad EC diskfile that didn't ess with any of it | 17:01 |
| acoles | clayg: ah, ok, like i said, i need to catch up on the suffix stuff, didn't spot mattoliverau comments there | 17:02 |
| acoles | (well, i spotted but didn;t dwell on them :?) | 17:02 |
| clayg | acoles: so the multi-frag change (setting asside suffix stuff) is really all about get_on_disk_files? | 17:02 |
| clayg | acoles: ok, well i guess I got hung up on the suffix stuff - so it's good we've got eyes on different things - but I really don't know at what point to merge that change | 17:03 |
| acoles | clayg: for my part, mostly get_on_disk_files, hash_cleanup_listdir, and some cleanup i think | 17:03 |
| clayg | acoles: ok ok - right on | 17:04 |
| acoles | clayg: i have one outstanding query in yield_hashes | 17:04 |
| clayg | acoles: so in your work on the diskfile stuff did you end up dropping those diskfile functions into the DiskFile class(es)? | 17:04 |
| acoles | clayg: yes pretty much, get_ondisk_files goes into the classes (module unc is still there for unit tests but thats just lack of time on my part to move it out) | 17:05 |
| *** mahatic has quit IRC | 17:05 | |
| acoles | hash_cleanup_listdir just routes through to a class method based on policy (could be refactored, i was 50/50 on that one) | 17:06 |
| *** mahatic has joined #openstack-swift | 17:06 | |
| clayg | acoles: no it's fine - no one wants to review all that test churn - unless it's part of a targeted refactor - which shouldn't stand in the way of the EC patches IMHO | 17:06 |
| acoles | clayg: glad to hear that! | 17:06 |
| clayg | acoles: so I just change all of the DiskFile(Manager|Writer|Reader) classes to call self.blah_blah instead of the module methods directly by some python tomfollery where I "attach" the module level functions to the class. | 17:07 |
| clayg | strip_self in obj.diskfile | 17:07 |
| *** dmorita has joined #openstack-swift | 17:08 | |
| clayg | means that I can overwrite the functions with methods in the EC diskfile - but there's no code/test churn in diskfile for the repl policy diskfile | 17:08 |
| clayg | now I just need to figure out what behaviors I acctully need to get into the EC diskfile? | 17:08 |
| acoles | ic. clever. | 17:08 |
| acoles | clayg: here's a suggestion... | 17:09 |
| acoles | clayg: i'll rebase my patch on yours. i'll revert to unmodified Repl diskfile and put all the 'new' impl into ECDiskFile | 17:09 |
| clayg | acoles: :) | 17:09 |
| clayg | it works best when it's your idea ;) | 17:09 |
| *** rledisez has quit IRC | 17:09 | |
| acoles | clayg: then as and when we like the new impl, we roll it into repl diskfile | 17:10 |
| clayg | acoles: yeah totally! but like that maybe doesn't even have to happen as part of the EC review? maybe? Like if we say "here let's merge this crazy new stuff and not change any of the old stuff - then after the release we'll go refactoring crazy town everywhere" | 17:11 |
| acoles | clayg: well, i like the idea of knowing that nothing changed with classic diskfile while we get EC landed | 17:11 |
| clayg | acoles: yeah at as long as we have those cleanups "in the wings" - we may *want* to not "cleanup" a bunch of critical code paths right before a named openstack release? | 17:12 |
| clayg | at least that's my misguided logic | 17:12 |
| clayg | anyway - i'd love to see some of the acctual get_on_disk_files and hash_cleanup_listdir changes inside the crazy routed diskfile manager nested class thing I cooked up | 17:12 |
| acoles | clayg: "clean up".almostEquals("f*** up") ?? | 17:12 |
| clayg | acoles: I mean - that's the joke | 17:13 |
| acoles | clayg: so , are you thinking propose your router patch to master? | 17:13 |
| clayg | "hey this looks like it's just refactoring - what does this patch *do*" - "oh you know, hopefully nothing, theere's probably a few new bugs tho" | 17:13 |
| clayg | acoles: if we can prove it out on feature/ec - i think it'd be a good review in the patch chain to master - "here | 17:14 |
| clayg | "here's what we need to change in ReplDiskfile so we can subclass it and override what we mostly only what we need to" | 17:14 |
| acoles | clayg: ok makes sense | 17:15 |
| clayg | "here's where we add in EC diskfile behaviors and notice *nothing changes in the repl diskfile*" (accept what we already changed to make it so we can hide ec specific behaviors in a ec diskfile) | 17:15 |
| clayg | i man we *could* probably even get a way with less repl diskfile changes - but it'd mean more duplicated code - like reimplementing all of the methods that call out to module level functions just so we can call instance methods in their place | 17:16 |
| clayg | i had convinced myself that it "only* takes three classes to implement a subclass of the existing implementation - but i forgot about the manager - so really it's *4* | 17:18 |
| acoles | clayg: ok, i'll take a crack at that rebase mine on yours with untouched diskfile, and see if i get there before end of day | 17:18 |
| clayg | acoles: ok - well i was going to suggest if you wanna take a stab at that tomorrow (my tonight) even - I could work on proxy stuff today - or I can try to carry on your good work in that area - i'll follow your lead | 17:18 |
| clayg | jrichli: did notmyname trick you into working on plumbing a non-default policy option through to all the container create methods in functests? | 17:19 |
| *** erlon has joined #openstack-swift | 17:19 | |
| acoles | clayg: ok, well i'll make a start but it may well end up being my tomorrow, hopefully something for you to laugh at on your tuesday morning :D | 17:19 |
| * notmyname *tricky*tricky* | 17:19 | |
| clayg | i've also got the container-sync and account-reaper stuff that is pretty cheap | 17:20 |
| dencaval | clayg acoles notmyname Hey core guys, could you please take a look on bp/sorting-method-storage-policy https://review.openstack.org/#/c/160877/ | 17:20 |
| notmyname | dencaval: I'll try to take a look later. but know that most of us are focusing on EC right now. huge amount of work there to get done before kilo | 17:21 |
| notmyname | dencaval: and thanks for working on it! I think it sounds like a really cool idea! | 17:22 |
| dencaval | notmyname ok, I know you are busy. I hope you have the priority things done soon, thanks | 17:23 |
| notmyname | dencaval: and I just added your patch to the bottom of https://wiki.openstack.org/wiki/Swift/PriorityReviews (because I want to keep an eye on it) | 17:24 |
| acoles | clayg: notmyname : jrichli : re func test and policies - i looked for an easy path to that when i did patch 159205 | 17:24 |
| patchbot | acoles: https://review.openstack.org/#/c/159205/ | 17:24 |
| acoles | but didn't see one ! | 17:24 |
| *** o5k_ has joined #openstack-swift | 17:24 | |
| dencaval | notmyname oh pretty nice, thanks | 17:25 |
| acoles | notmyname: i was wondernig about adding an option to swiftclient to specify 'default' policy for client to use when creating containers. Then just setting the swiftclient env var before running tests? | 17:26 |
| *** o5k has quit IRC | 17:26 | |
| notmyname | acoles: so plumb it through swiftclient instead of the tests themselves? interesting | 17:26 |
| notmyname | it would work | 17:26 |
| acoles | notmyname: if i'm right, its a *awful* lot simpler and *maintainable* | 17:27 |
| notmyname | and clayg was thinking of an env var for the setting too (but IIRC for the tests instead of swiftclient) | 17:27 |
| acoles | notmyname: yup. env var. so 'ST_POLICY=banana && ./.functests' | 17:28 |
| notmyname | acoles: ya. at the least, it will be a lot faster to do. jrichli^^ | 17:28 |
| acoles | notmyname: it could actually be a useful feature for swiftclient as well, to have your chosen policy set in env vars | 17:29 |
| notmyname | yup | 17:29 |
| *** zaitcev has joined #openstack-swift | 17:29 | |
| *** ChanServ sets mode: +v zaitcev | 17:29 | |
| clayg | acoles: notmyname: ok, well how about we monkey patch swiftclients put_container in the functests *first* and see if it would work? | 17:34 |
| clayg | i think in tests.py (the Env tests) it'd be easy enough to patch the "Container" class | 17:34 |
| clayg | in the other tests (the retry(method, **kwargs) tests) you'll find that that do their on http thing - so swiftclient just won't help | 17:35 |
| jrichli | acoles: ok, thx. I will take a look. | 17:35 |
| openstackgerrit | Merged openstack/swift: Merge master to feature/ec https://review.openstack.org/164667 | 17:35 |
| clayg | i'm heading in' bbiab | 17:35 |
| openstackgerrit | Merged openstack/swift: Merge 'remotes/origin/master' into feature/crypto https://review.openstack.org/164700 | 17:36 |
| acoles | jrichli: see wot clayg just said too | 17:36 |
| jrichli | acoles: yes. I just got back from lunch and am reading the exchange. :-) | 17:36 |
| acoles | clayg: jrichli : oh yeah, not all tests use swiftclient | 17:37 |
| jrichli | clayg: it wasn't a trick :-) but yes, I would like to help. | 17:39 |
| *** jordanP has quit IRC | 17:44 | |
| *** jistr has quit IRC | 17:51 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 18:02 |
| *** madgriff_ has joined #openstack-swift | 18:03 | |
| jrichli | clayg: checking my understanding: should I look at modifying class Container in swift_test_client.py so that it consumes an ENV var for policy? | 18:03 |
| openstackgerrit | Alistair Coles proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 18:05 |
| acoles | peluse: clayg: ^^ think I am done tinkering with this one, apart from fixing anything review uncovers | 18:06 |
| *** tacotuesday has quit IRC | 18:06 | |
| acoles | peluse: i just left the TODO note under yield_hashes as a reminder to revisit | 18:06 |
| peluse | acoles, awesome | 18:07 |
| peluse | clayg, provided it looks good on review are you cool landing it and then refactoring separately? | 18:08 |
| *** welldannit has joined #openstack-swift | 18:16 | |
| notmyname | funny https://twitter.com/jkmcnk/status/577523694680940545 | 18:18 |
| *** jh_ has joined #openstack-swift | 18:27 | |
| *** Fin1te has joined #openstack-swift | 18:28 | |
| *** jh_ has left #openstack-swift | 18:28 | |
| *** tsg has quit IRC | 18:35 | |
| *** geaaru has quit IRC | 18:37 | |
| jrichli | Is he claiming it is crappy code because there is one (ok two) bugs? At least he is appreciating that things still work | 18:38 |
| *** welldannit has quit IRC | 18:38 | |
| clayg | notmyname: i guess maybe we could land something to fix https://bugs.launchpad.net/swift/+bug/1424108 | 18:39 |
| openstack | Launchpad bug 1424108 in OpenStack Object Storage (swift) "KeyError: 'storage_policy_index' in _really_merge_items swift/account/backend.py" [Undecided,New] | 18:39 |
| clayg | adding the setdefault thing seems pretty sane | 18:39 |
| dmsimard | cschwede: Just came across https://github.com/enovance/swift-ring-tool | 18:40 |
| dmsimard | cschwede: Nice work but that sounds way too dangerous :p | 18:40 |
| peluse | clayg, FYI got the adjacent + far node tets added to probe and reconstructor updated to work with them... will post soon | 18:40 |
| clayg | cschwede: NICE! | 18:40 |
| clayg | peluse: were you able to steal what you needed from the proxy GET path - or did you go the internal client path? | 18:41 |
| clayg | peluse: i'm torn on landing the multi-frag index stuff - i feel like there's good momentum on the per-policy df - what's the downside of just waiting and getting it landed on feature/ec how we're planning on doing the merge to master? | 18:42 |
| peluse | clayg, actually neither... added just a small amount of straightforward error handling and a simple iter chain for 404s. Will want you to take a look for sure, seemed simpler than either other solution | 18:42 |
| clayg | peluse: NICE! | 18:43 |
| * clayg loves simple | 18:43 | |
| * clayg not sure if torgomatic believes clayg knows the meaning of the word | 18:43 | |
| peluse | clayg, I guess the only downside is that I'm dependent on MFI for recon to work at all. If you get another solution that I can munge over to as a D then that works too, wouldn't want to wait till the last minute though of course... | 18:43 |
| notmyname | lol | 18:43 |
| peluse | clayg, but I'm on board with that route if that's how you want to go! Will keep using MFI in the meantime | 18:44 |
| clayg | peluse: +1 you're doing god's work on the reconstructor | 18:45 |
| * peluse ponders the learning's of working on MFI.... | 18:45 | |
| clayg | peluse: when you say "recon" you're talking about the reconstructor - not "swift-recon" | 18:45 |
| peluse | yeah, I shouldn't shorten it like that, maybe ECrecon instead | 18:45 |
| peluse | reconstructor is such a long word | 18:45 |
| clayg | yeah I dig on it - i was sure somewhere there was a comment about "recon" and I was totally thinking you ment swift-recon | 18:45 |
| * peluse heads to grab some grub | 18:46 | |
| clayg | tdasilva: let me know what I can to to help with PUT/versioned_writes/copy on master/feature-ec | 18:47 |
| tdasilva | clayg: I guess first I'd like to figure out a plan...do we want to move everything to EC branch so that it is easier later to merge back to master | 18:48 |
| tdasilva | ? | 18:48 |
| clayg | tdasilva: that is *a* plan - is it the *best* plan? I have no idea :'( | 18:49 |
| tdasilva | ok, well, we have both the refactoring there and versioning is half-way done | 18:49 |
| tdasilva | I can jump on copy and see if we can make progress on that | 18:49 |
| clayg | tdasilva: so you have the PUT method extraction change up against feature/ec with tests passing? | 18:52 |
| tdasilva | clayg: yes | 18:52 |
| clayg | tdasilva: and versioned_writes too!? | 18:53 |
| clayg | tdasilva: ok I'll definately review https://review.openstack.org/#/c/164561/ | 18:53 |
| tdasilva | clayg: I tried to make it simple and really just split up the code to small functions | 18:53 |
| tdasilva | no new code added | 18:53 |
| clayg | tdasilva: hell yeah! | 18:53 |
| tdasilva | so it should be simple to code review | 18:53 |
| tdasilva | clayg: obj. versioning is here: https://github.com/matthewoliver/swift/tree/feature/ec | 18:54 |
| tdasilva | you will noticed that it's already based on the refactor patch (164561) - thanks to mattoliverau | 18:55 |
| clayg | wtg tdasilva and mattoliverau! | 18:55 |
| clayg | oh... but he didn't push it to gerrit - weak | 18:55 |
| clayg | :P | 18:55 |
| tdasilva | lol...not yet, but he should be online soon, I just pointed out a couple more methods he can remove and he should be all set | 18:56 |
| clayg | how is "Group-based-policy" an openstack project? is "Group-based-policy" a *service*???? | 18:56 |
| tdasilva | clayg: could probably use your help with something thou... | 18:56 |
| clayg | tdasilva: name it! | 18:57 |
| tdasilva | clayg: you dropped a comment on obj. versioning about placement of filter in the pipeline (before or after slo/dlo) and that got me thinking about some crazy use cases...unfortunately we don't have have a lot of cross-feature tests, so I think we might need to start thinking about that | 18:58 |
| *** welldannit has joined #openstack-swift | 18:59 | |
| clayg | fuuuuuu | 19:00 |
| clayg | i think we have *some* cross-feature tests - but not a lot of tests for *versioning* with other features? | 19:00 |
| tdasilva | clayg: for example: if you version slo manifest and some segments, then receive a multipart-manifest=delete request | 19:00 |
| clayg | sounds terrible | 19:01 |
| tdasilva | clayg: yes, i meant especifically slo/dlo/versioning cross feature, sorry | 19:01 |
| *** acoles is now known as acoles_away | 19:01 | |
| *** tsg_ has joined #openstack-swift | 19:03 | |
| clayg | tdasilva: yup - ok well - that's a disaster - so... i guess "help write some tests for dlo/slo * versioning and try to document existing behavior" would be a good starting point | 19:04 |
| clayg | so maybe the versioned middleware and copy extraction isn't in the cards in the near? that's fairly disappointing. Maybe we'll need to touch base with matt. | 19:05 |
| tdasilva | clayg: yeah, I think we need to sync | 19:08 |
| openstackgerrit | paul luse proposed openstack/swift: Erasure Code Reconstructor https://review.openstack.org/131872 | 19:08 |
| openstackgerrit | paul luse proposed openstack/swift: Multiple fragment Archive Index Support https://review.openstack.org/159637 | 19:08 |
| tdasilva | notmyname, clayg: is the current plan to land all the patches there are up for review on ec and merge them to master in two weeks? | 19:08 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: Refactor server side copy as middleware https://review.openstack.org/156923 | 19:09 |
| *** mahatic has quit IRC | 19:12 | |
| *** silor1 has joined #openstack-swift | 19:12 | |
| *** silor has quit IRC | 19:13 | |
| *** Fin1te has quit IRC | 19:14 | |
| openstackgerrit | paul luse proposed openstack/swift: wip: ec probe test https://review.openstack.org/164291 | 19:17 |
| notmyname | tdasilva: ya, like with storage policies there will be refactor to get it into a smaller series of commits and we'll get that as a merge commit again | 19:18 |
| *** Fin1te has joined #openstack-swift | 19:21 | |
| *** tsg_ has quit IRC | 19:24 | |
| *** tab____ has joined #openstack-swift | 19:27 | |
| *** Fin1te has quit IRC | 19:28 | |
| *** Fin1te has joined #openstack-swift | 19:30 | |
| *** rdaly2 has quit IRC | 19:32 | |
| tdasilva | notmyname: and is kilo RC going to be cut right after? I guess my question is: would we want new work done in master to be merged before tagging RC? trying to figure out if it makes sense to move obj. versioning and copy to ec or just keep working on master | 19:35 |
| notmyname | tdasilva: yes, current plan is to cut the RC very soon after the merge finishes. mostly to catch any last thing that needs to land | 19:38 |
| notmyname | eg bugfixes or security issues | 19:38 |
| tdasilva | notmyname: ok, thanks | 19:40 |
| notmyname | tdasilva: if there's a ton of merge conflicts for object versioning, then doing it on ec (or at least the proxy refactors) would more easily allow it to land on master in kilo | 19:41 |
| *** dmorita has quit IRC | 19:42 | |
| tdasilva | notmyname: agreed... | 19:43 |
| notmyname | and from what peluse/clayg tell me, it would actually help the merge to master anyway since it shows that the proxy is better | 19:44 |
| clayg | well - that's just like my *opinion* man | 19:45 |
| *** os1 has joined #openstack-swift | 19:48 | |
| os1 | Hi | 19:49 |
| os1 | quick question | 19:49 |
| os1 | What's the purpose of account.lock, container.lock, and object.lock in the /var/lock directory? | 19:49 |
| os1 | And are they always supposed to be there? | 19:49 |
| clayg | what the crap? those must be for recon. | 19:50 |
| clayg | well... s/must/might be/ | 19:51 |
| *** dmorita has joined #openstack-swift | 19:51 | |
| os1 | Yes, they are for recon. | 19:53 |
| clayg | see! s/might be/must/ ! | 19:53 |
| os1 | But why does recon need them, and does anything else need them as well? | 19:53 |
| clayg | is it just using them for the read modify write on the recon cache files? | 19:54 |
| *** nellysmitt has quit IRC | 19:54 | |
| notmyname | or for rsync | 19:55 |
| clayg | oh... maybe | 19:55 |
| notmyname | I'd expect account.lock to be for rsync. but there are a few other places that swift might drop a ".lock" file | 19:56 |
| clayg | notmyname: yup right there in rsync.conf - doh! | 19:57 |
| notmyname | os1: so that. it's an rsync artifact. ie replication | 19:57 |
| os1 | Right - so whether it's for rsync or recon, either way, why do they need those lock files? | 19:59 |
| *** tsg_ has joined #openstack-swift | 20:01 | |
| openstackgerrit | Merged openstack/swift: Fix ContainerBroker to use policy-0 in default https://review.openstack.org/158219 | 20:02 |
| clayg | os1: well rsync needs them for max connections | 20:02 |
| clayg | os1: why do you care? are they bumming you out somehow? | 20:03 |
| os1 | Sometimes they are not installed with the proper unix user permissions. | 20:06 |
| os1 | swift-recon outputs errors because of permissions issues to those files. | 20:06 |
| clayg | oh really... hrmm... how strange | 20:06 |
| os1 | Also, when you said that rsync needs them for max connections, I can see that, but why would a process need a lock file for that? | 20:07 |
| clayg | do you have a traceback - given notmyname's comments i suppose I'm not sure why recon is looking at them - i think it does some locking - but maybe on the swift_recon_path files instead | 20:07 |
| clayg | why would a process need.... I guess it's ipc - maybe fork-on-connect | 20:08 |
| peluse | clayg, looks like kill_server | 20:11 |
| peluse | will take out 2 nodes in our EC config (per call) which means I can't really put together the revert test like we talked about since I can't kill just one node at a time | 20:12 |
| peluse | if that makes sense | 20:12 |
| clayg | peluse: oh, takes out one server - but two disks | 20:14 |
| peluse | yeah | 20:14 |
| clayg | maybe if you dig aroud in replica2part2dev until you find *just* the right placement you need? | 20:15 |
| peluse | yeah, I think I'll have to do something like that | 20:15 |
| os1 | clayg : Not sure that your explanation completely made sense. | 20:15 |
| clayg | other option i 'spose if plumb in something for mount_check so that you can "disable" a disk like it's unmounted or something... maybe... | 20:15 |
| os1 | clayg : Regarding the lock. | 20:15 |
| clayg | os1: np, just ignore me then | 20:15 |
| peluse | clayg, that might be easier actually.... have to run off to a meeting here soon. Will keep ya posted. For a second there I didn't think PUT was working with < parity nodes down! | 20:16 |
| *** lpabon has quit IRC | 20:16 | |
| clayg | er... > parity maybe? | 20:17 |
| clayg | like... *more* than :P | 20:17 |
| clayg | peluse: np, keep me posted | 20:17 |
| peluse | yeah, the other way :) | 20:17 |
| clayg | lol or s/down/up - there's lots of ways to spell "broke" | 20:18 |
| os1 | clayg : What were you trying to suggest with 'fork-on-connect' ? | 20:19 |
| clayg | idk, that maybe rsync does have a good reason for exposing the option instead of just... it hates administrators? | 20:20 |
| madgriff_ | hey guys, I am trying to serve content from a swift container via a heroku app | 20:25 |
| madgriff_ | does anyone have an example of this? | 20:25 |
| madgriff_ | or any experience doing such things :) | 20:25 |
| clayg | madgriff_: like cloud files via cdn or just directly out of swift? | 20:26 |
| *** tsg_ has quit IRC | 20:26 | |
| madgriff_ | clayg: I have an app that generates static html files and writes them to my swift container for persistent storage. Ive gotten that part down fine, id now like to serve these static files from my application | 20:27 |
| notmyname | either set the container to public or use a tempurl for time-limited access | 20:27 |
| clayg | can your application do like a "media" url option? just point it to the public container? | 20:28 |
| madgriff_ | clayg: how its set up now (prior to swift) is it writes the html files and stores them locally, and then serves them from the local dir | 20:28 |
| openstackgerrit | Merged openstack/swift: Fix the prefix of messages caputured from stderr https://review.openstack.org/164552 | 20:28 |
| madgriff_ | clayg: no, I need to serve them, i cant set world readable on my container | 20:28 |
| clayg | oic - sor proxy access through the app | 20:29 |
| madgriff_ | yes :) | 20:29 |
| clayg | there's lots of "depends how you want to do it" with that setup | 20:29 |
| clayg | well... mostly there's lots of "depends how you want to do auth token management" with that setup | 20:29 |
| clayg | once you have a token - the return resp(body=swiftclient.get_object()) part's always ends up about the same | 20:30 |
| madgriff_ | clayg: when I write I set up a swiftclient.Connection | 20:38 |
| madgriff_ | clayg: could I follow that same convention with the read | 20:38 |
| clayg | new connection for every request - sure - and make an auth request each time - would work - maybe not the most efficent thing you could do - might depend on how many of these requests you wanna do per second? | 20:41 |
| *** zhill has joined #openstack-swift | 20:45 | |
| mattoliverau | clayg: I didn't want to push to gerrit until I could run tests on it, etc.. And there seems to be pyeclib issues on current EC HEAD.. Tho am sure that probably is just my environment. Haven't had a chance to debug it yet, but wanted tdasilva to have access to the codec while I slept. Surpisingly there weren't too many merge conflicts but only testing with let me know if there is brokenness caused by the rebase (re: obj | 20:47 |
| mattoliverau | versioning) | 20:47 |
| tdasilva | mattoliverau: hi, i had run into some pyeclib issues too, and I had to delete my .tox dir | 20:48 |
| tdasilva | mattoliverau: just FYI... | 20:48 |
| tdasilva | mattoliverau: and also get the latest pyeclib :-) | 20:49 |
| mattoliverau | I thought I had tried that, but will check again. Once done you happy for me to push a new patchet in your name? | 20:49 |
| tdasilva | mattoliverau: sure! thanks for all your help!! | 20:50 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/88736 | 20:51 |
| openstackgerrit | Thiago da Silva proposed openstack/swift: Refactor server side copy as middleware https://review.openstack.org/156923 | 20:58 |
| *** os1 has quit IRC | 20:59 | |
| clayg | notmyname: ^ look at that community collaborate! | 21:02 |
| *** silor1 has quit IRC | 21:03 | |
| notmyname | woot | 21:08 |
| *** thumpba has joined #openstack-swift | 21:09 | |
| *** tsg has joined #openstack-swift | 21:10 | |
| notmyname | I just discovered that OS X autocorrects "defcore" to "deface". I find that humorous. | 21:10 |
| mattoliverau | Lol | 21:12 |
| *** tellesnobrega has quit IRC | 21:13 | |
| *** Fin1te has quit IRC | 21:17 | |
| *** tellesnobrega has joined #openstack-swift | 21:18 | |
| openstackgerrit | Tushar Gohad proposed openstack/swift: Bump PyECLib version to >= 1.0.3 https://review.openstack.org/164872 | 21:19 |
| clayg | tsg: why not just go for the 1.0.4!? | 21:19 |
| clayg | oh... maybe there is no pyeclib 1.0.4 :P | 21:20 |
| tsg | clayg: I had submitted 1.0.3 for global-requirements :) | 21:20 |
| tsg | clayg: need to match that on the swift side | 21:21 |
| tsg | clayg: not sure if can we do 1.0.4 in swift requirements (if yes, I will resubmit) - and yes, there is a pyeclib 1.0.4 :) | 21:22 |
| tsg | clayg: :) looks like you approved - so end of that discussion | 21:22 |
| swifterdarrell | tsg: btw, I successfully got all the pyeclib (and dependency) code packaged and built for Ubuntu Precise, Trusty, and CentOS/RHEL | 21:23 |
| tsg | swifterdarrell: that's awesome! :) | 21:23 |
| tsg | hope the ubuntu uploads last night came handy | 21:23 |
| swifterdarrell | tsg: yup | 21:23 |
| swifterdarrell | tsg: i had to fix a couple things, tho... | 21:23 |
| *** fbo has quit IRC | 21:24 | |
| tsg | swifterdarrell: might be useful to pass those on to zigo | 21:24 |
| swifterdarrell | tsg: let me see if I can get a concise list of things I had to fix | 21:24 |
| tsg | (and me :)) | 21:24 |
| tsg | swifterdarrell: that will be great! | 21:24 |
| *** fbo has joined #openstack-swift | 21:26 | |
| notmyname | tsg: which means https://review.openstack.org/#/c/164172/ needs to be abandoned, right? | 21:30 |
| tsg | notmyname: yes - I am assuming using a new branch triggers a new review (and change id) | 21:31 |
| *** NM has quit IRC | 21:31 | |
| notmyname | tsg: I guess. why do you need it on master? | 21:31 |
| tsg | notmyname: jenkins is not happy that feature/ec branch does not exist in the global requirements repo | 21:32 |
| notmyname | oh i see | 21:32 |
| notmyname | and they do have the same change-id | 21:32 |
| tsg | http://logs.openstack.org/72/164172/1/check/gate-swift-requirements/0ef8713/console.html.gz | 21:32 |
| notmyname | but yeah, I guess it's (branch, change-id) as the unique pair | 21:32 |
| * tsg nods | 21:32 | |
| notmyname | which makes searching for it "fun" https://review.openstack.org/#/q/Ie6e0c1b491017dd5096b14c23b610abc22b03d6a,n,z | 21:33 |
| tsg | :) | 21:33 |
| notmyname | I approved the one for master | 21:34 |
| tsg | notmyname: thanks | 21:34 |
| notmyname | clayg: dfg is all like "we've got hummingbird and don't want your stinkin' object server" ;-) | 21:36 |
| dfg | ha | 21:36 |
| clayg | dfg: well the "mess" i was refering to was the pid files - if you don't have a app:object-server section you don't get an object server regardless - right? | 21:38 |
| clayg | dfg: but thanks for the explination none-the-less - here I was assuming you were just trying to make something better on the general principle of thing - for some reason I assume you cloud files guys are more altruistic than the rest of us louts | 21:39 |
| dfg | clayg: ya. well we had this problem before hummingbird anyway. | 21:39 |
| dfg | clayg: and I think it is better in general- except for fixing what you had in the comment | 21:40 |
| dfg | like if you don't want to run the auditor | 21:41 |
| clayg | which part? about just letting the object-server fail to start because there's no app:object-server section and cleaning up the pid - or trying to somehow add paste config groking to the double-config-parse step so you better implement the coupling? | 21:41 |
| clayg | dfg: same thing? you don't add the audtior section. | 21:41 |
| clayg | dfg: I guess i'm confused - it seems like the "problem" was not unwanted servers *starting* - it was their failing to start leaving stale pids around? | 21:42 |
| dfg | well- the stale pids is def a consequence but i'd rather have them just not start if they are not configured | 21:42 |
| clayg | oic - LBYL is better than EAPF in this case for some reason? | 21:43 |
| dfg | the coding is probably a little messy with the double config parse | 21:43 |
| dfg | but i didn't think it was a huge deal. if you think its really ugly then i can rty to clean it up some | 21:43 |
| clayg | instead of swift-init all start - could you just run swift-init object-auditor object-replicator object-updater start? | 21:44 |
| dfg | ya. we could do that. | 21:44 |
| clayg | dfg: well i didn't see "let them try to start and fail" being a huge deal - so I think I was definately misunderstanding your goal. | 21:45 |
| dfg | i just thought that if swift-init had the config and could tell that the server would never run that it could just not try. | 21:45 |
| clayg | my point was only that LBYL in this case could potentially lead to a weird situation with paste were the config file was valid as far as paste was concerned but swift-init was trying to be cleaver and wouldn't try to start the object server because the pipleline:main pointed to app:object instead of app:object-server | 21:46 |
| dfg | like there's this: https://github.com/openstack/swift/commit/0a122c1575fac98693059b0ad3b4386e35b74f63 | 21:46 |
| dfg | but if you use that then you will always get these pids lying around | 21:47 |
| clayg | right so the pid cleanup thing seemed like a reasonable thing to fix - no reason to leave the stale pids around once we know the server failed to start - but I didn't see the patch as going after that issue - they could fail to start for other reasons than the config section not listed? | 21:48 |
| dfg | my point was that if you didn't have a section that you on purpose don't want them to start. as oppsed to an error | 21:48 |
| clayg | yeah status code checking could another issue | 21:49 |
| zaitcev | Hey, here's an idea: switch to Systemd. It tracks everything 100%, there aren't any stale PIDs, ever. | 21:49 |
| clayg | zaitcev: that's a silly idea :) | 21:49 |
| clayg | dfg: I think if you say swift-init all start it's somewhat ambiguous if a missing config section is a warning or error - i sure seems that swift object-auditor start with missing object-audtior section would be something closer to an error - not sure | 21:50 |
| zaitcev | well... We actually tell people only use systemctl start openstack-swift-proxy etc. in RDO, because swift-init proxy-server start saddles the proxy with wrong SElinux context and thus you must defeat SElinux if you use swift-init. | 21:50 |
| clayg | poor swift-init :'( | 21:51 |
| *** echevemaster_ has joined #openstack-swift | 21:52 | |
| dfg | clayg: ya thats true (about the swift-init object-auditor restart error thing). alright, i'll just drop it then. | 21:53 |
| clayg | sorry bro | 21:53 |
| clayg | wouldn't mind having a bug to track the cleaning up of stale pid files on failure immediately tho :\ | 21:53 |
| dfg | not a big deal- i didn't spend a lot of time on it. thought it might be a quick fix. | 21:53 |
| clayg | i know - nothing is simple - i'm really sorry | 21:54 |
| madgriff_ | clayg: thanks for the help earlier btw. Ill start with a connection for each request and then try and find a way to optimize | 21:56 |
| clayg | madgriff_: right on! let us know how it goes - or write something up something quick on a blog or something if you get a chance. Thanks! | 21:56 |
| *** echevemaster_ has quit IRC | 21:58 | |
| dfg | notmyname: i don't think i'm going to do that x-container-response-meta thing. i'm working on just changing static web to look for x-container-meta-cache-control and use it as a default Cache-Control for static web requests. i think it makes sense so far | 21:58 |
| *** echevemaster has joined #openstack-swift | 21:58 | |
| notmyname | dfg: ah | 22:05 |
| dmsimard | Hey guys is there a blueprint or something for https://review.openstack.org/#/c/123220/ ? Trying to track down if there's any commits towards that topic. | 22:05 |
| notmyname | so if we do eventually do the x-container-response-meta, then we'll have to deal with conflicts and migrations and always supporting it | 22:06 |
| notmyname | dmsimard: yup. acoles_away and jrichli are the principle people working on it right now (while others focus on EC work). there's a feature/crypto branch for the work | 22:06 |
| zaitcev | dmsimard: spec replaces blueprint as a concep. | 22:06 |
| dfg | well- thats what i was worried about to begin with. but when you said nobody has ever wanted this- why would it ever get implemented | 22:07 |
| dfg | like- it wasn't really the problem i was trying to solve. | 22:07 |
| dfg | and you already have that problem with CORS | 22:07 |
| dfg | also- it isn't a problem because if you set it up as. set these headers unless they are already set (which is already a problem you'd have because of custom object headers) then it wouldn't be an issue | 22:08 |
| dfg | i couldn't think of another header that you'd really want to do this with. | 22:10 |
| * clayg has already giving dfg enough headache for the day so he's not going to mention that it should technically be sysmeta with name translation to avoid user-meta name collision :D | 22:10 | |
| dfg | also there's a good reason to want to have cache control on static web container listings | 22:10 |
| notmyname | dfg: ya. I agree. you're right | 22:11 |
| dfg | clayg: sysmeta was how i was going to do it to begin with :) | 22:11 |
| dfg | but we already use x-container-meta for static web and CORS- how is this any different? | 22:11 |
| dmsimard | notmyname: Cool thanks, are you guys aiming for a particular milestone for release ? | 22:11 |
| clayg | i guess static web was one of those legacy middlwares that used a bunch of user-meta already :\ | 22:12 |
| notmyname | dmsimard: "when it's done" | 22:12 |
| dmsimard | notmyname: Good answer :) | 22:12 |
| dfg | granted i hate typing out x-container-meta-access-control-allow-origin as the next guy... | 22:12 |
| dfg | but that's how it goes | 22:13 |
| *** tab____ has quit IRC | 22:13 | |
| *** welldannit has quit IRC | 22:13 | |
| dfg | anyway- i think it kinda fits right in there. for now. we'll see what happens to the pr | 22:13 |
| swifterdarrell | tsg: Please see https://gist.github.com/dbishop/692f39d9081414b12699 (feel free to forward to anyone else who could benefit from the feedback) | 22:13 |
| *** tsg has quit IRC | 22:17 | |
| clayg | tdasilva: bah the _make_copy_request and _make_copy_context methods are confusing (I know I wrote them) | 22:19 |
| clayg | mabye it's just the middleware COPY call backs are confusing | 22:20 |
| clayg | torgomatic: are you sure that somehow gets better if copy is extracted? I mean slo/dlo will still need to set the copy_hook - it just gets consumed in middleware instead of the ObjectController | 22:20 |
| torgomatic | clayg: if the copy middleware is left of SLO/DLO, then the copy hook goes away | 22:21 |
| dfg | clayg: do you think x-container-meta-cache-control already being set for some unknown reason is an issue? thats the only real problem i didn't like | 22:27 |
| dfg | figured it would come up in review | 22:27 |
| tdasilva | torgomatic: yeah, I honestly just spent as much time on that patch to make unit tests pass. will start looking more at code now... | 22:31 |
| tdasilva | I meant clayg ^^^ | 22:31 |
| tdasilva | torgomatic: left of slo/dlo? I thought it had to be to the right.... | 22:31 |
| torgomatic | tdasilva: the earlier it goes, the fewer middlewares need to know that COPY is even a thing | 22:32 |
| tdasilva | torgomatic: mmm...i see...unless it does copies itself...like obj. versioning | 22:33 |
| torgomatic | tdasilva: right, but the fact that slo and dlo have this ugly copy-hook thing is a bug, not a feature IMO | 22:33 |
| torgomatic | so put it to the left, and life is good | 22:33 |
| tdasilva | torgomatic: got it, thanks for the insight. will continue to spend more time on i | 22:34 |
| tdasilva | it | 22:34 |
| *** panbalag has quit IRC | 22:42 | |
| *** jrichli has quit IRC | 22:42 | |
| *** Nadeem has quit IRC | 22:44 | |
| *** madgriff_ has quit IRC | 22:55 | |
| clayg | dfg: yeah i mean it'll come up - but mostly just because we have sysmeta and that's a better way to do it now - OTOH, it'd be stupid to make this option do the right thing and leave the rest a mess - so then you'd want to cleanup all the existing options with a compatibility shim and *then* add the new options - which may not be that attractive to you - so you're probably better of just trying to slip in one more piece of sy | 23:03 |
| clayg | it won't really make it any harder to cleanup the rest of them later - so you're left with only the pre-existing x-container-meta-cache-control shadowing the new behavior - probably not that likely (FLW) | 23:04 |
| *** panbalag has joined #openstack-swift | 23:14 | |
| *** gyee has quit IRC | 23:16 | |
| *** lnr has joined #openstack-swift | 23:33 | |
| *** lnr has left #openstack-swift | 23:33 | |
| *** tsg has joined #openstack-swift | 23:46 | |
| *** thumpba has quit IRC | 23:51 | |
| *** tsg has quit IRC | 23:52 | |
| *** NM has joined #openstack-swift | 23:58 | |
| *** ho has joined #openstack-swift | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!