| kota_ | wbhuber: still around? | 00:00 |
|---|---|---|
| *** changbl has quit IRC | 00:17 | |
| *** wshao has joined #openstack-swift | 00:24 | |
| *** flwang1 has quit IRC | 00:24 | |
| *** garthb has quit IRC | 00:25 | |
| *** zhill has quit IRC | 00:28 | |
| *** darrenc_afk is now known as darrenc | 00:32 | |
| *** jrichli has joined #openstack-swift | 00:37 | |
| *** gyee has quit IRC | 00:41 | |
| *** m_kazuhiro has joined #openstack-swift | 00:42 | |
| *** pberis has joined #openstack-swift | 00:51 | |
| *** bill_az_ has quit IRC | 01:10 | |
| *** baojg has joined #openstack-swift | 01:17 | |
| *** baojg has quit IRC | 01:17 | |
| *** baojg has joined #openstack-swift | 01:19 | |
| openstackgerrit | Tim Burke proposed openstack/python-swiftclient: Miscellaneous (mostly test) cleanup https://review.openstack.org/222009 | 01:21 |
| *** haomaiwang has joined #openstack-swift | 01:29 | |
| *** jlhinson has joined #openstack-swift | 01:35 | |
| *** wshao has quit IRC | 01:36 | |
| *** jlhinson has quit IRC | 01:44 | |
| *** flwang1 has joined #openstack-swift | 01:44 | |
| *** jrichli has quit IRC | 01:47 | |
| *** jrichli has joined #openstack-swift | 01:47 | |
| *** pberis has quit IRC | 01:48 | |
| *** SkyRocknRoll has joined #openstack-swift | 01:57 | |
| *** haomaiwang has quit IRC | 02:01 | |
| *** haomaiwang has joined #openstack-swift | 02:01 | |
| wbhuber | kota_: yes, i am. | 02:04 |
| kota_ | wbhuber: good, any updates? | 02:07 |
| wbhuber | kota_: i got sidetracked with family for a bit - putting kids to bed, etc. all right, we know that X-Backend-Storage-Policy-Index has been assigned to self.headers in process_job and i debugged further that in _get_response both self.headers and headers do not have policy established. | 02:09 |
| kota_ | wbhuber: np and i bet you know what going on in reconstructor. | 02:11 |
| wbhuber | kota_: i haven't delved much on self.headers whether policy info should be used as a parameter when doing a http_conn. I don't think so based on what I've seen thus far. | 02:12 |
| wbhuber | kota_: if you go to my comment for Patch Set 1 after yours, you'd see "p headers" and "p self.headers". the only missing parameter in self.headers is X-Backend-Node-Index. | 02:14 |
| *** bkopilov has quit IRC | 02:14 | |
| kota_ | ok, we need X-Backend-Node-Index...no? | 02:16 |
| kota_ | AFAIK, X-Backend-Node-Index is needed to put a fragment | 02:16 |
| wbhuber | kota_: In my comment, that index has not been used anywhere in swift. | 02:16 |
| wbhuber | kota_: really? | 02:16 |
| kota_ | not sure. | 02:17 |
| kota_ | but | 02:17 |
| kota_ | _get_response will be used for gathering extra fragments to rebuild a new fragment to push. | 02:17 |
| wbhuber | Yes, it gets a single EC fragement archive from each node in order to rebuild an object. | 02:18 |
| kota_ | curious | 02:18 |
| *** AndreiaKumpera has joined #openstack-swift | 02:19 | |
| kota_ | wbhuber: Was the "self" instance Reconstructor? | 02:21 |
| wbhuber | kota_: let me check. | 02:22 |
| kota_ | no, confused x-backend-ssync-node-index | 02:22 |
| kota_ | i mean ssync-node-index but you said...backend-node-index. | 02:23 |
| wbhuber | <swift.obj.reconstructor.ObjectReconstructor object at 0x7f9141be9c10> | 02:23 |
| wbhuber | that's the self | 02:23 |
| kota_ | looks correctly ObjectReconstructor. | 02:24 |
| wbhuber | you can look at my debug : http://pastebin.com/SkXevBPs | 02:25 |
| kota_ | thanks | 02:26 |
| kota_ | let me see existing unit tests for reconstructor | 02:26 |
| wbhuber | fyi, this is running on a real EC cluster (not SAIO) | 02:26 |
| kota_ | nice | 02:26 |
| kota_ | perhaps, you are running around 2.3.0? | 02:28 |
| wbhuber | right. | 02:28 |
| *** AndreiaKumpera has quit IRC | 02:28 | |
| *** alejandrito has joined #openstack-swift | 02:28 | |
| kota_ | I am seeing 2.4.0 or later master, node-index seem not to be used anymore. | 02:29 |
| wbhuber | interesting | 02:29 |
| kota_ | I'm now bisecting which commit changes the behavior, wait a bit please. | 02:30 |
| wbhuber | no problem | 02:30 |
| *** SkyRocknRoll has quit IRC | 02:34 | |
| kota_ | hah, the change is here, 518262ab6ecd8faa2b915df118ffc70a30112a18 | 02:35 |
| wbhuber | checking.. | 02:36 |
| kota_ | that change is just 1 line minus | 02:36 |
| kota_ | - headers['X-Backend-Node-Index'] = node['index'] | 02:36 |
| kota_ | Paul commented "Assuming nobody intentionally left this in here for some reason..." | 02:36 |
| wbhuber | yeah, just saw that | 02:37 |
| kota_ | so we don't have to care about X-Backend-Node-Index anymore. | 02:37 |
| kota_ | we can forcus on just what we should do. | 02:38 |
| wbhuber | https://review.openstack.org/#/c/182479/1/swift/obj/reconstructor.py | 02:38 |
| kota_ | wbhuber: right | 02:39 |
| *** eranrom has joined #openstack-swift | 02:41 | |
| kota_ | Just my opiniton and it might be only me, I like to keep self.headers for general use case, i.e. we don't modify self.headers for each request. | 02:41 |
| kota_ | so I want to change something like self.headers['X-Backend-Policy-Index'] = index | 02:42 |
| kota_ | because the policy index might be different for each request when we have more than 2 EC policies in the cluster. | 02:43 |
| kota_ | I like to bet reconstructor could generate headers for the request like as follows: | 02:44 |
| kota_ | headers = self.headers.copy() | 02:45 |
| kota_ | headers['something'] = anything | 02:45 |
| kota_ | and pass to _get_response | 02:45 |
| wbhuber | that can be worked out | 02:45 |
| kota_ | And we can write unit tests for that "self.headers are consistent" but each request has expected headers. | 02:47 |
| kota_ | that's current thought from my side right now, and not yet tested :\ | 02:47 |
| *** SkyRocknRoll has joined #openstack-swift | 02:48 | |
| kota_ | wbhuber: does it you make sense? | 02:48 |
| wbhuber | kota_: it does | 02:48 |
| wbhuber | kota_: i will get something out in the morning for you | 02:48 |
| kota_ | good | 02:49 |
| *** eranrom has quit IRC | 02:49 | |
| *** ctrath has joined #openstack-swift | 02:49 | |
| kota_ | wbhuber: thanks for working this and poke me to review actual behavior :) | 02:49 |
| wbhuber | kota_: ありがとう | 02:50 |
| wbhuber | kota_: I hope i got it right | 02:50 |
| jrichli | wbhuber: I think the second character isn't quite right, but the rest looks good :-) | 02:52 |
| *** ctrath has quit IRC | 02:53 | |
| wbhuber | jrichli: glad that you know more than just English and ASL :) | 02:53 |
| jrichli | wbhuber: oh, I dont claim to know Japanese. I am just now starting on it. I just googled arigato to see the spelling :-) | 02:54 |
| jrichli | and as you know, I can't claim to know ASL either ;-) | 02:54 |
| *** ctrath has joined #openstack-swift | 02:55 | |
| jrichli | mainly, I wanted to make sure that is what you had said. | 02:55 |
| wbhuber | jrichli: thanks for checking! kota_, i am sure, will validate whehter it is correct or not. | 02:56 |
| jrichli | I know the second to last character is pronounced "to", so I am not sure now about what the sound the last character is for - I will have to look that up. I guess just more "o". | 02:56 |
| *** haomaiwang has quit IRC | 03:01 | |
| *** haomaiwa_ has joined #openstack-swift | 03:01 | |
| *** alejandrito has quit IRC | 03:10 | |
| *** silor has joined #openstack-swift | 03:14 | |
| kota_ | wbhuber: perfect Japanese spelling | 03:15 |
| *** silor has quit IRC | 03:15 | |
| notmyname | good evening | 03:17 |
| kota_ | notmyname: good evening | 03:17 |
| notmyname | jrichli: thanks for the encryption update | 03:20 |
| jrichli | notmyname: yw :-) | 03:21 |
| *** ctrath has quit IRC | 03:21 | |
| kota_ | jrichili: strictly, the last character is pronounced as like..."woo" in English? | 03:21 |
| kota_ | I'm not sure. | 03:21 |
| kota_ | Japanese writes as arigatou | 03:21 |
| jrichli | ah, ok. thx! | 03:21 |
| *** mahatic has joined #openstack-swift | 03:23 | |
| notmyname | mahatic: good morning | 03:23 |
| mahatic | good evening notmyname ! | 03:24 |
| *** silor has joined #openstack-swift | 03:24 | |
| *** sanchitmalhotra has joined #openstack-swift | 03:28 | |
| *** links has joined #openstack-swift | 03:30 | |
| janonymous | morning :) | 03:35 |
| *** SkyRocknRoll has quit IRC | 03:40 | |
| *** bkopilov has joined #openstack-swift | 03:43 | |
| notmyname | kota_: are you still working on https://review.openstack.org/#/c/186735/ to resolve https://bugs.launchpad.net/swift/+bug/1460920 ? | 03:46 |
| openstack | Launchpad bug 1460920 in OpenStack Object Storage (swift) "Successful PUT object might be missing container update on EC" [High,In progress] - Assigned to Kota Tsuyuzaki (tsuyuzaki-kota) | 03:46 |
| kota_ | notmyname: not yet, that's pending on me right now. | 03:46 |
| kota_ | but I can put higher priority for it. | 03:47 |
| kota_ | to me | 03:47 |
| notmyname | kota_: thanks. it seems like a pretty big deal for EC. I'd love to see a patch land soon | 03:47 |
| kota_ | ok, I added it to my task for working in this week. | 03:48 |
| notmyname | http://www.tripadvisor.com/Restaurants-g1066854-c38-Shinagawa_Tokyo_Tokyo_Prefecture_Kanto.html | 03:48 |
| kota_ | Sushi! | 03:49 |
| jrichli | can't wait! | 03:56 |
| jrichli | looking forward to some good ramen as well | 03:56 |
| *** changbl has joined #openstack-swift | 03:56 | |
| kota_ | jrichli: there is a lot of ramen restraunt in Shinagawa. | 03:57 |
| jrichli | kota_: do you happen to know of any go cafes? | 03:58 |
| jrichli | i should say, igo | 03:58 |
| *** bill_az has quit IRC | 03:59 | |
| kota_ | jrichli: idk in detail...but i can google the place | 03:59 |
| kota_ | igo cafe is not so popular, I guess. | 03:59 |
| wbhuber | kota_: thx for the confirmation on the Japanese spelling. :-] | 04:00 |
| jrichli | that's ok. It would be neat to go to one. It would be even better to know beforehand if they are ok with kyu levels :-) | 04:00 |
| *** haomaiwa_ has quit IRC | 04:01 | |
| *** haomaiwang has joined #openstack-swift | 04:01 | |
| kota_ | wbhuber: np :) | 04:01 |
| *** baojg has quit IRC | 04:02 | |
| kota_ | jrichili: what does "kyu" mean? | 04:02 |
| *** SkyRocknRoll has joined #openstack-swift | 04:03 | |
| jrichli | beginner levels - doesn't have to be complete beginner, but that sort of thing. | 04:03 |
| jrichli | no "dan" levels only | 04:03 |
| *** kei_yama has quit IRC | 04:03 | |
| kota_ | I see, thanks | 04:03 |
| *** kei_yama has joined #openstack-swift | 04:04 | |
| peluse__ | notmyname, clayg - sorry been away for a few days dealing with personal stuff. EC perf work back on track and I'll start looking through bugs again tomorrow. | 04:04 |
| notmyname | peluse__: thanks. we got all of the ec tagged bugs set with a priority. I've starred the medium/high ones that have patches | 04:05 |
| notmyname | peluse__: hope everything is ok | 04:05 |
| *** wbhuber has quit IRC | 04:06 | |
| *** morgan has quit IRC | 04:06 | |
| jrichli | good night all | 04:07 |
| *** morganfainberg has joined #openstack-swift | 04:07 | |
| *** proteusguy__ has quit IRC | 04:07 | |
| notmyname | jrichli: good night | 04:08 |
| *** proteusguy_ has joined #openstack-swift | 04:08 | |
| kota_ | jrichli: good night. | 04:09 |
| *** morganfainberg is now known as morgan | 04:09 | |
| *** marzif has joined #openstack-swift | 04:10 | |
| *** jrichli has quit IRC | 04:11 | |
| *** kei_yama has quit IRC | 04:14 | |
| *** kei_yama has joined #openstack-swift | 04:15 | |
| *** trifon_ has joined #openstack-swift | 04:16 | |
| *** ppai has joined #openstack-swift | 04:20 | |
| *** SkyRocknRoll has quit IRC | 04:22 | |
| *** esker has joined #openstack-swift | 04:24 | |
| *** rjaiswal has quit IRC | 04:30 | |
| *** esker has quit IRC | 04:33 | |
| *** flwang1 has quit IRC | 04:37 | |
| *** jith_ has quit IRC | 04:40 | |
| *** flwang1 has joined #openstack-swift | 04:52 | |
| *** mahatic has quit IRC | 04:57 | |
| *** fifieldt has joined #openstack-swift | 04:58 | |
| *** mahatic has joined #openstack-swift | 05:00 | |
| *** haomaiwang has quit IRC | 05:01 | |
| *** haomaiwa_ has joined #openstack-swift | 05:02 | |
| *** hrou has quit IRC | 05:07 | |
| *** flwang1 has quit IRC | 05:08 | |
| *** esker has joined #openstack-swift | 05:11 | |
| *** esker has quit IRC | 05:19 | |
| *** mahatic has quit IRC | 05:36 | |
| *** m_kazuhiro has quit IRC | 05:40 | |
| *** sanchitmalhotra1 has joined #openstack-swift | 05:47 | |
| *** sanchitmalhotra has quit IRC | 05:50 | |
| *** jerrygb has joined #openstack-swift | 05:53 | |
| *** fifieldt has quit IRC | 05:56 | |
| *** esker has joined #openstack-swift | 05:56 | |
| *** breitz has quit IRC | 05:58 | |
| *** mahatic has joined #openstack-swift | 05:58 | |
| *** breitz has joined #openstack-swift | 05:59 | |
| *** haomaiwa_ has quit IRC | 06:01 | |
| *** eranrom has joined #openstack-swift | 06:01 | |
| *** haomaiwang has joined #openstack-swift | 06:01 | |
| *** esker has quit IRC | 06:07 | |
| *** m_kazuhiro has joined #openstack-swift | 06:11 | |
| *** jerrygb has quit IRC | 06:19 | |
| *** mahatic has quit IRC | 06:22 | |
| *** breitz has quit IRC | 06:22 | |
| *** esker has joined #openstack-swift | 06:25 | |
| *** esker has quit IRC | 06:31 | |
| *** trifon_ has quit IRC | 06:33 | |
| *** trifon_ has joined #openstack-swift | 06:35 | |
| *** SkyRocknRoll has joined #openstack-swift | 06:37 | |
| *** sanchitmalhotra has joined #openstack-swift | 06:43 | |
| *** sanchitmalhotra1 has quit IRC | 06:45 | |
| *** mahatic has joined #openstack-swift | 06:48 | |
| *** km has quit IRC | 06:50 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 06:50 | |
| *** SkyRocknRoll has quit IRC | 06:50 | |
| *** km has joined #openstack-swift | 06:51 | |
| *** haomaiwang has quit IRC | 07:01 | |
| *** haomaiwang has joined #openstack-swift | 07:01 | |
| *** rledisez has joined #openstack-swift | 07:05 | |
| *** esker has joined #openstack-swift | 07:11 | |
| *** breitz has joined #openstack-swift | 07:16 | |
| *** baojg has joined #openstack-swift | 07:22 | |
| *** SkyRocknRoll_ has quit IRC | 07:23 | |
| *** esker has quit IRC | 07:24 | |
| *** SkyRocknRoll_ has joined #openstack-swift | 07:39 | |
| *** esker has joined #openstack-swift | 07:41 | |
| *** baojg has quit IRC | 07:42 | |
| *** esker has quit IRC | 07:50 | |
| *** baojg has joined #openstack-swift | 07:51 | |
| *** baojg has quit IRC | 07:54 | |
| *** geaaru has joined #openstack-swift | 07:56 | |
| *** baojg has joined #openstack-swift | 07:58 | |
| *** bapalm has quit IRC | 07:58 | |
| *** StevenK has quit IRC | 07:59 | |
| *** bapalm has joined #openstack-swift | 08:00 | |
| *** haomaiwang has quit IRC | 08:01 | |
| *** jistr has joined #openstack-swift | 08:01 | |
| *** haomaiwang has joined #openstack-swift | 08:01 | |
| *** StevenK has joined #openstack-swift | 08:02 | |
| *** eranrom has quit IRC | 08:03 | |
| *** wer_ has quit IRC | 08:04 | |
| *** esker has joined #openstack-swift | 08:10 | |
| *** wer_ has joined #openstack-swift | 08:12 | |
| *** marzif has quit IRC | 08:13 | |
| *** acoles_ is now known as acoles | 08:18 | |
| *** baojg has quit IRC | 08:19 | |
| *** jerrygb has joined #openstack-swift | 08:19 | |
| *** esker has quit IRC | 08:23 | |
| *** jerrygb has quit IRC | 08:24 | |
| *** sanchitmalhotra has quit IRC | 08:30 | |
| *** sanchitmalhotra has joined #openstack-swift | 08:34 | |
| *** sanchitmalhotra1 has joined #openstack-swift | 08:43 | |
| *** sanchitmalhotra has quit IRC | 08:45 | |
| *** silor has quit IRC | 08:55 | |
| *** eranrom has joined #openstack-swift | 09:00 | |
| *** haomaiwang has quit IRC | 09:01 | |
| *** haomaiwang has joined #openstack-swift | 09:01 | |
| *** wer_ has quit IRC | 09:19 | |
| *** jerrygb has joined #openstack-swift | 09:20 | |
| *** jerrygb has quit IRC | 09:26 | |
| *** baojg has joined #openstack-swift | 09:27 | |
| *** wer_ has joined #openstack-swift | 09:28 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Trivial Key Master for encryption https://review.openstack.org/193749 | 09:28 |
| *** baojg has quit IRC | 09:29 | |
| *** T0m_ has joined #openstack-swift | 09:32 | |
| *** T0m_ has quit IRC | 09:34 | |
| *** T0m_ has joined #openstack-swift | 09:35 | |
| *** flwang1 has joined #openstack-swift | 09:40 | |
| *** eranrom has quit IRC | 09:42 | |
| *** aix has quit IRC | 09:45 | |
| *** kota_ has quit IRC | 09:46 | |
| *** haomaiwang has quit IRC | 10:01 | |
| *** silor has joined #openstack-swift | 10:01 | |
| *** haomaiwang has joined #openstack-swift | 10:01 | |
| *** wbhuber has joined #openstack-swift | 10:02 | |
| *** mahatic has quit IRC | 10:05 | |
| *** mahatic_ has joined #openstack-swift | 10:05 | |
| *** wbhuber has quit IRC | 10:07 | |
| *** silor1 has joined #openstack-swift | 10:08 | |
| openstackgerrit | Alistair Coles proposed openstack/swift: Fix purge for tombstone only REVERT job https://review.openstack.org/218023 | 10:08 |
| *** silor has quit IRC | 10:09 | |
| *** silor1 is now known as silor | 10:09 | |
| *** aix has joined #openstack-swift | 10:13 | |
| *** eranrom has joined #openstack-swift | 10:23 | |
| *** haomaiwang has quit IRC | 11:01 | |
| *** haomaiwang has joined #openstack-swift | 11:01 | |
| *** jerrygb has joined #openstack-swift | 11:21 | |
| *** m_kazuhiro has quit IRC | 11:25 | |
| *** jerrygb has quit IRC | 11:26 | |
| *** aix has quit IRC | 11:34 | |
| *** sanchitmalhotra1 has quit IRC | 11:35 | |
| *** aix has joined #openstack-swift | 11:35 | |
| *** jamielennox is now known as jamielennox|away | 11:49 | |
| *** km has quit IRC | 11:50 | |
| *** bkopilov has quit IRC | 11:59 | |
| *** haomaiwang has quit IRC | 12:01 | |
| *** haomaiwang has joined #openstack-swift | 12:01 | |
| *** cdelatte has joined #openstack-swift | 12:02 | |
| *** delattec has joined #openstack-swift | 12:02 | |
| *** jkugel has quit IRC | 12:03 | |
| *** jistr is now known as jistr|mtg | 12:03 | |
| *** flwang1 has quit IRC | 12:04 | |
| *** ppai has quit IRC | 12:07 | |
| *** flwang1 has joined #openstack-swift | 12:10 | |
| *** jordanP has joined #openstack-swift | 12:13 | |
| *** SkyRocknRoll_ is now known as SkyRocknRoll | 12:19 | |
| *** flwang1 has quit IRC | 12:20 | |
| *** ppai has joined #openstack-swift | 12:21 | |
| *** changbl has quit IRC | 12:22 | |
| *** haomaiwang has quit IRC | 12:35 | |
| *** jerrygb has joined #openstack-swift | 12:40 | |
| *** dustins has joined #openstack-swift | 12:42 | |
| *** jerrygb has quit IRC | 12:45 | |
| acoles | peluse__: you around? | 12:50 |
| *** jistr|mtg is now known as jistr | 12:52 | |
| *** ho has quit IRC | 12:54 | |
| *** bill_az_ has joined #openstack-swift | 12:55 | |
| *** kei_yama has quit IRC | 13:00 | |
| *** ppai has quit IRC | 13:02 | |
| *** jkugel has joined #openstack-swift | 13:06 | |
| *** ctrath has joined #openstack-swift | 13:09 | |
| *** pberis has joined #openstack-swift | 13:09 | |
| *** ppai has joined #openstack-swift | 13:18 | |
| *** links has quit IRC | 13:23 | |
| *** eranrom has quit IRC | 13:30 | |
| *** changbl has joined #openstack-swift | 13:31 | |
| *** hrou has joined #openstack-swift | 13:33 | |
| *** jerrygb has joined #openstack-swift | 13:41 | |
| *** jerrygb has quit IRC | 13:45 | |
| *** ppai has quit IRC | 13:50 | |
| *** lcurtis has joined #openstack-swift | 14:04 | |
| *** jrichli has joined #openstack-swift | 14:06 | |
| *** bkopilov has joined #openstack-swift | 14:08 | |
| *** jerrygb has joined #openstack-swift | 14:15 | |
| *** haomaiwang has joined #openstack-swift | 14:16 | |
| *** esker has joined #openstack-swift | 14:19 | |
| *** wbhuber has joined #openstack-swift | 14:20 | |
| *** marzif has joined #openstack-swift | 14:23 | |
| *** esker has quit IRC | 14:23 | |
| *** esker has joined #openstack-swift | 14:23 | |
| *** marzif has quit IRC | 14:30 | |
| *** marzif has joined #openstack-swift | 14:31 | |
| *** jlhinson has joined #openstack-swift | 14:32 | |
| mahatic_ | jrichli: good morning | 14:35 |
| mahatic_ | jrichli: need a confirmation: what error do you get on TestSerialization? (wrt content-type)? Is that 500 internal error? or something more specific? | 14:36 |
| *** proteusguy_ has quit IRC | 14:36 | |
| jrichli | mahatic_: good morning. yes, It is 500 internal error. It is a formatting issue with the container listing, caused by the content-type - maybe because it was not encrypted. so decryption makes it invalid? but that is just my guess. haven't dug in. | 14:41 |
| mahatic_ | jrichli: So far, I did add a content type in case it is missing on a PUT, but that doesn't solve it. I'm looking into that, will let you know | 14:42 |
| jrichli | mahatic_: ok, thanks! | 14:43 |
| hrou | mahatic_, jrichli - hey there ! Now that we're done with the keymaster / crypto - would you like me to review any of your work ? Preference ? jrichli if you think you have her reviews covered I'll pick of one the cards instead. | 14:43 |
| jrichli | mahatic_: I will put your name on that card,in that case | 14:43 |
| jrichli | hrou: I think reviewing mahatic_'s changes would be best right now. I haven't done those reviews yet. High on my list though | 14:44 |
| hrou | jrichli, perfect, sounds good ! | 14:44 |
| jrichli | thanks! | 14:44 |
| mahatic_ | jrichli: sure, I think I already put mine on one of them (the guess content-type), but there is one more card too I guess on TestSerialization? | 14:45 |
| jrichli | yes | 14:45 |
| mahatic_ | hrou: sure, thanks! but, I haven't yet pulled down latest changes from keymaster and crypto - I'm waiting for them to get merged. Will push the latest soon as they get merged. Or atleast by tomm | 14:46 |
| clayg | acoles: heh destoryed | 14:46 |
| hrou | mahatic_, oh no worries at all, I don't think they'll effect any of your current out-going reviews though, acoles just did a lot of cleanup / refactoring | 14:47 |
| mahatic_ | hrou: true, you could go ahead, but you'll just see the old keymaster - I have changes on them | 14:47 |
| mahatic_ | s/them/that | 14:47 |
| hrou | mahatic_, hehe yep, I'll ignore that part : ) | 14:48 |
| mahatic_ | hrou: alright :) | 14:48 |
| *** proteusguy_ has joined #openstack-swift | 14:49 | |
| notmyname | good morning | 14:50 |
| *** minwoob has joined #openstack-swift | 14:50 | |
| *** wbhuber has quit IRC | 14:50 | |
| mahatic_ | good morning | 14:50 |
| *** wbhuber has joined #openstack-swift | 14:51 | |
| hrou | morning! | 14:51 |
| notmyname | hrou: you and I need to do summit prep ;-) | 14:52 |
| ctennis | notmyname: new EC bug https://bugs.launchpad.net/swift/+bug/1494359 | 14:52 |
| openstack | Launchpad bug 1494359 in OpenStack Object Storage (swift) "reconstructor list index out of range" [Undecided,New] | 14:52 |
| notmyname | ctennis: ack. thanks | 14:52 |
| *** aix has quit IRC | 14:53 | |
| notmyname | hrou: I'll be traveling next week, so we should schedule some time for the week of the 21st | 14:53 |
| acoles | clayg: heh. sadly it now has a -1 from jenkins. gate-grenade-dsvm has been failing today. problem installing pyeclib | 14:55 |
| *** hrou has quit IRC | 14:55 | |
| *** hrou has joined #openstack-swift | 14:55 | |
| acoles | clayg: peluse__ fyi i am working on patch 213147, hope to push some fixups later | 14:56 |
| patchbot | acoles: https://review.openstack.org/#/c/213147/ | 14:56 |
| acoles | peluse__: if that's ok with you? | 14:56 |
| hrou | notmyname, hey ! That'd be great, yea I agree : ) I'll send you an email so we can schedule sometime. | 14:56 |
| hrou | notmyname, we've been making some good progress here on the various elements of the presentation (e.g. our smr emulation scheme), looking forward to it ! | 14:57 |
| notmyname | great | 14:57 |
| notmyname | https://bugs.launchpad.net/openstack-gate/+bug/1494347 | 14:58 |
| openstack | Launchpad bug 1494347 in OpenStack-Gate "PyECLib 1.0.9 released on 9/10 blowing up grenade jobs due to "src/c/pyeclib_c/pyeclib_c.c:32:25: fatal error: erasurecode.h: No such file or directory"" [Undecided,New] | 14:58 |
| *** marzif has quit IRC | 14:59 | |
| *** haomaiwang has quit IRC | 15:00 | |
| *** lpabon has joined #openstack-swift | 15:00 | |
| *** wbhuber has quit IRC | 15:07 | |
| openstackgerrit | John Dickinson proposed openstack/swift: Pin PyECLib to 1.0.7 https://review.openstack.org/222237 | 15:11 |
| *** garthb has joined #openstack-swift | 15:13 | |
| *** jistr is now known as jistr|mtg | 15:14 | |
| *** wbhuber has joined #openstack-swift | 15:23 | |
| *** mahatic_ has quit IRC | 15:23 | |
| *** mahatic has joined #openstack-swift | 15:24 | |
| *** jistr|mtg is now known as jistr | 15:26 | |
| *** aix has joined #openstack-swift | 15:26 | |
| *** gyee has joined #openstack-swift | 15:34 | |
| *** rjaiswal has joined #openstack-swift | 15:43 | |
| *** jsalem has joined #openstack-swift | 15:46 | |
| jsalem | Where could I find Liberty release info for Swift? http://status.openstack.org/release/ doesn't show anything for Swift and https://launchpad.net/swift/+milestone/2.4.0 is too granual. | 15:51 |
| jsalem | Most interested in whether Erasure Coding will be considered production-ready in Liberty? Or does that wait for Mitaka. | 15:52 |
| notmyname | jsalem: the liberty release hasn't happened yet. what are you....ah ok | 15:52 |
| notmyname | jsalem: EC is our top priority right now. we're working on that right now, and we're planning to have all major issues knocked out by the end of the month | 15:53 |
| notmyname | it depends on what is discovered in further testing and how writing and reviewing patches goes | 15:53 |
| *** Fin1te has joined #openstack-swift | 15:55 | |
| jsalem | cool. so does that mean I should have a production-capable EC available by year end? Assuming you'll need some extra time for bug-fixing. Just looking for an order-of-magnitude here for some 6-12 month planning I'm doing. | 15:55 |
| notmyname | jsalem: if you did plan on that, you wouldn't be the only one ;-) | 15:56 |
| jsalem | Good. I always like company when things get screwed up. lol! | 15:57 |
| notmyname | jsalem: I'm hopeful that people can start using it in prod with the liberty release. but there are still some things that need to get resolved before then | 15:57 |
| notmyname | jsalem: https://bugs.launchpad.net/swift/+bugs?field.tag=ec | 15:57 |
| jsalem | thx… I'll stay tuned. | 16:00 |
| *** lpabon has quit IRC | 16:02 | |
| *** jlhinson has quit IRC | 16:03 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/129154 | 16:08 |
| openstackgerrit | David Goetz proposed openstack/swift: go: change dir permissions to what swift does https://review.openstack.org/222263 | 16:09 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/swift: Updated from global requirements https://review.openstack.org/174343 | 16:18 |
| *** mfalatic has joined #openstack-swift | 16:20 | |
| *** jsalem has left #openstack-swift | 16:20 | |
| *** AndreiaKumpera has joined #openstack-swift | 16:20 | |
| *** jordanP has quit IRC | 16:22 | |
| *** dabukalam has joined #openstack-swift | 16:24 | |
| openstackgerrit | Bill Huber proposed openstack/swift: Reconstructor GET excludes user_agent in log https://review.openstack.org/221506 | 16:25 |
| *** rledisez has quit IRC | 16:25 | |
| *** hemanthm_ is now known as hemanthm | 16:29 | |
| peluse__ | acoles, for sure | 16:30 |
| *** jlhinson has joined #openstack-swift | 16:30 | |
| peluse__ | need another core please on https://review.openstack.org/#/c/217830/ as clayg, acoles and I are done w/it | 16:31 |
| *** peluse__ is now known as peluse | 16:32 | |
| *** ChanServ sets mode: +v peluse | 16:32 | |
| *** lpabon has joined #openstack-swift | 16:35 | |
| *** jistr has quit IRC | 16:40 | |
| *** lpabon has quit IRC | 16:41 | |
| *** mahatic has quit IRC | 16:45 | |
| *** Fin1te has quit IRC | 16:45 | |
| *** SkyRocknRoll has quit IRC | 16:50 | |
| *** esker has quit IRC | 16:51 | |
| *** silor has quit IRC | 16:51 | |
| *** pgbridge has quit IRC | 16:58 | |
| *** geaaru has quit IRC | 17:04 | |
| *** aix has quit IRC | 17:09 | |
| *** esker has joined #openstack-swift | 17:09 | |
| *** changbl has quit IRC | 17:09 | |
| *** gyee has quit IRC | 17:21 | |
| *** zhill has joined #openstack-swift | 17:46 | |
| *** eranrom has joined #openstack-swift | 17:50 | |
| *** rjaiswal has quit IRC | 17:50 | |
| clayg | notmyname: torgomatic: I'm going to go ahead and add a change to validator in ring balancing to error out on the duplicate device thing | 17:53 |
| clayg | yay better graphite stats! | 17:55 |
| clayg | way to merge swift team! | 17:55 |
| clayg | I always feel so happy when I change I liked in principle but haven't made time to look at gets merged - it makes me feel totally superfluous in a nice way | 17:56 |
| notmyname | heh | 17:56 |
| *** ccavanna has joined #openstack-swift | 17:58 | |
| notmyname | clayg: ok, thanks | 18:11 |
| notmyname | clayg: mind if I mark https://bugs.launchpad.net/swift/+bug/1494359 as a duplicate bug? | 18:11 |
| openstack | Launchpad bug 1494359 in OpenStack Object Storage (swift) "reconstructor list index out of range" [Undecided,New] | 18:11 |
| clayg | notmyname: entirely sure - maybe there's something we could do to make the reconstrcutor some how more robust to the crazyness? If you ever changed replica count on EC ring (which is probably the worst thing I ever typed) the reconstructor may want to deal with this situation better than an IndexError? | 18:12 |
| clayg | notmyname: maybe peluse or ctennis or torgomatic could put on their thinking cap and say for sure? | 18:13 |
| openstackgerrit | Clay Gerrard proposed openstack/swift-specs: Adding expiring objecs spec https://review.openstack.org/221914 | 18:15 |
| *** changbl has joined #openstack-swift | 18:17 | |
| acoles | peluse: clayg you guys here? | 18:19 |
| *** garthb has quit IRC | 18:21 | |
| *** Fin1te has joined #openstack-swift | 18:25 | |
| *** garthb has joined #openstack-swift | 18:25 | |
| *** ahale has quit IRC | 18:29 | |
| *** ahale has joined #openstack-swift | 18:32 | |
| *** pgbridge has joined #openstack-swift | 18:34 | |
| *** esker has quit IRC | 18:39 | |
| *** esker has joined #openstack-swift | 18:39 | |
| *** T0m_ has quit IRC | 18:42 | |
| *** remix_tj has quit IRC | 18:56 | |
| *** remix_tj has joined #openstack-swift | 18:58 | |
| clayg | acoles: nope | 19:03 |
| *** pberis has quit IRC | 19:04 | |
| clayg | acoles: no seriously - i was about to walk away from desk - waht's up?! | 19:06 |
| clayg | torgomatic: can you explain to me one more time how partial replicas work? | 19:07 |
| *** jlhinson has quit IRC | 19:07 | |
| *** jlhinson has joined #openstack-swift | 19:07 | |
| acoles | clayg: i'm wondering what order we want to tackle alt frags vs optimistic gets. my optimistic gets patch is based on peluse's frags one (which i am working on right now) | 19:07 |
| torgomatic | clayg: so you've got _replica2part2dev in the ring, and it's 3 by 2^20 or whatever | 19:08 |
| *** pberis has joined #openstack-swift | 19:08 | |
| acoles | clayg: but wondered if we want to re-order them? like. whats the most pressing thing to fix? | 19:08 |
| clayg | replica2part2dev seems like it has to be a whole replica wide (like the ceil of partial replica) - but like not all the part "rows" are full? | 19:08 |
| torgomatic | if you have a partial replica, then you'll have 3 rows of 2^20 and one row of 2^18 or something shorter | 19:08 |
| clayg | acoles: optomistic GETS for *sure* | 19:08 |
| clayg | acoles: note when I say "for sure" I mean this is totally my opinion and I expect that peluse thinks the opposite | 19:09 |
| clayg | torgomatic: ok, so you think of the table as wide instead of tall | 19:09 |
| acoles | clayg: so the reason i based mine on peluse was cos peluse had done the great thing of making get_ondisk_files return a dict | 19:10 |
| clayg | acoles: yes that! lets' make everything depend on that ! | 19:10 |
| torgomatic | clayg: yep, I guess so... probably just because it prints out wide in the console | 19:10 |
| clayg | split up all the things | 19:10 |
| clayg | torgomatic: lol :) | 19:10 |
| acoles | clayg: i could flip them, steal some of his code and make his be based on mine. or i could make a common patch for diskfile that they both base on. | 19:11 |
| acoles | clayg: was 'split up all the things' to me or torgomatic ? | 19:11 |
| clayg | acoles: idk, I just feel like everyone that's stressed a ec cluster has seen data frags with no .durables - not returning them to the proxy is just a dick move on the object-servers part | 19:11 |
| clayg | acoles: to you - I think optomistic gets is going to be a annoyingly deep change :'( | 19:12 |
| clayg | acoles: starting with the diskfile, then object server, then finally into the proxy before it's useful :'( | 19:12 |
| clayg | acoles: seems like a lot of work - let's eat icecream instead! | 19:12 |
| acoles | clayg: lol | 19:12 |
| *** jlhinson has quit IRC | 19:13 | |
| clayg | not to mention reverse listings, ring part placement, rsync module per disk, container sync - and there was this one time I even cared about encryption! - (sorry jrichli :'( | 19:13 |
| acoles | clayg: fast post ? :P | 19:13 |
| clayg | notmyname: I'm doing that "freak out about all the stuff we've got going on" again :'( | 19:13 |
| clayg | OH GAWD!! that came up *again* just recently around here | 19:14 |
| notmyname | clayg: you only seem to freak out on days I choose to work from home. there might be a lesson here, but I'm not sure I like it ;-) | 19:14 |
| clayg | I keep telling people "see ANOTHER reason we need to do fast POST!!!" and they're all like "yup" but it's somehow never seems to make it into the top 10 #1 priorities! | 19:14 |
| acoles | clayg: i get that feeling too every morning - which of a zillion things shall i look at today? | 19:14 |
| acoles | notmyname: your physical presence must be so calming :) | 19:15 |
| clayg | acoles: lol - totally it | 19:16 |
| acoles | argh, lights went out on me :( | 19:16 |
| clayg | acoles: there was that ML post about "what a PTL does" - totally missed the "sit near core members and says 'there there, it'll be ok'" | 19:16 |
| *** dipe has joined #openstack-swift | 19:16 | |
| acoles | clayg: so i just need notmyname to wander up and down my corridor to keep triggering the light sensors :) | 19:17 |
| acoles | apparently i dont have enough body heat | 19:17 |
| clayg | brrrrr | 19:17 |
| clayg | acoles: a'ight well so anyway - umm.. yeah diskfile something | 19:17 |
| clayg | acoles: do you *have* to have alt frags dict stuff to solve for optomistic gets? | 19:18 |
| notmyname | acoles: I made a arduino-based tempurature sensor/display last night (speaking of being cold). it's currently 26C at my desk | 19:18 |
| clayg | like it seems like there'd just be a flag in the find_ondisk_files that instead of "keep going because I haven't seen a durable yet" it just stops and setups the diskfile | 19:18 |
| clayg | what's a C | 19:18 |
| notmyname | clayg: 79 in /murican | 19:18 |
| acoles | clayg: nope. i just want to use dict from get_ondisk_files. i think i could maybe pull that into a tiny separate patch. | 19:19 |
| clayg | yay tiny! | 19:20 |
| acoles | clayg: turned out that returning a non-durable data file wasnt that simple. | 19:20 |
| *** Fin1te has quit IRC | 19:20 | |
| acoles | well, for me anyway :) | 19:20 |
| clayg | acoles: if you can't do it - it can't be done - I'm sure of it | 19:20 |
| jrichli | clayg: lol, np. not to hint at even more work, but ... there is a proposal for a new approach to etag encryption to solve ifMatch requests. It'd be great to have your opinion. hrou will be posting details soon | 19:21 |
| clayg | jrichli: k, np - thanks :) | 19:21 |
| clayg | jrichli: please take some solace in that while I never *work* on encryption - i feel terrible about not working on it like on a daily basis | 19:22 |
| jrichli | clayg: that does not make me feel good. swift needs a happy clayg :-) | 19:22 |
| acoles | clayg: we nearly merged two patches on feature/crypto today. but the gate broke. | 19:23 |
| clayg | notmyname: knows I work best when I'm frustrated!!! | 19:23 |
| clayg | acoles: WTG! break the gate! | 19:23 |
| notmyname | clayg: that's because everyone leaves you alone then ;-) | 19:23 |
| jrichli | lol | 19:23 |
| *** tongli has joined #openstack-swift | 19:23 | |
| hrou | +1 to a happy clayg : ) Thanks jrichli yep I'll have an update in the etherpad we can discuss soon. | 19:24 |
| * acoles recommends join #openstack-counselling :P | 19:24 | |
| clayg | notmyname: ahhhhhhhhh | 19:24 |
| clayg | forever alone :'( | 19:24 |
| openstackgerrit | David Goetz proposed openstack/swift: go: change dir permissions to what swift does https://review.openstack.org/222263 | 19:25 |
| clayg | oh hey - minwoob had a birthday (this is what you get when you give me your skype username) | 19:26 |
| *** Fin1te has joined #openstack-swift | 19:26 | |
| notmyname | yay minwoob | 19:26 |
| clayg | also creith is having a birthday - but he's probably like 65 now | 19:26 |
| jrichli | minwoob: you didn't even tell us in the Austin team! | 19:26 |
| * jrichli goes to find minwoob in person | 19:27 | |
| tdasilva | minwoob: happy birthday! | 19:27 |
| openstackgerrit | Merged openstack/swift: Fix invalid frag_index header in ssync_sender when reverting EC tombstones https://review.openstack.org/217830 | 19:28 |
| clayg | ah hah! take that gate! | 19:28 |
| minwoob | clayg: notmyname: jrichli: tdasilva: Thanks!!! | 19:28 |
| hrou | Happy birthday minwoob ! But us ibmer's should be the first to know ; ) | 19:28 |
| minwoob | hrou: :P | 19:29 |
| clayg | torgomatic: the ring is so hard :'( | 19:29 |
| clayg | there's like a loop that yeilds part, replica - but it currently does it (0, 0), (1, 0), ... (2**20, 0), (0, 1), (1, 1) .. | 19:30 |
| clayg | and I need it go (0, 0), (0, 1), ... (0, <replica-count>), (1, 0), (1, 1) ... (2**20, <replica-count>) - but somehow this is hard for me :'( | 19:31 |
| clayg | MOAR COFFEE | 19:32 |
| *** rjaiswal has joined #openstack-swift | 19:40 | |
| notmyname | acoles: https://www.youtube.com/watch?t=19&v=fHxO0UdpoxM | 19:45 |
| notmyname | better link https://www.youtube.com/watch?v=fHxO0UdpoxM | 19:46 |
| jrichli | wow! thats a real place? | 19:51 |
| *** tongli has quit IRC | 19:54 | |
| wbhuber | did he say the word at 0:12 - 0:13? i couldnt tell by lip-reading him. the captions didn't get the town name right either. jrichli: it is: https://en.wikipedia.org/wiki/Llanfairpwllgwyngyll | 19:55 |
| notmyname | we could ask cschwede about https://en.wikipedia.org/wiki/Bielefeld_Conspiracy | 19:55 |
| notmyname | wbhuber: yeah, he said it and pretty much nailed it, from what I can tell. the full longer version of the name, not the short one in the wikipedia title | 19:56 |
| *** jlhinson has joined #openstack-swift | 19:56 | |
| jrichli | notmyname: lol. I learn something new everyday :-) | 19:57 |
| *** jerrygb has quit IRC | 19:58 | |
| clayg | jrichli: notmyname's good for that! | 19:59 |
| jrichli | so i have noticed. gotta give a phone interview now ... | 20:00 |
| notmyname | jrichli: ask them if Bielefeld exists | 20:00 |
| acoles | notmyname: we should have a hackathon there | 20:01 |
| clayg | why do lists not have a get(index) - stupid arrays | 20:01 |
| *** gyee has joined #openstack-swift | 20:01 | |
| notmyname | customs: "where are you going?" me: ".....I don't know how to say..." customs: "right. wales. carry on." | 20:02 |
| peluse | clayg, just reading scrollback... have you looked at the latest rebased alt_frags? | 20:02 |
| clayg | peluse: nope - nor the other thing you sent me in that email - i'm a total slacker :'( | 20:02 |
| peluse | BTW if I had to choose I'd probably want opt GETs first as well... | 20:02 |
| peluse | clayg, yeah, uh huh :) | 20:03 |
| peluse | clayg, but the update to get alt_frags rebased wasn't horrible and acoles is fixing it up as well so... its likely much closer to being suitable to land | 20:04 |
| acoles | peluse: clayg yeah i will push some changes for alt frgs then we can all take stock and see how best to proceed | 20:05 |
| clayg | peluse: that would be awesome - as you pointed out - i'm not the authority on the current state - no one should listen to me (this is good advice in most contexts) | 20:05 |
| acoles | clayg: we don't :P :PP | 20:05 |
| clayg | acoles: well it sounds like peluse is giving his blessing that the .durable timeout/optomistic get enhancement is a higher *priority* as a strategy - but tactically they may be reasons to do something else dependently first | 20:06 |
| acoles | trye | 20:06 |
| acoles | true | 20:06 |
| *** pberis has quit IRC | 20:07 | |
| peluse | clayh, yup that's my take | 20:07 |
| acoles | clayg: peluse i'm going to have to wrap up soon - are you in a rush to see alt frags/optimistic gets stuff today or can i push to gerrit tomorrow? | 20:07 |
| peluse | acoles, in the next 20 min would be great! | 20:07 |
| peluse | JK :) | 20:07 |
| peluse | acoles, tomorrow is awesome | 20:08 |
| acoles | lol | 20:08 |
| peluse | acoles, my main focus this week is still getting the EC perf tests done and shared. Then I'll start jumping in on the recon bugs that ctennis has been goijng hogwild entering :) | 20:09 |
| peluse | torgomatic, thanks for the review earlier BTW... | 20:09 |
| acoles | peluse: k. just didn't want y'all cursing me ;) | 20:10 |
| peluse | acoles, but when we do you know we do with with an accent... | 20:10 |
| acoles | torgomatic: oh yeah, i never got the fish/goals/school thing? | 20:11 |
| acoles | peluse: "jolly poor show" ;) | 20:11 |
| acoles | good night | 20:12 |
| torgomatic | acoles: just that fish goals rhymes with fish bowls, but fish travel in schools. not much to it. | 20:12 |
| torgomatic | peluse: sure | 20:12 |
| acoles | torgomatic: shoals rhymes with goals and bowls and coles | 20:14 |
| clayg | i work with a bunch of dorks | 20:14 |
| hrou | timburke, fyi I got some #s from a swift3 run I did, I'm going to drop you a note to share (and see if they're close to what you saw). | 20:16 |
| *** acoles is now known as acoles_ | 20:17 | |
| *** delattec has quit IRC | 20:20 | |
| *** cdelatte has quit IRC | 20:20 | |
| peluse | ctennis, you there? | 20:20 |
| *** dustins has quit IRC | 20:26 | |
| *** dustins has joined #openstack-swift | 20:28 | |
| *** jerrygb has joined #openstack-swift | 20:30 | |
| *** T0m_ has joined #openstack-swift | 20:31 | |
| clayg | sooooo many tests have replica count > len(devices) - grrrrr | 20:38 |
| clayg | rebalance is about to just start to say piss off when len(devices) < replica_count | 20:39 |
| *** aix has joined #openstack-swift | 20:40 | |
| *** bill_az_ has quit IRC | 20:43 | |
| jrichli | torgomatic acoles clayg: lol. and trolls | 20:43 |
| clayg | jrichli: hehehehehhe | 20:44 |
| torgomatic | clayg: yeah, test authors (myself included) got real lazy in there | 20:44 |
| clayg | I can't wait to write this commit message "Just say no to stupid rings" | 20:44 |
| clayg | RingValidationError: Replica count of 3 requires more than 2 devices | 20:45 |
| clayg | ^ anyone want to suggest a more flippant error message? | 20:45 |
| pgbridge | seems like that's about as flippant as you can get while still being somewhat professional :) | 20:46 |
| torgomatic | just append ", you dope" | 20:46 |
| torgomatic | oh, or this link: https://en.wikipedia.org/wiki/Pigeonhole_principle | 20:47 |
| clayg | hehehehhe - I like that or even -> http://lmgtfy.com/?q=pigeonhole+principle | 20:48 |
| torgomatic | YES | 20:49 |
| pgbridge | we got a winner | 20:51 |
| *** flwang1 has joined #openstack-swift | 20:53 | |
| clayg | bah, this is such a rabbit hole | 20:58 |
| *** prometheanfire has left #openstack-swift | 20:59 | |
| *** dustins has quit IRC | 20:59 | |
| clayg | so I add a few devices to a test - and instead of moving 87 parts now it moves 89 - which is *stupid* because only 86 parts go into the new region - so even the existing test was validating "oh and randomly one useless parition movement happens too" | 20:59 |
| notmyname | clayg: weren't a lot of those tests pretty fragile just so you know that things change. ie they might only test that things change, not that things are right (scary) | 21:01 |
| openstackgerrit | Merged openstack/swift: Pin PyECLib to 1.0.7 https://review.openstack.org/222237 | 21:01 |
| clayg | well sort of - i mean a fair number of them have moved into the relam of - "stand back and make some general observations about things not being terrible" | 21:01 |
| *** [1]eranrom has joined #openstack-swift | 21:04 | |
| *** eranrom has quit IRC | 21:04 | |
| *** [1]eranrom is now known as eranrom | 21:04 | |
| *** eranrom has quit IRC | 21:08 | |
| *** pberis has joined #openstack-swift | 21:13 | |
| *** panaflex has joined #openstack-swift | 21:24 | |
| *** jrichli has quit IRC | 21:25 | |
| panaflex | Question - what happens to a REST call in progress when authentication times out? | 21:25 |
| *** Fin1te has quit IRC | 21:25 | |
| *** wbhuber has quit IRC | 21:27 | |
| clayg | yay torgomatic is the best! | 21:27 |
| notmyname | panaflex: assuming it's already past the authorize step in swift, it completes | 21:27 |
| *** pberis2 has joined #openstack-swift | 21:27 | |
| clayg | well authn would be "get the identity of the token" - so if it's not in cache and it has to ask the remote store - and that raises a timeout - hell if I know - depends on the auth middleware? | 21:28 |
| panaflex | We setup an openstack swift system and set the auth timeout to 5 minutes, and we’re getting “This server could not verify that you are authorized to access the document you requested” with the new auth token for 1 call. Then everything works fine. | 21:30 |
| panaflex | We’re using swath | 21:31 |
| clayg | swiftstack ldap middleware does a 503 | 21:31 |
| clayg | I was going to check swauth next! | 21:31 |
| notmyname | "authentication times out" <-- is that the request from swift to the auth system fails or that the token expires? | 21:32 |
| panaflex | We’re uploading large files that span the auth timeout | 21:33 |
| notmyname | token expiry? | 21:33 |
| panaflex | Yup | 21:33 |
| clayg | swauth turns it into a default denied - it just calls req.get_response(self.app) - and intnerally if the app ends up timing out (like on node connections or w/e) it'll make a 503 | 21:33 |
| clayg | oh... | 21:34 |
| clayg | notmyname: good work - i was totally looking into a different question | 21:34 |
| clayg | notmyname: that's probably just the good 'ol auth after *lo bug | 21:34 |
| panaflex | So say uploading 200G takes 20 minutes, at 5 minutes in we expire. The retry then gets new token and seems to fail | 21:34 |
| notmyname | what retry? | 21:35 |
| panaflex | I have written a library using libcurl to sync data to/from swift. | 21:35 |
| notmyname | you start a request at time T. the token expires at T+300. the request finishes at T+3600 | 21:36 |
| panaflex | Yup, exactly | 21:36 |
| notmyname | ok, so you have only one request and response there | 21:36 |
| clayg | is it a bulk request? | 21:36 |
| notmyname | so what retry are you talking about? | 21:36 |
| panaflex | The one I’m about to write to correct the issue | 21:36 |
| notmyname | are you getting an error back from the first request? | 21:37 |
| panaflex | I am not getting an HTTP error - but that said I need to log my headers better to verify that. | 21:39 |
| notmyname | ok | 21:39 |
| peluse | notmyname, what's up w/the new pyeclib missing .h file thingy (or were you about to ask me the same thing)? | 21:41 |
| notmyname | the new request that you initiate at T+301 is getting an error, then | 21:41 |
| panaflex | Yes, that’s right | 21:41 |
| notmyname | peluse: I think the summary is something like "dependencies are hard" | 21:42 |
| notmyname | panaflex: because you're using the same token as the first request | 21:42 |
| panaflex | No - what happens is that the request finishes but the file isn’t in object storage. | 21:42 |
| notmyname | the first request? or the second one? | 21:43 |
| *** ccavanna has quit IRC | 21:43 | |
| notmyname | peluse: I think it's "just" that the version needed to be capped for kilo (and juno). since it wasn't, things broke because it ended up trying to do a downgrade | 21:44 |
| notmyname | peluse: I /think/ that's the issue. kevin and tsg have been looking at it | 21:44 |
| peluse | notmyname, OK cool | 21:45 |
| notmyname | panaflex: the second request is to do the same work as the first request, right? so your second request gets a 401 and then you make a third request to check on the result and you don't see the data in the cluster (the first request still hasn't completed) | 21:46 |
| notmyname | panaflex: is that correct? | 21:46 |
| panaflex | Yeah, I’m going off 2nd hand info but that’s what it looks like. | 21:46 |
| notmyname | panaflex: if that's the case, your second request seems completely superfluous. you don't need to retry until you get an error response back from swift | 21:47 |
| openstackgerrit | Merged openstack/python-swiftclient: make ClientException.http_status default to None rather than 0 https://review.openstack.org/220697 | 21:47 |
| notmyname | panaflex: and since the first request still hasn't finished by the time you're doing the cheking in the third request, it makes sense that you aren't finding any data | 21:48 |
| panaflex | The data never appears however, even after completion. | 21:48 |
| notmyname | panaflex: is this your cluster or are you using a third party cluster that you don't have internal access to (eg like logs) | 21:48 |
| panaflex | Using softlayer and local cluster. Issue appears on local cluster using the 5 minute timeout to replicate the issue. | 21:49 |
| *** T0m_ has left #openstack-swift | 21:50 | |
| notmyname | panaflex: ok, so you need to grab the x-trans-id head on the response and then look in the logs on your local cluster for that value | 21:50 |
| panaflex | I guess I need to get much better data and logs on the issue. | 21:50 |
| panaflex | Yes | 21:50 |
| panaflex | Thanks! I’ll pull the logs and be back. | 21:51 |
| *** AndreiaKumpera has quit IRC | 21:58 | |
| *** wbhuber has joined #openstack-swift | 22:01 | |
| mattoliverau | Morning from Sydney | 22:03 |
| notmyname | hello mattoliverau | 22:08 |
| *** marzif has joined #openstack-swift | 22:10 | |
| *** jkugel has quit IRC | 22:12 | |
| *** marzif_ has joined #openstack-swift | 22:18 | |
| openstackgerrit | Merged openstack/swift: Trivial Key Master for encryption https://review.openstack.org/193749 | 22:18 |
| *** marzif has quit IRC | 22:18 | |
| *** pgbridge has quit IRC | 22:19 | |
| *** setmason has joined #openstack-swift | 22:19 | |
| *** pgbridge has joined #openstack-swift | 22:20 | |
| *** remix_tj has quit IRC | 22:21 | |
| *** bill_az has joined #openstack-swift | 22:26 | |
| *** remix_tj has joined #openstack-swift | 22:27 | |
| *** lcurtis has quit IRC | 22:28 | |
| *** jlhinson has quit IRC | 22:30 | |
| *** ctrath has quit IRC | 22:30 | |
| *** remix_tj has quit IRC | 22:42 | |
| openstackgerrit | Zack M. Davis proposed openstack/python-swiftclient: enable autodocumentation for utils module; docstring fixes https://review.openstack.org/222391 | 22:43 |
| *** cdelatte has joined #openstack-swift | 22:44 | |
| *** remix_tj has joined #openstack-swift | 22:45 | |
| *** hrou has quit IRC | 22:46 | |
| *** remix_tj has quit IRC | 22:49 | |
| *** remix_tj has joined #openstack-swift | 22:51 | |
| *** marzif_ has quit IRC | 22:52 | |
| * panaflex Ok, pulled the logs and here’s my confusion… | 22:54 | |
| panaflex | Sep 10 09:30:56 localhost proxy-server: 172.16.0.119 172.16.0.119 10/Sep/2015/13/30/56 PUT /v1/AUTH_8c4e4c2f-60e7-4137-a6a8-5fddf1c9e735/accounting5/1GBtext2.txt-XshareX-1 HTTP/1.0 401 - - AUTH_tk1d3175157... - 131 - tx7d1bd19fd5884c92b1a93-0055f18610 - 0.0424 - - 1441891856.638288021 1441891856.680660009 - | 22:55 |
| panaflex | Sep 10 09:31:15 localhost proxy-server: 172.16.0.119 172.16.0.119 10/Sep/2015/13/31/15 PUT /v1/AUTH_8c4e4c2f-60e7-4137-a6a8-5fddf1c9e735/accounting5/1GBtext2.txt-XshareX-2 HTTP/1.0 201 - - mark:markid%2CAUTH... 525264500 - - tx6546063af03d4b19983c6-0055f18618 - 10.3158 - - 1441891864.728559971 1441891875.044395924 0 | 22:55 |
| panaflex | First item starts at 9:30:56, gets a 401 unauth, second item starts at 9:31:15 and gets a 201. These were not run parallel | 22:56 |
| panaflex | Note that auth token is the same for both, same container | 22:57 |
| *** minwoob has quit IRC | 22:58 | |
| *** km has joined #openstack-swift | 22:59 | |
| torgomatic | the auth token is not the same for both | 22:59 |
| torgomatic | at least, not according to those logs there; one token starts with "AUTH_tk" and the other with "mark:markid" | 23:01 |
| panaflex | Ok, thank you. | 23:02 |
| openstackgerrit | Merged openstack/swift: go: change dir permissions to what swift does https://review.openstack.org/222263 | 23:08 |
| *** cdelatte has quit IRC | 23:08 | |
| panaflex | Thanks for the help on the logs | 23:10 |
| *** pberis has quit IRC | 23:12 | |
| *** pberis2 is now known as pberis | 23:12 | |
| *** panaflex has quit IRC | 23:14 | |
| torgomatic | that's not to say that the client isn't *sending* the same ones, just that by the time it hits logs, it's different... keystoneauth may mangle tokens somehow, I don't know | 23:15 |
| openstackgerrit | Merged openstack/swift: go: Collecting runtime metrics https://review.openstack.org/216855 | 23:16 |
| *** pberis2 has joined #openstack-swift | 23:16 | |
| *** cdelatte has joined #openstack-swift | 23:17 | |
| *** delattec has joined #openstack-swift | 23:18 | |
| *** delattec has quit IRC | 23:19 | |
| *** cdelatte has quit IRC | 23:19 | |
| openstackgerrit | Merged openstack/swift: Better error output in TestHealthCheck https://review.openstack.org/212188 | 23:20 |
| *** kei_yama has joined #openstack-swift | 23:31 | |
| *** kaliya has quit IRC | 23:37 | |
| *** hrou has joined #openstack-swift | 23:49 | |
| *** aix has quit IRC | 23:53 | |
| *** pberis2 has quit IRC | 23:54 | |
| *** ho has joined #openstack-swift | 23:57 | |
| * mattoliverau gets the feeling torgomatic has spend alot of time looking at proxy logs :P | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!