| *** _JZ_ has quit IRC | 00:23 | |
| redbo | notmyname: I've never used that one. I have a slightly janky one in the branch that I use for benchmarks and stuff, https://github.com/openstack/swift/blob/feature/hummingbird/go/client/userclient.go | 00:23 |
|---|---|---|
| *** lyrrad has quit IRC | 00:23 | |
| *** klamath has joined #openstack-swift | 00:24 | |
| *** klamath has quit IRC | 00:39 | |
| *** klamath has joined #openstack-swift | 00:40 | |
| *** catintheroof has joined #openstack-swift | 00:47 | |
| *** klamath has quit IRC | 00:54 | |
| *** Suyash has quit IRC | 00:56 | |
| *** catintheroof has quit IRC | 01:07 | |
| kota_ | good morning | 01:14 |
| mattoliverau | kota_: Morning | 01:14 |
| kota_ | Amazing, Putter problem has been already fixed!?!? | 01:14 |
| kota_ | mattoliverau: o/ | 01:14 |
| *** klamath has joined #openstack-swift | 01:38 | |
| *** Suyash has joined #openstack-swift | 01:38 | |
| kota_ | notmyname: Did acoles fix both Putter and MIMEPutter in the crypto-review branch? | 01:43 |
| kota_ | I'm wondering if we should fix ECPutter which will be deprecated in the crypto-review work at master, or not. | 01:44 |
| timburke | kota_: i'm leaning toward letting crypto merge and fix it. saves acoles_ some merge conflicts | 01:46 |
| kota_ | timburke: ok, probably we can do it by adding closes-bug tag into the crypto commit. | 01:48 |
| kota_ | timburke: and agreed for your opinion, I also want to save acoles for complicated management of patch chain as possible. | 01:49 |
| notmyname | I definitely agree on saving acoles_ from extra work | 01:57 |
| *** klamath has quit IRC | 01:58 | |
| notmyname | and my statement was sortof asking for confirmation. I'm not 100% sure the issue is closed, but what previously showed it on my SAIO is no longer showing it | 01:58 |
| notmyname | and yeah, I agree about fixing it on crypto and letting that land and fix it on master | 01:59 |
| kota_ | notmyname: alright, I still have sort of points to be fixed (not yet digg and not yet look at the newest crypt-review branch though), I will look at crypt fist and then make sure if it would be 100% fixed :) | 02:05 |
| kota_ | Dump my thought right now, | 02:05 |
| kota_ | the patch in bug 1594739 can resolve the issue but it *might* not be 100%, IMHO. | 02:06 |
| openstack | bug 1594739 in OpenStack Object Storage (swift) "EC: ECPutter doesn't close backend HTTPConnection when exception raised" [Undecided,Confirmed] https://launchpad.net/bugs/1594739 | 02:06 |
| kota_ | because, the finally statemet to close backend connection will run through the putters list | 02:07 |
| kota_ | however, the putters list *may* be updated during around transfer_data | 02:07 |
| kota_ | if the putter has the flag "failed", the putter will be removed from the list. | 02:07 |
| kota_ | Right now, I'm not sure if the removed putter also is ensured to close the backend connection (or not). | 02:08 |
| kota_ | I'm trying to figure out it today. | 02:09 |
| kota_ | end - dumping :/ | 02:09 |
| *** jamielennox is now known as jamielennox|away | 02:16 | |
| *** jamielennox|away is now known as jamielennox | 02:17 | |
| kota_ | alright, looks like acoles did it. great. | 02:36 |
| openstackgerrit | Kazuhiro MIYAHARA proposed openstack/swift: WIP: Swift Automated Tiering https://review.openstack.org/287057 | 02:36 |
| kota_ | ah, maybe one more place, we could drop the failed putter? hmm...look at. | 02:38 |
| kota_ | hmm... it seems ok because it just re-assignment as a local variable, I don't like to overwrite a method arg variable with a new value though. | 02:48 |
| *** _JZ_ has joined #openstack-swift | 02:59 | |
| *** siva_krish has joined #openstack-swift | 03:08 | |
| *** siva_krish has left #openstack-swift | 03:08 | |
| *** sgundur1 has left #openstack-swift | 03:23 | |
| *** gyee has quit IRC | 03:24 | |
| *** links has joined #openstack-swift | 03:29 | |
| *** catintheroof has joined #openstack-swift | 03:44 | |
| *** dmorita has quit IRC | 03:55 | |
| *** dmorita has joined #openstack-swift | 03:58 | |
| *** catintheroof has quit IRC | 04:02 | |
| *** dmorita has quit IRC | 04:03 | |
| *** dmorita has joined #openstack-swift | 04:17 | |
| *** klrmn has quit IRC | 04:18 | |
| *** dmorita has quit IRC | 04:21 | |
| timburke | jrichli: go to sleep! | 04:28 |
| *** psachin has joined #openstack-swift | 04:30 | |
| *** sheel has joined #openstack-swift | 04:30 | |
| *** nadeem has joined #openstack-swift | 04:31 | |
| *** ppai has joined #openstack-swift | 04:31 | |
| jrichli | timburke: I was about to, but I decided I'd stay up just a bit more. I was already in bed the last time you told me to sleep :-) | 04:52 |
| timburke | good | 04:52 |
| timburke | :P | 04:52 |
| *** SkyRocknRoll has joined #openstack-swift | 04:53 | |
| *** rcernin has joined #openstack-swift | 04:56 | |
| *** Jeffrey4l_ has quit IRC | 05:01 | |
| *** Suyash has quit IRC | 05:08 | |
| *** rcernin has quit IRC | 05:16 | |
| *** ChubYann has quit IRC | 05:32 | |
| *** chlong has quit IRC | 05:34 | |
| *** chlong has joined #openstack-swift | 05:47 | |
| *** rcernin has joined #openstack-swift | 05:54 | |
| *** _JZ_ has quit IRC | 05:56 | |
| *** Jeffrey4l_ has joined #openstack-swift | 06:09 | |
| *** ozeri has joined #openstack-swift | 06:24 | |
| *** jmccarthy has quit IRC | 06:41 | |
| *** jmccarthy has joined #openstack-swift | 06:42 | |
| *** tesseract- has joined #openstack-swift | 06:47 | |
| *** hugokuo_ has joined #openstack-swift | 06:48 | |
| *** csmart_ has joined #openstack-swift | 06:51 | |
| *** dmellado_ has joined #openstack-swift | 06:52 | |
| *** charz has joined #openstack-swift | 06:54 | |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/swift: [WIP] Testing latest u-c https://review.openstack.org/318441 | 06:59 |
| openstackgerrit | Swapnil Kulkarni (coolsvap) proposed openstack/swift: [WIP] Testing latest u-c https://review.openstack.org/318441 | 06:59 |
| *** SkyRocknRoll_ has joined #openstack-swift | 07:00 | |
| *** SkyRocknRoll has quit IRC | 07:04 | |
| *** csmart has quit IRC | 07:04 | |
| *** mmcardle has quit IRC | 07:04 | |
| *** dmellado has quit IRC | 07:04 | |
| *** charz_ has quit IRC | 07:04 | |
| *** hugokuo has quit IRC | 07:04 | |
| *** hugokuo_ is now known as hugokuo | 07:04 | |
| *** permalac_ has quit IRC | 07:06 | |
| *** geaaru has joined #openstack-swift | 07:09 | |
| *** mmcardle has joined #openstack-swift | 07:14 | |
| *** nadeem has quit IRC | 07:15 | |
| *** pcaruana has joined #openstack-swift | 07:22 | |
| *** chlong has quit IRC | 07:25 | |
| *** rledisez has joined #openstack-swift | 07:25 | |
| *** nadeem has joined #openstack-swift | 07:29 | |
| *** nadeem has quit IRC | 07:41 | |
| *** geaaru has quit IRC | 07:47 | |
| *** ozeri has quit IRC | 07:52 | |
| *** geaaru has joined #openstack-swift | 08:00 | |
| *** acoles_ is now known as acoles | 08:16 | |
| *** dmk0202 has joined #openstack-swift | 08:23 | |
| *** ma9 has joined #openstack-swift | 08:27 | |
| ma9 | Hi | 08:27 |
| ma9 | I managed to connect with Cyberduck to my Swift instance… Now I'm tryng to connect, still with Cyberduck, using the S3 protocol, but I get the following error: | 08:28 |
| ma9 | Multiple Choices. Request Error [{"versions": {"values": [{"status": "stable", "updated": , "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": [{"href": "rel": "self"}]}, {"status": "stable", "updated": "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href": "rel": "self"}, {"h | 08:28 |
| ma9 | any idea of what I could be doing wrong? | 08:29 |
| acoles | kota_: I think that the only place a putter is removed from the list of putter during transfer_data is in the send_chunk function, and in that case I close the putter. Otherwise all putters should still be int he list handled by the finally clauses. But please tell me if I missed any. Thanks. | 08:33 |
| acoles | kota_: were you looking at line 816. that assignment is confusing, I may change the name of new variable to ok_putters | 08:34 |
| kota_ | acoles: hehe, I'm with. Just leaving comments for that :D | 08:37 |
| *** bapalm has quit IRC | 08:37 | |
| acoles | great | 08:37 |
| kota_ | acoles: but thanks for your explanation :) | 08:38 |
| kota_ | I was trying to ensure connection close if failed set to True, but it is getting bigger and I'm realizing it's overkill for us, so I gave up it there :/ | 08:39 |
| acoles | kota_: thanks for the comments. yes i'll fix line 816. BTW I did add some test assertions to check that conns get closed, search test_obj.py for capture_expect | 08:42 |
| *** bapalm has joined #openstack-swift | 08:42 | |
| acoles | kota_: oh, i think you already spotted that :) | 08:42 |
| kota_ | m9: it looks authentication error, are you using keystone? | 08:43 |
| kota_ | m9: or trying to Swift protocol might save your time. | 08:43 |
| kota_ | m9: honesty I'd like to continue to talk with you for helping because I'm maintaining s3 wrappers on top of swift but I'm leaving in 15 minutes, sorry. | 08:44 |
| kota_ | acoles: yeah, it's great to assert the backend connection closed :) | 08:45 |
| kota_ | acoles: it might be better to assert conn.resp is None, too? | 08:46 |
| kota_ | actually, we need to throw the response reference to kick the connection close :P | 08:46 |
| acoles | kota_: timburke ack re putter.resp, conn.resp...its that way on master, I'm sure it could be cleaner, I'm not sure how much clean up to do here vs leaving things to look vaguely familiar with master for reviewers ? | 08:47 |
| acoles | kota_: yeah, the test can't verify that an HTTPConnection actually closed because it is testing with Fake connections, but it's better than nothing | 08:47 |
| kota_ | acoles: I bet we can resolve in followup for putter.resp cleaner ;) | 08:49 |
| ma9 | kota_: thanks for the answer! sorry for my delay… yes I | 08:49 |
| acoles | kota_: OIC, yes we could assert that FakeConn.resp==None | 08:49 |
| ma9 | I'm using keystone | 08:49 |
| ma9 | I already tried with the Swift interface and it works… but I wanted to try s3 ;) | 08:49 |
| acoles | kota_: I bet timburke has already written all the follow ups ;) | 08:49 |
| ma9 | the port address I'm using is the one from Keystone 35357 | 08:50 |
| kota_ | m9: alright, if you could make some comments at launchpad bug report (or ask?), it could be easy to truck for us. probably I or timburke can see the detail later. | 08:50 |
| kota_ | acoles: great, that should be fixed while I am asleep :) | 08:51 |
| acoles | kota_: how long are you planning to sleep for ? ;) | 08:52 |
| ma9 | ok | 08:52 |
| *** mvk has joined #openstack-swift | 08:52 | |
| acoles | kota_: so do you think it is ok to add closes-bug 1594739 to patch 328204 | 08:56 |
| openstack | bug 1594739 in OpenStack Object Storage (swift) "EC: ECPutter doesn't close backend HTTPConnection when exception raised" [Undecided,Confirmed] https://launchpad.net/bugs/1594739 | 08:56 |
| patchbot | acoles: https://review.openstack.org/#/c/328204/ - swift (feature/crypto-review) - Support for http footers - Replication and EC | 08:56 |
| acoles | ? | 08:56 |
| kota_ | I think so | 09:03 |
| *** ouchkernel has quit IRC | 09:03 | |
| kota_ | It's ok i mean | 09:04 |
| *** ouchkernel has joined #openstack-swift | 09:04 | |
| kota_ | usually i prefer to make them separate, but in this case, fixing master will make us frustrating | 09:05 |
| kota_ | Just paying small pain if we want to backport | 09:07 |
| kota_ | Sorry its time to leave | 09:08 |
| acoles | kota_: k. thanks! | 09:16 |
| *** Jeffrey4l__ has joined #openstack-swift | 09:25 | |
| *** Jeffrey4l_ has quit IRC | 09:26 | |
| *** ppai has quit IRC | 09:34 | |
| *** mvk has quit IRC | 09:35 | |
| *** Jeffrey4l__ has quit IRC | 09:35 | |
| *** ahale_ has quit IRC | 09:36 | |
| *** kei_yama has quit IRC | 09:41 | |
| *** bapalm has quit IRC | 09:48 | |
| *** ppai has joined #openstack-swift | 09:48 | |
| *** bapalm has joined #openstack-swift | 09:54 | |
| *** mvk has joined #openstack-swift | 09:56 | |
| *** ahale has joined #openstack-swift | 10:02 | |
| *** daemontool has joined #openstack-swift | 10:07 | |
| *** hosanai has quit IRC | 10:14 | |
| *** daemontool has quit IRC | 10:19 | |
| *** Jeffrey4l has joined #openstack-swift | 10:27 | |
| *** ppai has quit IRC | 10:30 | |
| *** ozeri has joined #openstack-swift | 10:37 | |
| *** Jeffrey4l has quit IRC | 10:40 | |
| *** ppai has joined #openstack-swift | 10:46 | |
| acoles | timburke: oops, I didn't publish my replies to your comments on v3 of patch 328205 (I'll blame gerrit for hiding the fact that there are unpublished comments on older versions, which I am sure the old gerrit didn't do!) | 11:01 |
| patchbot | acoles: https://review.openstack.org/#/c/328205/ - swift (feature/crypto-review) - Make container update override headers persistent | 11:01 |
| acoles | timburke: in particular I'd left a question for you about your test patch | 11:02 |
| acoles | timburke: re. how middlewares should set override headers, I'll attempt some more words but I don't think we can prescribe, and encryption would be an immediate exception to the obvious guidance of "use setdefault" | 11:08 |
| *** chlong has joined #openstack-swift | 11:21 | |
| *** ppai has quit IRC | 11:35 | |
| *** ma91 has joined #openstack-swift | 11:46 | |
| *** ma9 has quit IRC | 11:48 | |
| *** ppai has joined #openstack-swift | 11:48 | |
| *** chlong has quit IRC | 11:51 | |
| *** ppai has quit IRC | 11:59 | |
| *** raildo-afk is now known as raildo | 12:09 | |
| *** ppai has joined #openstack-swift | 12:15 | |
| *** virus has joined #openstack-swift | 12:16 | |
| *** virus has left #openstack-swift | 12:20 | |
| *** cbartz has joined #openstack-swift | 12:23 | |
| *** d0ugal has quit IRC | 12:25 | |
| *** d0ugal has joined #openstack-swift | 12:25 | |
| *** hseipp has joined #openstack-swift | 12:26 | |
| *** Jeffrey4l has joined #openstack-swift | 12:26 | |
| *** SkyRocknRoll_ has quit IRC | 12:30 | |
| *** pauloewerton has joined #openstack-swift | 12:34 | |
| *** tanee is now known as tanee_away | 12:39 | |
| *** tanee_away is now known as tanee | 12:39 | |
| *** tanee is now known as tanee_away | 12:39 | |
| *** ujjain has quit IRC | 12:47 | |
| *** ujjain has joined #openstack-swift | 12:52 | |
| *** ujjain has joined #openstack-swift | 12:52 | |
| *** dmorita has joined #openstack-swift | 12:56 | |
| *** dmorita has quit IRC | 13:01 | |
| *** chlong has joined #openstack-swift | 13:06 | |
| *** permalac has joined #openstack-swift | 13:07 | |
| *** links has quit IRC | 13:09 | |
| *** StraubTW has joined #openstack-swift | 13:11 | |
| *** BigWillie has joined #openstack-swift | 13:24 | |
| ma91 | how do I see who is the "owner" of an object in swift? or better, who first uploaded the object? The command line swift stat does not show it... | 13:33 |
| *** siva_krish has joined #openstack-swift | 13:37 | |
| *** klamath has joined #openstack-swift | 13:43 | |
| *** mwheckmann has joined #openstack-swift | 13:43 | |
| *** chlong has quit IRC | 13:49 | |
| ma91 | is the ownership specified only at "project/tenant" level? | 13:49 |
| ma91 | i'm asking because I would like to see who is the person who uploaded a specific file | 13:50 |
| *** ppai has quit IRC | 13:52 | |
| *** cdelatte has joined #openstack-swift | 13:57 | |
| tdasilva | ma91: that information is not stored with the object. so if multiple users have access to a container, I don't think you could know | 13:58 |
| ma91 | ok thanks for the info | 14:00 |
| *** ametts has joined #openstack-swift | 14:01 | |
| *** siva_krish has quit IRC | 14:03 | |
| *** chlong has joined #openstack-swift | 14:05 | |
| *** siva_krish has joined #openstack-swift | 14:07 | |
| StraubTW | Anyone know of a FUSE Swift file system other than https://github.com/hironobu-s/swiftfs ? | 14:13 |
| *** chsc has joined #openstack-swift | 14:20 | |
| *** chsc has joined #openstack-swift | 14:20 | |
| *** sgundur1 has joined #openstack-swift | 14:23 | |
| *** ozeri has quit IRC | 14:31 | |
| *** tanee_away is now known as tanee | 14:32 | |
| *** tanee is now known as tanee_away | 14:33 | |
| *** tanee_away is now known as tanee | 14:35 | |
| *** ma91 has left #openstack-swift | 14:43 | |
| *** ma91 has joined #openstack-swift | 14:43 | |
| *** siva_krish has quit IRC | 14:51 | |
| *** zul has quit IRC | 14:54 | |
| *** zul has joined #openstack-swift | 14:55 | |
| *** SkyRocknRoll has joined #openstack-swift | 14:55 | |
| ma91 | tdasilva: so from what I see with the swift stat command each container has an Account field | 14:56 |
| ma91 | that Account is a project/tenant…. so each container does not have a specific user as owner, but it's owned by a project | 14:57 |
| ma91 | so i guess there is no way to know what user uploaded a file.. | 14:57 |
| *** jistr is now known as jistr|mtg | 14:58 | |
| *** siva_krish has joined #openstack-swift | 15:01 | |
| *** klamath has quit IRC | 15:03 | |
| *** permalac has quit IRC | 15:08 | |
| *** rcernin has quit IRC | 15:09 | |
| *** chsc_ has joined #openstack-swift | 15:09 | |
| *** npf has joined #openstack-swift | 15:10 | |
| *** chsc has quit IRC | 15:10 | |
| *** bapalm has quit IRC | 15:10 | |
| *** bapalm has joined #openstack-swift | 15:10 | |
| *** yarkot1 has quit IRC | 15:10 | |
| *** hogepodge has quit IRC | 15:10 | |
| *** andymccr_ has quit IRC | 15:10 | |
| *** andymccr has joined #openstack-swift | 15:10 | |
| *** tesseract- has quit IRC | 15:10 | |
| *** yarkot1 has joined #openstack-swift | 15:11 | |
| *** hogepodge has joined #openstack-swift | 15:11 | |
| tdasilva | ma91: yeah, it's important to understand the distinction between users and accounts. multiple users could have access to an account and/or container | 15:11 |
| ma91 | ok | 15:13 |
| *** klrmn has joined #openstack-swift | 15:14 | |
| tdasilva | acoles: Hello! for the footer patch, are you planning to add Sam's patch to unify Putter and MIMEPutter or will that be a follow up? just wondering... | 15:17 |
| *** tanee is now known as tanee_away | 15:20 | |
| *** pcaruana has quit IRC | 15:22 | |
| *** rcernin has joined #openstack-swift | 15:24 | |
| acoles | tdasilva: hi. not today because (a) I have not yet had chance to review it and (b) based on jrichli comment I am wondering if there will be other opinions. but mainly I've not had time yet (been studying the debate on etag encryption on patch 328208) | 15:25 |
| patchbot | acoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption | 15:25 |
| acoles | tdasilva: I have made a few changes to patch 328204 that may (or may not) get pushed today, nothing major though | 15:26 |
| patchbot | acoles: https://review.openstack.org/#/c/328204/ - swift (feature/crypto-review) - Support for http footers - Replication and EC | 15:26 |
| *** psachin has quit IRC | 15:26 | |
| tdasilva | acoles: ok, no worries, thanks for the heads up | 15:27 |
| *** mwheckmann has quit IRC | 15:29 | |
| *** StraubTW has quit IRC | 15:31 | |
| *** jistr|mtg is now known as jistr | 15:32 | |
| *** StraubTW has joined #openstack-swift | 15:36 | |
| *** pcaruana has joined #openstack-swift | 15:36 | |
| *** cbartz has quit IRC | 15:37 | |
| *** arch-nemesis has joined #openstack-swift | 15:42 | |
| openstackgerrit | Petr Kovar proposed openstack/swift: Add install-guide for swift https://review.openstack.org/330070 | 15:46 |
| *** nadeem has joined #openstack-swift | 15:48 | |
| *** catintheroof has joined #openstack-swift | 15:49 | |
| *** Suyash has joined #openstack-swift | 15:50 | |
| *** chlong has quit IRC | 15:50 | |
| *** adu has joined #openstack-swift | 15:51 | |
| *** lyrrad has joined #openstack-swift | 15:52 | |
| notmyname | good morning | 15:55 |
| *** daemontool has joined #openstack-swift | 16:00 | |
| *** dmk0202 has quit IRC | 16:02 | |
| *** npf_ has joined #openstack-swift | 16:03 | |
| *** npf_ has quit IRC | 16:06 | |
| *** npf has quit IRC | 16:07 | |
| *** mwheckmann has joined #openstack-swift | 16:12 | |
| *** gyee has joined #openstack-swift | 16:13 | |
| *** links has joined #openstack-swift | 16:16 | |
| *** Suyash has quit IRC | 16:17 | |
| *** ma91 has left #openstack-swift | 16:19 | |
| *** dmorita has joined #openstack-swift | 16:29 | |
| *** klrmn has quit IRC | 16:29 | |
| *** _JZ_ has joined #openstack-swift | 16:32 | |
| *** nadeem has quit IRC | 16:33 | |
| *** nadeem has joined #openstack-swift | 16:34 | |
| *** Suyash has joined #openstack-swift | 16:34 | |
| *** daemontool has quit IRC | 16:37 | |
| acoles | timburke: jrichli I didn't get all the way through comments today but left some replies on patch 328208 | 16:38 |
| patchbot | acoles: https://review.openstack.org/#/c/328208/ - swift (feature/crypto-review) - Enable object body and metadata encryption | 16:38 |
| acoles | I will make remaining updates and plan to push fresh crypto-review patches tomorrow | 16:38 |
| *** links has quit IRC | 16:40 | |
| *** rcernin has quit IRC | 16:40 | |
| *** pcaruana has quit IRC | 16:43 | |
| *** chsc_ has quit IRC | 16:53 | |
| *** awelleck has joined #openstack-swift | 16:55 | |
| *** rledisez has quit IRC | 17:06 | |
| *** siva_krish has quit IRC | 17:06 | |
| *** Jeffrey4l has quit IRC | 17:08 | |
| *** mvk has quit IRC | 17:11 | |
| notmyname | on today's meeting agenda, the first topic we'll talk about is container sync. oshrit is raising these patches again, and there are now some (very nice looking) numbers from test results | 17:12 |
| notmyname | links are on https://wiki.openstack.org/wiki/Meetings/Swift | 17:13 |
| *** rcernin has joined #openstack-swift | 17:20 | |
| notmyname | redbo: any progress on getting hummingbird stuff launched with swift-init? | 17:20 |
| timburke | good morning | 17:21 |
| *** hseipp has quit IRC | 17:26 | |
| *** hseipp has joined #openstack-swift | 17:27 | |
| *** hseipp has quit IRC | 17:27 | |
| *** pauloewerton has quit IRC | 17:28 | |
| jrichli | acoles: ok, ill take a look | 17:29 |
| clayg | some dood did a pr to vsaio for aws support - pretty cool | 17:29 |
| clayg | he says he's using vsaio to test some backup software - pretty cool | 17:30 |
| clayg | notmyname: so your theory about app developers using vsaio wasn't too nuts after all!? | 17:30 |
| notmyname | :-) | 17:30 |
| notmyname | something something about stopped clocks | 17:31 |
| *** klrmn has joined #openstack-swift | 17:32 | |
| zaitcev | go go gadget Bing vsaio | 17:36 |
| jrichli | acoles: thanks for tracking down the code at http://osxr.org:8080/openssl/source/crypto/modes/ctr128.c#0080 | 17:37 |
| notmyname | funny story, this morning we got an email to a general swiftstack address from someone who is building a swift cluster on a bunch of raspberry pis and is seeing some errors. | 17:38 |
| *** pauloewerton has joined #openstack-swift | 17:38 | |
| *** mvk has joined #openstack-swift | 17:41 | |
| *** daemontool has joined #openstack-swift | 17:41 | |
| ahale | :) | 17:42 |
| tdasilva | acoles: if a middleware sends a POST request to update a transient metadata, will that erase all user metadata too? | 17:43 |
| timburke | jrichli: acoles: fwiw, i didn't really dig into the openssl implementation; i was just going off of our adjustments to iv for ranged requests and the observed behavior in a python REPL. how certain are we that the offset handling will be correct on both big- and little-endian platforms? maybe more to the point, how much do we care? | 17:43 |
| *** daemontool has quit IRC | 17:44 | |
| *** nadeem has quit IRC | 17:44 | |
| timburke | zaitcev: vagrant swift all in one - https://github.com/swiftstack/vagrant-swift-all-in-one | 17:44 |
| *** dmorita has quit IRC | 17:44 | |
| *** dmorita has joined #openstack-swift | 17:44 | |
| *** chsc has joined #openstack-swift | 17:44 | |
| *** chsc has joined #openstack-swift | 17:44 | |
| zaitcev | timburke: thanks, I would never guess, although I saw Clay using it | 17:45 |
| *** silor has joined #openstack-swift | 17:45 | |
| timburke | tdasilva: i'd expect so, yeah. would a middleware send a POST that wasn't some manipulation of a POST from a client, though? | 17:46 |
| tdasilva | timburke: I was thinking about that and it is a fair point, I don't think we have that situation in swift's middleware code today, but still I think that's an important enough information to be documented so that third-party middleware developers are aware of it. | 17:48 |
| timburke | yeah. i know timur was playing around with a middleware that would send a POST immediately following any PUTs as part of some metadata-search work. one nice thing about the crypto work is that shouldn't be necessary any more; he can just use footers | 17:50 |
| *** nadeem has joined #openstack-swift | 17:50 | |
| notmyname | "just" | 17:50 |
| timburke | notmyname: hey, it's not *so* hard! | 17:51 |
| notmyname | it's just typing! | 17:51 |
| timburke | exactly. | 17:51 |
| *** silor has quit IRC | 17:51 | |
| *** silor has joined #openstack-swift | 17:52 | |
| notmyname | if I understand correctly, the middleware installs the footer callback, then the obj controller will do the right thing and do the multipart mime dance with the object server | 17:52 |
| *** nadeem has quit IRC | 17:52 | |
| timburke | yup | 17:52 |
| notmyname | which also had the huge advantage of removing the need for coordination across multiple requests. it's all one request | 17:52 |
| timburke | better all-around | 17:53 |
| notmyname | it's so easy! why aren't you done yet timur?! | 17:53 |
| timburke | yeah, it's even better when it's *someone else's* typing. that's like zero-effort! | 17:54 |
| *** geaaru has quit IRC | 17:54 | |
| tdasilva | lol | 17:54 |
| notmyname | "easy is a word used to describe someone else's job" | 17:54 |
| redbo | raspberry pi seems like a terrible idea, there's no way to get reasonable bandwidth to drives. There are similar boards with sata interfaces like the banana pi. | 18:04 |
| *** silor has quit IRC | 18:04 | |
| timur | ha sorry, but timburke is right, the footer is what I wanted for the case where the middleware was extracting jpeg EXIF metadata on PUT and then doing a POST to apply it | 18:04 |
| redbo | notmyname: not really. My python is all rusty. | 18:04 |
| jrichli | you guys are making me want some pie! | 18:04 |
| notmyname | redbo: http://eavesdrop.openstack.org/irclogs/%23openstack-swift/%23openstack-swift.2016-06-09.log.html#t2016-06-09T16:48:11 ;-) | 18:06 |
| zaitcev | I don't quite understand what makes anyone thinkg that a board based on Allwinner A20 has a better storage access than Raspberry Pi. There's no PCI! You can't connect a SATA controller to that thing. | 18:10 |
| *** siva_krish has joined #openstack-swift | 18:10 | |
| notmyname | that is interesting. I hadn't heard of the banana pi before | 18:11 |
| jrichli | timburke: so you are speaking now to the offset handling - and not about the derived_iv method, right? | 18:12 |
| jrichli | timburke: when you ask how much we care, I am not sure if I know which part you are talking about. If we our calculation does not result in the proper offset, then encryption or decryption would fail to produce the correct message. | 18:12 |
| jrichli | (in a ranged get case against the body) | 18:12 |
| jrichli | timburke: I plan on digging into the openssl impl so I can think about adding some cases to our offset calcuation unit test so we can be more sure that endianess will all work-out. Good thought! | 18:12 |
| zaitcev | Banana Pi is in the same boat as TrimSlice 2 | 18:13 |
| timburke | jrichli: yeah, i was thinking about endianness and offset handling. it may be somewhat related to the derived_iv issues, though | 18:14 |
| *** ChubYann has joined #openstack-swift | 18:15 | |
| timburke | jrichli: i think i'd be in favor of always using md5(path) for derived_iv; it would ensure that ivs are reasonably well-distributed | 18:15 |
| notmyname | zaitcev: my interest in running swift on stuff like that is directly proportional to how many ones I get for free from a conference ;-) | 18:16 |
| timburke | jrichli: acoles was right, of course, that we only care about two blocks, so there's only one opportunity for overlap. i thought of that last night after i'd posted. still a concern, i think | 18:17 |
| jrichli | timburke: I agree on hash guarantees well-distributed, but I think maybe acoles may be wanting to avoid the hash, if possible, because of the performance hit. so more reason i'd like to get more familiar with the exact incrementing taking place. | 18:18 |
| jrichli | timburke: right, only the 2 blocks. so ... | 18:18 |
| timburke | you can get the etag for /a/c/o if you know the etag for /a/c/n and /a/c/p (and you have the encrypted etag for all three of these...) | 18:21 |
| zaitcev | BeagleBoard people are promising a board with eSata, called X15. Expeted MSRP is $239. | 18:22 |
| zaitcev | Which is a far cry from the Pi | 18:22 |
| notmyname | zaitcev: yikes. for a small amount more, I'd just get an intel NUC | 18:22 |
| zaitcev | notmyname: You just explained why ARM is unable to get traction in datacenter, despite trying for many years. | 18:23 |
| zaitcev | ARM server is like Linux Desktop. Always around the corner ever since the days of Netwinder. | 18:24 |
| timburke | jrichli: this is another reason i'd like to *only* use the derived iv for the object server's etag if we can. a container server necessarily aggregates a lot of encrypted etags, while a single object server is unlikely to have all three of those encrypted etags | 18:26 |
| redbo | Oh, the sata port on the banana pi still goes over the usb bus. blah. | 18:29 |
| jrichli | timburke: good point. | 18:29 |
| *** nadeem has joined #openstack-swift | 18:30 | |
| *** mvk_ has joined #openstack-swift | 18:53 | |
| *** mvk has quit IRC | 18:55 | |
| *** mkrcmari__ has joined #openstack-swift | 19:02 | |
| *** mvk_ has quit IRC | 19:02 | |
| *** adu has quit IRC | 19:05 | |
| *** jordan_ has joined #openstack-swift | 19:07 | |
| *** jordan_ is now known as jordanP | 19:07 | |
| acoles | tdasilva: yes any POST will always replace all existing object user metadata, regardless of whether from middleware or client | 19:08 |
| acoles | and regardless of if it has transient sysmeta headers | 19:08 |
| acoles | timburke: jrichli I think I managed to get overlapping ciphertext when encrypting etags for /a/c/n and /a/c/o :/ | 19:10 |
| acoles | timburke: reason for using same encrypted etag in container update and with object was to save encryption cycles, but yeah if there's a good argument for making them different then we can change that. | 19:11 |
| *** jordanP has quit IRC | 19:13 | |
| *** jordanP has joined #openstack-swift | 19:13 | |
| acoles | but then IIRC its the md5's that are expensive anyway so it may not be a big deal to calculate a different ciphertext for the container update. | 19:13 |
| jrichli | acoles: how similar was the plaintext etag of /a/c/n to that of /a/c/o? | 19:14 |
| *** jordanP has quit IRC | 19:15 | |
| *** jordanP_ has joined #openstack-swift | 19:15 | |
| *** mkrcmari__ has quit IRC | 19:15 | |
| *** mvk has joined #openstack-swift | 19:16 | |
| *** jordanP has joined #openstack-swift | 19:18 | |
| *** jordanP_ has quit IRC | 19:18 | |
| *** tqtran has joined #openstack-swift | 19:19 | |
| *** jordanP has quit IRC | 19:19 | |
| acoles | jrichli: identical plaintext, just confirms that the IV used for a/c/n results in a counter value (for second block) equal to iv used for /a/c/o | 19:19 |
| *** jordanP has joined #openstack-swift | 19:19 | |
| *** jamielennox is now known as jamielennox|away | 19:21 | |
| jrichli | acoles: indeed. so are you going to try the leading/ending padding? | 19:21 |
| *** jordanP has quit IRC | 19:23 | |
| acoles | jrichli: not this evening ;) and it depends if we are going to hash every path, or pad short paths and hash longer ones, but if the latter then it does seem that the current implementation is wrong. | 19:23 |
| notmyname | jrichli: question I just got about the crypto work: "does it use FIPS approved suite B cryptographic algorithms"? | 19:24 |
| *** sheel has quit IRC | 19:25 | |
| jrichli | notmyname: good question ;-) As far as our cyrpto alg used for the body and user-meta, we use AES with 256-bit keys. | 19:26 |
| jrichli | I would be surprised if that doesn't fit FIP approved anything. But I'll have to pass this question along to people who know FIPS better than me | 19:26 |
| notmyname | yeah, it seems to. the list is at https://www.cnss.gov/CNSS/openDoc.cfm?ImqEU6DGqUoGCh/dUFPK1Q== | 19:26 |
| notmyname | so AES256 and SHA384 | 19:27 |
| notmyname | (for top secret) | 19:27 |
| notmyname | and some eliptic curve stuff | 19:27 |
| timburke | huh. apparently my browser doesn't trust the issuer (DOD ID SW CA-37) for the cert on that link | 19:28 |
| jrichli | timburke: me too. i thought it a little ironic | 19:29 |
| timburke | me too :-) | 19:29 |
| timburke | ...and they don't seem to like deep linking :-( | 19:30 |
| openstackgerrit | oshritf proposed openstack/swift: Add thread level concurrency to container sync https://review.openstack.org/332985 | 19:36 |
| timburke | that^ seems to have been proposed to crypto-review -- i wonder, was that intentional? | 19:41 |
| *** thumpba has joined #openstack-swift | 19:43 | |
| notmyname | interesting. likely yes | 19:45 |
| notmyname | I had asked oshrit via email if it applied cleanly on top of the crypto work | 19:45 |
| notmyname | so I suspect that's where it comes from | 19:46 |
| notmyname | we can raise it in the meeting | 19:46 |
| *** cdelatte has quit IRC | 19:48 | |
| *** mwheckmann has quit IRC | 19:50 | |
| *** mwheckmann has joined #openstack-swift | 19:51 | |
| *** manous has joined #openstack-swift | 19:53 | |
| *** mvk_ has joined #openstack-swift | 20:01 | |
| *** SkyRocknRoll has quit IRC | 20:02 | |
| *** mvk has quit IRC | 20:05 | |
| *** nadeem has quit IRC | 20:07 | |
| *** arch-nemesis has quit IRC | 20:12 | |
| *** manous has quit IRC | 20:18 | |
| *** mwheckmann has quit IRC | 20:19 | |
| *** adu has joined #openstack-swift | 20:22 | |
| jrichli | notmyname: i'd like to know what standard/complience our key derivation would be held against | 20:29 |
| *** manous has joined #openstack-swift | 20:31 | |
| *** kei_yama has joined #openstack-swift | 20:36 | |
| notmyname | jrichli: I don't know | 20:51 |
| *** oshritf has joined #openstack-swift | 20:53 | |
| jrichli | notmyname: I know cca was good with the key derivation used, so it's probably widely accepted. I will ask him, though, about how it relates to FIPS and such | 20:53 |
| notmyname | jrichli: but yeah, good question. so many people get worried when the characters m, d, and 5 happen to be right next to each other | 20:53 |
| notmyname | jrichli: oh, that would be great. thanks | 20:53 |
| jrichli | notmyname: indeed. but rest assured, the key derivation being used does not include md5 anyway. its uses a sha-256 | 20:54 |
| timburke | (md5 wouldn't give us enough bits anyway) | 20:54 |
| *** BigWillie has quit IRC | 20:54 | |
| notmyname | oh, I was referring to the md5 used in the derived iv | 20:55 |
| *** thumpba has quit IRC | 20:55 | |
| mattoliverau | morning | 20:56 |
| timburke | notmyname: ah, yeah. i wouldn't mind that being something with longer output | 20:58 |
| timburke | but i don't have any particular reason to have a problem with it | 20:58 |
| kota_ | good morning | 20:58 |
| notmyname | see? I'm one of those worried people, too. I'm not sure (yet) how/why it's being used, but I've been told it's bad ;-) | 20:58 |
| *** m_kazuhiro has joined #openstack-swift | 20:58 | |
| jrichli | notmyname: its bad if you are using it for message authentication. but we are not :-) | 20:59 |
| notmyname | ok, team meeting time in #openstack-meeting | 21:00 |
| notmyname | oshritf: awake for the meeting? :-) | 21:01 |
| *** arch-nemesis has joined #openstack-swift | 21:01 | |
| *** joeljwright has joined #openstack-swift | 21:04 | |
| *** ChanServ sets mode: +v joeljwright | 21:04 | |
| *** pauloewerton has quit IRC | 21:05 | |
| *** acoles is now known as acoles_ | 21:05 | |
| *** acoles_ is now known as acoles | 21:07 | |
| *** acoles is now known as acoles_ | 21:20 | |
| *** acoles_ is now known as acoles | 21:21 | |
| *** siva_krish has quit IRC | 21:22 | |
| *** acoles is now known as acoles_ | 21:22 | |
| *** acoles_ is now known as acoles | 21:24 | |
| *** siva_krish has joined #openstack-swift | 21:27 | |
| *** manous has quit IRC | 21:28 | |
| *** acoles is now known as acoles_ | 21:30 | |
| *** acoles_ is now known as acoles | 21:31 | |
| *** oshritf has quit IRC | 21:47 | |
| *** jamielennox|away is now known as jamielennox | 21:48 | |
| MooingLemur | Just to understand the workflow of middleware, the auth middleware usually also handles read ACLs (for referer) and validating auth tokens? Is it basically a monolith? | 21:49 |
| *** ametts has quit IRC | 21:50 | |
| MooingLemur | If I wanted to add a middleware to do something like using request source IP as authentication/authorization bypass, would my auth middleware of choice (swauth) need to know about it? | 21:51 |
| MooingLemur | or would there be a way of testing various types of auth methods using different middlewares until one worked or one was explicitly denied | 21:52 |
| MooingLemur | I guess another question similar in scope: Can I mix keystone auth and swauth in the same instance? They'll each generate auth tokens and store them in their respective data stores, but can the pipeline of swift easily be set to try the auth modules in turn so that an auth token that isn't known to swauth is then tried against keystone? | 21:56 |
| *** tqtran has quit IRC | 21:56 | |
| notmyname | MooingLemur: hi (was in swift team meeting) | 21:57 |
| notmyname | in general, the answer is yes | 21:57 |
| MooingLemur | no worries, I looked at the time and though so :) | 21:57 |
| * jrichli has got to run | 21:57 | |
| notmyname | with a few caveats | 21:57 |
| acoles | timburke: so, I have applied your test_copy patch for etag coverage that got lost a few reviews ago, but today sort of slipped away and not ready to push to gerrit - it will be in next version of patches | 21:57 |
| mattoliverau | jrichli: o/ | 21:57 |
| MooingLemur | I suppose that could involve writing a shim that calls the auth middlewares in sequence? | 21:57 |
| notmyname | MooingLemur: swauth is a well-behaved auth system in that it works well with other auth middlewares | 21:58 |
| notmyname | MooingLemur: by design, swift supports having multiple auth systems installed | 21:58 |
| timburke | acoles: yeah, i saw those replies. no worries | 21:58 |
| timburke | there's still plenty for me to review :-) | 21:58 |
| notmyname | MooingLemur: so first off, let's ignore keystone for a moment | 21:58 |
| notmyname | MooingLemur: if you add your own extra acl-by-ip thing, then you should probably add it to the right of swauth in the pipeline and write a callback method that wraps the swift.authorize environ variable | 21:59 |
| notmyname | MooingLemur: that would let you do your checking and hand it off to swauth for the rest of the checking | 21:59 |
| notmyname | MooingLemur: make sense? | 21:59 |
| acoles | timburke: yeah, keep going! :D | 21:59 |
| MooingLemur | notmyname: I think so. So my auth-by-ip middleware would call swauth itself? Or rather I would take the old value of swift.authorize and add my code before it, return None on success, but if it fails, then call the old value of swift.authorize? | 22:00 |
| notmyname | MooingLemur: right. exactly | 22:01 |
| acoles | good night | 22:01 |
| *** joeljwright has quit IRC | 22:01 | |
| notmyname | MooingLemur: ok, so let's talk about keystone now | 22:01 |
| MooingLemur | aha, that makes sense | 22:01 |
| notmyname | MooingLemur: with different auth systems (or multiple instances of swuath or something) in one swift cluster, the "right" way to do that is for each to have their own reseller prefix | 22:02 |
| *** adu has quit IRC | 22:02 | |
| notmyname | so authX gets FOO_ and authY get BAR_ and etc | 22:02 |
| MooingLemur | yeah, I had considered that that's what that's for. | 22:03 |
| notmyname | so maybe swauth gets SWAUTh or AUTh and keystone gets KEY or something like that | 22:03 |
| MooingLemur | so auth tokens get that prefix, too | 22:03 |
| notmyname | auth plugins to swift are supposed to cooperate. so if a request comes in for a reseller prefix that an auth plugin doesn't handle, it should simply pass it along (an perhaps add a default-fail if nothing's set in swift.authorize) | 22:04 |
| notmyname | right | 22:04 |
| notmyname | and this is how swauth works. and tempauth | 22:04 |
| notmyname | the keystone middleware doesn't, though. it will overwrite the swift.authorize callback | 22:05 |
| notmyname | honestly, this is probably simply a bug in our own code (ie we manage that middleware in the swift repo) | 22:05 |
| notmyname | but the point is that when you set up keystone + any other auth system, you gotta have keystone first so that it doesn't stomp on all the others | 22:06 |
| MooingLemur | so it has to come first in the pipeline and anything we do | 22:06 |
| notmyname | right (IIRC) | 22:06 |
| MooingLemur | afterwards can wrap the authorize callback | 22:06 |
| notmyname | so with that caveat, yes all of what you described Just Works (tm) | 22:06 |
| *** m_kazuhiro has quit IRC | 22:06 | |
| MooingLemur | now, the other case is if we want to seamlessly migrate users from swauth to something else using the same reseller prefix. If we really want to do that, I suspect we'd have to write shim code that intercepts the responses of the different middlewares and decides what to do. | 22:08 |
| *** manous has joined #openstack-swift | 22:08 | |
| MooingLemur | and we'd put our shim in the pipeline instead of any of the others | 22:09 |
| notmyname | not sure why you'd need to have an online migration. if you migrate the data in the back (eg from swauth to something else) then you need to configure the new auth middleware to use the same reseller prefix as the old middleware | 22:11 |
| notmyname | oh, well except for existing tokens. so if you want existing tokens to still be valid, then you might need some code somewhere to check the old system if the new one fails | 22:12 |
| MooingLemur | yeah, I was thinking of the online migration mostly for making sure tokens don't break. | 22:13 |
| *** acoles is now known as acoles_ | 22:14 | |
| MooingLemur | We haven't decided what we want to do yet, but I think I have a better understanding of the system and how much work would be required to do them. Thanks so much for your explanations :D | 22:14 |
| MooingLemur | notmyname: ^ | 22:14 |
| notmyname | great | 22:14 |
| MooingLemur | looks like swauth/tempauth don't wrap swift.authorize, but they only clobber it if they own the reseller prefix | 22:17 |
| notmyname | right | 22:18 |
| *** StraubTW has quit IRC | 22:18 | |
| notmyname | the idea is that if an auth system is configured with a reseller prefix, it's authoritative for that | 22:18 |
| notmyname | and don't configure two auth systems with the same reseller prefix | 22:18 |
| *** jamielennox is now known as jamielennox|away | 22:44 | |
| *** manous has quit IRC | 22:46 | |
| *** jamielennox|away is now known as jamielennox | 22:53 | |
| *** tqtran has joined #openstack-swift | 22:54 | |
| *** tqtran has quit IRC | 22:58 | |
| *** rcernin has quit IRC | 23:00 | |
| *** jamielennox is now known as jamielennox|away | 23:05 | |
| *** gyee has quit IRC | 23:11 | |
| *** chsc has quit IRC | 23:14 | |
| *** arch-nemesis has quit IRC | 23:16 | |
| *** awelleck has quit IRC | 23:18 | |
| *** csmart_ is now known as csmart | 23:25 | |
| *** cdelatte has joined #openstack-swift | 23:32 | |
| *** catintheroof has quit IRC | 23:35 | |
| *** sheel has joined #openstack-swift | 23:55 | |
| *** jamielennox|away is now known as jamielennox | 23:57 | |
| *** tqtran has joined #openstack-swift | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!