Wednesday, 2016-06-22

*** _JZ_ has quit IRC00:23
redbonotmyname: I've never used that one.  I have a slightly janky one in the branch that I use for benchmarks and stuff,
*** lyrrad has quit IRC00:23
*** klamath has joined #openstack-swift00:24
*** klamath has quit IRC00:39
*** klamath has joined #openstack-swift00:40
*** catintheroof has joined #openstack-swift00:47
*** klamath has quit IRC00:54
*** Suyash has quit IRC00:56
*** catintheroof has quit IRC01:07
kota_good morning01:14
mattoliveraukota_: Morning01:14
kota_Amazing, Putter problem has been already fixed!?!?01:14
kota_mattoliverau: o/01:14
*** klamath has joined #openstack-swift01:38
*** Suyash has joined #openstack-swift01:38
kota_notmyname: Did acoles fix both Putter and MIMEPutter in the crypto-review branch?01:43
kota_I'm wondering if we should fix ECPutter which will be deprecated in the crypto-review work at master, or not.01:44
timburkekota_: i'm leaning toward letting crypto merge and fix it. saves acoles_ some merge conflicts01:46
kota_timburke: ok, probably we can do it by adding closes-bug tag into the crypto commit.01:48
kota_timburke: and agreed for your opinion, I also want to save acoles for complicated management of patch chain as possible.01:49
notmynameI definitely agree on saving acoles_ from extra work01:57
*** klamath has quit IRC01:58
notmynameand my statement was sortof asking for confirmation. I'm not 100% sure the issue is closed, but what previously showed it on my SAIO is no longer showing it01:58
notmynameand yeah, I agree about fixing it on crypto and letting that land and fix it on master01:59
kota_notmyname: alright, I still have sort of points to be fixed (not yet digg and not yet look at the newest crypt-review branch though), I will look at crypt fist and then make sure if it would be 100% fixed :)02:05
kota_Dump my thought right now,02:05
kota_the patch in bug 1594739 can resolve the issue but it *might* not be 100%, IMHO.02:06
openstackbug 1594739 in OpenStack Object Storage (swift) "EC: ECPutter doesn't close backend HTTPConnection when exception raised" [Undecided,Confirmed]
kota_because, the finally statemet to close backend connection will run through the putters list02:07
kota_however, the putters list *may* be updated during around transfer_data02:07
kota_if the putter has the flag "failed", the putter will be removed from the list.02:07
kota_Right now, I'm not sure if the removed putter also is ensured to close the backend connection (or not).02:08
kota_I'm trying to figure out it today.02:09
kota_end - dumping :/02:09
*** jamielennox is now known as jamielennox|away02:16
*** jamielennox|away is now known as jamielennox02:17
kota_alright, looks like acoles did it. great.02:36
openstackgerritKazuhiro MIYAHARA proposed openstack/swift: WIP: Swift Automated Tiering
kota_ah, maybe one more place, we could drop the failed putter? hmm...look at.02:38
kota_hmm... it seems ok because it just re-assignment as a local variable, I don't like to overwrite a method arg variable with a new value though.02:48
*** _JZ_ has joined #openstack-swift02:59
*** siva_krish has joined #openstack-swift03:08
*** siva_krish has left #openstack-swift03:08
*** sgundur1 has left #openstack-swift03:23
*** gyee has quit IRC03:24
*** links has joined #openstack-swift03:29
*** catintheroof has joined #openstack-swift03:44
*** dmorita has quit IRC03:55
*** dmorita has joined #openstack-swift03:58
*** catintheroof has quit IRC04:02
*** dmorita has quit IRC04:03
*** dmorita has joined #openstack-swift04:17
*** klrmn has quit IRC04:18
*** dmorita has quit IRC04:21
timburkejrichli: go to sleep!04:28
*** psachin has joined #openstack-swift04:30
*** sheel has joined #openstack-swift04:30
*** nadeem has joined #openstack-swift04:31
*** ppai has joined #openstack-swift04:31
jrichlitimburke: I was about to, but I decided I'd stay up just a bit more.  I was already in bed the last time you told me to sleep :-)04:52
*** SkyRocknRoll has joined #openstack-swift04:53
*** rcernin has joined #openstack-swift04:56
*** Jeffrey4l_ has quit IRC05:01
*** Suyash has quit IRC05:08
*** rcernin has quit IRC05:16
*** ChubYann has quit IRC05:32
*** chlong has quit IRC05:34
*** chlong has joined #openstack-swift05:47
*** rcernin has joined #openstack-swift05:54
*** _JZ_ has quit IRC05:56
*** Jeffrey4l_ has joined #openstack-swift06:09
*** ozeri has joined #openstack-swift06:24
*** jmccarthy has quit IRC06:41
*** jmccarthy has joined #openstack-swift06:42
*** tesseract- has joined #openstack-swift06:47
*** hugokuo_ has joined #openstack-swift06:48
*** csmart_ has joined #openstack-swift06:51
*** dmellado_ has joined #openstack-swift06:52
*** charz has joined #openstack-swift06:54
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/swift: [WIP] Testing latest u-c
openstackgerritSwapnil Kulkarni (coolsvap) proposed openstack/swift: [WIP] Testing latest u-c
*** SkyRocknRoll_ has joined #openstack-swift07:00
*** SkyRocknRoll has quit IRC07:04
*** csmart has quit IRC07:04
*** mmcardle has quit IRC07:04
*** dmellado has quit IRC07:04
*** charz_ has quit IRC07:04
*** hugokuo has quit IRC07:04
*** hugokuo_ is now known as hugokuo07:04
*** permalac_ has quit IRC07:06
*** geaaru has joined #openstack-swift07:09
*** mmcardle has joined #openstack-swift07:14
*** nadeem has quit IRC07:15
*** pcaruana has joined #openstack-swift07:22
*** chlong has quit IRC07:25
*** rledisez has joined #openstack-swift07:25
*** nadeem has joined #openstack-swift07:29
*** nadeem has quit IRC07:41
*** geaaru has quit IRC07:47
*** ozeri has quit IRC07:52
*** geaaru has joined #openstack-swift08:00
*** acoles_ is now known as acoles08:16
*** dmk0202 has joined #openstack-swift08:23
*** ma9 has joined #openstack-swift08:27
ma9I managed to connect with Cyberduck to my Swift instance… Now I'm tryng to connect, still with Cyberduck, using the S3 protocol, but I get the following error:08:28
ma9Multiple Choices. Request Error [{"versions": {"values": [{"status": "stable", "updated": , "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v3+json"}], "id": "v3.4", "links": [{"href":  "rel": "self"}]}, {"status": "stable", "updated":  "media-types": [{"base": "application/json", "type": "application/vnd.openstack.identity-v2.0+json"}], "id": "v2.0", "links": [{"href":  "rel": "self"}, {"h08:28
ma9any idea of what I could be doing wrong?08:29
acoleskota_: I think that the only place a putter is removed from the list of putter during transfer_data is in the send_chunk function, and in that case I close the putter. Otherwise all putters should still be int he list handled by the finally clauses. But please tell me if I missed any. Thanks.08:33
acoleskota_: were you looking at line 816. that assignment is confusing, I may change the name of new variable to ok_putters08:34
kota_acoles: hehe, I'm with. Just leaving comments for that :D08:37
*** bapalm has quit IRC08:37
kota_acoles: but thanks for your explanation :)08:38
kota_I was trying to ensure connection close if failed set to True, but it is getting bigger and I'm realizing it's overkill for us, so I gave up it there :/08:39
acoleskota_: thanks for the comments. yes i'll fix line 816. BTW I did add some test assertions to check that conns get closed, search for capture_expect08:42
*** bapalm has joined #openstack-swift08:42
acoleskota_: oh, i think you already spotted that :)08:42
kota_m9: it looks authentication error, are you using keystone?08:43
kota_m9: or trying to Swift protocol might save your time.08:43
kota_m9: honesty I'd like to continue to talk with you for helping because I'm maintaining s3 wrappers on top of swift but I'm leaving in 15 minutes, sorry.08:44
kota_acoles: yeah, it's great to assert the backend connection closed :)08:45
kota_acoles: it might be better to assert conn.resp is None, too?08:46
kota_actually, we need to throw the response reference to kick the connection close :P08:46
acoleskota_: timburke ack re putter.resp, conn.resp...its that way on master, I'm sure it could be cleaner, I'm not sure how much clean up to do here vs leaving things to look vaguely familiar with master for reviewers ?08:47
acoleskota_: yeah, the test can't verify that an HTTPConnection actually closed because it is testing with Fake connections, but it's better than nothing08:47
kota_acoles: I bet we can resolve in followup for putter.resp cleaner ;)08:49
ma9kota_: thanks for the answer! sorry for my delay… yes I08:49
acoleskota_: OIC, yes we could assert that FakeConn.resp==None08:49
ma9I'm using keystone08:49
ma9I already tried with the Swift interface and it works… but I wanted to try s3 ;)08:49
acoleskota_: I bet timburke has already written all the follow ups ;)08:49
ma9the port address I'm using is the one from Keystone 3535708:50
kota_m9: alright, if you could make some comments at launchpad bug report (or ask?), it could be easy to truck for us. probably I or timburke can see the detail later.08:50
kota_acoles: great, that should be fixed while I am asleep :)08:51
acoleskota_: how long are you planning to sleep for ? ;)08:52
*** mvk has joined #openstack-swift08:52
acoleskota_: so do you think it is ok to add closes-bug 1594739 to patch 32820408:56
openstackbug 1594739 in OpenStack Object Storage (swift) "EC: ECPutter doesn't close backend HTTPConnection when exception raised" [Undecided,Confirmed]
patchbotacoles: - swift (feature/crypto-review) - Support for http footers - Replication and EC08:56
kota_I think so09:03
*** ouchkernel has quit IRC09:03
kota_It's ok i mean09:04
*** ouchkernel has joined #openstack-swift09:04
kota_usually i prefer to make them separate, but in this case, fixing master will make us frustrating09:05
kota_Just paying small pain if we want to backport09:07
kota_Sorry its time to leave09:08
acoleskota_: k. thanks!09:16
*** Jeffrey4l__ has joined #openstack-swift09:25
*** Jeffrey4l_ has quit IRC09:26
*** ppai has quit IRC09:34
*** mvk has quit IRC09:35
*** Jeffrey4l__ has quit IRC09:35
*** ahale_ has quit IRC09:36
*** kei_yama has quit IRC09:41
*** bapalm has quit IRC09:48
*** ppai has joined #openstack-swift09:48
*** bapalm has joined #openstack-swift09:54
*** mvk has joined #openstack-swift09:56
*** ahale has joined #openstack-swift10:02
*** daemontool has joined #openstack-swift10:07
*** hosanai has quit IRC10:14
*** daemontool has quit IRC10:19
*** Jeffrey4l has joined #openstack-swift10:27
*** ppai has quit IRC10:30
*** ozeri has joined #openstack-swift10:37
*** Jeffrey4l has quit IRC10:40
*** ppai has joined #openstack-swift10:46
acolestimburke: oops, I didn't publish my replies to your comments on v3 of patch 328205 (I'll blame gerrit for hiding the fact that there are unpublished comments on older versions, which I am sure the old gerrit didn't do!)11:01
patchbotacoles: - swift (feature/crypto-review) - Make container update override headers persistent11:01
acolestimburke: in particular I'd left a question for you about your test patch11:02
acolestimburke: re. how middlewares should set override headers, I'll attempt some more words but I don't think we can prescribe, and encryption would be an immediate exception to the obvious guidance of "use setdefault"11:08
*** chlong has joined #openstack-swift11:21
*** ppai has quit IRC11:35
*** ma91 has joined #openstack-swift11:46
*** ma9 has quit IRC11:48
*** ppai has joined #openstack-swift11:48
*** chlong has quit IRC11:51
*** ppai has quit IRC11:59
*** raildo-afk is now known as raildo12:09
*** ppai has joined #openstack-swift12:15
*** virus has joined #openstack-swift12:16
*** virus has left #openstack-swift12:20
*** cbartz has joined #openstack-swift12:23
*** d0ugal has quit IRC12:25
*** d0ugal has joined #openstack-swift12:25
*** hseipp has joined #openstack-swift12:26
*** Jeffrey4l has joined #openstack-swift12:26
*** SkyRocknRoll_ has quit IRC12:30
*** pauloewerton has joined #openstack-swift12:34
*** tanee is now known as tanee_away12:39
*** tanee_away is now known as tanee12:39
*** tanee is now known as tanee_away12:39
*** ujjain has quit IRC12:47
*** ujjain has joined #openstack-swift12:52
*** ujjain has joined #openstack-swift12:52
*** dmorita has joined #openstack-swift12:56
*** dmorita has quit IRC13:01
*** chlong has joined #openstack-swift13:06
*** permalac has joined #openstack-swift13:07
*** links has quit IRC13:09
*** StraubTW has joined #openstack-swift13:11
*** BigWillie has joined #openstack-swift13:24
ma91how do I see who is the "owner" of an object in swift? or better, who first uploaded the object? The command line swift stat does not show it...13:33
*** siva_krish has joined #openstack-swift13:37
*** klamath has joined #openstack-swift13:43
*** mwheckmann has joined #openstack-swift13:43
*** chlong has quit IRC13:49
ma91is the ownership specified only at "project/tenant" level?13:49
ma91i'm asking because I would like to see who is the person who uploaded a specific file13:50
*** ppai has quit IRC13:52
*** cdelatte has joined #openstack-swift13:57
tdasilvama91: that information is not stored with the object. so if multiple users have access to a container, I don't think you could know13:58
ma91ok thanks for the info14:00
*** ametts has joined #openstack-swift14:01
*** siva_krish has quit IRC14:03
*** chlong has joined #openstack-swift14:05
*** siva_krish has joined #openstack-swift14:07
StraubTWAnyone know of a FUSE Swift file system other than ?14:13
*** chsc has joined #openstack-swift14:20
*** chsc has joined #openstack-swift14:20
*** sgundur1 has joined #openstack-swift14:23
*** ozeri has quit IRC14:31
*** tanee_away is now known as tanee14:32
*** tanee is now known as tanee_away14:33
*** tanee_away is now known as tanee14:35
*** ma91 has left #openstack-swift14:43
*** ma91 has joined #openstack-swift14:43
*** siva_krish has quit IRC14:51
*** zul has quit IRC14:54
*** zul has joined #openstack-swift14:55
*** SkyRocknRoll has joined #openstack-swift14:55
ma91tdasilva: so  from what I see with the swift stat command each container has an Account field14:56
ma91that Account is a project/tenant…. so each container does not have a specific user as owner, but it's owned by a project14:57
ma91so i guess there is no way to know what user uploaded a file..14:57
*** jistr is now known as jistr|mtg14:58
*** siva_krish has joined #openstack-swift15:01
*** klamath has quit IRC15:03
*** permalac has quit IRC15:08
*** rcernin has quit IRC15:09
*** chsc_ has joined #openstack-swift15:09
*** npf has joined #openstack-swift15:10
*** chsc has quit IRC15:10
*** bapalm has quit IRC15:10
*** bapalm has joined #openstack-swift15:10
*** yarkot1 has quit IRC15:10
*** hogepodge has quit IRC15:10
*** andymccr_ has quit IRC15:10
*** andymccr has joined #openstack-swift15:10
*** tesseract- has quit IRC15:10
*** yarkot1 has joined #openstack-swift15:11
*** hogepodge has joined #openstack-swift15:11
tdasilvama91: yeah, it's important to understand the distinction between users and accounts. multiple users could have access to an account and/or container15:11
*** klrmn has joined #openstack-swift15:14
tdasilvaacoles: Hello! for the footer patch, are you planning to add Sam's patch to unify Putter and MIMEPutter or will that be a follow up? just wondering...15:17
*** tanee is now known as tanee_away15:20
*** pcaruana has quit IRC15:22
*** rcernin has joined #openstack-swift15:24
acolestdasilva: hi. not today because (a) I have not yet had chance to review it and (b) based on jrichli comment I am wondering if there will be other opinions. but mainly I've not had time yet (been studying the debate on etag encryption on patch 328208)15:25
patchbotacoles: - swift (feature/crypto-review) - Enable object body and metadata encryption15:25
acolestdasilva: I have made a few changes to patch 328204 that may (or may not) get pushed today, nothing major though15:26
patchbotacoles: - swift (feature/crypto-review) - Support for http footers - Replication and EC15:26
*** psachin has quit IRC15:26
tdasilvaacoles: ok, no worries, thanks for the heads up15:27
*** mwheckmann has quit IRC15:29
*** StraubTW has quit IRC15:31
*** jistr|mtg is now known as jistr15:32
*** StraubTW has joined #openstack-swift15:36
*** pcaruana has joined #openstack-swift15:36
*** cbartz has quit IRC15:37
*** arch-nemesis has joined #openstack-swift15:42
openstackgerritPetr Kovar proposed openstack/swift: Add install-guide for swift
*** nadeem has joined #openstack-swift15:48
*** catintheroof has joined #openstack-swift15:49
*** Suyash has joined #openstack-swift15:50
*** chlong has quit IRC15:50
*** adu has joined #openstack-swift15:51
*** lyrrad has joined #openstack-swift15:52
notmynamegood morning15:55
*** daemontool has joined #openstack-swift16:00
*** dmk0202 has quit IRC16:02
*** npf_ has joined #openstack-swift16:03
*** npf_ has quit IRC16:06
*** npf has quit IRC16:07
*** mwheckmann has joined #openstack-swift16:12
*** gyee has joined #openstack-swift16:13
*** links has joined #openstack-swift16:16
*** Suyash has quit IRC16:17
*** ma91 has left #openstack-swift16:19
*** dmorita has joined #openstack-swift16:29
*** klrmn has quit IRC16:29
*** _JZ_ has joined #openstack-swift16:32
*** nadeem has quit IRC16:33
*** nadeem has joined #openstack-swift16:34
*** Suyash has joined #openstack-swift16:34
*** daemontool has quit IRC16:37
acolestimburke: jrichli I didn't get all the way through comments today but left some replies on patch 32820816:38
patchbotacoles: - swift (feature/crypto-review) - Enable object body and metadata encryption16:38
acolesI will make remaining updates and plan to push fresh crypto-review patches tomorrow16:38
*** links has quit IRC16:40
*** rcernin has quit IRC16:40
*** pcaruana has quit IRC16:43
*** chsc_ has quit IRC16:53
*** awelleck has joined #openstack-swift16:55
*** rledisez has quit IRC17:06
*** siva_krish has quit IRC17:06
*** Jeffrey4l has quit IRC17:08
*** mvk has quit IRC17:11
notmynameon today's meeting agenda, the first topic we'll talk about is container sync. oshrit is raising these patches again, and there are now some (very nice looking) numbers from test results17:12
notmynamelinks are on
*** rcernin has joined #openstack-swift17:20
notmynameredbo: any progress on getting hummingbird stuff launched with swift-init?17:20
timburkegood morning17:21
*** hseipp has quit IRC17:26
*** hseipp has joined #openstack-swift17:27
*** hseipp has quit IRC17:27
*** pauloewerton has quit IRC17:28
jrichliacoles: ok, ill take a look17:29
claygsome dood did a pr to vsaio for aws support - pretty cool17:29
clayghe says he's using vsaio to test some backup software - pretty cool17:30
claygnotmyname: so your theory about app developers using vsaio wasn't too nuts after all!?17:30
notmynamesomething something about stopped clocks17:31
*** klrmn has joined #openstack-swift17:32
zaitcevgo go gadget Bing vsaio17:36
jrichliacoles: thanks for tracking down the code at
notmynamefunny story, this morning we got an email to a general swiftstack address from someone who is building a swift cluster on a bunch of raspberry pis and is seeing some errors.17:38
*** pauloewerton has joined #openstack-swift17:38
*** mvk has joined #openstack-swift17:41
*** daemontool has joined #openstack-swift17:41
tdasilvaacoles: if a middleware sends a POST request to update a transient metadata, will that erase all user metadata too?17:43
timburkejrichli: acoles: fwiw, i didn't really dig into the openssl implementation; i was just going off of our adjustments to iv for ranged requests and the observed behavior in a python REPL. how certain are we that the offset handling will be correct on both big- and little-endian platforms? maybe more to the point, how much do we care?17:43
*** daemontool has quit IRC17:44
*** nadeem has quit IRC17:44
timburkezaitcev: vagrant swift all in one -
*** dmorita has quit IRC17:44
*** dmorita has joined #openstack-swift17:44
*** chsc has joined #openstack-swift17:44
*** chsc has joined #openstack-swift17:44
zaitcevtimburke: thanks, I would never guess, although I saw Clay using it17:45
*** silor has joined #openstack-swift17:45
timburketdasilva: i'd expect so, yeah. would a middleware send a POST that wasn't some manipulation of a POST from a client, though?17:46
tdasilvatimburke: I was thinking about that and it is a fair point, I don't think we have that situation in swift's middleware code today, but still I think that's an important enough information to be documented so that third-party middleware developers are aware of it.17:48
timburkeyeah. i know timur was playing around with a middleware that would send a POST immediately following any PUTs as part of some metadata-search work. one nice thing about the crypto work is that shouldn't be necessary any more; he can just use footers17:50
*** nadeem has joined #openstack-swift17:50
timburkenotmyname: hey, it's not *so* hard!17:51
notmynameit's just typing!17:51
*** silor has quit IRC17:51
*** silor has joined #openstack-swift17:52
notmynameif I understand correctly, the middleware installs the footer callback, then the obj controller will do the right thing and do the multipart mime dance with the object server17:52
*** nadeem has quit IRC17:52
notmynamewhich also had the huge advantage of removing the need for coordination across multiple requests. it's all one request17:52
timburkebetter all-around17:53
notmynameit's so easy! why aren't you done yet timur?!17:53
timburkeyeah, it's even better when it's *someone else's* typing. that's like zero-effort!17:54
*** geaaru has quit IRC17:54
notmyname"easy is a word used to describe someone else's job"17:54
redboraspberry pi seems like a terrible idea, there's no way to get reasonable bandwidth to drives.  There are similar boards with sata interfaces like the banana pi.18:04
*** silor has quit IRC18:04
timurha sorry, but timburke is right, the footer is what I wanted for the case where the middleware was extracting jpeg EXIF metadata on PUT and then doing a POST to apply it18:04
redbonotmyname: not really.  My python is all rusty.18:04
jrichliyou guys are making me want some pie!18:04
notmynameredbo: ;-)18:06
zaitcevI don't quite understand what makes anyone thinkg that a board based on Allwinner A20 has a better storage access than Raspberry Pi. There's no PCI! You can't connect a SATA controller to that thing.18:10
*** siva_krish has joined #openstack-swift18:10
notmynamethat is interesting. I hadn't heard of the banana pi before18:11
jrichlitimburke: so you are speaking now to the offset handling - and not about the derived_iv method, right?18:12
jrichlitimburke: when you ask how much we care, I am not sure if I know which part you are talking about.  If we our calculation does not result in the proper offset, then encryption or decryption would fail to produce the correct message.18:12
jrichli(in a ranged get case against the body)18:12
jrichlitimburke: I plan on digging into the openssl impl so I can think about adding some cases to our offset calcuation unit test so we can be more sure that endianess will all work-out.  Good thought!18:12
zaitcevBanana Pi is in the same boat as TrimSlice 218:13
timburkejrichli: yeah, i was thinking about endianness and offset handling. it may be somewhat related to the derived_iv issues, though18:14
*** ChubYann has joined #openstack-swift18:15
timburkejrichli: i think i'd be in favor of always using md5(path) for derived_iv; it would ensure that ivs are reasonably well-distributed18:15
notmynamezaitcev: my interest in running swift on stuff like that is directly proportional to how many ones I get for free from a conference ;-)18:16
timburkejrichli: acoles was right, of course, that we only care about two blocks, so there's only one opportunity for overlap. i thought of that last night after i'd posted. still a concern, i think18:17
jrichlitimburke: I agree on hash guarantees well-distributed, but I think maybe acoles may be wanting to avoid the  hash, if possible, because of the performance hit.  so more reason i'd like to get more familiar with the exact incrementing taking place.18:18
jrichlitimburke: right, only the 2 blocks.  so ...18:18
timburkeyou can get the etag for /a/c/o if you know the etag for /a/c/n and /a/c/p (and you have the encrypted etag for all three of these...)18:21
zaitcevBeagleBoard people are promising a board with eSata, called X15. Expeted MSRP is $239.18:22
zaitcevWhich is a far cry from the Pi18:22
notmynamezaitcev: yikes. for a small amount more, I'd just get an intel NUC18:22
zaitcevnotmyname: You just explained why ARM is unable to get traction in datacenter, despite trying for many years.18:23
zaitcevARM server is like Linux Desktop. Always around the corner ever since the days of Netwinder.18:24
timburkejrichli: this is another reason i'd like to *only* use the derived iv for the object server's etag if we can. a container server necessarily aggregates a lot of encrypted etags, while a single object server is unlikely to have all three of those encrypted etags18:26
redboOh, the sata port on the banana pi still goes over the usb bus.  blah.18:29
jrichlitimburke: good point.18:29
*** nadeem has joined #openstack-swift18:30
*** mvk_ has joined #openstack-swift18:53
*** mvk has quit IRC18:55
*** mkrcmari__ has joined #openstack-swift19:02
*** mvk_ has quit IRC19:02
*** adu has quit IRC19:05
*** jordan_ has joined #openstack-swift19:07
*** jordan_ is now known as jordanP19:07
acolestdasilva: yes any POST will always replace all existing object user metadata, regardless of whether from middleware or client19:08
acolesand regardless of if it has transient sysmeta headers19:08
acolestimburke: jrichli I think I managed to get overlapping ciphertext when encrypting etags for /a/c/n and /a/c/o :/19:10
acolestimburke: reason for using same encrypted etag in container update and with object was to save encryption cycles, but yeah if there's a good argument for making them different then we can change that.19:11
*** jordanP has quit IRC19:13
*** jordanP has joined #openstack-swift19:13
acolesbut then IIRC its the md5's that are expensive anyway so it may not be a big deal to calculate a different ciphertext for the container update.19:13
jrichliacoles: how similar was the plaintext etag of /a/c/n to that of /a/c/o?19:14
*** jordanP has quit IRC19:15
*** jordanP_ has joined #openstack-swift19:15
*** mkrcmari__ has quit IRC19:15
*** mvk has joined #openstack-swift19:16
*** jordanP has joined #openstack-swift19:18
*** jordanP_ has quit IRC19:18
*** tqtran has joined #openstack-swift19:19
*** jordanP has quit IRC19:19
acolesjrichli: identical plaintext, just confirms that the IV used for a/c/n results in a counter value (for second block) equal to iv used for /a/c/o19:19
*** jordanP has joined #openstack-swift19:19
*** jamielennox is now known as jamielennox|away19:21
jrichliacoles: indeed.  so are you going to try the leading/ending padding?19:21
*** jordanP has quit IRC19:23
acolesjrichli: not this evening ;) and it depends if we are going to hash every path, or pad short paths and hash longer ones, but if the latter then it does seem that the current implementation is wrong.19:23
notmynamejrichli: question I just got about the crypto work: "does it use FIPS approved suite B cryptographic algorithms"?19:24
*** sheel has quit IRC19:25
jrichlinotmyname: good question ;-)  As far as our cyrpto alg used for the body and user-meta, we use AES with 256-bit keys.19:26
jrichliI would be surprised if that doesn't fit FIP approved anything.  But I'll have to pass this question  along to people who know FIPS better than me19:26
notmynameyeah, it seems to. the list is at
notmynameso AES256 and SHA38419:27
notmyname(for top secret)19:27
notmynameand some eliptic curve stuff19:27
timburkehuh. apparently my browser doesn't trust the issuer (DOD ID SW CA-37) for the cert on that link19:28
jrichlitimburke: me too.  i thought it a little ironic19:29
timburkeme too :-)19:29
timburke...and they don't seem to like deep linking :-(19:30
openstackgerritoshritf proposed openstack/swift: Add thread level concurrency to container sync
timburkethat^ seems to have been proposed to crypto-review -- i wonder, was that intentional?19:41
*** thumpba has joined #openstack-swift19:43
notmynameinteresting. likely yes19:45
notmynameI had asked oshrit via email if it applied cleanly on top of the crypto work19:45
notmynameso I suspect that's where it comes from19:46
notmynamewe can raise it in the meeting19:46
*** cdelatte has quit IRC19:48
*** mwheckmann has quit IRC19:50
*** mwheckmann has joined #openstack-swift19:51
*** manous has joined #openstack-swift19:53
*** mvk_ has joined #openstack-swift20:01
*** SkyRocknRoll has quit IRC20:02
*** mvk has quit IRC20:05
*** nadeem has quit IRC20:07
*** arch-nemesis has quit IRC20:12
*** manous has quit IRC20:18
*** mwheckmann has quit IRC20:19
*** adu has joined #openstack-swift20:22
jrichlinotmyname: i'd like to know what standard/complience our key derivation would be held against20:29
*** manous has joined #openstack-swift20:31
*** kei_yama has joined #openstack-swift20:36
notmynamejrichli: I don't know20:51
*** oshritf has joined #openstack-swift20:53
jrichlinotmyname: I know cca was good with the key derivation used, so it's probably widely accepted.  I will ask him, though, about how it relates to FIPS and such20:53
notmynamejrichli: but yeah, good question. so many people get worried when the characters m, d, and 5 happen to be right next to each other20:53
notmynamejrichli: oh, that would be great. thanks20:53
jrichlinotmyname: indeed.  but rest assured, the key derivation being used does not include md5 anyway.  its uses a sha-25620:54
timburke(md5 wouldn't give us enough bits anyway)20:54
*** BigWillie has quit IRC20:54
notmynameoh, I was referring to the md5 used in the derived iv20:55
*** thumpba has quit IRC20:55
timburkenotmyname: ah, yeah. i wouldn't mind that being something with longer output20:58
timburkebut i don't have any particular reason to have a problem with it20:58
kota_good morning20:58
notmynamesee? I'm one of those worried people, too. I'm not sure (yet) how/why it's being used, but I've been told it's bad ;-)20:58
*** m_kazuhiro has joined #openstack-swift20:58
jrichlinotmyname: its bad if you are using it for message authentication.  but we are not :-)20:59
notmynameok, team meeting time in #openstack-meeting21:00
notmynameoshritf: awake for the meeting? :-)21:01
*** arch-nemesis has joined #openstack-swift21:01
*** joeljwright has joined #openstack-swift21:04
*** ChanServ sets mode: +v joeljwright21:04
*** pauloewerton has quit IRC21:05
*** acoles is now known as acoles_21:05
*** acoles_ is now known as acoles21:07
*** acoles is now known as acoles_21:20
*** acoles_ is now known as acoles21:21
*** siva_krish has quit IRC21:22
*** acoles is now known as acoles_21:22
*** acoles_ is now known as acoles21:24
*** siva_krish has joined #openstack-swift21:27
*** manous has quit IRC21:28
*** acoles is now known as acoles_21:30
*** acoles_ is now known as acoles21:31
*** oshritf has quit IRC21:47
*** jamielennox|away is now known as jamielennox21:48
MooingLemurJust to understand the workflow of middleware, the auth middleware usually also handles read ACLs (for referer) and validating auth tokens?  Is it basically a monolith?21:49
*** ametts has quit IRC21:50
MooingLemurIf I wanted to add a middleware to do something like using request source IP as authentication/authorization bypass, would my auth middleware of choice (swauth) need to know about it?21:51
MooingLemuror would there be a way of testing various types of auth methods using different middlewares until one worked or one was explicitly denied21:52
MooingLemurI guess another question similar in scope: Can I mix keystone auth and swauth in the same instance?  They'll each generate auth tokens and store them in their respective data stores, but can the pipeline of swift easily be set to try the auth modules in turn so that an auth token that isn't known to swauth is then tried against keystone?21:56
*** tqtran has quit IRC21:56
notmynameMooingLemur: hi (was in swift team meeting)21:57
notmynamein general, the answer is yes21:57
MooingLemurno worries, I looked at the time and though so :)21:57
* jrichli has got to run21:57
notmynamewith a few caveats21:57
acolestimburke: so, I have applied your test_copy patch for etag coverage that got lost a few reviews ago, but today sort of slipped away and not ready to push to gerrit - it will be in next version of patches21:57
mattoliveraujrichli: o/21:57
MooingLemurI suppose that could involve writing a shim that calls the auth middlewares in sequence?21:57
notmynameMooingLemur: swauth is a well-behaved auth system in that it works well with other auth middlewares21:58
notmynameMooingLemur: by design, swift supports having multiple auth systems installed21:58
timburkeacoles: yeah, i saw those replies. no worries21:58
timburkethere's still plenty for me to review :-)21:58
notmynameMooingLemur: so first off, let's ignore keystone for a moment21:58
notmynameMooingLemur: if you add your own extra acl-by-ip thing, then you should probably add it to the right of swauth in the pipeline and write a callback method that wraps the swift.authorize environ variable21:59
notmynameMooingLemur: that would let you do your checking and hand it off to swauth for the rest of the checking21:59
notmynameMooingLemur: make sense?21:59
acolestimburke: yeah, keep going! :D21:59
MooingLemurnotmyname: I think so. So my auth-by-ip middleware would call swauth itself?  Or rather I would take the old value of swift.authorize and add my code before it, return None on success, but if it fails, then call the old value of swift.authorize?22:00
notmynameMooingLemur: right. exactly22:01
acolesgood night22:01
*** joeljwright has quit IRC22:01
notmynameMooingLemur: ok, so let's talk about keystone now22:01
MooingLemuraha, that makes sense22:01
notmynameMooingLemur: with different auth systems (or multiple instances of swuath or something) in one swift cluster, the "right" way to do that is for each to have their own reseller prefix22:02
*** adu has quit IRC22:02
notmynameso authX gets FOO_ and authY get BAR_ and etc22:02
MooingLemuryeah, I had considered that that's what that's for.22:03
notmynameso maybe swauth gets SWAUTh or AUTh and keystone gets KEY or something like that22:03
MooingLemurso auth tokens get that prefix, too22:03
notmynameauth plugins to swift are supposed to cooperate. so if a request comes in for a reseller prefix that an auth plugin doesn't handle, it should simply pass it along (an perhaps add a default-fail if nothing's set in swift.authorize)22:04
notmynameand this is how swauth works. and tempauth22:04
notmynamethe keystone middleware doesn't, though. it will overwrite the swift.authorize callback22:05
notmynamehonestly, this is probably simply a bug in our own code (ie we manage that middleware in the swift repo)22:05
notmynamebut the point is that when you set up keystone + any other auth system, you gotta have keystone first so that it doesn't stomp on all the others22:06
MooingLemurso it has to come first in the pipeline and anything we do22:06
notmynameright (IIRC)22:06
MooingLemurafterwards can wrap the authorize callback22:06
notmynameso with that caveat, yes all of what you described Just Works (tm)22:06
*** m_kazuhiro has quit IRC22:06
MooingLemurnow, the other case is if we want to seamlessly migrate users from swauth to something else using the same reseller prefix.  If we really want to do that, I suspect we'd have to write shim code that intercepts the responses of the different middlewares and decides what to do.22:08
*** manous has joined #openstack-swift22:08
MooingLemurand we'd put our shim in the pipeline instead of any of the others22:09
notmynamenot sure why you'd need to have an online migration. if you migrate the data in the back (eg from swauth to something else) then you need to configure the new auth middleware to use the same reseller prefix as the old middleware22:11
notmynameoh, well except for existing tokens. so if you want existing tokens to still be valid, then you might need some code somewhere to check the old system if the new one fails22:12
MooingLemuryeah, I was thinking of the online migration mostly for making sure tokens don't break.22:13
*** acoles is now known as acoles_22:14
MooingLemurWe haven't decided what we want to do yet, but I think I have a better understanding of the system and how much work would be required to do them.  Thanks so much for your explanations :D22:14
MooingLemurnotmyname: ^22:14
MooingLemurlooks like swauth/tempauth don't wrap swift.authorize, but they only clobber it if they own the reseller prefix22:17
*** StraubTW has quit IRC22:18
notmynamethe idea is that if an auth system is configured with a reseller prefix, it's authoritative for that22:18
notmynameand don't configure two auth systems with the same reseller prefix22:18
*** jamielennox is now known as jamielennox|away22:44
*** manous has quit IRC22:46
*** jamielennox|away is now known as jamielennox22:53
*** tqtran has joined #openstack-swift22:54
*** tqtran has quit IRC22:58
*** rcernin has quit IRC23:00
*** jamielennox is now known as jamielennox|away23:05
*** gyee has quit IRC23:11
*** chsc has quit IRC23:14
*** arch-nemesis has quit IRC23:16
*** awelleck has quit IRC23:18
*** csmart_ is now known as csmart23:25
*** cdelatte has joined #openstack-swift23:32
*** catintheroof has quit IRC23:35
*** sheel has joined #openstack-swift23:55
*** jamielennox|away is now known as jamielennox23:57
*** tqtran has joined #openstack-swift23:59

Generated by 2.14.0 by Marius Gedminas - find it at!