Wednesday, 2015-11-18

*** ho has joined #openstack-swift00:07
hogood morning!00:10
mattoliverauho: Morning00:16
homattoliverau: morning!00:17
openstackgerritOpenStack Proposal Bot proposed openstack/swift: Updated from global requirements  https://review.openstack.org/8873600:23
*** kei_yama_ has joined #openstack-swift00:28
*** kei_yama has quit IRC00:29
*** diazjf1 has quit IRC00:34
*** mwheckmann has quit IRC00:37
*** m_kazuhiro has joined #openstack-swift00:42
*** dmorita has joined #openstack-swift00:46
*** diazjf has joined #openstack-swift00:49
*** jerrygb has quit IRC00:55
*** hrou has joined #openstack-swift00:55
*** gyee has quit IRC00:57
torgomaticwell, I'm now pretty confident that clayg's ring change won't break on initial rebalance; I fuzz-tested about 11K random rings but didn't manage to crash it01:02
notmynamecool01:02
torgomaticnow to work on making changes, then rebalancing01:02
torgomaticalso, now to let my laptop quiet down01:02
*** hrou has quit IRC01:04
*** thumpba has joined #openstack-swift01:06
*** mzhou has quit IRC01:08
*** jamielennox is now known as jamielennox|away01:10
*** thumpba has quit IRC01:12
openstackgerritMerged openstack/swift: go: fix problem with using single config file  https://review.openstack.org/24650001:13
*** zhill has quit IRC01:50
*** diazjf has quit IRC01:55
mattoliverauLol, nice work torgomatic01:58
*** mwheckmann has joined #openstack-swift02:05
*** mwheckmann has quit IRC02:17
*** mwheckmann has joined #openstack-swift02:17
*** jerrygb has joined #openstack-swift02:29
*** 7JTAA216T has joined #openstack-swift02:30
*** 7JTAA216T has quit IRC02:30
*** 21WAACTNP has joined #openstack-swift02:31
*** sanchitmalhotra has joined #openstack-swift02:43
jrichlitimburke: I just debugged the "old" version of what I had working.   It used to work because seg_hash AND seg_resp.etag were both always None.02:53
*** 21WAACTNP has quit IRC03:01
*** haomaiwang has joined #openstack-swift03:01
kota_notmyname: hello, i'd notify you that i will be absent from tommorow IRC meeting because of offline in my business flight.03:04
kota_notmyname: sorry03:04
*** kei_yama_ has quit IRC03:05
*** lpabon has quit IRC03:05
*** venkat_p has joined #openstack-swift03:06
*** kei_yama has joined #openstack-swift03:07
*** janonymous has quit IRC03:26
*** tsg has joined #openstack-swift03:30
kota_oh...tomorrow meeting addressed PyECLib updates I'd catch up the status, I won't be able to attend on time tho :(03:34
*** janonymous has joined #openstack-swift03:36
*** mac_ified has joined #openstack-swift03:40
notmynamekota_: thanks for letting me know03:42
*** diazjf has joined #openstack-swift03:42
*** tsg has quit IRC03:43
*** haomaiwang has quit IRC04:01
*** haomaiwang has joined #openstack-swift04:01
*** km has joined #openstack-swift04:01
*** km is now known as Guest828504:02
*** Guest6043 has quit IRC04:03
*** kei_yama_ has joined #openstack-swift04:03
*** kei_yama has quit IRC04:04
*** sanchitmalhotra1 has joined #openstack-swift04:11
*** sanchitmalhotra has quit IRC04:13
*** sanchitmalhotra has joined #openstack-swift04:20
*** elligottmc has quit IRC04:22
*** sanchitmalhotra1 has quit IRC04:22
*** ppai has joined #openstack-swift04:26
*** mwheckmann has quit IRC04:36
jrichlitimburke:  sorry for the alert.  I think that it might be some crypto changes of late causing the crpyto-meta for the etag to be missing.  Still looking at verifying that.  i still think its interesting those values used to be None, but oh well.04:40
*** hrou has joined #openstack-swift04:43
*** jrichli has quit IRC04:43
*** trifon has joined #openstack-swift04:44
*** kei_yama_ has quit IRC04:46
*** Guest8285 has quit IRC04:47
*** km_ has joined #openstack-swift04:47
*** mac_ified has quit IRC04:49
*** trifon has quit IRC04:49
*** kei_yama has joined #openstack-swift04:52
*** eariasn has quit IRC04:52
*** eariasn has joined #openstack-swift04:53
*** jamielennox|away is now known as jamielennox04:54
*** eariasn has quit IRC04:57
*** janonymous has quit IRC04:59
*** haomaiwang has quit IRC05:01
*** klrmn2 has joined #openstack-swift05:01
*** klrmn1 has quit IRC05:01
*** haomaiwang has joined #openstack-swift05:01
openstackgerritMatthew Oliver proposed openstack/swift: Add concurrent reads option to proxy  https://review.openstack.org/11771005:03
*** tsg has joined #openstack-swift05:11
tsgkota_: https://review.openstack.org/#/c/244287/ has the latest update in comments05:20
tsgkota_: s/update/status05:20
*** jerrygb has quit IRC05:25
*** SkyRocknRoll has joined #openstack-swift05:33
kota_tsg: thx, I'll see that.05:35
tsgkota_: I added you to https://bugs.launchpad.net/precise-backports/+bug/1515710.  Please see if you can ping someone on #ubuntu-devel to get a resolution accelerated.  :-)  I haven't had much luck during day time here last few days.  Thank you05:38
openstackLaunchpad bug 1515710 in vivid-backports "Please backport liberasurecode-1.1.0, python-pyeclib-1.1.1 to precise, trusty, vivid" [Undecided,New]05:38
tsgkota_: thanks05:38
*** tsg has left #openstack-swift05:38
*** diazjf1 has joined #openstack-swift05:42
*** diazjf has quit IRC05:45
*** trifon has joined #openstack-swift05:56
*** hrou has quit IRC06:00
*** haomaiwang has quit IRC06:01
*** haomaiwang has joined #openstack-swift06:01
*** sanchitmalhotra1 has joined #openstack-swift06:04
*** SkyRocknRoll has quit IRC06:05
*** sanchitmalhotra has quit IRC06:07
*** SkyRocknRoll has joined #openstack-swift06:20
*** rcernin has joined #openstack-swift06:37
*** diazjf1 has quit IRC06:37
*** haomaiwang has quit IRC06:42
*** ppai has quit IRC06:43
*** haomaiwang has joined #openstack-swift06:49
*** ppai has joined #openstack-swift06:54
*** zaitcev has quit IRC06:54
*** haomaiwang has quit IRC07:01
*** haomaiwang has joined #openstack-swift07:01
*** ppai has quit IRC07:01
*** pchng_ has joined #openstack-swift07:05
*** pchng has quit IRC07:09
*** ppai has joined #openstack-swift07:15
*** jerrygb has joined #openstack-swift07:25
*** jerrygb has quit IRC07:30
*** jmccarthy1 has joined #openstack-swift07:33
*** jmccarthy1 has quit IRC07:35
*** SkyRocknRoll has quit IRC07:36
*** eranrom has joined #openstack-swift07:43
*** eranrom has left #openstack-swift07:43
*** jmccarthy has quit IRC07:47
*** jmccarthy has joined #openstack-swift07:48
*** SkyRocknRoll has joined #openstack-swift07:49
*** breitz has quit IRC07:58
*** breitz1 has joined #openstack-swift07:58
*** haomaiwang has quit IRC08:01
*** haomaiwang has joined #openstack-swift08:01
*** arnox has joined #openstack-swift08:03
*** hseipp has joined #openstack-swift08:09
*** rledisez has joined #openstack-swift08:10
*** hseipp has quit IRC08:14
*** hseipp has joined #openstack-swift08:15
*** mahatic has joined #openstack-swift08:27
*** geaaru has joined #openstack-swift08:33
*** SkyRocknRoll has quit IRC08:43
*** jordanP has joined #openstack-swift08:49
*** ppai has quit IRC08:50
*** ho has quit IRC08:54
*** haomaiwang has quit IRC09:01
*** haomaiwang has joined #openstack-swift09:01
*** janonymous has joined #openstack-swift09:02
*** ppai has joined #openstack-swift09:04
*** mahatic has quit IRC09:05
*** mahatic has joined #openstack-swift09:08
*** dmorita has quit IRC09:19
*** m_kazuhiro has quit IRC09:19
*** joeljwright has joined #openstack-swift09:27
*** ChanServ sets mode: +v joeljwright09:27
*** janonymous has quit IRC09:36
*** SkyRocknRoll has joined #openstack-swift09:40
*** janonymous has joined #openstack-swift09:42
*** jistr has joined #openstack-swift09:42
*** SkyRocknRoll has quit IRC09:49
*** km_ has quit IRC09:51
*** kei_yama has quit IRC09:52
*** haomaiwang has quit IRC10:01
*** haomaiwang has joined #openstack-swift10:01
*** jmccarthy has quit IRC10:05
*** aix has joined #openstack-swift10:08
*** jmccarthy has joined #openstack-swift10:24
*** jerrygb has joined #openstack-swift10:26
*** jmccarthy has quit IRC10:26
*** jmccarthy has joined #openstack-swift10:27
*** daemontool has joined #openstack-swift10:28
*** daemontool has quit IRC10:30
*** daemontool has joined #openstack-swift10:30
*** jerrygb has quit IRC10:31
*** daemontool has quit IRC10:31
*** haomaiwang has quit IRC11:01
*** haomaiwang has joined #openstack-swift11:01
*** jeblair has quit IRC11:04
*** jeblair has joined #openstack-swift11:05
*** jmccarthy has quit IRC11:09
*** jmccarthy has joined #openstack-swift11:10
*** daemontool has joined #openstack-swift11:17
*** daemontool has quit IRC11:18
*** daemontool has joined #openstack-swift11:19
*** daemontool has quit IRC11:21
*** daemontool has joined #openstack-swift11:24
*** aix has quit IRC11:31
*** aix has joined #openstack-swift11:32
*** sanchitmalhotra1 has quit IRC11:34
*** SkyRocknRoll has joined #openstack-swift11:39
*** daemontool has quit IRC11:47
*** jmccarthy has quit IRC11:47
*** jmccarthy has joined #openstack-swift11:47
*** daemontool has joined #openstack-swift11:47
*** haomaiwang has quit IRC11:51
*** silor has joined #openstack-swift12:06
*** wolsen has quit IRC12:20
*** alejandrito has joined #openstack-swift12:23
*** eariasn has joined #openstack-swift12:27
*** hrou has joined #openstack-swift12:29
*** eariasn has quit IRC12:29
*** eariasn has joined #openstack-swift12:30
*** eariasn_ has joined #openstack-swift12:31
*** hrou has quit IRC12:34
*** eariasn has quit IRC12:34
venkat_pHi all12:44
venkat_pI am not able to delete the endpoint for swift using keystone12:44
venkat_pwhat can be the problem12:44
venkat_pThis is devstack installation12:44
venkat_pI have not done anything after installation12:44
*** eariasn_ has quit IRC12:49
*** eariasn has joined #openstack-swift12:49
*** eariasn has quit IRC12:53
*** NM has joined #openstack-swift12:54
*** joeljwright has quit IRC12:59
*** jerrygb has joined #openstack-swift13:06
*** jmccarthy has quit IRC13:09
*** jerrygb has quit IRC13:10
*** tab has joined #openstack-swift13:14
*** eariasn has joined #openstack-swift13:15
*** tab is now known as Guest9945213:15
*** mac_ified has joined #openstack-swift13:16
*** eariasn_ has joined #openstack-swift13:16
*** joeljwright has joined #openstack-swift13:19
*** ChanServ sets mode: +v joeljwright13:19
*** eariasn has quit IRC13:19
*** joeljwright has quit IRC13:25
*** venkat_p has quit IRC13:25
*** joeljwright has joined #openstack-swift13:28
*** ChanServ sets mode: +v joeljwright13:28
*** jmccarthy has joined #openstack-swift13:30
*** mac_ified has quit IRC13:44
*** bill_az has joined #openstack-swift13:53
*** hrou has joined #openstack-swift13:59
*** bapalm has joined #openstack-swift14:08
*** ppai has quit IRC14:08
*** mwheckmann has joined #openstack-swift14:14
*** wbhuber has joined #openstack-swift14:19
*** daemontool_ has joined #openstack-swift14:25
openstackgerritMike Fedosin proposed openstack/python-swiftclient: Fix swift client reauthentication  https://review.openstack.org/24694314:25
*** daemontool_ has quit IRC14:26
*** daemontool_ has joined #openstack-swift14:27
*** daemontool has quit IRC14:27
*** daemontool_ has quit IRC14:28
*** daemontool_ has joined #openstack-swift14:29
*** hrou has left #openstack-swift14:30
*** dustins has joined #openstack-swift14:30
*** openstackgerrit has quit IRC14:31
*** openstackgerrit has joined #openstack-swift14:32
*** hrou has joined #openstack-swift14:33
*** bapalm has quit IRC14:43
*** bapalm has joined #openstack-swift14:43
*** lpabon has joined #openstack-swift14:46
*** pchng_ is now known as pchng14:51
*** daemontool_ has quit IRC14:51
*** petertr7_away is now known as petertr714:52
*** zaitcev has joined #openstack-swift14:55
*** ChanServ sets mode: +v zaitcev14:55
*** dustins has quit IRC14:57
*** dustins has joined #openstack-swift15:00
*** daemontool has joined #openstack-swift15:01
jordanPguys, I'd like your opinion on https://review.openstack.org/#/c/246601/15:01
jordanP"Add liberasurecode-dev as a swift dependency"15:01
jordanPif this gets merged, it won"t be possible to install swift-master on Ubuntu 1215:02
jordanPbecause liberasurecode-dev is not available, as a package, on Ubuntu 1215:02
*** blmartin has joined #openstack-swift15:05
*** jrichli has joined #openstack-swift15:05
jordanPI don't understand why, now, the system package liberasurecode-dev would be a requirement.15:15
*** silor1 has joined #openstack-swift15:28
*** silor has quit IRC15:32
*** silor1 is now known as silor15:32
*** haomaiwang has joined #openstack-swift15:37
pchngjordanP: I believe that change was done because later PyECLib versions don't include an EC implementation15:40
pchngjordanP: Perhaps related to this commit, but someone else might know better: https://github.com/openstack/swift/commit/8086a0e53406dc95856df2d3e015d3eaaf81380a15:41
pchngjordanP: So I think the change was done to "get ready" for the later versions of PyECLib, to include the requirements for it15:41
jordanPpchng, I am aware of this issue, but pining pyeclib to 1.0.7 worked so far15:41
jordanPso am not sure why we have to change this15:42
pchngjordanP: I believe you are right, things work fine, but I think the change to pin to PyECLib 1.0.7 was considered a temporary/interim one, at least that's what I can tell from that github commit message15:43
jordanPpchng, I understand. I still think that dropping support of Ubuntu 12 is not worth it and until we really can"t live with pyeclib 1.0.7, then we should stick to 1.0.7 but in the end it's not my call15:45
jordanPI'll wait and see then15:45
notmynamejordanP: tsg is working on getting that available https://bugs.launchpad.net/precise-backports/+bug/151571015:47
openstackLaunchpad bug 1515710 in vivid-backports "Please backport liberasurecode-1.1.0, python-pyeclib-1.1.1 to precise, trusty, vivid" [Undecided,New]15:47
notmynameactually, that's one of the reasons the update to pyeclib hasn't landed yet15:48
jordanPnotmyname, yup I've seen that bug report, I really hope his request will be accepted15:49
notmynameyeah, me too :-)15:49
jordanPI'll test liberasurecode 1.1.0-2 on Ubuntu 12 and leave a comment on the bug report15:50
*** Yeo09 has joined #openstack-swift15:51
jordanPnotmyname, If you could leave a comment on https://review.openstack.org/#/c/246601 to slow things a bit that would help, cause at $my_job, I need to test swift on Ubuntu 12 and that patch will only make my job harder15:51
jordanP(I guess I shouldn't have picked devstack as my swift installer)15:52
*** proteusguy__ has quit IRC15:53
Yeo09I have questions. How to create (PUT) and retrieve (GET) object files from (device, partition, node)15:54
Yeo09tuple? (For example, when you store 'object_00001' object into 'container_1'15:54
Yeo09container and 'account_1', describe which directories & files of that object15:54
Yeo09are created in local filesystem.)15:54
*** mzhou has joined #openstack-swift15:55
*** minwoob has joined #openstack-swift15:55
notmynamejordanP: on the other hand, this whole process has taken so long I'm reticent to try to put the brakes on a particular aspect of it (except for things that are dependent on something else)15:56
*** jerrygb has joined #openstack-swift15:57
notmynameYeo09: I'm not sure I understand your question15:57
jordanPagreed, but LP#1515710 was opened only 5 days ago. Maybe things will finally move15:57
*** silor1 has joined #openstack-swift15:57
*** silor has quit IRC15:58
*** silor1 is now known as silor15:58
*** daemontool has quit IRC15:58
*** dustins has quit IRC15:58
*** dustins has joined #openstack-swift15:58
*** daemontool has joined #openstack-swift15:59
*** haomaiwang has quit IRC16:01
*** haomaiwang has joined #openstack-swift16:01
*** swiftchobo has joined #openstack-swift16:03
swiftchoboplease could you explain to me that the effect of configuration parameters defined in /etc/swift/***.conf.16:04
pchngswiftchobo: Configuration parameters are documented here: http://docs.openstack.org/developer/swift/deployment_guide.html16:06
*** proteusguy__ has joined #openstack-swift16:06
pchngswiftchobo: Is there a question about a specific configuration parameter you had?16:06
*** rcernin has quit IRC16:09
swiftchobopchng : what are proxy-server.conf-write_affinity, read_affinity parameter16:09
*** wuhg has quit IRC16:11
pchngswiftchobo: I am not really familiar with those, but there is some documentation about it in this admin guide: http://docs.openstack.org/developer/swift/admin_guide.html#geographically-distributed-clusters16:12
*** sayali has quit IRC16:13
*** jmccarthy has quit IRC16:14
swiftchobopchng : Okay. I will check it. thank you16:15
*** jlhinson has joined #openstack-swift16:17
*** wolsen has joined #openstack-swift16:19
*** diazjf has joined #openstack-swift16:22
*** klrmn2 has quit IRC16:24
*** wbhuber_ has joined #openstack-swift16:24
*** sayali has joined #openstack-swift16:26
*** wbhuber has quit IRC16:26
*** jmccarthy has joined #openstack-swift16:27
*** trifon has quit IRC16:30
*** jmccarthy has quit IRC16:32
*** jmccarthy has joined #openstack-swift16:33
timburkejrichli: interesting...it makes sense that if seg_resp.etag is None, then seg_hash would be as well. i'm assuming some recent crypto work started passing back an etag, which triggered the md5 check16:39
timburkein the error that gets logged, does the "headers had ____" etag match either of the encrypted etags?16:40
jrichlitimburke: i'll have an answer in a big16:42
jrichlis/big/bit/16:42
*** jlhinson has quit IRC16:45
*** jlhinson has joined #openstack-swift16:48
jrichlitimburke: I am pretty sure its the one that is encrypted with the obj key.  it just isnt decrypted because the crypto-meta is not present at decrypt time.  that is what i am looking into now.16:49
jrichlibut, there is no added passing back an etag or anything from the time when the items were "none" last time I tested.  at least, nothing that was knowingly different.  i guess a bug could have been fixed.16:51
timburkejrichli: cool, makes sense. another thought i had: you said yesterday that the slo PUT path isn't working yet, so...how is encryption coming into play here? is it that we had an existing slo, then replaced some segments with encrypted versions of themselves?16:51
*** blmartin has quit IRC16:51
timburkehuh16:51
timburkeweird16:51
*** haomaiwang has quit IRC16:51
jrichlithe slo PUT path does work for the normal stuff on our feature branch16:51
*** blmartin has joined #openstack-swift16:52
timburkeagainst encrypted segments?16:52
jrichliit just isnt working right now with my patch that replaces fake_footers with real footers support16:52
*** chsc has joined #openstack-swift16:52
timburkeah :)16:52
*** petertr7 is now known as petertr7_away16:52
jrichliyes, against encrypted segments - normal slo PUT works16:52
timburkelove it16:52
*** arnox has quit IRC16:54
notmynamegood morning (again)16:57
*** tongli has joined #openstack-swift17:00
*** jlhinson_ has joined #openstack-swift17:00
*** jlhinson has quit IRC17:02
*** klrmn1 has joined #openstack-swift17:03
*** SkyRocknRoll_ has joined #openstack-swift17:05
*** diazjf has quit IRC17:06
*** diazjf has joined #openstack-swift17:08
*** petertr7_away is now known as petertr717:09
*** sileht has quit IRC17:10
*** sileht has joined #openstack-swift17:11
*** jistr has quit IRC17:11
notmynamejoeljwright: timburke: follow-up from last week's py26 conversation. I think we should drop py26 in swiftclient, and if a user is on a distro (ie red hat) that is supporting/pinning py26, the user needs to ask that distro for support17:13
notmynamejoeljwright: timburke: I also think this should warrant a major version bump for swiftclinet17:13
joeljwrightnotmyname: agreed17:13
joeljwrightI found enough ways to work round the problem17:13
joeljwrightius and softwarecollections both have a parallel installable py2717:14
timburkeseems fair. if we're planning a major version bump *anyway*, are there any breaking api changes we'd like to see?17:14
*** diazjf has quit IRC17:16
joeljwrightnotmyname: are we talking 2.6 -> 2.7?17:16
joeljwrightwith 2.7+ losing py26 support (nice parallel there)17:17
timburkejoeljwright: 2.6 -> 3.017:17
timburkedun-dun-DUUUUN17:18
joeljwright:D17:18
joeljwrightare there any swift api updates in the pipeline?17:18
joeljwrightthere have been chats about a v2 api bubbling under the surface for a while17:18
joeljwrightbut not sure if anyone's actually serious…17:19
*** diazjf has joined #openstack-swift17:19
timburkei think probably not terribly. notmyname is over at my desk now, though, tossing about the idea of changing the default auth version as part of this...i was thinking purely client api breaks17:19
*** daemontool has quit IRC17:20
*** diazjf1 has joined #openstack-swift17:20
*** gyee has joined #openstack-swift17:20
joeljwrightthere was a patch a while back that we couldn't implement because of potentially breaking clients17:21
timburkeyeah...lemme look for that...17:21
* joeljwright digging in history17:22
joeljwrighthttps://review.openstack.org/#/c/189815/17:22
timburkeyep, just found it too :)17:23
*** diazjf has quit IRC17:23
notmynamethere would essentially be 2 releases if we did that. the last 2.x series and then a 3.017:23
joeljwrightit never felt important enough at the time17:23
joeljwrightI'd be happy enough to cut a 2.7 release and leave breaking changes for now17:24
notmynamegenerally, I'd think that it's not good to go do a lot of API changes right now just because we're doing a bump for py support17:24
notmynameyeah, what joeljwright said :-)17:24
joeljwrightit's no great trauma to install python27 in parallel to the system python on rhel based distros anyway17:25
notmynamehowever, we had talked about raising the default auth version and that would be a major version bump too17:26
joeljwrightyeah17:26
notmynamethat's the only think I'd consider17:26
notmyname*thing17:27
joeljwrightit needs to be done eventually17:27
joeljwrightand now is as good a time as any :)17:27
notmynamemaybe ;-)17:27
joeljwrightnow, how do we make pip flash red 'sorry I broke your client' messages on install...?17:28
notmynamefor the auth version change?17:29
joeljwrightyeah :)17:29
joeljwrightlet's face it, that's gonna bite more people17:29
notmynameoh yeah17:29
notmynameincluding us. since there's no way right now for a v3 auth to even work with our out-of-the box installs17:30
notmynameor rather it just makes it a lot harder (/me hasn't looked at the keystone vSAIO stuff yet)17:31
joeljwrightand you're still putting this forward? you might want to check whether any of your support guys are dangerous first...17:31
notmynamelol17:31
notmynameit's the swift devs I'm more worried about ;-)17:31
joeljwright:)17:31
notmynameI think the default auth version bump should happen. and whenever it does happen, it should have a major version bump17:32
notmynamealso, we probably shouldn't do more than one major version bump in a single openstack release cycle (that's an assumption)17:32
*** jordanP has quit IRC17:33
notmynamebut changing the default will have significant impact on end users17:33
joeljwrightso do we lose py26 now and go to 2.7.017:34
joeljwrightand save the authv3 for the N-release?17:34
joeljwrightor intermediate release17:34
*** tsg has joined #openstack-swift17:34
joeljwrighttie the client breaking change to a significant event17:34
notmynameIIRC what we talked about in tokyo is making a right default based on envvars and options given, and then respecting the version if explicitly given17:35
notmynamejoeljwright: I don't know17:35
* joeljwright needs to look carefully at all those envvars17:35
timburkenotmyname: i have a fear related to the auth v3 idea...i feel like it'll just push more swift devs to curl instead of our own cli17:35
notmynameon the other hand, waiting another 6 months for an auth version update seems to be a long time17:35
timburkeit's a hell of a lot of env vars...17:36
notmynamethe OS_* ones17:36
notmynametimburke: I *like* curl ;-)17:36
timburkei like it too! but if we want a better client, we need people using the client, finding the stupid, and fixing it17:37
*** jlhinson_ has quit IRC17:37
joeljwrightthere must be combinations of OS_* options that would let us make the right choice in most cases17:39
notmynamemaking the right choice based on the options given might be a reasonable midpoint17:39
notmynameyeah17:39
notmynameand that might be possible to do in a release along with another major version bump17:41
notmynameor rather, would that require a major version bump at all?17:41
joeljwrightis parsing the auth_url not also an option for a guess?17:41
notmynameI don't know. someone said that's a bad idea. or that we can't/shouldn't rely on that for version discovery17:42
notmynamebut I don't remember who was saying that or why17:42
*** jlhinson_ has joined #openstack-swift17:42
joeljwrightit could always be a maybe17:42
notmynameit's definitely a maybe17:42
*** jmccarthy has quit IRC17:43
joeljwrightI think there will be plenty of people with string opinons later17:43
notmynamewe're never short on that17:43
joeljwrightI know I currently don't specify an auth version in the OS_* vars we give to clients17:43
joeljwrightso I'm somewhat nervous17:43
joeljwrightwould like to hear other people's opinions on the matter17:45
*** mac_ified has joined #openstack-swift17:45
notmynamedefinitely17:46
notmynameI added it to a swiftclient topic for today's meeting17:46
joeljwrightI wonder if the ksclient has any logic for this…17:46
*** jlhinson_ has quit IRC17:46
*** jlhinson_ has joined #openstack-swift17:47
*** diazjf1 has quit IRC17:47
*** hseipp has quit IRC17:48
*** rledisez has quit IRC17:50
*** aix has quit IRC17:51
joeljwrightwe must be able to query the auth_url for supported versions17:52
notmynamejoeljwright: timburke: the side effect of even just the py26 stuff is doing another swiftclient release, so that's good :-)17:52
notmynamejoeljwright: yeah, I think keystone supports that17:52
*** zhill has joined #openstack-swift17:52
joeljwrightso let's pick a supported version starting at 3.0 :)17:53
*** petertr7 is now known as petertr7_away17:53
joeljwrightI'll have a think about this before the meeting17:53
joeljwrightunfortunately I now have to go collect my wife from the station and buy&cook dinner :S17:54
*** diazjf has joined #openstack-swift17:54
*** jerrygb_ has joined #openstack-swift17:54
*** jerrygb_ has quit IRC17:54
*** diazjf has quit IRC17:55
*** jerrygb_ has joined #openstack-swift17:55
*** diazjf has joined #openstack-swift17:57
*** jerrygb has quit IRC17:57
*** jerrygb_ has quit IRC17:59
*** jerrygb_ has joined #openstack-swift18:01
*** jerrygb_ has quit IRC18:01
*** itlinux has joined #openstack-swift18:01
*** jerrygb_ has joined #openstack-swift18:01
*** joeljwright has quit IRC18:01
*** wbhuber has joined #openstack-swift18:04
*** daemontool has joined #openstack-swift18:05
*** wbhuber_ has quit IRC18:07
*** daemontool has quit IRC18:16
*** jerrygb_ has quit IRC18:17
*** diazjf has quit IRC18:18
*** mac_ified has quit IRC18:20
*** wbhuber_ has joined #openstack-swift18:42
*** trifon has joined #openstack-swift18:43
*** jerrygb has joined #openstack-swift18:44
*** wbhuber has quit IRC18:44
*** petertr7_away is now known as petertr718:45
*** geaaru has quit IRC18:51
*** badari has joined #openstack-swift18:56
*** petertr7 is now known as petertr7_away19:00
*** jlhinson_ has quit IRC19:00
*** wbhuber has joined #openstack-swift19:06
*** itlinux_ has joined #openstack-swift19:07
*** wbhuber_ has quit IRC19:08
*** diazjf has joined #openstack-swift19:08
*** itlinux has quit IRC19:08
*** petertr7_away is now known as petertr719:21
*** bapalm has quit IRC19:23
*** wbhuber_ has joined #openstack-swift19:25
*** bapalm has joined #openstack-swift19:26
*** wbhuber has quit IRC19:28
*** blmartin has quit IRC19:28
*** mahatic has quit IRC19:30
*** jlhinson_ has joined #openstack-swift19:41
*** afaranha has joined #openstack-swift19:47
*** tsg has quit IRC19:48
*** onder has quit IRC19:50
*** onder has joined #openstack-swift19:53
afaranhaHey people, I just tried to run the devstack with swift and I got an error while installing the PyECLib: http://paste.openstack.org/show/479333/19:57
afaranhacan anyone see this problem?19:57
*** nadeem has joined #openstack-swift19:57
afaranhadoes anyone have any idea?19:58
*** itlinux_ has quit IRC19:59
NMafaranha: Can you do  pip install PyECLib===1.0.7  ?19:59
*** zhill has quit IRC19:59
afaranhaNM, yes, I can19:59
afaranhait install without problem, but when running stack.sh it raises error20:00
afaranhaI just tried run "from pyeclib.ec_iface import ECDriver, ECDriverError, VALID_EC_TYPES" in a python and I got an error20:00
afaranhaah, When running pip install I get: "Failed to build PyECLib"20:01
*** itlinux has joined #openstack-swift20:02
afaranhaNM, some news, running pip install with -v we got: Error: liberasurecode-1.0.7 build failed!  Please install liberasurecode-1.0.7 manually and retry20:03
*** silor1 has joined #openstack-swift20:06
*** diazjf has quit IRC20:08
*** jlhinson_ has quit IRC20:08
*** NM has quit IRC20:09
*** diazjf has joined #openstack-swift20:09
*** [1]eranrom has joined #openstack-swift20:09
*** petertr7 is now known as petertr7_away20:09
*** silor has quit IRC20:09
*** silor1 is now known as silor20:09
*** jlhinson_ has joined #openstack-swift20:09
*** itlinux has quit IRC20:10
*** wbhuber_ is now known as wbhuber20:11
*** itlinux has joined #openstack-swift20:11
[1]eranromGreetings everyone. I got a question on get_info https://github.com/openstack/swift/blob/master/swift/proxy/controllers/base.py Line 53420:11
*** zhill has joined #openstack-swift20:13
[1]eranromseems like we rely on GetorHead_base to populate the environment with the info, which is true only on another environment. I beleive that the env GetorHead_base operates on is different then the response env in line 53420:13
[1]eranromI feel I am missing something, as this implies that any first call to get_info just populates the cache but do not actually return anything usefull20:14
[1]eranroms/usefull/useful20:14
*** swiftchobo has quit IRC20:15
*** silor1 has joined #openstack-swift20:15
*** silor has quit IRC20:16
*** silor1 is now known as silor20:16
*** diazjf has quit IRC20:16
*** NM has joined #openstack-swift20:17
*** diazjf has joined #openstack-swift20:18
NMNice. Can you find that lib?20:18
*** petertr7_away is now known as petertr720:24
*** diazjf1 has joined #openstack-swift20:27
*** diazjf has quit IRC20:30
*** diazjf has joined #openstack-swift20:30
*** diazjf2 has joined #openstack-swift20:31
*** itlinux has quit IRC20:33
*** diazjf1 has quit IRC20:34
*** diazjf has quit IRC20:34
*** tongli has quit IRC20:38
*** ho-mobile has joined #openstack-swift20:46
*** SkyRocknRoll has quit IRC20:48
*** ho-mobile_ has joined #openstack-swift20:48
*** wbhuber has quit IRC20:49
*** wbhuber has joined #openstack-swift20:50
*** ho-mobile has quit IRC20:51
*** ho-mobile_ is now known as ho-mobile20:51
*** itlinux has joined #openstack-swift20:51
*** vinsh_ has joined #openstack-swift20:51
torgomatic[1]eranrom: yeah, that get_info stuff is pretty darn crazy IMO. should probably get reworked to populate stuff based on the response, not based on fishing around in getorhead_base20:52
*** vinsh has quit IRC20:54
torgomaticat a minimum, get_info() doesn't work with the 410 response from an account, for example20:54
torgomaticlike, if you GET or HEAD an account that's recently deleted, the proxy gives you back a 410 instead of a 40420:54
torgomaticbut get_info() returns 404 because it's doing its work underneath that code, down in the base controller, instead of reading the response20:55
torgomatica coworker of mine ran into that recently20:55
torgomaticI'm sure there are other such things that you also can't read via get_info() as well, but that's the only one I can think of at the moment.20:56
mattoliveraumorning20:57
torgomatichi20:57
notmynamemeeting time21:00
[1]eranromtorgomatic: ok, thanks very much for this.21:00
*** vinsh has joined #openstack-swift21:00
*** vinsh_ has quit IRC21:00
*** [1]eranrom has left #openstack-swift21:00
*** joeljwright has joined #openstack-swift21:02
*** ChanServ sets mode: +v joeljwright21:02
*** mzhou has quit IRC21:02
*** openstack has joined #openstack-swift21:05
*** itlinux has quit IRC21:05
*** trifon has quit IRC21:07
*** petertr7 is now known as petertr7_away21:08
*** petertr7_away is now known as petertr721:09
*** jistr has joined #openstack-swift21:10
*** jerrygb_ has joined #openstack-swift21:12
*** NM has quit IRC21:12
timburkeoh, joeljwright: mind taking a look at patch 226897 when you get a chance? i think it should address the underlying issue that patch 246943 is trying to work around21:13
patchbottimburke: https://review.openstack.org/#/c/226897/ - Make LengthWrappers resettable if their _readable ...21:13
patchbottimburke: https://review.openstack.org/#/c/246943/ - Fix swift client reauthentication21:13
joeljwrighttimburke: kk, will do21:13
timburkethanks!21:14
*** jerrygb has quit IRC21:15
*** jerrygb_ has quit IRC21:22
*** diazjf2 has quit IRC21:22
*** jerrygb_ has joined #openstack-swift21:23
*** breitz1 is now known as breitz21:25
*** jerrygb__ has joined #openstack-swift21:26
*** jerrygb__ has quit IRC21:27
*** jerrygb has joined #openstack-swift21:28
*** jerrygb_ has quit IRC21:28
openstackgerritAlan Erwin proposed openstack/swift-specs: Updating spec to reflect a few changes.  https://review.openstack.org/24718521:28
*** silor has quit IRC21:30
*** diazjf has joined #openstack-swift21:30
*** diazjf has joined #openstack-swift21:30
*** mzhou has joined #openstack-swift21:37
*** jistr has quit IRC21:41
*** blmartin has joined #openstack-swift21:44
*** dustins has quit IRC21:45
*** NM has joined #openstack-swift21:47
*** tsg has joined #openstack-swift21:56
*** ho-mobile has quit IRC21:57
notmynametsg: oh hi. just missed the meeting. any updates on pyeclib? just waiting for those distro packaging things?21:57
gmmahanotmyname: didnt want the hijack the meeting, but when my env has both ST_* and OS_* set & keystone enabled, swift client always default to ST_* and fails to auth with keystone. is this what discover feature will solve?21:59
notmynamegmmaha: not immediately21:59
notmynamegmmaha: the plan would be that if you had no ST_* options and no auth version specified, it would do auth version discoverability22:00
gmmahaaaah22:00
*** jerrygb has quit IRC22:00
gmmahaguess then the other is an enhancement at some point.. cool.. thanks22:00
notmynameof course, if you want to future-proof your app, always explicitly set the auth version22:01
jrichliwill a lot of people be out next week due to Thanksgiving?  I will be on holiday technically, but will prob still be doing swift and asking questions :-)22:01
notmynameoh yeah22:02
notmynamenext week22:02
notmynameis US holiday22:02
* torgomatic will be in a turkey coma and completely unresponsive22:02
acolesyeah i was wondering if you guys would be here22:02
gmmahanotmyname: yeah. thats definitely something that might solve that22:02
notmynameshouldn't be any meeting22:02
torgomaticso, you know, about like normal22:02
notmynametorgomatic: on wednesday?22:02
wbhuberlol at torgomatic22:02
minwoobQuick question22:03
acolesnotmyname: all week is holiday?22:03
wbhuberacoles: usually thurs and fri of next week22:03
notmynameI'm taking off wed-fri22:03
minwoobwhy do I not see the process/daemon that I created, when I do swift-init, or even a ps -ef ?22:03
acoleswbhuber: ah22:03
notmynameI'll be on an airplane or in a car during the meeting time next week22:03
joeljwrightnotmyname: I'll be away for the first half of December, so I guess the next meeting I can make is Dec 16th22:04
minwoobI've modified common/manager.py, bin, setup.cfg, and added the actual file in swift/container/22:04
joeljwrightbut I'll keep an eye out for swiftclient docs info22:04
jrichlinotmyname: anywhere new?22:04
notmynamejoeljwright: yikes. ok. we definitely should be talking more than just in the meetings though :-)22:04
acolesjoeljwright has THE best plans for december!22:04
joeljwrightacoles: don't tell everyone!22:05
notmynamejrichli: I'll be in the exciting town of dallas ;-)22:05
acolesjrichli: which days are you on holiday?22:05
notmynamejrichli: majorca? ibiza?22:05
notmynamejoeljwright: ^22:05
acolesjoeljwright: i was vague!22:05
acoles:)22:05
jrichliBig D!  have fun22:05
joeljwrightnotmyname: Zermatt22:05
notmynamenice22:06
joeljwrighttaking a course to become a ski guide22:06
notmynamecool22:06
tsgnotmyname: sorry missed the meeting due to a conflict.  on the pkg updates, the backports (https://bugs.launchpad.net/trusty-backports/+bug/1515710) are still pending, although I finally heard from ubuntu-backporters - need to resubmit using the ubuntu tools, which I will do this afternoon.22:06
openstackLaunchpad bug 1515710 in vivid-backports "Please backport liberasurecode-1.1.0, python-pyeclib-1.1.1 to precise, trusty, vivid" [Undecided,Invalid]22:06
jrichliacoles: again, technically - I am taking off work all week next week.  but that just means I might be more productive ;-)22:06
wbhuberjrichli: that makes 2 of us22:06
tsgIn the meantime, I submitted a new review request for devstack to get liberasurecode-dev in - you may have seen it22:06
notmynamejrichli: I think that's a common sentiment among swift devs ;-)22:06
acolesjrichli: no, think long term, take your vacation22:07
mattoliverauminwoob: does it need a configuration section in a config or it's own config?22:08
jrichliacoles: its ok, really.  i will get enough of the relaxing stuff.  the work parts i will do are just the fun parts of work - i wont have to do the things that feel more like work.22:08
gmmahanotmyname: if you dont mind, wanted to get your thoughts on one more thing. I ran the testr patch and functests on my SAIO setup and ran into a bunch of errors22:08
gmmahahttp://paste.openstack.org/show/479352/22:08
minwoobmattoliverau: I modified the container-server config file and added it in (just like the container-sharder).22:08
gmmahai did verify that all the requirements are satisfied. Not sure what i might be missing22:08
minwoobmattoliverau: This is for the local table partitioner daemon that I'm trying to create, btw.22:09
mattoliverauminwoob: I guessed as much ;)22:09
minwooblol22:09
mattoliverauminwoob: you said you added it to manager.py, when happens when you start it manually "swift-init <service> start -n"22:11
notmynamegmmaha: looking22:11
gmmahanotmyname: thanks.. No rush, i am most certain its a user error.. ;)22:11
*** diazjf has quit IRC22:12
notmynamegmmaha: you've validated your test.conf22:12
notmyname?22:12
minwoobmattoliverau: dev@dev-VirtualBox:~/swift/bin$ sudo swift-init container-partitioner start -n22:13
gmmahayes. havent changed it in a while.. its the one i have been using22:13
minwoobStarting container-partitioner...(/etc/swift/container-server/1.conf)22:13
gmmahawithout the testr patch, functests passes just fine22:13
minwoobswift-container-partitioner does not exist22:13
* gmmaha goes to look at it again22:13
minwoobThat's it.22:13
notmynamegmmaha: that was just an initial guess. not sure what's going on22:13
minwoobIt's there in bin, though.22:13
minwoobPermissions are the same, etc.22:14
gmmahanotmyname: if it helps, this is my test.conf http://paste.openstack.org/show/479354/22:15
mattoliverauminwoob: have you 'pip installed -e .' or 'sudo python setup.py develop' in case it isn't actually installed to the system properly22:15
minwoobmattoliverau: I issued setup.py again and now it works.22:17
minwoobThanks a lot!22:17
*** diazjf has joined #openstack-swift22:18
minwoobI'll let you know how the rest of it goes.22:18
mattoliverauminwoob: awesome! please do!22:19
*** acoles is now known as acoles_22:31
pelusecan't seem to get the final steps of keystone on my SAIO working right.  can interact with keystone OK, created all the relevant enteties I think and updated my proxy settings.  Puking on me though and likely something silly.  When anyone has a second: https://gist.github.com/anonymous/bfa221c146ffad435eb822:31
*** petertr7 is now known as petertr7_away22:35
timburkepeluse: what are these headers? i thought v2 auth expected you to POST json...?22:36
pelusetimburke: beats the hell out of me, I can't find devent docs showing exactly what is supposed to work so I've tried a few different things, that's just the last thing I tried22:36
notmynamepeluse: heh https://twitter.com/alrs/status/66710789324672614522:37
timburkegive http://developer.openstack.org/api-ref-identity-v2.html a try22:37
pelusenotmyname: no shit!22:38
*** tsg has quit IRC22:38
timburkepeluse: i'm guessing you'll want something like {"auth":{"tenantName":"swift","passwordCredentials":{"username":"swift","password":"swift"}}}22:39
pelusetimburke: yeah, was just looking at the output of 'get versions' and trying to figure out which is the simplest22:39
*** jrichli has quit IRC22:40
*** diazjf has quit IRC22:42
*** diazjf has joined #openstack-swift22:43
pelusebah nothing works, this is a pile of dogshit22:44
notmynamepeluse: http://cdn.honestnetworker.com/dns/CRVCgHBUEAAj-Vg.mp4.gif22:49
*** rjaiswal has joined #openstack-swift22:52
peluseok, still dogshit but timburke helped me get a token finally.... thanks tim!23:04
timburkehappy to help23:04
*** jerrygb has joined #openstack-swift23:05
*** tsg has joined #openstack-swift23:05
peluseBTW if anyone is looking for a simple curl to get a token:  curl -d '{"auth": {"tenantName": "swift", "passwordCredentials": {"username": "swift", "password": "swift"}}}' -H 'Content-type: application/json' http://localhost:5000/v2.0/tokens23:06
*** km has joined #openstack-swift23:06
*** km is now known as Guest5379423:07
*** arcimboldo has joined #openstack-swift23:08
arcimboldohi all, I'm trying to configure swift to use keystone, but I get EmptyCatalog on proxy-server23:09
*** jerrygb has quit IRC23:09
*** minwoob has quit IRC23:11
*** jerrygb has joined #openstack-swift23:14
pelusearcimboldo: I'm no expert but just now got my swift & keystone working.  I only have like 3 min before I'm walking out the door but if you can explain a little more about what's happening maybe I can help23:15
arcimboldothnx peluse, I think I've got it23:15
arcimboldoi was using the wrong sample fro proxy-server.conf23:15
hrouAh no meeting next week !   Cool that means I'll just get to bug some of you here re: symlinks on this channel ;)23:16
pelusearcimboldo: great!23:16
arcimboldoI've spent like 2 hours before finding the correct configuration :(23:17
notmynamearcimboldo: is there something you can recommend changing in the sample config or the docs? if it's fresh on your mind, any help would be appreciated23:17
notmyname(even just filing a bug)23:18
arcimboldowell, I'm not sure it's an issue with the doc or the sample23:19
*** jlhinson_ has quit IRC23:19
arcimboldofor sure, the sample in /usr/share/doc/swift-proxy for the debian package is incorrect23:19
arcimboldomissing identity_uri, still using auth_host auth_port etc23:20
arcimboldothen, I was following the documentation in http://docs.openstack.org/developer/swift/admin_guide.html and then liberty, but I was using kilo :(23:20
notmynamearcimboldo: in which section? auth_token?23:20
arcimboldo[filter:authtoken]23:20
arcimboldobtw: it's never clear what you are supposed to do when using keystone v3 or v223:21
notmynamearcimboldo: how does https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L284 look?23:21
arcimboldolooks like the one I'm using23:21
notmynamethe good one or the bad one?23:21
arcimboldobut the debian package is not using that23:21
arcimboldoyes the good one23:21
notmynameoh23:21
notmynamezigo: :-(23:21
notmynamezigo: who'd you pass the debian swift packaging on to?23:22
*** kei_yama has joined #openstack-swift23:25
arcimboldook, now I can list, but can't create a container :(23:25
arcimboldoit gives me a 40423:26
notmynamearcimboldo: what the list you did that worked?23:32
arcimboldoswift -V 3 list23:33
arcimboldoalso openstack container list23:33
notmynameand you got an empty list back?23:33
notmynamebut you can't create a container?23:33
arcimboldoyes23:33
arcimboldoyes23:33
arcimboldoNov 18 23:34:18 swift1 proxy-server: 172.23.0.3 172.23.0.3 18/Nov/2015/23/34/18 POST /v1/AUTH_6a8c8c3ed987477b82f475742d695fef/test HTTP/1.0 400 - python-swiftclient-2.6.0 137d8ebd3d3c457c... - 65 - tx15e05f44603c4e6da4960-00564d0afa - 0.0264 - - 1447889658.110282898 1447889658.13668608723:34
notmynamemy first guess would be incorrect roles settings in keystone23:34
*** chsc has quit IRC23:34
claygreading the logs I think the meeting went smoother without me today23:34
arcimboldoI am admin and _member_ in this tenant, and operator_roles = admin, Member, _member_, user in [filter:keystoneauth]23:35
notmynamearcimboldo: that sounds right23:37
notmynamearcimboldo: do you have account_autocreate set to true? https://github.com/openstack/swift/blob/master/etc/proxy-server.conf-sample#L13123:37
arcimboldouhm23:38
arcimboldoyes23:38
arcimboldoI wonder23:38
arcimboldoI might have created the rings wrong23:38
arcimboldothey *all* use port 600223:38
notmynamearcimboldo: ok. use `swift-recon --validate-servers`23:39
notmynameoh, yeah. that's a problem23:39
arcimboldoI've just deleted rings and data, will re-run my playbook23:40
arcimboldook, while it runs, I have a more theoretical question23:40
notmynameoh. not normally the first swift troubleshooting step I'd suggest ;-)23:40
arcimboldoit's just 2 vms23:41
*** asettle is now known as asettle-bfast23:41
arcimboldoso my question is: how swift distribute the data over the various servers? And what happens when a disk/server goes down?23:41
arcimboldoI have experience with ceph, which is, I think, pretty different23:41
arcimboldo(btw, now it created the container)23:42
notmynamearcimboldo: using a modified consistent hash ring. oh. similar to crush. and swift works around failures. a disk failure rebuilds the data somewhere else. a server failure doesn't23:42
arcimboldoso, when a disk fails, it automatically rebuilds the data somewhere else, as ceph does?23:42
timburkejoeljwright: thanks for the review! don't forget to sleep, though :)23:43
notmynamearcimboldo: here's a starting point on the swift rings http://docs.openstack.org/developer/swift/overview_ring.html23:43
arcimboldois it the *-sync daemons taht take care of syncing the data?23:43
arcimboldoand what happen if I have, let's say, only one replica and the server goes down?23:43
arcimboldo(let's say one replica for objects, but 3 replica for container and accounts)23:44
notmynamearcimboldo: yes. for an individual drive failure. for a server failure, it's treated as an availability issue, not durability. https://www.youtube.com/watch?v=_sUvfGKhaMo23:44
*** joeljwright has quit IRC23:44
notmynameif you have one replica and the server goes down, you won't be able to read it. but overwrites will work, and everything will be resolved (based on timestamps) when the server comes back up23:44
arcimboldothe idea I have in mind is to use swift for some sort of not-necessarly-always-online storage, for archive/cold storage23:45
arcimboldoand I would like to keep the cost down23:45
arcimboldosince we will need a lot of data23:45
notmynamewhere are your costs today?23:45
arcimboldowhat do you mean?23:45
notmynameare your costs around keeping servers online?23:46
notmynameas opposed to drives or other hardware23:46
arcimboldowell, there is the energy question, of course, but this it's unavoidable unless we use tapes23:46
arcimboldoand there is the large amount of disks/servers you need to have a storage with 3 replica23:47
arcimboldowhat I would like to have it's something that I guess it's not possible right now23:47
notmynameI know many people using swift for archival(-ish) storage. but swift is designed for high availability23:47
*** mwheckmann has quit IRC23:47
notmynamethere's also erasure codes you could use to get better storage efficiency23:47
arcimboldoI would like to fill the storage pods and *power off* them23:47
arcimboldowhen they are full23:47
notmynameno, swift isn't designed for turning off servers23:47
*** diazjf has quit IRC23:47
arcimboldoit would be enough to spin down the disks23:48
notmynamethat won't work since the data distribution is all about spreading it across every server (drive) in the cluster23:48
arcimboldobut I can't find anything open source to do it23:48
arcimboldoI see23:48
notmynamethat sounds similar to facebook's haystack. or maybe whatever backblaze uses23:48
notmyname(guessing on backblaze)23:48
arcimboldothat's right23:49
arcimboldoI think it makes sense23:49
notmynamewhat scale of storage are you talking about?23:49
arcimboldoI work for an university, and we have this use cases, where people produces huge amount of data that *might need in 2-3 years or maybe also next week*23:49
notmynameright :-)23:50
arcimboldo~10PB23:50
notmynameyeah, swift is really really good at that (keeping stuff available for whenever you need it)23:50
arcimboldobut if it works, can very easily reach the 100PB over the years23:50
notmynamecompany I work for is called swiftstack and we have several university (and research/scientific) customers doing stuff very similar to what you describe23:50
notmynamearcimboldo: that's what everyone says ;-)23:51
*** m_kazuhiro has joined #openstack-swift23:51
*** ho has joined #openstack-swift23:51
arcimboldonotmyname, well, what can we do: they have microscopes and sequencing machines that produce 100x times the data they used to produce 10 years ago23:51
notmynameoh cool!23:52
arcimboldoand the researchers often do not have the experience to filter the raw data23:52
arcimboldoit's a brand new world for a lot of scientists23:52
notmynameany chance you're using illumina sequencers? I've seen those hacked to put the data directly in swift23:52
notmynameoh yeah23:52
arcimboldoyes we have them23:52
arcimboldoreally? That's really interesting23:52
arcimboldodo you have any pointers?23:52
notmynamecool. we've been working with hudson alpha for putting stuff directly from the sequencers into swift23:53
arcimboldo(please do not quote xkcd: https://xkcd.com/138/)23:53
notmynamelol23:53
notmynameactually, get the book :-) https://swiftstack.com/books/23:53
notmynamethe genomics book talks all about that workflow23:53
notmynamefill out the marketing form (of course ;-) and we'll mail you one for free23:54
notmynamearcimboldo: where are you based?23:54
arcimboldowhich form?23:55
arcimboldozurich23:55
notmynamearcimboldo: sorry, this one http://info.swiftstack.com/object-storage-for-genomics-deploying-and-administering-openstack-swift-with-swiftstack-for-bioinformatics23:56
arcimboldothe "preorder"?23:56
arcimboldoah ok23:56
notmynameyeah, I just asked our marketing people about that "preorder" word. doesn't make sense23:56
*** eariasn_ has quit IRC23:56
arcimboldowhat would the marketing people say about the "added value" of swiftstack? :)23:57
notmyname"lots"23:57
notmyname;-)23:57
notmyname(1) I try to keep marketing stuff out of this community channel (2) swiftstack's product is about making it easier for you to deploy, manage, and run swift23:58
arcimboldoI just wanted to know if it's "just" a deployment solution or you build something on top of swift23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!