19:01:41 <notmyname> #startmeeting swift
19:01:42 <openstack> Meeting started Wed Feb  6 19:01:41 2013 UTC.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:43 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:45 <openstack> The meeting name has been set to 'swift'
19:01:56 <notmyname> should be a pretty short meeting, I think
19:02:04 <notmyname> at least I don't have much on my agenda to talk about
19:02:37 <notmyname> 1) is encryption in the scope of swift? 2) docs / copyright 3) any patches need considering
19:02:53 <notmyname> and at the end, torgomatic has volunteered to answer all your questions about global clusters
19:03:11 <notmyname> #topic is encryption in the scope of swift?
19:03:44 <notmyname> ok, there have been some mailing list threads and ideas floating around about adding various flavors of encryption to swift
19:04:05 <chmouel> I think having tools or client librarie making it maybe easier to encrypt bfore uploading to swift would be more in scope
19:04:26 <notmyname> there are 2 use cases that I know of: users don't want anyone to read their data and deployers want to easily recycle drives by simply throwing away a key
19:04:44 <redbo> dmcrypt is good for the second one
19:05:09 <swifterdarrell> We've had at least one hard requirement from a customer deploying Swift for encryption of data, but only at rest; that was solved with disk-level encryption out-of-band to Swift
19:05:11 <davidha> if adding encryption in the server side, we need key management
19:05:14 <notmyname> does anyone want to argue in favor of either of these being in scope for swift itself (ie should there be code in swift's codebase that handles this)?
19:05:15 <torgomatic> yep, dm-crypt or luks or something else underneath Swift
19:05:15 <chmouel> yeah that or luks
19:06:06 <notmyname> so does that mean we all agree that encryption should be out-of-band to swift itself?
19:06:19 <swifterdarrell> I'm perfectly happy with keeping encryption (and the associated problems and complexity which come with key management) outside the scope of Swift
19:06:20 <chmouel> +1 to out
19:06:27 <swifterdarrell> Push that stuff out to deployments
19:06:43 <swifterdarrell> (IOW +1 to out)
19:06:45 <torgomatic> +1 to encryption being outside the scope of Swift
19:06:54 <redbo> buy a drive shredder
19:07:09 <notmyname> redbo: or a ball-peen hammer ;-)
19:07:21 <davidha> How would you shred only files form one customer?
19:07:40 <notmyname> davidha: that data is spread out over all drives in the entire cluster
19:07:50 <redbo> why would I need to shred one customer's files?
19:08:13 <davidha> Right, this is a real use case for server side encryption.
19:08:42 <notmyname> "this"? the user use case or the deployer use case?
19:08:44 <swifterdarrell> Let's not pretend that encrypting data at rest and possibly in-flight as well are not valid requirements for an object storage system.  The question is whether that use-case should be satisfied by code in Swift or not.
19:08:55 <notmyname> swifterdarrell: exactly
19:09:38 <davidha> I think Swift should enable it - we at least need to see that one can add it to Swift
19:09:52 <notmyname> davidha: what is "it"?
19:10:05 <davidha> It being server side encryption
19:10:14 <notmyname> davidha: full drive encryption or encryption of the object data?
19:10:18 <davidha> It is possible it can be done as a middleware ... :)
19:10:42 <davidha> encr of the object data by  a per user key
19:11:10 <notmyname> davidha: so that the deployer then has access to both the encrypted data and the key? what use case does that solve?
19:11:31 <davidha> notmyname: Shreding of data whne the user leaves
19:11:54 <davidha> I need to go deaper on that
19:12:11 <davidha> I will ask the team here, lets leave that to next meeting
19:12:25 <redbo> who's that serving, the user or the provider?
19:12:47 <chmouel> that's going bad for performances for both
19:13:26 <notmyname> ok. for now it looks like we are agreed that encryption should stay outside of swift, unless there is another use case that cannot be efficiently implemented in middleware
19:13:40 <davidha> Agreed
19:14:05 <notmyname> #agreed that encryption should stay outside of swift
19:14:24 <notmyname> #topic docs and copyright
19:15:09 <notmyname> just as an FYI, as I've been looking at some of the docs lately (like the SAIO and multinode install instructions), it seems that they are getting somewhat dated
19:15:32 <notmyname> if you are looking for something to do in your free time (hah!), re-reading the docs is a good place to start
19:15:45 <notmyname> this applies as much to me as anyone
19:15:49 <notmyname> just wanted to bring it up
19:16:05 <chmouel> (or get the newcommers in your company to do it while learning swift 8-))
19:16:11 <notmyname> heh
19:16:19 <notmyname> chmouel: aren't you the newcomer in your company?
19:16:21 <notmyname> :-)
19:16:27 <chmouel> haha
19:16:55 <notmyname> annegentle proposed a patch that mentions copyright (https://review.openstack.org/#/c/18889/). looks like there may be a little more clarification needed (as per the comments)
19:17:09 <notmyname> but review the http://wiki.openstack.org/Documentation/Copyright page
19:17:47 <notmyname> also, I've heard that we should stop assigning copyright to openstack (btw, openstack llc doesn't exist any more) and should be keeping it for whatever company you work for
19:17:48 <swifterdarrell> (I hate to be annoying, but if we got legal guidance, we might as well follow all of it.)
19:18:38 <notmyname> #topic patches?
19:18:54 <notmyname> are there any patches that we need to discuss in here that can't be handled in gerrit comments?
19:19:10 <notmyname> https://review.openstack.org/#/q/status:open+swift,n,z
19:19:35 <davidha> I would like to see account qouta as an enhancement to the container one - seem to be trivial enhancement
19:19:57 <davidha> I can work on this unless this is already being worked on
19:19:59 <chmouel> davidha: yeah i can do that since we are going to use it
19:20:30 <notmyname> cool
19:20:54 <notmyname> redbo: any big reason not to extend your container quotas to handle entire accounts?
19:21:26 <swifterdarrell> patches welcomed, perhaps?  There's at least one account quota middleware floating around outside Swift (don't have link handy)
19:21:42 <chmouel> #link https://github.com/cschwede/swquota
19:22:00 <davidha> I think it should be a single middleware doing both
19:22:00 <swifterdarrell> chmouel: thanks!
19:22:42 <chmouel> #link https://github.com/AlexYangYu/StackLab-swift/tree/dev-quota
19:22:49 <notmyname> chmouel: davidha: I look forward to seeing your patches :-)
19:22:56 <redbo> Not that I know of.  It just wasn't a use case we had.
19:22:57 <chmouel> and last one
19:23:02 <chmouel> #link https://github.com/chmouel/swift-quota
19:23:14 <chmouel> (it actually works need to update the README)
19:23:25 <swifterdarrell> I'm happy having two middlewares with different names which differentiate their different behavior (most importantly that container quotas are mostly voluntary and account quotas would presumably be most useful to resellers enforcing restrictions on customers);  but that assesment of the use-cases could be biased/inaccurate.
19:24:19 <davidha> Why would that make more sense?  A single quota middleware will not be less efficient if only container quota is used
19:24:25 <chmouel> I think having them merged would make more sense
19:24:43 <swifterdarrell> IOW, I think if two things both implement "quotas" but they're substantially different, they needn't be implemented in one piece of code/middleware
19:24:44 <chmouel> it may be confusing for deployers and it's not much of biggie to have the two
19:25:01 <davidha> as much as I enjoy long pipelines..... this is becoming too long :)
19:25:08 <notmyname> chmouel: really? _this_ is what will be confusing to deployers about swift? ;-)
19:25:09 <redbo> we could call them user_quotas and provider_quotas or something
19:25:20 <chmouel> notmyname: good point :)
19:25:21 <redbo> or deployer_quotas
19:25:24 <chmouel> redbo: +1
19:25:35 <notmyname> redbo: +1
19:26:17 <chmouel> and a nice quota.txt explaining the difference between the two :)
19:26:22 <chmouel> .rst
19:26:40 <davidha> I would guess userrs would expect both to work together and be aligned, and the performance of both will be affected
19:26:44 <davidha> so -1 for me
19:27:15 <notmyname> let's carry this on in the gerrit comments on the patch that comes in
19:27:35 <notmyname> #topic other
19:27:53 <notmyname> anything else need to be brought up today?
19:28:08 <davidha> Is there any work on one of the following topics:
19:28:16 <davidha> Metadata, search
19:28:17 <davidha> ?
19:28:52 <notmyname> I wanted to point out that I got a lot of positive comments last week at linux conf australia. people like the stuff you all are working on. good job :-)
19:29:14 <torgomatic> davidha: you mean metadata beyond Swift's existing support for account/container/object metadata?
19:29:16 <notmyname> davidha: softlayer has a metadata search feature using an external system for their swift cluster
19:29:16 <chmouel> i think the guys of keystone v3 want to talk more about swift
19:29:24 <redbo> umm I have a branch somewhere where I just stuck a fts3 table in the container db and indexed metadata
19:29:25 <chmouel> swift integration i mean with ACL
19:29:30 <davidha> torgomatic: yep
19:30:03 <davidha> chmouel: right, I know of a change needed to support v3 also
19:30:22 <notmyname> chmouel: they can define whatever they want. the ACL tokens are opaque to swift, and the only thing that needs possibly updating is the keystoneauth middleware in swift. if they decide to change things, they'll have to accound for a migration path, of course
19:30:43 <zykes-> is v3 supported ? : p
19:30:54 <notmyname> zykes-: does v3 exist? ;-)
19:30:58 <davidha> We may need to add domain id as a MD for containers
19:31:02 <chmouel> notmyname: yep that's correct, it just has to be to authorize to whatever acl they are giving to us by auth_token
19:31:38 <davidha> Since there is not another hirarchy in v3, we would need to add some support for it to work
19:31:58 <chmouel> I don't think I would like much if they are taking in account migration (we have a large v2 swift/keystone cluster here)
19:32:07 <chmouel> if they are not
19:32:17 <zykes-> doesn't it in G notmyname ? :)
19:32:37 <notmyname> zykes-: I'm just half joking. I thought they were still defining it
19:32:59 <chmouel> from the discussion on the ml they seem so
19:33:08 <chmouel> (i.e: making email attr compulsory)
19:33:28 <notmyname> chmouel: but that shouldn't change anything in swift, from what I could tell
19:33:56 <chmouel> if they decide to give tenant_email instead of tenant_id
19:34:07 <chmouel> that would get difficult for migration
19:34:37 <torgomatic> so accounts would look like /v1.0/AUTH_#{tenant_email} ?
19:34:46 <notmyname> a user gives X to keystone in order to get back a token and and endpoint. swift's middleware then give the request and the token to keystone for authorization. I don't see what changes
19:35:09 <zykes-> notmyname: v3 apparantly has a goal to be fully implemented in G
19:35:15 <torgomatic> notmyname: as long as the endpoint is immutable, that's true
19:35:15 <redbo> torgomatic: I think he just means for acls
19:35:16 <chmouel> torgomatic: that's what i am worried about will clarify with those guys
19:35:17 <zykes-> fyi :)
19:35:24 <notmyname> they can redefine how they are passing out the endpoints all they want, but you can't do that for existing data
19:36:09 <torgomatic> redbo: I hope you're right
19:37:41 <notmyname> chmouel: you've got an existing keystone v2 cluster. when you get a clarifying answer, can you let me know what you find out? (I thought this had been settled on the ML)
19:38:15 <chmouel> notmyname: yep will do
19:38:19 <davidha> As far as I understand it, there is an additional hirarchy - this mean that you may have the same "account name"  in two different domains -  we would therfore need to store another attribute as a system MD being the domain id to which the account belong to
19:38:20 <notmyname> thanks
19:38:58 <redbo> we have accounts, keystone can delegate access to those however it wants
19:39:32 <davidha> redbo, the previous mapping betweeing users and account no longer stanbd
19:41:05 <davidha> I expect to see a blueprint on that soon that will elaboarte on how it can be done, than we can discuss issues it may bring
19:41:24 <redbo> there's been a huge mailing list discussion
19:41:48 <notmyname> let's see what chmouel comes back with
19:41:50 <notmyname> anything else that needs to be brought up today?
19:42:22 <torgomatic> if anyone has spare review time, the multi-range GET patch for segmented objects could use some eyeballs
19:42:44 <torgomatic> it's the oldest in the review queue, and it's been resurrected at least once
19:43:10 <notmyname> ok
19:43:25 <notmyname> next meeting is in two weeks
19:43:29 <notmyname> #endmeeting