19:00:36 <notmyname> #startmeeting swift
19:00:37 <openstack> Meeting started Wed May 29 19:00:36 2013 UTC.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:38 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:40 <openstack> The meeting name has been set to 'swift'
19:01:00 <litong> @notmyname, good afternoon
19:01:06 <notmyname> hellow everyone
19:01:22 <notmyname> things to cover today: next swift release, API, summit, and a few other things
19:01:23 <lpabon> hello
19:01:31 <torgomatic> ahoy
19:01:37 <alexpecoraro> hi
19:01:40 <notmyname> first, a few simple things
19:01:40 <shri1> hey
19:01:42 <portante> helo
19:01:44 <portante> hello
19:01:50 <notmyname> #topic housekeeping/misc
19:02:19 <notmyname> The naming for the next OpenStack release (the "I" release) is open for a vote.
19:02:28 <notmyname> 'the poll is at https://launchpad.net/~openstack/+poll/i-release-naming
19:02:58 <notmyname> on the topic of openstack releases, the next summit is in hing kong
19:03:03 <notmyname> hong kong
19:03:15 <notmyname> dates are nov 5-8
19:03:15 <notmyname> http://www.openstack.org/summit/openstack-summit-hong-kong-2013/
19:03:38 <notmyname> and if you are a US citizen, here's your visa info: http://hongkong.usconsulate.gov/acs_hkvisa.html
19:04:01 <notmyname> if you are planning on attending, make sure your passport is up to date. it can take up to 2 months to get one, so do it now
19:04:48 <notmyname> a little more specific to swift now, let's talk about the next release of swift
19:05:03 <notmyname> #topic next swift release
19:05:22 <notmyname> I'd like to cut the next release when the final global clusters features land
19:05:42 <litong> @notmyname, for US citizen , you do not need visa if you stay there less than 90 days.
19:05:51 <notmyname> litong: correct
19:05:57 <litong> but you do need a valid passport.
19:06:00 <notmyname> the proxy affinity is the major one, but I've also got my eye on the cross-region replication
19:06:33 <notmyname> any comments or questions on when we should do the next release? is that a good plan or should we do something else?
19:07:32 <notmyname> I'll assume either everyone agrees or I've dropped out of IRC again... ;-)
19:07:42 <torgomatic> I'd like to see https://review.openstack.org/30797 merged before the next release
19:08:02 <portante> I'd like to see thread pools make the next release
19:08:05 <litong> @notmyname, cross-region replication is a good one.
19:08:10 <torgomatic> or at least, I'd like to see that bug fixed; doesn't have to be my patchset
19:08:23 <shri1> notmyname: whats the tentative release date for these features to be completed?
19:08:28 <notmyname> torgomatic: portante: agreed on both
19:08:42 <notmyname> shri1: there is no date as of yet. it's done when it's done
19:08:42 <shri1> *if these features have to be completed
19:08:49 <shri1> I see
19:08:51 <notmyname> shri1: sooner is better than later
19:09:11 <litong> if anyone can review this one (just need one more +1 to merge), that will be really nice. https://review.openstack.org/#/c/22569/
19:09:15 <notmyname> but aside from the 6 month openstack cycle, we aren't doing any sort of time-based releases
19:09:56 <portante> notmyname: do you want to keep the DiskFile and DatabaseBroker out of the next release to not delay it?
19:10:11 <notmyname> so basically, all this comes down to doing reviews
19:10:19 <notmyname> (what's new?)
19:10:24 <portante> ;)
19:10:40 <litong> @notmyname, yes, yes, yes. review, please
19:10:44 <notmyname> #topic reviews
19:10:48 <notmyname> speaking of reviews...
19:12:02 <notmyname> just a quick note, especially for core reviewers: if someone has already given a +2 and there is a minor issue (eg typo in docs or commit message), it probably shouldn't be given a -1 by another reviewer. that resets the previous +2 and slows everything down
19:12:51 <notmyname> quick follow-up commits could be a good way to address small issues like that, as long as master remains deployable
19:13:16 <notmyname> just something to keep in mind and try to see if we can speed up the review time
19:13:37 <notmyname> #topic bot in channel
19:13:50 * portante ack
19:14:10 <notmyname> last quick thing: I added the openstackgerrit bot to #openstack-swift. anyone think it should be removed?
19:14:22 <notmyname> it reports on new patches and merges
19:14:34 <notmyname> but if it gets too annoying, we can easily remove it
19:14:42 * torgomatic likes having the bot there
19:14:47 * portante agrees
19:15:10 <litong> I think it is a good thing.
19:15:27 <notmyname> cool. I like it too. let's keep it until it gets to be a problem
19:16:01 <notmyname> ogelbukh1: glad to see you. I'm interested in the mirantis patches wrt global clusters, but they are all marker WIP
19:16:23 <notmyname> ogelbukh1: basically, I want the functionality to be in the next release
19:16:39 <notmyname> and I doubt many people have looked at them since they are marked WIP
19:17:33 <dosaboy> is there a bp/wiki entry for the global clusters work?
19:18:01 <notmyname> dosaboy: https://blueprints.launchpad.net/swift/+spec/multi-region
19:18:17 <dosaboy> notmyname: cool thanks
19:18:26 <notmyname> portante: ready to talk about DiskFile?
19:18:35 <portante> sure
19:18:38 <notmyname> #topic DiskFile refactor
19:18:43 <notmyname> portante: go for it
19:19:06 <portante> so how many folks have heard about this effort?
19:19:17 <notmyname> o/
19:19:23 <portante> I not that notmyname and torgomatic have looked at?
19:19:42 <torgomatic> o/
19:19:47 <notmyname> (quiet crowd today)
19:20:17 <portante> From April's summit, we are working to refactor the DiskFile and DatabaseBroker to define them as "public" classes so that we can have various implementations of those classes
19:20:26 <alexpecoraro> i've heard of it
19:20:31 <portante> the existing implementation would be the first
19:20:50 <portante> to date, we've proposed https://review.openstack.org/#/c/30051/
19:21:04 <portante> which is a stab at refactoring just DiskFile to define the APIs
19:21:29 <portante> It still has 18 unit test failures, so it is a work-in-progress for sure
19:21:58 <portante> but I am seeing folks to take a look at the changes to see if they make sense and if they are in a direction we want to go in
19:22:09 <notmyname> aside from the minor comments I left, I like the direction of it
19:22:35 <torgomatic> I like the way it's going
19:22:52 <portante> notmyname has suggested that I pull out the "reader" work, which is sister work to the "writer" work performed in https://review.openstack.org/#/c/27149/
19:23:05 <portante> that might make it easier to review
19:23:07 <alexpecoraro> i haven't yet, but i'm interested in taking a look and will provide any feedback that i can think of
19:23:19 <portante> alexpecoraro: thanks
19:23:51 <portante> I personally think pulling the "reader" work out would be a good thing, but want to balance too many things to review against too large a change to review
19:25:10 <portante> The next question is how to take this in pieces so that it does not straddle a release and possibly cause problems
19:25:39 <portante> I would like each piece to be fully functional, but still, not a fan of half-way for a release
19:25:42 <notmyname> portante: well, anything that lands should be distinct enough to work itself
19:25:48 <torgomatic> is it really a problem if it straddles a release? I mean, at each step, Swift is still working
19:25:57 <notmyname> eg extracting the reader doesn't break anything
19:26:02 <portante> true enough
19:26:41 <portante> if folks are fine with that, then let's work to target the "reader" work pulled out of the next release, and then the full API definition to follow
19:26:52 <notmyname> I'm ok with that
19:26:55 <notmyname> anyone else have comments?
19:27:13 <portante> certainly I am available for questions afterwards
19:27:39 <litong> when it is in, how one can replace with a different implementation?
19:28:13 <portante> ah, em, ah, well, er, so, still working on those mechanics. :)
19:28:20 <notmyname> heh
19:28:35 <notmyname> worst-case (ie now), monkey-patching, right?
19:28:42 <portante> right
19:29:01 <litong> @portante, I am thinking about LTFS.
19:29:03 <portante> best case, defined backends associated with a know string
19:29:10 <portante> LFS patch?
19:29:13 <portante> litong?
19:29:20 <litong> right.
19:29:28 <litong> Linear Tape File System
19:29:36 <portante> The DiskFile and DatabaseBroker refactoring work is going to be a subclass of that
19:29:44 <litong> very low cost storage.
19:29:47 <portante> ah, yes
19:29:48 <portante> I see now
19:30:03 <litong> in thoery, we should be able to do that.
19:30:33 <portante> yes
19:30:33 <litong> swift cluster can be a mixed disk , tape, very fast storage and some what slow storage.
19:30:34 <notmyname> I've got a couple of uses too (the most obvious is optimizing some XFS-specific things, if possible)
19:30:59 <torgomatic> I'd like to use it to let me audit for missing objects on non-XFS filesystems
19:31:18 <notmyname> well, more quickly audit ;-)
19:31:32 <portante> to that end, it would be helpful to get the thread pools work in before this work
19:31:43 <torgomatic> notmyname: true; right now, though, it takes until another update invalidates hashes.pkl before anyone notices a missing file
19:31:45 <litong> so the question is, when this patch is in, will it be every node with same driver or object node can have mixed drivers.
19:31:49 <portante> I would like to make sure this refactoring is considered on top of that
19:31:52 <torgomatic> so let's say "update in bounded time" :)
19:32:51 <portante> Also, regarding the DatabaseBroker side, the work that David Haddas did for the "db_file" exists code, would be useful
19:32:51 <notmyname> litong: that will be determined by the yet-to-be-defined inclusion mechanism, I'm usre
19:33:13 <notmyname> portante: so what you're saying is "do more reviews" ;-)
19:33:14 <portante> Also, zaitcev proposed a patch there as well
19:33:18 <portante> ;)
19:33:25 * portante smiles quietly to himself
19:33:56 <notmyname> #agreed do more reviews
19:34:03 <notmyname> (now it's official)
19:34:11 <portante> ;)
19:34:13 <notmyname> ok, shall we move on?
19:34:18 <portante> I'm good
19:34:24 <notmyname> #topic swift API
19:34:33 <notmyname> last thing to discuss today
19:34:38 <notmyname> https://wiki.openstack.org/wiki/Swift/API
19:35:25 <notmyname> this is a description of what I believe is the subset of functionality that can be defined as "swift" and that we can therefore call swift API v1
19:35:52 <notmyname> but there are a couple of outstanding questions, and I'm not sure who has actually looked at the page (there certainly haven't been many edits)
19:36:00 <litong> @notmyname, totally agree with the first 2 items. ACL and Auth.
19:36:34 <litong> multi-range is not supported on large objects
19:36:56 <notmyname> the goal is not to restrict functionality but to find the subset of functionality that currently-deployed swift clusters (ie from the swift source code) support
19:37:03 <litong> we could have that, if the patch got reviewed. I worked on that for 4 months,
19:37:17 <notmyname> litong: right, I think that distinction was lost in my last edit
19:37:32 <portante> So ACLs is interesting, because the code chose key names stored in the metadata on disk already, right?
19:37:52 <litong> can we have that back? I mean the multi-range on large objects.
19:37:56 <portante> Most auth filters rely on that format, don't they?
19:38:21 <notmyname> portante: yes, probably
19:39:25 <portante> would it cause a problem to define ACLs as part of the v1 API?
19:39:53 <portante> it is kinda asymetrical, since an auth filter would not be part of the v1 API
19:40:18 <litong> @portante, once it is in API, it alomst feel like required. some company may want to ACL completely different way.
19:40:44 <portante> but this is what it is today, these ACLs are stored on disk that way
19:40:44 <litong> I mean "do ACL differently."
19:41:04 <litong> @portante, you do not have to use it.
19:41:19 <notmyname> I want to have a v2 API that has pretty much all of the functionality in the codebase, but I don't think we should ever deprecate v1. This is to give a baseline of what can be called an implementation of the swift api
19:41:26 <litong> you can add your own ACL at the proxy server any way you want.
19:42:24 <notmyname> portante: since ACL definitions are only meaningful to a particular auth system, it doesn't make sense to me to define an ACL format as part of the swift api (especially a v1 api)
19:42:48 <litong> @notmyname, not in v1, v2, or any other version.
19:42:55 <notmyname> perhaps v1 and v2 are bad names. perhaps "basic functionality" and "complete functionality" are better
19:43:20 <notmyname> litong: perhaps. I'm not sure I'm willing to go that far yet
19:43:23 <litong> it conflicts with separation of concerns (design principals)
19:43:50 <litong> no matter how you define it , someone won't like it
19:44:19 <portante> what are the metadata key names used by the tempauth and other auth system?
19:44:23 <notmyname> I'd like to agree that the wiki page is an accurate reflection of what we as a group can call the v1 API
19:44:27 <portante> do we have to define a names apced for the v1 API then?
19:44:55 <notmyname> portante: I'd just leave X-Container-Read and X-Container-Write out of it
19:45:13 <notmyname> the other questions I had were around tempurl and large object support
19:45:21 <portante> okay
19:46:26 <notmyname> I like them both. I want to have a definition that says both "here is something that defines swift" but also not so restrictive that existing clusters aren't swift any more
19:46:53 <notmyname> so at this very moment, I'm leaning towards not including them. but I go back and forth
19:47:01 <portante> is tempurl purely a filter?
19:47:23 <portante> Or does it rely on some code in the app server side?
19:47:32 <notmyname> it is implemented as such, but that is immaterial to its inclusion
19:48:24 <portante> hmm, messy
19:49:02 <notmyname> I think it's important to separate the implementation decisions (ie middleware or not) from the functionality
19:49:19 <notmyname> eg dynamic large objects vs static large objects. one is middleware and the other isn't
19:49:24 <torgomatic> notmyname: agreed 100%
19:50:03 <portante> well, if I have an implementation that can use the existing tempurl filter, then my implementation could say it supports it
19:50:05 <notmyname> to me the question is "when someone claims they have a swift cluster, what functionality should my client assume exists as a baseline?"
19:50:30 <portante> if I don't, then I have to implement it (like Ceph folks can use filters)
19:51:24 <notmyname> portante: correct. and in your case, when Red Hat says that gluster supports the Swift API, what does that mean for clients?
19:51:44 <portante> today, we support all the filters because we reuse most of the code
19:51:55 <notmyname> as you should!
19:51:58 <notmyname> ;-)
19:52:00 <portante> ;)
19:52:23 <torgomatic> the way I see it, if a cluster lets me set X-Account-Meta-Temp-Url-Key on my account, and then do some HMAC magic to make a URL like /v1/AUTH_me/con/obj?temp_url_sig=X&temp_url_expires=Y and that url *works*, then that cluster supports Swift signed URLs
19:52:30 <torgomatic> it's all from the client's perspective
19:52:34 <notmyname> yes
19:52:45 <portante> okay, I'm easily convinced
19:53:04 <notmyname> but should that be in the swift v1 api? :-)
19:53:12 <torgomatic> I don't care if the implementation is using swift.common.middleware.tempurl verbatim, hacked up, or if it's got a hand-rolled proxy server written in node.js and befunge in front that handles it ;)
19:53:26 <litong> @notmyname, if that is really useful to customers, yes it should be in.
19:53:34 <litong> we are defining an API after all.
19:53:38 * torgomatic thinks signed requests *should* be in the Swift v1 API
19:53:57 <notmyname> IOW, should any clusters out there that don't support it today no longer be able to be called a swift cluster?
19:54:25 <notmyname> litong: usefulness isn't the criteria IMO, since we are only formalizing an API post-facto
19:55:16 <notmyname> and we're running out of time, so it seems that there isn't a broad consensus
19:55:22 <portante> hmmm, its the post-facto thingy that has me wondering here, not the core principle (which I like)
19:56:11 <notmyname> portante: basically, I don't want so write a tester against the v1 spec and have mercado libre, wikimedia, vimeo, rackspace cloud files, or HP fail it :-)
19:56:18 <notmyname> and all the others
19:56:20 <portante> yes
19:56:36 <portante> that is why I am thinking it should be based on the app server code vs filter code
19:56:42 <portante> for post-facto definition
19:56:47 <portante> not for the principle or the future
19:57:02 <portante> define v1 based on that, immediately define v2 based on the principle
19:57:03 <litong> are we starting the API on the wrong foot?
19:57:43 <litong> I think stick with the principal is important. we can not change an api in v2 which was just defined in v1.
19:57:45 <portante> litong: it is the foot we are on already, I think.
19:58:08 <notmyname> we should continue this in the -swift channel
19:58:09 <portante> we won't be, if I understandthings, we'll only be adding to the API
19:58:11 <portante> k
19:58:16 <notmyname> we're out of time
19:58:25 <notmyname> #agreed to nothing yet
19:58:37 <notmyname> thanks for your time everyone
19:58:42 <notmyname> #endmeeting