19:00:54 <notmyname> #startmeeting swift
19:00:55 <openstack> Meeting started Wed Mar  6 19:00:54 2013 UTC.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:56 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:58 <openstack> The meeting name has been set to 'swift'
19:01:41 <notmyname> the openstack grizzly release is coming up soon, so we'll spend some time on that today
19:01:51 <notmyname> but first, some general announcements
19:02:21 <notmyname> #info reminder that the CFP for talks at the summit is open at summit.openstack.org
19:03:11 <notmyname> I'm curious as to how many swift devs will be coming from RAX. any idea right now?
19:03:45 <notmyname> dfg: have you heard anything?
19:03:55 <dfg> ya
19:04:09 <dfg> remember anything- is the question :)
19:04:11 <notmyname> heh
19:04:27 <davidh_> Do we have a deadline for submitting items for discussion? Can it wait till end of next week?
19:04:40 <notmyname> ok. well, if you all aren't there, I'll go gripe at your managers (who will be there) ;-)
19:04:51 <dfg> notmyname: chuck, redbo, maybe glange
19:05:19 <notmyname> davidh_: I don't remember the deadline off the top of my head. I think you've still got time. I'll need to check with ttx
19:05:25 <notmyname> dfg: k, thansk
19:06:05 <davidh_> As this is my first sumit -  these are items for technical discussions right?
19:06:11 <notmyname> as we get closer to the next swift release, please let me know if your email needs to change in the AUTHORS or .mailmap file. I'll be updating that soon. let me know via email, IRC, or with patches
19:06:37 <notmyname> davidh_: yes. these are intended to be more like facilitated discussions rather than official presentations
19:06:49 <davidh_> notmyname: great
19:07:05 <notmyname> davidh_: presenting material is ok, but you should have a good chunk of time devoted to discussion
19:07:16 <davidh_> np
19:07:33 <clayg> hi
19:07:45 <notmyname> also, if you haven't re-signed the oepnstack CLA, please do so ASAP. it's all done through gerrit now, and it should be much simpler (yay!)
19:08:06 <dfg> notmyname: are ther instructions somewhere?
19:08:21 <chmouel> there is a faq on the list that got posted
19:08:28 <chmouel> i can forward it to you if you like
19:08:38 <chmouel> done
19:08:40 <notmyname> chmouel: thanks (or paste a link here)
19:08:52 <dfg> chmouel: that would be nice- i delete a lot of openstack stuff :)
19:09:00 <chmouel> yeah i figured ;)
19:09:09 <notmyname> everyone needs to sign a CLA, even if you are also covered by a corporate one
19:09:56 <clayg> dfg: but you come to the meetings!  Should be easy to keep up with the important stuff...
19:10:20 <notmyname> last announcement: pycon starts next week (3/15) in Santa Clara. on monday 3/18 there is a swift sprint focused on building things for the swift API (ie apps). if you are going to pycon, try to stay around for the sprint too. we have fabulous prizes
19:10:54 <dfg> clayg: thats why go to the meetings. hi back btw
19:11:08 <notmyname> any questions so far? next topic is the swift 1.8 release
19:11:38 <chmouel> #link Gerrir CLA faq http://pastie.org/private/hkpk4lxekjlnhxs3w53dww
19:12:06 <notmyname> chmouel: thanks
19:12:07 <dfg> chmouel: thx
19:12:20 <notmyname> moving on to the upcoming release
19:12:28 <notmyname> #topic swift 1.8.0 for grizzly
19:12:53 <notmyname> the official openstack grizzly release is april 4
19:13:19 <notmyname> which pretty much gives us the rest of this month to finalize and QA our release
19:13:56 <notmyname> dfg: (since you're the only racker in here) can you confirm that y'all's QA will not be able to do a sign off by mid next week?
19:14:24 <dfg> notmyname: thats what i hear- they're being stretched pretty thin
19:14:34 <notmyname> ok
19:15:07 <dfg> plus there's a lot of new stuff
19:15:58 <notmyname> so that means we will cut our 1.8.0~rc1 next wednesday (the 13th). QA (from all of us, including RAX) will have the rest of the month to validate it. when/if things are found, patches will be backported and ~rcX+1 releases will be made
19:16:03 <notmyname> yes
19:16:31 <notmyname> the sooner we have a QA signoff, the sooner we don't have to worry about maintaining the separate milestone-proposed branch
19:16:37 <chmouel> when is the deadline for features to get merged?
19:16:57 <dfg> notmyname: if you hear a sneeze in the background in a few minutes- it was chuck
19:17:04 <dfg> (sorry)
19:17:06 <creiht> lol
19:17:17 <notmyname> chmouel: I'll cut the release with ttx next wednesday
19:17:27 <chmouel> notmyname: ok thanks.
19:17:32 <notmyname> chqyou have the advantage of being many time zones ahead of me :-)
19:17:48 <notmyname> chmouel: ^
19:17:53 <notmyname> dfg: :-)
19:17:57 <chmouel> heh
19:18:28 <notmyname> there are 3 patches that I would like to see merged by the release (of course if a patch is proposed, its author wants it merged..)
19:18:47 <notmyname> first, davidh_ wants https://review.openstack.org/#/c/23585/ reviewed and included if possible
19:19:03 <clayg> idk, sometimes I think redbo submits stuff that he doesn't acctually care if it gets merged...
19:19:09 <chmouel> it would be nice for us if we can get the quota account patch get merged  as well https://review.openstack.org/#/c/23434/
19:19:15 <creiht> clayg: hah
19:19:30 <notmyname> second, there is the separate replication network patch https://review.openstack.org/#/c/19618/ that needs reviewed. I'd liek to see it merged, if possible
19:19:36 <notmyname> chmouel: thanks. good to know
19:19:57 <notmyname> and torgomatic has a patch that he'll submit (today?) that includes the region tier
19:20:15 <notmyname> if you're wondering what patches to review first, please take a look at these 4
19:20:26 <torgomatic> I'll submit it as soon as its dependency clears Jenkins
19:20:57 <torgomatic> So between 10 and 10000 minutes from now ;)
19:21:39 <notmyname> don't forget about python-swiftclient patches. closer to the grizzly date I'll cut a new release there too
19:21:43 <tongli> @torgomatic, sam, this one https://review.openstack.org/#/c/22569/
19:22:11 <tongli> you asked for it to be fixed. please review it. it is about the swift client busy-waiting fix.
19:22:16 <notmyname> any questions about the swift 1.8.0 release or the schedule until 4/4?
19:22:17 <dfg> notmyname: btw- i'm just about done with static large update support for python-swiftclient
19:22:23 <notmyname> dfg: cool
19:22:50 <notmyname> dfg: I got to talk about that yesterday to someone who was visiting our office. he likes it :-)
19:22:52 <dfg> notmyname: i'll try to get in today so it can go out with the release i guess
19:22:59 <torgomatic> tongli: ok, I'll look at it soon
19:23:11 <dfg> notmyname: cool.
19:23:13 <tongli> it will be nice to include that as well in the relase.
19:23:15 <notmyname> dfg: no rush. the client release won't happen for another few weeks
19:23:22 <dfg> ok
19:23:43 <torgomatic> Although the client follows its own release cycle, so it doesn't have this hard deadline
19:23:50 <notmyname> if no questions about the 1.8.0 release, let's move on to a specific patch that needs some discussion
19:23:53 <chmouel> yep client is easy :)
19:23:54 <tongli> @notmyname, john, are you saying that swift and swift client releases are on different schedule?
19:24:12 <chmouel> tongli: everything openstack works like that
19:24:34 <notmyname> tongli: yes. they have a their own separate cadence (but in the case of the semi-annual openstack releases, it's nice to have them close)
19:24:48 <tongli> ok
19:24:55 <notmyname> #topic endpoints patch
19:25:11 <swifterdarrell> So I asked for this to be brought up...
19:25:27 <notmyname> a patch has been proposed that adds the funtionality to get the storage nodes for an object: https://review.openstack.org/#/c/21015/
19:25:45 <swifterdarrell> I'm happy with including it, but thought it was worth discussing one more time the criteria for inclusion in "core swift" for middleware
19:25:47 <notmyname> the question came up: "if this is included, why not swift3?"
19:26:13 <chmouel> swift3 is basically a screenscraping of an api
19:26:34 <davidh_> This patch allows a client to know on which nodes his data is right?
19:26:35 <chmouel> it's not really maintainable compared to the endpoint listing IMO
19:26:36 <notmyname> I've not heard anything say that this endpoints patch is bad functionality, but I think the question is more about if it should be in core (right swifterdarrell?)
19:26:49 <torgomatic> This is a reasonable extension to the swift API, while  swift3 implements a different API with the same functionality
19:26:57 <torgomatic> That's why, IMHO
19:27:09 <creiht> davidh_: this is needed for hadoop functionality to run on top of swift
19:27:12 <swifterdarrell> notmyname: correct, and I'm not even trying to argue that it shouldn't be in core (I did +2 it at one point, after all)
19:27:27 <davidh_> This patch is problematic - as it exposes internal info out
19:27:30 <creiht> notmyname: yeah I don't see why to not include it
19:27:37 <notmyname> I agree, re swift3: the S3 api is a separate (closed) api, and we've rejected other things like that too (*wink* tongli)
19:27:37 <creiht> davidh_: it is optional
19:27:54 <davidh_> There should be a way for haddop to get the info - but there shouldnt be a way for a regular client to get that info
19:28:04 <tongli> @notmyname, that is right john,
19:28:06 <torgomatic> Also it is sufficiently generic that any app that bypasses the proxy can use it; its not hadoop specific
19:28:06 <clayg> creiht: it's only "optional" until clients start depending on it
19:28:08 <davidh_> creiht: than its less of an issure
19:28:26 <cschwede> maybe limit the info to configurable client ips?
19:28:26 <creiht> clayg: how would a client depend on it?
19:28:30 <davidh_> Default should be not to offer this service
19:28:46 <clayg> creiht: a map reduce client?
19:29:01 <caitlin-nexenta> How could a client *ever* depend on it. The set of servers that an object is stored upon is subject to change without notice.
19:29:20 <notmyname> caitlin-nexenta: a client could depend on the info, not what the info is
19:29:27 <caitlin-nexenta> It's useful information, but you could never *rely* on an answer.
19:29:33 <clayg> sorry, client may have been the wrong term, a service that is using this info to implement it's funtionality
19:29:33 <chmouel> caitlin-nexenta: i think they are talking about providing an api contract
19:29:37 <davidh_> It assumes that the haddop is running on the same servers right?
19:29:49 <clayg> it's api access to the ring
19:30:04 <notmyname> it's more general than hadoop. it's..ya, what clayg jsut said
19:30:05 <clayg> why expose it all if NO ONE can use or depend on it
19:30:24 <clayg> neway, I think it's great, different than s3
19:30:36 <davidh_> clayg: why do we need to offer an API access to the ring
19:30:41 <clayg> I just think it's a paper tiger argument to keep calling api expansion "optional"
19:30:47 <cschwede> might be useful for a future erasure coding implementation as well (have to think about it)
19:31:15 <swifterdarrell> Okay, so the criteria for middleware inclusion is something like A) adds something (presumably useful) to the Swift API, and B) does not implement a different API than the Swift API?
19:31:21 <clayg> davidh_: I didn't write it, but the gneral idea is that stuff can do cleaver things running on the nodes, and if it has to first write a native binding to the ring structure (which may change) then it's a barrier to entry
19:32:01 <notmyname> swifterdarrell: well, there's also the "do the core devs want to maintain it"
19:32:08 <clayg> lol
19:32:13 <clayg> that's the only one that matters
19:32:25 <clayg> sorry, bad joke
19:32:34 <dfg> its not a lot of code
19:32:43 <cschwede> do the core devs maintain all patches from contributors?
19:32:51 <creiht> notmyname: so if you go down this route, then do you plan to do the same with the rest of the middleware?
19:32:59 <clayg> cschwede: only if they break or if we need to change something they use
19:33:00 <davidh_> This exposes internal data out - does not make sense -  if it is used only by internal elements - we need to offer an internal API - not one that can be used by a client
19:33:01 <notmyname> creiht: we tried that already :-)
19:33:02 <creiht> like for example the domain remap/cname stuff?
19:33:40 <creiht> davidh_: From their description the purpose would be to run on a proxy that is only running internally
19:33:46 <clayg> davidh_: I think there's a class of "services" that will run "in cluster" which can make use of ring information directly
19:34:11 <swifterdarrell> btw, IIRC, the "why" of this was discussed at the patch by the actual author
19:34:23 <creiht> swifterdarrell: correct
19:34:35 <creiht> they need the info for their hadoop integration
19:34:42 <SergeyLukjanov> this feature will provide an ability to run "in cluster" services using "data locality"
19:34:48 <davidh_> Using the ring directly is fine  as long as it uses the Ring.py or some CLI that uses Ring.Py
19:35:21 <dfg> i think its simple and generic enough to be useful to a bunch of different applications. also serves as an example of how you can do stuff like this.  i don't think the swift3 is a good comparison. what's the problem with it?
19:35:28 <torgomatic> also, it's as close to disabled-by-default as any middleware can be, in that it's not in the sample config's middleware pipeline
19:35:48 <swifterdarrell> Personally, I'm not interestedin a discussion of the merits of the patch for its intended purpose.  I care more about whether it's fine to include it in core and what the criteria for that are.
19:36:04 <swifterdarrell> So notmyname, you mentioned that core-dev interest in maintenance is a pre-req to middleware inclusion?
19:36:25 <creiht> well there isn't any official criteria that I am aware of
19:36:34 <creiht> so are you proposing that we define it?
19:36:35 <swifterdarrell> creiht: correct
19:36:38 <clayg> i feel like swifterdarrell wants to try out the vote fucntion in this meeting thing...
19:36:39 <torgomatic> swifterdarrell: I think what you said earlier are good criteria, FWIW.
19:36:41 <dfg> please lets not try to make one ;)
19:36:44 <creiht> lol
19:36:46 <notmyname> swifterdarrell: it's an implicit one, I think. your list earlier is good too
19:37:10 <torgomatic> although there's some official OpenStack thingy saying you can't implement third-party APIs unless you're Nova
19:37:23 <notmyname> the only real criteria there is right now is that we have one officially supported API (eg we wont' include S3 or CDMI in core)
19:37:28 <swifterdarrell> I don't care if it's "official"
19:37:47 <creiht> torgomatic: no that is up to the project to decide on 3rd party apis
19:37:54 <torgomatic> creiht: ah, I misunderstood then
19:38:00 <clayg> why do we need to "define the critera" if we can change them on a case by case basis... if it's just a question of posting a "guideline" somewhere we can point contributers to so they know our currently line of thinking... let's just put what swifterdarrell said in the sphinx docs under "contributing"
19:38:06 <creiht> to which we did decide to not include 3rd party apis
19:38:07 <notmyname> creiht: actually, torgomatic is right
19:38:17 <creiht> oh did they change that again?
19:38:32 <notmyname> creiht: ya, while you were off being a manager ;-)
19:38:36 <creiht> heh
19:38:54 <davidh_> Regardless of maintainance - the problem is not with what this patch wish to achieve  - it is with the how  - it should preferably not be an external API.
19:39:18 <tongli> I think that patch should be in is because it enables swift to serve the purpose that is important now and in future.
19:39:19 <notmyname> ok, so to summarize: there is no official requirements for inclusion in core other than it's got to be a good patch and can't implement a 3rd party API
19:39:22 <creiht> davidh_: so you suggest that we should have an official java api for the ring then?
19:39:28 <dfg> 2 core-devs approval means core-devs are willing to support it. i'll look at it again later and approve it. if someone else agrees- done.  what's wrong with that? any kind of official criteria can be put off til needed. cause it will be a pain
19:39:35 <swifterdarrell> sounds great; so there's no prob w/this patch or the account quota
19:39:40 <creiht> or insert favorite language
19:39:41 <torgomatic> notmyname: agreed 100%
19:39:44 <swifterdarrell> we done with this topic?
19:39:45 <swifterdarrell> :)
19:39:45 <notmyname> davidh_: that discussion should be taken to the review
19:39:48 <tongli> if someone wants to use swift for hadoop, then they can.
19:40:17 <davidh_> creiht: we will have (already have) a python one - if someone would convert it to a Java one - he can use it
19:40:33 <clayg> davidh_: until we change the ring - then he has to fix it
19:40:34 <swifterdarrell> dfg: agreed
19:40:40 <creiht> that's the problem is that it isn't trivial, and possibly very error prone
19:40:50 <davidh_> notmyname: ok.
19:40:51 <creiht> and the reason he wanted to expose it as an api
19:41:22 <notmyname> swifterdarrell: ya, I think we're done with this. you can move forward on your review now :-)
19:41:25 <clayg> ... so we DON'T get to use the vote feature :'(
19:41:31 <swifterdarrell> sweet
19:41:49 <swifterdarrell> clayg: sorry :)
19:41:52 <notmyname> #topic general discussion
19:41:53 <creiht> #vote no to bringing it to a vite
19:42:04 <notmyname> anything else?
19:42:04 <creiht> vote even
19:42:05 <creiht> :)
19:42:14 <clayg> object-replicator-2 looks pretty sweet
19:42:17 <notmyname> I wanted to ask gholt about a blueprint he has...
19:42:20 <notmyname> ya that one
19:42:30 <notmyname> but he's not here (and it's not time critical)
19:42:43 <creiht> I'm sure it will solve world hunger ;)
19:42:48 <clayg> last I checked there wasn't a sqlite schema, but the native rest calls where starting to form up
19:43:42 <notmyname> ok, let's call it. thanks for your time today
19:43:44 <davidh_> clayg: does replicator-2 uses rsync or just PUT?
19:43:45 <notmyname> #endmeeting