19:00:43 <notmyname> #startmeeting swift
19:00:44 <openstack> Meeting started Wed Jan 23 19:00:43 2013 UTC.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:00:45 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:00:47 <openstack> The meeting name has been set to 'swift'
19:01:12 <notmyname> I sent out an agenda outline to the mailing list
19:01:40 <notmyname> first up: slight change in versioning/tagging at common release time
19:01:50 <notmyname> nothing major, but something I think you should be aware of
19:02:23 <notmyname> old way: we release when we're ready (hopefully close to the combined release time), and our last release during the cycle gets included
19:02:58 <notmyname> in the case that we want to cut another release before the combined release comes out, we do it and versions get bumped
19:03:39 <notmyname> new way: we release when we're ready (hopefully close to the combined rleease time), and that does the normal milestone-proposed dance as normal
19:04:24 <notmyname> except now, at the combined release time, the final version on milestone-proposed will not get the final version flag until the actual release, and instead will get a -rcX tag
19:05:09 <notmyname> no real change to us, but the impact is that we cannot release a full new version between what we think will be in the combined release and when the combined release actually releases
19:05:19 <notmyname> any questions?
19:05:21 <chmouel> so on milestone-proposed we get -rc? and on release remove the -rc?
19:05:41 <notmyname> chmouel: technically, a new non-rc tag (rather than deleting a tag), but yes
19:05:54 <chmouel> ok
19:05:59 <swifterdarrell> notmyname: "combined release" is like Essex or Folsom?  or something inbetween?
19:06:11 <notmyname> ya, the essex, folsom, etc releases
19:06:34 <notmyname> but since the milestone-proposed will still be "active", we cannot use it for another release until the combined release comes out
19:07:17 <notmyname> oh, I should use the meetbot flags
19:07:26 <swifterdarrell> so the tradeoff is potentially backporting (to -rcX) instead of rushing a new release if we have late changes which are important (between the milestone-proposed release and combined release), correct?
19:07:36 <notmyname> correct
19:07:55 <caitlinbestler> Does this reduce the amount of project-wide testing on a non-openstack-integrated release?
19:07:57 <notmyname> but that's essentially what we do now anyway
19:08:31 <tongli> @notmyname, is there a link to access the 'what's new in the release on Swift'?
19:09:03 <notmyname> caitlinbestler: I think this mostly comes from distros being able to plan for the combined release. I think it may help the openstack CI team have consistent version numbers, though
19:09:30 <notmyname> tongli: the CHANGELOG os the authoritative source, but I also normally send out an email about it too
19:09:30 <malini> notmyname: -rcX is the X an integer, just bunp up X? or is it -rcGrisly .. I am wondering because we have milestones such as G1, G2 etc
19:09:37 <creiht> notmyname: in other words, avoids everyone from going crazy because we add a .1 to the release? :)
19:09:40 <notmyname> malini: integer
19:09:45 <notmyname> creiht: ya :-)
19:09:51 <creiht> lol
19:10:04 <notmyname> it doesn't really change anything we've been doing, but I'm told it makes other people's lives easier :-)
19:10:10 <creiht> seems like a reasonable compromise though
19:10:18 <creiht> just funny
19:10:20 <notmyname> ya
19:11:01 <notmyname> #info swift candidates for the openstack combined release will have an -rcX tag until the combined release actually happens to avoid a version number dance in the case of backports
19:11:11 <swifterdarrell> important changes between milestone-proposed release and combined release are a PITA any way you slice it, so seems reasonable (not any worse, etc.)
19:11:25 <notmyname> yup
19:11:42 <notmyname> #topic swift next API wiki
19:12:03 <notmyname> creiht and chmouel talked about and put together http://wiki.openstack.org/SwiftNextAPI. awesome. let's all add to it
19:12:08 <notmyname> #link http://wiki.openstack.org/SwiftNextAPI
19:13:08 <clayg> cool hadn't seen that
19:13:17 <notmyname> some of the minor changes are things we should consider addressing sooner than later, but for the big stuff, I'd like you to think about "what does swift 2.0 mean?"
19:13:58 <notmyname> creiht: thanks for putting it up
19:14:45 <notmyname> I don't have much to add there. just wanted to say I liked it and make sure people knew about it
19:14:59 <notmyname> #topic pycon sprint
19:15:33 <malini> For me from an API standpoint, it means new user functionality.  Including server side encryption, it could include stuff like cold-storage, reduce number of copies
19:15:35 <notmyname> swiftstack and redhat are working together to host a sprint at pycon this year. our goal is not to work on swift directly, but to encourage client apps written against the swift api
19:15:47 <creiht> notmyname: I'm planning on going through the apis before the conference
19:15:58 <notmyname> If you would like to be involved, please let me know
19:16:20 <notmyname> creiht: cool
19:16:52 <cschwede> i'd like to be involved, might add some code to server side encryption
19:17:08 <notmyname> we have 2 ideas for the sprint, and I'd like your feedback: 1) add swift support to tools like boto and s3cmd 2) hackathon to build swift clients
19:17:29 <notmyname> thoughts?
19:17:42 <notmyname> what would be interesting to you to hack on?
19:17:45 <creiht> notmyname: That will likely depends on who shows up for the sprints right?
19:18:11 <notmyname> creiht: it does affect how we promote it though
19:18:15 <chmouel> will probably not be able to make it
19:18:28 <creiht> true
19:18:38 <creiht> I'm still looking into weather I can stay for the sprints or not
19:19:06 <notmyname> although the sprints are 3 days after the conference, I think we'll just be doing stuff on monday the 18th
19:19:14 <notmyname> a single day
19:19:18 <malini> I am new to all things Swift and would like to participate .. so I have some ramping to do till March before hackthon. Just went through John's swift workshop from the last summit.
19:19:48 <creiht> k
19:20:12 <malini> +1 for swift clients
19:20:31 <notmyname> malini: that's what I'm leaning towards too
19:20:53 <swifterdarrell> I like the adding-swift-support to existing tools idea... since this is PyCon, we'd be talking tools implemented in python or python client libraries, right?
19:20:57 <creiht> notmyname: that's cool with me, perhaps with a "or we will help you with whatever you would like to hack on"
19:21:00 <clayg> yeah I'd rather see a new tool that is built for swift than adding swift support to something I don't know the guts of and has an impedence mismatch
19:21:18 <clayg> heh
19:21:19 <notmyname> creiht: of course
19:21:31 <cschwede> does swift clients includes addons to the current swift.py?
19:21:31 <notmyname> swifterdarrell: clayg: can I agree with both of you? ;-)
19:22:04 <creiht> notmyname: another intersting one would be improving swift3
19:22:20 * clayg is not that interested...
19:22:27 <clayg> ;)
19:22:28 <notmyname> cschwede: perhaps, but also like "build a better cyberduck" or, in a hackathon sense, "build a small site that stores data in swift"
19:22:39 <notmyname> creiht: indeed
19:22:58 <creiht> lol
19:23:21 <creiht> notmyname: I think a general cross-platform gui written in python would be very nice
19:23:28 <creiht> and well received
19:23:48 <notmyname> ya
19:23:49 <clayg> cross-platform?  you mean the web right?
19:23:57 <cschwede> creiht: web or desktop?
19:24:01 <clayg> lol
19:24:07 <clayg> srly who does desktop apps anymore
19:24:07 <creiht> of course there is also improving swiftly, or the user space wrappers (like fuse, etc.)
19:24:15 <creiht> cschwede: desktop
19:24:30 <tongli> probably swift-dropbox for mobile device is the best thing to do.
19:24:36 <creiht> if you are looking at web based, actually it wouldn't be a bad idea to improve horizon support
19:24:43 <swifterdarrell> cschwede: I hope so :)  having python-swiftclient be a well-regarded, easy to use  python client for Swift is a good goal; I'd like python devs who want to store and retrieve things to look at python-swiftclient and say, "Oh yeah, this is a breeze and does what I want" (not saying it's not already--I haven't looked at it from that perspective)
19:24:44 <chmouel> or improve things like that https://github.com/reidrac/swift-nbd-server
19:25:07 <creiht> I think there are plenty of opportunities
19:25:13 <creiht> just depends on the interest of those that show up
19:25:14 <notmyname> ya, tons of great ideas
19:25:32 <chmouel> we probably should start  a wiki page with the list of ideas
19:25:39 <swifterdarrell> clayg: who does desktop apps?  Um, Cyberduck?  (which keeps coming up from real people as a client they're pointing at Swift)
19:25:45 <tongli> I've noticed a lot of people have dropbox on their phone these days, including few 9 year-olders and I was bit surprised.
19:25:53 <clayg> I acctally had a question about the api thing (tahnks for bringing that up notmyname) - it kinda moved by quickly - should I wait?
19:26:10 <creiht> tongli: it is more difficult to do mobile dev with python
19:26:22 <notmyname> clayg: let's move on. next topic is future plans for swift, so it fits there
19:26:34 <notmyname> thanks for the great ideas for the sprint
19:26:34 <tongli> developed a mobile app based on swift API will be useful and get product recongized by large number of people quickly.
19:26:54 <creiht> tongli: I totaly agree that mobile apps would also be useful :)
19:26:54 <tongli> true!
19:27:05 <notmyname> tongli: provide a moble framework that app devs can use to store data in swift (ie what dropbox is doing)
19:27:07 <malini> i like tongli mobile drop-box idea
19:27:21 <malini> that meets GUI, swift client ..
19:27:37 <creiht> malini: but this is a python conference, and the sprints are usually on python apps
19:27:43 <creiht> which don't match well with mobile
19:27:50 <chmouel> yeah not sure about the java part
19:27:53 <tongli> since we were talking about client. right true for that conf.
19:27:56 <notmyname> creiht: it did with the nokia OS ;-)
19:28:14 <tongli> but client app can be built regardless that that meeting/conf
19:28:21 <creiht> oh great, so the 1 person with a nokia os phone and a swift cluster to talk to could use it :)
19:28:26 <creiht> tongli: certainly
19:28:36 <tongli> it will spread the Swift to bigger crowd.
19:28:37 <notmyname> creiht: symbian
19:28:57 <notmyname> creiht: http://en.wikipedia.org/wiki/Python_for_S60
19:29:15 <notmyname> ok, moving on... :-)
19:29:20 <creiht> lol
19:29:23 <notmyname> #topic swift in 2013
19:29:26 <swifterdarrell> we gonna wikify the ideas somewhere?
19:29:38 <notmyname> swifterdarrell: ya, I'll scape the transcript for it
19:29:42 <creiht> we should start a wiki to keep track of all the stuff we want to wiki
19:29:43 <swifterdarrell> cool
19:29:46 <notmyname> clayg: what's your API question?
19:30:28 <clayg> well so... we have some "optional" middleware right now "enahcnes" the api
19:31:19 <clayg> do those ever becomes part of the core api?  Like when you talk about what a POST means to the swift api - you have to talk about multipart forms and form-post (e.g.)
19:33:00 <creiht> *crickets*
19:33:01 <creiht> :)
19:33:02 <clayg> I just ask because we talk about client tools - they want to know what they can count on from a swift endpoint, and a major version bump might be a good time to "officialize" some of the api enhancements that already have momentum - or double check to see if they have any warts?
19:33:06 <notmyname> clayg: so since there is no doc defining the swift api as a spec, the "swift API" has pretty much meant "what can an app reasonably expect a deployer to do"
19:33:40 <notmyname> ya, a mojor version bump (or any version bump) would require some more formal definition
19:33:46 <notmyname> *probably
19:34:50 <swifterdarrell> discoverability of what's supported in a particular deployment's implementation of the API could be handy; don't want to overcomplicate things, but being able to tell something about what middleware is around or how proxy-server's configured, etc could be useful
19:34:56 <notmyname> but I don't think that code organization affects what the API is (at least too much). IOW, it doesn't matter from an API perspective if it's in middleware or in the process
19:35:07 <swifterdarrell> I know the idea of exposing the configured swift limits was floated at one point
19:35:31 <clayg> ^ good example!
19:35:32 <uvirtbot> clayg: Error: "good" is not a valid command.
19:35:43 <clayg> i hate you SO much uvirtbot
19:36:09 <yuanz> notmyname: can I say that you will focus on the multi-region for swift 2013? seems there are lots of dependency on this in the blueprints page
19:36:17 <notmyname> clayg: example of discovering configured constraints as part of the API?
19:36:28 <swifterdarrell> clayg: Error: "i hate you SO much uvirtbot" is not nice.
19:36:31 <torgomatic> I like the middleware-queryability idea
19:36:43 <notmyname> yuanz: ya, I'll get to that in just a bit
19:36:45 <creiht> https://developers.google.com/discovery/
19:37:19 <clayg> swifterdarrell: sorry
19:37:34 <swifterdarrell> torgomatic: I guess you can always "query" it by trying the API a middleware might be implementing and deduce its absence from the response
19:37:44 <alpha_ori> creiht: interesting.
19:37:47 <swifterdarrell> torgomatic: doh, misread that as "I don't like"
19:37:52 <clayg> notmyname: yeah something like discovery would "need to be there" so if it's done in middleware it couldn't really be "optional" in your pipeline?
19:38:05 <notmyname> swifterdarrell: essentially the web client model for feature degredation
19:38:14 <creiht> alpha_ori: yeah I haven't had a chance to dig real deep into it yet, but seems like they have done a lot in that domain
19:38:33 <notmyname> clayg: ya, but things like cache and check_errors aren't really optional either
19:38:48 <alpha_ori> notmyname: That's what we do now with our web client to some degree.
19:39:15 <alpha_ori> notmyname: but that puts some burden on the client.
19:39:45 <notmyname> ya, it's not great. it's where we are today, and I do like the idea of discoverability
19:40:16 <creiht> the easiest (and could still be v1 compatible) would be to add some sort of capabilities resource
19:40:25 <caitlinbestler> Aren't we talking about two different things here? 1) a presumably wsgi method to find out exactly what middleware modules are enabled, and 2) what the relevant configuration options are for the swift relevant modules?
19:40:29 <creiht> a client could get that to know what apis are available
19:40:38 <malini> would python decorators saying deprecated be adequate (java has deprecated tags)
19:40:45 <creiht> caitlinbestler: it could be depending on how it is implemented
19:40:51 <clayg> discoverability is great for *optional* elements, my originial question was really about getting things that clients want to count on in the api spec explicitly
19:40:57 <swifterdarrell> Swift API 2.0 (or whatever) could define an API middlewares would implement, with, perhaps, proxy-server coughing up the proper "not present" response (for when an optional middleware isn't in the pipeline)
19:41:11 <notmyname> creiht: ya, I've got a (really, really dumb 30-minutes-spent-on-it) middleware in my github account that does that
19:41:39 <notmyname> clayg: that would mean that we'd have to be explicit about the api
19:41:44 <clayg> I also don't like the idea of "discoveribility" being "what middleware is in the pipeline" - I should be able to conform to the api (including a well speced ehancement) even if I'm running a *different* middleware that the one that comes stock?
19:41:45 <notmyname> which would be a change
19:42:29 <notmyname> clayg: like the deb packaging "provides" directive (eg you get "mail" with both postfix and sendmail)
19:42:33 <alpha_ori> clayg: interesting, but that would require defining standards for features themselves.
19:42:43 <alpha_ori> clayg: not that we couldn't do that.
19:42:46 <torgomatic> could group things into conceptual "features"; then do something like OPTIONS /features/temporary-urls, and if it 404s, then tempurl's functionality isn't there
19:43:05 <alpha_ori> clayg: What does it mean to provide S3 compatibility, for instance?
19:43:20 <torgomatic> then you can reimplement tempurl's API to your heart's content, and just make sure your new fancy middleware says 200 to OPTIONS /features/temporary-urls
19:43:47 <swifterdarrell> clayg: my idea was that middleware would say kind of what it can do; so like you said, the Swift API would define an optional form-post subset; any middleware purporting to support form-post functionality would implement that API including answing some kind of discovery req about it
19:44:17 <clayg> swifterdarrell: ok, that sounds reasonable
19:44:19 <swifterdarrell> clayg: if an alternate implementation of Swift's optional subset API for form-post came along, it could conform to the discovery API response and be a drop-in replacement from a client perspective
19:44:26 <swifterdarrell> clayg: (the point of APIs, right?)
19:44:51 <alpha_ori> swifterdarrell: Do we yet have any examples of popular middleware that duplicate each other's efforts?
19:44:55 <clayg> alpha_ori: s3 is hard, but if someone wrote a "better than swift3" middleware for swift - clients checking for "does this endpoint support s3 api" shouldn't have to know if I've installed swift3 or the other?
19:44:58 <notmyname> alpha_ori: auth
19:45:24 <alpha_ori> notmyname: ok...anything else?
19:45:39 <notmyname> ok, we only have a few more minutes. let's put dsicoverability off till later (would make a great summit topic)
19:45:42 <notmyname> ya
19:45:57 <notmyname> general 2013 goals.
19:46:05 <clayg> swifterdarrell: I agree with 100%, i was worried for some reason that we'd  "return a list of middleware - discoverable - done"
19:46:21 <notmyname> what use case are we looking to solve or enhance this year?
19:46:43 <creiht> clayg: hehe
19:46:53 <creiht> clayg: you always asume the worst :)
19:46:55 <notmyname> global clusters is something that we (swiftstack) are working on now
19:47:17 <creiht> notmyname: any update as to where that is?
19:47:20 <swifterdarrell> As mentioned, global clusters has been a community desire since at least the Boston conference (and probably before)
19:47:23 <creiht> how things are coming?
19:47:42 <swifterdarrell> creiht: well the first two patches are languishing in Gerrit pretty well
19:47:45 <torgomatic> well, ring enhancements are sitting in Gerrit (https://review.openstack.org/18263)
19:47:58 <creiht> there is a lot languishing in gerrit :)
19:48:03 <notmyname> creiht: isn't one of those waiting for a follow up from you? ;-)
19:48:42 <creiht> notmyname: like I said, my list keeps getting longer :)
19:49:14 <notmyname> ya, so global clusters has some patches in gerrit now, and the other functionality will be coming, I hope, over the next month
19:49:35 <creiht> swifterdarrell: I was asking that rather than submit a couple of patches and wait, could you at least throw some stuff in WIP so people could see how things are taking shape overall?
19:50:20 <notmyname> creiht: our list is long too, we'll submit them when we can :-)
19:50:28 <swifterdarrell> creiht: because that'd speed up the reviews on the ready patches or because that'd be a good unrelated thing to have?
19:51:01 <creiht> ok, so then future patches aren't hinging on those approvals... good :)
19:51:05 <swifterdarrell> creiht: I think we'd like to submit a couple of patches and not have to wait (forever)
19:51:10 <creiht> swifterdarrell: it would just be nice to see where things are going
19:51:13 <swifterdarrell> ;)
19:51:31 <notmyname> beyond global clusters, are there any other big things like that that your associated companies are working on? tiering? encryption ( caitlinbestler?) etc?
19:51:45 <creiht> swifterdarrell: I think we can all agree that we all have a lot on our plates, and we are getting to things as we can
19:51:55 <swifterdarrell> creiht: yup!
19:51:59 <notmyname> generally, what are you working on or planning that will go into the core of Swift?
19:52:47 <cschwede> Maybe not a big thing - but I'm working on a quota middleware
19:53:17 <creiht> cschwede: have you seen the quota stuff that is currently in review?
19:53:19 <notmyname> cschwede: ah, cool. have you seen mike barton's (redbo) patches
19:53:33 <swifterdarrell> We'll continue to improve Swift monitoring as we get to it (see: plates and the count of things on them), but nothing earth-shattering planned there
19:53:44 <cschwede> creiht: yes, my work is currently on an account level
19:54:09 <chmouel> we'll probaby work on some migration tools for swift (cluster to cluster) probably not something for Core
19:54:16 <cschwede> notmyname: no, will look into that. I talked to nijaba and chmouel (thanks for your comments!)
19:54:44 <notmyname> chmouel: cool
19:55:12 <clayg> wasn't rax looking at enhancements to object replication to leverage xfs internal data structures or do better prefix-hash-tree-rsync-faster-ninja-something?  or is object repliation still cycling "fast enough"?
19:55:24 <cschwede> notmyname: current working implementation supports tempauth, keystone and swauth: https://github.com/cschwede/swquota README is a little bit out of date - I'll update it tomorrow (added a tool for simpler quota set recently)
19:55:40 <creiht> clayg: replication enhancements are still being looked into, but I think they have moved onto another idea
19:55:43 <caitlinbestler_> Reconnecting -- on encryption - Nexenta is working on encryption but specifically on solutions that would *not* be part of swift core.
19:55:58 <yuanz> swifterdarrell: by monitoring do you mean the statsd+graphit work, or a new tool?
19:56:06 <notmyname> caitlinbestler_: ok, good to know (I liek that approach, thanks for your ML comments)
19:56:15 <clayg> creiht: awesome!
19:56:16 <chmouel> cschwede: nice.. i am planning to test that sometime next week and we'll give you some feedback it could be nice to merge this with the redbo's container quota
19:56:44 <notmyname> and we're pretty much out of time.. anything else for the last 3 minutes?
19:56:57 <swifterdarrell> yuanz: I generally mean changes to core swift for emission of monitoring data (eg. statsd, but not graphite or any other consumer of StatsD output, which is outside the scope of core swift code)
19:57:13 <notmyname> next meeting is Feb 6 at 1900UTC
19:57:18 <yuanz> ok
19:57:47 <notmyname> I'll put together a summary based on the transcript and send it out
19:57:49 <creiht> notmyname: redbo has some work on bit torrent support
19:57:52 <swifterdarrell> yuanz: an example of "ongoing improvement" to that would be https://review.openstack.org/#/c/20089/
19:57:57 <notmyname> creiht: nice!
19:58:01 <creiht> https://github.com/rackspace/cloudfiles-sworrent
19:58:08 <creiht> if anyone would like to provide some feedback
19:58:17 <notmyname> creiht: 404
19:58:19 <creiht> oh dangit
19:58:20 <creiht> heh
19:58:25 <creiht> I thought he made that public :)
19:58:42 <creiht> well once he is comfortable with others seeing it, I'll let you know :)
19:58:49 <notmyname> cool
19:58:51 <cschwede> chmouel: that would be nice - thanks!
19:59:26 <creiht> notmyname: I'm trying to play with better ways with handling metadata
19:59:33 <creiht> but that's very low priority
19:59:36 <creiht> at the moment
19:59:36 <notmyname> k
19:59:45 <notmyname> and we're out of time
19:59:56 <notmyname> thanks for attending. great feedback
19:59:59 <notmyname> #endmeeting