19:00:43 #startmeeting swift 19:00:44 Meeting started Wed Jan 23 19:00:43 2013 UTC. The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot. 19:00:45 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 19:00:47 The meeting name has been set to 'swift' 19:01:12 I sent out an agenda outline to the mailing list 19:01:40 first up: slight change in versioning/tagging at common release time 19:01:50 nothing major, but something I think you should be aware of 19:02:23 old way: we release when we're ready (hopefully close to the combined release time), and our last release during the cycle gets included 19:02:58 in the case that we want to cut another release before the combined release comes out, we do it and versions get bumped 19:03:39 new way: we release when we're ready (hopefully close to the combined rleease time), and that does the normal milestone-proposed dance as normal 19:04:24 except now, at the combined release time, the final version on milestone-proposed will not get the final version flag until the actual release, and instead will get a -rcX tag 19:05:09 no real change to us, but the impact is that we cannot release a full new version between what we think will be in the combined release and when the combined release actually releases 19:05:19 any questions? 19:05:21 so on milestone-proposed we get -rc? and on release remove the -rc? 19:05:41 chmouel: technically, a new non-rc tag (rather than deleting a tag), but yes 19:05:54 ok 19:05:59 notmyname: "combined release" is like Essex or Folsom? or something inbetween? 19:06:11 ya, the essex, folsom, etc releases 19:06:34 but since the milestone-proposed will still be "active", we cannot use it for another release until the combined release comes out 19:07:17 oh, I should use the meetbot flags 19:07:26 so the tradeoff is potentially backporting (to -rcX) instead of rushing a new release if we have late changes which are important (between the milestone-proposed release and combined release), correct? 19:07:36 correct 19:07:55 Does this reduce the amount of project-wide testing on a non-openstack-integrated release? 19:07:57 but that's essentially what we do now anyway 19:08:31 @notmyname, is there a link to access the 'what's new in the release on Swift'? 19:09:03 caitlinbestler: I think this mostly comes from distros being able to plan for the combined release. I think it may help the openstack CI team have consistent version numbers, though 19:09:30 tongli: the CHANGELOG os the authoritative source, but I also normally send out an email about it too 19:09:30 notmyname: -rcX is the X an integer, just bunp up X? or is it -rcGrisly .. I am wondering because we have milestones such as G1, G2 etc 19:09:37 notmyname: in other words, avoids everyone from going crazy because we add a .1 to the release? :) 19:09:40 malini: integer 19:09:45 creiht: ya :-) 19:09:51 lol 19:10:04 it doesn't really change anything we've been doing, but I'm told it makes other people's lives easier :-) 19:10:10 seems like a reasonable compromise though 19:10:18 just funny 19:10:20 ya 19:11:01 #info swift candidates for the openstack combined release will have an -rcX tag until the combined release actually happens to avoid a version number dance in the case of backports 19:11:11 important changes between milestone-proposed release and combined release are a PITA any way you slice it, so seems reasonable (not any worse, etc.) 19:11:25 yup 19:11:42 #topic swift next API wiki 19:12:03 creiht and chmouel talked about and put together http://wiki.openstack.org/SwiftNextAPI. awesome. let's all add to it 19:12:08 #link http://wiki.openstack.org/SwiftNextAPI 19:13:08 cool hadn't seen that 19:13:17 some of the minor changes are things we should consider addressing sooner than later, but for the big stuff, I'd like you to think about "what does swift 2.0 mean?" 19:13:58 creiht: thanks for putting it up 19:14:45 I don't have much to add there. just wanted to say I liked it and make sure people knew about it 19:14:59 #topic pycon sprint 19:15:33 For me from an API standpoint, it means new user functionality. Including server side encryption, it could include stuff like cold-storage, reduce number of copies 19:15:35 swiftstack and redhat are working together to host a sprint at pycon this year. our goal is not to work on swift directly, but to encourage client apps written against the swift api 19:15:47 notmyname: I'm planning on going through the apis before the conference 19:15:58 If you would like to be involved, please let me know 19:16:20 creiht: cool 19:16:52 i'd like to be involved, might add some code to server side encryption 19:17:08 we have 2 ideas for the sprint, and I'd like your feedback: 1) add swift support to tools like boto and s3cmd 2) hackathon to build swift clients 19:17:29 thoughts? 19:17:42 what would be interesting to you to hack on? 19:17:45 notmyname: That will likely depends on who shows up for the sprints right? 19:18:11 creiht: it does affect how we promote it though 19:18:15 will probably not be able to make it 19:18:28 true 19:18:38 I'm still looking into weather I can stay for the sprints or not 19:19:06 although the sprints are 3 days after the conference, I think we'll just be doing stuff on monday the 18th 19:19:14 a single day 19:19:18 I am new to all things Swift and would like to participate .. so I have some ramping to do till March before hackthon. Just went through John's swift workshop from the last summit. 19:19:48 k 19:20:12 +1 for swift clients 19:20:31 malini: that's what I'm leaning towards too 19:20:53 I like the adding-swift-support to existing tools idea... since this is PyCon, we'd be talking tools implemented in python or python client libraries, right? 19:20:57 notmyname: that's cool with me, perhaps with a "or we will help you with whatever you would like to hack on" 19:21:00 yeah I'd rather see a new tool that is built for swift than adding swift support to something I don't know the guts of and has an impedence mismatch 19:21:18 heh 19:21:19 creiht: of course 19:21:31 does swift clients includes addons to the current swift.py? 19:21:31 swifterdarrell: clayg: can I agree with both of you? ;-) 19:22:04 notmyname: another intersting one would be improving swift3 19:22:20 * clayg is not that interested... 19:22:27 ;) 19:22:28 cschwede: perhaps, but also like "build a better cyberduck" or, in a hackathon sense, "build a small site that stores data in swift" 19:22:39 creiht: indeed 19:22:58 lol 19:23:21 notmyname: I think a general cross-platform gui written in python would be very nice 19:23:28 and well received 19:23:48 ya 19:23:49 cross-platform? you mean the web right? 19:23:57 creiht: web or desktop? 19:24:01 lol 19:24:07 srly who does desktop apps anymore 19:24:07 of course there is also improving swiftly, or the user space wrappers (like fuse, etc.) 19:24:15 cschwede: desktop 19:24:30 probably swift-dropbox for mobile device is the best thing to do. 19:24:36 if you are looking at web based, actually it wouldn't be a bad idea to improve horizon support 19:24:43 cschwede: I hope so :) having python-swiftclient be a well-regarded, easy to use python client for Swift is a good goal; I'd like python devs who want to store and retrieve things to look at python-swiftclient and say, "Oh yeah, this is a breeze and does what I want" (not saying it's not already--I haven't looked at it from that perspective) 19:24:44 or improve things like that https://github.com/reidrac/swift-nbd-server 19:25:07 I think there are plenty of opportunities 19:25:13 just depends on the interest of those that show up 19:25:14 ya, tons of great ideas 19:25:32 we probably should start a wiki page with the list of ideas 19:25:39 clayg: who does desktop apps? Um, Cyberduck? (which keeps coming up from real people as a client they're pointing at Swift) 19:25:45 I've noticed a lot of people have dropbox on their phone these days, including few 9 year-olders and I was bit surprised. 19:25:53 I acctally had a question about the api thing (tahnks for bringing that up notmyname) - it kinda moved by quickly - should I wait? 19:26:10 tongli: it is more difficult to do mobile dev with python 19:26:22 clayg: let's move on. next topic is future plans for swift, so it fits there 19:26:34 thanks for the great ideas for the sprint 19:26:34 developed a mobile app based on swift API will be useful and get product recongized by large number of people quickly. 19:26:54 tongli: I totaly agree that mobile apps would also be useful :) 19:26:54 true! 19:27:05 tongli: provide a moble framework that app devs can use to store data in swift (ie what dropbox is doing) 19:27:07 i like tongli mobile drop-box idea 19:27:21 that meets GUI, swift client .. 19:27:37 malini: but this is a python conference, and the sprints are usually on python apps 19:27:43 which don't match well with mobile 19:27:50 yeah not sure about the java part 19:27:53 since we were talking about client. right true for that conf. 19:27:56 creiht: it did with the nokia OS ;-) 19:28:14 but client app can be built regardless that that meeting/conf 19:28:21 oh great, so the 1 person with a nokia os phone and a swift cluster to talk to could use it :) 19:28:26 tongli: certainly 19:28:36 it will spread the Swift to bigger crowd. 19:28:37 creiht: symbian 19:28:57 creiht: http://en.wikipedia.org/wiki/Python_for_S60 19:29:15 ok, moving on... :-) 19:29:20 lol 19:29:23 #topic swift in 2013 19:29:26 we gonna wikify the ideas somewhere? 19:29:38 swifterdarrell: ya, I'll scape the transcript for it 19:29:42 we should start a wiki to keep track of all the stuff we want to wiki 19:29:43 cool 19:29:46 clayg: what's your API question? 19:30:28 well so... we have some "optional" middleware right now "enahcnes" the api 19:31:19 do those ever becomes part of the core api? Like when you talk about what a POST means to the swift api - you have to talk about multipart forms and form-post (e.g.) 19:33:00 *crickets* 19:33:01 :) 19:33:02 I just ask because we talk about client tools - they want to know what they can count on from a swift endpoint, and a major version bump might be a good time to "officialize" some of the api enhancements that already have momentum - or double check to see if they have any warts? 19:33:06 clayg: so since there is no doc defining the swift api as a spec, the "swift API" has pretty much meant "what can an app reasonably expect a deployer to do" 19:33:40 ya, a mojor version bump (or any version bump) would require some more formal definition 19:33:46 *probably 19:34:50 discoverability of what's supported in a particular deployment's implementation of the API could be handy; don't want to overcomplicate things, but being able to tell something about what middleware is around or how proxy-server's configured, etc could be useful 19:34:56 but I don't think that code organization affects what the API is (at least too much). IOW, it doesn't matter from an API perspective if it's in middleware or in the process 19:35:07 I know the idea of exposing the configured swift limits was floated at one point 19:35:31 ^ good example! 19:35:32 clayg: Error: "good" is not a valid command. 19:35:43 i hate you SO much uvirtbot 19:36:09 notmyname: can I say that you will focus on the multi-region for swift 2013? seems there are lots of dependency on this in the blueprints page 19:36:17 clayg: example of discovering configured constraints as part of the API? 19:36:28 clayg: Error: "i hate you SO much uvirtbot" is not nice. 19:36:31 I like the middleware-queryability idea 19:36:43 yuanz: ya, I'll get to that in just a bit 19:36:45 https://developers.google.com/discovery/ 19:37:19 swifterdarrell: sorry 19:37:34 torgomatic: I guess you can always "query" it by trying the API a middleware might be implementing and deduce its absence from the response 19:37:44 creiht: interesting. 19:37:47 torgomatic: doh, misread that as "I don't like" 19:37:52 notmyname: yeah something like discovery would "need to be there" so if it's done in middleware it couldn't really be "optional" in your pipeline? 19:38:05 swifterdarrell: essentially the web client model for feature degredation 19:38:14 alpha_ori: yeah I haven't had a chance to dig real deep into it yet, but seems like they have done a lot in that domain 19:38:33 clayg: ya, but things like cache and check_errors aren't really optional either 19:38:48 notmyname: That's what we do now with our web client to some degree. 19:39:15 notmyname: but that puts some burden on the client. 19:39:45 ya, it's not great. it's where we are today, and I do like the idea of discoverability 19:40:16 the easiest (and could still be v1 compatible) would be to add some sort of capabilities resource 19:40:25 Aren't we talking about two different things here? 1) a presumably wsgi method to find out exactly what middleware modules are enabled, and 2) what the relevant configuration options are for the swift relevant modules? 19:40:29 a client could get that to know what apis are available 19:40:38 would python decorators saying deprecated be adequate (java has deprecated tags) 19:40:45 caitlinbestler: it could be depending on how it is implemented 19:40:51 discoverability is great for *optional* elements, my originial question was really about getting things that clients want to count on in the api spec explicitly 19:40:57 Swift API 2.0 (or whatever) could define an API middlewares would implement, with, perhaps, proxy-server coughing up the proper "not present" response (for when an optional middleware isn't in the pipeline) 19:41:11 creiht: ya, I've got a (really, really dumb 30-minutes-spent-on-it) middleware in my github account that does that 19:41:39 clayg: that would mean that we'd have to be explicit about the api 19:41:44 I also don't like the idea of "discoveribility" being "what middleware is in the pipeline" - I should be able to conform to the api (including a well speced ehancement) even if I'm running a *different* middleware that the one that comes stock? 19:41:45 which would be a change 19:42:29 clayg: like the deb packaging "provides" directive (eg you get "mail" with both postfix and sendmail) 19:42:33 clayg: interesting, but that would require defining standards for features themselves. 19:42:43 clayg: not that we couldn't do that. 19:42:46 could group things into conceptual "features"; then do something like OPTIONS /features/temporary-urls, and if it 404s, then tempurl's functionality isn't there 19:43:05 clayg: What does it mean to provide S3 compatibility, for instance? 19:43:20 then you can reimplement tempurl's API to your heart's content, and just make sure your new fancy middleware says 200 to OPTIONS /features/temporary-urls 19:43:47 clayg: my idea was that middleware would say kind of what it can do; so like you said, the Swift API would define an optional form-post subset; any middleware purporting to support form-post functionality would implement that API including answing some kind of discovery req about it 19:44:17 swifterdarrell: ok, that sounds reasonable 19:44:19 clayg: if an alternate implementation of Swift's optional subset API for form-post came along, it could conform to the discovery API response and be a drop-in replacement from a client perspective 19:44:26 clayg: (the point of APIs, right?) 19:44:51 swifterdarrell: Do we yet have any examples of popular middleware that duplicate each other's efforts? 19:44:55 alpha_ori: s3 is hard, but if someone wrote a "better than swift3" middleware for swift - clients checking for "does this endpoint support s3 api" shouldn't have to know if I've installed swift3 or the other? 19:44:58 alpha_ori: auth 19:45:24 notmyname: ok...anything else? 19:45:39 ok, we only have a few more minutes. let's put dsicoverability off till later (would make a great summit topic) 19:45:42 ya 19:45:57 general 2013 goals. 19:46:05 swifterdarrell: I agree with 100%, i was worried for some reason that we'd "return a list of middleware - discoverable - done" 19:46:21 what use case are we looking to solve or enhance this year? 19:46:43 clayg: hehe 19:46:53 clayg: you always asume the worst :) 19:46:55 global clusters is something that we (swiftstack) are working on now 19:47:17 notmyname: any update as to where that is? 19:47:20 As mentioned, global clusters has been a community desire since at least the Boston conference (and probably before) 19:47:23 how things are coming? 19:47:42 creiht: well the first two patches are languishing in Gerrit pretty well 19:47:45 well, ring enhancements are sitting in Gerrit (https://review.openstack.org/18263) 19:47:58 there is a lot languishing in gerrit :) 19:48:03 creiht: isn't one of those waiting for a follow up from you? ;-) 19:48:42 notmyname: like I said, my list keeps getting longer :) 19:49:14 ya, so global clusters has some patches in gerrit now, and the other functionality will be coming, I hope, over the next month 19:49:35 swifterdarrell: I was asking that rather than submit a couple of patches and wait, could you at least throw some stuff in WIP so people could see how things are taking shape overall? 19:50:20 creiht: our list is long too, we'll submit them when we can :-) 19:50:28 creiht: because that'd speed up the reviews on the ready patches or because that'd be a good unrelated thing to have? 19:51:01 ok, so then future patches aren't hinging on those approvals... good :) 19:51:05 creiht: I think we'd like to submit a couple of patches and not have to wait (forever) 19:51:10 swifterdarrell: it would just be nice to see where things are going 19:51:13 ;) 19:51:31 beyond global clusters, are there any other big things like that that your associated companies are working on? tiering? encryption ( caitlinbestler?) etc? 19:51:45 swifterdarrell: I think we can all agree that we all have a lot on our plates, and we are getting to things as we can 19:51:55 creiht: yup! 19:51:59 generally, what are you working on or planning that will go into the core of Swift? 19:52:47 Maybe not a big thing - but I'm working on a quota middleware 19:53:17 cschwede: have you seen the quota stuff that is currently in review? 19:53:19 cschwede: ah, cool. have you seen mike barton's (redbo) patches 19:53:33 We'll continue to improve Swift monitoring as we get to it (see: plates and the count of things on them), but nothing earth-shattering planned there 19:53:44 creiht: yes, my work is currently on an account level 19:54:09 we'll probaby work on some migration tools for swift (cluster to cluster) probably not something for Core 19:54:16 notmyname: no, will look into that. I talked to nijaba and chmouel (thanks for your comments!) 19:54:44 chmouel: cool 19:55:12 wasn't rax looking at enhancements to object replication to leverage xfs internal data structures or do better prefix-hash-tree-rsync-faster-ninja-something? or is object repliation still cycling "fast enough"? 19:55:24 notmyname: current working implementation supports tempauth, keystone and swauth: https://github.com/cschwede/swquota README is a little bit out of date - I'll update it tomorrow (added a tool for simpler quota set recently) 19:55:40 clayg: replication enhancements are still being looked into, but I think they have moved onto another idea 19:55:43 Reconnecting -- on encryption - Nexenta is working on encryption but specifically on solutions that would *not* be part of swift core. 19:55:58 swifterdarrell: by monitoring do you mean the statsd+graphit work, or a new tool? 19:56:06 caitlinbestler_: ok, good to know (I liek that approach, thanks for your ML comments) 19:56:15 creiht: awesome! 19:56:16 cschwede: nice.. i am planning to test that sometime next week and we'll give you some feedback it could be nice to merge this with the redbo's container quota 19:56:44 and we're pretty much out of time.. anything else for the last 3 minutes? 19:56:57 yuanz: I generally mean changes to core swift for emission of monitoring data (eg. statsd, but not graphite or any other consumer of StatsD output, which is outside the scope of core swift code) 19:57:13 next meeting is Feb 6 at 1900UTC 19:57:18 ok 19:57:47 I'll put together a summary based on the transcript and send it out 19:57:49 notmyname: redbo has some work on bit torrent support 19:57:52 yuanz: an example of "ongoing improvement" to that would be https://review.openstack.org/#/c/20089/ 19:57:57 creiht: nice! 19:58:01 https://github.com/rackspace/cloudfiles-sworrent 19:58:08 if anyone would like to provide some feedback 19:58:17 creiht: 404 19:58:19 oh dangit 19:58:20 heh 19:58:25 I thought he made that public :) 19:58:42 well once he is comfortable with others seeing it, I'll let you know :) 19:58:49 cool 19:58:51 chmouel: that would be nice - thanks! 19:59:26 notmyname: I'm trying to play with better ways with handling metadata 19:59:33 but that's very low priority 19:59:36 at the moment 19:59:36 k 19:59:45 and we're out of time 19:59:56 thanks for attending. great feedback 19:59:59 #endmeeting