19:01:17 <notmyname> #startmeeting swift
19:01:17 <openstack> Meeting started Wed Feb 12 19:01:17 2014 UTC and is due to finish in 60 minutes.  The chair is notmyname. Information about MeetBot at http://wiki.debian.org/MeetBot.
19:01:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
19:01:20 <openstack> The meeting name has been set to 'swift'
19:01:34 <notmyname> welcome to the first weekly swift team meeting. who's here?
19:01:44 <portante> o/
19:01:47 <torgomatic> <-
19:01:48 <peluse> yo
19:01:50 <cschwede_> o/
19:02:17 <gvernik> hello
19:02:35 <tdasilva> hello
19:02:39 <notmyname> I think we've got a pretty full schedule for today (potentially)
19:02:40 <notmyname> #link https://wiki.openstack.org/wiki/Meetings/Swift
19:02:42 <tgohad> Hi John
19:02:47 <creiht> \o/
19:03:15 <notmyname> creiht is excited!
19:03:19 <creiht> heh
19:03:22 <peluse> touchdown
19:03:41 <notmyname> mostly today I want to talk about storage policy status and python-swiftclient status
19:03:58 <notmyname> but first
19:04:07 <notmyname> #topic summit CFP due this week
19:04:16 <notmyname> #link http://www.openstack.org/summit/openstack-summit-atlanta-2014/call-for-speakers/
19:04:40 <notmyname> if you are wanting to submit a talk for the conference section of the summit in atlanta, friday is your deadline
19:04:44 <notmyname> at the link above
19:05:02 <notmyname> all you need by friday is an abstract, so it's not too hard to do
19:05:09 <creiht> "Openstack, the good parts"
19:05:17 <creiht> >:)
19:05:41 <notmyname> and if you want to submit something about swift (and who doesn't?), then I'd be happy to help you do that. but I'd like to see what you have by the end of today
19:06:32 <notmyname> I know there will be talk submissions for quite a few things. but the mroe the merrier :-)
19:06:32 <portante> fwiw, been talking to notmyname about a workshop on object server backends
19:06:45 <notmyname> that's one of them :-)
19:06:55 <portante> would like others to participate and help with it
19:06:58 <notmyname> portante: for which I'll be bugging you later today :-)
19:07:01 <peluse> portante:  I can help, sure
19:07:02 <portante> k
19:07:14 <portante> thx, peluse
19:07:41 <peluse> also FYI notmyname and I are putting a few out there including, of course, a talk on policies and EC :)
19:07:48 <notmyname> yay
19:08:33 <notmyname> we're also building a test cluster for CI integration (eg let's do automatic probe tests on a real cluster!) and submitting a talk on that
19:09:04 <creiht> notmyname: I don't think I will be submitting a talk this time
19:09:21 <notmyname> creiht: I liked your idea above ;-)
19:09:26 <creiht> heh
19:09:39 <notmyname> so to sum up, friday is your deadline. if you are working with someone else, try to have it done today so you can sleep on it, polish it tomorrow, and be in before the deadline
19:10:07 <notmyname> for the tech tracks, the CFP will come later
19:10:23 <notmyname> any questions on the summit?
19:10:44 <notmyname> ok, moving on
19:10:52 <notmyname> #topic storage policies status
19:10:58 <notmyname> this could be a big one
19:11:04 <peluse> a big topic?
19:11:06 <notmyname> before getting updates from peluse torgomatic and portante
19:11:10 <notmyname> ya, maybe
19:11:18 <notmyname> I've updated https://wiki.openstack.org/wiki/Swift/PriorityReviews
19:11:20 <notmyname> #link https://wiki.openstack.org/wiki/Swift/PriorityReviews
19:11:54 <notmyname> I've added a few dates to the storage policies. nothing new to the people who have been working on it, but these are pretty much what will keep us on track for getting it in icehouse
19:12:09 <peluse> I like the new queries up there.... cool
19:12:21 <notmyname> thanks, torgomatic for the new review links :-)
19:12:41 <notmyname> importantly, I'd like to see all the final patches proposed into gerrit by friday (yes, that means we're just a little behind so far)
19:12:57 <notmyname> the trello board tracking storage policies is at https://trello.com/b/LlvIFIQs/swift-erasure-codes-and-storage-policies
19:12:58 <notmyname> #link https://trello.com/b/LlvIFIQs/swift-erasure-codes-and-storage-policies
19:13:18 <notmyname> and you can see that peluse and torgomatic have some patches up for review now
19:13:24 <notmyname> #link https://review.openstack.org/#/q/branch:feature/ec+AND+-status:workinprogress+AND+status:open,n,z
19:13:30 <notmyname> (lots of links!!!)
19:14:10 <notmyname> so the final pieces of functionality for storage policies on the feature/ec branch are (1) rolled up accounting and (2) policy reconciler
19:14:20 <peluse> yup
19:14:23 <notmyname> peluse: torgomatic: is that correct? can you give an update on the status?
19:14:43 <peluse> acct rollup:  good progress (thanks to copying off of torgomatic's paper)
19:14:55 <peluse> I expect *maybe* Fri but could be next week....
19:15:00 <notmyname> ok
19:15:09 <peluse> will try
19:15:20 <notmyname> torgomatic: reconciler?
19:15:21 <torgomatic> the patches in Gerrit now are the foundation for the reconciler; you can't move misplaced objects unless you know they're misplaced, and that's what you get with my patch series
19:16:03 <torgomatic> reconciler itself is still highly theoretical :)
19:16:11 <peluse> nice :)
19:16:13 <notmyname> small matter of typing left to do?
19:16:16 <torgomatic> a bit
19:16:39 <peluse> torgomatic:  what do we do with James's 'Preserving older storage policies'?
19:17:00 <torgomatic> peluse: I don't know how usable that is now that the policy index isn't in the metadata... I'll take a look
19:17:13 <peluse> K
19:18:09 <notmyname> any questions about storage policies or the plan there?
19:19:09 <notmyname> ok then :-)
19:19:24 <notmyname> then on to python-swiftclient
19:19:31 <notmyname> #topic python-swiftclient status
19:19:37 <notmyname> chmouel: around?
19:19:48 <notmyname> big changes have been happening here
19:19:55 <chmouel> notmyname: yeah just arrived :)
19:20:05 <notmyname> #link https://review.openstack.org/#/c/69187/
19:20:15 <notmyname> that's the patch to port swiftclient to requests
19:20:18 <chmouel> I think requests is good to go as far goes my testing and revieweing
19:20:25 <chmouel> the port of swiftclient to requests indeed
19:20:34 <notmyname> and we just landed the change to stop swift itslef from depending on swiftclient
19:20:42 <chmouel> that's correct
19:20:50 <chmouel> i think we should start merge this and do a release straight away
19:20:50 <notmyname> which means that py3 support in python-swiftclient can start happening
19:20:52 <chmouel> to 2.0
19:21:00 <notmyname> requests or py3?
19:21:06 <chmouel> notmyname: requests
19:21:12 <chmouel> and put in announcement that we will start reviewing the py3 support
19:21:15 <notmyname> right. that's the plan, IMO
19:21:19 <chmouel> +1
19:21:31 <notmyname> I think it makes sense to start py3 patches after a 2.0 release
19:21:50 <chmouel> we will have to do some reconciling since a lot of those patches are conflicting with each others
19:21:54 <notmyname> so what I want to do is get requests port landed and then tag HEAD~1 as the last 1.X release
19:22:02 <notmyname> to wrap us all that
19:22:10 <notmyname> then tag the reqeusts port as 2.0
19:22:17 <notmyname> and then move on the py3 changes
19:22:31 <chmouel> perfect plan IMO
19:22:38 <notmyname> that way we don't get the py3 dependencies into a 1.x series without actually having full py3 support
19:22:56 <notmyname> any questions or alternate suggestions or concerns?
19:23:31 <notmyname> you guys are making this easy :-)
19:23:47 <portante> thanks
19:23:48 <torgomatic> as long as we don't start talking about py3 for Swift before eventlet or gevent sorts itself out, I'm happy
19:23:55 <torgomatic> swiftclient is A-OK
19:23:56 <notmyname> +1
19:24:10 <notmyname> #topic s3 DiskFile
19:24:20 <notmyname> marcusvrn: around? this was a topic that has your name on it
19:24:25 <erlon> ok, so we started to work with in a code that implements a s3 backend to swift.
19:24:29 <marcusvrn> hi
19:24:35 <portante> cool
19:24:39 <notmyname> interesting
19:24:43 <erlon> working but didn't used the Diskfile API, so we are porting the code to use the API
19:25:04 <notmyname> seems like an alternative way to do the container migration patch
19:25:18 <notmyname> https://review.openstack.org/#/c/64430/
19:25:20 <erlon> hopefully we can make it ready to icehouse
19:25:44 <torgomatic> I dunno; being able to import data from external sources seems very different from using another object storage system as a backend
19:25:45 <portante> happy to help in anyway I can
19:25:47 <chmouel> i think it has other benefits than just migration right?
19:26:28 <notmyname> in general I like the idea of migration. I don't really like the idea of "Swift as an abstraction for other object storage systems"
19:26:33 <erlon> portante: yes, if you can give us a hand in this next days  we will reaally apreciate :)
19:26:35 <notmyname> (to torgomatic's point)
19:26:37 * torgomatic isn't a big fan of Swift turning into an API translation layer
19:26:50 <gvernik> data migration is about to import data into swift from other sources...
19:26:56 <erlon> there still are some obscure areas that we can't figure aout
19:26:56 <creiht> "Swift as an abstraction for * storage systems"
19:26:58 <creiht> :)
19:27:26 <creiht> well we kind of opened that can of worms with diskfile
19:27:45 <notmyname> creiht: well the point of swift is to abstract volumes as an object storage system. not to abstract any arbitrary storage engine
19:27:47 <chmouel> agreed
19:28:00 <notmyname> creiht: so I disagree about DiskFile opening that can of worms
19:28:05 <creiht> I'm not against people doing it, but I don't think it should be in core swift
19:28:35 <portante> I am not sure I follow all the concerns
19:28:38 * torgomatic can't wait for the bug reports saying that users can't tag S3 buckets when using Swift as an API translation layer
19:29:24 <creiht> notmyname: I'm just saying by making the abstraction, you invite these type of ideas
19:29:27 <portante> can somebody enumerate what might cause a problem?
19:30:00 <creiht> perhaps another question might be, will the gluster diskfile plugin be part of openstack swift?
19:30:02 <notmyname> creiht: ya. that is true. and I agree that they generally should not be in swift's codebase
19:30:09 <notmyname> creiht: imo no
19:30:12 <creiht> right
19:30:14 <erlon> one thing I can see is the way that files appear on s3
19:30:20 <torgomatic> portante: as a quick first pass, Swift provides eventual consistency atop immediately-consistent backends (a/c/o). Sticking eventual consistency in the object server seems like it'll introduce all sorts of intermittent troubles
19:30:47 <erlon> just hashes, which makes would make the s3 storage unsuable without swift
19:30:47 <portante> torgomatic: well stated
19:30:53 <torgomatic> I don't have a specific bug in mind, but my spidey-sense is tingling like mad
19:31:05 <portante> did you make it into the next movie?
19:31:31 <torgomatic> :)
19:31:31 <creiht> there will always be an impedence mismatch
19:31:38 <creiht> same as with swift3
19:31:54 <creiht> either way, this seems simple for me, it just lives outside of swift
19:31:58 <creiht> I can see people wanting to use it
19:31:59 <notmyname> it's the same general idea to me as "let's write a DiskFile to put Swift on top of our proprietary SAN device that also includes some distributed filesystem". not something that should be in swift itself
19:32:07 <notmyname> creiht: yes, exactly
19:32:07 <torgomatic> creiht: yeah, and I'm no great fan of that one either ;)
19:32:33 <creiht> torgomatic: lol, well when you create a plugin system you can't control what plugs into it :)
19:32:38 <chmouel> i don't think i want to see swift becoming like neutron with drivers not maintained
19:32:51 <cschwede_> chmouel: +1
19:32:58 <creiht> chmouel: they have that even with drivers in the core tree :)
19:33:08 <torgomatic> creiht: true, but I can try to control what plugins make it into the Swift tree
19:33:13 <notmyname> chmouel: but isnt' the problem with drivers that are in the repo?
19:33:26 <creiht> torgomatic: absolutely, which is what I'm arguing for as well
19:33:28 <chmouel> notmyname: yeah that's the problem
19:33:47 <chmouel> notmyname: i'm good with having a plugin system in core swift and driver staying outside
19:33:51 <notmyname> chmouel: so I don't think we'll get there. ie we won't be accepting many "drivers" into the swift tree
19:34:01 <notmyname> chmouel: I think we're all in agreement
19:34:10 <creiht> cool
19:34:24 <denis_cavalcante> Well, maybe is a positive point: s3 backend could increase the adoption of the openstack/swift softly... and if it is inside, it can
19:34:37 <notmyname> FWIW, I'd like to see the migration containers functionality in core swift :-)
19:34:51 <creiht> tons of people run with swift3, and it is outside the codebase
19:34:55 <denis_cavalcante> and if it is inside, it can smooth the deployment
19:35:09 * portante 's head spins
19:35:10 <chmouel> notmyname: yeah i'd like to spend some time reviewing it
19:35:15 <notmyname> denis_cavalcante: yes, I agree (especially with your 2nd statement)
19:35:59 <creiht> if it is just a manner of adding a piece of middleware, I'm not sure how being inside or outside swift makes much of a difference
19:36:01 <notmyname> it's something that must be weighed
19:36:14 <notmyname> creiht: I think it's the "Defaults matter" idea.
19:36:14 <gholt> Being outside means you can work on it faster. ;)
19:36:20 <notmyname> true :-)
19:36:21 <creiht> hah
19:36:36 <notmyname> but being inside means more people will be able to use it
19:36:38 <creiht> notmyname: but if it were in swift, I would also think that it wouldn't be a default
19:36:45 <notmyname> I think that's all denis_cavalcante is saying
19:37:11 <gholt> I'm not sure I want a bunch of users that don't know how to install stuff. It's pretty easy to find outside stuff. We have a whole section for it in the docs.
19:37:11 <chmouel> creiht: yeah but somebody would need to commit to maintain it in the long term
19:37:44 <creiht> chmouel: I don't want to maintain it inside swift ;)
19:37:52 <torgomatic> chmouel: +1 on that; I don't want to be responsible for maintaining an S3 translation layer
19:38:11 <gholt> I won't be. :D
19:38:15 <creiht> haha
19:38:23 <notmyname> ok, I think we're not really arguing about anything. we all feel the same way from what I can see. I think we should move on
19:38:29 <antonioc> actually, hybrid cloud environments are a good options for a lot of companies that doesn't have enough infrastructure, so supporting an S3 backend would be great for it
19:39:06 <notmyname> nobody denies that people use S3 or that S3 affects Swift's adoption
19:39:22 <chmouel> antonioc: i agree, but that doesn't mean it need to be shipped with core swift?
19:39:37 <lifeless> antonioc: there are lots of OpenStack public cloud providers
19:39:46 <notmyname> but actually running swift on top of s3 is something that no core dev wants to support ins wift's source tree
19:40:18 <notmyname> although we'd all support that functionality being available in the ecosystem
19:40:26 <notmyname> (from what I can tell by the conversation)
19:40:29 <creiht> yes
19:40:35 <chmouel> +1
19:40:42 <steveisoft> where does it belong if not in core swift?
19:40:42 <torgomatic> yup
19:40:50 <tdasilva> that's the idea behind having the plugin system, right?
19:40:52 <creiht> steveisoft: github
19:40:54 <creiht> :)
19:41:18 <gholt> Heh, I didn't realize we were still talking about that. Thought we were talking about https://review.openstack.org/#/c/64430/ now, heh.
19:41:23 <chmouel> or stackforge
19:41:48 <notmyname> I want to move on to current outstanding patches
19:42:11 <notmyname> #topic review queue
19:42:13 <antonioc> yeap, but S3 would be just one more interface or option... amazon is one of the greatest public cloud providers,  it wouldn't be bad to have some support to that
19:42:24 <erlon> notmyname: what's is the impact of supportin s3 as an API, like swift API? it seems to me an option too
19:42:40 * gholt just got caught up by creiht
19:42:43 <creiht> antonioc: we aren't saying that it shouldn't be an option, just not part of core
19:42:52 <chmouel> erlon, antonioc: perhaps let's talk about that in #openstack-swift?
19:42:59 <creiht> just like the s3 api compat layer isn't part of swift core
19:43:29 <chmouel> so the review queu
19:43:32 <torgomatic> so, what about the review queue?
19:43:36 <torgomatic> hehe
19:43:36 <notmyname> torgomatic's links he provided last week have been a good help
19:43:49 <notmyname> pathes with one core review:
19:43:51 <notmyname> #link https://review.openstack.org/#/q/-owner:notmyname+AND+-reviewer:notmyname+AND+status:open+AND+(project:openstack/swift+OR+project:openstack/python-swiftclient+OR+project:openstack/swift-bench+OR+project:openstack/object-api)+AND+CodeReview%252B2+AND+Verified%252B1+AND+-Verified-1+AND+-CodeReview-2+AND+-Approved%252B1,n,z
19:44:02 <notmyname> patches with no core reviews:
19:44:04 <notmyname> #link https://review.openstack.org/#/q/-reviewer:torgomatic+AND+-reviewer:notmyname+AND+-reviewer:gholt+AND+-reviewer:peter-a-portante+AND+-reviewer:darrellb+AND+-reviewer:chmouel+AND+-reviewer:clay-gerrard+AND+-reviewer:zaitcev+AND+-reviewer:david-goetz+AND+-reviewer:cthier+AND+-reviewer:redbo+AND+-reviewer:greglange+AND+status:open+AND+-status:workinprogress+AND+(project:openstack/swift+OR+project:openstack/python-swiftclient+OR+project:o
19:44:04 <notmyname> penstack/swift-bench),n,z
19:44:07 <notmyname> fun
19:44:08 <chmouel> https://review.openstack.org/#/c/71224/ is the easiest review someone can do
19:44:18 <torgomatic> of course , s/owner:notmyname/owner:$ME/g
19:44:32 <creiht> chmouel: did you add tests for that?
19:44:33 <creiht> ;)
19:44:40 <notmyname> torgomatic: both of those should be generic :-)
19:45:03 <chmouel> creiht: that's a good point we need to sort the bin/swift tests like we do in swift.cli stuff
19:45:20 <cschwede_> chmouel: working on it :)
19:45:30 <chmouel> cschwede_: nice :)
19:45:48 <notmyname> that's good to see
19:45:56 <notmyname> the bin scripts have had no tests for far too long
19:46:39 <notmyname> cschwede_: chmouel: there's actually something about that I want to talk to you about in -swift later
19:46:52 <cschwede_> notmyname: ok, will be there
19:47:12 <notmyname> so ingeneral, our review queue is somewhat shorter now, so that's nice
19:47:24 <chmouel> notmyname:  I think for reviews gvernik was asking about the container migration status (and you as well)
19:47:31 <notmyname> most of my time has been spend on summit abstracts and storage policy/swiftclient stuff
19:47:41 <chmouel> #link https://review.openstack.org/#/c/64430/
19:48:37 <notmyname> chmouel: gvernik: I don't have anything else to add that I didn't say earlier :-) I like the idea a lot
19:49:11 <chmouel> i'd spend some time to commit into it since i like the idea as well
19:49:20 <chmouel> *to commit review time
19:49:24 <notmyname> what about the account-to-account copy patch? who's looked at it?
19:49:32 <gholt> <
19:49:34 <portante> I looked at it briefly
19:49:34 <notmyname> #link https://review.openstack.org/#/c/72157/
19:49:48 <torgomatic> I took a pass at it once; I need to go back and look again
19:49:53 <chmouel> cschwede_: as well i think
19:49:54 <cschwede_> notmyname: me, works so far for me and looks good
19:49:56 <notmyname> looking at the comments, it seems to be on track
19:50:02 <gholt> That one's been going reasonably fast, with feedback loops
19:50:06 <notmyname> cleaning up the API issues
19:50:12 * portante is sorry that he has not had more time these past few weeks to work on swift code reviews
19:50:43 <creiht> interesting correlation that the queue is shorter, and portante hasn't been reviewing
19:50:46 <creiht> ;)
19:50:50 <chmouel> heh heh
19:50:51 <chmouel> burn
19:50:54 <portante> wow
19:50:57 <notmyname> ouch
19:50:59 <portante> thanks creiht
19:51:12 <portante> I can take a hint
19:51:13 <notmyname> correlation doesn't mean causation :-)
19:51:17 <creiht> of course we know that correlation != caustaion :)
19:51:36 <portante> ;)
19:51:56 <creiht> but doesn't mean we can't have fun with it :)
19:52:01 <notmyname> that was one of the most well-played burns I've seen in a while
19:52:07 <portante> that is what I am here for
19:52:13 <notmyname> (completely untrue, for the record)
19:52:18 <portante> or not
19:52:48 * portante wonders if that correlates with the sped up gate?
19:52:49 <notmyname> what other significant patches do we have outstanding?
19:53:01 <notmyname> gholt: creiht: anything on ssync updates?
19:53:39 <chmouel> creiht: are you back on swift btw?
19:53:46 <creiht> chmouel: yes
19:53:56 <creiht> notmyname: still testing as far as I know
19:54:02 <chmouel> creiht: coool :)
19:54:06 <notmyname> creiht never really leaves swift
19:54:12 <creiht> lol
19:55:41 <notmyname> I don't see any other reviews that warrant discussion here in the last five minutes that can't be handled in gerrit or -swift. anyone have anything else?
19:55:45 <notmyname> #topic open discussion
19:56:51 <notmyname> and so I guess that's it for this week. see you next week!
19:56:56 <notmyname> thanks for being here
19:57:01 <notmyname> #endmeeting