16:01:07 <smcginnis> #startmeeting Cinder
16:01:08 <openstack> Meeting started Wed Mar 16 16:01:07 2016 UTC and is due to finish in 60 minutes.  The chair is smcginnis. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:11 <openstack> The meeting name has been set to 'cinder'
16:01:16 <fernnest> here
16:01:19 <Swanson> hello
16:01:20 <smcginnis> Courtesy ping: dulek duncant eharney geguileo winston-d e0ne jungleboyj jgriffith thingee smcginnis hemna xyang tbarron scottda erlon rhedlind jbernard vincent_hou kmartin patrickeast sheel dongwenjuan JaniceLee cFouts Thelo vivekd adrianofr mtanino yuriy_n17 karlamrhein diablo_rojo
16:01:21 <dulek> hi! :)
16:01:22 <jseiler_> hi
16:01:23 <yuriy_n17> hi
16:01:24 <yhayashi> hi
16:01:25 <jungleboyj> o/
16:01:28 <tbarron> hi
16:01:28 <vivekdhayaal> hi
16:01:28 <geguileo> Hi
16:01:28 <sheel> hello all
16:01:31 <mc_nair> o/
16:01:32 <smcginnis> Hey everyone.
16:01:33 <vincent_hou> hi
16:01:33 <baumann> Hello!
16:01:34 <diablo_rojo> Hello :)
16:01:34 <patrickeast> Hey
16:01:36 <hemna> hi
16:01:38 <jgregor> Hola
16:01:40 <bswartz> ¬_¬
16:01:41 <cfouts> hi
16:01:47 <smcginnis> Lot's today. Let's get started.
16:01:47 * DuncanT wakes up
16:01:52 <xyang1> hi
16:01:54 <smcginnis> #topic Release Status
16:02:03 <smcginnis> Today's the day.
16:02:06 <e0ne> hi
16:02:11 <smcginnis> I would like to cut RC-1 today, if possible.
16:02:24 <smcginnis> We have a few final bug fixes and important updates to get in first.
16:02:30 <ntpttr> hi
16:02:33 <smcginnis> So hopefully gerrit doesn't have other things in mind.
16:02:41 <jungleboyj> Wow, it came so quickly!
16:02:59 <bswartz> jungleboyj: it's 1 week shorter this cycle
16:03:02 <e0ne> good news!
16:03:25 <sheel> first release after i joined cinder...
16:03:26 <kmartin> hi
16:03:31 <smcginnis> Thanks to DuncanT - he started this etherpad: https://etherpad.openstack.org/p/cinder-mataka-release-final-push
16:03:39 <smcginnis> With help from sheel!
16:03:47 <smcginnis> #link https://etherpad.openstack.org/p/cinder-mataka-release-final-push Final push
16:04:04 <JayXu> hi
16:04:05 <sheel> :)
16:04:16 <xyang> mataka?:)
16:04:17 <smcginnis> The colors are killing me, but please add anything important there and I'll make sure we are covered.
16:04:23 <smcginnis> xyang: LOL
16:04:24 <jungleboyj> smcginnis: So we should focus review there and -2 anything else?
16:04:34 <diablo_rojo> smcginnis: +1
16:04:37 <e0ne> xyang: :)
16:04:40 <smcginnis> jungleboyj: Yes, please.
16:04:42 <hemna> my eyes!
16:04:44 <diablo_rojo> My eyes are burning
16:04:47 * jungleboyj cries as I open the page.
16:04:55 <jungleboyj> Oh thank you!
16:04:55 <smcginnis> OK, I had to clear the colors.
16:04:56 <Swanson> mine eyes!  The goggles, they do nothing!
16:05:02 <geguileo> Thank you for the color change
16:05:06 <smcginnis> :)
16:05:06 <dulek> Is master open for Newton just after we cut RC-1?
16:05:13 <smcginnis> dulek: Yes, good question.
16:05:23 <smcginnis> Once we cut Mitaka RC a new branch will be created.
16:05:29 <sheel> its ok now
16:05:32 <smcginnis> That effectively makes master switch over to Newton.
16:05:43 <smcginnis> Our focus still needs to be on wrapping up Mitaka.
16:05:54 <dulek> smcginnis: Okay, so until stable/mitaka is there, the master isn't opened. :)
16:06:00 <smcginnis> But it is possible to loosen things up a little again and start letting things through for N.
16:06:08 <smcginnis> dulek: Right!
16:06:20 <DuncanT> Note that people should be focusing on testing the release where possible, rather than forging ahead with master
16:06:29 <smcginnis> DuncanT: +1000
16:06:31 <dulek> DuncanT: +1 :)
16:06:39 <e0ne> DuncanT: +2
16:06:40 <bswartz> traditionally there's a quiet period after cutting the branch, because anything that goes into master that doesn't get backported makes it harder to backport fixes to RC2
16:06:51 <smcginnis> bswartz: Good call!
16:06:53 <bswartz> and the testing thing too
16:07:07 <hemna> testing?
16:07:09 <hemna> what's that
16:07:24 <smcginnis> Yes. Cores please be aware of making backports too complicated until M is final.
16:07:25 <sheel> haha
16:07:25 <Swanson> hemna, hpe slogan?
16:07:30 <hemna> :)
16:07:36 <e0ne> hemna: it's something after you depoy Mitaka to production:)
16:07:37 <smcginnis> Having some quiet time is good.
16:07:44 <hemna> ah yes
16:08:11 <smcginnis> #topic Volumes in error take up quota
16:08:14 <smcginnis> DuncanT: All yours
16:08:25 <DuncanT> Oh, this one
16:08:36 <jungleboyj> smcginnis: So, anything on that review list that doesn't make it in before the RC cut will have to be backported to RC-1.  RIght?
16:08:58 <DuncanT> So it looks like (and I might be totally wrong, I've not do more than read the code) that volumes in error take up quota
16:09:15 <DuncanT> This does not seem like desired behaviour from a tenant PoV
16:09:17 <bswartz> shouldn't they take up quota? if they don't take quota but they take actual space you could formulate a DOS attack around creating huge numbers of error state volumes
16:09:17 <dulek> DuncanT: I think you're right on that.
16:09:20 <mc_nair> DuncanT: pretty sure they do
16:09:24 <smcginnis> jungleboyj: If it's critical enough, yes.
16:09:43 <jungleboyj> smcginnis: Thanks.
16:09:47 <DuncanT> bswartz: If you can cause volumes to go into error as a tenant, then something is broken
16:10:06 <bswartz> fair enough
16:10:22 <DuncanT> bswartz: and they shouldn't be taking up resources other than a db entry anyway
16:10:52 <dulek> DuncanT: Any retype errors or migration errors cannot cause that?
16:10:53 <bswartz> that's not true -- delete volume can fail
16:11:03 <smcginnis> DuncanT: Guess it depends how/where they error.
16:11:05 <dulek> bswartz: That would be error_deleting,
16:11:17 <geguileo> smcginnis: +1
16:11:54 <jungleboyj> Seems like they shouldn't take up quota if they error while created but should if they error while deleting.
16:12:00 <e0ne> DuncanT: we have to be careful, if volume is in error state and it takes some space on storage - it could be in issue
16:12:08 <eharney> can we actually differentiate between "failed and is using space" and "failed but is not using any space" ?
16:12:19 <smcginnis> eharney: I think we _could_.
16:12:21 <vincent_hou> dulek: currectly migration retype can cause migration_status error, not volume status in error
16:12:23 <smcginnis> But I don't think we do.
16:12:28 <e0ne> eharney: good question
16:12:35 <eharney> i'm not so sure, since there are a lot of ways a volume can end up in an error state
16:12:43 <DuncanT> eharney: Not the way the code is currently
16:13:06 <patrickeast> eharney: that would be nice, but probably requires modifying all the drivers
16:13:31 <smcginnis> I've always thought ideally we should have a failure rollback.
16:13:35 <dulek> For a tenant if he ends up out of volume quota it isn't really hard to delete all the error ones by a single bash one-liner…
16:13:37 <eharney> the point was, i think we have to assume they are using space because we probably can't reliably tell
16:13:49 <smcginnis> So if it fails after the backend has created the volume we actually delete it.
16:13:58 <smcginnis> eharney: I think you're right.
16:14:01 <DuncanT> dulek: They often go into error_deleting then, depending on the cause of the error
16:14:12 <patrickeast> eharney: +1
16:14:14 <dulek> DuncanT: Fair point.
16:14:19 <geguileo> eharney: +1
16:14:21 <smcginnis> To bswartz's point, if that happens someone could exhaust all resources.
16:15:09 <bswartz> the DOS attack doesn't even have to be malicious -- with the right kind of automation and retries, a well-meaning script could end up in a loop creating error volumes forever
16:15:13 <DuncanT> Ok, so any fix is not trivial. I might look at not using quota in the unable-to-schedule case, and see how much that complicates things
16:15:25 <smcginnis> DuncanT: OK, good.
16:15:29 <DuncanT> Since that doesn't use any disk, ever
16:15:42 <e0ne> agree
16:16:04 <smcginnis> A good thing to make better at any rate.
16:16:14 <smcginnis> DuncanT: Anything else on the topic?
16:16:21 <mc_nair> DuncanT: you'd need to make sure not to reduce quota on the volume delete then, right? that way it's not double counted
16:16:25 <geguileo> You have to be carefull on deleting the volume
16:16:35 <geguileo> Because if you haven't used quota, you cannot free it
16:16:39 <DuncanT> smcginnis: Nope, mostly wanted to get thoughts before I put any effort into it
16:16:49 <smcginnis> DuncanT: Great, thanks!
16:16:57 <smcginnis> #topic Gerrit IP change
16:16:58 <DuncanT> And yes, one of the complications I mentioned is the delete code :-)
16:17:03 <smcginnis> So this is just an announcement.
16:17:13 <smcginnis> The IP address of gerrit will be changing soon.
16:17:30 <smcginnis> #link http://lists.openstack.org/pipermail/openstack-dev/2016-March/088985.html Gerrit IP change
16:17:45 <smcginnis> If you're CI has specific firewall rules, please don't get caught off guard.
16:18:00 <e0ne> should we notify 3rd party CI maintainers?
16:18:15 <geguileo> e0ne: +1
16:18:23 <smcginnis> It was announced on the mailing list, so just highlighting it here.
16:18:40 <smcginnis> But if someone wants to volunteer to notify everyone directly, I certainly have no objections.
16:19:02 <smcginnis> #topic Summit session planning
16:19:11 <e0ne> smcginnis:  I didn't find it in openstack-infra  ML
16:19:25 <smcginnis> So I probably should have sent out a ML on session planning, in hindsight.
16:19:32 <smcginnis> But oh well.
16:19:38 <bswartz> e0ne: dev list tagged with [infra]
16:19:51 <smcginnis> e0ne: Probably should have been cross posted I guess.
16:19:59 <e0ne> bswartz: yes, but it's a differrent ML
16:20:04 <smcginnis> #link https://etherpad.openstack.org/p/newton-cinder-summit-ideas Session planning etherpad
16:20:23 <smcginnis> We have rooms allocated as of this morning.
16:20:40 <JayXu> Will the new IP address be available in April 11, 2016 20:00 UTC and the old IP will be removed at that time?
16:21:14 <smcginnis> We have 4 fishbowls. 5 working rooms, and a space all day Friday for a contributors meetup.
16:21:37 <smcginnis> JayXu: It will be a hard switchover from one to the other, so once the new one is live the old one goes away.
16:22:05 <smcginnis> We can beg for more space if absolutely needed, but on the other hand, we can be nice and give space back if not needed.
16:22:29 <dulek> We don't have a lot of propositions yet.
16:22:35 <smcginnis> So if everyone can update session topics on the etherpad, I can start allocating space and see where we are.
16:22:42 <hemna> odd, I don't see a Gerrit UI bitch session planned.......
16:22:58 <dulek> And all the geguileo's updates for developers can be done at Friday I guess?
16:22:59 <smcginnis> hemna: We can reserve some time Friday for that one. ;)
16:23:02 <bswartz> hemna: that's crossproject track
16:23:04 <e0ne> :)
16:23:14 <hemna> bswartz, ah yes, different etherpad
16:23:17 <geguileo> dulek: I think that would be the right moment
16:23:18 <smcginnis> dulek: That's kind of what I was thinking.
16:23:26 <dulek> hemna: Isn't that for contributors beer evening? ;)
16:23:30 <smcginnis> That way if one topic is shorter we can just move on to the next.
16:23:32 <geguileo> At least that was my idea when I proposed them
16:23:41 <dulek> :)
16:23:47 <smcginnis> Although some of them are important enough that we may want to allocate a fishbowl to it.
16:23:54 <smcginnis> We'll see how things end up.
16:24:26 <DuncanT> I think some of them definitely could do with more format than the Friday session
16:24:32 <jungleboyj> hemna: Still frustrated by that?  I had forgotten about it and moved on.
16:24:42 <hemna> jungleboyj, daily...
16:24:58 <smcginnis> Good new is we have another cross session with nova. Good to keep that rolling.
16:25:05 <hemna> smcginnis, +1
16:25:06 <jungleboyj> hemna: There there ...
16:25:09 <hemna> we have a lot to cover in that
16:25:16 <jungleboyj> smcginnis: ++
16:25:25 <sheel> smcginnis: will this all be through video conf as well?
16:25:30 <dulek> DuncanT: But the session format geguileo is proposing is purely developer-oriented.
16:25:53 <sheel> or only recorded sessions?
16:25:58 <smcginnis> sheel: Depends if our A/V department can bring the gear (hemna), but yes, I'm hoping to.
16:26:10 <DuncanT> dulek: Yes, but it is largely one-to-many, and the friday sessions make it difficult to do that sort of presentation
16:26:11 <hemna> :)
16:26:20 <hemna> I need to find a compact lightweight tripod
16:26:31 <hemna> that fits in my backpack
16:26:31 <smcginnis> sheel: We should at least be able to put a hangout up on the projector for folks that can't make it.
16:26:42 <diablo_rojo> hemna: If you don't wanna bring everything I can cover the tripod.
16:26:49 <smcginnis> hemna: Sick of holding the stick the whole time? :)
16:26:50 <sheel> smcginnis:  will be good to have
16:26:58 <hemna> yah the stick makes people sick
16:27:02 <dulek> DuncanT: We can be disciplined enough to do that I think. :)
16:27:08 <smcginnis> sheel: I'm assuming you're not able to attend in person?
16:27:19 <hemna> diablo_rojo, ok that'd be cool if I can't find one thanks
16:27:34 <sheel> smcginnis: yes, I tried to manage but could not make it ...
16:27:43 <sheel> :(
16:27:50 <smcginnis> sheel: Hopefully next time.
16:27:51 <jungleboyj> hemna: I also have one ... we will cover you if you need it.
16:28:09 <smcginnis> #topic Any suggestion about the driver replacement policy
16:28:16 <jungleboyj> sheel: Bummer.  Start planning for Barcelona.  :-)
16:28:16 <sheel> smcginnis: yep... :)
16:28:17 <smcginnis> JayXu: Hi, you're up.
16:28:20 <JayXu> Oh, it is my turn
16:28:22 <JayXu> In Tokyo summit, I had some conversation with some of Core team members, and at that time, we are thinking to re-architect VNX Cinder driver. Move some storage specific business logic out of Cinder driver and put them into pypi as the third party library. Even though, it still is hard to deliver the re-implementation of VNX Cinder driver phase by phase. So I would ask any possible to deliver...
16:28:23 <JayXu> ...a new implementation of a driver in one commit.
16:28:31 <sheel> jungleboyj: hehe, sure..
16:28:58 <smcginnis> JayXu: I think we need to treat it as a new driver submission then.
16:29:06 <smcginnis> And apply the same policies around that.
16:29:12 <e0ne> smcginnis: +1
16:29:24 <smcginnis> JayXu: So same deadlines, CI requirements, etc.
16:29:25 <JayXu> okay, good to know that.
16:29:26 <fernnest> JyaXu, we have similar issue w/ Hitachi and HPE driver so policy is good.
16:29:33 <diablo_rojo> smcginnis: +1
16:29:44 <DuncanT> JayXu: I'd suggest a parallel driver for a time, to work the issues out, give it a new name, then once it is sorted use the mapping file to map the old name over
16:29:47 <xyang> smcginnis: do we need to deprecate the old driver or can we keep the same driver name
16:29:47 <JayXu> we can submit the new driver at the beginning of Newton timeframe
16:30:07 <JayXu> yes, this is what we are expecting, thanks a lot.
16:30:18 <smcginnis> xyang: Good question.
16:30:24 <JayXu> we will deprecate the old driver.
16:30:38 <JayXu> after the new driver is merged.
16:30:50 <smcginnis> JayXu: OK, that works for me.
16:31:23 <smcginnis> JayXu: Anything else on this topic, or should I move on to the next agenda item?
16:32:02 <JayXu> go ahead. I am done. thanks!
16:32:09 <smcginnis> JayXu: OK, thanks.
16:32:13 <smcginnis> #topic Some operation passes even if volume service is disabled
16:32:15 <smcginnis> sheel: Hey
16:32:26 <sheel> smcginnis: we can skip this for now..
16:32:27 <smcginnis> Oh, now I see your note on there.
16:32:41 <smcginnis> sheel: Sounds good. Important, but we can defer for now.
16:32:46 <sheel> smcginnis: yes..
16:32:58 <smcginnis> #topic Dynamic reconfiguration
16:33:12 <smcginnis> diablo_rojo: Your turn
16:33:13 <diablo_rojo> #link https://review.openstack.org/#/c/286234/
16:33:22 <e0ne> +1 on using SIGHUP
16:33:23 <jungleboyj> Man, look at us being efficient!
16:33:57 <sheel> +1 diablo_rojo for this thought..
16:33:57 <geguileo> lol
16:34:05 <diablo_rojo> So after talking about the different approaches with DuncanT we thought the approach in the spec would be a good start.
16:34:11 <dulek> I've already commented on that, but - reloading by watching doesn't reload what we already have in the memory.
16:34:20 <diablo_rojo> But it sounds like SIGHUP might be preferred?
16:34:30 <eharney> the watch and automatic reload thing won't work
16:34:32 <diablo_rojo> I just wanted to get more opinions on it
16:34:34 <bswartz> +1 for SIGHUP
16:34:40 <dulek> Certainly we need to kill the service threads and make them restarted.
16:34:46 <dulek> To reload memory content.
16:34:49 <DuncanT> SIGHUP is already used
16:34:58 <geguileo> +1 for SIGHUP
16:35:01 <dulek> And SIGHUP + oslo.service will do that for us.
16:35:08 <dulek> DuncanT: Yes, do we have a conflict here?
16:35:26 <e0ne> DuncanT: yes, but we need to fix some issues with it. it doesn't work for all params
16:35:39 <dulek> DuncanT: Reload the config + reload the pinned versions of APIs.
16:35:41 <DuncanT> I don't know if there's any downside to doing both the version pin stuff and the config reload on a SIGHUP
16:35:56 <DuncanT> But it needs looking at for sure
16:36:00 <dulek> DuncanT: This doesn't sound like something people would really need to be separated.
16:36:10 <diablo_rojo> It sounds like it might get a little complicated, but not impossible to do
16:36:16 <DuncanT> dulek: Probably not, no
16:36:29 <DuncanT> dulek: I meant regarding bugs rather than anything else
16:36:30 <geguileo> dulek: +1
16:36:40 <dulek> DuncanT: Yeah, definitely mentioning it in the spec would rise some operators awareness.
16:36:49 <diablo_rojo> Okay I can include that
16:37:13 <geguileo> Should we stop everything if we just want to change the debug level?
16:37:21 <dulek> diablo_rojo: Hey, had you looked how other services reload configs (if they do?)
16:37:42 <e0ne> geguileo: we don't know what was changed
16:37:43 <geguileo> Because I think it shouldn't
16:37:51 <diablo_rojo> dulek: I have done a little research, but I definitely need to spend more time on it. Any project I should check out in particular?
16:37:54 <geguileo> e0ne: We can tell
16:38:07 <geguileo> e0ne: We have our current configuration and we can load the new one and compare
16:38:10 <geguileo> right?
16:38:13 <dulek> diablo_rojo: Anything using oslo_service will support SIGHUP by default.
16:38:16 <e0ne> geguileo: IMO, it's good to have one solution for every config option and dont' care on a specific one
16:38:31 <diablo_rojo> dulek: Okay yeah that sounds familiar.
16:38:35 <cfouts> eOne +1
16:38:43 <geguileo> e0ne: In my opinion debug and verbose are important enought to be a special case
16:38:46 <dulek> diablo_rojo, geguileo: I'm not sure, but I think logging level reload is implemented in oslo_service?
16:38:53 <dulek> At least there were ideas like that.
16:39:18 <jungleboyj> e0ne: ++
16:39:18 <geguileo> dulek: I thought that too, but I think there might be something we have to do in cinder...
16:39:26 <diablo_rojo> jungleboyj: Captain oslo? Do you know if its in oslo_service?
16:39:39 <geguileo> We should check it anyway
16:39:51 <jungleboyj> diablo_rojo: I believe dulek is right.
16:39:59 <diablo_rojo> jungleboyj: Okay cool.
16:40:08 <DuncanT> The testing for this is also going to be a pain.... things like checking the ssh pools in drivers and such get rebuilt properly
16:40:18 <e0ne> geguileo: you can change debug mode and, for example, some connection string (db, rabbitmq, keystone, glance, etc)
16:40:35 <geguileo> e0ne: Then you have to drain
16:41:03 <geguileo> e0ne: Anyway, I'll check if it currently works reloading of debug mode
16:41:06 <diablo_rojo> So it sounds like I have a lot of changes to make..
16:41:20 <dulek> diablo_rojo: https://blueprints.launchpad.net/glance/+spec/sighup-conf-reload
16:41:35 <dulek> diablo_rojo: Looks like at least Glance implemented that?
16:41:42 <diablo_rojo> geguileo: Could you let me know what you find?
16:41:58 <geguileo> diablo_rojo: I will ping you once I check it
16:42:02 <diablo_rojo> dulek: Yeah that was in my list of things to look at.
16:42:08 <diablo_rojo> geguileo: Thank you :)
16:42:23 <smcginnis> diablo_rojo: Good for now?
16:42:29 <jungleboyj> Good that we aren't the very first.
16:42:32 <jungleboyj> :-)
16:42:41 <jungleboyj> We can't always be the trail blazers.  :-)
16:42:43 <diablo_rojo> smcginnis: Yeah I think so. Lots of work to do :)
16:42:54 <smcginnis> diablo_rojo: For sure. Thanks!
16:43:01 <diablo_rojo> smcginnis: Thank you :)
16:43:02 <smcginnis> #topic Open Discussion
16:43:04 <jungleboyj> Go diablo, go diablo, go go!
16:43:24 <DuncanT> jungleboyj: The glance approach we can't really take though
16:43:26 <e0ne> I thought we've got diablo release few years ago ;)
16:43:32 <smcginnis> :)
16:43:42 <DuncanT> jungleboyj: Our design is fundamentally different
16:43:56 <smcginnis> Just to reiterate: I'll be working off of this before the cut: https://etherpad.openstack.org/p/cinder-mataka-release-final-push
16:43:59 <jungleboyj> DuncanT: :-(  Bummer.
16:44:12 <smcginnis> Anyone, feel free to ping me directly if there is something critical I need to watch for.
16:44:38 <sheel> :👍
16:45:05 <smcginnis> I'll end early if there isn't anything else.
16:45:07 <diablo_rojo> I honestly didn't think we would get through that whole agenda.
16:45:14 <DuncanT> Note that I didn't look at many driver bugs when putting that list together
16:45:16 <smcginnis> diablo_rojo: Me too, I'm surprised!
16:45:26 <Swanson> unavailable in a number field?
16:45:28 <jungleboyj> :-)
16:45:36 <Swanson> or unknown or infinite?
16:45:39 <DuncanT> If you know of any in your driver, be prepared to shout up or backport
16:45:51 <jungleboyj> Does everyone have their cheesecake patches in?
16:46:04 <Swanson> jungleboyj, god help me, yes.
16:46:06 <smcginnis> All the ones that had patches out at the midcycle are in.
16:46:11 <smcginnis> So I feel good about that.
16:46:19 <Swanson> petition to roll back to v2.0.
16:46:28 <jungleboyj> smcginnis: ++
16:46:30 <smcginnis> Swanson: O_o
16:46:49 * jungleboyj proposes returning to V1.
16:46:51 <xyang> smcginnis: are we going to disable 2.0?
16:47:04 <vincent_hou> why is 2.0? replication?
16:47:13 <xyang> vincent_hou: yes
16:47:14 * jungleboyj is waiting for jgriffith to reach through the computer to strangle me.
16:47:18 <DuncanT> vincent_hou: Yes, replication
16:47:19 <smcginnis> xyang: Most stuff was removed from the client, but looks like a couple things got left behind.
16:47:30 <vincent_hou> I thought we did remove 2.0
16:47:41 <smcginnis> Once the freeze is over on client libs I'll do a new release getting those out of there so there's no confusion.
16:48:03 <smcginnis> vincent_hou: Just a few cleanup items to take care of yet.
16:48:22 <smcginnis> OK, go forth and test. Thanks everyone!
16:48:22 <e0ne_> smcginnis: we've already got stable/mitaka branch for python-cinderclient
16:48:29 <smcginnis> #endmeeting