16:09:36 <thingee> #startmeeting Cinder
16:09:37 <openstack> Meeting started Wed Dec 11 16:09:36 2013 UTC and is due to finish in 60 minutes.  The chair is thingee. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:09:38 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:09:39 <jungleboyj> avishay: Everyone is bailing out here for Christmas already.  Not be just an IBM thing.  :-)
16:09:41 <openstack> The meeting name has been set to 'cinder'
16:09:46 <thingee> hi all
16:09:51 <bswartz> woop
16:09:53 <xyang__> hi
16:09:58 <joel-coffman> hey
16:10:04 <zhaoqin__> hello
16:10:05 <jungleboyj> Howdy!
16:10:08 <bswartz> better late than never
16:10:09 <thingee> agenda today folks https://wiki.openstack.org/wiki/CinderMeetings
16:10:15 <avishay> bswartz: not always :)
16:10:17 <winston-d> o/
16:10:30 <thingee> thingee: you have the floor
16:10:36 <avishay> haha
16:10:47 <thingee> #topic Ideas with extensions
16:10:57 <thingee> #link https://etherpad.openstack.org/p/cinder-extensions
16:11:04 <jungleboyj> Oh no, thingee is talking to himself.
16:11:40 <avishay> thingee: so an extension could add a new flow, or inject into existing flows, yes?
16:12:00 <thingee> inject into an existing flow
16:12:07 <avishay> yes
16:12:23 <avishay> or a new one, right?
16:12:54 <thingee> so that's debatable. I thnk things get more complex then.
16:12:59 <avishay> i see
16:13:02 <kmartin> so reading the etherpad, does multi-attach fall into core since its part of "attach it" basic function
16:13:08 <thingee> because you could end up with different results
16:13:30 <zhiyan> kmartin: hehe i asked same question on the summit, but seems it isn't
16:13:38 <caitlin56> If it is a new flow, how do users know when to invoke it and how? If you are modifying a flow, how do you allow two exensions to modify the same flow?
16:13:51 <avishay> where would retype go?  it's related to types, but it has it's own flow.  where does migration go?  replication can definitely inject into create_volume.
16:15:38 <thingee> caitlin56: exactly. there maybe other extensions injecting to that flow...and then you got a whole mess if an extension just does a new flow.
16:15:49 <thingee> sorry guys, ym irssi session is really laggy atm
16:16:14 <winston-1> i like the idea 'core feature should be avaiable (cannot be disabled/turned off) everywhere'.
16:16:24 <caitlin56> thingee: that's my concern, the phrase "injection" brings back ancient memories of DOS TSR routines.
16:16:34 <jungleboyj> +1 winston-1
16:16:44 <thingee> winston-d: +1
16:16:57 <thingee> avishay: so I see migration as a new flow that could be registered with the manager.
16:17:01 <bswartz> DOS TSR routines: -1000
16:17:32 <avishay> thingee: OK so that's what i meant by an extension adding a new flow
16:17:40 <rushiagr> hey hi all!
16:17:44 <avishay> rushiagr: hi
16:17:56 <thingee> avishay: Thanks. I just need a good example to get to that point :)
16:18:04 <avishay> thingee: :)
16:18:22 <thingee> so harlowja and I have been talking about ways to inject beginning and end
16:18:26 <thingee> that's pretty simple
16:18:38 <avishay> i like the idea in terms of code, the DB is where it gets tricky (and also conflicting extensions as caitlin56 mentioned)
16:19:27 * DuncanT forgot the meeting, sorry
16:19:30 <thingee> when you talk middle..I have this sort of silly idea that you take the the core tasks...whatever task is in the middle, you inject before that. I think this might lead to problems where the core tasks can change..and then extensions would break.
16:19:32 <avishay> need tempest tests that try all combinations?
16:19:34 <caitlin56> avishay: we can learn from past mistakes, the key is to prevent extension X from modifying the chain as previously modified by extension Y while assuming it was unmodifed.
16:20:21 <thingee> You have things like FC zone manager that'll basically need to change how attach/detach works.
16:20:52 <avishay> caitlin56: thingee: yep
16:21:57 <hemna> thingee, do you have a wiki on what it means to be an extension and how one codes it up?   I'm not clear on the distinction.
16:22:05 <thingee> anyways, this is kind of what I've been throwing around.
16:22:35 <rushiagr> I agree, we don't have good docs to explain what an extension is
16:22:43 <thingee> hemna: not yet. I wanted to start with just rexplaining my idea since there was a big disagreement at the summit
16:22:47 <caitlin56> henma: there isn't anything I could find. That might be a first step -- fully document what is there today.
16:22:59 <thingee> I wanted to start with everyone agreeing what is core
16:23:09 <thingee> if we can agree on what core is, the rest are extensions to cinder.
16:23:28 <hemna> well I thought the real disagreement was simply around disallowing extensions to touch/change the db schema
16:23:46 <avishay> thingee: your list of core seems good to me
16:24:32 <caitlin56> henma: with some solid examples on a wiki page there might be less opposition to specific restrictions. Show us how it would work with the restriction.
16:24:34 <thingee> hemna: yeah and I just want people on the same page of what core is. I really see the future of cinder is mostly just talking about extensions. Very rarely does something new come into core..it's just improvements to core mostly.
16:24:35 <avishay> hemna: i agree that it would be cool to not have them change schema, but i think we need a better alternative than joins or metadata
16:24:58 <hemna> avishay, +1
16:25:25 <avishay> not that i have any bright ideas :)
16:25:30 <hemna> heh
16:25:37 <hemna> the FC Zone Manager is a grey area
16:25:46 <hemna> which makes it a good discussion point
16:25:59 <thingee> avishay: my only concern is when you have extensions changing the model schema to say volumes. What happens when that plugin gets disabled...
16:26:40 <hemna> thingee, could be a use at your own risk documentation issue
16:26:46 <thingee> Neutron is dealing with that right now since vendors dictate what the core table schemas look like.
16:26:47 <bswartz> Where can I read about why multi-attach can't be part of core?
16:27:19 <hemna> thingee, disabling in that case breaks cinder
16:27:42 <hemna> once you enable one of those plugins it can't get disabled, unless there is a migration that gets run on disable ?
16:27:42 <avishay> is this blocking multi-attach?
16:27:44 <hemna> pain
16:28:19 <thingee> avishay: no. This will not block anything because of how fast it's progressing. Extensions should continue with development.
16:28:24 <caitlin56> One question I have: could you define an extension that made some new type of volume available where you could *only* access that volume with the extension?
16:28:28 <avishay> thingee: ok good
16:28:38 <thingee> avishay: the problem will just get worse, but we'll get there when we get there.
16:29:02 <avishay> thingee: we have to rewrite all the code anyway for taskflow, and all the unit tests for mock :)
16:29:22 <thingee> caitlin56: I see that volume still part of the basic idea of what cinder thinks a volume is. it just has added information with it.
16:29:29 <thingee> if you cinder list, it shows up.
16:29:32 <thingee> you do*
16:30:28 <caitlin56> So you use extensions to add new operations/etc., not totally new things that surpass the base. That's livable, especially if well explained on a wiki page.
16:30:53 <jungleboyj> That makes more sense to me.
16:31:04 <thingee> ok so it sounds like no one disagrees with this and I'm ok with continuing to spend time with find alternatives to joins, what to do with extensions that require a complete rewrite of a flow like FC zone manager attach/detach...and provide a useful example of how this all works.
16:31:39 <avishay> thingee: sounds good to me
16:31:42 <thingee> the persistent data thing I'm really unsure about atm.
16:31:46 <caitlin56> at the summit avishay discussed the idea of having a new type of volume that would be the passive end of a mirrored relationship. Is that something that an extension could handle?
16:31:55 <DuncanT> bswartz: That discussion about multi-attach being core has popped up many times, and come out with several different answers. Not sure the discussion has been documented anywhere. A big part of it not being core is that many installations might want to turn it off, since it is hard to use correctly and *will* generate support load and data corruption
16:32:15 <bswartz> DuncanT: thanks for the explanation
16:32:22 <thingee> DuncanT: +1
16:32:31 <avishay> caitlin56: the passive volume will not be visible to users, so maybe that will ease things?
16:33:15 <caitlin56> avishay: interesting, I'd like to hear how you use it if it isn't visible, but after this topic is finished.
16:33:33 <thingee> #todo thingee needs to figure out 1) ext persistent data 2) work with ext that require redoing an entire flow 3) provide a useful example
16:33:36 <avishay> caitlin56: https://etherpad.openstack.org/p/icehouse-cinder-continuous-volume-replication-v2
16:33:49 <thingee> thanks folks for the input
16:34:00 <avishay> thingee: thanks for the effort
16:34:08 <thingee> if anyone is interested with working on this with me, please ping me
16:34:18 <winston-1> another impact of extension is that they sometimes modified internal RPC APIs
16:34:39 <avishay> winston-1: mmm good point
16:35:02 <thingee> winston-1: yes good point. noted on the etherpad
16:35:25 <thingee> #topic icehouse-2 progress
16:35:40 <winston-1> even if you can turn off some extension, but it already modified RPC APIs, which means backwards compatibility is messed up
16:35:47 <thingee> DuncanT: https://blueprints.launchpad.net/cinder/+spec/filtering-weighing-with-driver-supplied-functions
16:35:56 <thingee> I think you started on this from discussions yesterday?
16:36:40 <DuncanT> thingee: Yup, I've got the evaluatior done, the filter itself seems to work. Unit test still need sorting, but I'm waiting for Avisay's branch to be merged since he re-wrote them all
16:37:04 <rushiagr> winston-1: so should we just disallow extensions to make backward incompatible changes?
16:37:19 <thingee> DuncanT: do you mind marking that as started?
16:37:29 <DuncanT> Sure
16:37:55 <DuncanT> Done
16:38:01 <thingee> thanks
16:38:09 <caitlin56> rushiagr: extensions will need to *add* methods. And those methods would be disabled/disappaear when the extension is disabled. But they shouldn't *change* an existing method.
16:38:13 <thingee> avishay: you're waiting jenkins failures for https://review.openstack.org/#/c/44881/ ?
16:38:16 <thingee> volume retype
16:38:52 <avishay> thingee: just put up a new version that addressed all issues so far (i think).  yea, jenkins sucks today.
16:38:54 <rushiagr> caitlin56: oh
16:39:05 <avishay> thingee: it works and is ready for re-review
16:39:15 <rushiagr> caitlin56: thats messy
16:39:23 <thingee> great, so everyone should help out out on the retype review. :)
16:39:32 <thingee> #link https://review.openstack.org/#/c/44881
16:39:39 <thingee> is Abhishek here?
16:39:43 <avishay> :)
16:40:29 <rushiagr> caitlin56: err.. that can make things messy while disabling extensions
16:40:42 <avishay> thingee: Refactor code for delete volume using TaskFlow 0.1 ?
16:40:54 <thingee> avishay: yes.
16:41:06 <thingee> alright going to assume not
16:41:10 <jungleboyj> thingee: avishay I will try to look at 44881 later today.
16:41:17 <avishay> thingee: as far as i understood, all taskflow-related work is blocked on the create_volume patch currently in the queue
16:41:21 <avishay> jungleboyj: thanks!
16:41:24 <thingee> jungleboyj: thanks. I should too since I already went through it once and have an env ready
16:41:46 <thingee> #Open topic
16:41:56 <thingee> #topic open topic
16:41:58 <thingee> :)
16:42:07 <thingee> so any bps people want to discuss?
16:42:22 <winston-1> avishay: will review retype tomorrow, err, later today
16:42:31 <avishay> winston-1: thanks a lot
16:42:51 <winston-1> thingee: https://blueprints.launchpad.net/cinder/+spec/deprecate-chance-and-simple-schedulers
16:42:51 <avishay> we should also give create_volume w/ taskflow a high priority.  it looks ok to me, but i'm not an expert.
16:43:06 <rushiagr> avishay: I'll have a look at it too. I always found reasons to skip reviewing it so far :)
16:43:21 <avishay> rushiagr: awesome
16:44:14 * jungleboyj needs to look at some of the longer patches soon.
16:44:49 <hemna> avishay, I reviewed the taskflow -> 0.1.1 yesterday and it looks good minus a few minor tweaks
16:44:54 <thingee> winston-1: looks like you were the one stopping it :) https://review.openstack.org/#/c/60690/
16:45:03 <winston-d> thingee: i'm working on a patch to actually replace chance/simple scheduler with filter scheduler.
16:45:12 <avishay> hemna: yep saw that
16:45:14 <thingee> oh ok
16:45:36 <hemna> I'll try looking at the retyping review today
16:45:42 <avishay> winston-d: is it clear which filters to use in those cases?  everything except capablities?
16:45:45 <winston-d> thingee: not really sure if anyone still using chance/simple scheduler
16:46:00 <thingee> winston-d, avishay: I noticed the replace the schedulers isn't in I-2
16:46:35 <avishay> thingee: it is now :)
16:47:29 <winston-d> avishay: i think every filters in default filter list should be included, even capablities filter. it's about weigher
16:47:30 <hemna> has jgriffith mentioned when he is getting to the brick work ?
16:47:49 <DuncanT> I'm going to make my usual point about back-compatibility here...
16:47:52 <avishay> BTW, all, please stop rechecking...poor jenkins is broken, it won't help and only adds load
16:47:55 <thingee> winston-d, avishay: ok so this bp captures deprecating...is there another for replacing with filter?
16:48:19 <xyang__> winston-d: do you know why the filters are in two different folders in cinder
16:48:20 <thingee> hemna: I was wondering that myself...I have not heard anything on it yet
16:48:21 <avishay> thingee: so winston-d's suggestion is not to deprecate, but replace them with filter scheduler under the covers
16:48:38 <avishay> DuncanT: ^
16:48:39 <thingee> avishay: got it
16:48:46 <hemna> I spoke with him at the start of I-1, and he said he slated it for I-1, but I guess he got busy
16:49:05 <DuncanT> avishay: That's fantastic, as long as running config don't stop running, I'm entirely happy
16:49:16 <avishay> DuncanT: yea sounds good to me too
16:49:33 <winston-d> xyang__: yes, i do. i can explain that offline in #openstack-cinder if you like
16:49:45 <xyang__> winston-d: ok, thanks
16:49:48 <hemna> DuncanT, did you get a chance to write the BP on the scheduluer driver methods as discussed at HK ?
16:49:49 <thingee> bswartz: do you need anything for https://blueprints.launchpad.net/cinder/+spec/multiple-capability-sets-per-backend ?
16:50:22 <DuncanT> hemna: https://blueprints.launchpad.net/cinder/+spec/filtering-weighing-with-driver-supplied-functions ?
16:50:29 <hemna> nice
16:50:31 <hemna> thanks
16:50:32 <winston-d> avishay, DuncanT replacing scheduler patches should be ready in a day or 2.
16:50:45 <avishay> bswartz: i like that BP
16:51:19 <DuncanT> winston-d: I've got patches in a holding pattern waiting on Avishay's unit test rewrite to land
16:52:10 <avishay> winston-d: cool.  note also that 'chance' has that silly ignore_hosts thing...not sure if it needs to stay
16:52:30 <avishay> thingee: since you're the resident mock expert, can you review that one and free up DuncanT ?
16:52:56 <thingee> avishay: are you going to pass the bp onto winston-d then?
16:53:00 <thingee> avishay: sure
16:53:03 <avishay> thingee: yep
16:53:09 <winston-d> avishay: i prefer no -> 'ignore_hosts', force_hosts
16:53:16 <avishay> thingee: BTW, mock is a million times better than mox
16:53:22 <thingee> avishay: I think dosaboy has taught me a thing or two from previous reviews with mock, but I'll take it :)
16:53:40 <avishay> thingee: :)
16:54:20 <thingee> bswartz: ping me later about the bp? or jgriffith whenever he wakes up :)
16:54:26 <avishay> thingee: reassigned to winston-d
16:54:39 <winston-d> thingee, DuncanT, avishay, hemna do you guys really want to make scheduler aware pools inside one back-end?
16:55:18 <bswartz> thingee: sorry I got distracted
16:55:26 <avishay> winston-d: i think it will help scalability
16:55:28 <bswartz> thingee: yes I'm working on that one
16:56:22 <thingee> winston-d: I'm not sure I follow. Though I've worked / reviewed a little with scheduler.
16:56:25 <winston-d> pools inside back-ends, small pools inside pools, smaller poools ...
16:56:56 <DuncanT> winston-d: I'm not sure of the point, but I do remember there being talk of it
16:57:07 <winston-d> thingee: well, whether there's oone pool or multiple pools within a back-end, it's transparent to scheduler, for now.
16:57:24 <avishay> bswartz: can you update the BP with its motivation please?
16:57:29 <bswartz> winston-d: almost
16:57:35 <bswartz> avishay: yes
16:57:45 <winston-d> DuncanT, bswartz do you remember if we have any consensus about that?
16:57:48 <thingee> avishay, bswartz: yeah I just want to know if its been started yet
16:57:58 <thingee> it was marked unknown
16:58:16 <bswartz> it's transparent until the scheduler picks the wrong backend and there's a failure
16:58:24 <hemna> winston-d, the pools are part of volume types for our backend
16:58:29 <xyang__> winston-d: one backend should be able to manage multiple pools
16:58:57 <winston-d> xyang__: i'm fine with that, but not sure if pools should be expose to Cinder
16:59:38 <bswartz> here's one motivation: suppose a backend has 2 pools with 50GB space each -- it reports 100GB capacity to the scheduler, if the scheduler sends a 100GB create request, it will fail. It would be better if the backend could just report that it has 2 pools of 50GB each
17:00:02 <thingee> alright lets take the discussion for pools into #openstack-cinder
17:00:04 <thingee> we're out of time
17:00:07 <bswartz> that's a simple example, there are more complicated ones -- I'll update teh BP though
17:00:08 <thingee> #endmeeting