16:04:11 <jgriffith> #startmeeting cinder
16:04:12 <openstack> Meeting started Wed Nov 28 16:04:11 2012 UTC.  The chair is jgriffith. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:04:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:04:15 <openstack> The meeting name has been set to 'cinder'
16:04:17 <thingee> o/
16:04:31 <bswartz> hello
16:04:31 <kmartin> hi
16:04:34 <avishay> hi
16:04:34 <eharney> hello
16:05:00 <jgriffith> bswartz: The official cut is hopefully tomorrow
16:05:16 <jgriffith> bswartz: the mail list has quite a bit of info/discussion/updates
16:05:37 <jgriffith> bswartz: The cut date however has passed and we got everything for cinder in that we wanted
16:06:01 <jgriffith> alrighty... I didn't update the meetings wiki this week.... sorry about that
16:06:15 <jgriffith> I just had a couple of things then I'll open it up for everybody else
16:06:25 <jgriffith> #topic FC update
16:06:33 <jgriffith> kmartin: Have any updates for us?
16:07:11 <kmartin> working on the code will add the details on Friday to the spec
16:07:29 <jgriffith> kmartin: Ok... is the code up on github or somewhere that we can get a preview?
16:07:42 <kmartin> did you see hemna question yesterday regarding the the need to enter a nova bp or not for the linvirt changes?
16:07:54 <kmartin> libvirt
16:07:56 <jgriffith> kmartin: I did not see that I don't think
16:08:10 <jgriffith> kmartin: But yes, there would need to be BP's for the nova side as well if that was the question
16:08:54 * thingee gives bad advice, sorry guys!
16:08:57 <jgriffith> kmartin: Was that the question?
16:08:59 <kmartin> basically, do we need to enter a bp for the libvirt changes on the nova side or just reference the cinder bp in the commit
16:09:22 <jgriffith> kmartin: I'd do a BP for the Nova side and reference it as required for the Cinder BP
16:09:36 <jgriffith> kmartin: make sense?
16:09:44 <kmartin> ok, will do that
16:09:47 <jgriffith> cool
16:09:56 <jgriffith> alright... anything else on the subject?
16:09:59 <kmartin> I hope we don't have to go through legal again
16:10:06 <jgriffith> kmartin: me too :)
16:10:11 <kmartin> :)
16:10:16 <jgriffith> kmartin: If that's a risk we can talk offline
16:10:31 <kmartin> jgriffith: ok thanks
16:10:34 <jgriffith> kmartin: Stupid to tie things up for another period of weeks for that silly crap
16:10:43 <kmartin> I agree
16:10:54 <jgriffith> alright... well, let me know how that shapes up
16:11:03 <kmartin> sure, will do
16:11:09 <jgriffith> and if you can get the code public sooner rather than later that would be good too
16:11:15 <jgriffith> kmartin: thanks
16:11:20 <jgriffith> #topic LIO target
16:11:26 <jgriffith> eharney: :)
16:11:35 <eharney> hi there
16:11:38 <jgriffith> eharney: So first sorry if I misundersood your initial intent there
16:12:00 <jgriffith> eharney: I think that doing an LIO version of iSCSI is the right way to start
16:12:07 <eharney> it's ok... so, it sounds like the current leading idea is to do a targetcli-based one first
16:12:26 <jgriffith> eharney: I think that's the right way to go
16:12:50 <jgriffith> eharney: I'd like to start with that as the iscsi subclass like we talked about
16:12:58 <eharney> i don't see any problem with that from my end
16:12:59 <eharney> right
16:13:15 <jgriffith> eharney: Then I'd like to see how things might shape up for a full remote driver for Grizz
16:13:32 <jgriffith> eharney: That use case is particularly interesting IMO
16:13:42 <jgriffith> eharney: Remote LVM boxes :)
16:14:03 <jgriffith> eharney: But I think phasing it and learning as we go might be good
16:14:24 <eharney> this makes sense to me
16:14:31 <jgriffith> eharney: great
16:14:39 <jgriffith> eharney: anything else you want to bring up with folks?
16:14:40 <eharney> my only real issue i've found was that targetcli may need some work, but it shouldn't be anything too bad
16:14:55 <eharney> i don't think so
16:15:05 <jgriffith> eharney: what kind of *work*?
16:15:12 <jgriffith> eharney: error reporting etc?
16:15:14 <bswartz> eharney: are you doing anything on the iscsi initiator side, or just target stuff?
16:15:32 <eharney> yes, error reporting -- it doesn't really do it in a useful fashion for us
16:15:48 <eharney> i.e. never returns codes, only prints colorful messages
16:15:59 <jgriffith> eharney: hmm... yeah that was one of the concerns that somebody mentioned on the ML
16:16:09 <eharney> bswartz: just the target side
16:16:32 <jgriffith> eharney: well, depending it might be a good opportunity to improve tgt-cli as well :)
16:16:53 <eharney> the alternative was to interface with python-rtslib rather than targetcli, but it has a different set of issues
16:17:03 <eharney> since it's library code that expects to run as root, and doesn't really fit in the rootwrap scheme well
16:17:11 <jgriffith> eharney: yeah, that didn't seem *ready* IMO
16:17:25 <eharney> well, targetcli uses that library
16:17:38 <eharney> but yes, i think i have a good plan to go forward for now
16:17:49 <jgriffith> eharney: Ok, keep us posted
16:17:58 <eharney> ok
16:18:03 <jgriffith> eharney: thanks!
16:18:15 <jgriffith> #topic capabilities reporting
16:18:27 <jgriffith> So we started talking about this a bit
16:18:50 <jgriffith> I'd like to propose we start moving forward with what is on the etherpad
16:18:53 <jgriffith> https://etherpad.openstack.org/cinder-backend-capability-report
16:19:12 <jgriffith> It's been idle with the exception of a few things I added this morning
16:19:35 <jgriffith> I'd like to start implementing this and we can always tear it apart in review if folks aren't happy with it
16:19:39 <bswartz> jgriffith: is this going to be a new driver entrypoint to be consumed by the scheduler?
16:19:53 <jgriffith> bswartz: the capabilities?
16:19:57 <bswartz> yes
16:20:24 <jgriffith> bswartz: Could be I suppose
16:20:36 <bswartz> what else is it intended for?
16:20:49 <jgriffith> bswartz: The goal for me right now was to be able to report back to a user what a back-end can do
16:20:58 <jgriffith> bswartz: Well... a user but more importantly the API
16:21:13 <jgriffith> bswartz: One example is the 'online_clone' parameter
16:21:31 <jgriffith> bswartz: unless someobdy has a better idea for how to store/report that sort of thing?
16:21:41 <bswartz> by "user" do you mean "end user" or "administrator"?
16:21:59 <jgriffith> bswartz: might be either...
16:22:06 <avishay> jgriffith: with something like "online_clone", what would a user do with that knowledge?
16:22:13 <jgriffith> bswartz: There are cases for both
16:22:25 <jgriffith> avishay: That's not a good example :)
16:22:34 <avishay> :)
16:22:35 <jgriffith> avishay: That's more for use by the API and Manager IMO
16:23:00 <jgriffith> avishay: The user would just get an error message if they tried to do a clone of an attached volume if it was False
16:23:04 <avishay> So what's a good example of something the user should be aware of?  I think the manager/scheduler should be making most/all of these decisions
16:23:21 <jgriffith> avishay: actually.. I thnk you're correct
16:23:21 <bswartz> it seems like users should be kept in the dark about driver capabilities (because there may be multiple drivers, and the administrator doesn't want them to know) and the administrators could get everything they need from the driver documentation
16:23:33 <jgriffith> bswartz: agreed
16:23:49 <jgriffith> so correct my wording above....
16:24:09 <bswartz> the scheduler seems to be the main consumer of the this API as far as I can see
16:24:12 <jgriffith> The goal for me anyway was the ability to do things in the manager and API
16:24:23 <jgriffith> bswartz: sure, ok
16:24:37 <jgriffith> bswartz: so just to clarify...
16:25:00 <bswartz> okay, I like the idea of a capabilities API so that schedulers can make smarter decisions
16:25:00 <jgriffith> bswartz: The intent was to have a methods in the drivers to query and return this info
16:25:57 <jgriffith> bswartz: Ok, sounds like we all agree at least at a high level
16:26:10 <jgriffith> bswartz: avishay think we need to get some code up to get in to the details
16:26:33 <gregjacobs> greetings
16:26:34 <jgriffith> Any other thoughts on this?  Anything horrible on the etherpad, or anything obviously missing?
16:27:14 <avishay> jgriffith: this is very closely tied to the filter scheduler that's under review, right?
16:27:15 <bswartz> well unless we can see other users for the API, I think we should design the API around things that we know the scheduler will be able to make use of
16:27:55 <bswartz> and it probably ties in with volume_type_extra_specs
16:27:58 <jgriffith> avishay: yes, there's potential for that
16:28:06 <jgriffith> bswartz: yes
16:28:21 <jgriffith> So I think there's a number of possibilities this opens up
16:28:46 <jgriffith> anyway... just wanted to make sure we had at leat general agreement to move forward on it
16:29:02 <jgriffith> s/leat/least/
16:29:33 <jgriffith> bswartz: So to be clear I had not intended a *new* API
16:29:58 <avishay> what about support for modifying a volume's type? (obviously not for the first pass)
16:30:20 <jgriffith> avishay: I dunno... interesting, but what's the use case for that?
16:30:36 <avishay> jgriffith: tiering, for example
16:30:49 <avishay> move a mostly idle volume to slower, cheaper disks
16:31:01 <bswartz> avishay: would modifying the volume's type just change the metadata, or actually move the volume to new storage?
16:31:02 <jgriffith> avishay: ahh... a migration
16:31:07 <avishay> maybe within a back-end, maybe across back-ends
16:31:21 <avishay> possibly migration
16:31:31 <bswartz> migration could get complicated
16:31:35 <jgriffith> avishay: yeah... so that's an interesting one that I thought through and *thought* I had a solution for but maybe not
16:31:40 <bswartz> especially cross-backend
16:31:42 <avishay> if the same back-end supports the new capabilities, it could do some transformation
16:31:59 <avishay> bswartz: nothing worth doing is easy :)
16:31:59 <jgriffith> avishay: My thought was that for same back-end you leave the type (the type can define the back-end actually)
16:32:11 <jgriffith> avishay: extra specs could be used for the different levels
16:32:31 <jgriffith> avishay: this works really well for the tiering/moving that you described IMO
16:33:01 <bswartz> jgriffith: the way volume_types are today, it's up to the administrator
16:33:08 <jgriffith> avishay: The question I was still dealing with is whether it's an active admin operation, or part of the audit to check for changes in the extra_specs
16:33:21 <jgriffith> bswartz: Yes, and I think it should stay that way
16:33:24 <bswartz> jgriffith: a backend can have multiple volume_types and a volume_type can map to multiple backends
16:33:53 <jgriffith> bswartz: I think if that's how someobdy wants to implement it that's fine
16:33:55 <avishay> like i said, we don't need to figure this out right now, but it could be nice for the future
16:34:00 <jgriffith> bswartz: Or I should say *utilize* i8t
16:34:01 <jgriffith> it
16:34:28 <jgriffith> avishay: Actually I have plans for this very thing in G3 so we'll talk about it in the new year :)
16:34:41 <avishay> jgriffith: OK cool - anything written?
16:34:54 <jgriffith> avishay: somewher...
16:35:15 <bswartz> migration is certainly a good idea -- but I don't think we'll be able to do anything so simple as changing the volume_type and then getting the right thing to happen
16:35:17 <jgriffith> avishay: I'll throw together a google-doc or etherpad in the next couple weeks
16:35:20 <avishay> OK, if you come across it I'd be interested to see
16:35:34 <jgriffith> bswartz: Yeah, I'm not ready to tackle the migration thing yet
16:35:47 <jgriffith> bswartz: unless it's migration within a single back-end
16:36:12 <jgriffith> avishay: I'll get info out to everyone on it, but it's not time criticial right now
16:36:13 <avishay> bswartz: at first we'll probably need to mount both volumes on a server and dd from old to new
16:36:21 <jgriffith> avishay: ewwwwwww
16:36:28 <jgriffith> avishay: :)
16:36:30 <avishay> jgriffith: i said "at first" :)
16:36:35 <bswartz> avishay: that would be the lowest common denomiator case
16:36:37 <jgriffith> LOL
16:36:39 <avishay> yes
16:36:55 <jgriffith> ok... I think we've covered that sufficiently for now.  Let's move on
16:36:57 <avishay> OK, this isn't urgent - we can move on
16:37:02 <jgriffith> #topic G2 workload
16:37:25 <jgriffith> #link https://launchpad.net/cinder/+milestone/grizzly-2
16:37:36 <jgriffith> so check out the BP list
16:37:45 <jgriffith> note the assignments...
16:37:53 <jgriffith> It's a bit lop-sided :(
16:38:18 <jgriffith> I'd really like to see folks step up and help with some of these.
16:38:35 <jgriffith> particularly the API enhancements, we've assigned almost all of them to thingee
16:38:52 <jgriffith> While he can do it and won't complain we don't want to take advantage of him either ;)
16:39:08 <jgriffith> So if you can.. work with thingee and find ways to help him out on these
16:39:28 <jgriffith> there's a ton of work here and I also suspect we'll come up with more to add in the next couple weeks
16:39:38 <thingee> or buy me a beer at the next summit :)
16:39:43 <jgriffith> :)
16:39:52 <jgriffith> Another thing to keep in mind on G2...
16:40:15 <jgriffith> So I made the mistake on G1 of scheduling everything up until the schedule date
16:40:36 <jgriffith> Remember that a few days before we go in to lock-down
16:40:42 <jgriffith> and the candidate gets cut
16:40:48 <jgriffith> so we loose a couple days there
16:40:54 <jgriffith> Then... add in the holidays
16:41:19 <jgriffith> So realisticly we're almost looking at Christmas we need to have a pretty good handle on these
16:41:28 <kmartin> jgriffith: I can speak for hemna, still good progress... going through the final review process(yep... with HP legal) for the 3PAR array driver.
16:41:45 <jgriffith> kmartin: ok... sounds good
16:42:16 <jgriffith> So if somebody is assigned something here and they don't think they're going to work on it let me know earlier rather than later
16:42:35 <jgriffith> and again, if you wanna help out, sync up with whoever is assigned and see what you can do to help
16:43:11 <jgriffith> anybody have any questions/concerns on the G2 proposals thus far?
16:44:04 <jgriffith> cool...
16:44:09 <jgriffith> #topic NAS support
16:44:32 <jgriffith> So there wasn't near as much interest on the ML as I had anticipated
16:44:56 <bswartz> what were you anticipating?
16:44:59 <jgriffith> but regardless, I think if bswartz and folks want to work on this it's useful
16:45:18 <jgriffith> bswartz: response from more than the three people that I'd already heard from on the subject :)
16:45:41 <bswartz> yeah it seems like a small number are really excited about it
16:46:00 <jgriffith> bswartz: Yeah, and I think it's useful/good soo...
16:46:01 <bswartz> others either don't care or aren't willing to speak up
16:46:10 <avishay> this is support for file in addition to block?
16:46:20 <jgriffith> avishay: yup
16:46:35 <jgriffith> bswartz: So I'd like to re-iterate my proposal for an iterative path on this
16:46:53 <avishay> i missed it on the ML i guess.
16:47:02 <jgriffith> bswartz: Start with enhancing/improving the NFS driver that you guys already worked on
16:47:23 <jgriffith> bswartz: we don't *have* to export it as iscsi if there's time to change it
16:47:35 <bswartz> so the NFS driver is unrelated to the NAS stuff
16:47:48 <jgriffith> bswartz: how so?
16:48:15 <bswartz> well -- I guess it's a matter of perspective again
16:48:21 <jgriffith> ;)
16:48:29 <bswartz> the 2 features don't overlap code-wise whatsoever
16:48:42 <jgriffith> bswartz: I guess what I'm suggesting is that maybe they should
16:49:06 <jgriffith> bswartz: In other words rather than start with everything under the sun, why not build up a usable NAS driver in Cinder
16:49:25 <jgriffith> bswartz: We can work on that to use connection info other than iscsi as well
16:49:27 <bswartz> the NFS-based drivers can do everything they need to do inside of a driver, with some modifications on the nova side that have already been made
16:49:35 <jgriffith> bswartz: CEPH is a good example here
16:50:03 <bswartz> the NAS support that we're proposing is really about delivering CIFS/NFS directly to the guests
16:50:27 <jgriffith> bswartz: Yup, and that's what I was eluding to in terms of the provider_location and connection info changes
16:50:30 <bswartz> or if cinder is being used outside of OpenStack (as some are doing) then delivering CIFS/NFS to whatever
16:51:07 <jgriffith> bswartz: So I had proposed one of two directions...
16:51:24 <jgriffith> bswartz: Either a Driver model (which I still think can be done pretty well)
16:51:27 <jgriffith> bswartz: or
16:51:37 <jgriffith> bswartz: A seperate service in Cinder
16:51:57 <jgriffith> bswartz: the seperate service is in everybodys best interest for a number of reasons
16:52:04 <bswartz> what would a seperate service in cinder mean?
16:52:20 <jgriffith> bswartz: So use Nova as an example
16:52:36 <jgriffith> bswartz: Inside nova we had: compute, networking, volumes
16:52:50 <jgriffith> so I'm saying in Cinder we have: block, NAS
16:53:05 <bswartz> okay, I think that's a good model conceptually
16:53:09 <jgriffith> bswartz: it keeps the seperation of topics that I was concerned about
16:53:18 <avishay> i think that sounds good
16:53:29 <avishay> +1
16:53:41 <jgriffith> bswartz: It also makes things gateable/testable, and if done right it makes life easier for a break out later
16:53:48 <thingee> jgriffith: +1
16:54:07 <jgriffith> this is weird, we're all agreeing on something NAS related :)
16:54:25 <jgriffith> or.. I should say, *I'm* agreeing :)
16:54:43 <jgriffith> bswartz: it's a bit of work, but I think it's worth the effort
16:55:05 <jgriffith> bswartz: if we're going to do this, we should do it as well as possible
16:55:13 <bswartz> jgriffith: we still need to get together to talk, and I'd like to understand what kind of structural changes to the code would allow it to be "a seperate service in cinder" with the reduced risks you mention
16:55:30 <jgriffith> bswartz: sure
16:55:41 <jgriffith> bswartz: maybe later today we can skype or something
16:55:50 <bswartz> it's a question of at what level we seperate blocks and NAS
16:56:26 <avishay> this meeting may be scheduled too early in the west coast and too late in europe/ME/asia - people are too tired to be confrontational :)
16:56:41 <jgriffith> avishay: HaHa!!
16:56:50 * jgriffith realizes they're on to him
16:56:55 <avishay> :)
16:57:12 <bswartz> we're all programmers -- we have caffeine to counter tiredness
16:57:22 <jgriffith> bswartz: hehe... so true
16:57:32 <jgriffith> Ok.. anything else on this topic we have to address right now?
16:57:52 <jgriffith> #topic LVM
16:58:01 <jgriffith> One other thing I wanted to touch on real quick
16:58:19 <jgriffith> I started looking at some of our LVM/dd issues again and it sucks
16:58:45 <jgriffith> particularly throughput to an LVM volume that has a snapshot is horrid
16:59:07 <jgriffith> if anybody has any tricks up their sleeve or ideas they've been kicking around on this please let me know
16:59:33 <bswartz> Is it the LVM code that's slow or the way the I/O goes to disk?
16:59:49 <bswartz> put another way: would it still suck if an SSD was used intead of spinning rust?
16:59:53 <jgriffith> bswartz: it's more of the way LVM snapshots work than anything else
17:00:02 <jgriffith> bswartz: Yeah, pretty much
17:00:24 <jgriffith> bswartz: because when there's an LVM snap you're going through an exception table for every write and mapping it to the correct device etc
17:00:29 <jgriffith> bunch of overhead there
17:00:34 <bswartz> perhaps LVM is not the right base for the generic driver then?
17:00:44 <jgriffith> bswartz: correct
17:00:55 <avishay> what's the status of the re-write going on for the kernel component of LVM?  is that done?  it was supposed to support thin provisioning and have better snapshot support
17:01:03 <avishay> i stopped tracking it a while ago
17:01:18 <jgriffith> avishay: I've been looking at that a bit, but the results aren't that much better TBH
17:01:30 <jgriffith> avishay: the thin-provisioning itself is still compelling though
17:01:35 <avishay> jgriffith: OK, disappointing
17:01:48 <avishay> what other options are viable?
17:01:55 <jgriffith> Really I could live with all of this EXCEPT for the kernel bug
17:02:21 <jgriffith> remember the dd on delete kernel hang....
17:02:35 <jgriffith> Anyway, I wanted to just sort of throw this out again for folks to think about
17:02:57 <jgriffith> #topic free for all
17:03:09 <jgriffith> speak up if you have something :)
17:03:24 <bswartz> can LIO help with the above problem?
17:03:33 <jgriffith> bswartz: funny you should ask that :)
17:03:35 <bswartz> if we had a LIO driver could we wire it up to some less-sucky backend?
17:03:48 <jgriffith> bswartz: I had this crazy idea of using LIO with a file back instead of LVM
17:04:01 <jgriffith> bswartz: but I don't thnk that would fly in a production env
17:04:10 <bswartz> you still need to get snapshot support somehow
17:04:20 <avishay> jgriffith: that's what i thought too
17:04:25 <jgriffith> bswartz: So with LIO you can literally use a file as your storage, but it's recommended to use LVM
17:04:45 <avishay> btrfs?  too immature?
17:04:50 <jgriffith> avishay: :)
17:04:52 <bswartz> what if you used LIO with a file-based storage, but you used a more powerful filesystem like BTRFS?
17:04:54 <eharney> i was just thinking about btrfs too, as a longer goal...
17:05:19 <jgriffith> avishay: eharney BTRFS has been my latest, but I've read mixed results on how ready it is
17:05:33 <jgriffith> avishay: eharney I'm still willing to investigate it a bit
17:05:44 <eharney> it appears to be pretty rough around the edges still from what i understand
17:05:52 <avishay> is it still being actively worked on?  unfortunately i lost touch with my kernel roots a bit
17:06:17 <jgriffith> avishay: it is, but seems to have lost some of it's appeal from what I've been seeing
17:06:18 <eharney> avishay: i think there is active development ongoing, yes
17:06:51 <bswartz> NetApp actually implements LUNs as files inside our storage controllers, and all of the power comes from the filesystem used
17:07:05 <bswartz> if BTRFS was more production ready I think it would make a great solution
17:07:17 <avishay> bswartz: so everyone should just buy netapps ;P
17:07:22 <jgriffith> avishay: :)
17:07:26 <bswartz> YES~
17:07:29 <bswartz> YES!
17:07:33 <jgriffith> I was thinking NetApps should be free!
17:07:54 <russellb> i would totally test it out if you wanted to send me a free one.
17:08:04 <jgriffith> So I have considered making RBD the ref implementation but I dunno... that's a big switch
17:08:16 <russellb> that applies to all storage vendors in the room :-p
17:08:25 <jgriffith> russellb: :)
17:08:29 <avishay> :)
17:09:02 <jgriffith> alright... we're over, and nobody bit on the RBD thing so let's wrap up for now
17:09:14 <avishay> have the kernel issues been reported?  any response?  maybe moving to something else is a bit hasty?  i don't know the history
17:09:14 * jgriffith fishes, fishes, fishes
17:09:34 <jgriffith> avishay: Oh yes, it's reported and there is some work on it but very slow progress
17:09:45 <jgriffith> avishay: You're correct, I may be being very hasty
17:09:49 <avishay> is RBD more mature than btrfs?
17:09:50 <bswartz> jgriffith: if ceph was packaged by the distros and stable then I'd say it was a reasonable idea
17:10:18 <zykes-> any FC news ?
17:10:26 <jgriffith> avishay: well, I believe so... but the real tie in there is it's already an OpenStack option
17:10:33 <jgriffith> zykes-: You missed it, check the logs
17:10:42 <avishay> OK, I need to go - 10 minutes over - see you all next week!
17:10:48 <jgriffith> alright folks, I've kept you 10 over already
17:10:52 <jgriffith> thanks everybody
17:11:01 <jgriffith> catch me on IRC if you need anything or have further questions
17:11:05 <jgriffith> #endmeeting