18:03:12 <jdg> #startmeeting
18:03:13 <openstack> Meeting started Thu Mar 22 18:03:12 2012 UTC.  The chair is jdg. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:03:14 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic.
18:03:29 <jdg> Any folks present for the block storage meeting today?
18:04:47 <jdurgin> I'm here if people have things to discuss
18:05:49 <jdg> I was hoping to discuss boot from volume some more, as well as updates on volume uuid
18:06:10 <jdg> Looks like it's another no show
18:06:13 <DuncanT> I'm here
18:06:39 <jdg> Ok...
18:07:06 <jdg> Ok, so there was a pretty lively discussion on boot from volume last time
18:07:35 <jdg> I didn't see any note written up or proposals, was thinking I'd go ahead and do a high level description on what we talked about
18:07:43 <jdg> submit on the wiki or something
18:08:15 <jdg> Mainly wanted to focus on defining what we're looking for, as there seems to be different ideas about what BFV is
18:08:46 <jdg> ie booting an instance with a volume attached versus actually having an instance on the volume
18:08:48 <DuncanT> We've been having some internal discussions here on same, they got lively too for the same reasons
18:08:50 <jdg> Any thoughts?
18:09:06 <jdg> DuncanT: Sorry... didn't mean to cut you off
18:09:14 <jdg> :)
18:09:37 <jdg> So I can see both use cases, however the second is NOT boot from volume in my mind.  It's boot with volume
18:09:41 <jdg> plain and simple
18:09:45 <DuncanT> Agreed
18:09:52 <jdurgin> yeah
18:09:56 <DuncanT> Different issue, also needs doing :-)
18:10:01 <jdg> Agreed
18:10:13 <jdg> Ok, so we're looking at two seperate blueprints then
18:10:42 <timr1> tim here as well, yep agree "boot from" is different to" boot with" we are intertsted in the former
18:10:55 <DuncanT> Maybe more than two... it all gets a bit complicated when you start looking a possible semantics for BFV
18:11:24 <jdg> DuncanT: Good point, the other wrench to throw in is breaking out block storage
18:11:45 <jdg> But I think if that's done correctly and the API looks right it will allow alot of those use cases
18:12:18 <DuncanT> If it doesn't allow both these cases it is broken and will need fixing IMO
18:12:40 <jdg> Seems like the right thinking to me
18:12:48 <timr1> agree
18:12:52 <DuncanT> We (HP) have been discussing BFV a lot in the last week or so, we plan on having a blueprint out before the summit for discussion there
18:12:57 <jdg> the trouble I'm having is I don't want to wait until the summit to figure out what we're doing.
18:14:04 <timr1> sure, we would do the bprint before then,
18:14:05 <jdg> I think it might be good to get some ideas around use cases going into the summit
18:14:47 <jdg> timr1: ok... what sort of timeline are you thinking?
18:15:01 <jdurgin> a thread on the mailing list might collect more use cases from a wider audience, even before blueprints are ready
18:15:23 <jdg> jdurgin: good idea, do you want to send it out or shall I?
18:15:56 <jdurgin> jdg: I can do it
18:16:18 <jdg> #action jdurgin to send out mailing list request for BFV use cases
18:16:22 <DuncanT> We hope to have a blueprint in two weeks, purely so there is something concrete to discuss at the summit
18:16:34 <DuncanT> Maybe a week if other pressures allow
18:16:49 <jdg> DucnanT: sounds good
18:17:27 <jdg> #action DuncanT to submit blueprint for BFV in the next 2 weeks
18:17:32 <DuncanT> Part of the problem is that you need not just the use cases, but the cognative model to go with it, e.g. is a snapshot of a bootable volume the same as a glace image?
18:18:20 <jdg> DuncanT: I would say that's something that would be good to have
18:18:33 <jdg> DuncanT: But I also don't know that it needs to be first pass
18:19:01 <jdurgin> there are already two kinds of snapshots in nova, and cleaning up that sounds like a separate issue
18:19:08 <jdg> I guess I'm thinking of coming up with the cognitive model based on use cases
18:19:21 <jdg> Otherwise you end up with a screwy paradigm
18:19:52 <DuncanT> jdg: The two go together. There should be some beginnings of our thoughts on the subject in the blueprint
18:20:18 <DuncanT> If you implement use-cases without thinking of the model then you end up with screwy APIs :-)
18:20:35 <jdg> DuncanT: Agreed
18:21:15 <jdg> But the other direction is true as well
18:21:24 <DuncanT> Indeed
18:21:39 <jdg> Ok, so this is good.  We at least have some steps for where to go
18:22:10 <jdg> I think there are quite a few things that can be done with block storage, seems like a priority list should be put together at the summit
18:22:38 <jdg> hint, hint... propose sessions on the web page :)
18:24:03 <jdg> Ok, does anybody have any topics they want to discuss today?  We have a couple action items already.
18:24:40 <jdg> If anybody is interested in uuid conversion for volume ID's I'll fill ya in on that?
18:25:25 <jdurgin> sounds good
18:25:36 <DuncanT> We have some interest in getting some firming up of the semantics of snapshot & backup
18:25:59 <jdg> DuncanT: any specific proposals/ideas?
18:26:30 <DuncanT> There's an old blueprint about it, but getting some agreement would be nice, and we only really have a view of what our storage system can do rather than what other people want/can do
18:27:08 <jdg> DuncanT: Do you have a link to the blueprint?
18:28:12 <DuncanT> https://blueprints.launchpad.net/nova/+spec/nova-volume-snapshot-backup-api
18:29:27 <DuncanT> Basically, a snapshot is a lightweight point-in-time frozen copy of a volume, a backup is an external, pan-availabily zone thing e.g. to swift
18:30:27 <jdg> DuncanT: So is there some controversy around these definitions?
18:31:03 <DuncanT> jdg: There seemed to be a couple of weeks ago, yes
18:31:35 <jdg> DuncanT: Ahh, yes... from within this meeting in fact
18:31:43 <DuncanT> jdg: Indeed
18:32:02 <jdg> Personally I don't necessarily see why.  I'd have to go back through the notes.
18:32:26 <jdg> WRT lifetime of the snapshot, I would tie it to the originating volume
18:32:44 <timr1> there was also controversy when there was only one term "SNAPSHOT"  some people wanted it to be a quick point in time snap, others wanted it to be a swift copy.
18:33:02 <timr1> that lifecycle has issues:
18:33:28 <jdg> So I think that's why you have clones and snapshots no?
18:34:23 <timr1> if I create a new volume from a s snapshot (a clone) then that should be able to live even if the original is deleted - no?
18:34:36 <timr1> original volume that is
18:34:37 <jdg> timr1: agreed
18:34:58 <jdg> timr1: But that's assuming you do a "create volume from snapshot"
18:35:10 <jdg> timr1: so I would propose it's an additional step
18:35:10 <timr1> eh, yes
18:35:42 <jdg> In other words the definition:  snapshots are for point in time restores to a volume
18:35:55 <jdg> They can be used to create a new volume (clone)
18:36:14 <jdg> The new volume is not tied to the snapshot it orignates from
18:36:53 <jdg> does anybody have any issues with going that route?
18:37:19 <DuncanT> jdg: We see no reason to tie the lifetime of a snapshot with that of any volume
18:37:27 <timr1> can you expand on what you mean by: "snapshots are for point in time restores to a volume"
18:37:56 <jdg> DuncanT: So you're proposing that even if you delete the originating volume the snapshot should "live on"
18:38:20 <DuncanT> jdg: Yes
18:38:38 <timr1> agree
18:38:53 <jdg> DuncanT: timr1:  Ok... so why?
18:38:59 <timr1> but you cannot delete the snap if clones are in existence ?
18:39:18 <DuncanT> A use-case:
18:39:21 <jdg> timr1: seems like you're making life difficult by doing that
18:39:38 <jdurgin> it sounds like different volume backends have different assumptions about the lifetime of snapshots/volumes/clones, and whether they're tied together, so we can't have a single global policy
18:39:57 <timr1> jdg: sorry 0 that was a question, not an assertion :)
18:40:30 <jdg> timr1: Oh... sorry.
18:41:23 <DuncanT> I setup a volume with my basic ubuntu install, and snapshot it. I then clone this many times for different purposes. The fact my first volume eventually moves on and maybe becomes useless doesn't stop the snapshot still being useful
18:41:49 <jdg> DuncanT:  Ok, good use case
18:42:24 <jdg> However, depending on how the clone is implemented it may or may not know anything about the snapshot
18:42:41 <jdg> The other problem is then you have to deal with managing which snapshots are valid for which clones
18:42:54 <timr1> ok, so we agree we can delete the original volume and the snap and any clones can live on - OK ?
18:42:56 <DuncanT> valid?
18:43:18 <DuncanT> I don't understand what you mean by that
18:43:32 <jdg> timr1: Not sure I agree, however if the majority likes that approach that's cool
18:44:07 <jdg> DucanT: Say you have volume-1, you create snapshot-a
18:44:08 <timr1> jdg, I was tryign to sumarise what Duncan was saying - thougth you did agree with his use case ?
18:44:10 <jdurgin> timr1: it still seems backend dependent - i.e. lvm and rbd can't use snapshots after their associated volume has been deleted (not sure about other backends)
18:44:52 <DuncanT> jdurgin: Then the implementation can in that case refuse to delete the volume until the snaps are deleted?
18:44:56 <jdg> Then you clone
18:45:18 <jdg> Then you snap volume-a again... you didn't do anything with volume-b
18:45:34 <jdg> Point in time discrepencies start cropping up all over the place
18:46:32 <jdg> jdurgin: The backend dependency is another concern
18:46:45 <jdurgin> DuncanT: yeah, my point is that the varying semantics support different use cases, but this means drivers have to handle the discrepancies
18:46:50 <jdg> Depending on how snapshots are implemented it might not work at all
18:47:28 <DuncanT> jdg: I don't see the descrepacies... both snaps are logically copies of the volume at the instant of time they were created. Once you clone them, the clone becomes a (semantically) separate entity that can do its own thing
18:49:03 <jdg> DucanT: right but then you're talking about restoring snaps from "other" volumes
18:49:12 <jdg> That's not a snapshot, that's a clone
18:49:38 <jdg> Sorry...
18:50:04 <jdg> So what I'm saying is... once you create volume from a snapshot that should be the start of it's lifetime
18:50:51 <DuncanT> Right, I see, we're using the terms differently here. In your parlance, creating any volume from a snapshot is a clone
18:50:54 <jdg> Depending on how snapshots are implemented it actually has to be this way
18:51:18 <jdg> DucanT: correct, thanks :)
18:51:44 <jdg> Does that mean we actually agree on this?
18:52:10 <DuncanT> I /think/ we're talking about the same thing here. If I email out some examples and see if they match your idea?
18:52:28 <jdg> Sounds like a great plan.
18:52:34 <timr1> I alos agree that the capabilities of back end implementations will differ depending on the technology used. For this reason the Nova semantics should be more relaxed and allow the back end decide what is possible in its own context
18:52:47 <jdg> timr1: +1
18:53:06 <timr1> ok - more write ups needed from Duncan :)
18:53:11 <jdurgin> timr1: agreed
18:53:21 <jdg> It seems to me there are some generalized semantics already around these terms, we should stick with them
18:53:23 <timr1> I'll help him
18:53:51 <jdg> Do you want to do mass email first, or just to this group to start?
18:54:07 <jdg> Might be easier to hash it out amongst the smaller group then propose it to the entire list
18:54:15 <DuncanT> Agreed
18:54:21 <timr1> yep
18:54:26 <jdg> Cool
18:55:07 <jdg> Alright, anything else?  Or should we keep going and assign more tasks to DuncanT  :)
18:55:14 <rnirmal> jdg: sorry I missed the earlier part... do you have an update on moving id's to uuid's
18:55:29 <jdg> #topic volume uuid
18:55:41 <jdg> rnirmal:  thanks for reminding me
18:56:07 <jdg> So I implemented the migrations as well as the model and the db api
18:56:20 <rnirmal> is there a way we can get this for essex or is it just folsom
18:56:21 <jdg> fighting with the EC2 tests now
18:56:50 <jdg> rnirmal: So I haven't been really agressive with it because it sounded like it was nixed from essex
18:57:01 <rnirmal> yeah that's what I thought too
18:57:05 <rnirmal> just wanted to make sure
18:57:46 <jdg> rnirmal: So now I'm sort of in a spot, trying to keep up with changes but not submitting until after Essex
18:58:16 <jdg> May be smarter to just get it figured out and then wait until after release
18:58:30 <rnirmal> yeah I think we can work on getting it now... the release has been cut and trunk open for folsom
18:58:48 <rnirmal> atleast from one of the last commits... but I may be wrong
18:58:53 <jdg> Oh, that's right forgot Thierry sent out an email
18:59:22 <rnirmal> I'm interested in testing it out. I'll probably merge from your branch
18:59:39 <jdg> Ok, I'll try devote some more time to it next week and try to get it done.
18:59:59 <jdg> rnirmal: I'll shoot you a note when there's something ready on my github branch.
19:00:07 <rnirmal> jdg: perfect thanks.
19:00:07 <jdg> It's terribly out of date right now  :(
19:00:20 <jdg> Ok, anybody have anything else?
19:00:51 <jdg> Alrighty, thanks everyone for showing up.  Same time and place next week.  :)
19:00:57 <DuncanT> I've volenteered for enough for one week :-)
19:00:59 <timr1> ciao
19:01:15 <jdg> Dont' forget if you want to add agenda items to the wiki feel free.
19:01:18 <jdg> Later!
19:01:22 <jdg> #endmeeting