18:03:12 #startmeeting 18:03:13 Meeting started Thu Mar 22 18:03:12 2012 UTC. The chair is jdg. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:03:14 Useful Commands: #action #agreed #help #info #idea #link #topic. 18:03:29 Any folks present for the block storage meeting today? 18:04:47 I'm here if people have things to discuss 18:05:49 I was hoping to discuss boot from volume some more, as well as updates on volume uuid 18:06:10 Looks like it's another no show 18:06:13 I'm here 18:06:39 Ok... 18:07:06 Ok, so there was a pretty lively discussion on boot from volume last time 18:07:35 I didn't see any note written up or proposals, was thinking I'd go ahead and do a high level description on what we talked about 18:07:43 submit on the wiki or something 18:08:15 Mainly wanted to focus on defining what we're looking for, as there seems to be different ideas about what BFV is 18:08:46 ie booting an instance with a volume attached versus actually having an instance on the volume 18:08:48 We've been having some internal discussions here on same, they got lively too for the same reasons 18:08:50 Any thoughts? 18:09:06 DuncanT: Sorry... didn't mean to cut you off 18:09:14 :) 18:09:37 So I can see both use cases, however the second is NOT boot from volume in my mind. It's boot with volume 18:09:41 plain and simple 18:09:45 Agreed 18:09:52 yeah 18:09:56 Different issue, also needs doing :-) 18:10:01 Agreed 18:10:13 Ok, so we're looking at two seperate blueprints then 18:10:42 tim here as well, yep agree "boot from" is different to" boot with" we are intertsted in the former 18:10:55 Maybe more than two... it all gets a bit complicated when you start looking a possible semantics for BFV 18:11:24 DuncanT: Good point, the other wrench to throw in is breaking out block storage 18:11:45 But I think if that's done correctly and the API looks right it will allow alot of those use cases 18:12:18 If it doesn't allow both these cases it is broken and will need fixing IMO 18:12:40 Seems like the right thinking to me 18:12:48 agree 18:12:52 We (HP) have been discussing BFV a lot in the last week or so, we plan on having a blueprint out before the summit for discussion there 18:12:57 the trouble I'm having is I don't want to wait until the summit to figure out what we're doing. 18:14:04 sure, we would do the bprint before then, 18:14:05 I think it might be good to get some ideas around use cases going into the summit 18:14:47 timr1: ok... what sort of timeline are you thinking? 18:15:01 a thread on the mailing list might collect more use cases from a wider audience, even before blueprints are ready 18:15:23 jdurgin: good idea, do you want to send it out or shall I? 18:15:56 jdg: I can do it 18:16:18 #action jdurgin to send out mailing list request for BFV use cases 18:16:22 We hope to have a blueprint in two weeks, purely so there is something concrete to discuss at the summit 18:16:34 Maybe a week if other pressures allow 18:16:49 DucnanT: sounds good 18:17:27 #action DuncanT to submit blueprint for BFV in the next 2 weeks 18:17:32 Part of the problem is that you need not just the use cases, but the cognative model to go with it, e.g. is a snapshot of a bootable volume the same as a glace image? 18:18:20 DuncanT: I would say that's something that would be good to have 18:18:33 DuncanT: But I also don't know that it needs to be first pass 18:19:01 there are already two kinds of snapshots in nova, and cleaning up that sounds like a separate issue 18:19:08 I guess I'm thinking of coming up with the cognitive model based on use cases 18:19:21 Otherwise you end up with a screwy paradigm 18:19:52 jdg: The two go together. There should be some beginnings of our thoughts on the subject in the blueprint 18:20:18 If you implement use-cases without thinking of the model then you end up with screwy APIs :-) 18:20:35 DuncanT: Agreed 18:21:15 But the other direction is true as well 18:21:24 Indeed 18:21:39 Ok, so this is good. We at least have some steps for where to go 18:22:10 I think there are quite a few things that can be done with block storage, seems like a priority list should be put together at the summit 18:22:38 hint, hint... propose sessions on the web page :) 18:24:03 Ok, does anybody have any topics they want to discuss today? We have a couple action items already. 18:24:40 If anybody is interested in uuid conversion for volume ID's I'll fill ya in on that? 18:25:25 sounds good 18:25:36 We have some interest in getting some firming up of the semantics of snapshot & backup 18:25:59 DuncanT: any specific proposals/ideas? 18:26:30 There's an old blueprint about it, but getting some agreement would be nice, and we only really have a view of what our storage system can do rather than what other people want/can do 18:27:08 DuncanT: Do you have a link to the blueprint? 18:28:12 https://blueprints.launchpad.net/nova/+spec/nova-volume-snapshot-backup-api 18:29:27 Basically, a snapshot is a lightweight point-in-time frozen copy of a volume, a backup is an external, pan-availabily zone thing e.g. to swift 18:30:27 DuncanT: So is there some controversy around these definitions? 18:31:03 jdg: There seemed to be a couple of weeks ago, yes 18:31:35 DuncanT: Ahh, yes... from within this meeting in fact 18:31:43 jdg: Indeed 18:32:02 Personally I don't necessarily see why. I'd have to go back through the notes. 18:32:26 WRT lifetime of the snapshot, I would tie it to the originating volume 18:32:44 there was also controversy when there was only one term "SNAPSHOT" some people wanted it to be a quick point in time snap, others wanted it to be a swift copy. 18:33:02 that lifecycle has issues: 18:33:28 So I think that's why you have clones and snapshots no? 18:34:23 if I create a new volume from a s snapshot (a clone) then that should be able to live even if the original is deleted - no? 18:34:36 original volume that is 18:34:37 timr1: agreed 18:34:58 timr1: But that's assuming you do a "create volume from snapshot" 18:35:10 timr1: so I would propose it's an additional step 18:35:10 eh, yes 18:35:42 In other words the definition: snapshots are for point in time restores to a volume 18:35:55 They can be used to create a new volume (clone) 18:36:14 The new volume is not tied to the snapshot it orignates from 18:36:53 does anybody have any issues with going that route? 18:37:19 jdg: We see no reason to tie the lifetime of a snapshot with that of any volume 18:37:27 can you expand on what you mean by: "snapshots are for point in time restores to a volume" 18:37:56 DuncanT: So you're proposing that even if you delete the originating volume the snapshot should "live on" 18:38:20 jdg: Yes 18:38:38 agree 18:38:53 DuncanT: timr1: Ok... so why? 18:38:59 but you cannot delete the snap if clones are in existence ? 18:39:18 A use-case: 18:39:21 timr1: seems like you're making life difficult by doing that 18:39:38 it sounds like different volume backends have different assumptions about the lifetime of snapshots/volumes/clones, and whether they're tied together, so we can't have a single global policy 18:39:57 jdg: sorry 0 that was a question, not an assertion :) 18:40:30 timr1: Oh... sorry. 18:41:23 I setup a volume with my basic ubuntu install, and snapshot it. I then clone this many times for different purposes. The fact my first volume eventually moves on and maybe becomes useless doesn't stop the snapshot still being useful 18:41:49 DuncanT: Ok, good use case 18:42:24 However, depending on how the clone is implemented it may or may not know anything about the snapshot 18:42:41 The other problem is then you have to deal with managing which snapshots are valid for which clones 18:42:54 ok, so we agree we can delete the original volume and the snap and any clones can live on - OK ? 18:42:56 valid? 18:43:18 I don't understand what you mean by that 18:43:32 timr1: Not sure I agree, however if the majority likes that approach that's cool 18:44:07 DucanT: Say you have volume-1, you create snapshot-a 18:44:08 jdg, I was tryign to sumarise what Duncan was saying - thougth you did agree with his use case ? 18:44:10 timr1: it still seems backend dependent - i.e. lvm and rbd can't use snapshots after their associated volume has been deleted (not sure about other backends) 18:44:52 jdurgin: Then the implementation can in that case refuse to delete the volume until the snaps are deleted? 18:44:56 Then you clone 18:45:18 Then you snap volume-a again... you didn't do anything with volume-b 18:45:34 Point in time discrepencies start cropping up all over the place 18:46:32 jdurgin: The backend dependency is another concern 18:46:45 DuncanT: yeah, my point is that the varying semantics support different use cases, but this means drivers have to handle the discrepancies 18:46:50 Depending on how snapshots are implemented it might not work at all 18:47:28 jdg: I don't see the descrepacies... both snaps are logically copies of the volume at the instant of time they were created. Once you clone them, the clone becomes a (semantically) separate entity that can do its own thing 18:49:03 DucanT: right but then you're talking about restoring snaps from "other" volumes 18:49:12 That's not a snapshot, that's a clone 18:49:38 Sorry... 18:50:04 So what I'm saying is... once you create volume from a snapshot that should be the start of it's lifetime 18:50:51 Right, I see, we're using the terms differently here. In your parlance, creating any volume from a snapshot is a clone 18:50:54 Depending on how snapshots are implemented it actually has to be this way 18:51:18 DucanT: correct, thanks :) 18:51:44 Does that mean we actually agree on this? 18:52:10 I /think/ we're talking about the same thing here. If I email out some examples and see if they match your idea? 18:52:28 Sounds like a great plan. 18:52:34 I alos agree that the capabilities of back end implementations will differ depending on the technology used. For this reason the Nova semantics should be more relaxed and allow the back end decide what is possible in its own context 18:52:47 timr1: +1 18:53:06 ok - more write ups needed from Duncan :) 18:53:11 timr1: agreed 18:53:21 It seems to me there are some generalized semantics already around these terms, we should stick with them 18:53:23 I'll help him 18:53:51 Do you want to do mass email first, or just to this group to start? 18:54:07 Might be easier to hash it out amongst the smaller group then propose it to the entire list 18:54:15 Agreed 18:54:21 yep 18:54:26 Cool 18:55:07 Alright, anything else? Or should we keep going and assign more tasks to DuncanT :) 18:55:14 jdg: sorry I missed the earlier part... do you have an update on moving id's to uuid's 18:55:29 #topic volume uuid 18:55:41 rnirmal: thanks for reminding me 18:56:07 So I implemented the migrations as well as the model and the db api 18:56:20 is there a way we can get this for essex or is it just folsom 18:56:21 fighting with the EC2 tests now 18:56:50 rnirmal: So I haven't been really agressive with it because it sounded like it was nixed from essex 18:57:01 yeah that's what I thought too 18:57:05 just wanted to make sure 18:57:46 rnirmal: So now I'm sort of in a spot, trying to keep up with changes but not submitting until after Essex 18:58:16 May be smarter to just get it figured out and then wait until after release 18:58:30 yeah I think we can work on getting it now... the release has been cut and trunk open for folsom 18:58:48 atleast from one of the last commits... but I may be wrong 18:58:53 Oh, that's right forgot Thierry sent out an email 18:59:22 I'm interested in testing it out. I'll probably merge from your branch 18:59:39 Ok, I'll try devote some more time to it next week and try to get it done. 18:59:59 rnirmal: I'll shoot you a note when there's something ready on my github branch. 19:00:07 jdg: perfect thanks. 19:00:07 It's terribly out of date right now :( 19:00:20 Ok, anybody have anything else? 19:00:51 Alrighty, thanks everyone for showing up. Same time and place next week. :) 19:00:57 I've volenteered for enough for one week :-) 19:00:59 ciao 19:01:15 Dont' forget if you want to add agenda items to the wiki feel free. 19:01:18 Later! 19:01:22 #endmeeting