17:05:40 #startmeeting 17:05:42 Meeting started Thu Nov 3 17:05:40 2011 UTC. The chair is renuka. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:05:43 Useful Commands: #action #agreed #help #info #idea #link #topic. 17:05:54 #topic Volume Affinity 17:06:20 It might be useful to have your thoughts online, so anyone else can read it later 17:06:21 http://wiki.openstack.org/VolumeAffinity has the details I've put up so far 17:06:36 #link http://wiki.openstack.org/VolumeAffinity 17:07:06 So what do you expect will go as key value pairs? 17:08:09 Have you decided how you will extend the volume create API? 17:08:22 Yes, it looked like "affinity:volume1,volume2,volume3 anti-affinity:volume4" notation could express what we are thinking of 17:08:33 No, the API was an open issue 17:08:59 volume-types doesn't allow key-value pairs at create time as it stands 17:09:31 vishy: Should API related things be taken up in the nova api meeting or should we come up with extensions here 17:09:41 extensions here 17:09:54 Vladimir and I couldn't see a reason not to add an 'extras' field, that can contain key-value pairs, with only affinity and anti-affinity being defined for now 17:09:58 volume api will be separated out so you guys will be responsible for it 17:10:23 ultimately affinity to computes would be interesting as well 17:10:24 vishy: right, thanks 17:10:30 but that seems like a good start 17:10:57 vishy: I agree. That will likely help boot from volume 17:10:58 Should we call it 'volume-affinity' and volume-anti-affinity' to avoid ambiguities later then? 17:11:19 sure 17:11:42 or perhaps it could be affinity:volume- 17:11:46 Do you think we should have something as generic as extras in the API? 17:11:56 and we could use instance- for computes 17:11:58 If this has structure, why not make it an actual option? 17:12:19 vishy: +! 17:12:31 I meant +1 :P 17:12:34 leaving room for other types in the future without having an explosion of keys 17:12:42 I'm thinking we're likely to want to add other options in future 17:13:37 I don't understand. (Potential noob question). If someone has to use the option, they would have to know the key affinity 17:13:57 So anyone using specific things in extras needs to know the keywords 17:14:21 By adding extras, we have to add the parsing for it.. doesn't that get more complicated later? 17:15:49 true 17:16:04 key:value pairs end up being a json dictionary, which has a standard parsing 17:16:10 Plus the API is not self explanatory.. because there could be all sorts of features that go in as "extras" 17:16:31 this is the general idea behind api extensions 17:16:46 extensions can be added to support specific features 17:17:42 vishy: but that is exactly why I would think we need not have another layer of generic options... unless we think we need them. Like in the case of connectivity, there is no way we can support all the options that all storage types have 17:17:46 key value do seem like undocumented features 17:18:28 internally they make sense 17:18:48 but i think what is exposed to the api should attempt to be actual options in the request 17:18:49 but in case of affinity/anti-affinity, we seem to have the keywords nailed. Anything else like compute that gets added later, can go in, as you said affinity: volume-id, compute-id, etc 17:18:51 I'm not wedded to the idea, it just seems cleaner than having to make structural changes to clients to support some minor API feature 17:18:58 so that they can be documented. 17:19:19 if it is an optional parameter 17:19:25 clients can ignore it 17:19:41 i guess i see where you are going though 17:20:24 if a vendor wants to add a new parameter, they would have to go and modify the client to know about it 17:20:43 vish: Quite 17:20:54 that seems like something that is useful to bring up in the nova-api meeting 17:20:59 Why in this case, in particular? 17:21:11 scheduler hints 17:21:18 wouldn't affinity and anti-affinity keywords be sufficient? 17:21:35 renuka: in this case, but what if there are other scheduler hints 17:21:52 ssd:preferred 17:21:57 or something along those lines 17:22:24 vishy: I thought we agreed that types should be used for that 17:22:26 every new hint would have to be added to the client 17:22:40 renuka: types are great for requirements but not hints 17:23:01 optimize:latency vs optimize:bandwidth 17:23:19 makes sense 17:23:26 If we rename 'extras' as 'hints' and define it as information the volume driver/scheduler is free to ignore, does that help? 17:23:47 right, so I like the renaming. Just to ensure that we don't end up adding other functionality there, just because we can 17:23:58 in any case, i think we should discuss it in the api meeting. What should be codified into a documented option, and what is ok to stick in as key value pairs 17:24:27 my rough pass would be ignorable data (like hints) are ok as key/value pairs 17:24:49 vish: That makes good sense 17:25:02 #action vishy to bring up key-value pairs vs extensions in the nova-api meeting 17:25:18 #action vishy to bring up separate-volume-api in the nova-api meeting 17:25:57 i have to go grab some lunch before it is gone, can I throw in a request before i go? 17:26:04 sure 17:26:23 I would really like you guys to start reviewing all volume related patches that are thrown into gerrit 17:26:27 for example: https://review.openstack.org/#change,1202 17:26:55 even if it is just to say, that looks fine 17:27:11 will do 17:27:16 ok willn do 17:27:20 vish: I've been playing with that code today funny enough 17:27:21 will 17:27:46 #action volume team to start reviewing volume related code in gerrit 17:27:49 great 17:28:32 yup will do 17:29:25 Right, so anything else to be discussed for affinity? This blueprint will be implemented by HP, correct? 17:30:27 renuka: Yes, though it is closely tied to the volume-types work in places 17:31:48 DuncanT: right, so the volume-types work is already in, correct? 17:33:15 renuka: I'm not certain it is in its final form, but I'll check with Vladimir and make sure we don't cause each other headaches 17:34:00 #action HP to look into volume types requirements and implement Volume Affinity/Anti Affinity 17:34:39 The next thing i wanted to bring up was the need for an admin API for dynamic storage 17:34:53 Does anyone else feel the need for this? 17:35:25 renuka, what do you mean by dynamic storage? 17:36:02 tim-at-home: We do not use storage local to the nova-volume node for volumes 17:36:30 so we end up having to "add" our storage backends/arrays and remove them when they need to be offline etc 17:36:38 renuka, gotcha - 17:36:48 renuka: Is this something that will end up generic, or is it better as a vendor extension? 17:36:58 wont this be very specific? 17:37:05 At the moment, the xen storage manager driver has its own table, and uses the nova-manage command to add/remove this 17:37:44 this is how the nova-manage command looks at the moment: 17:37:46 nova-manage sm backend_add [config connection parameters] 17:38:06 flavor here means type (needs to be refactored) 17:38:13 for example we have a proprietary API for managing our storage - it is very specific to our particular archtiectrure 17:38:55 So I guess we are leaning towards vendor extensions then 17:39:31 I do not feel strongly about it, 17:39:37 ohters? 17:39:38 I think so. We might look at how much commonality there is later, but I don't see any advantage at the moment 17:39:57 each vendor already has existing tools to manage this, should we recreate them within the context of nova 17:41:39 without a common table that contains the storage available in the zone, would be still be able to function well across various vendors/storage types? 17:42:27 Surely the available storage is know dynamically by the scheduler based on what nova-volume servers present themselves? 17:43:08 right, i guess report capabilities can be used appropriately 17:43:26 agree 17:44:48 next up was the netapp blueprint that robert mentioned on the emails. Is anyone from netapp here? 17:45:32 guess not 17:45:47 Anything else anyone would like to bring up? 17:46:23 Is boot-from-volume going to be discussed at some point? 17:46:38 We are certainly interested in that 17:46:49 is anyone using it? 17:46:52 The current situation, aims and functionality is not very clear, and the current shape of the API is very limiting 17:47:10 I agree 17:47:22 me 2 17:47:29 The code seems to be there for libvirt, but I haven't used it 17:48:13 should we ask Morita-san to come along? 17:49:48 should we bring this up on an email thread for visibility? 17:50:21 I think that is a good idea. I certainly find it difficult to understand exactly what is there 17:50:27 It would definitely be worth getting all of the interested parties involved, yes 17:50:44 DuncanT: Is there anything in particular you would like to see in the API for example? 17:51:29 renuka: I'd like to see an API for generically plugging volumes onto an instance/server before it is booted, and setting the boot device 17:51:51 Rather than limiting it to the ec2 boot-onto-a-cloned-copy only 17:52:20 DuncanT: there is a BootFromVolumeController in the openstack api 17:52:26 extensions 17:52:45 renuka: I saw it, it is not clear how it is supposed to work 17:53:24 #action bring up boot from volume on email thread to get an idea of current state/goals 17:53:46 For example, if I have vm1 create & prepare (fill) a volume, then detach I'd like to be able to provision a new vm that boots from that volume 17:53:54 (or volumes) 17:55:02 DuncanT: this exists as an OS api extension, but it could use some cleaning up 17:55:17 I think it would be good to consolidate the boot from volume *& the recent autocreatebootvoluems into a logical set of volume & boot capabilities 17:55:39 agreed 17:57:12 Right, we have about 3 minutes remaining. Anything else that needs to be brought up in the future? 17:57:34 i'm good 17:57:36 I can't think of anything at the moment 17:57:45 good here too 17:58:05 so I noticed we have the nova volume meeting set to happen every Thursday 17:58:19 just wanted to confirm that everyone feels the need for a weekly meeting 17:58:41 There have been things to discuss every week so far 17:59:04 I think at present it is a good timing,. As it happens I will be AWOL next week 17:59:21 ok. let us keep it that way 17:59:47 alright, thanks everyone. 17:59:51 ciao 18:00:02 #endmeeting