18:04:55 #startmeeting 18:04:56 Meeting started Thu Nov 17 18:04:55 2011 UTC. The chair is renuka. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:04:57 Useful Commands: #action #agreed #help #info #idea #link #topic. 18:05:08 #topic openstack volume 18:05:36 vladimir3p: did you have a chance to decide on whether you could implement the scheduler? 18:05:49 yes, I will do it 18:05:56 are there any other volunteers? 18:06:05 did not hear back from anyone 18:06:20 :-) 18:06:57 do you have an estimate of when you could start, how long it might take, etc? 18:07:32 we were quite busy with our internal tasks 18:07:44 most likely closer to the end of next week I will have time to start it 18:08:19 ah, it will be a thanksgiving 18:08:23 so, week after that 18:08:38 right :) and how much work do you think it is? 18:08:51 IMHO, max couple of days :-) 18:09:18 I will actually need to extend the new distributed scheduler 18:09:25 to allow operations with volumes as well 18:10:13 right, did you get a chance to look at the meeting minutes/logs for the affinity work? 18:10:28 sorry, had no chance to look at them 18:10:38 can you pls briefly summarize what you've decided? 18:10:47 or I can take a look later 18:11:01 DuncanT: would you like to give a summary/update? 18:12:03 Not sure if he is around, the blueprint is here: http://wiki.openstack.org/VolumeAffinity 18:12:08 I've not had chance to start on anything yet, and I've not yet heard back on a definitive answer on generic key/value .v. adding a new named field 18:13:09 I've seen this BP 18:13:16 so i think we agreed that we should have, at create, a field for affinity and anti-affinity 18:13:19 The basics of it are in the blueprint though - I was hoping to get vladimir's opinion on whether it is just a special case of general free-form constraints? 18:13:42 which expresses the volumes/instances to which affinity/anti-affinity exists 18:14:01 for example affinity: vol-0000001, i-000006 18:14:42 I agree that it is a special case. The question if we would like to address the afinity/anti-af in "general" way or just allow vendors to implement it by their own 18:15:10 I can imagine the discussion re: what afinitiy/anti-afinity rules may have higher priorities, etc 18:15:17 renuka: That's certainly the the approach I'm assuming, but vish was going to check with the API folks as to whether a generic key-value pair system was appropriate instead since placement rules in general are hard to predict and we don't want to be adding ten million new fields 18:15:36 like if you indicate both affinity to vol and instance - which one is the preferred one 18:15:38 vladimir3p: I'd assume it would be vendor specific, at least for a while 18:16:12 yeah. in the past I've suggested to add a special placement/scheduling requirements 18:16:28 vladimir3p: why not order the affinity: .... 18:16:33 but it didn't come through I suppose 18:16:53 renuka: yep, this is definitely an option 18:17:12 at the same time you may want to have a list of objects with the same priority 18:17:20 DuncanT: for now I suggest we go with the extension that has the basic affinity/anti-affinity keywords 18:17:41 in volume_create request? 18:17:42 this seems fairly well defined a problem, so I do not see the need for a generic field at this point 18:17:49 or in volume_type params? 18:18:00 renuka: That's what I play on doing, it won't take long to change it if a different interface is desired 18:18:35 *plan 18:18:48 vladimir3p: It probably doesn't matter if non-conflicting things have the same priority and are expressed one after another 18:19:19 if we do not do this, we will have a lot of complexity in the code to deal with "what happens if the user specifies conflicting goals to be at the same priority" 18:20:50 ok, anyway meanwhile we can plan to put an infrastructure in and allow other vendors to override it 18:21:20 vladimir3p: Agreed 18:21:49 #idea In volume create, affinity is expressed as affinity_to: ... etc 18:22:10 vladimir3p: do you have an ideas about this infrastructure? 18:22:20 renuka: or anti-affinity 18:22:45 I propose to add those fields as "per request" and not in volume types 18:23:01 as we may need to specify for each volume (of the same type) where to put it 18:23:04 vladimir3p: They absolutely are 'per-request' 18:23:04 #idea same as above for anti-affinity 18:23:19 anti-affinity exactly the same 18:23:24 DuncanT: yep 18:23:50 there were ideas to use special metadata keywords 18:23:55 this is an option as well... 18:24:18 As far as whether the scheduler can do anything useful with them, I have no strong view - we have very little requirement from the scheduler 18:24:28 regardless of how we will deliver it - the idea is to be able on scheduler level to acces them 18:24:46 meta-data keywords is basically the generic key-value idea 18:25:15 vladimir3p: you mean keywords other than affinity and anti-affinity? 18:25:28 DuncanT: yes. Don't know if we want to reserve some keywords for that 18:25:37 vladimir3p: or were you saying this in general, and not with respect to affinity alone 18:26:04 renuka: yes. For now we have no other keywords that we need 18:26:33 from our perspective we have following requirements for the scheduler: 18:26:44 1. to be able to create multiple volumes in one call 18:26:56 2. to be able on scheduler level to determine host abilities 18:27:10 vladimir3p: I am usually wary of adding generic key value pairs to requests where behavior can be defined... the documentation and behavior becomes hard to keep up with it 18:27:27 3. to be able to specify that particular volume should not be allocated from hosts where other volumes are allocated (anti-affinity) 18:27:41 renuka; agree 18:27:56 renuka: The problem with that is it means all values must be defined, which removes vendor flexibiltiy 18:28:31 DuncanT: at this stage, we are having vendors come up and state their requirements, so we should deal with it individually ... 18:28:53 Allowing arbitary keywords, even with the proviso that they are only for advisory data, gives more flexibility 18:29:15 DuncanT: what I mean is, where we need generic key-value pairs like at some point for scheduler rules, we can use them... but we should not use them before that point 18:31:15 DuncanT: like i said, if these are scheduler rules, then maybe we should have a keyword "scheduler-rules" and add these as arbitrary strings that can be passed 18:31:51 this will ensure that suddenly no one starts using the generic key value input for defining entirely different behavior 18:32:00 In our case they aren't really scheduler rules though, they're just extra data to pass to the driver 18:32:16 renuka: I'll think about what other examples I can come up with. It isn't something that needs a decision now 18:32:27 agreed 18:33:34 so, seems like our options are: a. in request structure to have a special field like scheduler/placement requirement. b. put same keyword in meta-data. c. no special keywords - just put to meta-data whatever you want and driver later will look (or not look) at metadata 18:34:01 vladimir3p: would it be alright to add an action for you to start the implementation of the scheduler first week of december? 18:34:13 yep, put it in 18:34:41 #action vladimir3p to start scheduler implementation in the first week of december 18:36:37 the other reason to go with known keywords as far as possible is to be able to provide usage strings and maybe at some point auto completion 18:38:17 ok guys, do we have anything else to discuss? 18:38:34 I've nothing this week 18:38:49 me neither 18:38:55 should we call it a day then? 18:39:05 yep. 18:39:09 Yup 18:39:13 #endmeeting