18:04:55 <renuka> #startmeeting
18:04:56 <openstack> Meeting started Thu Nov 17 18:04:55 2011 UTC.  The chair is renuka. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:04:57 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic.
18:05:08 <renuka> #topic openstack volume
18:05:36 <renuka> vladimir3p: did you have a chance to decide on whether you could implement the scheduler?
18:05:49 <vladimir3p> yes, I will do it
18:05:56 <vladimir3p> are there any other volunteers?
18:06:05 <renuka> did not hear back from anyone
18:06:20 <vladimir3p> :-)
18:06:57 <renuka> do you have an estimate of when you could start, how long it might take, etc?
18:07:32 <vladimir3p> we were quite busy with our internal tasks
18:07:44 <vladimir3p> most likely closer to the end of next week I will have time to start it
18:08:19 <vladimir3p> ah, it will be a thanksgiving
18:08:23 <vladimir3p> so, week after that
18:08:38 <renuka> right :) and how much work do you think it is?
18:08:51 <vladimir3p> IMHO, max couple of days :-)
18:09:18 <vladimir3p> I will actually need to extend the new distributed scheduler
18:09:25 <vladimir3p> to allow operations with volumes as well
18:10:13 <renuka> right, did you get a chance to look at the meeting minutes/logs for the affinity work?
18:10:28 <vladimir3p> sorry, had no chance to look at them
18:10:38 <vladimir3p> can you pls briefly summarize what you've decided?
18:10:47 <vladimir3p> or I can take a look later
18:11:01 <renuka> DuncanT: would you like to give a summary/update?
18:12:03 <renuka> Not sure if he is around, the blueprint is here: http://wiki.openstack.org/VolumeAffinity
18:12:08 <DuncanT> I've not had chance to start on anything yet, and I've not yet heard back on  a definitive answer on generic key/value .v. adding a new named field
18:13:09 <vladimir3p> I've seen this BP
18:13:16 <renuka> so i think we agreed that we should have, at create, a field for affinity and anti-affinity
18:13:19 <DuncanT> The basics of it are in the blueprint though - I was hoping to get vladimir's opinion on whether it is just a special case of general free-form constraints?
18:13:42 <renuka> which expresses the volumes/instances to which affinity/anti-affinity exists
18:14:01 <renuka> for example affinity: vol-0000001, i-000006
18:14:42 <vladimir3p> I agree that it is a special case. The question if we would like to address the afinity/anti-af in "general" way or just allow vendors to implement it by their own
18:15:10 <vladimir3p> I can imagine the discussion re: what afinitiy/anti-afinity rules may have higher priorities, etc
18:15:17 <DuncanT> renuka: That's certainly the the approach I'm assuming, but vish was going to check with the API folks as to whether a generic key-value pair system was appropriate instead since placement rules in general are hard to predict and we don't want to be adding ten million new fields
18:15:36 <vladimir3p> like if you indicate both affinity to vol and instance - which one is the preferred one
18:15:38 <DuncanT> vladimir3p: I'd assume it would be vendor specific, at least for a while
18:16:12 <vladimir3p> yeah. in the past I've suggested to add a special placement/scheduling requirements
18:16:28 <renuka> vladimir3p: why not order the affinity: <highest priority>  <next highest>....
18:16:33 <vladimir3p> but it didn't come through I suppose
18:16:53 <vladimir3p> renuka: yep, this is definitely an option
18:17:12 <vladimir3p> at the same time you may want to have a list of objects with the same priority
18:17:20 <renuka> DuncanT: for now I suggest we go with the extension that has the basic affinity/anti-affinity keywords
18:17:41 <vladimir3p> in volume_create request?
18:17:42 <renuka> this seems fairly well defined a problem, so I do not see the need for a generic field at this point
18:17:49 <vladimir3p> or in volume_type params?
18:18:00 <DuncanT> renuka: That's what I play on doing, it won't take long to change it if a different interface is desired
18:18:35 <DuncanT> *plan
18:18:48 <renuka> vladimir3p: It probably doesn't matter if non-conflicting things have the same priority and are expressed one after another
18:19:19 <renuka> if we do not do this, we will have a lot of complexity in the code to deal with "what happens if the user specifies conflicting goals to be at the same priority"
18:20:50 <vladimir3p> ok, anyway meanwhile we can plan to put an infrastructure in and allow other vendors to override it
18:21:20 <DuncanT> vladimir3p: Agreed
18:21:49 <renuka> #idea In volume create, affinity is expressed as affinity_to: <highest priority> <next highest>... etc
18:22:10 <renuka> vladimir3p: do you have an ideas about this infrastructure?
18:22:20 <vladimir3p> renuka: or anti-affinity
18:22:45 <vladimir3p> I propose to add those fields as "per request" and not in volume types
18:23:01 <vladimir3p> as we may need to specify for each volume (of the same type) where to put it
18:23:04 <DuncanT> vladimir3p: They absolutely are 'per-request'
18:23:04 <renuka> #idea same as above for anti-affinity
18:23:19 <DuncanT> anti-affinity exactly the same
18:23:24 <vladimir3p> DuncanT: yep
18:23:50 <vladimir3p> there were ideas to use special metadata keywords
18:23:55 <vladimir3p> this is an option as well...
18:24:18 <DuncanT> As far as whether the scheduler can do anything useful with them, I have no strong view - we have very little requirement from the scheduler
18:24:28 <vladimir3p> regardless of how we will deliver it - the idea is to be able on scheduler level to acces them
18:24:46 <DuncanT> meta-data keywords is basically the generic key-value idea
18:25:15 <renuka> vladimir3p: you mean keywords other than affinity and anti-affinity?
18:25:28 <vladimir3p> DuncanT: yes. Don't know if we want to reserve some keywords for that
18:25:37 <renuka> vladimir3p: or were you saying this in general, and not with respect to affinity alone
18:26:04 <vladimir3p> renuka: yes. For now we have no other keywords that we need
18:26:33 <vladimir3p> from our perspective we have following requirements for the scheduler:
18:26:44 <vladimir3p> 1. to be able to create multiple volumes in one call
18:26:56 <vladimir3p> 2. to be able on scheduler level to determine host abilities
18:27:10 <renuka> vladimir3p: I am usually wary of adding generic key value pairs to requests where behavior can be defined... the documentation and behavior becomes hard to keep up with it
18:27:27 <vladimir3p> 3. to be able to specify that particular volume should not be allocated from hosts where other volumes are allocated (anti-affinity)
18:27:41 <vladimir3p> renuka; agree
18:27:56 <DuncanT> renuka: The problem with that is it means all values must be defined, which removes vendor flexibiltiy
18:28:31 <renuka> DuncanT: at this stage, we are having vendors come up and state their requirements, so we should deal with it individually ...
18:28:53 <DuncanT> Allowing arbitary keywords, even with the proviso that they are only for advisory data, gives more flexibility
18:29:15 <renuka> DuncanT: what I mean is, where we need generic key-value pairs like at some point for scheduler rules, we can use them... but we should not use them before that point
18:31:15 <renuka> DuncanT: like i said, if these are scheduler rules, then maybe we should have a keyword "scheduler-rules" and add these as arbitrary strings that can be passed
18:31:51 <renuka> this will ensure that suddenly no one starts using the generic key value input for defining entirely different behavior
18:32:00 <DuncanT> In our case they aren't really scheduler rules though, they're just extra data to pass to the driver
18:32:16 <DuncanT> renuka: I'll think about what other examples I can come up with. It isn't something that needs a decision now
18:32:27 <renuka> agreed
18:33:34 <vladimir3p> so, seems like our options are: a. in request structure to have a special field like scheduler/placement requirement. b. put same keyword in meta-data. c. no special keywords - just put to meta-data whatever you want and driver later will look (or not look) at metadata
18:34:01 <renuka> vladimir3p: would it be alright to add an action for you to start the implementation of the scheduler first week of december?
18:34:13 <vladimir3p> yep, put it in
18:34:41 <renuka> #action vladimir3p to start scheduler implementation in the first week of december
18:36:37 <renuka> the other reason to go with known keywords as far as possible is to be able to provide usage strings and maybe at some point auto completion
18:38:17 <vladimir3p> ok guys, do we have anything else to discuss?
18:38:34 <DuncanT> I've nothing this week
18:38:49 <renuka> me neither
18:38:55 <renuka> should we call it a day then?
18:39:05 <vladimir3p> yep.
18:39:09 <DuncanT> Yup
18:39:13 <renuka> #endmeeting