14:59:45 <n0ano> #startmeeting scheduler
14:59:46 <openstack> Meeting started Tue Jul 30 14:59:45 2013 UTC and is due to finish in 60 minutes.  The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:47 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:59:49 <openstack> The meeting name has been set to 'scheduler'
15:00:02 <n0ano> show of hands, anyone here for the scheduler meeting?
15:00:57 <jog0> o/
15:02:02 <n0ano> jog0, just you & me so far, this could be quick :-)
15:02:14 <jgallard> hi
15:03:15 <alaski> o/
15:03:54 <n0ano> well, while we're waiting for people...
15:04:00 <n0ano> #topic administrivia
15:04:34 <n0ano> I will be out the months of Sept & Oct (back in time for Hong Kong), anyone want to chair this meeting during those 2 months?
15:05:26 <jog0> n0ano: I would offer but, the time of day doesn't always work for me
15:05:57 <alaski> kind of the same for me.  I can take it sometimes, but I know I'll be out a few of those weeks.
15:06:15 <glikson> hi
15:06:16 <n0ano> we don't need an asnwer today, I just wanted to get it out and we'll find someone, I don't want the meeting to die on the vine.
15:06:38 <n0ano> moving on to more interesting stuff
15:06:45 <boris-42> hi all
15:06:46 <boris-42> =)
15:06:54 <n0ano> #topic multiple active scheduler policies/drivers
15:07:08 <jgallard> n0ano, to remember this point, can you "action" it?
15:07:12 <n0ano> glikson, jog0 I believe you've been active on the list on this, can you summarize?
15:07:47 <glikson> I can try
15:07:48 <n0ano> #action need new moderator for months of Sept & Oct
15:07:55 <n0ano> jgallard, tnx, good point
15:08:00 <jgallard> n0ano, thanks :)
15:08:31 <glikson> we have a blueprint and a patch submitted. there have been several concerns with the current design approach.
15:09:02 <glikson> seems that the main one was whether we need API-driven management of policies, or we can start with something more simple for Havana
15:10:09 <n0ano> especially given the short timeline remaining for Havana seems like starting simple with the option to make it more flexible in future would be the way to go
15:10:23 <n0ano> is the issue config files?
15:11:20 <jog0> simple isn't always good either.  we have to keep in mind that any 'simple' short term solution will have to have a migration path to the next thing  etc
15:11:31 <glikson> yes (I think). whether defining policies in nova.conf is good enough for Havana (similarly to cinder multi-backed support)
15:11:53 <jog0> IMHO, defining policies in nova.conf is not good enough for Havana
15:12:32 <glikson> jog0: agree, but I don't expect major issues with migration..
15:12:35 <n0ano> are they really going to be so dynamic that this is an unreasonable restriction?
15:12:48 <glikson> I don't expect them to be dynamic in most cases
15:13:06 <jog0> sure, they may not be dynamic but we don't want more config options in nova.conf
15:13:26 <glikson> adding policies typically means adding new kinds of hardware, or changing overall cloud management policies..
15:13:36 <n0ano> jog0, give the config file has well defined sections, what's wrong with putting them there
15:13:41 <jog0> adding this to nova.conf means making nova.conf even more complex
15:14:16 <jog0> n0ano: its not about the right section or not.  We want things to use the REST APIs when possible
15:14:38 <jgallard> jog0, +1
15:14:42 <glikson> in fact, adding APIs and DB support adds a lot to the overall complexity..
15:14:47 <n0ano> jog0, yes, APIs are the ultimate goal but is using the config file that much of a problem for Havana
15:15:12 <jog0> well I think this is missing a big piece, cross project policies
15:15:28 <jog0> such as cinder  and neutron related
15:15:40 <glikson> jog0: fully agree, but it shouldn't prevent us from solving an important subset of the problem in Havana
15:16:07 <jog0> I disagree, I would like to see a plan for a better solution, and how this gets us on the right track
15:16:58 <n0ano> seems to me we all want the APIs but think that's too complex a change for Havana
15:17:02 <jog0> If there is a roadmap for how to get to the better solution, and this is an temporary solution then sure.  Also what about russellb's idea of using flavors
15:17:25 <glikson> jog0: do you think we can come up with such a plan without a discussion at the summit?
15:17:38 <n0ano> maybe this is something that should be targeted for Icehouse and not Havana
15:17:44 <jog0> glikson: true
15:17:47 <glikson> jog0: I accepted his idea
15:17:59 <jog0> glikson: so using flavors sounds likea good first step
15:18:03 <glikson> the patch under review uses flavor extra specs
15:18:07 <jog0> its doesn't change things in any major way
15:18:23 <jog0> and addresses most of your basic requirments as far as I can tell
15:18:23 <glikson> the extra spec specifies the policy
15:19:17 <glikson> jog0: in theory, we can specify all the scheduler parameters (driver, filters, etc) in the flavor -- but I think it would be an abuse..
15:19:37 <glikson> also much harder in terms of compatibility and migration going forward
15:19:53 <jog0> glikson: yeah
15:20:20 <jgallard> glikson, yes, and complicated in term of flavors management and update too
15:20:25 <jog0> well I think there is a bigger issue at hand.  Which is there are several competing scheduler blueprints and no overall plan
15:21:15 <n0ano> jog0, that should be handled by a session at the next summit, trying to come up with a comprehensive plan
15:21:25 <glikson> jog0: IMO, the proposed design is flexible enough to seamlessly move to different mechanism for persistency and lifecycle of policies, when we know what exactly we are trying to achieve.
15:22:01 <jog0> glikson: the issue is when there are a bunch of BPs pulling the scheduler in different directions we end up with a mess
15:22:44 <n0ano> we kind of had that at the last summit but it just wound up being a quick review of multiple BPs, not an overall plan
15:22:46 <glikson> jog0: in general, I agree that some sort of overall architecture and roadmap would be useful
15:22:48 <jog0> yes, its too late to work on agree before the summit.  but its a good time to figure out the proposal for the summit
15:23:18 <shanewang1> the duedate for feature proposal is Aug 23?
15:23:21 <jog0> Some of the best summit sessions are the ones where there is a well thought out proposal that gets discussed
15:23:29 <jog0> shanewang1: no new BPs already
15:23:31 <glikson> jog0: I will surely vote for having a discussion on this at the summit
15:23:50 <jog0> glikson: a discussion over a fully proposed plan
15:23:55 <n0ano> let's not get too side tracked, we're diverting from multiple schedulers into overall scheduler design
15:24:22 <jog0> n0ano: sorry, I just think its hard to have this discussion without the overall one
15:24:24 <n0ano> both are good topics but let's finish the multiple scheduler discussion first
15:25:15 <n0ano> jog0, we're not going to finalize the overall picture today so I think it's still useful to discuss the specific issues
15:26:04 <n0ano> trust me, I'm adding a `overall scheduler plan' as a topic (most likely for next week when we have a chance to think about it).
15:26:22 <jog0> n0ano: no we aren't but we can start that discussion, perhaps schedule it off meeting
15:26:32 <glikson> jog0: the question is whether this particular BP, which has been discussed at high level at the last summit and approved, should be the one to "suffer" from this. It doesn't introduce any revolutionary changes, and does not seem to be contradicting any other major design goal that we may potentially want to enforce..
15:26:42 <n0ano> jog0, +1 (we're in violent agreement)
15:26:58 <jog0> glikson: can you link to the patch
15:27:18 <glikson> https://review.openstack.org/#/c/37407/
15:28:27 <jog0> glikson: I thought you said this used the flavors, not a list of policies
15:29:13 <glikson> jog0: flavor specifies the policy
15:29:40 <n0ano> glikson, then where are the actual policies defined
15:29:46 <glikson> the details of each policy is in nova.conf
15:30:17 <n0ano> ahh, now I see the issue, that does seem to be a bit of an abuse of the nova.conf
15:30:25 <jog0> n0ano: yup
15:30:43 <glikson> why?
15:31:06 <glikson> specify the default driver/filters is ok, but adding couple of extra ones is not?
15:31:07 <n0ano> you are now putting a potentially unbounded amount of info in the config file, not what it was defined for
15:31:55 <jog0> n0ano: we want things like policies to be controlled via APIs when possible
15:32:18 <jog0> this makes things like RBAC and  domains work nice
15:32:18 <glikson> n0ano: in theory..
15:32:22 <jgallard> hummm, it's not the same thing for instance when you configure several cinder backend into cinder.conf ? (multiple sections into cinder.conf)
15:32:37 <glikson> jgallard: indeed, it is
15:32:39 <jgallard> in that case, its not an issue
15:32:50 <jog0> jgallard: isn't that saying this machine has these backends
15:33:01 <jog0> and this is different per machine, and is only local.
15:33:02 <glikson> that's where we 'stolen' the idea
15:33:18 <jog0> while here we are saying I have these global policies and I am defining them in the nova.conf file
15:33:54 <jog0> where an admin API user cannot touch them even if the cloud operator wanted that
15:33:55 <n0ano> I'm with jog0 on this one (he's expressing my concerns better)
15:33:56 <glikson> jog0: not really -- you can potentially control all of your storage pools via a single machine running cinder-volume
15:34:57 <glikson> in fact, having a lot of storage pools, with dynamics, is a much more reallistic scenario than having lots of scheduling policies..
15:34:59 <jog0> glikson: sure, but that is an edge case of this?  and adding new hardware is different than changing a policy
15:36:01 <n0ano> note, just because cinder works this way doesn't mean we agree with that design, I'd say cinder is wrong in this case
15:36:15 <glikson> jog0: can you have an example, what kind of scheduling policies do you have in mind when thinking about having lots of them?
15:37:05 <jog0> glikson: who knows, but why can't I as a owner of a cloud say, you as a seperate domain and maybe a virtual private cloud inside of mine cannot set your own arbitrary scheduling policies
15:38:45 <glikson> jog0: you surely can, but it is not supported by the patch we've submitted. we support many other use-cases, which do not require this kind of programmability. we don't need to support all the possible policy-related use-cases in a single patch, right?
15:39:43 <n0ano> glikson, as long as you have a way of ultimately supporting all the use cases starting simple is fine
15:39:55 <glikson> I am not aware of any inherent problems with our implementation that would prevent anyone from extending it to support better progammability
15:39:57 <jog0> glikson: anyway, I think the next step here is:
15:40:26 <jog0> clearly explain your current patch in great detail, and push to the ML thread and see what happens
15:40:34 <glikson> on ther other hand, I am also not aware of any alternative implemetation that would solve the bigger problem in Havana time frame..
15:41:22 <jog0> glikson: this isn't always about the first implementation proposal wins, its about the 'right one,' however that is defined
15:41:34 <glikson> jog0: you mean, use cases or implementation?
15:41:44 <jog0> glikson: implementation
15:42:05 <jgallard> does this patch was not already discussed on the ML?
15:42:26 <jog0> jgallard: the patch has had several revisions since then, not sure if that has changed anything major
15:42:34 <glikson> there is no single "right one" in most cases. if this solves a real problem, and it seems to be flexible enough to extend to support additional use-cases -- this should be enough, IMO..
15:42:45 <jgallard> jog0, ok
15:43:00 <glikson> jog0: we addressed some of Russell's suggestions.
15:43:04 <hk_peter> it is a nova meeting or a ceilometer meeting?
15:43:12 <n0ano> hk_peter, nova
15:43:23 <russellb> glikson: needs to be re-reviewed now?
15:43:41 <hk_peter> @n0ano thx
15:43:59 <glikson> hi russel. yes, would appreciate if you could take another look (and maybe remove your -2)
15:44:14 <jog0> glikson: well for one thing the commit msg needs work
15:44:24 <russellb> ack, will look
15:44:53 <hk_peter> is there anybody can help to design the screen for nova's schduler setup?
15:44:53 <glikson> russellb: you may also take a look at the discussion we just had here, in the last 1/2 hour..
15:46:38 <glikson> jog0: ok
15:47:18 <n0ano> so, are we winding down on this subject (subject to mailing list activity coming up)?
15:48:03 <n0ano> taking silence as assent...
15:48:07 <n0ano> #topic opens
15:48:50 <n0ano> I wanted to talk about instance groups (but gary doesn't seem to be here) and/or simple way to improve the scheduler but I don't think there's enough time
15:48:57 <n0ano> we'll defer those to next week.
15:49:08 <n0ano> anyone have anything new to bring up toda?
15:49:09 <hk_peter> can i say something about my project pandora? i scare i am disturbing
15:49:38 <n0ano> hk_peter, is this scheduler related?
15:49:45 <hk_peter> kind of
15:49:57 <n0ano> go ahead
15:49:57 <hk_peter> but i really want to say something about pandora background first
15:50:04 <glikson> hk_peter, got a link?
15:50:27 <n0ano> hk_peter, I was going to say, is this something you should start on the dev mailing list first?
15:50:57 <hk_peter> pandora is a admin console http://peter.kingofcoders.com  , we are *NOT* forking horizon, we have the horizon API. We want to create a better GUI, so customer is easier to make the decision to purchase openstack
15:51:38 <jog0> hk_peter: is this opensource?
15:51:47 <hk_peter> yes, completely open source, apache license
15:52:00 <jog0> why not push code up into horizon?
15:52:09 <n0ano> hk_peter, sounds to me like you `are` trying to replace horizon, you'd better expect a certain amount of controversy
15:52:15 <hk_peter> two days ago, hong kong openstack community chairman want to submit our news to openstack blog, but refused, we are very down.
15:52:20 <n0ano> hk_peter, why not just enhance/change horizon
15:52:32 <hk_peter> I just want to say, pandora rely on horizon, we are not forking it.
15:52:42 <hk_peter> Pandora is java, hard to push code into horizon
15:53:01 <hk_peter> Pandora is not web base, i think our team can do better in a standalone app.
15:53:35 <jgallard> what is the link with scheduling?
15:53:36 <n0ano> hk_peter, do you have a detailed design you can point us to (I'm not liking what I'm hearing so far I have to tell you)
15:53:42 <hk_peter> Honestly, i still not sure how to submit code review to horizon team, we want to enhance the scheduler, give it a nice UI
15:54:23 <hk_peter> here you go http://peter.kingofcoders.com/?p=442
15:55:12 <n0ano> hk_peter, we don't have time to discuss this more today, I strongly suggest you raise this on the dev mailing list
15:55:21 <hk_peter> thanks n0ano
15:55:27 <n0ano> anything else new anyone want to raise?
15:55:30 <hk_peter> take you time
15:56:00 <jog0> hk_peter: the short of it is you can't adjust QOS scheduling today via any REST APIs
15:56:18 <jog0> well there are a few ways starting to show up, but we are discussing that here
15:57:09 <hk_peter> if we give it a REST APIs, letting people/third party program to reprogram the scheduler, isn't it great?
15:57:20 <jog0> hk_peter: see above
15:57:58 <n0ano> hk_peter, in a word, no, that's not necessarily a good idea
15:58:16 <n0ano> anyway, times up, so I'll thank everyone and we'll talk again next week.
15:58:27 <n0ano> #endmeeting