14:00:16 <bauzas> #startmeeting nova_scheduler
14:00:17 <openstack> Meeting started Mon Nov  9 14:00:16 2015 UTC and is due to finish in 60 minutes.  The chair is bauzas. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:18 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:21 <openstack> The meeting name has been set to 'nova_scheduler'
14:00:28 <edleafe> o/
14:00:32 <bauzas> mornooning
14:00:48 <edleafe> Good UGT morning!
14:01:30 <bauzas> so n0ano is enjoying OPNFV Summit, I'm just holding the meeting :)
14:01:35 <jichen> o/
14:02:04 <edleafe> bauzas: thanks for running it. Didn't see Don's message until this morning
14:02:09 <bauzas> np
14:02:14 <edleafe> I have a nasty habit of not working on weekends :)
14:02:41 <bauzas> I don't have a precise agenda in mind so I guess it should be short
14:02:42 * johnthetubaguy lurks
14:03:34 <bauzas> lxsli: around for talking about scheduling bits?
14:04:16 <bauzas> that DST shift for both US and EU folks probably threw some people under the bus
14:04:17 <edleafe> if not, I wanted to discuss https://review.openstack.org/#/c/242636/
14:04:24 <bauzas> okay, let's start then
14:04:30 <bauzas> people can join lately
14:04:34 <johnthetubaguy> so I have some requests, but we can leave that to open
14:04:41 <bauzas> johnthetubaguy: surre
14:04:48 <edleafe> It simply makes the resource tracker class a config option
14:04:52 <bauzas> so, last week's meeting was pretty brief http://eavesdrop.openstack.org/meetings/nova_scheduler/2015/nova_scheduler.2015-11-02-14.01.log.html
14:04:55 <PaulMurray> o/
14:05:26 <bauzas> #topic summit
14:05:57 <bauzas> #link https://etherpad.openstack.org/p/mitaka-nova-resource-modeling
14:06:04 <bauzas> #link https://etherpad.openstack.org/p/mitaka-nova-scheduler
14:06:15 <bauzas> were the two sessions dedicated to the scheduler
14:06:31 <bauzas> given most of us were present at the summit, I won't speak about, unless someone steps in
14:06:40 <bauzas> just to make sure everyone is aware of the decisions we had
14:07:18 <bauzas> #info Nova-scheduler has been identified as a priority for Mitaka, in particular Resource modeling and usage
14:08:02 <bauzas> most of our specs are merged for Mitaka unless I'm wrong ?
14:08:28 <edleafe> there's still jaypipes' resource provider spec
14:08:36 <bauzas> oh right
14:08:42 <bauzas> that one isn't approved yet
14:08:45 <edleafe> which I'm having conceptual difficulties with
14:08:56 <edleafe> I'mm thinkning of providing a simpler spec
14:08:57 <bauzas> #link https://review.openstack.org/#/c/225546/
14:09:08 <edleafe> but it's not fully cooked in my head yet
14:09:17 <edleafe> probably by later today
14:09:19 <johnthetubaguy> the list in the etherpad for the scheduler seems very small: https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking
14:09:43 <johnthetubaguy> it just includes this blueprint: https://blueprints.launchpad.net/nova/+spec/configurable-resource-tracker
14:09:43 <bauzas> johnthetubaguy: fair point, we need a bit of summit mojo before going further
14:10:47 <johnthetubaguy> edleafe: I don't see that blueprint as part of the scheduler priority, at least as discussed at the summit?
14:10:49 <bauzas> #action bauzas to update https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking with https://review.openstack.org/#/c/225546/ and possibly others
14:11:07 <bauzas> it hasn't been discussed AFAIR
14:11:13 <edleafe> johnthetubaguy: it has to do with the distributed scheduler efforts
14:11:31 * alex_xu lurks
14:11:34 <johnthetubaguy> edleafe: didn't think that was a focus for mitaka, but thats maybe a different discussion
14:11:40 <edleafe> I spoke with a few people who had ideas on this, and it seemed that the RT was a sticking point for most
14:11:51 <johnthetubaguy> edleafe: feels like it needs a spec anyways
14:11:54 <edleafe> johnthetubaguy: agreed, not a focus
14:12:02 <bauzas> johnthetubaguy: yup, the priority was about providing resource objects to the scheduler, not more than that
14:12:09 <PaulMurray> bauzas, I'm going to have a question resource tracking, but I think its better for opens - would you remind me when we get there
14:12:17 <bauzas> okay, I feel we can move on
14:12:25 <johnthetubaguy> edleafe: OK, so lets catch up afterwards and work out how to get that reviewed
14:12:26 <bauzas> since we should have some questions of opens
14:12:30 <bauzas> #topic bugs
14:12:33 <edleafe> johnthetubaguy: ok, I can write a spec if you think it needs one
14:12:50 <bauzas> just to make sure people don't forget to lookup any open bug
14:13:01 <bauzas> https://bugs.launchpad.net/nova/+bugs?field.tag=scheduler is a bit high (30)
14:13:40 <bauzas> we should probably do some triage
14:14:29 <bauzas> I can handle that with markus_z
14:14:51 <bauzas> any open bug that people want to raise?
14:15:08 <bauzas> I don't see a recent one tho
14:15:41 <bauzas> ok, moving on
14:15:46 <bauzas> #topic opens
14:16:00 <bauzas> so, edleafe you had one spec to discuss
14:16:09 <bauzas> #link https://blueprints.launchpad.net/nova/+spec/configurable-resource-tracker
14:16:15 <bauzas> s/spec/blueprint
14:16:34 <edleafe> well, let's consider that on hold for now
14:16:42 <edleafe> it simply makes RT a config option
14:16:48 <edleafe> no change in default behavior
14:16:59 <bauzas> a config option like what ?
14:17:11 <bauzas> you mean a pluggable RT ?
14:17:19 <edleafe> which class to instantiate for the RT object
14:17:29 <edleafe> just like the scheduler, the host manager, etc
14:17:45 * jaypipes joins conversation...
14:17:48 <bauzas> mmm, I have to doublecheck but process-wise, I feel we wanted to avoid to have config opts pointing classes
14:18:08 <jaypipes> bauzas: ++
14:18:11 <bauzas> and use other stuff like stevedore rather
14:18:17 <johnthetubaguy> it would have to be stevedore or similar, I feel, not sure it makes sense otherwise
14:18:23 <johnthetubaguy> yeah ++
14:18:29 <bauzas> which concerns me a bit for RT, but that can be discussed in the spec
14:18:40 <edleafe> ok, understood
14:18:46 <edleafe> I was following the existing pattern
14:19:03 <bauzas> which is not the one we want to reproduce :D
14:19:16 <johnthetubaguy> edleafe: its one we are trying to destroy, if you could patch the scope doc to document that, it would be great
14:19:24 <bauzas> +1
14:19:35 <bauzas> that could go into the review doc
14:19:38 <johnthetubaguy> or anyone for that matter, that would be cool
14:19:55 <edleafe> ok
14:20:02 <bauzas> speaking of http://docs.openstack.org/developer/nova/code-review.html
14:20:03 <edleafe> I literally copy/pasted/edited https://github.com/openstack/nova/blob/master/nova/scheduler/driver.py#L29-L33
14:20:08 <bauzas> I know...
14:20:20 <bauzas> edleafe: but consider jaypipes's BP for metric monitors
14:20:31 <bauzas> edleafe: you'll see he used stevedore for that
14:20:47 <bauzas> sorry if that's undocumented, hence very good johnthetubaguy's point - that needs to be explained
14:20:55 <johnthetubaguy> we should fix that up, like we did for the virt drivers
14:21:03 <bauzas> but that's an implementation detail
14:21:13 <bauzas> johnthetubaguy: sounds a good low-hanging-fruit
14:21:38 <edleafe> so would a (low priority) change be to replace existing code like the one I referenced with stevedore-type imports?
14:21:45 <bauzas> #action bauzas to open a low-hanging-fruit for using stevedore with scheduler driver
14:21:53 <bauzas> edleafe: yeah that
14:21:58 <edleafe> ok, cool
14:21:59 <jaypipes> edleafe, bauzas, ndipanov, johnthetubaguy, lxsli: would appreciate another review round on https://review.openstack.org/#/c/225546/
14:22:19 <bauzas> I know that's not a bug, but that's the only way we have to log an easy feature people can do
14:22:28 <edleafe> jaypipes: been going over that a lot
14:22:49 <bauzas> jaypipes: yeah we covered that in the previous section, that's top prio
14:22:49 <edleafe> jaypipes: still seems like a huge amount of complexity for a small variation in behavior
14:23:09 <johnthetubaguy> so thats maybe a good segway to my question
14:23:26 <bauzas> johnthetubaguy: go for it
14:23:27 <jaypipes> edleafe: alternatives welcome.
14:23:31 <johnthetubaguy> we should make a list of our spec and blueprint review priorities in here: https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking
14:23:46 <bauzas> fair point
14:23:50 <johnthetubaguy> I can try make sure we get the most focus on those
14:24:04 <johnthetubaguy> the code still goes in the usual place here:
14:24:12 <johnthetubaguy> #link https://etherpad.openstack.org/p/mitaka-nova-priorities-tracking
14:24:18 <johnthetubaguy> #link https://etherpad.openstack.org/p/mitaka-nova-spec-review-tracking
14:24:30 <johnthetubaguy> just adding those into the meeting notes
14:24:44 <johnthetubaguy> we spoke about getting help for implementing some of these
14:24:52 <bauzas> the reqspec-object had very nice review traction thanks to dansmith and jaypipes but anyone can also contribute
14:24:53 <johnthetubaguy> once we have the overall direction agreed
14:25:01 <bauzas> johnthetubaguy: yup
14:25:19 <johnthetubaguy> just wondering how we want to handle that
14:25:23 <bauzas> johnthetubaguy: for the moment, the only way I see is to open wishlist bugs as placeholders for referencing a BP
14:25:27 <bauzas> it's another hop tho
14:25:37 <johnthetubaguy> thats not really worth it, IMHO
14:25:54 <bauzas> ideally, I'd love to see a low-hanging-fruit tag for blueprints
14:25:57 <johnthetubaguy> we do have an etherpad, #link https://wiki.openstack.org/wiki/Nova/Mentoring#What_should_I_work_on.3F
14:26:26 <johnthetubaguy> we can't add tags to blueprints, otherwise that would be great
14:26:28 <bauzas> oh snap, indeed
14:26:41 <johnthetubaguy> but this is more about stuff in the middle I think
14:26:44 <bauzas> there is the low-hanging-fruit etherpad, awesomeness
14:26:54 <johnthetubaguy> stuff we have a plan for, but folks who know Nova could chew on it
14:27:12 <bauzas> that should usually be balanced in those meetings IMHO
14:27:13 <johnthetubaguy> probably just starting with an ML list post, talking about good places to jump in an help would be good
14:27:31 <johnthetubaguy> asking folks to jump into this meeting, I guess is how the get started
14:27:36 <bauzas> yup
14:27:43 <johnthetubaguy> probably need to make that clearer
14:27:46 <johnthetubaguy> somehow
14:28:00 <johnthetubaguy> that was all for me really
14:28:14 <bauzas> okay, we just need to make sure people are aware of that
14:28:15 <johnthetubaguy> basically communication bits and bobs
14:28:36 <bauzas> anyone who want to step up on scheduling bits can walk into the meeting bar
14:29:17 <johnthetubaguy> a quick ML post I think is a good action
14:29:25 <johnthetubaguy> who fancies doing that?
14:29:30 <bauzas> I can handle that
14:29:30 <johnthetubaguy> bauzas?
14:29:44 <johnthetubaguy> bauzas: thank you
14:29:57 <bauzas> #action bauzas to write an ML thread about how to step up for the scheduler bigs
14:30:00 <bauzas> meh
14:30:02 <bauzas> #undo
14:30:03 <openstack> Removing item from minutes: <ircmeeting.items.Action object at 0xa376150>
14:30:07 <bauzas> #action bauzas to write an ML thread about how to step up for the scheduler bits
14:30:37 <bauzas> ok, are we taking the 30min left to discuss about jaypipes's resource-providers spec ?
14:30:46 <bauzas> or are we considering it's worth reviewing it rather ?
14:31:05 <PaulMurray> I had a question - its a pretty open one
14:31:12 <bauzas> PaulMurray: sure
14:31:28 <PaulMurray> now that I've removed the ERT
14:31:39 <PaulMurray> (or at least as soon as it gets through the gate)
14:31:42 <bauzas> s/removed/deprecated :)
14:31:57 <PaulMurray> I wanted to circle back on why it was there in the first place
14:32:12 <PaulMurray> We needed a way to deal with accounting cpus differently
14:32:33 <PaulMurray> If we created a resource representing cpus that was accounted differently
14:32:54 <PaulMurray> would that be seen as another new resource?
14:33:02 <PaulMurray> jaypipes, ^^ :)
14:33:20 <johnthetubaguy> PaulMurray: we should totally have got that problem specification into a backlog spec, I think
14:33:24 <bauzas> PaulMurray: you thinking of a new Resource class representing vcpus ?
14:33:45 <PaulMurray> maybe
14:33:52 <johnthetubaguy> PaulMurray: this is a consistent way of handing out CPU resources across CPUs of different speed/capacities?
14:33:57 <PaulMurray> there was also a problem with ironic htat could be addressed this way
14:34:23 <PaulMurray> ironic takes all memory for example - not just deduct it
14:34:24 <jaypipes> PaulMurray: it's a vCPU. it's not a different resource class.
14:34:48 <bauzas> PaulMurray: that's done in the IronicHostManager, right?
14:35:01 <jaypipes> PaulMurray: Ironic's issues are dealt with using the min_unit and max_unit fields in the resource-providers spec.
14:35:06 <bauzas> PaulMurray: and then the virt driver is reporting the real resource usage
14:35:37 <PaulMurray> jaypipes, I'll need to read that
14:36:00 <jaypipes> PaulMurray: and yes, the resource reporting problem is handled in the ironic-multi-compute spec, too (temporarily until resource-providers modeling solves the issue once and for all)
14:36:29 <PaulMurray> jaypipes, but for cpu example, its not quite vcpu - because we would always say how many vcpu
14:36:56 <PaulMurray> jaypipes, its about what amount of time gets allocated to the vm vcpu
14:37:19 <PaulMurray> we wanted to give an idea of performance (like 1GHz worth for example)
14:37:40 <PaulMurray> if I want 4 vcpu each worth about 1GHz
14:38:10 <PaulMurray> how could I account taking 1GHz worth from my available resources given each has say 2GHz worth of performance
14:38:56 <jaypipes> PaulMurray: that isn't quantitative. that is qualitative, and thus is capabilities, not resoruces.
14:39:19 <PaulMurray> sorry - I disagree
14:39:33 <PaulMurray> same applies for bandwidth on nics
14:39:45 <edleafe> PaulMurray: So can you divide one 2Ghz vcpu into 2 1Ghz vcpus?
14:39:58 <PaulMurray> edleafe, yes
14:40:11 <edleafe> Or 10 0.2 Ghz vcpus?
14:40:12 <PaulMurray> we do it in HP public cloud - always have done
14:40:13 <edleafe> Or...
14:40:28 <PaulMurray> edleafe, you get the idea
14:40:40 <edleafe> IOW, is it discreet chunks, or infinitely divisible?
14:40:45 <jaypipes> erm, you can't divide one 2ghz CPU into 2 1Ghz CPUs.
14:41:00 <PaulMurray> jaypipes, no, but you give them time slices
14:41:19 <bauzas> I feel it's more about how to provide a quantity over time than just a finite quantity
14:41:22 <PaulMurray> yo can certainly run 2 vcpu on one cpu
14:41:29 <bauzas> yeah that
14:41:36 <bauzas> so, like concurrency
14:41:40 <jaypipes> PaulMurray: that is oversubscription.
14:41:50 <bauzas> I feel it's not exactly that
14:41:51 <PaulMurray> jaypipes, not if you give half each
14:41:52 <jaypipes> PaulMurray: has nothing to do with dividing Ghz.
14:41:53 <johnthetubaguy> its basically varying the over commit to try and make the performance feel the same across different CPUs running at different clock speeds right?
14:42:08 <jaypipes> it's cgroups limits.
14:42:09 <johnthetubaguy> so trying to have CPU agnosic measure for CPU
14:42:10 <jaypipes> that's all.
14:42:39 <jaypipes> johnthetubaguy: that is normalized compute units.
14:42:52 <johnthetubaguy> jaypipes: I think thats what they are wanting here
14:42:54 <jaypipes> https://review.openstack.org/#/c/192609/
14:43:15 <johnthetubaguy> yeah, that sounds similar to me
14:43:19 <jaypipes> johnthetubaguy: sure, but it has nothing to do with "accounting for vCPUs differently"
14:43:49 <johnthetubaguy> well, it does affect the overcommit rations and things, right?
14:44:04 <jaypipes> no
14:44:26 <johnthetubaguy> so you want the flavor to save 6 units, across 2 vCPUs I guess?
14:44:34 <PaulMurray> jaypipes, there is a total amount of time slices per core - you can provide a given performance so long as you don't over commit those time slices
14:44:38 <johnthetubaguy> PaulMurray: am I miss interpreting your request here?
14:44:48 <jaypipes> PaulMurray: and that would be a new resource class.
14:45:00 <PaulMurray> jaypipes, that's cool by me
14:45:02 <jaypipes> PaulMurray: it's not "multiple ways of accounting for vCPU"
14:45:08 <jaypipes> PaulMurray: cool.
14:45:11 <PaulMurray> jaypipes, agreed
14:45:14 <johnthetubaguy> right, that makes sense to me
14:45:29 <jaypipes> vCPU == number of threads of qemu-kvm process for the VM.
14:45:40 <jaypipes> CPU shares is a different resource class altogether.
14:45:46 <PaulMurray> absolutely
14:45:54 <jaypipes> cool, we're on the same page.
14:45:54 <bauzas> I see
14:45:57 <johnthetubaguy> yup, makes sense
14:46:19 <jaypipes> PaulMurray: once the resource-object models initial patches are in, let's add a new resource class for CPU shares (or units, or whatevs)
14:46:37 <PaulMurray> jaypipes, so if we made a resource for that - would they both always be there or is it up to the operator to chose which are used?
14:47:03 <johnthetubaguy> PaulMurray: I think you need both right? unless you always have 1 vCPU exposed to the guest or something?
14:47:16 <jaypipes> PaulMurray: the CPU units resource could not always be there, because it depends on code running on nova-compute startup that would run a normalization thread to determine the supported unit for CPU.
14:47:33 <jaypipes> PaulMurray: kinda like NUMA resources, right?
14:47:42 <johnthetubaguy> jaypipes: yeah, +1 on it being optional that way
14:48:02 <PaulMurray> jaypipes, yes, so are resources selectable - or does this come up as a None in some field?
14:48:06 <jaypipes> PaulMurray: so we'd need to have an addition to the virt driver's get_available_resource() command that would return information about the available/capacity of CPU units for that hardware
14:48:27 <bauzas> the problem is that resource-objects doesn't take care of persistence
14:48:42 <jaypipes> PaulMurray: it would not be a None field, but rather a lack of a record in the inventories table for that compute node and the "CPU units" resource class.
14:48:46 <bauzas> so, whatever a new Resource is created, it needs to come up with a plan on how to persist it
14:48:49 <jaypipes> PaulMurray: make more sense now?
14:49:05 <bauzas> oh, because of resource-providers then
14:49:09 <jaypipes> rigt
14:49:30 <jaypipes> no more adding fields to compute_nodes. instead, we'd be adding records to the inventories table.
14:49:40 <PaulMurray> I need to read resource-providers (I did - but I need to again)
14:49:52 <bauzas> so that not only requires resource-objects but resource-providers
14:49:59 <PaulMurray> jaypipes, sounds like your doing everything I always wanted in ERT !
14:50:14 <jaypipes> PaulMurray: sure, no worries. I just want you to know I've considered the vCPU accounting use case from HP Cloud in my thinking on the resource-objects and resource-providers stuff.
14:50:26 <PaulMurray> cool
14:50:28 <bauzas> okay, I feel we need to all review resource-providers
14:50:44 <bauzas> (although I already did...)
14:50:46 <jaypipes> bauzas: actually, gimme about an hour more. pushing more changes to it.
14:50:54 <bauzas> jaypipes: good to know, ty
14:51:00 <jaypipes> lxsli, PaulMurray, edleafe, johnthetubaguy ^^
14:51:08 <edleafe> jaypipes: gotcha
14:51:11 <johnthetubaguy> ack
14:51:15 <jaypipes> I forgot to rework the biolerplate shit.
14:51:54 <bauzas> I felt we basically had an agreement on resource-providers at the summit, so that should be straightforward
14:52:09 <jaypipes> bauzas: not really :)
14:52:18 <jaypipes> bauzas: edleafe had alternative proposal.
14:52:18 <bauzas> jaypipes: well, I remember some implementation details
14:52:29 <PaulMurray> did I go to the same summit?
14:52:35 <jaypipes> bauzas: which I believe lxsli also had negative feedback
14:52:36 <bauzas> jaypipes: but I'd like to see the spec approved, and then us discussing about the implementation bits
14:52:40 <jaypipes> PaulMurray: yes :)
14:52:56 <jaypipes> bauzas: erm, I'd rather have more discussion on the spec right now.
14:53:03 <bauzas> to be clear, yeah we all had questions about how to model that, but is that really spec-wise?
14:53:07 <jaypipes> bauzas: because this is a foundational change...
14:53:10 <bauzas> jaypipes: fair enough then
14:53:19 <bauzas> jaypipes: you mean the data model impact?
14:53:26 <edleafe> yeah, working oit out on the spec seems better
14:53:55 <bauzas> jaypipes: okay, then we all know what to do
14:54:09 <bauzas> 6 mins left, anything else to discuss ?
14:55:02 <bauzas> crickets
14:55:14 <bauzas> okay, lemme save you 5 mins of your lifetime
14:55:19 <bauzas> #endmeeting