15:00:19 <n0ano> #startmeeting gantt
15:00:20 <openstack> Meeting started Tue Jun  2 15:00:19 2015 UTC and is due to finish in 60 minutes.  The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:24 <openstack> The meeting name has been set to 'gantt'
15:00:28 <n0ano> anyone here to talk about the scheduler?
15:00:28 <edleafe> o/
15:00:38 <watanabe_isao> mestery, amotoki_ , night night sir. ZZZzzz�
15:00:40 <bauzas> \o
15:01:25 <lxsli> o/
15:01:30 <lxsli> I have a clash so won't be listening hard
15:01:47 * n0ano ignore my groans, I think I cracked a rib snorkeling :-(
15:01:49 <edleafe> yeah, I have a phone call now, too, so I may be slower than usual
15:02:05 <edleafe> n0ano: ouch
15:02:22 <PaulMurray> hi
15:02:32 <n0ano> edleafe, lxsli NP, I know where all the ARs are going...
15:02:40 <n0ano> anyway, let's start
15:02:54 <n0ano> #topic Vancouver recap
15:03:15 <n0ano> I hope everyone survied the return trip and we're all back hard at work.
15:03:54 <n0ano> I think the main take away I got was that we are pretty much staying the course, cleaning up the sched interfaces and working on the know specs/BPs
15:03:58 <PaulMurray> I'm hardly working if that counts
15:04:25 <lxsli> badoom tish
15:04:34 <n0ano> anyone want to expand upon that?
15:05:02 <bauzas> currently in another discussion about the reqspec in #openstack-nova so feel free to continue :)
15:05:37 <n0ano> bauzas, I just saw that, are you coming to any conclusions in that discusssion?
15:06:03 <bauzas> n0ano: not yet :)
15:06:24 <n0ano> no surprise but it's all good.
15:07:20 <n0ano> we don't have to go into all the gory details here but I want to update my liberty wiki to track all of our specs/BPs, I just got back and I'm dealing with computer issues so I'll bug everyone to update that later.
15:08:10 <edleafe> n0ano: I posted the BP for the NoValidHost reporting: https://blueprints.launchpad.net/nova/+spec/no-valid-host-reporting
15:08:21 <edleafe> n0ano: The spec should be up later today
15:09:19 <n0ano> cool, I'll look for it and hopefully comment
15:10:07 <n0ano> if nothing else on Vancouver let's move on
15:10:19 <n0ano> #topic updating concept of resource
15:10:55 <n0ano> edleafe, I read your email & blog and I think you have some good ideas I just don't want to throw the baby out with the bath water
15:10:58 <edleafe> I wrote a blog post and then followed it up with the ML post yesterday
15:11:23 <n0ano> #link http://blog.leafe.com/rethinking-resources/
15:11:36 <edleafe> A resource in OpenStack is more than a server to be carved up into VPS-like chuncks
15:11:39 <edleafe> chunks
15:12:09 <n0ano> well, you could argue that 100% is a chunk and that would deal with the all or nothing resources
15:12:34 <edleafe> not really
15:12:59 <edleafe> we have lots of things that don't split up on RAM or disk or CPU
15:13:20 <edleafe> we shouldn't be considering that as the model for all resources
15:14:26 <n0ano> but dealing with multiple models could really complicate things and I'd like to be sure there's a definite benefit to doing that
15:14:44 <edleafe> n0ano: that's where a ResourceObject comes in
15:15:05 <edleafe> It should encapsulate the way each type of resource is handled
15:15:48 <n0ano> are you seeing ram as one type of resource object and numa topology as a different type of resource object?
15:15:57 <edleafe> n0ano: no
15:16:09 <edleafe> a resource is something that we need to allocate
15:16:46 <edleafe> the compute resources that flavors address are the traditional type of resource that openstack pretty much assumes is the *only* type of resource
15:17:09 <edleafe> as a result, we're cramming other things, such as NUMA and PCI, into the flavor-based design
15:17:14 <edleafe> that's a hack, IMO
15:18:11 <edleafe> And if you look ahead to things such as container-based hypervisors (such as LXD), the model totally falls apart
15:18:18 <n0ano> using extra_specs to define things is arguably a hack but encapsulating everything in a flavor is OK to me, you have to encapsulate this info somehow
15:18:36 <lxsli> I'd like to make it a Nova problem not a scheduler problem
15:18:51 <edleafe> n0ano: then every possible permutation of specs is a distinct flavor
15:19:14 <n0ano> lxsli, I would too but I don't think we can, I think the sched needs to be aware of this info
15:19:16 <lxsli> let the scheduler just deal with resource requests
15:19:43 <lxsli> n0ano: the API can translate "boot flavor=X" into "claim resources=[...]"
15:20:35 <edleafe> lxsli: yes, but that still requires that we clean up how we think of resources
15:21:01 <n0ano> we still need to remember the cloud provider vs. the user, users ask to boot `something` while the provider defines what is available
15:21:08 <lxsli> edleafe: can you expand on that?
15:21:17 <PaulMurray> edleafe I don't think that's disputed is it - isn't that what we are looking to do?
15:21:31 <lxsli> the provider also defines what the user is allowed to ask for
15:21:32 <edleafe> PaulMurray: not in the way that I'm  thinkning
15:21:43 <edleafe> we're still insisting that "everything is a flavor"
15:21:55 <n0ano> lxsli, that's my poing and that definition is done by flavors right now
15:21:59 <lxsli> imo flavors are to prevent the user asking for weird-shaped chunks, they're nothing to do with the core job of scheduling
15:22:04 <n0ano> s/poing/point
15:22:14 <edleafe> lxsli: if you move 'flavor' to the API, and leave it in the scheduler, what's the gain?
15:22:26 <lxsli> edleafe: nope, remove it totally from scheduler
15:22:35 <lxsli> so scheduler just deals with resources
15:22:39 <edleafe> lxsli: yeah, I covered the weird chunks in my blog post
15:22:55 <PaulMurray> edleafe, I think at some point a user wants to say I want an X
15:23:14 <edleafe> PaulMurray: yes
15:23:16 <PaulMurray> as opposed to an {a,b,c,d,e,f,g,h,... oh did I forget something}
15:23:18 <lxsli> I'm agreeing with your problems, just suggesting rather than eliminating flavors entirely we shove the concept out to the API
15:23:31 <PaulMurray> so I think we all agree on the resource point
15:23:36 <n0ano> PaulMurray, +1
15:23:39 <edleafe> PaulMurray: but they'll say I want an X with this network config and this NUMA topology and ...
15:24:01 <edleafe> PaulMurray: and thus the provider will have to offer a flavor for every possible combination
15:24:14 <PaulMurray> edleafe, yes and no
15:24:18 <edleafe> lxsli: no, don't eliminate flavors
15:24:26 <PaulMurray> edleafe, they may chose to support only a certain combination
15:24:33 <lxsli> if we constrain the scope of flavor code, potentially it becomes pluggable so operators can decide what request validation works for them
15:24:35 <PaulMurray> edleafe, but they may allow you to make up combinations
15:24:40 <edleafe> lxsli: just apply them to resources that are divisible (i.e., like VPS instances)
15:25:15 <edleafe> PaulMurray: anything that the user might possibly pick is a defined flavor
15:26:05 <edleafe> lxsli: flavors would only apply to ResourceObjects that follow the "traditional" computing model
15:26:41 <edleafe> i.e., things that can be sliced up should have a sane way of limiting the slicing
15:26:51 <n0ano> edleafe, then how does a user as for, say, a specific SR/IOV device?
15:27:05 <n0ano> s/as for/ask for
15:27:20 <edleafe> n0ano: that's an additional requirement for the request
15:27:34 <edleafe> it would be evaluated separately from the flavor constraint
15:27:44 <edleafe> it would *not* be part of the flavor
15:27:59 <n0ano> and how does the provider control what a additional requirements are available
15:28:41 <edleafe> n0ano: that would go back to lxsli's point of enforcing that at the API level
15:28:48 <edleafe> that's not a Scheduler decision
15:29:20 <lxsli> edleafe: so you agree that the scheduler API should not know about flavors? Just resources?
15:29:42 <edleafe> lxsli: a flavor is simply one type of request constraint
15:29:46 <n0ano> well, it's never really been a sched decision, we're done that implicitly in the definition of the flavors
15:30:17 <edleafe> n0ano: but look at the mess we have created by trying to jam everything into a flavor
15:30:25 <lxsli> the scheduler looks at flavors today though, rather than the flavor having been fully unpacked before the request gets to the scheduler
15:30:37 <edleafe> the cloud technology choices are only going to expand in the future
15:31:40 <n0ano> I think i'm beginning to agree with lxsli have nova decompose flavor (or whatever) and only have the scheduler see resource requests
15:32:07 <edleafe> n0ano: yes, that would make things a lot cleaner
15:32:22 <lxsli> \o/
15:32:30 <edleafe> it would also eliminate the current issue with cells not getting defined flavors replicated to each cell
15:32:49 <edleafe> since cells would only get the resource request
15:33:14 <n0ano> and would also make the sched less nova centric
15:33:45 <edleafe> What I'd like to get you all thinking about is the notion that there are different types of Resources
15:34:05 <edleafe> and each different type can have different request constraints
15:34:25 <edleafe> n0ano: this would also move things to making the scheduler not Nova-centric
15:34:42 <lxsli> In my rewrite of Jay's spec, I'm proposing Integer, NUMA and PCI ResourceTypes each with their own matching and consumption logic
15:34:45 <edleafe> since other services could define their own resource classes and request constraints
15:35:15 <n0ano> edleafe, but I still want to allow a user to request a provider defined instance, we just need to expand the definition of a flavor maybe
15:35:18 <edleafe> lxsli: how is an Integer a resource?
15:35:27 <edleafe> sounds more like a constraint type
15:35:40 <lxsli> it's not, it's a resourcetype - eg vcpus is an integer resource, so is memory
15:36:01 <edleafe> lxsli: ah, we're overloading terms
15:36:03 <lxsli> both of those are matched (a <= b) and consumed (a -= b) the same
15:36:25 <lxsli> edleafe: yes naming is hard >.<
15:36:39 <edleafe> I was using resource type to mean the type of resource being requested: compute, network, bare metal, etc.
15:36:50 <lxsli> I'm calling a Resource an amount, with a type, optionally from a ResourcePool (IE a compute host)
15:37:01 <edleafe> I was calling the things like vcpus a constraint
15:37:04 <n0ano> edleafe, call yours `class` and we should be good
15:37:44 <lxsli> maybe if we called compute, network etc rsource cells we could make some heads explode? :)
15:38:03 <n0ano> lxsli, don't even go there :-)
15:38:04 <edleafe> lxsli: +1  lol
15:38:25 <edleafe> no, "resource zones"
15:39:16 <lxsli> anyway... it sounds like we have broad agreement that flavors aren't needed in scheduler and they make it nova-centric? any volunteers to spec that, edleafe?
15:39:20 <n0ano> edleafe, back to your classes, note that compute encompasses multiple things, some enumeratable (RAM), some not (NUMA topology), how dos that fit?
15:39:24 <edleafe> PaulMurray: to get back to the provider view, they could say "we offer compute instances in the following sizes: ..."
15:39:38 <edleafe> PaulMurray: "and you can add these options: ..."
15:39:49 <n0ano> lxsli, me, I'm going to see if I can write a spec on moving flavors out of the scheduler
15:39:54 <lxsli> n0ano: awesome
15:40:07 <lxsli> btw I was chatting with claudiub about multiple flavours at the summit
15:40:08 <n0ano> lxsli, say that `after` I've written something
15:40:31 <edleafe> n0ano: those different constraints would be handled the way the lxsli was talking about in jaypipe's spec
15:40:36 <lxsli> it might be quite nice to be able to have "VPS" flavors, "SRIOV" flavors and to be able to pick one of each
15:41:02 <edleafe> lxsli: yes, that's sort of what I had in mind
15:41:11 <edleafe> different types of resources
15:41:23 <edleafe> although the name 'flavor' should DIAFF
15:41:28 <lxsli> hehe
15:41:30 <n0ano> treating impossible combinations of multiple flavors as a user problem?
15:41:34 <claudiub> o/
15:41:37 <lxsli> treating it as a validation problem
15:42:02 <lxsli> a good start might be "at most one flavor per resource"
15:42:32 <n0ano> still a user problem, I predict the user will get a very informative `boot request failed'
15:42:52 <edleafe> n0ano: No, they should get back an API validation error
15:43:00 <lxsli> edleafe: +1
15:43:24 <lxsli> that can easily be very informative as it doesn't have to get copied through multiple services
15:44:01 <edleafe> lxsli: it could also be customizable middleware that different vendors could set up for their own products
15:44:10 <n0ano> still a little worried that a user asks for something that he thinks should work and it doesn't but that's probably Ok
15:44:11 <lxsli> edleafe: +1
15:44:13 <bauzas> n0ano: since I'm fully uberloaded by the discussion on the chan, could you please ping me when you're at the open topic ?
15:44:34 <n0ano> bauzas, sure, probably soon
15:45:34 <n0ano> edleafe, good discussion, do you want to maybe update your blog with some of the results from today?
15:45:42 <edleafe> n0ano: If they're using the web interface, there isn't any problem. If they're using the command line, they're probably use to deciphering messages
15:46:16 <edleafe> n0ano: I would prefer that people respond on the ML instead - more eyes than my puny blog :)
15:46:30 <edleafe> I'll certainly post a reply
15:46:41 <edleafe> you all can argue with that
15:46:44 <edleafe> :)
15:46:49 <n0ano> edleafe, reasonable, a reply to that would be good
15:47:10 <n0ano> edleafe, cool
15:47:15 <n0ano> moving on...
15:47:22 <n0ano> #topic opens
15:47:26 <n0ano> bauzas, ping
15:47:26 <lxsli> edleafe: I responded and no one replied v.v
15:47:56 <n0ano> lxsli, I intend to reply today (traveling yesterday)
15:47:56 <edleafe> lxsli: I saw it but waited for this meeting
15:48:03 <lxsli> yay <3
15:48:08 <edleafe> n0ano: two things
15:48:20 <edleafe> one, jaypipes has a standing meeting at this time
15:48:30 <edleafe> he requested that we move this meeting if possible
15:49:04 <edleafe> second, can we remove the 'gantt' stuff from the meeting topic?
15:49:10 <bauzas> n0ano: yea
15:49:14 <lxsli> I can do 1-2 hours later or up to 7 hours earlier
15:49:25 <n0ano> sigh, I can try, finding an open slot that works for everyone can be difficult
15:49:39 <bauzas> n0ano: so I was about discussing about 2 points, but edleafe hijacked me
15:49:50 <bauzas> n0ano: doodle
15:49:50 <edleafe> bauzas: you snooze you lose
15:50:10 <n0ano> edleafe, I don't know why people object to gantt, it's just a name, but sure, I can remove it
15:50:25 <edleafe> n0ano: it has baggage
15:50:30 <lxsli> n0ano: I find it's best to survey the most busy people first then present options to the less busy people
15:50:34 <claudiub> so... I did not get to read all the meeting log, but I do have a question, I don't know if it was asked before or not... how can compute nodes report any extra resources they have?
15:50:57 <claudiub> that might be needed for any of the extra-flavors that users might need?
15:51:05 <claudiub> e.g. GPU?
15:51:07 <n0ano> lxsli, 7 hrs earlier is problematic for my timezone so I probably won't try that :-)
15:51:24 <lxsli> up to :)  guessing later will be what happens
15:51:57 <lxsli> claudiub: we need a pluggable resource provider system
15:52:19 <n0ano> claudiub, most of the discussion has been on how to request resources, not on how they are reported
15:53:04 <claudiub> n0ano: I get that, but having resources reported seems to me like a part of a whole.
15:53:26 <lxsli> to make the scheduler usable outside Nova, other projects need to be able to supply the appropriate parts of resource tracker
15:53:32 <claudiub> lxsli: are there some details / spec / bp for that?
15:53:52 <lxsli> claudiub: I'm working on a spec around resource objects which may touch on it but it's probably a separate spec
15:53:53 <n0ano> claudiub, no arguement, that should be part of the resource tracker
15:54:38 <edleafe> n0ano: so we
15:54:41 <edleafe> ugh
15:54:54 <lxsli> 5 mins
15:54:57 <edleafe> so we'll need a RT for any project that need scheduling services
15:55:04 <claudiub> lxsli: can you add me as a reviewer to that spec? I'm interested on how that will be implemented
15:55:07 <lxsli> if edleafe + claudiub are done, should we let bauzas speak?
15:55:12 <lxsli> claudiub: sure thing!
15:55:12 <edleafe> or some comparable reporting thing
15:55:19 <claudiub> i'm done
15:55:24 <claudiub> ty folks
15:55:29 <n0ano> lxsli, edleafe spoke for bauzas
15:55:35 <lxsli> ahaa great
15:55:36 <edleafe> when has bauzas ever needed permission to speak? :)
15:55:46 <lxsli> True, true
15:55:49 <bauzas> speak
15:56:08 <edleafe> n0ano: +1 for a doodle to find a time
15:56:18 <bauzas> anyway, I didn't really followed that meeting, so I won't say something more than zero :)
15:56:25 <edleafe> n0ano: do you want me to set it up?
15:56:33 <n0ano> edleafe, sure, that'd be great
15:56:50 * n0ano doesn't do doodle or etherpads, I'm a luddite
15:56:55 <edleafe> #action edleafe to create doodle for new meeting time
15:57:10 <n0ano> anyway, any last minute opens?
15:57:30 <lxsli> who's coming to midcycle?
15:57:36 <n0ano> lxsli, me for one
15:57:45 * edleafe is
15:58:08 <edleafe> lxsli: can you make it?
15:58:10 <lxsli> Yep
15:58:14 <edleafe> kewl
15:58:29 <n0ano> OK, tnx everyone, we have to close
15:58:33 <lxsli> thanks all
15:58:33 <n0ano> #endmeeting