15:01:04 <n0ano> #startmeeting gantt
15:01:04 <openstack> Meeting started Tue Mar  4 15:01:04 2014 UTC and is due to finish in 60 minutes.  The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:01:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:01:08 <openstack> The meeting name has been set to 'gantt'
15:01:15 <n0ano> anyone here to talk scheduler?
15:02:36 <bauzas> \o
15:02:47 <bauzas> (left-handed)
15:02:58 <n0ano> you and my wife :-)
15:03:15 <bauzas> :-)
15:04:15 <n0ano> looks like everyones busy today, maybe you & I can talk a little about the forklift
15:04:22 <n0ano> #topic scheduler code forklift
15:04:31 <bauzas> sure
15:04:58 <n0ano> I've been trying to update gantt to match the current nova tree, just to see what interfaces need to be updated, not much progress yet, have you been able to look at anything?
15:05:17 <bauzas> https://etherpad.openstack.org/p/icehouse-external-scheduler
15:05:30 <bauzas> I looked at the 2nd bp
15:05:48 <n0ano> the scheduler-lib one?
15:05:55 <bauzas> yup
15:06:21 <n0ano> that would be good, how does it look to do that?
15:06:23 <bauzas> the discussion was around where to update metrics
15:07:11 <bauzas> ie. if we say that the scheduler lib should stand by its own, does it mean it could be a different SQL backend ?
15:07:33 <bauzas> atm, johnthetubaguy told me it was not necessary for a first release
15:08:03 <n0ano> given it stands alone I'd say yes, it could use a different SQL backend but why would we
15:09:00 <bauzas> where the HostState should be placed ?
15:09:14 <bauzas> I was thinking it should stand by its own as Gantt DB
15:09:32 <bauzas> so that queries would look in it directly within the filters
15:09:55 <n0ano> my goal would be to hold the hoststate in scheduler memory, potentially backed up by something like memcached to support multiple schedulers
15:10:31 <bauzas> n0ano: that's sexy, but that means we should wait for the nodb-schedule to be implemented
15:11:10 <n0ano> note I said goal, nodb would be `really` good but, while waiting for it, we can just keep the current SQL database
15:11:16 <bauzas> ok
15:11:33 <bauzas> I agree with the goal
15:11:53 <bauzas> my concern is just how we make sure to backport changes to Gantt once forklifted
15:12:03 <n0ano> I would imagine the first step would be to have the SQL database updated and queried only by the scheduler
15:12:09 <bauzas> +1
15:12:19 <bauzas> let's talk baby steps
15:12:24 <n0ano> +1
15:12:32 <n0ano> I think we're in violent agreement :-)
15:12:54 <n0ano> backporting of changes is doable I think, just a lot of grunt work on my part
15:13:30 <n0ano> not that we need to make the scheduler-lib and the SQL accesses in the nova tree first, we'll do a new generation of the gantt tree once plan B is done
15:14:55 <bauzas> ok, looking at the resource_tracker, I think the goal would be first to move the conductor calls to compute_nodeupdate() to scheduler lib
15:15:56 <n0ano> sounds good, how does that affect the calls in the compute code to send that info
15:16:19 <bauzas> all the code is factorized
15:16:32 <bauzas> within the _update() method
15:16:51 <n0ano> so it's well isolated and easy to change
15:16:58 <bauzas> I think (and hope) so
15:17:23 <n0ano> sounds like a plan, do you think you can work on doing that soon?
15:17:34 <bauzas> but my question is, do we consider this call should not use the nova conductor then ?
15:17:55 <n0ano> not sure what you mean and what's the implication
15:17:56 <bauzas> gantt currently doesn't have conductors in place
15:18:27 <bauzas> the call to update the host is done by self.conductor_api.compute_node_update(
15:18:27 <bauzas> context, self.compute_node, values)
15:19:32 <n0ano> as long as that compute_node_update method winds up calling an API rather than linking into the scheduler code we should be OK
15:19:53 <bauzas> that doesn't use scheduler code
15:20:00 <bauzas> maybe I misexplained
15:20:20 <bauzas> resource_tracker is updating stats to ComputeNode table
15:20:53 <bauzas> thanks to compute_node_update() which finally goes to update ComputeNode table
15:20:58 <bauzas> thru the conductor
15:21:46 <bauzas> there is no scheduler code in it
15:22:33 <n0ano> well, if there's no scheduler code in it then it shouldn't be a problem for the forklift, move the scheduler out to a separate tree and the compute_node_update method should still work fine
15:22:50 <bauzas> that's only HostState from the Scheduler which is periodically updating its states from the ComputeNode table
15:23:18 <bauzas> the problem is that both projects share same table
15:23:23 <bauzas> Nova and Gantt
15:23:53 <n0ano> sharing the same table is OK for the forklift (maybe an issue for the no-db work but that's boris-42
15:23:59 <n0ano> ) problem.
15:24:36 <bauzas> see the rationale here https://blueprints.launchpad.net/nova/+spec/scheduler-lib
15:26:31 <n0ano> I think there's some confusion between splitting the code out and creating well defined interfaces
15:27:00 <n0ano> that BP is more about creating interfaces (which is good) but not necessarily necessary to splitting the code out
15:27:15 <bauzas> well, I'm ok with it
15:28:14 <n0ano> no arguement, clean interfaces are good, personally I'm kind of focused on splitting the code out, I think we can do both in parallel
15:28:32 <bauzas> think so too
15:28:46 <bauzas> basically the proposal is to wrap up some interfaces within Gantt, that's it
15:29:19 <n0ano> yep, that's the idea
15:29:46 <bauzas> we're entering FeatureFreeze
15:30:04 <bauzas> when do you think it's worth working on it ?
15:31:20 <n0ano> well, we can work on a private tree of course but I don't think we should try to push patches until after the Juno tree opens up, there's too much that needs to done on Icehouse right now
15:31:59 <n0ano> it would be nice to get as much done as possible early in the Juno cycle, otherwise things will have a tendency to slip a lot
15:32:23 <bauzas> +1
15:32:23 <bauzas> anyway, i'm new to nova
15:32:23 <bauzas> I was more focused on other project
15:32:23 <bauzas> so, I can do some drafts there
15:32:34 <bauzas> and you would be able to review me
15:32:45 <bauzas> and johnthetubaguy as well
15:33:06 <bauzas> so that once Juno would go live, we could focus on real patchsets
15:33:18 <n0ano> absolutely (I consider myself relatively new myself)
15:33:58 <bauzas> I'm moving from Climate to Nova scheduler things, so I have a little ramp-up for understanding Nova concepts
15:33:58 <n0ano> I want to keep working until juno goes live so we can immediately push stuff if possible but we do have a little bit of time.
15:34:06 <bauzas> sounds good
15:34:18 <n0ano> we can learn together :-)
15:34:21 <bauzas> :)
15:34:54 <bauzas> I hope to be present at the Juno summit, that will depend on the Travel Support Program agreement... or not :)
15:35:22 <n0ano> I'll be there (these conferences are one I can talk my company into), let's hope you can make it too
15:35:49 <n0ano> where are you physically located?
15:36:22 <bauzas> Grenoble, France
15:36:55 <n0ano> Ahh, well, the K summit should be easy for you (Paris), Atlanta I feel your pain
15:37:06 <bauzas> I was present in KH
15:37:10 <bauzas> HK
15:37:42 <n0ano> I was in HK also but I have a tendency to keep a low profile
15:37:52 <bauzas> :)
15:38:22 <bauzas> I did a quick nova unconference presentation on Friday about scheduling interactions with Climate
15:38:23 <n0ano> I'm good for today, unless you have anything else you want to discuss?
15:38:28 <bauzas> nope, I'm fine too
15:38:37 <bauzas> let's back to work
15:38:45 <n0ano> cool, good talking to you and we'll meet again next week.
15:38:50 <bauzas> sure
15:38:53 <n0ano> #endmeeting