15:00:50 <n0ano> #startmeeting gantt
15:00:51 <openstack> Meeting started Tue Nov 18 15:00:50 2014 UTC and is due to finish in 60 minutes.  The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:54 <openstack> The meeting name has been set to 'gantt'
15:01:00 <n0ano> anyone here to talk about the scheduler?
15:01:01 <edleafe> \o
15:01:06 <lxsli> o/
15:02:01 <jgallard> hi!
15:02:04 <vigneshvar> hi
15:02:14 <bauzas> \o
15:02:20 <bauzas> (sorry, was late)
15:02:22 <jaypipes> o/
15:02:31 <n0ano> just about to ping you two :-)
15:02:38 <n0ano> anyway, let's start
15:02:50 <n0ano> #topic code cleanup tasks
15:03:13 <bauzas> so, maybe we can take a look at the wiki page ?
15:03:15 <n0ano> I hope everyone has studied the email threads and the wiki page at:
15:03:19 <n0ano> https://wiki.openstack.org/wiki/Gantt/kilo#Tasks
15:03:22 <bauzas> +1
15:03:32 <lxsli> #link https://wiki.openstack.org/wiki/Gantt/kilo#Tasks
15:04:05 <n0ano> I've tried to distill down what needs to be done at the wiki, the general question is did I miss anything or is something there that's not needed
15:04:24 <bauzas> n0ano: lgtm
15:04:33 <PaulMurray> n0ano, the RT objects one needs updating - I'll do it
15:04:40 <n0ano> PaulMurray, tnx
15:04:42 <jaypipes> n0ano: there's refactoring of the RT unit tests ongoing, along with my work to make nova.virt.hardware use nova.objects framework.
15:05:15 <n0ano> jaypipes, should I add a couple of line items to the table about those?
15:05:17 <bauzas> jaypipes: oh right
15:05:34 <jaypipes> n0ano: no worries, I can do that
15:06:12 <n0ano> jaypipes, cool, I want to tweak the table a little, I want to add a column about approval for the specs, that's pretty crucial
15:06:21 <bauzas> so, about https://blueprints.launchpad.net/nova/+spec/detach-service-from-computenode
15:06:28 <jaypipes> n0ano: k, will wait for you to change.
15:06:42 <n0ano> jaypipes, np
15:06:43 <bauzas> item #4
15:06:46 <n0ano> bauzas, go ahead
15:07:08 <bauzas> I've been told yesterday that data migrations are no longer accepted for Kilo
15:07:31 <n0ano> before even K1?  that seems a little harsh
15:08:01 <jaypipes> schema migrations are fine, just not data migrations I guess.
15:08:09 <bauzas> jaypipes: exactly
15:08:19 <bauzas> I meant *data* migration
15:08:29 <bauzas> so adding a new col is ok
15:08:54 <bauzas> but now we have to use the objects backwards compatibility to provide data for it
15:08:56 <jaypipes> bauzas: right.
15:09:01 <bauzas> so I'll update the spec
15:09:03 <jaypipes> k
15:09:09 <jaypipes> and I will +1 it again :)
15:09:27 <jaypipes> bauzas: let's get this one approved ASAP, so the sooner you push a fixed spec, the better :)
15:09:35 <jaypipes> bauzas: danpb already +2d it
15:09:47 <bauzas> jaypipes: so that means that the fields won't be changed on upgrade and downgrade, but the compute_node object will do it by itself
15:10:11 <bauzas> jaypipes: it will require some change in the code, but I don't expect so much things
15:10:39 <bauzas> the bad thing is that I'm the first to test the new process so maybe some discussion has to be made :)
15:11:00 <jaypipes> bauzas: why wouldn't you be able to change the fields on update and downgrade?
15:11:02 <bauzas> jaypipes: yeah, but johnthetubaguy -1'd it for a good reason
15:11:28 <bauzas> jaypipes: because now it's asked to not do that using dbmigrate tool
15:11:50 <bauzas> jaypipes: instead, objects have to check the version and update by themselves
15:12:19 <bauzas> IIUC of course
15:12:26 <jaypipes> bauzas: :( I think I need to know more specifics then...
15:12:58 <bauzas> jaypipes: I suggest that we discuss about it offline
15:13:02 <bauzas> with johnthetubaguy
15:13:12 <n0ano> bauzas, are you saying the same column would contain rows with different versions, that seems iffy
15:13:21 <bauzas> n0ano: nope, that's simple for us
15:13:29 <PaulMurray> bauzas, I would like to follow this - or at least find out what is done - maybe I should just read the patches
15:13:31 <bauzas> n0ano: there is a new col
15:13:36 <bauzas> called host
15:13:48 <bauzas> ergh, a new 'field'
15:13:57 <johnthetubaguy> basically objects does data migrations, flavor stuff is the first example of that
15:14:03 <johnthetubaguy> thats the current plan at least
15:14:11 <johnthetubaguy> the details are… hairy at best
15:14:16 <bauzas> johnthetubaguy: thanks for clarifying
15:14:34 <PaulMurray> johnthetubaguy, all this is a nightmare for rebasing - especially if someone else is working in the same code
15:14:41 <johnthetubaguy> there will have to be a spec I recon, for the data migration pattern,
15:14:51 <johnthetubaguy> PaulMurray: the other stuff is already hell for that same reason
15:14:59 <n0ano> and we're the first guinea pig - fun :-(
15:15:04 <bauzas> indeed :)
15:15:08 <PaulMurray> johnthetubaguy, I really want to make that kind of thing easier
15:15:09 <johnthetubaguy> n0ano: yeah, thats the sucky bit
15:15:13 <bauzas> I like to be a guinea pig :)
15:15:35 <johnthetubaguy> PaulMurray: yeah, ideas welcome, I think the version is now per object, rather than global, which helps a bit
15:15:35 <PaulMurray> bauzas, you said that out loud
15:15:59 * johnthetubaguy fills out medical testing application for bauzas
15:15:59 <bauzas> :)
15:16:00 <PaulMurray> johnthetubaguy, yes, but if it is referred to by another object that one has to change too
15:16:14 <PaulMurray> johnthetubaguy, etc.
15:16:20 <johnthetubaguy> PaulMurray: right, this is the data version, not the object version, but yeah
15:16:42 <johnthetubaguy> we might make them the same thing, but we need to hash all that out yet
15:16:42 <n0ano> johnthetubaguy, is there a BP about all this, I need to sit back and think on it.
15:16:43 <bauzas> PaulMurray: yeah, the nested object thing is quite a pain to maintain, but I think the idea is pretty good
15:17:09 * PaulMurray end of side-track
15:17:10 <bauzas> PaulMurray: johnthetubaguy: maybe we should free up what's possible with nested objects
15:17:15 <johnthetubaguy> n0ano: not quite, is the simple answer, I am owning that priority thing, so I need to pull my finger out an make something more formal happen
15:17:40 <n0ano> johnthetubaguy, fer sure, this looks hard and it would be nice to have something concrete to read on it
15:17:48 <bauzas> johnthetubaguy: I would apprieciate if you could review my patch series when it will be updated :)
15:18:07 <johnthetubaguy> right now, I would plan on doing expand and not contract, and leave the contract for the upgrade folks to fix
15:18:19 <lxsli> did someone say they'd create a hacking check to alert if a dependent object version was updated without the depending object version being updated?
15:18:29 <johnthetubaguy> and follow the pattern dansmith is doing for flavor for the expand bit
15:18:52 <PaulMurray> lxsli, there is a test for that
15:19:03 <lxsli> Ah great, ty
15:19:05 <PaulMurray> lxsli, but any new objects have to be added to the test
15:19:16 <bauzas> johnthetubaguy: well, I was not expecting data migrations to the flavor BP
15:19:30 <bauzas> johnthetubaguy: because it's purely additive nope ?
15:19:32 <bauzas> johnthetubaguy: oh
15:19:57 <bauzas> johnthetubaguy: nope, my bad, metadata has to be populated to the new table
15:19:58 <johnthetubaguy> bauzas: its a move, it doesn't really need the contract part, becuase we are not killing system metadata (yet)
15:20:04 <johnthetubaguy> right
15:20:13 <johnthetubaguy> anyways, sorry, end of side track, hopefully!
15:20:17 <bauzas> johnthetubaguy: I then exactly have the same pattern
15:20:21 * johnthetubaguy goes back to lurking
15:20:30 <bauzas> johnthetubaguy: thanks
15:20:37 <johnthetubaguy> bauzas: yeah, I would copy it, add the thing, and use it, leave the old
15:21:06 <bauzas> johnthetubaguy: that's barely already done
15:21:19 <johnthetubaguy> perfect
15:21:25 <bauzas> johnthetubaguy: except the thing that I was updating the existing fields
15:22:09 <bauzas> k, I think we're done with this bp
15:22:19 <bauzas> I also have another one to mention
15:22:21 <n0ano> well, seems like we know what we're doing on this one
15:22:29 <n0ano> bauzas, go ahead
15:22:42 <bauzas> https://blueprints.launchpad.net/nova/+spec/request-spec-object
15:22:55 <bauzas> ie. https://review.openstack.org/#/c/127610/
15:23:06 <bauzas> at the moment, it's a pure mess
15:23:22 <bauzas> so, the basic plan is to objectify request_spec
15:23:31 <bauzas> but there is also filter_properties dict
15:23:50 <bauzas> so I discussed with jaypipes about that, but I would like to get the consensus here too
15:24:19 <bauzas> (because I'll still implementing this next week, albeit for K2)
15:24:27 <PaulMurray> bauzas, can you get consensus with one other person :)
15:24:54 <bauzas> well, from my past experience, I really like to get feedback before doing anything
15:25:08 <bauzas> because this scheduler.client thing merged with +70 iterations
15:25:20 <bauzas> and I don't talk about ERT ;)
15:25:34 <PaulMurray> +70 is a record by me
15:25:46 <n0ano> anyway, so what specifically do you need concensus on?
15:25:50 <bauzas> so
15:26:02 <bauzas> select_destinations() is admitting 2 args
15:26:10 <bauzas> filter_properties and request_spec
15:26:35 <bauzas> later in the scheduler, request_spec is nested into filter_properties
15:26:55 <bauzas> so the filters access the spec by doing filter_prop['req_spec']
15:27:01 <bauzas> my question is
15:27:24 <bauzas> should we consider having 2 objects for that select_dest() interfaces or only one, and so which one ?
15:27:54 <bauzas> I was about saying RequestSpec being the object and filter_properties a field of it
15:28:10 <bauzas> but in that case, it's just a non-sense to invert that for the filters
15:28:10 <n0ano> if one param it should be the request (which contains the filter prop) but I don't see a problem with 2 parameters
15:28:20 <PaulMurray> bauzas, I agree on that way around
15:28:31 <bauzas> PaulMurray: which is ?
15:28:33 <edleafe> It seems that logically the filter_properties is part of the request
15:28:40 <bauzas> edleafe: agreed
15:28:52 <bauzas> edleafe: so we would need to invert the logic in the scheduler too
15:29:00 <bauzas> that's a massive and invasive change
15:29:02 <edleafe> bauzas: yep
15:29:05 <lxsli> +1 to RequestSpec being the object and filter_properties a field of it
15:29:08 <PaulMurray> bauzas, what you said: filter props inside request
15:29:26 <PaulMurray> bauzas, but I wonder if it is early to do that?
15:29:30 <n0ano> bauzas, it's not inverting the logic, it's changing the data access, is that really such a big change
15:29:37 <bauzas> n0ano: it is
15:29:41 <bauzas> n0ano: trust me :)
15:30:01 <bauzas> n0ano: because we need to change all the filters, the tests etc.
15:30:07 <PaulMurray> bauzas, could remove request spec from filter props but keep both as first step
15:30:30 <bauzas> PaulMurray: I like having baby steps here
15:30:30 <n0ano> well, the target is K2, if we start now we should be able to make it, I can get someone else to help on this
15:30:33 <PaulMurray> bauzas, then make object
15:30:41 <PaulMurray> bauzas, for request
15:30:56 <edleafe> bauzas: could we change it in steps: first outside the scheduler, and just have one place inside the scheduler that inverts it to match current design?
15:31:09 <bauzas> edleafe: I was thinking about it
15:31:19 <n0ano> edleafe, I like that idea
15:31:26 <vigneshvar> edleafe: good idea
15:31:32 <bauzas> edleafe: it's quickier to only change the RPC interface of the scheudler
15:31:39 <edleafe> bauzas: and then later clean up the references inside the scheduler
15:31:48 <bauzas> and then keep the actual logic within the scheduler
15:32:02 <bauzas> so we only have one place to change
15:32:13 <bauzas> k, agreed ?
15:32:14 <n0ano> remember, the objective for the Kilo release is to clean up interfaces, not necessarily internal implementations
15:32:16 <bauzas> jaypipes: ?
15:32:18 <lxsli> +1
15:32:28 <edleafe> n0ano: precisely
15:32:28 <n0ano> bauzas, +1
15:32:34 <bauzas> n0ano: I know, I know
15:32:56 <bauzas> sounds like we lost jaypipes :)
15:33:28 <n0ano> might want to email/irc him directly and make sure he's on board also
15:33:38 <bauzas> k, will go this way and will notify jaypipes
15:33:55 <bauzas> anyway, it was the first step
15:34:10 <n0ano> cool, moving on, one other item I'd like to discuss is like 3 of the table
15:34:17 <bauzas> the second step was to change anything in the scheduler, we can defer it or ask someone else to handle it for K3
15:34:28 <bauzas> edleafe: interested ?
15:34:35 <n0ano> PaulMurray, that's your's, since it's targed for K1 do you have an update on where we stand with it.
15:34:56 <edleafe> bauzas: sure, but that will come later, no?
15:35:00 <PaulMurray> n0ano, sorry got distracted
15:35:10 <n0ano> PaulMurray, NP
15:35:25 <bauzas> edleafe: yep, but we can do a temptative attempt for K3 nope ?
15:35:26 <PaulMurray> n0ano, we could take it on if you like
15:35:32 <bauzas> edleafe: if you have time of course
15:35:54 <edleafe> bauzas: sure, let's tentatively plan on that
15:36:09 <n0ano> PaulMurray, not sure what you meant
15:36:13 <bauzas> edleafe: k, will put you as a reviewer of my first drafts then
15:36:36 <PaulMurray> n0ano, you want an update on rt objects
15:36:45 <PaulMurray> yes?
15:36:52 <n0ano> yes if you can
15:37:22 <PaulMurray> lxsli, has started to work on the migrations patch and working with jay on redoing the tests for resource tracker
15:37:32 <PaulMurray> there is that one patch up and in  progress
15:37:41 <PaulMurray> we will both start getting more on the way
15:37:45 <PaulMurray> that's about it for now
15:38:05 <n0ano> PaulMurray, cool, good to hear, send me links to the patches and I'll update the table
15:38:17 <PaulMurray> n0ano, already in the table - refresh
15:38:31 <n0ano> PaulMurray, you're too quick, tnx
15:39:07 * PaulMurray has to pay attention to something else for now... sorry
15:39:18 <n0ano> we've talked about the K1 items (except Berrange's ones, I'll deal with those later).
15:39:47 <n0ano> I think that covers the immediate concerns how about I go to opens for now
15:39:56 <n0ano> #topic opens
15:40:01 <n0ano> Anything?
15:40:39 <vigneshvar> would there be anyhelp i would like to
15:40:57 <vigneshvar> typo would there be any help required. i would like to
15:41:10 <n0ano> vigneshvar, tnx, one help would be to review things, that is always needed
15:41:12 <bauzas> vigneshvar: reviewing is the first bit to do
15:41:31 <vigneshvar> sure will
15:41:44 <n0ano> from there we can see what else makes sense
15:42:13 <n0ano> I don't want to keep everyone so, unless there are any last minute opens
15:42:49 <n0ano> I will say thanks and talk to you all again next week (if not earlier)
15:42:56 <n0ano> #endmeeting