13:59:56 <n0ano> #startmeeting nova-scheduler
13:59:57 <openstack> Meeting started Mon Feb 29 13:59:56 2016 UTC and is due to finish in 60 minutes.  The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:59:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:01 <openstack> The meeting name has been set to 'nova_scheduler'
14:00:06 <edleafe> \o
14:00:08 <n0ano> anyone here to talk about the scheduler?
14:00:10 <Yingxin> o/
14:00:23 <bauzas> \o
14:00:47 <bauzas> n0ano: you're early today, you beated me at the clock
14:01:07 <n0ano> bauzas, yeah, I beat the top of the hour by abouot 10 sec :-)
14:01:33 * johnthetubaguy lurking
14:01:50 <bauzas> oh, beat, beat, beat, irregular verb
14:01:54 <n0ano> well, the usual suspects are here so let's get started
14:02:11 <n0ano> #topic Patches/Reviews – https://etherpad.openstack.org/p/mitaka-nova-priorities-tracking
14:02:29 <bauzas> beat, beat, beaten even
14:02:33 <n0ano> I see I go away for a week have you guys have a major discusson on this
14:02:47 <bauzas> cdent: around ?
14:02:51 <cdent> o/
14:02:53 <n0ano> anything to continue this week or are we good?
14:03:11 <cdent> so yeah, there's the resource-provider/pool stuff
14:03:14 <bauzas> n0ano: we're closing M-3 this week, so we are going to sprint merging some of cdent's teeth
14:03:27 <cdent> starts here: https://review.openstack.org/#/c/281837/
14:03:41 <cdent> there are some questions/issues that need to resolved
14:04:04 <n0ano> cdent, are they minor enough they can be closed in 1 week?
14:04:08 <bauzas> cdent: I missed the unresolved questions ?
14:04:11 <cdent> yes,I think so
14:04:30 <cdent> bauzas: one of them is my last comment on that review
14:04:48 <cdent> the other are the several question in the commit message on: https://review.openstack.org/#/c/284963/
14:04:49 <bauzas> oh missed that one
14:05:09 <edleafe> me too (took the weekend off)
14:05:09 <cdent> that comment (about the constraint) was discovered as a result of the work on the latter review
14:05:21 <Yingxin> cdent: can I ask you some questions about the resource pool after the meeting?
14:06:08 <cdent> Yingxin: of course, and I'll try to answer, but I'm not sure I will have all the answers. My ignorance is why I'm hoping for feedback from bauzas, johnthetubaguy, dansmith and jaypipes
14:06:39 <bauzas> cdent: so, I was on PTO last Friday
14:06:42 * johnthetubaguy hopes to hit those ones soon
14:06:55 <bauzas> cdent: so I missed the problem, could you please rephrase it ?
14:07:08 <bauzas> I can see it's an UC problem, right?
14:07:57 <cdent> bauzas: Yeah. It's possible, as the database is currently constructed to create an inventory for the same resource-provider resource-class pair.
14:08:09 <cdent> According to the comments on the resource-pools spec this is not desired
14:08:16 <bauzas> cdent: for the moment, nothing is populating that table, right?
14:08:26 <cdent> Or at least the comments can be interpreted that way
14:08:37 <cdent> bauzas: only tests
14:08:54 <cdent> (that I'm aware of, but some of the patches in that stack I dont know)
14:09:41 <bauzas> cdent: so, let's be clear, we're missing this https://review.openstack.org/#/c/283253/ at least ?
14:10:12 <cdent> in talking with dansmith we decided that if we we did need to add another migration it would be better to do it in the existing one that hasn't merged yet, not add another
14:10:16 <bauzas> cdent: I don't see a discussion around UCs in there
14:10:25 <bauzas> cdent: and I agree with him
14:10:33 <cdent> bauzas: that's because I only discovered it late friday
14:10:52 <bauzas> cdent: okay, gotcah
14:11:09 <bauzas> cdent: so, what's the rationale behind needing an UC ? sorry, missing context obviously
14:11:22 <bauzas> is that point being persisted in a gerrit comment ?
14:11:33 <bauzas> or is this an IRC convo ?
14:11:41 <bauzas> I need to understand *why* we need an UC
14:11:54 <bauzas> and if so, we could quickly iterate on that
14:11:55 <cdent> look at the conversation between Roman and Jay at the end of this: https://review.openstack.org/#/c/253187/
14:11:57 <johnthetubaguy> what is a UC in this context?
14:12:04 <cdent> unique constraint
14:12:19 <johnthetubaguy> ah, doh, gotcha
14:12:58 <bauzas> cdent: oooooh, all of the discussion was not in files, just general messages, missed that :/
14:13:00 <cdent> so setting aside the technical terms for a moment: the questinon is: "Can a resource provider have more than one inventory for the same resource class?"
14:13:13 <bauzas> gotcha
14:13:33 <bauzas> and that's a good point :-)
14:14:04 <cdent> I'm pretty sure the answer has to be no, otherwise the structure for accounting for things falls apart
14:14:19 <cdent> as it is hard to know which inventory an allocation would be associated with
14:14:39 <n0ano> not sure why you would `need` multiple inventories anyway
14:14:45 <johnthetubaguy> yeah, that has to be a no, I feel
14:15:13 <_gryf> jay has explained that matter on irc to me
14:15:17 <cdent> n0ano: there was some discussion of a provider having two different batches of the same thing
14:15:29 <edleafe> Seems that while it might be physically possible, it isn't something that we should encourage
14:15:37 <cdent> edleafe++
14:15:49 <bauzas> cdent: +1 for a no, that's one of the motivations we had for having resource-providers, just because we don't have a clear way of having consistent resources
14:15:56 <n0ano> cdent, I'd want a stronger justification than a discussion, more like a valid use case
14:16:03 <_gryf> so the answer is no in this case - if there is a need to have another resource of the same class, than another pool will be created
14:16:11 <bauzas> _gryf: yeah
14:16:13 <edleafe> n0ano: and not just a *possible* use case
14:16:16 <cdent> I'm happy to barrell forward, but I'm a bit cautious as there's a _lot_ of work I've done for this stack which has been discovered to be off track well past the time I did it, so I'm a bit...shy and frustrated?
14:16:21 <n0ano> edleafe, +1
14:16:47 <bauzas> so we shouldn't care about "possible" usecases
14:16:56 <bauzas> we should care about what's in the spec
14:16:56 <cdent> k, I'll go ahead and put the constraint on the existing migration
14:17:00 <n0ano> cdent, don't get too discouraged, that's just the nature of changing complex systems
14:17:27 <cdent> n0ano: I'm familiar. It's the feedback latency that has me frustrated not the complexity of the systems.
14:17:34 <bauzas> cdent: ping me when you're done with https://review.openstack.org/#/c/281837/
14:17:39 <johnthetubaguy> so the dropping a unique constraint is a non-impacting DB change I am guessing?
14:17:51 <n0ano> cdent, indeed
14:17:51 <cdent> johnthetubaguy: we're adding
14:17:58 <johnthetubaguy> so this is adding
14:18:05 <edleafe> cdent that was one of the motivations for specs: so we could agree on the big picture before writing code
14:18:05 <cdent> but to a table that doesn't have anything in it
14:18:06 <johnthetubaguy> I am thinking if we find we need to drop it
14:18:08 <johnthetubaguy> thats easy I think?
14:18:10 <bauzas> johnthetubaguy: yeah, it's just a matter of amending https://review.openstack.org/#/c/281837/ with adding an UC on an empty table
14:18:14 <cdent> johnthetubaguy: yes
14:18:20 <johnthetubaguy> cool, so lets just do it
14:18:22 <edleafe> the freeze is making us write code before the specs have been settled
14:18:30 <bauzas> johnthetubaguy: that https://review.openstack.org/#/c/281837/ is already for adjusting the model
14:19:02 <bauzas> johnthetubaguy: with https://review.openstack.org/#/c/283253/ reflecting that on nova-specs
14:19:20 <bauzas> (so for example, explaining the 'generation' field)
14:19:29 <johnthetubaguy> so the spec is probably too detailed for some of these
14:19:42 <cdent> johnthetubaguy++
14:19:44 <johnthetubaguy> the large brush strokes agreements we made in person, so I am OK with how we are going here
14:20:02 <johnthetubaguy> normally thats a spec merge, but lets just do whats needed at this point
14:20:43 <cdent> So I'm good to go on the migration, but there's plenty of other stuff in that stack that needs feedback, pronto, if we want to get it merged. Most of it is model or object changes so stuff we want in M
14:21:40 <johnthetubaguy> right, so which is the last change we need in this cycle, so we have all the migrations complete?
14:21:47 <Yingxin> bauzas: 'generation' looks like a field to force atomic transactions and resolve races between schedulers' claims.
14:22:40 <cdent> johnthetubaguy: I'm not entirely certain. I don't really understand the rules that impact what limits things.
14:23:02 <cdent> I'm pretty sure it's something very close to "all of it"
14:23:02 <johnthetubaguy> its really about upgrade
14:23:07 <bauzas> Yingxin: it's rather because of a compare-and-update model in a transactional model
14:23:20 <johnthetubaguy> we need those data migrations and schema migrations in place I think
14:23:27 <johnthetubaguy> the data ones being the uuid generation on compute node, I think
14:23:30 <cdent> because that stuff leaves out all the actually doing stuff with resource providers in favor of getting just the groundwork in place
14:23:44 <bauzas> cdent: johnthetubaguy: so, the thing is, we need computes to update things in DB to get that not require an online migration for Newton
14:24:05 <bauzas> cdent: johnthetubaguy: not only the model to be present AFAIK
14:24:16 <johnthetubaguy> the uuid is the only key bit, really
14:24:21 <bauzas> not only the model to be /up-to-date/ rather
14:24:22 <johnthetubaguy> migration wise
14:24:27 <cdent> bauzas, johnthetubaguy If you guys review that entire stack I think it will be more clear which is required
14:24:33 <johnthetubaguy> we can read from old ad new location
14:24:40 <cdent> (even if it is just a light drive-by review)
14:24:42 <johnthetubaguy> cdent: yeah, thats probably quicker at this point
14:24:49 <cdent> johnthetubaguy++
14:25:41 <n0ano> so, have we beat this to death and just review the patch series is the next step?
14:26:04 <cdent> I think so, and those with particular interest/investment can help determine which is the required stuff
14:26:20 <cdent> and also help me figure out the ResourcePool objct
14:26:21 <bauzas> cdent: well, I was seeing https://review.openstack.org/#/c/279313/ as a needed merge for preventing data online migrations in Newton, unless I'm wrong ?
14:27:03 <cdent> we also need all the objects, don't we, otherwise actually using them is delayed by an additional cycle?
14:27:05 <bauzas> because if old computes don't store that information new-way-ish, then we need to do an online migration when calling that
14:27:31 <cdent> yes, we do need 279313
14:28:13 <bauzas> cdent, tbc we need object remotable methods as well, yes
14:29:51 <n0ano> OK, moving on
14:29:57 <n0ano> #topic Bugs - https://bugs.launchpad.net/nova/+bugs?field.tag=scheduler
14:30:22 <johnthetubaguy> so bauzas cdent: lets catch up about https://review.openstack.org/#/c/279313 I think thats the most important thing we need
14:30:23 <n0ano> I believe we've actually reduced the bug count by about 4, we down to `only` 35 bugs
14:31:09 <n0ano> I don't know who closed a few but that's good news
14:31:22 <bauzas> johnthetubaguy: that's the last patch of the branch, yes
14:32:33 <n0ano> are there any specific bugs anyone is concerned about?
14:33:26 <n0ano> if not, let's move on
14:33:33 <n0ano> #topic opens
14:34:03 <Yingxin> I was modeling my design based on Jay Pipes benchmarking tool. The work is nearly done.
14:34:04 <n0ano> actually, I do have a slightly personal matter to tell everyone about...
14:34:41 <n0ano> after a `long` career I have decided to retire as of April this year...
14:35:19 <edleafe> n0ano: we'll miss you, but that's awesome news for you!
14:35:20 <cdent> Do you want to be congratulated or consoled? :)
14:35:28 <n0ano> I'll be doing other things, hopefully involved in open source, but it looks like I'll be dropping off of the Open Stack project
14:35:37 <n0ano> cdent, yes :-)
14:36:11 <n0ano> it's been a pleasure to work with you all but you'll have to find someone else manage you guys :-)
14:36:20 <Yingxin> n0ano: :) I've just started my open source career.
14:36:40 <n0ano> Yingxin, may it be long
14:36:59 <edleafe> I volunteer jaypipes - he doesn't have very much going on :)
14:37:11 <n0ano> edleafe, +1
14:38:08 <bauzas> n0ano: oh, pleasure as well
14:38:26 <n0ano> so, winding down, anything else for today?
14:38:41 <mlavalle> n0ano: yes, I have something to bring up
14:38:49 <n0ano> mlavalle, go ahead
14:40:08 <n0ano> mlavalle, you there?
14:40:36 <edleafe> (probably typing an essay)
14:41:04 <n0ano> edleafe, I try and use vi and then paste in that situation but to each their own :-)
14:41:32 <mlavalle> n0ano: I would like the team to take a look at this spec we are working for Newton: https://review.openstack.org/#/c/263898/
14:41:33 <mlavalle> n0ano: it complements an effort we will be conducting on the Neutron side to implement routed networks
14:41:33 <mlavalle> n0ano: this has impact on the nove scheduler
14:41:33 <mlavalle> nova scheduler^^^^
14:41:33 <mlavalle> it's not urgent, but I would like to start giving it visibility in this meeting
14:41:33 <mlavalle> n0ano: so when the team has time, I encourage everybody to give us feedback on it
14:42:24 <n0ano> mlavalle, NP, just realize that, given the schedule pressures, it might be a little while before we can give you good feedback
14:42:48 <bauzas> mlavalle: ack
14:42:49 <n0ano> mlavalle, but tnx for pointing this out and we'll add it to our queues
14:43:19 <edleafe> mlavalle: was just going to say the same as n0ano. I've starred it, but don't be shy about pinging us with reminders when Newton opens up
14:43:53 <n0ano> OK, anything else?
14:44:02 <Yingxin> By the way, the modeling work is at https://github.com/cyx1231st/placement-bench/tree/shared-state-demonstration
14:44:05 <bauzas> mmm, its a backlog spec, we can litterally review it when we want
14:44:41 <edleafe> bauzas: sure, but time is the limit here
14:45:20 <mlavalle> n0ano: I know, I just want to start early drawing attention to it
14:45:20 <mlavalle> :-)
14:45:20 <mlavalle> that's all I have. Thanks
14:45:33 <n0ano> mlavalle, NP
14:46:26 <bauzas> edleafe: sure, just wanted to explain it's just a matter of finding time, not a release process issue :)
14:47:25 <n0ano> OK, anything else?
14:47:31 <cdent> nosir
14:48:07 <n0ano> then let's close for today and I want to thank everyone, it's been a real trip
14:48:18 <n0ano> #endmeeting