14:00:25 <n0ano> #startmeeting nova-scheduler
14:00:26 <openstack> Meeting started Mon Nov 23 14:00:25 2015 UTC and is due to finish in 60 minutes.  The chair is n0ano. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:29 <openstack> The meeting name has been set to 'nova_scheduler'
14:00:38 <n0ano> anybody here to talk about the scheduler?
14:00:44 <edleafe> o/
14:00:58 <ralonsoh> yes
14:01:36 <n0ano> ralonsoh, a new name - welcome
14:01:55 <ralonsoh> hello, nice to meet you
14:02:50 <bauzas> \o
14:03:05 <jichen> o/
14:03:13 * bauzas just caffeinating at the same time
14:03:39 <n0ano> OK, bleery eyed (out of milk so I can't make a latte :-( but I'm back, let's start
14:03:46 <Yingxin> \o
14:03:49 <n0ano> #topic specs & BPs
14:04:17 <n0ano> edleafe, I believe you were going to create a new spec, did you get around to it?
14:04:21 <PaulMurray> o/
14:04:29 <edleafe> n0ano: yes: https://review.openstack.org/245940
14:04:45 <edleafe> not much feedback yet
14:05:24 <n0ano> well, let's ping everybody here at least
14:05:39 <n0ano> #action all to review the spec https://review.openstack.org/245940
14:05:58 <johnthetubaguy> does that add a new configuration variable?
14:05:59 <bauzas> edleafe: ack, CC'ing myself
14:06:19 <bauzas> johnthetubaguy: yes it should
14:06:25 <edleafe> yes
14:06:25 <bauzas> I mean technically
14:06:35 <bauzas> conceptually, I need to read the spec :)
14:06:46 <edleafe> you can't have a choice without a way to specify your choice :)
14:06:51 <lxsli> o/
14:06:54 <johnthetubaguy> my problem is we are working hard to get rid of variables that point to out of tree code, and it feels to go in the opposite direction
14:07:28 <johnthetubaguy> but anyways, I should take that to the spec review I guess
14:07:36 <bauzas> the main problem I see is to add an entrypoint which would make an implicit contract
14:07:39 <lxsli> edleafe: I left a comment
14:07:52 <edleafe> johnthetubaguy: wasn't that the point of using stevedore?
14:08:02 <johnthetubaguy> edleafe: no
14:08:33 <johnthetubaguy> being more modular in the code is great, on its own
14:09:00 <johnthetubaguy> its this bit I always refer to, I think: http://docs.openstack.org/developer/nova/policies.html#public-contractual-apis
14:09:01 <bauzas> edleafe: in general, stevedore plugins imply that you have 2 in-tree plugins :)
14:09:13 <bauzas> johnthetubaguy: thanks, I was looking to that doc
14:09:24 <bauzas> we only contract on the REST APIs
14:09:32 <edleafe> bauzas: simple then: make the new RT for cassandra in-tree! :-P
14:09:41 <bauzas> edleafe: yeah I know
14:10:00 <bauzas> edleafe: I'm just thinking of anyone who could complain because we changed the interface
14:10:30 <bauzas> edleafe: althought it's perfectly clear that we don't guaranttee anything but the REST APIs
14:10:35 <johnthetubaguy> supporting upgrade is generally where life becomes hard with these things
14:10:37 <edleafe> how would it change? If they don't do a thing, it works exactly the same
14:10:46 <bauzas> edleafe: like, for example, I changed the filters interface
14:11:16 <edleafe> bauzas: yes, but that changed how they *worked*
14:11:23 <n0ano> bauzas, isn't that an API issue, don't change the API the upgrade should just work
14:11:28 <edleafe> bauzas: how they were *loaded* didn't change
14:11:28 <bauzas> edleafe: even if that was something internal, the entrypoint it has makes me provide an UpgradeImpact with very cautious statement that it will break out-of-tree filters
14:11:51 <edleafe> bauzas: IMO not the same at all
14:12:08 <bauzas> edleafe: well, you'll have a BaseRT interface I guess ?
14:12:22 <bauzas> edleafe: that anyone could consule
14:12:24 <bauzas> consume
14:12:45 <edleafe> bauzas: but we were discussing *upgrade* pain
14:12:46 <bauzas> I mean, any out-of-tree driver could implement that interface
14:13:07 <edleafe> Now you're talking about someone actively changing things
14:13:17 <bauzas> edleafe: sure, now take the idea that resource-providers will change how we write the DB and will modify the RT interface for writing data
14:13:34 <bauzas> edleafe: that would impact the BaseRT interface, right?
14:14:08 <n0ano> bauzas, I would think service providers would not change the internal code which is what changing the way the RT wrote data would require
14:14:37 <edleafe> bauzas: you should probably read the spec
14:14:44 <bauzas> n0ano: resource-objects doesn't, but I'm unsure about service-providers
14:14:50 <bauzas> edleafe: sure, fair point
14:15:01 <edleafe> bauzas: the only thing I'm looking at changing is how the RT does its thing
14:15:08 <edleafe> resoruce providers are not affected at all
14:15:28 <bauzas> edleafe: okay, touché, will read the spec first
14:15:29 <bauzas> :)
14:15:35 <edleafe> :)
14:15:54 <n0ano> I'm seeing a confusion between publicly available REST APIs and internal nova APIs, we should keep that distinction in mind
14:16:04 <n0ano> anyway, read the spec and comment
14:16:29 <edleafe> n0ano: but the way filters work is essentially a public API, since we allow out-of-tree filters
14:16:43 <bauzas> n0ano: yup, to be clear, I'll read the spec with http://docs.openstack.org/developer/nova/policies.html#public-contractual-apis in mind
14:16:56 <bauzas> edleafe: that's where the problem is
14:17:10 <bauzas> edleafe: there is a big misunderstanding about what nova guarantees
14:17:19 <lxsli> edleafe: not explicitly preventing out-of-tree filters doesn't make it a public API
14:17:28 <bauzas> yeah that
14:17:35 <n0ano> edleafe, well, `any` plugin essentially becomes a public API but the plugin has to follow the internal APIs, an external plugin for Juno is not guaranteed to work with Liberty
14:17:41 <edleafe> well, either nova has to provide every filtering possibility in-tree, or allow out-of-tree with a standard interface
14:17:59 <bauzas> edleafe: to be clear, we have plugins for in-tree choiice
14:18:00 <lxsli> edleafe: or accept limited filtering potential :)
14:18:21 <edleafe> lxsli: translation: we know what's best for your cloud!
14:18:30 <bauzas> edleafe: ie. if you have 2 implementations of the same interface in-tree, then you open a entrypoint
14:18:56 <bauzas> having that interface plus that entrypoint makes out-of-tree hacking easier, but it's not a contract
14:19:11 <edleafe> bauzas: that was never the design goal. It was always the case that we considered people would need their custom filters
14:19:21 <bauzas> edleafe: sure
14:19:39 <bauzas> edleafe: but since, we clarified our position, hence johnthetubaguy's doc
14:20:04 <bauzas> edleafe: so, if you prefer, I'm fine with having a pluggable RT if we have *tw
14:20:15 <bauzas> *two* in-tree implementations
14:20:30 <lxsli> +1
14:20:36 <bauzas> edleafe: I'm a bit more concerned about a driver which would propose *one* in-tree implementation
14:20:36 <edleafe> Fine with me
14:21:07 <johnthetubaguy> I still think the scheduler client should be extended to the point, that its easy for experiments to just use that seam
14:21:24 <johnthetubaguy> anyways, left comments on the spec
14:21:34 <n0ano> I'm actually +1 with the two implementations because why bother to expose the plugin if there's only one
14:21:35 <bauzas> johnthetubaguy: I feel it's doable to experiment some stuff by overriding the sched client, indeed
14:21:38 <edleafe> johnthetubaguy: but the same arguments apply to the number of in-tree clients, no?
14:21:59 <jaypipes> hey guys, sorry for being late :(
14:22:04 <edleafe> isn't that why you didn't want me to make the choice of client configurable?
14:22:26 <bauzas> edleafe: do you really need stevedore for testing a Cassandra RT driver?
14:22:31 <bauzas> to be franck
14:22:33 <bauzas> ah
14:22:36 <bauzas> frank even
14:22:54 <bauzas> since you explained it's your goal
14:22:57 <edleafe> bauzas: or kafka, or cassandra, or whatever else people might be interested in
14:23:14 <bauzas> edleafe: just speaking of the experiment you want to have
14:23:21 <edleafe> I've already created the POC I had at Tokyo that hacked the RT
14:23:59 <bauzas> edleafe: so what's the purpose of enabling an out-of-tree hook ?
14:24:01 <edleafe> any alternative proposal that involve handling the data differently will need the same hack
14:24:43 <bauzas> edleafe: sure, but that's because we don't need any alternative in-tree for the moment :)
14:25:11 <n0ano> guys, I think we're arguing in circles a little here
14:25:22 <n0ano> maybe we should just read the spec and comment there?
14:25:27 <lxsli> yes, any other specs to discuss?
14:25:37 <bauzas> n0ano: yup, agreed
14:25:37 <ralonsoh> https://blueprints.launchpad.net/nova/+spec/aggregate-extra-specs-filter
14:25:37 <edleafe> eh, it's already -2'd, so...
14:26:20 <n0ano> ralonsoh, any specific issue with the spec or do you just want to get attention on it?
14:26:40 <ralonsoh> It was approved last release
14:26:53 <ralonsoh> And there was code in review
14:27:16 <ralonsoh> But the code was rejected by bauzas because the new class created
14:27:32 <ralonsoh> He proposed to include this code in the SpecsFilter
14:27:34 <lxsli> ooh that bauzas!
14:27:44 <ralonsoh> Sorry hehehe
14:27:57 * n0ano I wonder if that bauzas guy is around to comment :-)
14:28:09 <claudiub> hi. expose-host-capabilities spec, if you could read it, it would be great. could use some suggestions on the scheduler part. Uploading a new PS shortly: https://review.openstack.org/#/c/222200/4
14:28:42 <ralonsoh> I don't want to use your time now
14:28:46 <ralonsoh> Only to point it
14:29:30 <n0ano> ralonsoh, that's fine, this is exactly what we're here for, I'm guessing bauzas is review the spec again
14:29:33 <bauzas> sorry, was diverted
14:29:45 <ralonsoh> no problem
14:30:01 <lxsli> johnthetubaguy: can we remove the procedural -2 on that?
14:30:04 <n0ano> bauzas, NP, want to comment on ralonsoh's issue or just comment on the spec
14:30:08 <bauzas> ralonsoh: so, IIRC, I was having some concerns about some concept duplication that you were introducing
14:30:13 <lxsli> johnthetubaguy: https://review.openstack.org/#/c/189279
14:30:44 <bauzas> ralonsoh: so I need to reboot my mind with your spec
14:30:55 <lxsli> action bauzas to re-review?
14:31:09 <ralonsoh> thanks!
14:31:16 <bauzas> lxsli: agreed
14:31:39 <n0ano> #action bauzas to re-review https://blueprints.launchpad.net/nova/+spec/aggregate-extra-specs-filter
14:31:48 <lxsli> thanks n0ano
14:32:09 * n0ano wields the action power - bwahaha
14:32:22 <bauzas> claudiub: in my pipe
14:32:26 <n0ano> anyway, moving on
14:32:33 <n0ano> #topic bugs
14:32:58 <claudiub> bauzas: cool, ty.
14:33:05 <n0ano> although we have 34 bugs I note that 12 are wishlist items but that still leaves 22 outstanding
14:33:09 <lxsli> claudiub: initial reaction is that this is treading on a very uncertain area and it's going to take quite a bit of work to get agreement
14:33:13 <bauzas> well, markus_z is not around
14:33:46 <claudiub> lxsli: yeah, I agree, which is why I need help on that, M-1 is steadly approaching.
14:33:53 <bauzas> n0ano: the bug triage could probably have some lagging
14:33:57 <n0ano> I've got a team I can point at the list, I'll see if I can get them to take on some of these
14:34:16 <bauzas> n0ano: we are missing nova bug triagers you know :)
14:34:26 <lxsli> n0ano: I think markus_z would appreciate the help
14:34:46 <n0ano> let me triage, I just mark everything as NOTABUG and move on :-)
14:34:57 <lxsli> BROKEN_AS_DESIGNED
14:34:57 <bauzas> yup, that's confirmed, some bugs are untriaged
14:34:59 <claudiub> lxsli: but during the mitaka summit, during some nova session, it was concluded that we need something like this in nova.
14:35:07 <edleafe> lxsli: +1 (was just typing the same comment)
14:35:10 <bauzas> like https://bugs.launchpad.net/nova/+bug/1517770
14:35:10 <openstack> Launchpad bug 1517770 in OpenStack Compute (nova) "NULL free_disk_gb causes scheduler failure" [Undecided,New]
14:35:21 <edleafe> lxsli: about the help
14:35:28 <n0ano> claudiub, lxsli are you guys trying to hijack this channel :-)
14:35:44 <lxsli> n0ano: sorry, overflow from specs part :)
14:35:49 <claudiub> sorry. :(
14:36:03 <n0ano> lxsli, claudiub NP it just makes things even more confusing
14:36:35 <n0ano> #action more scheduler bug triaging by all
14:36:53 <n0ano> moving on
14:36:57 <n0ano> #topic opens
14:37:01 <n0ano> anything new from anyone?
14:37:35 <n0ano> hearing crickets, last chance
14:37:40 <edleafe> just a reminded that this will be a slow week for US-based stackers
14:37:49 <edleafe> big holiday
14:37:54 <lxsli> bauzas: how do you feel about persist request spec?
14:38:07 <johnthetubaguy> lxsli: I can't remove a procedural -2 until the blueprint is approved
14:38:14 <n0ano> edleafe, indeed, technically I'm on vacation but I just couldn't stay away :-)
14:38:20 <bauzas> lxsli: some weird issue is occurring, I need time for investigating that
14:38:46 <johnthetubaguy> lxsli: happy to follow up with folks on how to get the blueprint approved
14:38:57 <lxsli> ralonsoh: ^^
14:39:00 * bauzas not joking about the whole pack of holidays that US folks have
14:39:02 <lxsli> bauzas: OK thanks
14:39:07 <n0ano> johnthetubaguy, I though the BP wasn't approve because there wasn't a spec - I'm so confused
14:39:24 <lxsli> n0ano: the spec got -2 for "should be specless BP"
14:39:47 <edleafe> bauzas: Thursday is the holiday. Wife and daughter have the whole week off.
14:40:04 <bauzas> :)
14:40:05 <n0ano> lxsli, but at the last meeting the result was needs a spec
14:40:07 <johnthetubaguy> n0ano: basically the same process as the last two cycles, as documented here: https://wiki.openstack.org/wiki/Nova/Process#How_do_I_get_my_blueprint_approved.3F
14:40:54 <bauzas> well, the problem I had with that filters wasn't code-related, but rather a problem about whether that filter was necessary or not
14:41:03 <bauzas> (IIRC)
14:41:45 <bauzas> so I don't know if that's a specless BP, but I still have some concerns about if all of that is okay or not :)
14:41:46 <ralonsoh> bauzas that filter give the admin more choices
14:42:19 <edleafe> bauzas: are you saying that the filtering need can be met with the existing filter(s)?
14:42:20 <ralonsoh> bauzas but thanks to give a second review
14:42:26 <bauzas> edleafe: perhaps
14:42:48 <lxsli> even if it can't quite, might be better to extend the existing filter than create a whole new one
14:42:50 <bauzas> ralonsoh: the problem that I have is that we really suck (IMHO :D) about what's a good filter UI
14:42:58 <bauzas> well, not UI, rather UX
14:43:09 <bauzas> ie. how we define the interaction between the user and the filter
14:43:35 <bauzas> for example we don't fully commit ourselves on verifying the semantics overlapping between filters
14:43:52 <ralonsoh> that's fair enough
14:43:55 <lxsli> for in-tree filters I definitely don't want duplication
14:44:07 <bauzas> and as a consequence, they are 2 filters that are using the same metadata key for example
14:44:11 <edleafe> lxsli: duplication is one thing; overlap is another
14:44:24 <bauzas> that's very confusing for operators
14:44:45 <lxsli> edleafe: duplication of functionality I mean, same as overlap afaict?
14:44:46 <bauzas> lots of them are complaining on how we suck about filters documentation
14:45:05 <bauzas> for all of us, did you ever tried to play with a filter without reading code?
14:45:16 <n0ano> bauzas, too true
14:45:18 <edleafe> lxsli: not necessarily. Everything is intertwined to some degree :)
14:45:36 <bauzas> so, that's my biggest concern with that new filter
14:45:40 <bauzas> I mean
14:45:45 <edleafe> bauzas: +1 to not having to read the code
14:45:46 <bauzas> I totally got the usecase
14:45:49 <lxsli> Time to take this offline?
14:45:54 <johnthetubaguy> bauzas: similar to the REST API and config options right now, sadly, its a big issue we must fix very soon :(
14:46:07 <johnthetubaguy> clearer documentation, that is
14:46:22 <bauzas> so, I said in the precedent meeting that I have an action about providing some bits for newcomers
14:46:25 <bauzas> and I sucked at that too
14:46:26 <n0ano> johnthetubaguy, why do I have visions of the Agean stables
14:46:57 <bauzas> so, you should rather hassle me for fulfilling my action items and do the necessary for having people able to write docs
14:47:09 <johnthetubaguy> at least one of our priorities this release is to improve our REST API docs, and thats the interface we have the most users for, so some progress is good
14:47:57 <bauzas> johnthetubaguy: true, but the scheduler filters are out of the doc scope that has been prioritized, true?
14:48:32 <johnthetubaguy> bauzas: do you mean this stuff, or something different? https://wiki.openstack.org/wiki/Nova/Mentoring
14:48:39 <bauzas> johnthetubaguy: http://docs.openstack.org/developer/nova/filter_scheduler.html
14:48:44 <lxsli> bauzas: yes, but there's nothing stopping us improving filter doc meanwhile
14:48:46 <bauzas> johnthetubaguy: ^ that has to vhange
14:49:05 <johnthetubaguy> its a priority call, REST API has more users
14:49:10 <johnthetubaguy> I hope
14:49:10 <n0ano> I think we're rambling a bit, let's take anything else to the nova channel
14:49:25 <Yingxin> https://review.openstack.org/#/c/246476/5 host manager is already implemented to use stevedore :)
14:49:28 <bauzas> lxsli: true, and see 4a) here http://eavesdrop.openstack.org/meetings/nova_scheduler/2015/nova_scheduler.2015-11-16-14.01.html
14:49:39 <bauzas> johnthetubaguy: sure
14:49:48 <lxsli> bauzas: :)
14:49:52 <n0ano> So tnx everyone and we'll talk again next week
14:49:56 <lxsli> cheers all o/
14:49:58 <n0ano> #endmeeting