13:00:04 <alex_xu> #startmeeting nova api
13:00:05 <openstack> Meeting started Wed Feb  8 13:00:04 2017 UTC and is due to finish in 60 minutes.  The chair is alex_xu. Information about MeetBot at http://wiki.debian.org/MeetBot.
13:00:06 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
13:00:09 <openstack> The meeting name has been set to 'nova_api'
13:00:11 <alex_xu> who is here today?
13:00:11 <cdent> o/
13:00:35 <gmann> o/
13:01:19 <alex_xu> cdent: gmann let us wait johnthetubaguy and sdague if they are around
13:01:20 * johnthetubaguy waves
13:02:26 <alex_xu> let us start the meeting
13:02:33 <alex_xu> #topic Pike ideas
13:02:42 <alex_xu> #link https://etherpad.openstack.org/p/nova-api-pike
13:03:08 <alex_xu> ^ I created an etherpad to collect the idea for Pike
13:03:29 <gmann> nice, thanks
13:03:48 <johnthetubaguy> did we want to discussion the policy ideas, I am curious what folks think
13:03:53 <alex_xu> Matt helps to input some items
13:03:57 <johnthetubaguy> also big thank you to alex_xu for reviewing that
13:04:11 <alex_xu> johnthetubaguy: np
13:04:26 <alex_xu> sure, we can discuss the policy ideas
13:04:35 <gmann> yea
13:04:53 <johnthetubaguy> so operator feedback first
13:04:54 <johnthetubaguy> most folks struggle editing policy right now
13:05:17 <johnthetubaguy> many of them are forced into re-writing all rules because the want more RBAC levels than we have in the default policy file
13:05:46 <johnthetubaguy> my thinking is we define a rough end goal for where we want to be, and get that reviewed by operators
13:05:59 <johnthetubaguy> then we go through each rule, one by one, and make it match the plan
13:06:11 <johnthetubaguy> ...so to the "plan" I have suggested
13:06:26 <alex_xu> 'reviewed by operators' means sending email to operator ML?
13:06:33 <johnthetubaguy> basically we add more roles, like observer roles, roles for neutron and cinder, operator rules, etc
13:06:45 <johnthetubaguy> alex_xu: hopefully means talking to them at an ops meetup, at a minimum
13:06:57 <johnthetubaguy> alex_xu: but thats part of it
13:07:01 <gmann> i see but there might be lot of roles depends on each operator
13:07:06 <johnthetubaguy> I really want something we agree on first
13:07:07 <gmann> or we add common one
13:07:15 <johnthetubaguy> so there are two things here
13:07:21 <johnthetubaguy> the name of the role, and the common patterns
13:07:35 <johnthetubaguy> if we can cover 80% of deployments with the default, thats great
13:07:55 <johnthetubaguy> folks will just need to say what the role name is in their keystone, and we take care of the rest, for the 80%
13:08:08 <johnthetubaguy> for the 20% we need to add doc strings on each of the rules, so they know what they control
13:08:18 <sdague> o/
13:08:38 <johnthetubaguy> basically, my idea is stop most folks having to change much of the policy file
13:08:53 <alex_xu> whether the name of role need to get more widly agreement, so other project can use the same name
13:09:02 <johnthetubaguy> but also give folks who need to change everything (or audit it) a bit of a helping hand (i.e. no need to read code to work out what each rule means)
13:09:34 <johnthetubaguy> alex_xu: we actually we need it to be project specific, like nova_operator etc, but yes common patterns are good
13:09:46 <johnthetubaguy> so I am talking to keystone folks a bit about this, they have started a policy meeting
13:10:01 <johnthetubaguy> they have a proposal for some roles we could create
13:10:02 <sdague> johnthetubaguy: can you be more specific on folks that are changing policy so much?
13:10:30 <johnthetubaguy> sdague: I think they want observer roles, and operator roles that are not admin everywhere
13:10:32 <sdague> johnthetubaguy: yeh, I honestly thought we had 5 roles decided on back in tokyo
13:10:41 <johnthetubaguy> sdague: I am hoping we can get access to some specific ones
13:10:57 <johnthetubaguy> sdague: yeah, its basically those
13:11:22 <sdague> ok, what stands between those hallway conversations and just doing it?
13:11:38 <johnthetubaguy> upgrade story, the spec I wrote out tried to cover that
13:12:12 <johnthetubaguy> I think I have the OSIC folks who are willing to do the work to implement it
13:12:32 <sdague> johnthetubaguy: ok, so... every time I brought this up at ops meetups, people mostly didn't change much in policy
13:12:32 <johnthetubaguy> kinda a follow on from config work that was related
13:12:33 <alex_xu> ++OSIC folks
13:12:53 <johnthetubaguy> sdague: I was told they tried, it failed, and so gave up
13:13:02 <sdague> johnthetubaguy: I guess I need to understand the spec more
13:13:09 <johnthetubaguy> at the manchester meetup
13:13:38 <sdague> maybe the issue is the following, today, we do policy as "policy_point: can be done by this complicated formula of things"
13:13:49 <sdague> vs the way people would think about things:
13:13:58 <sdague> giant list of policy points...
13:14:19 <sdague> observer_can: p1, p2, p4, p8
13:15:04 <johnthetubaguy> thats a good point, this is still reversed in my spec
13:15:10 <sdague> and... it almost seems like you could add role stanzas like that as an overlay to the existing content
13:15:12 <johnthetubaguy> p1: observer and admin can, etc
13:15:23 <sdague> so that the current stuff is just the current stuff
13:15:31 <johnthetubaguy> sdague: yeah, it should be equivalent
13:15:32 <sdague> and the new stuff is new stanzas
13:15:49 <sdague> which... makes the upgrade a lot more viable for users, right?
13:16:28 <johnthetubaguy> maybe... need more visualization on it
13:17:00 <sdague> johnthetubaguy: ok
13:17:04 <johnthetubaguy> I was also talking about renaming the rules (via deprecation aids) to better match the REST API rather than the code
13:17:16 <sdague> yeh, which I'm all for
13:17:28 <sdague> but if there is a different upgrade issue, we should probably address that
13:17:54 <johnthetubaguy> maybe the staging is (1) fix names and docs, (2) look at making what each role does more visible (3) think about adding extra roles
13:18:25 <johnthetubaguy> it was mostly the "we need to evolve like config" upgrade issue that I was referring too
13:18:58 <sdague> johnthetubaguy: ok cool
13:19:17 <sdague> sorry, rewrapping my head on the whole thing
13:19:36 <johnthetubaguy> sdague: totally, been going through that a bit myself
13:20:05 <johnthetubaguy> FWIW, we really have made a lot of progress at this point
13:20:24 <johnthetubaguy> no policy file needed by default, defaults in the code, reduced the number of rules by half
13:20:30 <sdague> yeh
13:20:37 <johnthetubaguy> we are in a much better place to move forward now, which is kinda awesome
13:20:39 <sdague> there is still a docs gap though, right?
13:20:54 <sdague> pretty much the policy rules only make sense if you go and read the code
13:20:55 <johnthetubaguy> yeah, and a naming to avoid the docs having to say everything
13:20:58 <johnthetubaguy> yeah
13:21:06 <sdague> well, the docs should be there anyway
13:21:13 <sdague> names are never as obvious as you think :)
13:21:27 <johnthetubaguy> yeah, I tried it, its hard
13:21:54 <gmann> doc for rules will be as description right ? so that it can be seen from sample file like config one
13:22:03 <sdague> yeh
13:22:08 <johnthetubaguy> my current leaning is towards compute:servers:action:live-migration
13:22:20 <johnthetubaguy> gmann: turns out we have two doc strings in our sample file already
13:22:34 <gmann> yea :)
13:22:40 <sdague> johnthetubaguy: do we have anything published web side?
13:22:44 <johnthetubaguy> #link http://docs.openstack.org/developer/nova/sample_policy.html
13:23:08 <sdague> um... I wouldn't call that docs :)
13:23:26 <johnthetubaguy> I mean two of them have strings
13:23:30 <sdague> What I really expect is:
13:23:41 <johnthetubaguy> but its totally not docs
13:23:43 <sdague> - os_compute_api:os-aggregates:set_metadata
13:24:18 <sdague> this allows the roles in question to set metadata on aggregates.
13:24:20 <gmann> how about embeding policy rule inside api-ref for corresponding APIs
13:24:34 <johnthetubaguy> sdague: I was meaning the rule near os_compute_api:os-attach-interfaces:create has added some docs already
13:24:36 <sdague> default_permissions: admin role only
13:24:48 <sdague> ok
13:24:53 <johnthetubaguy> gmann: thats mixing operator and API user though
13:25:04 <sdague> it's not api-ref for sure
13:25:06 <johnthetubaguy> yeah, we need to render the default better, like the config stuff
13:25:09 <gmann> ah yea
13:25:14 <sdague> but it should end up with/near the config guide
13:25:30 <johnthetubaguy> but we all seem agreed with the same as config, you shouldn't need to read the code to understand it
13:25:35 <sdague> yeh
13:25:55 <gmann> yes
13:26:16 <johnthetubaguy> if we go and add docs, I am tempted to consider a rename, but maybe that is premature?
13:26:33 <sdague> johnthetubaguy: honestly, we can document things first, that would be of immediate help to people
13:26:41 <johnthetubaguy> sdague: thats fair
13:26:52 <johnthetubaguy> I was wanting to avoid visiting all the rules twice
13:26:54 <sdague> I'd rather get us in the habbit of writing down what is instead of assuming we can make a change that means we don't need docs
13:26:56 <johnthetubaguy> but we have less rules now
13:27:15 <johnthetubaguy> sdague: yeah, we totally need the docs, even with the most perfect of names
13:27:30 <sdague> do we have other topics? I don't want to take over the whole meeting with this
13:27:49 <johnthetubaguy> yeah, I think we should review the other ideas
13:27:53 <alex_xu> other items in https://etherpad.openstack.org/p/nova-api-pike
13:28:18 <johnthetubaguy> well, before we leave...
13:28:25 <johnthetubaguy> I can add a spec for adding in the docs
13:28:34 <johnthetubaguy> so folks stop needing to read the code
13:28:48 <sdague> johnthetubaguy: that would be cool, I would be up for helping get that all sorted
13:28:57 <johnthetubaguy> sdague: cool, thanks
13:29:10 <johnthetubaguy> I think the next step might be making the rules mean something (actually adding targets)
13:29:32 <johnthetubaguy> like admin_or_owner is largely a no op today
13:29:40 <johnthetubaguy> because of the deafault rule target
13:29:48 <johnthetubaguy> we are safe due to the hardcoded DB checks
13:30:09 <johnthetubaguy> it feels like we should add in the targets, then consider removing the hard coded things once we are happy with our unit tests
13:30:15 <johnthetubaguy> but thats a separate discussion
13:30:19 <sdague> johnthetubaguy: I thought the hardcoded db checks were mostly gone
13:30:37 <johnthetubaguy> sdague: they are gone for is_admin, they are everywhere for membership in the project
13:30:54 <sdague> oh, you mean owner
13:31:06 <johnthetubaguy> sorry, yeah, owner
13:31:20 <johnthetubaguy> that is not checked in the policy later, even though the rule says its checking
13:31:28 <sdague> right
13:31:39 <johnthetubaguy> we just check context.project_id == context.project_id right now, which is amuzing
13:32:04 <sdague> heh
13:32:15 <sdague> yeh, well the project scoping like that goes back years
13:32:18 <johnthetubaguy> although there is a separate discussion about that being a good idea, as it keeps policy from making the API interoperable
13:32:33 <johnthetubaguy> so we totally said we should move on lol, we should do that...
13:33:28 <sdague> :)
13:33:38 <johnthetubaguy> turns out that was a conversation stopper
13:33:44 <sdague> what are team members excited about for the next cycle?
13:33:50 <sdague> what would they most like to solve
13:34:20 <johnthetubaguy> https://review.openstack.org/#/c/386555/ on capabilities would be awesome
13:34:47 <gmann> yea that is nice to do
13:34:53 <johnthetubaguy> query param schema excites my less, but would be nice
13:35:16 <sdague> do we feel like capabilities can come to a resolution on design for the PTG?
13:35:22 <alex_xu> query param schema is just a refactor, so it can be background work for new people
13:35:27 <alex_xu> so +1 for capabilities
13:35:41 <gmann> and cleanup extension things as sdague started, make code more easy
13:36:13 <johnthetubaguy> sdague: so turns out we didn't have many people from keystone or API-wg in the summit session, if we can fix that, maybe?
13:36:24 <sdague> I guess the concern I have about capabilities... it relies a lot on link following
13:36:34 <sdague> do we know of any sdk tooling that's regularlly doing that?
13:37:04 <johnthetubaguy> there is a proposal to remove the link following, for the global scope and per type scope
13:37:10 <johnthetubaguy> that might be a good idea
13:37:16 <sdague> spending some time poking on some other APIs recently, they tend not to expose anything on their API until the SDK also implements it
13:37:43 <johnthetubaguy> so I keep thinking about to horizon and enabling buttons in the GUI
13:37:54 <sdague> johnthetubaguy: sure, but they aren't writing their own client
13:38:10 <sdague> so... I'm fine exposing capabilities
13:38:11 <johnthetubaguy> like ironic and libvirt based servers, choose which turn on
13:38:18 <sdague> I think it's hugely needed
13:38:31 <johnthetubaguy> sdague: they do, I was more thinking efficient calling patterns down that thought path
13:38:38 <johnthetubaguy> link follwing is expensive
13:38:58 <sdague> I mostly get concerned that it is useful only if we have sdk drops with the functions
13:39:26 <johnthetubaguy> thats true, would this be adaptive help in the SDK?
13:39:51 <johnthetubaguy> I guess I am struggling on the consuming part, if I don't think about horizon
13:40:49 <sdague> yeh, I don't know.
13:40:52 <cdent> who were the original people or group that said "we need capabilities"?
13:41:04 <sdague> cdent: definitely people making guis
13:41:30 <johnthetubaguy> ironic snapshots always fail, why not hide the button, etc
13:41:38 <johnthetubaguy> the other side was live-resize
13:41:52 <johnthetubaguy> we expect folks to turn that off, and want that to be discoverable when talking to multiple clouds
13:42:11 <johnthetubaguy> but thats a little bit of a vanity use case, in some ways
13:42:22 <cdent> in the stuff I've read, that particular use case is not made as clear as it potentially could be. Instead we end up with discussions on how to make capabilities as (uh) capable as possible
13:42:50 <sdague> I know that mordred ended up basically writing crazy autoconf like stuff where he tries a bunch of things in shade because there is no way to figure out that he can't do a thing in advance
13:42:58 <sdague> and doesn't want to fail deep in complicated logic
13:43:13 <sdague> this is mostly about OpenStack clouds could do A
13:43:33 <sdague> but some clouds turn off B, C, D for performance, security, hypervisor support reasons
13:43:51 <johnthetubaguy> right, the how do I get an external IP on my server question
13:44:47 <sdague> cdent: so perhaps the issue is framing the use case a bit better
13:44:52 <johnthetubaguy> maybe shade is the SDK use case
13:45:00 <sdague> the way I imagined this was more along the fact that
13:45:33 <cdent> as a user I'd prefer to ask a "capabilities service" for that information, not go grubbing around in all the projects at several urls. perhaps a service could register it's capabilities at startup or regularly (not deployment)
13:46:23 <sdague> cdent: yeh
13:47:02 <sdague> cdent: that works for top level constructs for sure. It doesn't solve the "this cloud has baremetal and libvirt computes"
13:47:11 <sdague> and this compute you have can't be live migrated
13:47:47 <cdent> I think anything that can be started as "this cloud..." ought to be able to be "registered"
13:48:02 <sdague> cdent: sure
13:48:13 <cdent> and I think anything that says "this <entity, like this compute>" should be in the representation of that entity itself
13:48:28 <sdague> cdent: yep, no disagreement there
13:48:53 <johnthetubaguy> would aggregated centrally work better, i.e. exposed in each service, but also aggregated (via checking everything in the service catalog, etc), or does that mean we do the work twice?
13:48:56 <sdague> ok... so where does that leave us, because this seems circular
13:49:08 <sdague> which probably means the thing being carved off is too big
13:49:17 <sdague> and maybe it can be started as something smaller
13:50:02 <johnthetubaguy> maybe per endpoint global, and per endpoint type only?
13:50:15 <johnthetubaguy> leave the aggregation / central piece for later?
13:50:50 <sdague> or, find a specific problem, like a chunk of code in shade or horizon we'd like to go away, and work backwards on solution to that
13:50:52 <johnthetubaguy> stop the per resource capabilitiy thing at first, just go for the shade and horizon use cases first
13:51:29 <sdague> johnthetubaguy: yeh
13:51:38 <johnthetubaguy> so horizon just shows all buttons based on VM state, and shade just hard codes
13:51:51 <johnthetubaguy> so it would be improving both of those to some degree I guess
13:52:00 <johnthetubaguy> maybe just do the horizon one to start with?
13:52:14 <johnthetubaguy> ironic and VM mixed cloud
13:52:17 <johnthetubaguy> get the buttons right
13:52:26 * alex_xu reminds 8 mins left
13:52:34 <johnthetubaguy> (for user operations)
13:52:50 <sdague> cdent: from an api-wg perspective, how would you like to / expect to constrain scope so we can get feedback and iterate on something real?
13:54:15 <cdent> sdague: that's an excellent question. I don't really have a good answer. I agree with you that the current scope is too large, and have said as much on the ongoing review of the capabilities spec. In the context of the api-wg what matters is what a capability looks like and where it is retrieved from
13:54:52 * mordred waves
13:54:55 <cdent> (and of course that whatever it is, everyone is doing the same thing)
13:55:08 <johnthetubaguy> cdent: OK with doing it per endpoint first, as s stepping stone towards centralised?
13:55:44 <cdent> johnthetubaguy: I think that's probably the only way it is likely to happen, but I don't like it that much because it means it will never truly be centralised :)
13:56:00 <mordred> sdague, cdent: so, for several of these things (although admittedly not enough of them) we have flags in os-client-config so that discovery code can be skipped for known clouds
13:56:29 * mordred reminds self to go make a "this cloud requires floating ips" flag
13:56:36 <sdague> mordred: code reference?
13:56:41 <mordred> one sec ...
13:56:56 <sdague> I think that shade might be more concrete than horizon for these discovery routines
13:56:57 <johnthetubaguy> cdent: yeah, I am with you on that
13:57:32 <johnthetubaguy> sdague: assuming they are not harder questions to answer, +1
13:57:42 <mordred> sdague: line 14 and line 16 are both concrete examples:  http://git.openstack.org/cgit/openstack/os-client-config/tree/os_client_config/defaults.json#n14
13:57:44 <sdague> cdent: I think that if we can convince keystone folks that more things should flow into service catalog, there is a centralization plan
13:57:53 <cdent> sdague: yes
13:58:43 <johnthetubaguy> I think we could argue thats more useful than exposing policy, and so it might be what they are wanting to do
13:58:45 <sdague> cdent: so, I think that's the centralizing plan
13:59:09 <sdague> mordred: it makes me slightly sad that all the api versions are hardcoded in shade
13:59:14 <alex_xu> ok, we need to close the meeting now
13:59:16 <mordred> sdague: those are just defaults
13:59:17 <sdague> sure
13:59:23 <alex_xu> we should back to nova channel
13:59:30 <mordred> sdague: they're overridable in config - and they also can be auto-discovered
13:59:31 <sdague> mordred: ah, ok
13:59:32 <mordred> sdague: but yes
13:59:42 <alex_xu> and johnthetubaguy sdague there is patch looking for feedback https://review.openstack.org/430497, that is also mordred faced problem
13:59:52 <mordred> sdague: like, if your config says "image api version 2" and the cloud only has 1, we'll use 1 but give you a warning
14:00:00 <alex_xu> #endmeeting