16:01:08 <lbragstad> #startmeeting policy
16:01:08 <openstack> Meeting started Wed Feb 15 16:01:08 2017 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:12 <openstack> The meeting name has been set to 'policy'
16:01:14 <lbragstad> ping antwash, raildo, ktychkova, dolphm, dstanek, rderose, htruta, atrmr, gagehugo, lamt, thinrichs, edmondsw, ruan, ayoung, stevemar, ravelar, morgan, raj_singh
16:02:00 <johnthetubaguy> o/
16:02:04 <lbragstad> johnthetubaguy o/
16:02:04 <lamt> o/
16:02:25 <rderose> o/
16:02:44 <lbragstad> we will wait for a few others to show up
16:03:10 <lbragstad> who's excited for next week?!
16:03:30 <raj_singh> o/
16:03:33 <edmondsw> o/
16:03:52 <gagehugo> o/
16:03:58 <lbragstad> there we go - now we're getting some people
16:04:06 <antwash> o/
16:04:11 <dstanek> o/
16:04:23 <gagehugo> this meeting always sneaks up on me
16:04:32 <edmondsw> gatehugo +1
16:04:45 <lbragstad> #topic PTG Policy Meeting
16:05:00 <lbragstad> so it looks like we were able to find a time to get a couple projects together to talk at the PTG
16:05:12 <lbragstad> specifically nova, cinder, and keystone
16:05:20 <lbragstad> we're going to meet on Thursday 1:30 - 2:30 PM in South Capital (level 1)
16:05:33 * lbragstad is still in the middle of scheduling sessions
16:05:57 <lbragstad> but I plan to send out something a little more official along with a dedicated etherpad by the end of the week (at the absolute latest)
16:06:26 <lbragstad> I wanted to bring it up here so that folks could get it on their calendars if they are planning to participate in that discussion
16:06:36 <edmondsw> will plan on it
16:06:42 <lbragstad> any questions on the time or the place?
16:07:21 <ayoung> lbragstad, let me know if you can get remote presence
16:07:27 <lbragstad> ayoung ack
16:07:55 * lbragstad sticks a post-it on his monitor
16:08:07 <lbragstad> alright - moving on
16:08:08 <lbragstad> #topic Review/discuss policy specs
16:08:18 <lbragstad> #link https://review.openstack.org/#/c/433010 (nova-specs: Add policy-docs spec)
16:08:18 <lbragstad> #link https://review.openstack.org/#/c/433037 (nova-specs: Add policy-remove-scope-checks spec)
16:08:18 <lbragstad> #link https://review.openstack.org/#/c/427872 (nova-specs: Add additional-default-policy-roles spec)
16:08:18 <lbragstad> #link https://review.openstack.org/#/c/428453 (keystone-specs: Policy in code)
16:08:39 <lbragstad> so - johnthetubaguy has been doing a bunch of work in nova-specs that outline what they are trying to do
16:09:06 <dstanek> lbragstad: are these part of what we'll be discussing at the PTG?
16:09:13 <lbragstad> dstanek yeah
16:09:59 <lbragstad> those are likely going to be required reading before the session
16:10:35 <lbragstad> johnthetubaguy do you have anything in particular you want to discuss about any of those?
16:10:48 <johnthetubaguy> the remove scope checks one is interesting
16:10:57 <lbragstad> johnthetubaguy i know you recently reworked a couple of them based on the outcomes of last weeks meeting
16:11:00 <johnthetubaguy> idea came from dstanek's comment
16:11:36 <johnthetubaguy> basically I am proposing we remove the use of target from all our policy checks
16:11:40 <johnthetubaguy> roughly speaking
16:11:44 <ayoung> johnthetubaguy, you are aware of the work I was doing to fix is_admin, right?
16:12:02 <johnthetubaguy> ayoung: roughly yeah, its very similar idea I think
16:12:13 <ayoung> johnthetubaguy, can you take that and fix the whole thing?
16:12:26 <ayoung> johnthetubaguy, https://review.openstack.org/#/c/384148/
16:12:36 <ayoung> it needs changes to the Tempest tests in order to pass
16:12:49 <johnthetubaguy> oh, didn't know there was a patch out there
16:12:55 <johnthetubaguy> so if tempest fails, we broke the API right, so thats really bad surely?
16:13:10 <edmondsw> no, just bad tests
16:13:14 <ayoung> johnthetubaguy, nope
16:13:20 <ayoung> it is just test assumptions:
16:13:27 <ayoung> that admin can be in any random project
16:13:43 <johnthetubaguy> so my proposed change is quite different to your proposal
16:13:45 <ayoung> and with that patch, and enforcement turned on, admin needs to be in the admin_project
16:13:48 <ayoung> bad tests
16:14:15 <ayoung> johnthetubaguy, I know.  I am not working on Keystone full time anymore. If you don't take it and run with it, it is not going to happen
16:14:29 <ayoung> and without nova support, nothing happens in OpenStack
16:14:31 <edmondsw> ayoung I'm still trying to find time to push that patch
16:14:46 <edmondsw> but could definitely use some nova core help
16:15:00 <johnthetubaguy> so lets turn that around a second
16:15:11 <johnthetubaguy> if we implemented it the way proposed here: https://review.openstack.org/#/c/433037
16:15:31 <johnthetubaguy> I believe that also fixes the bug you are trying to fix in the above patch (and fixes a few other gremlins too)
16:15:56 <ayoung> johnthetubaguy, um...does it fix it, or just push it around?
16:16:05 <johnthetubaguy> I believe it fixes it
16:16:15 <johnthetubaguy> if it doesn't thats great feedback to get
16:16:26 <johnthetubaguy> probably means I miss understood the bug
16:16:27 <ayoung> johnthetubaguy, does not look like it.  But the solution could be rewritten in terms of that spec
16:16:31 <edmondsw> johnthetubaguy I'll add that to my reading list and let you know :)
16:16:48 <ayoung> johnthetubaguy, there are 2 types of admin checks
16:16:49 <johnthetubaguy> ah, so maybe its this spec that fixes it all together
16:16:53 <ayoung> project scoped and global
16:16:55 <johnthetubaguy> https://review.openstack.org/#/c/427872
16:17:14 <lbragstad> i've review a couple of them ( #link https://review.openstack.org/#/c/433010 is really straight forward)
16:17:17 <ayoung> johnthetubaguy, yes, that looks better
16:17:29 <johnthetubaguy> basically doing the fix in two steps
16:17:44 <johnthetubaguy> forgot how I split that up, oops
16:17:48 <ayoung> johnthetubaguy, there was a cross project spec along those lines years ago.  I think you are on the right track
16:17:54 <lbragstad> ayoung ++
16:18:00 <johnthetubaguy> yeah, its using lots of stuff from that one
16:18:11 <ayoung> lbragstad, meanwhile, the Keystone patches are also sitting there...
16:18:44 <ayoung> https://review.openstack.org/#/c/387161/
16:18:48 <lbragstad> ayoung I need to finish reading johnthetubaguy final proposal (I ran out of battery last night)
16:18:51 <ayoung> https://review.openstack.org/#/c/387710/
16:18:56 <ayoung> and then
16:19:35 <ayoung> https://review.openstack.org/#/c/257636/16
16:21:17 <ayoung> lbragstad, this is very tactical, practical, and acheiveable goals.  I am not going to continue to update the patches. Merge them, or abandon them as you see fit.  Let someone else hack on them...I'm, willing to review
16:21:36 <lbragstad> ayoung added them to my list
16:21:39 <ayoung> but this is pre-req to any future policy work being on a sound footing
16:21:53 <lbragstad> we also have #link https://review.openstack.org/#/c/428453 (keystone-specs: Policy in code)
16:22:01 <lbragstad> which ravelar and antwash have spoken for
16:22:04 <lbragstad> (thanks guys)
16:22:30 <antwash> lbragstad : you're welcome ++
16:22:35 <ayoung> lbragstad, there are comparable patches for Glance and Cinder.  But, if johnthetubaguy is going to redo Nova's approach, perhaps we wait to see wha he learns before redoing those patches
16:22:55 <lbragstad> ayoung totally on board there
16:23:17 <johnthetubaguy> so I was also adding the nova-member thing
16:23:24 <lbragstad> i'm excited to get into these discussions at the PTG to see what the consensus is *as a group*
16:23:42 <johnthetubaguy> so folks by default have zero access to nova
16:24:07 <lbragstad> johnthetubaguy by access do you mean write-access?
16:24:17 <johnthetubaguy> I mean *any* access
16:24:24 <johnthetubaguy> so no access by default
16:24:27 <ayoung> antwash, I'd recommend grabbing johnthetubaguy 's follow on specs and doing a keystone version of them as well.  Let Nova set the approach for Policy enforcement makes it easier to implement consistently in the other projects
16:24:35 * lbragstad is a fan of deny-all by default
16:24:57 <johnthetubaguy> now, there might be a big hole in my proposal somewhere
16:25:10 <ayoung> johnthetubaguy, define "Default" here?  An unscoped token should be rejected by Nova.
16:25:16 <johnthetubaguy> it just feels like the current best bet
16:25:23 <johnthetubaguy> ayoung: good question, if you have no roles assigned
16:25:39 <johnthetubaguy> so with no roles assigned, you get no access
16:25:42 <ayoung> johnthetubaguy, no role mean you cannot get a token scoped to that project
16:25:54 <ayoung> but a second check is not going to hurt
16:25:55 <johnthetubaguy> you are scoped to a project, thats cool
16:26:07 <johnthetubaguy> but then you can't do anything, becuase you don't have a role in that project
16:26:12 <ayoung> johnthetubaguy, I have a follow on proposal.
16:26:24 <edmondsw> johnthetubaguy I think you mean you can't just have any role assigned (the current state), but rather have to have a role that nova has allowed to do something
16:26:44 <lbragstad> edmondsw ++
16:26:44 <edmondsw> no?
16:26:53 <ayoung> johnthetubaguy, looking for the spec...
16:27:01 <johnthetubaguy> edmondsw: oh, right, I was missing you need a role to be "in" a project, so yet
16:27:09 <edmondsw> cool
16:27:17 <ayoung> johnthetubaguy, http://specs.openstack.org/openstack/keystone-specs/specs/keystone/ongoing/role-check-from-middleware.html
16:27:22 <johnthetubaguy> I keep meaning to do a spec on supporting domains as well
16:27:27 <ayoung> move the role check into middleware
16:27:29 <lbragstad> and right now the evaluation is *any* role on a project implies membership (right?)
16:27:40 <edmondsw> johnthetubaguy what do you mean when you say "supporting domains"?
16:27:40 <johnthetubaguy> ayoung: yeah, our API is too screwed up for that to work, unless I am missing something
16:27:49 <ayoung> No, it should work
16:27:54 <johnthetubaguy> edmondsw: we just ignore domain right now
16:27:58 <ayoung> johnthetubaguy, the scope check stays in code
16:28:14 <johnthetubaguy> ayoung: we do policy based on random things in the body of the API payload
16:28:17 <ayoung> johnthetubaguy, Oh, wait, you mean the "pack everything into one post call"  API
16:28:21 <edmondsw> johnthetubaguy, I know, but that's true in a lot of senses... and some of them rightly... which ones are you thinking of addressing?
16:28:34 <ayoung> johnthetubaguy, yeah...so you can still make use of the mechanism.
16:28:55 <johnthetubaguy> we could do a basic "can I access Nova at all" check
16:29:03 <ayoung> you just have to manually implement that same logic:
16:29:14 <ayoung> instead of the URL being the URL, it becomes some magic string
16:30:06 <lbragstad> so nova would have to maintain the mapping?
16:30:19 <johnthetubaguy> the problem is I want policy checks to be simpler and easier to audit, I think having to do two levels of checks is going to bad, but I could be missing this
16:30:26 <lbragstad> because we coded to store a URL?
16:30:51 <johnthetubaguy> we could create some "magic" to make action work, for sure
16:32:57 <ayoung> johnthetubaguy, I want the same things
16:33:09 <ayoung> the scope check should be immutable
16:33:15 <johnthetubaguy> anyways, lets step back, right now I am trying to work through a set of deployer problems really, much less worried about the how, open to options, we should look at them at the PTG
16:33:15 <ayoung> all end users can do is break that
16:33:25 <ayoung> the only part people should be morphing is the role->api mapping
16:33:37 <ayoung> johnthetubaguy, it supports your spec
16:33:45 <johnthetubaguy> ah, OK, I missed that being the aim
16:33:58 <ayoung> this will allow you to get the granularity of readonly/readwrite/projectadmin etc
16:34:12 <ayoung> johnthetubaguy, so, here is how I see it working
16:34:30 <ayoung> we get the proposed mechanism built and deployed. but nothing changes on day 1
16:34:48 <ayoung> if a user has Member, oon a projhect, every thing works the same
16:34:59 <ayoung> admin iplies member, so admin on a project can still do everything
16:35:17 <ayoung> then, we change member implies read_only
16:35:34 <ayoung> that starts showing up in the tokens...nothing changes
16:35:45 <ayoung> we modify a set of APIs to be read_only....
16:35:49 <ayoung> everything keeps working.
16:36:08 <ayoung> now....we create a new users, and, instead of granting them Member, we grant them "read_only"
16:36:17 <ayoung> now that use can only do the read_only APIs, not all of them
16:36:30 <ayoung> the process is backwards compatble
16:37:08 <johnthetubaguy> yeah, I think thats basically what I am shooting for in my spec, it falls back to use the old roles for a cycle or so
16:37:25 <johnthetubaguy> with some warnings if haven't updated your users roles to match the "future"
16:37:46 <ayoung> johnthetubaguy, note that implied_roles are already in keystone
16:38:40 <ayoung> so, the big thing is to make sure that Nova only enforces the current standard.  If you go too far, you ruin it for everyone
16:38:50 <ayoung> don't bake the roles into code.  Only scope check
16:38:58 <ayoung> admin we treat as a special
16:39:40 <johnthetubaguy> so not quite sure if thats what we have done
16:39:56 <edmondsw> but scope checks should be baked into code for the most part. For a select few APIs where we want scope to be more malleable, have an additional policy check to control that, but should be the exception
16:39:56 <johnthetubaguy> the stuff we have in the code is more about the default policy
16:40:32 <johnthetubaguy> edmondsw: yeah, there are a few exceptions, we are trying to slowly kill those, in the name of interop
16:40:38 <edmondsw> ++
16:40:45 <ayoung> johnthetubaguy, right, and the default should be "you need a token with a project that matches"
16:40:55 <johnthetubaguy> edmondsw: FWIW, we need to fix hierarchical multi-tenancy to get rid of those
16:40:58 <ayoung> and, for some "you need admin on the admin_project"
16:41:10 <edmondsw> oh?
16:41:16 <johnthetubaguy> yeah, I don't like that
16:41:30 <johnthetubaguy> it means you get access to everywhere by default
16:41:36 <johnthetubaguy> so I must have missed something
16:41:45 <lbragstad> i think by default we should have a granular set of roles that are well-defined
16:42:16 <johnthetubaguy> so "by default" is causing confusion here
16:42:20 <johnthetubaguy> I think what I mean is...
16:42:30 <johnthetubaguy> when in a project, you don't automatically get access to Nova
16:42:50 <johnthetubaguy> you can do implied roles, to make sure all Members get the nova-member rule, or some such
16:42:54 <johnthetubaguy> if thats what you want
16:43:51 <johnthetubaguy> ah, there is the whole in my proposal...
16:44:20 <johnthetubaguy> you can't have a token say read access to the world, but write access to just my project
16:44:27 <johnthetubaguy> but I think thats probably OK
16:45:07 <ayoung> johnthetubaguy, you cannot get a token scoped to a project without having a role on that project
16:45:25 <ayoung> johnthetubaguy, yes you can....
16:45:33 <ayoung> iff those are done by separate roles
16:46:05 <lbragstad> but you can't do that with a single token though, tokens are either unscoped or scoped
16:46:18 <ayoung> you still need a token scoped to the project to perform operations on that project
16:46:19 <johnthetubaguy> right, you could if its separate roles, but I was proposing not to do that
16:46:31 <lbragstad> what it sounds like johnthetubaguy wants it a token that means two different things depending on where you're using it
16:46:35 <lbragstad> is a*
16:46:37 <ayoung> is_admin_project being the backdoor to allow admins global access
16:46:58 <johnthetubaguy> right, I have gone for is_admin and is_global_scope being separate roles
16:47:12 <ayoung> you could do something comparable like role:read_only with a scope checkthat enforces is_admin_project=yes
16:47:25 <ayoung> scope is not role.  role is not scope
16:47:46 <johnthetubaguy> so its a problem with my proposal, but I also don't think its a valid use case
16:47:53 <johnthetubaguy> I just should pull that out explicitly
16:48:09 <johnthetubaguy> you just re-authenticate in the project where you have the permissions you want
16:48:13 <johnthetubaguy> (as a work around)
16:48:22 <ayoung> yep
16:48:51 <dstanek> johnthetubaguy: ++ on calling that out
16:49:42 <edmondsw> and you can get a token from a token, as long as you're ok with the new token having the same expiration of the first token, without requiring credentials again... so getting a second token with a different scope shouldn't be too problematic
16:50:32 <johnthetubaguy> true, I don't think its a big deal
16:50:41 <johnthetubaguy> just a non-obvious restriction of the system I am proposing
16:51:09 <dstanek> yeah, calling it out will let others looking at the docs/specs know that it was thought about
16:51:26 <lbragstad> ++
16:51:38 <lbragstad> 9 minute mark
16:52:25 <johnthetubaguy> so I think I have a much better understanding of some of the background here, which is great
16:53:07 <johnthetubaguy> so domains...
16:53:15 <johnthetubaguy> I am thinking about how to add that
16:53:24 <edmondsw> what about them?
16:53:28 <johnthetubaguy> I am thinking everywhere we have project_id we also need to add domain_id
16:53:36 <johnthetubaguy> well, project_id and user_id
16:53:47 <edmondsw> why?
16:54:07 <johnthetubaguy> oh, is project_id unique in the system?
16:54:12 <edmondsw> yes
16:54:19 <johnthetubaguy> so I totally missed that
16:54:25 * johnthetubaguy face palm
16:54:27 <lbragstad> project name is not
16:54:30 <edmondsw> project_name is only unique within a domain, but project_id is unique globally
16:54:37 <lbragstad> project name must be unique within the domain
16:54:48 <johnthetubaguy> right, that makes *way* more sense to me now
16:54:55 <lbragstad> :)
16:55:05 <johnthetubaguy> hmm, maybe this is over kill
16:55:16 <johnthetubaguy> if you put domain_id everywhere....
16:55:23 <johnthetubaguy> when we get a domain scoped token
16:55:42 <johnthetubaguy> we could simply update the scope check to look up things limited to the domain_id
16:55:49 <edmondsw> today nova doesn't work with domain scoped tokens at all, does it?
16:55:58 <lbragstad> edmondsw i don't believe so
16:56:02 <johnthetubaguy> edmondsw: I don't know what we do, honestly
16:56:10 <johnthetubaguy> we might just call it a project
16:56:20 <edmondsw> I'm pretty sure the only service that handles domain-scoped tokens is keystone
16:56:20 <johnthetubaguy> and get on with things as normal
16:56:23 <edmondsw> and that may be fine
16:56:49 <johnthetubaguy> I was thinking about an admin that wanted visibility only in their domain
16:56:52 <edmondsw> or there may be use cases where it would be nice for nova/etc. to allow domain-scoped tokens... I don't know
16:57:04 <lbragstad> johnthetubaguy i have a question about policy in code too if you have a minute
16:57:18 <johnthetubaguy> lbragstad: yeah, thats probably a better use of our time
16:57:30 * johnthetubaguy needs to go learn domains, and come back to that
16:57:49 <lbragstad> johnthetubaguy not to derail - just saw one of the post-its on my monitor and I had to ask
16:57:54 <johnthetubaguy> heh
16:58:09 <lbragstad> johnthetubaguy nova enforces policy in code, but how does oslo.policy grab the registered values?
16:58:26 <lbragstad> johnthetubaguy does it reach into nova and ask for it like config?
16:58:29 <lbragstad> cc antwash ravelar ^
16:59:29 <dstanek> johnthetubaguy: i also had a question about how horizon handles the policy in code thing. don't they need a copy of the policy to work correctly?
16:59:44 <johnthetubaguy> ah, its just like conifg
16:59:47 <johnthetubaguy> with entry point
16:59:48 * ravelar listening intently
16:59:56 <dstanek> or did you already make it discoverable?
17:00:16 <johnthetubaguy> https://docs.openstack.org/developer/oslo.policy/usage.html#sample-file-generation
17:00:26 <lbragstad> ah - so you can access an policy in oslo.policy (to do the evaluation) by doing `from nova import policies; policies.instances...`
17:00:30 <johnthetubaguy> alaski did all the good work here
17:00:38 <johnthetubaguy> oh, let me find our wiring
17:00:49 <edmondsw> lbragstad I think you're looking for this? https://github.com/openstack/nova/blob/master/nova/policy.py#L207
17:01:08 <lbragstad> edmondsw aha!
17:01:22 <johnthetubaguy> ah, yeah
17:01:28 <lbragstad> edmondsw yes - so that's what makes oslo.policy use the registered policy rules instead of some file
17:01:37 <lbragstad> (i.e. policy.json or policy.yaml)
17:01:46 <johnthetubaguy> well, overrides still come from the file
17:01:51 <edmondsw> right
17:01:53 <johnthetubaguy> we aslo use a new method
17:01:54 <lbragstad> right - but they are registered together
17:02:01 <johnthetubaguy> https://github.com/openstack/nova/blob/master/nova/context.py#L274
17:02:05 <johnthetubaguy> at least I thought that was new
17:02:23 <johnthetubaguy> I think we error out if the rule is no pre-registered
17:02:39 <lbragstad> got it - so just like config, policy must be registered before use
17:02:55 <lbragstad> (which finds the overrides, if any, and populates the rest of the policies with the default defined in code)
17:03:32 <johnthetubaguy> yeah
17:03:40 <lbragstad> awesome - that helps
17:03:46 <lbragstad> we're over time (sorry!)
17:03:50 <lbragstad> thanks for coming everyone!
17:03:56 <lbragstad> #endmeeting