16:00:09 <lbragstad> #startmeeting policy
16:00:10 <openstack> Meeting started Wed May 31 16:00:09 2017 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:11 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:13 <openstack> The meeting name has been set to 'policy'
16:00:13 <lbragstad> ping lbragstad, raildo, ktychkova, rderose, htruta, hrybacki, atrmr, gagehugo, lamt, thinrichs, edmondsw, ruan, ayoung, morgan, raj_singh, johnthetubaguy, knikolla, nhelgeson
16:00:19 <lbragstad> #link https://etherpad.openstack.org/p/keystone-policy-meeting
16:00:22 <lbragstad> agenda ^
16:00:23 <hrybacki> o/
16:00:27 <blancos> o/
16:00:48 <edmondsw> o/
16:00:48 <gagehugo> o/
16:00:53 <nhelgeson> o/
16:01:17 <lbragstad> looks like we have the same agenda that we did last week
16:01:22 <knikolla> o/
16:01:42 <lbragstad> #topic RBAC in middleware
16:01:44 * morgan lurks
16:01:59 <lbragstad> I've been reviewing the original and reproposed spec for the last 24 hours
16:02:06 <lbragstad> i'm still in the process of reviewing
16:02:49 <lbragstad> I think one thing that is becoming apparent to me for that work is the isolation of the scope check from the RBAC check in each of the projects
16:03:11 <lbragstad> the part that worries me is the release or two where we have both in place in the policy file and in keystone
16:03:35 <lbragstad> while other projects work to remove the RBAC checks from policy and move scope checks completely into code
16:03:57 <lbragstad> i'm not sure if anyone else shares that view point or has given that any thought and wants to air it out here
16:04:14 <edmondsw> I'm not sure I completely followed
16:04:24 <knikolla> lbragstad: no matter how you look at it, there's bound to be a transition period
16:04:42 <edmondsw> lbragstad what exactly concerned you about the transition period?
16:04:44 <lbragstad> edmondsw: something clicked for me yesterday
16:04:51 <edmondsw> :)
16:05:10 <lbragstad> and i realized that there wouldn't be any sort of role check in the policy files *eventually*
16:05:19 <knikolla> yes
16:05:24 <lbragstad> edmondsw: does that make sense
16:05:42 <lbragstad> because the proposal says that needs to be pulled into keystone
16:05:48 <edmondsw> lbragstad yes, that's the theory... unproven theory, but it's the theory
16:06:01 <knikolla> lbragstad: role check pulled into keystone. not scope check
16:06:13 <lbragstad> knikolla: correct
16:06:25 <lbragstad> both are currently done in the policy file
16:06:26 <knikolla> lbragstad: scope check moved into code, which is what is happening.
16:06:45 <lbragstad> knikolla: but - that is something that is driven by the project
16:07:00 <edmondsw> lbragstad I'm still not convinced we'll ever be able to get all role checks into the middleware... but we should be able to get at least most there
16:07:12 <knikolla> lbragstad: fully agreed.
16:07:30 <knikolla> lbragstad: but all can be disabled until the project buys in.
16:07:36 <lbragstad> edmondsw: ok - so here is my concern (real quick)
16:07:50 <knikolla> lbragstad: with nothing to aim towards, it's a chicken and egg buy in problem.
16:08:17 <lbragstad> knikolla: sure - but we need more buy in, which i'm getting to
16:08:24 <lbragstad> if a service has a policy file
16:08:35 <lbragstad> and their operation -> role mapping is stored in keystone
16:08:39 <lbragstad> we have a duplicated thing
16:08:42 <lbragstad> makes sense,
16:09:04 <lbragstad> what do we do with duplicated things? we have a migration or transition period, right?
16:09:04 <edmondsw> yes
16:09:21 <edmondsw> middleware is checked first, and then if it passes the policy is checked
16:09:21 <lbragstad> that migration or transition is dependent on some level of service work
16:09:58 <lbragstad> ideally - we want the middleware to check the role and if that passes the project should check the scope
16:10:08 <edmondsw> yes
16:10:11 <lbragstad> (if there isn't an overridden policy on disk somewhere)
16:11:06 <lbragstad> the thing that I thought was important was ensuring we have project buy in so that they prioritze the work to move the role checks out of policies
16:11:16 <lbragstad> that way we don't have duplicate storage of the same information
16:11:23 <hrybacki> +1
16:11:25 <lbragstad> to me - that is only going to lead to inconsistent user experience
16:11:54 <lbragstad> i don't think the approach is wrong
16:12:12 <lbragstad> but i want to make sure we don't let user experience degrade because we failed to convince or get proper project buy in
16:12:45 <lbragstad> edmondsw: does that make more sense?
16:13:02 <knikolla> lbragstad: we can work with the projects to have a policy.json file without role checks in case operators want to enable the role check in middleware during the transition period
16:13:05 <hrybacki> lbragstad: do you see an specific projects that might need a little extra encouragement? What ux issues are you foreseeing?
16:13:46 <knikolla> lbragstad: also this is something entirely up to ops to enable or not, with a "caution: these projects haven't reworked their policy yet"
16:14:37 <edmondsw> lbragstad yes, it's going to be inconsistent
16:14:45 <lbragstad> hrybacki: for example - if a project doesn't discourage the use of roles in policy.json files, then it's possible for an operator to make changes and not update keystone (i.e. the keystone check passes or a user sees that they can do something through keystone, but can't through the service API)
16:15:14 <hrybacki> thank you for the clarification, makes sense
16:15:18 <edmondsw> lbragstad by default, operators won't have anything set in middleware, right?
16:15:36 <lbragstad> edmondsw: i don't think so?
16:15:50 <knikolla> edmondsw: it's disabled by default.
16:16:08 <edmondsw> so they would need to set something before any of this matters
16:16:20 <lbragstad> if it is enabled - the operator has to persist the role and routes in keystone in order for it to be useful
16:16:35 <knikolla> edmondsw: what lbragstad said.
16:16:37 <edmondsw> but that doesn't change the fact that it could be confusing that, when they set middleware, it works to varying degrees for different projects
16:16:41 <lbragstad> because that's what keystonemiddleware has to query in order to make a decision
16:16:53 <edmondsw> right
16:17:07 <lbragstad> edmondsw: elaborate on varying degrees a bit?
16:18:23 <edmondsw> lbragstad I was thinking about different projects being at various stages in the rework of their policy files
16:18:34 <lbragstad> oh - sure
16:18:36 <edmondsw> moving scope checks into code, etc.
16:18:56 <lbragstad> ideally - with the rbac in middleware bits, scope should be moved into the project
16:18:58 <knikolla> edmondsw: which is why it's enablement is on a service per service basis
16:19:13 <lbragstad> and role checks should be moved into keystone, according to the proposal
16:20:46 <lbragstad> edmondsw: you made a comment earlier about not all roles being enforced in middleware, can you walk me through that so i understand it?
16:21:20 <edmondsw> lbragstad role checks can't ever move from default policy into middleware, though, unless we then require middleware be setup
16:21:31 <edmondsw> is that something we can eventually do at some point?
16:21:46 <lbragstad> edmondsw: that's a good question - i would expect operators to answer that for us
16:21:54 <lbragstad> after some reasonable time in the wild
16:21:58 <lbragstad> and looking at adoption
16:22:03 * lbragstad shrug
16:22:50 <lbragstad> another thing that i noticed in the spec is that the routes api would be open to all users
16:22:58 <edmondsw> lbragstad my earlier comment about not necessarily being able to do all role checks in middleware has to do with special cases like the actions APIs, a bunch of neutron APIs, etc.
16:23:07 <lbragstad> oh - sure
16:23:11 <edmondsw> where the body of the request is important
16:23:20 <knikolla> edmondsw: there is a follow up spec on the body
16:23:32 <lbragstad> edmondsw: we wouldn't have to worry about that if we queried of the operation/policy
16:23:33 <edmondsw> knikolla yeah... but I don't buy that it will solve all cases
16:23:56 <edmondsw> knikolla do you have a link, or is that not written yet?
16:24:16 <lbragstad> there is a section in the reproposed spec on that work
16:24:19 <knikolla> edmondsw: gimme a sec for the link.
16:24:31 <lbragstad> i'm not sure if there is another spec proposed detailing those steps
16:24:38 <lbragstad> #link https://review.openstack.org/#/c/452198/8
16:24:42 <knikolla> #link https://review.openstack.org/#/c/456974/
16:24:52 <knikolla> edmondsw: ^^
16:25:02 <lbragstad> oh - there is one
16:25:20 <edmondsw> knikolla yeah, that doesn't work as written
16:25:23 <edmondsw> way too simplistic
16:25:25 <lbragstad> but - i don't think we'd need that follow on spec if we replaced the URL with the operation
16:25:31 <knikolla> edmondsw: please comment on your concerns
16:25:34 <edmondsw> will do
16:25:47 <lbragstad> i.e. compute:create_service instead of POST /v2/servers/
16:25:56 <edmondsw> lbragstad how would you replace the url with the operation, since the middleware doesn't know the operation?
16:26:04 <lbragstad> edmondsw: exactly
16:26:08 <lbragstad> edmondsw: that's the tough part
16:26:22 <lbragstad> you might be able to do it in the service, or oslo.policy somewhere
16:26:45 <lbragstad> but i think the operation would be a better fit, since multiple URLs can map to the same API
16:26:50 <knikolla> edmondsw: we can work with nova to expand their actions api into separate urls. with their microversion support it shouldn't be hard.
16:27:23 <edmondsw> could we have middleware-enabled services and non-middleware enabled services, and make the middleware-enabled ones provide a yaml or something that maps from POST /v2/servers to compute:create_service
16:27:50 <edmondsw> then have the middleware expose compute:create_service to users, since that's what they know about?
16:27:50 <lbragstad> yeah - that's an option
16:27:56 <knikolla> edmondsw: yes. the check is enabled in the service.conf
16:28:14 <knikolla> edmondsw: it's actually on an endpoint basis
16:28:23 <lbragstad> either way - we wouldn't have to store the url information in keystone for other service APIs
16:28:58 <lbragstad> which goes back to usability (i.e. if an operator updates a role requirement for booting a server on one API but doesn't on another, now we have an inconsistency)
16:29:18 <knikolla> edmondsw: lbragstad: during the transition period, role check in middleware is disabled by default, nothing changes unless explicitly changed. once services rework policy, they can guide their users to enable role check in middleware.
16:29:43 <edmondsw> knikolla yeah, we got that
16:29:52 <lbragstad> knikolla: sure - that makes sense, but does it address the url concerns?
16:29:55 <edmondsw> no
16:30:13 <lbragstad> dstanek: had a pile of them and i've been working with him to understand those
16:30:28 <knikolla> edmondsw: it's up to the services.
16:30:35 <knikolla> lbragstad: *
16:30:43 <edmondsw> knikolla what is?
16:30:45 <lbragstad> knikolla: what's up to the service?
16:31:42 <knikolla> in the future to have real REST-like apis instead of the actions api.
16:31:58 <knikolla> the only solution without their work is check in the json body.
16:32:16 <lbragstad> knikolla: why couldn't we check the operation key?
16:32:35 <knikolla> lbragstad: that's the proposal, to check the operation key in the json.
16:32:42 <lbragstad> knikolla: no
16:32:49 <lbragstad> knikolla: the policy key, sorry
16:33:05 <lbragstad> compute:create_server, compute:reboot_server, etc...
16:33:12 <edmondsw> knikolla we're not suggesting they transition to "real REST-like apis"... not going to happen
16:33:22 <knikolla> lbragstad: middleware has no knowledge of the mapping
16:33:48 <lbragstad> knikolla: what would it take to get your proposal to adhere to that requirement then?
16:34:49 <lbragstad> because if we could come up with a way to do what you want and solve that requirement, then i think you'll get buy in from several of the people who had concerns
16:34:58 <knikolla> lbragstad: a mapping from urls into policy keys. which is still the issue in middleware because there are similar urls.
16:35:51 <knikolla> lbragstad: a mapping (url, json key in the body) might solve that.
16:35:52 <lbragstad> knikolla: yes - that's the problem
16:36:15 <knikolla> which brings me back to the role check on body key proposal. please comment on that about your concerns.
16:36:16 <lbragstad> that still requires keystone to store the URL
16:36:24 <knikolla> lbragstad: not necessarily.
16:36:46 <knikolla> lbragstad: it can be read from a conf file. but i think that is suboptimal.
16:37:00 <knikolla> since middleware is deployed with the service
16:37:26 <lbragstad> why can't we wait to query keystone for the required role until later in the request?
16:37:27 <knikolla> it has access to the service configuration, including policy.json
16:37:44 <lbragstad> like - why can't the service call out to keystone for that information once it knows exactly which operation it's doing
16:38:35 <knikolla> lbragstad: what would it benefit from for doing it that late?
16:38:58 <lbragstad> because then the service knows the policy key it needs since it's processing the request (URL, body, method, etc..)
16:39:21 <knikolla> lbragstad: then the only thing that changes is that policy.json is stored as it is in keystone
16:39:35 <knikolla> lbragstad: however it has roles instead of overcomplicated rules
16:39:37 <lbragstad> so instead of storing a bunch of URL for other services in keystone (which can be fragile) keystone only has to say "compute:create_server requires the member" role
16:40:07 <lbragstad> then we don't need a follow on spec for supporting json body parsing for complex (non-restful) APIs
16:40:35 <knikolla> lbragstad: i don't have an argument against or pro, i defer to ayoung to answer that.
16:41:15 <lbragstad> fair enough - it was just something that came up as i was reviewing it and reflecting on feedback about the storage or URLs :)
16:41:51 <knikolla> lbragstad: i thought about that too. i know that at some point there was a centralized/dynamic policy approach which was shut down. but that predates my keystone involvement.
16:42:31 <lbragstad> the difference between that approach and this one is that the dynamic policy stuff was a push of all existing policy files into keystone
16:42:43 <lbragstad> so when a service needed a policy, it fetched it from keystone
16:42:46 <knikolla> while this is role check, gotcha.
16:43:12 <knikolla> as i said. i don't have a strong argument against doing routes with policy keys instead of urls.
16:43:15 <knikolla> the only thing is that
16:43:21 <knikolla> now it needs to be in the service code
16:43:25 <lbragstad> right
16:43:26 <knikolla> rather than keystonemiddleware code
16:43:29 <knikolla> so we don't own that
16:43:30 <lbragstad> yep
16:43:42 <lbragstad> which is going to require cross-project communication in order to work
16:43:44 <lbragstad> which isn't a bad thing
16:44:02 <lbragstad> and that's something we can use project tags and community goals to get traction on if the TC thinks it's worth while
16:44:04 <knikolla> lbragstad: the actionable plan to split role check from scope check is still there
16:44:30 <knikolla> task*
16:44:37 <knikolla> no matter how you look at it, that has to be done
16:44:46 <lbragstad> agree - that makes sense
16:44:52 <lbragstad> and some projects have already started working on that
16:45:01 <lbragstad> (maybe just nova, but it's a start)
16:45:05 <knikolla> and no matter how you look at it, that might break backwards compat with policy.sjon
16:45:19 <edmondsw> knikolla why do you say that?
16:45:54 <knikolla> edmondsw: emphasis on might.
16:45:57 <edmondsw> moving scope checks into code doesn't break backward compate
16:46:23 <samueldmq> edmondsw: it depends
16:46:30 <edmondsw> not really
16:46:39 <samueldmq> there might be complex rules that mix roles and scopes
16:46:46 <edmondsw> yes...
16:46:48 <knikolla> edmondsw: depends on if you allow them to be ovewritten. if you don't, and someone used that, it will.
16:46:59 <samueldmq> splitting them might end up losing the power of defining those complex rules
16:46:59 <edmondsw> what is "them" in that sentence?
16:47:20 <knikolla> edmondsw: policy rules with mixed scope and role.
16:47:21 <samueldmq> in my sentence it's role+scope
16:47:54 <edmondsw> I haven't heard anyone propose that policy.json goes away, or that you can no longer do scope checks there
16:48:21 <lbragstad> it would be possible for it to go away - but i'm not sure we could ever remove it
16:48:29 <samueldmq> yes I believe by default it will be better, even if we change what we have to allow splitting role from scope
16:48:49 <samueldmq> however people could still continue doing complex/complicated in their overrides in their policy.json, correct/
16:48:54 <knikolla> edmondsw: then it's removing role checks from there. either way, something will change.
16:49:19 <lbragstad> but - that's something that needs to be done on a per service basis
16:49:20 <edmondsw> knikolla I'm not following
16:49:24 <samueldmq> knikolla: from the default, I expect it to continue allowing it
16:49:58 <knikolla> edmondsw: i'm moving too far head after the transition period.
16:50:19 <edmondsw> that's fine... I'm thinking way ahead to.. but I'm not following your line of reasoning
16:51:20 <knikolla> edmondsw: do we agree that policy.json needs to evolve eventually if we split role check?
16:51:31 <knikolla> edmondsw: if we store it in keystone it will be there
16:52:01 <edmondsw> knikolla what is "there"?
16:52:39 <knikolla> edmondsw: role check in keystone. either as a mapping with policy key -> role. or url -> verb -> etc -> role.
16:52:53 <edmondsw> knikolla default policy.json is going away, as we move policy defaults into code
16:53:13 <edmondsw> using policy.json to override things, whether that is scope checks in code or role checks in middleware, is not going away
16:54:03 <knikolla> edmondsw: including long after the transition period?
16:54:12 <edmondsw> knikolla yes
16:54:19 <edmondsw> ever
16:54:25 <samueldmq> moving into code means moving into projects code
16:54:30 <edmondsw> yes
16:54:32 <samueldmq> not moving them all into keystone code
16:54:40 <edmondsw> right
16:54:48 <lbragstad> correct - keystone should never have to deal with scope checks
16:54:51 <lbragstad> IMO
16:55:04 <knikolla> lbragstad: i never mentioned scope checks into keystone
16:55:11 <lbragstad> knikolla: just clarifying
16:55:50 <samueldmq> lbragstad: except for its own scope checks that goes into its code :-)
16:55:59 <knikolla> edmondsw: what i'm pointing to is: right now policy.json can override both role and scope. in response to lbragstad's concern about having two sources of truth for role checks, would we deprecate the possibility of having role checks in policy.json
16:56:24 <knikolla> emphasis on eventually (which i forgot to write)
16:56:55 <edmondsw> knikolla I don't think so
16:57:07 <lbragstad> 4 minutes remaining
16:57:11 <samueldmq> I think we still allow scope+roles checks in the policy.json files, for the sack of backward compatibility, even if we do not do it upstream (or do not advise it)
16:57:21 <samueldmq> edmondsw: lbragstad is that correct?
16:57:40 <edmondsw> knikolla maybe if we really solve things in middleware... but again, I don't think bodykeys necessarily do that
16:57:43 <lbragstad> probably? but i'll defer to edmondsw
16:58:11 <knikolla> edmondsw: that is fine.
16:58:17 <knikolla> edmondsw: more ops choice :)
16:58:22 <lbragstad> ok - real quick
16:58:29 <knikolla> edmondsw: as long as they don't mix them…
16:58:51 <lbragstad> there are a couple general documents i've proposed to specs that i'd like get reviews on (and i need to address samueldmq's comments)
16:58:54 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone-specs+branch:master+topic:460344
16:59:12 <samueldmq> lbragstad: ++ I will review the others too
16:59:27 <samueldmq> and we're out of time, thanks lbragstad knikolla edmondsw o/
16:59:35 <lbragstad> they are in a linear series and i attempted to make it clear that they build on each other
16:59:43 <lbragstad> thanks for coming - today was good
16:59:44 <knikolla> please reach out to me in -keystone for more rbac middleware concerns
16:59:49 <knikolla> i really want this merged :)
16:59:50 <hrybacki> thanks all o/
16:59:50 <lbragstad> knikolla: will do
16:59:57 <lbragstad> #endmeeting