16:00:02 #startmeeting policy 16:00:03 Meeting started Wed May 17 16:00:02 2017 UTC and is due to finish in 60 minutes. The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:04 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:07 The meeting name has been set to 'policy' 16:00:10 ping lbragstad, raildo, ktychkova, dolphm, dstanek, rderose, htruta, atrmr, gagehugo, lamt, thinrichs, edmondsw, ruan, ayoung, morgan, raj_singh, johnthetubaguy, knikolla, nhelgeson 16:00:13 o/ 16:00:14 hi 16:00:19 o/ 16:00:21 #link https://etherpad.openstack.org/p/keystone-policy-meeting 16:00:25 agenda ^ 16:00:26 o/ 16:00:52 let see how many folks we have - we might be able to squeeze by with a google hangout 16:00:54 o/ 16:01:14 o/ 16:01:20 o/ 16:01:36 o/ 16:02:06 o/ 16:02:17 wow - we're getting a pretty good turn out 16:02:23 o/ 16:03:11 o/ 16:04:04 Heh 16:04:05 11, i believe that takes us over the hangout limit. 16:04:12 yep - i think so 16:04:32 o/ 16:04:43 ok - let's go ahead and get started 16:04:45 Its OK, that was a special occasion. We should try to make IRC work 16:04:55 #topic goals 16:05:04 https://etherpad.openstack.org/p/keystone-policy-meeting 16:05:36 imo - part of the reason why we seem to stall out on various policy dicussions is because there are *so* many things to fix 16:05:41 to help with that 16:05:54 I took a stab at defining the goals we want to strive for 16:05:58 ++ 16:06:04 1. Admin-ness 16:06:15 2. mapping easy of understanding and main 16:06:19 i wanted to be able to provide a document that folks can use as a reference when reviewing or even discussing policy 16:06:27 3. default set of roles 16:06:37 lbragstad: that sounds great 16:06:38 #link https://review.openstack.org/#/c/460344/ 16:06:50 I'd love to get some feedback on that ^ 16:07:01 not only from keystone folks, but from other projects as well 16:07:08 do we have representation here from other projects? 16:07:28 lbragstad, so, some prioritization 16:07:41 I guess I am a nova-ish voice here 16:07:54 I don't think we can do the "standard set of roles" until we have an interim state of "Be able to add a new role to an already running system" 16:07:54 johnthetubaguy: glad you could make it :) 16:08:11 lbragstad: i work on qa, murano, reading it 16:08:23 felipemonteiro: awesome 16:08:28 I think the "standard set of roles" is the right top level goal 16:09:20 So, one thing we should probably make explicit, somehow, is that "some" role needs to be enforced for every policy check unless explicitly opted out 16:09:28 there is probably a better way to say that 16:09:45 but what I want to address is the cases where the current policy checks scope, but no role. We need a good default 16:09:54 So we don't reallt talk about user/operator aims in that policy goals document, its still quite implementation based 16:10:18 I wonder if stepping back a tiny bit would help here 16:10:25 johnthetubaguy, did you make it to my talk? I did address some of it there. 16:10:33 johnthetubaguy I don't know that I agree with that... I thought it was quite operator based 16:11:06 ayoung: didn't make any talks, had forum sessions in nevery slot, I do plan on watching yours ASAP 16:11:23 johnthetubaguy, feel free to hit me up for the accellerated version 16:11:31 two slides I want to call out 16:11:32 edmondsw: maybe its just the way I am reading it 16:11:33 https://docs.google.com/presentation/d/1L31_x977Yrz9im3MapBxVUpcQSv7IGTJnCf7s6kbtFc/edit#slide=id.g1d8796a54f_1_75 16:11:36 can everyone see that? 16:11:48 #link https://www.youtube.com/watch?v=EYYzl0rFCVU&t=147s 16:11:48 need permission 16:11:58 yup need permission 16:12:01 I have a public version somewhere...let me see if I can find it.. 16:12:12 ayoung I requested access 16:12:32 try now 16:12:37 i updated the permissions 16:12:50 works for me now 16:13:04 how about https://docs.google.com/presentation/d/1L31_x977Yrz9im3MapBxVUpcQSv7IGTJnCf7s6kbtFc/pub?start=false&loop=false&delayms=3000#slide=id.p 16:13:18 slides 17 and 18 right? 16:13:26 ayoung: that link works 16:13:34 they both work now 16:13:43 OK, that is preso mode. The two things I want to call out 16:14:03 Without 16:14:03 An Explicit 16:14:03 Scope Check 16:14:03 Every Project 16:14:03 Has Access 16:14:03 to Every Project’s Resources 16:14:05 GAh 16:14:09 Heh 16:14:27 Without An Explicit Scope Check Every Project Has Access to Every Project’s Resources 16:14:42 Without An Explicit Role Check for an API Every Role Has Access 16:14:51 every token scoped to that project has access 16:15:01 the whole master keyring issue 16:15:05 * johnthetubaguy nods 16:15:40 gagehugo, yeah...if you don't look too close at my analogy 16:15:56 so, close those two things should be at the top. 16:16:03 from the operator point of view, which are the critical use cases? 16:16:17 like I get those are problems that break lots of flows 16:16:25 johnthetubaguy, #1 thing I've gotten a request is for a read-only role 16:16:26 just wondering which flows we should think about the most 16:16:46 ayoung: yeah, I guess number 2 is the whole per project admin? 16:16:52 yep 16:17:10 so this all ties back to admin-ness 16:17:11 cool, so those seem like the two things I was expecting to see in the goals doc 16:17:22 so a default role for all APIs for a service would go a long way to covering those. 16:17:33 lbragstad, yeah, sortof 16:17:40 johnthetubaguy ayoung my #1 operator request is fixing the admin scoping 16:18:09 edmondsw, good to know. And its is also a pre-req to the other two 16:18:16 then #2 making it easier to customize policy 16:18:33 and then having better pre-defined roles would come after that somewhere 16:19:05 as the ordering is in the spec lbragstad has up on policy goals 16:19:07 "customize policy" isn't really specific enough for me, but its good context for sure 16:19:14 ./ 16:19:23 lurking, just recovering from vacation ;) 16:19:24 oh man, we have a morgan 16:19:25 \O 16:19:31 "customize policy" = "define my own roles and what they can do" 16:19:38 cool. 16:20:01 morgan: welcome back! 16:20:12 OK. So...I think I am OK with this Doc as is, providing we make it a living document, and are not afraid to update it as we get better clarity. 16:20:35 Some of the things we are talking about is mechanism, but the doc does a good job, I think, of abstracting the use cases to general goals 16:20:41 edmondsw: I was really meaning, today what many operators want is the read-only role, which causes them to cusomtize stuff 16:20:42 i'm good with that - so long as it stays concise and direct 16:21:10 OK. So, I'm a put my +2 on it 16:22:09 Any future changes can be posted as their own reviews. What is important for the upcoming release is work items 16:22:22 and, the priority of those is directed at goal #1 16:22:29 Agreed? 16:22:34 ++ on the whole living document thing. it would be very beneficial as we move forward. 16:22:47 having a clear documented path of goals 16:22:48 johnthetubaguy readonly is one needed role, for sure. But the ability to customize and create whatever role you want is really what I'm after 16:23:07 ok - so i have a question 16:23:22 edmondsw: totally, I think they are both separate aims, I added a comment in the spec 16:23:35 Fire away lbragstad 16:23:37 we know fixing admin-ness is a priority, but can that be decoupled from determining which role is required to do an operation? 16:23:41 johnthetubaguy right... #2 vs #3 in the spec 16:23:45 lbragstad, yes 16:23:48 #2 gives you the ability to create whatever you want 16:23:54 johnthetubaguy: ++ 16:23:57 lbragstad, that is, in essence, a scope check, and should be done first 16:24:06 #3 gives you some pre-defined roles so that you don't have to use #2 if you're happy with what we predefine 16:24:38 for the moment, admin is checked in the oslo-policy mechanism or the nova/keystone equivalents 16:24:49 and the checks are all done as Or checks 16:24:56 either the user has Admin or the scope must match 16:25:07 to ensure adminess, we need an and check like this 16:25:15 edmondsm: ah, I think I miss understood your point, I get what you mean now (I think) 16:25:42 either ((role == admin and is_admin_project ) or (token.project_)id == resource.project_id)) 16:26:18 note that the == condition ensures that role == admin on the project still works 16:26:57 so that's where this starts to get tricky because we'll have to work with each of the projects to ensure the resource they are checking against makes sense within the scope of a project 16:27:10 (i.e. instances make sense within project, endpoints do not) 16:27:21 lbragstad, right. THat has been Sisyphean 16:27:31 ayoung more like "(role == ) and (is_admin_project or (token.project_id == resource.project_id))) 16:27:39 separate role check from scope check 16:28:06 edmondsw, Yes, that is the end state. I was answering lbragstad 's question about whether we could serialize the work, though 16:28:16 the role check work can come afterwards 16:28:31 I don't think I agree 16:28:39 but I think you are referring to the "admin" part of that, and, yes, we should be smarter about that 16:28:53 if we had context.is_global_admin implemented, would that help the project do what they need to do? 16:28:56 edmondsw, think about it as a patch ordering 16:29:21 johnthetubaguy, you mean context.is_admin_project> 16:29:25 I get that you're talking about patch ordering... I don't understand why you think you should split that into separate patches 16:29:25 and yes, that is required 16:29:26 nope 16:29:41 I'm not sure you can split it, much less that you should 16:29:58 or why you would want to 16:30:27 edmondsw, because I think role modificiation is a trickier thing to implement, and, even if we go with a policy.json based implementation, we will have trouble getting it into a single, understandable patch 16:31:12 ayoung what "role modification" do you think I'm proposing? 16:31:26 I don't think I am proposing anything I'd call that 16:31:40 edmondsw, taking an API that only allows 'admin' and making it accest 'admin or reader' for example 16:31:43 this is my super rough idea of what projects would have to do - http://paste.openstack.org/show/609824/ 16:32:01 i'm not proposing we add any new roles in this first patch stage 16:32:07 lbragstad, not quite 16:32:15 which would force them to make sense of operations in a global, domain, and project scope 16:32:17 lbragstad, lets leave domain out for now, as that is keystone only 16:32:22 its more like: 16:32:29 if operation is global_scoped: 16:32:30 if token.is_admin_project == True: 16:32:33 (i.e. does it make sense to do instance things in a global sense - no it's a project thing) 16:32:42 Oh...wait 16:32:48 I misread what you wrote.... 16:32:53 yeah, what you have is right 16:33:00 \o/ 16:33:11 * lbragstad highfives the dog 16:33:12 with the exception that is_admin_project might be needed for domain scope operations, too 16:33:37 lbragstad I would only change the domain check to allow both domain_scoped token or is_admin_project True 16:33:39 ayoung: sure - that could be applied at the project level too 16:33:40 so. lets leave that out for now, as that is Keystone only, and we can fix that 16:34:16 there are user scoped things too for Nova, but that follows the same pattern 16:34:30 edmondsw, so, why do you think the role stuff needs to be handled along with is_admin_project? 16:34:54 ayoung I think you are just misunderstanding me 16:34:56 johnthetubaguy, what resources are user scoped? 16:35:00 edmondsw, likely! 16:35:04 ayoung: keypairs 16:35:10 johnthetubaguy, got it 16:35:21 johnthetubaguy, those are "also" project scoped, right? 16:35:26 ayoung I'm not suggesting we add any roles... just that we not tie is_admin_project checks to the admin role... there's no reason or need to do that 16:35:27 *also* 16:35:32 it doesn't help with anything 16:35:35 edmondsw, roger 16:35:37 and it's not the end goal we want 16:35:45 rodger dodger. Agree 16:35:49 ++ 16:36:26 OK...so on the agenda, I sketched out the pieces that need to be done. 16:36:36 lines 15 -30 16:36:45 hack on it, please 16:37:03 so if you have an admin-only policy check, then it would be "((role == admin) and (is_admin_project or (token.project_id == resource.project_id)))" 16:37:13 edmondsw, right 16:37:19 edmondsw, and a reader only check would be 16:37:27 but an operation that is allowed for any role would be that minus the role == admin part 16:37:27 "((role == reader) and (is_admin_project or (token.project_id == resource.project_id)))" 16:37:34 right 16:37:36 yes 16:37:45 I'd like to get rid of the "any role will do" stuff, but THAT comes later 16:37:49 and so, with implied roles, we could then make that, implicitly 16:37:56 "((role == reader or role == admin) and (is_admin_project or (token.project_id == resource.project_id)))" 16:38:07 without implied roles, that would be added to the policy file 16:38:40 otherwise, an admin would not be able to to it without also explicitly having the reader role 16:38:52 updated - http://paste.openstack.org/show/609825/ 16:39:19 lbragstad, ++ 16:39:30 so is_admin_project seems really odd, that was the feedback in one of the previous discussions, is this something we want to train our operators to do? 16:39:38 OK...so johnthetubaguy can you take on the nova part, or do we need to find someone to do that? 16:39:52 johnthetubaguy: i did write up https://review.openstack.org/#/c/464763/ 16:39:54 I don't have a job right now, so its tricky to know either way 16:39:57 johnthetubaguy, it was based on a lot of discussion... 16:40:00 which is an alternative 16:40:03 johnthetubaguy, ah, joy 16:40:33 lbragstad: I should read through that 16:40:44 johnthetubaguy, is_admin_project was based on the way openstack was origianlly designed to do this, at least according to termie 16:41:01 lbragstad johnthetubaguy if/when I can find time to help, my priority will most likely be helping in nova 16:41:08 my proposal is that we eventually move away from that model 16:41:12 it also allowed us a way to get global roles without radically modifying the Keystone server 16:41:50 now I want us to make progress as fast as possible, but we do need to take care of training operators to do a thing, then changing that a release or two later 16:41:53 I will try to help in keystone as well though 16:41:56 i have an idea that might allow us to move away from that model without making the projects consuming the information change more than once 16:42:15 that sounds nice 16:42:16 its the operators that worry me the most here 16:42:24 johnthetubaguy: ++ 16:42:29 but the minimal project chagnes would be awesome too 16:42:55 do folks want to hear my plan? 16:42:58 I would prefer something that checks for the concept "global admin" that could be implemented however 16:43:03 lbragstad: I do :) 16:43:09 can we not switch from is_admin_project to global-scoped role assignments via a db migration? 16:43:24 I would think that's just a keystone db migration 16:43:31 edmondsw: I am thinking API calls, and general concepts in all our users heads 16:43:44 edmondsw, yes, we can do that 16:44:07 its the changes to oslo-context that require changes in the remote projects 16:44:07 1.) implement the is_admin_project stuff but make it so that the token represents something like token.global_scope=True 16:44:19 2.) make oslo.context changes to honor token.global_scope 16:44:32 3.) project consume that and apply it to their operations 16:44:37 lbragstad, so...what iffffff 16:44:41 johnthetubaguy yeah, they would need to understand that "oh, I can take advantage of global-scoped role assignments now as I add new users/groups instead of being tied to what I had defined as is_admin_project" 16:44:44 we had a library or something to do the check 16:44:44 4.) implement global roles in keystone 16:44:54 what if we did the scope check in oslo-context itself? 16:45:12 5.) migrate is_admin_project to global role assignment 16:45:19 and oslo-context could then handle is_admin_project OR global_roles? 16:45:33 can't this just be context.is_global_scope? 16:45:53 ayoung: right - that's the magic bit that makes it so other services don't care about is_admin_project 16:46:09 or global roles 16:46:12 cool, I think thats going to be much easier to describe to folks 16:46:18 exactly 16:46:28 all the other services should care about is token.global_scope == True 16:46:40 so whats the check for admin now? 16:46:56 then they can evaluate according to their "global" operations at the service level 16:47:15 sorry I was thinking back to the ealier paste with the if statements 16:47:20 how do they change in this model? 16:47:46 johnthetubaguy they look at token.is_global instead of token.is_admin_project... just a rewording 16:47:52 yeah 16:48:03 as that will be in code, not in policy, it's not a concern for users/operators 16:48:05 but the important bit is that it's not exposing implementation details in the interface 16:48:06 so if we want a global read_only role? 16:48:35 take the read-only role, and assign it with global scope instead of a project scope 16:48:40 role==reader and token.global_scope 16:48:50 is_global* 16:48:58 so what I mean is, we are really checking for role=admin and token.is_global_scope 16:49:04 then bob should be able to get a global token with the reader role 16:50:32 johnthetubaguy if you want to restrict something to something that is a global admin, yes 16:51:09 so the current admin folks access the APIs, they would fail this new test I assume? 16:51:21 what are we doing for the transition? 16:51:37 the big thing, though, is that Nova is not even properly loading up the context, is it? 16:52:05 ayoung: the context object is fine, we don't always pass the target 16:52:11 johnthetubaguy, we need that 16:52:37 unless we pull out all scope checks from the policy file right? 16:53:04 I am finding it hard separating all these concerns and making it work across upgrade 16:53:20 without feeling like we are about to break everything 16:53:51 johnthetubaguy upgrade is tricky... I think the only way we can solve that and maintain backward compat is to have some transitional policy checks that do scope checking 16:53:51 johnthetubaguy, I'm guessing that the scope checks are in the database now, but I can't see how that would work for global admin operations. But also, Nova must enforce some scope checks, somehow. 16:53:59 I had a plan to do all this in Nova, but its a long old process, and the team happy to do it are no more, but struggling to see the lighterweight approach here 16:54:41 ayoung: it all depends on which exact API we are talking about 16:54:43 OK, first question, is it even possible to do this in oslo-context, assuming we load the target 16:54:57 what is "this"? 16:55:06 johnthetubaguy, scope check IAW the rules above 16:55:10 so, ignoring the role: 16:55:20 (is_admin_project or (token.project_id == resource.project_id))) 16:55:34 e.g. servers:list:other_projects = "role:admin and is_admin_project==True" and have the code use that as an override for the normal scope checks it would do... and deprecated this for eventual removal 16:55:46 and, when policy runs, have it do "context.scope_check" 16:55:46 the bit I thought was going in context was the extra is_global_scope = True/False property 16:56:20 what if... 16:56:33 we did that anyway, and have the scope check ignore the target if it is not set? 16:56:48 then this resolves to a series of smaller bugs in Nova/cinder etc to get the scope checks right? 16:56:49 I did create this thing: https://review.openstack.org/#/c/435485/2/nova/context.py 16:57:23 johnthetubaguy, yeah, that. but it would be nice if it were in oslo-context, so we could do the same thing in cinder 16:57:50 ++ on oslo 16:58:02 ayoung: yeah, makes sense, its just that would need to be called from each bit of the API separately, the way I initial proposed this 16:59:02 two minute warning 16:59:48 hmm 16:59:49 is the best way forward to do a POC in keystone for this change, that includes dealing with upgrade, etc? 17:00:21 this is my nova POC: https://review.openstack.org/#/c/435485 but it would take lots of work to add that everywhere 17:00:23 johnthetubaguy, not a bad starting point 17:00:48 we're out of time 17:01:07 do folks want to spill into #openstack-dev or #openstack-keystone? 17:01:17 lbragstad, yes please 17:01:24 johnthetubaguy the problem with a keystone-only PoC is that the admin-is-special-across-projects thing is very different in keystone than in nova/elsewhere 17:01:32 is anthing scheduled in here for the next hour? 17:01:41 edmondsw: ah, that could be a problem 17:01:54 edmondsw, yes, but... 17:02:17 I think we have keystone policy enforce on oslo-context now. If we don't we should make that happen anyway, and we can do the heavy liftin in oslo 17:02:57 there is a bit missing for me, between oslo-context and the current policy check calls 17:03:08 ayoung I definitely agree with leaning on oslo here as much as possible. I'm inclined to implement the logic the way it needs to be without that first, though, and then figure out how to clean it up with moving things into oslo 17:03:29 ++ for a move to olso after we have it working 17:03:38 ++ 17:03:46 so, gagehugo you up for that? 17:04:57 the bit I am missing on the Nova side is how we avoid touching every API call (like proposed in https://review.openstack.org/#/c/435485), maybe we can't avoid that, and thats fair enough. 17:05:26 ayoung sure 17:05:30 we're going to have to end the meeting 17:05:38 * johnthetubaguy has to go and cook his dinner 17:05:38 i don't want to roll over 17:05:58 johnthetubaguy unfortunately I think we will need to touch every API call... I don't see a way around that 17:06:21 edmondsw: OK, maybe this can be the POC then: https://review.openstack.org/#/c/435485 17:07:13 I'd like the PoC to take several different hard cases and show that we can solve them... not be a general change like that 17:08:05 i.e. actually prove something :) 17:15:22 lbragstad need to end the meeting 17:15:25 we are going to have to end the meeting 17:15:46 #endmeeting