17:59:50 #startmeeting keystone 17:59:51 Meeting started Tue Dec 13 17:59:50 2016 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:59:53 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:59:55 The meeting name has been set to 'keystone' 18:00:10 hi 18:00:19 * morgan doesn't want to meeeeet. 18:00:25 * morgan is here though. 18:00:44 o/ 18:00:50 morgan: i feel you, i'm only half way through lunch 18:00:56 stevemar: i need breakfast. 18:01:05 stevemar: and coffeeeeeeee 18:01:21 alright, let's kick things off 18:01:25 #topic Announcements 18:01:33 Cutting Ocata-2 this week! This is also spec freeze week, specs must merge now! 18:01:41 errr, agenda: https://etherpad.openstack.org/p/keystone-weekly-meeting 18:01:57 hi 18:02:12 #link https://etherpad.openstack.org/p/keystone-weekly-meeting 18:02:37 so i'm kinda concerned about the status of new features and the short release cycle 18:02:51 we've got 1 complete, 6 in progress, 4 not started 18:03:00 and ocata-2 is *this* week 18:03:11 stevemar: also those specs (whose haven't merged) are related to trusts, and ayoung is against 18:03:25 samueldmq: ignore the specs for now 18:03:32 i have time blocked off for http://specs.openstack.org/openstack/keystone-specs/specs/keystone/ocata/pci-dss-password-requirements-api.html today and tomorrow 18:03:36 these are approved ones 18:03:43 i plan to have something up to review this week 18:03:46 * samueldmq nods 18:03:52 See gerrit topics for where to start reviewing: https://docs.google.com/spreadsheets/d/156q820cXcEc8Y9YWQgoc_hyOm3AZ2jtMQM3zdDhwGFU/edit?usp=sharing 18:04:05 end of ocata-3 is only about 5-6 weeks away 18:04:14 theres no stopping it for holiday break either 18:04:22 so i'm pessimistic on a lot of these 18:04:35 especially the "not started" ones 18:04:37 with all the policy discussions i haven't revisited my saml work, but i'll get on that this week again 18:05:00 stevemar is going to be manager! 18:05:03 theres also the fact that if everyone is heads down coding, then no one is reviewing 18:05:04 :P 18:05:06 ravelar will help with extending user API, so we'll have something soon 18:05:32 the MFA spec is on it's way (working on the migrations etc), but unfortunately, I have a move happening as well. 18:05:37 lbragstad: since you're focusing on that one, perhaps shaddow mappings could be postponed ? 18:05:39 i'm trying to toss in reviews around coding and packing 18:05:45 lbragstad: or you think you can do both ? 18:05:48 mfa was close IIRC 18:05:54 lbragstad: shadow mapping doesn't seem to be a small one 18:06:01 ayoung: mfa spec merged. just needs code 18:06:01 this is just a heads up -- we can re-evaluate in a few weeks based on how much a specific feature affects critical paths 18:06:29 stevemar: I am reviewing. no features on me 18:06:30 but it is ok if mfa slips, we can implement parts now and final code / enable next cycle if needed 18:06:47 samueldmq i'm still planning on doing both - but i wanted to tackle the password requirements one first since horizon is expecting it 18:06:50 it isn't a ton of code 18:07:06 fwiw, expect a bit more code/reviews from me once i'm moved (next week) and over holidays 18:07:17 hopefully everyone has a few quiet last week or two (at work) before the holidays kick in and can review a bit more 18:07:21 lbragstad: gotcha 18:07:25 o/ 18:07:26 * morgan also has shade code to write/review 18:07:36 ocata-3 is Jan 23-27 18:08:12 at which point anything that isn't merged will get bumped to backlog/pike, or can ask for an exception 18:08:49 anywho, i'm tired of acting like a PM for 8 minutes :P 18:08:54 next topic! 18:09:04 #topic Require domain_id when registering Identity Providers 18:09:07 rderose: ^ 18:09:10 In preparation for extending the user API and to help Horizon with domain admin support, 18:09:10 We want to replace the hardcoded 'Federated' domain with a real domain for federated users. 18:09:10 https://review.openstack.org/#/c/399684/ 18:09:20 The new domain will be related to the IdP (1:1) and thus, we'll require a domain_id when registering new IdPs. 18:09:30 To be backwards compatible, if the domain is not explicitly set via the API, we'll automatically create the domain and relate it to the IdP. 18:09:38 I'm just looking to get this reviewed, but let me know if you have any questions or concerns. 18:09:48 rderose: that sounds all good to me 18:09:56 rderose: quick observation... 18:09:58 I like this. Should have been this way from the get go 18:10:13 ayoung: ++ 18:10:26 rderose: any specific question or something to discuss or just a "this is the status, needs review"? 18:10:27 rderose: automatically create the 'Federated' domain right? 18:10:28 rderose: what about auto-creating a domain with the same name if one isn't supplied when creating an idp? 18:10:42 as long as we avoid more magic domains like default 18:10:46 samueldmq: yeah, just want to bring into attention 18:10:51 rderose: ++ 18:11:02 dstanek: that can possibly be done with a migration 18:11:02 dstanek: auto create the IdP domain 18:11:05 if we can, in any way avoid it 18:11:12 rderose, the domain_id attribute on the IDP is going to be editable, right, so that people can munge things that they got wrong when creating IdPs? 18:11:22 rderose, like the idea! 18:11:23 and by magic i mean "default:" being the ID 18:11:27 stevemar: it will create a domain and the name will be the IdP id 18:11:34 IdP doesn't have a name 18:11:35 rderose: cool 18:11:47 so as long as it doesn't get magic ids on the new domain, i'm ok with it 18:11:48 rderose: is that backward compatible? 18:12:02 rderose, if there is a clash, if there is a domain already named that way, what do you propose? 18:12:03 ayoung: no updates 18:12:13 right now all idps share a domain namespace (err....kinda, if it were working) 18:12:13 ayoung: concerned about the implications of changing the domain 18:12:24 ayoung: so you'd have to delete and recreate the IdP 18:12:39 dstanek: yes, backwards compatible 18:12:43 rderose, if I create the domain first, then create the IdP, and it creates a new domain, I guess delete the IdP and recreate with the new API 18:12:49 rderose: that is... not great. can we provide an admin a mechanism to map the idp to a domain sideband in that case? 18:13:04 morgan, ++ 18:13:13 morgan, it is suboptimal, but it will also be rare 18:13:25 it would mean things change for user(s), but it's really letting the admin do something about it 18:13:28 I say lets go with this for now, and add additional mechanisms as a follow on 18:13:30 rderose: rather than answering this flurry of questions, can you map out the different cases that'll happen in an etherpad or in a spec or release note ? 18:13:37 rderose: in an existing cloud with 2 idps will you migrate them to have two new domains? 18:13:38 morgan: what's the concern? if we create a domain for the IdP 18:13:45 stevemar: ++ 18:13:47 rderose: if the domain exists 18:13:52 BTW, do NOT cascade the delte 18:13:54 delete 18:13:56 dstanek: yes 18:14:01 if we delete the IdP, do not delete the domain 18:14:04 rderose: let the admin map the idp ito the domain. 18:14:08 also waht ayoung said. 18:14:13 if there is a domain with the name that matches the IdP, grab it 18:14:19 rderose: address the migration case, the create case and the update case 18:14:20 anyway. 18:14:20 let me restart that 18:14:27 *cough* etherpad for this *cough* 18:14:30 stevemar: will do 18:14:37 when creating, if the idp_id matches an existing domain name, make that the domain for the IdP 18:14:56 way too many combinations to flesh out in a meeting ayoung -- but i'll give you a quick go at it 18:14:56 discuss after meeting. Lets get this one in 18:15:13 etherpad: https://etherpad.openstack.org/p/keystone-idp-domain-questions 18:15:21 dstanek: ty 18:15:44 okay 18:15:50 post in the code review, please 18:16:09 ayoung: the code review being https://review.openstack.org/#/c/399684/ 18:16:18 I mean post response in the review request 18:16:24 not in etherpad 18:16:28 that is what review is for 18:16:35 ayoung: can be easily done 18:16:56 next topic 18:16:58 #topic Role check from middleware 18:17:15 i think this is the only spec that is close to making the ocata cut-off 18:17:17 this had 2 +2s, but I left it open 18:17:24 there are not radical changes from last weekl 18:17:28 are there any last minute dissenters on this one ? 18:17:31 1: better defaults 18:17:43 2: allowing multiple roles per API 18:17:52 i am not opposed to this. 18:18:03 i think henrynash gave it his blessing 18:18:07 i think it is closest to the right direction for policy updates that has been proposed. 18:18:09 indeed :-) 18:18:24 i had a bunch of comments - specifically around mapping things using the URLs, but dstanek ayoung edmondsw and I have been talking through it since Friday 18:18:30 i can add a +2 if you'd like. 18:18:39 please do 18:18:49 yeah, i'll still not a fan of using URLs for this at all 18:18:50 edmondsw: o/ 18:18:54 We can amend if there are new details 18:18:57 ayoung: link? 18:19:02 https://review.openstack.org/#/c/391624/ 18:19:05 ty 18:19:15 ayoung have you made any of the changes we were discussing end of last week? 18:19:27 edmondsw, yes, I added in the multiple roles part 18:19:27 I was out yesterday, haven't looked 18:19:30 fyi - i'm probably going to -2 the rest of the open specs 18:19:35 RBAC by URL and policy by operational target 18:19:53 dstanek: what is operational target ? 18:20:01 identity:get_user 18:20:07 dstanek, I think what we can work toward over time is a closer mapping from URL to the existing policy rules 18:20:10 ayoung: done 18:20:10 samueldmq: what we use in policy today 18:20:15 kk 18:20:21 samueldmq target = the resource you're acting on 18:20:27 ayoung: it is explicitly stated as ok since it's ongoing and such 18:20:32 without some way to map that, though, we need to start with the URL 18:20:39 edmondsw: it's not the resource it the operation you are doing 18:20:41 lbragstad: you had the policy working group going on, does this align what that? 18:20:47 edmondsw: gotcha 18:21:02 lbragstad: i'll admit i haven't been keeping up with that one 18:21:05 stevemar we're starting from square one there 18:21:06 dstanek I'd say it's both 18:21:28 get user isn't a resource 18:21:36 user is a resource 18:21:46 exactly 18:21:49 stevemar we want to define usecases on what the idea solution should be and we're (mainly dstanek and myself) still working on that 18:21:59 and policy can check attributes of the resource 18:22:07 user is a specific instance of the get_user operation with additional context...namely the id you want to get 18:22:15 lbragstad: nice, starting from usecases is always the right path 18:22:47 as i understand it the RBAC is on the operation and the existing policy is on the resource 18:22:54 dstanek: ++ 18:23:03 lbragstad: my concern is that we get ayoung working on this solution and then you and dstanek come up with another overhaul in a few months 18:23:09 that is what the core of this change is started 18:23:24 stevemar: for the policy meeting we have been going around in circles...just begining to have an architectural vision defined 18:23:31 dstanek, right... well, existing policy is on both operation and resource... that's what I'm saying 18:23:43 stevemar ^ yeah.. that's a problem, too 18:23:45 stevemar, so, I don't think having the mechanism defined this way is super committing 18:23:56 stevemar: that's my concern as well - that's why lbragstad and i have been trying to capture all the things 18:23:58 right now - i'd be happy if we could agree on a vision 18:24:03 ayoung: right, this is pretty isolated code, i liked that 18:24:14 we can alwasy build better automation to tie oslo policy and RBAC together if we find they start to diverge 18:24:20 the split of operation and resource action aslo makes a lot of sense in general 18:24:24 what is really lacking in code now is a mapping between URL and Role. 18:24:28 ayoung: and i think this will take the Pike cycle as well right? 18:24:38 regardless of what policy meeting comes up with 18:24:45 stevemar, I think that I can get a proof of concept of this workig in Ocata 18:24:53 since it's mostly an enhancement of what we do today 18:25:07 the changes are pretty minimal: keystone server change needs to be updated to reflect the spec 18:25:10 it should give the policy folks more fidelity in definition even if there are changes. 18:25:21 keystone client is essentially a REST API 18:25:29 OK, it sounds like we still have runway for dstanek and lbragstad to figure out if they can come up with something and live with the RBAC check in middleware regardless 18:25:33 middleware needs to use the keystoneclient and cache 18:25:36 morgan: i agree on the split. my biggest issue with the spec as-is is that we use the URL 18:26:00 dstanek: what else is there to use? 18:26:28 stevemar: we already use the operation target - so know you have to know that an the URL 18:26:48 and to be honest i don't know the URLs for 95% of what i use in openstack 18:27:25 actually it's not just the URL you need, you also need to know what verb is going to be used 18:27:28 almost everything is //resource/ thankfully 18:27:30 dstanek: unfortuantely from middleware you can't easily map operation to url. 18:27:34 dstanek: that's funny, i prefer looking at the APIs / URLs 18:27:38 * 18:27:42 since middlweware is above the application 18:27:54 i also tend to prefer the URL vs the underlying method(s) like we have today 18:28:21 morgan: yeah, i think there would have be be a mapping defined and i think it would be pretty easy to pull that out of routes 18:28:24 knikolla: there has been a real effort to remove from URIs in projects 18:28:25 at least in keystone 18:28:34 dstanek, but the URL+VERB is what is documented 18:28:36 dstanek: in keystone is different then in nova 18:28:39 not 100% of course 18:28:44 as edmondsw was about to state, I am sure 18:28:45 dstanek stevemar's question is a good one, though... what other option do we have? How would middleware map from the Verb+URL of the request to something you would recognize? 18:28:49 dstanek: so, look at it from not-keystone perspective. 18:28:57 but that is what is in the api-refs 18:29:12 morgan: no, i agree we need to. i'm saying we should :-) 18:29:20 ayoung ++ 18:29:35 so if i want to know if i can restart a vm i'll have to know the URL and the target to check? 18:29:54 i've always hated the policy file referencing the underlying methods 18:30:03 URLs are far far more stable 18:30:04 morgan: oh, cool. right now only glance doesn't have the project_id in the url. 18:30:09 dstanek but "you" aren't a user in that case... you're horizon or another client 18:30:10 dstanek, yes, that is the contract 18:30:13 knikolla: nova was working on it 18:30:21 knikolla: and has made strides to support both 18:30:41 edmondsw: i could be the user...what can i do with this token...right? 18:30:48 knikolla: and i think cinder is really the only other one.... the project id is in the token. 18:30:54 morgan: now if they only supported a unified pagination method :) 18:31:01 knikolla: not now. 18:31:04 i just think in general that a service should be responsible for what it's URLs mean and not everyone else 18:31:09 RESTful and all that 18:31:09 knikolla: lets not derail this 18:31:33 edmondsw dstanek doubtful... a user would use a client, and that client would abstract this from them 18:31:56 dstanek: ah you're asking for something different then. not urls but also not what we have today 18:32:01 dstanek if the user is using the APIs directly rather than a client, then obviously they already understand APIs 18:32:05 * morgan still thinks URLs is most correct. 18:32:16 put a description on it 18:32:22 action+url+description 18:32:24 edmondsw: sure, but they also have to now the target so that case is moot 18:32:36 edmondsw: they understand the type of resource and the action they want to do. 18:33:41 dstanek knikolla of course they understand the target resource, but the middleware doesn't 18:33:44 i know i want to list domains not /v3/domains/list or is it /v3/list/domains or /v3/domains? 18:34:33 ok. so instead of making the serivce keep a mapping for the middleware to use we are making then keep a mapping that they have to jam directly into keystone? 18:34:43 dstanek so you go to openstackclient, right? and it can map Verb+URL information from the keystone API's response into something prettier if we want 18:34:43 {'action':'GET', 'url':' /v3/domains', 'description': 'List domains'} 18:35:02 dstanek, but how would middleware know anything but the Verb+URL? 18:35:18 edmondsw: the service would somehow configure that mapping 18:35:23 so it's easy to operators to see what is the semantics on the operation 18:35:28 dstanek how? 18:36:05 dstanek, I think, over time, we can do that 18:36:05 edmondsw: simple was is config file, complex way is a callback, and i can probably think of other ways to do this 18:36:11 but not out the gate 18:36:11 keystone already has the service catalog, maybe services would expose a URL with mappings when registering to the catalog 18:36:21 dstanek: would you be ok with URLs and an addition to make it easier / friendlier down the line once you've thought of it 18:36:24 dstanek: so both could be done? 18:36:29 dstanek if you can solve that, then I don't see why we couldn't support 2 different formats... one that is Verb+URL and another based on that mapping, and let the caller determine which they want to provide/receive 18:36:34 i really think URLs is a correct method to use. 18:36:39 the router framework in Keystone, for example, gives us a logical place to link to the policy rule 18:36:41 even if we have another as well 18:36:45 morgan: well right out of the gate how does nova upgrade? 18:36:48 but that differs for every project 18:36:52 so we do that manually to start 18:37:18 ayoung: yeah in keystone it's trivial to look up the route and inspect @protected 18:37:19 we Need it at the URL level. the trick is to then map that to the policy rules, and that will take a lot of effort 18:37:28 ayoung: ++ 18:37:40 dstanek, and I suspect that, over time, we could do the same for other projects 18:37:46 ayoung: yeah, i'm just arguing who should own the mapping 18:37:52 the way Nova does it, for example 18:38:09 dstanek, so, to start, we use good defaults on the ROles and punt on detailed mapping 18:38:18 we can make a first hack using the api-ref 18:38:38 but if most APIs require the Member role, we can focus on excludimng those that would be broken by an RBAC checjk 18:38:42 like discovery 18:39:50 ayoung: keeping URLs? 18:40:10 dstanek, we need to be able to report at the URL level 18:40:11 20m left (timecheck) 18:40:13 it is a requirement 18:40:24 "what role do I need for this operation" 18:40:37 we can change the mechanism, but that requirement has to be met 18:41:09 i'm just saying that the service should own the mapping and that is best left out of keystone - not that roles shouldn't be checked at the verb+url level 18:41:13 thanks morgan 18:42:02 we can continue in -keystone later if needed 18:42:13 dstanek: we actually have time, only 1 topic left 18:42:26 dstanek: your topic, actually 18:43:34 stevemar: no expired tokens? 18:43:48 i don't see jamielennox 18:43:55 dstanek: no, i think jamie and i figured it out 18:44:15 lets switch gears to your topic dstanek 18:44:21 #topic versioning mapping 18:44:30 ayoung: when you ask "what role do I need for this operation" are you asking the service or keystone? 18:44:53 version mapping....i just wanted to get community thoughts on this 18:44:55 dstanek keystone 18:45:16 we have mapping for federation right now that is nice, but has its warts 18:45:16 dstanek, what does that mean? 18:45:27 also maybe needs a feature or two 18:45:28 Vesioning of what? APIS? 18:45:56 would anyone be opposed to adding something in the mapping to indicated a version identifier? 18:46:05 dstanek: can you describe the use-case 18:46:11 ayoung: not APIs. the mapping itself 18:46:23 so some possible things that have been thrown around: 18:46:25 i'm not clear on the use. i'm sure it's not something i'm opposed to 18:46:26 dstanek, Federation mapping? 18:46:32 but i want to know what you're talking about 18:46:42 ayoung: yes federation mapping 18:46:47 - fix the way to do indexed matching that that all remote matches are available 18:47:03 - add the provisioning stuff that dolphm & lbragstad have talked about 18:47:29 i think what dstanek is getting at is that if you create a new mapping and have something like {version: 2} in the payload, it'll go through a different set of rules ? 18:47:31 - adding some extra features allowing mapping to grab data directly out of SAML instead of just the environment 18:47:36 dstanek, ++ 18:47:51 sure 18:47:52 stevemar ++ 18:47:57 no issues with that idea at all 18:47:59 I am not 100% sure it is required, but it does let us deprecate features 18:48:02 stevemar: yeah, and first step is to say the next version is an object and not a list 18:48:06 similar to fernet token types 18:48:07 dstanek: how will you "grab data directly out os SAML" ? 18:48:09 just for the mapping. 18:48:11 wfm 18:48:23 dstanek: thanks for clarifying 18:48:25 stevemar that would probably come with native saml support 18:48:34 lbragstad: okay, thought as much 18:48:35 stevemar: in my saml work i have to recreate the shib mapping stuff already 18:48:46 morgan: yep, just for mapping 18:48:53 right right, it just looked funny when comparing it to today's functionality 18:48:56 it's fine to give versions for rulesets like that 18:49:02 just make sure it's usable 18:49:03 ayoung: i would love the ability to deprecate anything we see as cruft :-) 18:49:15 :) 18:49:17 dstanek: i think this is fine -- nova did similar things with block_mapping 18:49:34 dstanek: it's much the same as i want for auth in general 18:49:49 dstanek: so we can move auth data/types/payloads forward without breaking everyone 18:49:52 dstanek so - kind of related, but does that make the shadow mapping stuff dependent on introducing versioning first? 18:49:55 dstanek: the engine mapper needs love, but if starting from scratch is easier, then go for it 18:51:09 dstanek: is that something i should look for soon or no? 18:51:10 lbragstad: good question. i don't think it needs to be if you dn't change the behavior of existing mappings 18:51:29 stevemar: i'll write up a spec today 18:51:41 stevemar: if push comes to shove it can wait a release 18:51:49 dstanek: why not piggy back that the existing mapping engine spec? 18:52:03 stevemar: sure, i can do that 18:52:06 dstanek, you might recall the jdennis had a pretty exhaustive mapping mechanism which he proposed a few years back 18:52:07 dstanek do we have a good reason to wait for mapping versions for provisioning? 18:52:11 dstanek: i think it aligns with http://specs.openstack.org/openstack/keystone-specs/specs/keystone/ocata/shadow-mapping.html 18:52:33 timecheck 8 min left 18:52:44 lbragstad: i thought versioning would make provisioning easier? 18:52:49 thats why you wanted to do it? 18:53:15 ayoung: yeah, this would allow us to use other mapping engines like what he propesed. i actually was thinking of his demo when we were talking about this yesterday 18:53:22 cool 18:53:27 stevemar yeah - it probably would.. because you'd be able to create mappings with versions and that could imply auto provisioning 18:54:00 stevemar: yes, among other reasons 18:54:38 dstanek: okay, so it would go versioning -> provisioning -> profit? 18:55:16 dstanek rderose lbragstad thanks for all the federation work -- really nice to see it become a first class citizen 18:55:16 so in that case - provisioning would only work for mappings that have specific versions. 18:55:23 5 minute warning 18:55:49 lbragstad: i'm open to either, i assumed you wanted it that way for ease of impl. 18:56:07 rather than add a bunch of new checks to already hacky mapping code 18:56:12 stevemar: ayoung inspired me to think about federation in a different way :-) 18:56:19 Uh oh 18:56:25 ayoung inspires all of us 18:56:28 stevemar that's true 18:56:35 ++ 18:56:39 lbragstad: i'll leave that call to you 18:56:45 lbragstad: you and i can talk about that offline and come up with a strategy 18:56:46 #topic open discussion 18:56:47 * lbragstad ponders 18:56:59 any last minute stuff n things? 18:57:13 well - we have a policy meeting tomorrow 18:57:17 just fyi... think we are close to have federated auth tests 18:57:37 i'll run the meeting next week, but probably cancel the one on the 27th 18:57:50 ok 18:58:04 perfect 18:58:15 i'll run the meeting on the 3rd again, half-expecting low attendance :) 18:58:17 makes senes 18:58:47 i think we're all good 18:58:52 thanks for coming folks 18:58:56 thanks stevemar 18:58:59 good meeting 18:58:59 thanks all 18:59:00 see you next week, same bat-time, same bat-channel 18:59:06 :) 18:59:08 pow! 18:59:10 o/ 18:59:15 o/ 18:59:17 henrynash: bam! 18:59:21 #endmeeting