Monday, 2017-12-04

*** itlinux has joined #openstack-keystone00:03
ayoungjamielennox, lets see, the ones I override are mostly keystone specific values that should not be used for policy on other services00:07
ayoungI'm cool with follow on work that cleans this up, though00:07
ayoungI want to get rid of common/ now, but in another patch00:07
*** sticker_ is now known as sticker00:08
ayoungOn the  Deprecated Values, I am guessing that I can append new values as opposed to passing them in the constructor, so, yeah, that should be cleaner00:08
ayoungjamielennox, for you next question...00:09
ayoung"Can't we create the context first and then just add the stuff we need after.?"  I'd be fine with that00:09
ayoungit seems like that stuff is supposed to be passed in to the constructor, but I really Don't care.  I do want to get that whole logic extracted out into a seperate function that we can test, as a replacement for's token_to_auth_context00:10
ayoungBut, this change allowed me to get this one to finally pass Tempest:  and that is what I really care about00:11
*** thorst has joined #openstack-keystone00:16
*** thorst has quit IRC00:21
*** mancdaz has joined #openstack-keystone00:22
jamielennoxthat's cool on the tempest front, i've no idea why it changes anything because it should be the same00:27
jamielennoxso there's no reason things have to be passed to constructor, it's just built like that because from_environ works that way and everyone used to pass things to constructor00:28
jamielennoxit's just an object you can set whatever you like, and i don't feel like there should need to be any switching logic on the constructor side, just add the values to the object that you need00:28
jamielennoxre override, there shouldn't be anything we _override_ differently to what is already there - there's definitely stuff we need to add, but i just mean to reuse as much as the underlying as possible as it makes keystone less special00:29
*** david-lyle has joined #openstack-keystone00:30
*** david-lyle has quit IRC00:31
*** david-lyle has joined #openstack-keystone00:31
ayoungjamielennox, how about this:00:31
ayoungOn tempest, it defaults the is_admin_projects values.  Which is what I want tested.  But you are right, I could get it to pass without that. I had to pass an additioanl value to get tempest to pass:00:33
ayoungand thati s just, I think, to keep from opening up a security hold00:34
ayoungas for the values handled by __init__ I'd like to minimize this patch, and clean up existing behaviour in a future patch, if that is OK.  Trying to keep this reviewable, and already it asks for some deep understanding on the part of the reviewer00:36
*** jmlowe has joined #openstack-keystone00:47
*** thorst has joined #openstack-keystone00:52
jamielennoxayoung: i am fine with keep it small00:54
jamielennoxstrange that is_domain_scoped is a problem, why isn't is just a @property self.domain_id != None00:56
*** thorst has quit IRC00:57
ayoungjamielennox, so, the problem was origianally that the tempest tests used admin scoped to the default domain.  I changed the Keystone policy to grandfather this in...but with out is_domain_scoped set, it will match any token's domain_id value, not just domain scoped.  Perhaps I am being paranoid01:05
ayoungI know that at that point, it should be project_domain_id01:05
openstackgerritayoung proposed openstack/keystone master: Use oslo-context
ayoungjamielennox, how's that01:08
jamielennoxlol where'd this come from:01:09
jamielennox       self.user_id = kwargs.pop('user_id ', None)01:09
jamielennox        self.project_id = kwargs.pop('project_id ', None)01:09
jamielennox        self.user_domain_id = kwargs.pop('user_domain_id ', None)01:09
jamielennox        self.project_domain_id = kwargs.pop('project_domain_id ', None)01:09
jamielennoxthere are spaces in those strings01:09
jamielennox(wasn't you)01:10
jamielennoxbut all that can just be removed01:10
ayoungYep, And I acutally got messed up by that01:16
ayoungI think I git blamed it and found out it was you01:16
ayoungNope, stanek01:17
ayoungso my patch is gonna fail...01:18
ayoungif I try to drop the group_ids thing01:18
*** edmondsw has joined #openstack-keystone01:19
*** edmondsw has quit IRC01:24
*** annp has quit IRC01:27
*** thorst has joined #openstack-keystone01:27
*** thorst has quit IRC01:28
*** thorst has joined #openstack-keystone01:29
*** thorst has quit IRC01:30
*** jmlowe has quit IRC01:44
openstackgerritayoung proposed openstack/keystone master: Use oslo-context
*** gmann_afk is now known as gmann01:54
*** thorst has joined #openstack-keystone02:01
*** thorst has quit IRC02:06
*** thorst has joined #openstack-keystone02:10
*** thorst has quit IRC02:14
openstackgerritayoung proposed openstack/keystone master: Use oslo-context
openstackgerritayoung proposed openstack/keystone master: Add is_admin_project check to policy for non scoped operations
openstackgerritayoung proposed openstack/keystone master: Add is_admin_project check to policy for token validations
*** daidv has joined #openstack-keystone02:28
openstackgerritAdrian Turjak proposed openstack/keystone master: Make name fields a consistent size of 255
*** edmondsw has joined #openstack-keystone03:07
*** edmondsw has quit IRC03:12
*** prashkre has joined #openstack-keystone03:15
*** thorst has joined #openstack-keystone03:17
*** thorst has quit IRC03:22
*** itlinux has quit IRC03:32
*** edmondsw has joined #openstack-keystone03:32
*** edmondsw has quit IRC03:36
*** thorst has joined #openstack-keystone03:57
*** thorst has quit IRC04:02
*** annp has joined #openstack-keystone04:10
*** links has joined #openstack-keystone04:13
*** jmlowe has joined #openstack-keystone04:17
openstackgerritAdrian Turjak proposed openstack/keystone master: Make name fields a consistent size of 255
*** ricolin has joined #openstack-keystone04:21
*** swain has joined #openstack-keystone04:23
openstackgerritAdrian Turjak proposed openstack/keystone master: Make name fields a consistent size of 255
*** thorst has joined #openstack-keystone04:37
*** itlinux has joined #openstack-keystone04:39
*** thorst has quit IRC04:41
*** edmondsw has joined #openstack-keystone04:58
*** jmlowe has quit IRC05:02
*** edmondsw has quit IRC05:03
*** thorst has joined #openstack-keystone05:10
*** thorst has quit IRC05:15
*** prashkre has quit IRC05:15
*** jaosorior has joined #openstack-keystone05:16
*** jmlowe has joined #openstack-keystone05:24
*** itlinux has quit IRC05:28
*** jmlowe has quit IRC05:28
*** zhurong has joined #openstack-keystone05:31
*** andymccr has quit IRC05:32
*** andymccr has joined #openstack-keystone05:33
*** cloudnull has quit IRC05:33
*** thorst has joined #openstack-keystone05:37
*** cloudnull has joined #openstack-keystone05:41
*** thorst has quit IRC05:42
*** thorst has joined #openstack-keystone05:45
*** thorst has quit IRC05:50
*** clarkb has quit IRC06:02
*** sticker_ has joined #openstack-keystone06:10
*** sticker has quit IRC06:13
*** sticker_ has quit IRC06:14
*** thorst has joined #openstack-keystone06:24
*** thorst has quit IRC06:29
*** swain has quit IRC06:32
*** clarkb has joined #openstack-keystone06:33
*** edmondsw has joined #openstack-keystone06:46
*** edmondsw has quit IRC06:51
*** prashkre has joined #openstack-keystone06:55
*** thorst has joined #openstack-keystone06:57
*** zhurong has quit IRC06:57
*** thorst has quit IRC07:01
*** zhurong has joined #openstack-keystone07:20
*** thorst has joined #openstack-keystone07:36
*** thorst has quit IRC07:41
*** rcernin has quit IRC07:49
openstackgerritwangxiyuan proposed openstack/keystone master: Correct error message for request token
*** tesseract has joined #openstack-keystone08:13
*** prashkre has quit IRC08:13
*** AlexeyAbashkin has joined #openstack-keystone08:16
*** zhurong has quit IRC08:20
*** pcaruana has joined #openstack-keystone08:23
*** prashkre has joined #openstack-keystone08:30
*** magicboiz has joined #openstack-keystone08:33
*** edmondsw has joined #openstack-keystone08:35
*** rcernin has joined #openstack-keystone08:35
*** magicboiz has quit IRC08:37
*** edmondsw has quit IRC08:40
*** magicboiz has joined #openstack-keystone08:42
*** magicboiz has quit IRC08:47
*** magicboiz has joined #openstack-keystone08:47
*** zhurong has joined #openstack-keystone09:03
*** mvk has joined #openstack-keystone09:05
*** Guest13268 is now known as zigo09:09
*** aloga has quit IRC09:21
*** aloga has joined #openstack-keystone09:22
openstackgerritMerged openstack/ldappool master: Avoid for constraints support
*** zhurong has quit IRC10:26
*** annp has quit IRC10:33
*** mvk has quit IRC10:40
*** gmann is now known as gmann_afk10:52
*** mvk has joined #openstack-keystone11:09
*** prashkre has quit IRC11:49
*** raildo has joined #openstack-keystone11:51
*** edmondsw has joined #openstack-keystone12:11
*** avi_ has quit IRC12:13
*** edmondsw has quit IRC12:16
*** rcernin has quit IRC12:31
*** links has quit IRC12:54
*** edmondsw has joined #openstack-keystone13:23
*** links has joined #openstack-keystone13:31
*** edmondsw has quit IRC13:43
*** panbalag has joined #openstack-keystone13:48
*** itlinux has joined #openstack-keystone13:50
*** thorst has joined #openstack-keystone13:59
*** jmccarthy has joined #openstack-keystone14:12
jmccarthyAny suggestions to help determine why I seem to have this issue with a missing endpoint ?
*** spilla has joined #openstack-keystone14:30
lbragstadhrybacki: not sure if you've seen this yet -
*** thorst has quit IRC14:46
*** thorst has joined #openstack-keystone14:48
*** thorst has quit IRC15:03
*** links has quit IRC15:10
*** jsheeren has joined #openstack-keystone15:12
*** phalmos has joined #openstack-keystone15:12
jsheerenhi all, what's the easiest way to configure keystone to accept requests on eg AND openstack.net15:12
*** edmondsw has joined #openstack-keystone15:13
*** jmccarthy has left #openstack-keystone15:17
*** edmondsw has quit IRC15:18
*** AlexeyAbashkin has quit IRC15:20
*** phalmos has quit IRC15:22
*** thorst has joined #openstack-keystone15:24
*** itlinux has quit IRC15:26
openstackgerritayoung proposed openstack/keystone master: Enforce policy on oslo-context
*** thorst has quit IRC15:29
hrybackilbragstad: ack, we knew it was coming fortunately15:31
lbragstadhrybacki: last we talked, did we want to use trello?15:32
lbragstadhrybacki: also - do we want to go through the existing board before it goes away?15:32
hrybackilbragstad: ack. Use Trello until we switch to story board15:32
hrybackilbragstad: yes, we should do that tomorrow if you have time?15:33
lbragstaddamn -
lbragstadthey already turned the lights off15:33
hrybackiAh hmm. Well, we did a good job of making sure we didn't drop anything from the ptg retro which was the most important so far15:34
cmurphyjsheeren: it depends on what you mean, if I interpret your question correctly you can just point your DNS A or CNAME records to wherever your keystone is, but that's not really a keystone question so maybe you mean something else15:36
jsheerencmurphy, yes, i meam something else.  the catalog points to; how can i make it that when a user requests the catalog at it shows the endpoints on and not openstack.com15:40
jsheerendns is all good and such, but if the endpoints configured in keystone still refer to ..15:41
*** phalmos has joined #openstack-keystone15:43
jsheerenmaybe i need to look into something like this?
bretonjsheeren: so you have 2 domains pointing to the same keystone and you want keystone to return different urls depending on which domain you make a query to?15:51
jsheerenbreton, yes, thank you for rephrasing my question to something that's better :)15:52
jsheerenis it possible though?15:52
bretonjsheeren: The urls in the catalog are static and cannot be computed15:53
bretonjsheeren: so without magic i think no15:55
jsheerenah :(15:55
jsheerenthanks anyway! i'll look into magic then :)15:55
*** jsheeren has quit IRC15:55
bretonjsheeren: theoretically you can invent some magic with 2 instances of keystone15:56
bretoni wanted to describe the magic15:56
bretonbut he quit15:56
*** david-lyle has quit IRC15:58
*** david-lyle has joined #openstack-keystone15:59
mordredcmurphy: "Could we do the same thing for keystonemiddleware?" ... absolutely - and certainly shold16:23
cmurphymordred: cool16:26
*** edmondsw has joined #openstack-keystone16:26
*** thorst has joined #openstack-keystone16:28
*** itlinux has joined #openstack-keystone16:31
*** thorst has quit IRC16:33
lbragstadkeystone retrospective board is up and ready for input16:37
lbragstadif you don't have the ability to add things, please ping me and i'll send you an invite16:37
lbragstad(it should be open to the public to view)16:37
*** gyee has joined #openstack-keystone16:43
* cmurphy wonders where kmalloc is16:43
cmurphyi want his eyes on
*** mvk has quit IRC16:44
*** itlinux has quit IRC16:46
lbragstadcmurphy: responded16:48
lbragstadi remember talking about that in sydney16:48
cmurphylbragstad: hmm interesting16:49
*** itlinux has joined #openstack-keystone16:50
*** ricolin has quit IRC16:53
*** kmalloc has joined #openstack-keystone17:01
*** panbalag has quit IRC17:02
*** thorst has joined #openstack-keystone17:02
* kmalloc yawns17:08
cmurphykmalloc: hi o/17:08
cmurphykmalloc: i was hoping you could look at I think it breaks the api :/17:09
kmallocso, this does break our api contract, in so much as that someone who was relying on some names being 64 (lets say for CRM integrations) could potentially now fail17:12
kmallocnot that anyone should rely on names17:12
kmallocThis one is not as big a break as some, but it changes API behavior and therefore breaks the contract.17:12
kmallocsecond, it could really break anyone who has custom identity backends17:13
kmallocsince they may not support > 64 characters (bytes?) for these fields.17:13
kmalloci'm going to -2 it. until lbragstad or more cores say we need this.17:13
cmurphyon the identity backends do we explicitly guarantee non-breakage? we removed all that driver version stuff and we've definitely had interface changes in those since then17:15
kmalloccmurphy: we do not, but we also typically do not change column sizes because of E_NO_DOWNTIME_UPGRADES17:17
kmallocwe typically add new columns, which could be much easier to deal with from a custom backend standpoint17:17
kmallocsince you're not "changing a data type"17:17
*** panbalag has joined #openstack-keystone17:17
cmurphywell adriant ^ i wonder if there's something else we could do to solve your use case17:19
kmallocadded my comment re no-downtime upgrades as well to my -2.17:20
kmalloci am 100% ok reversing course on the -2 if cores/ptl say we need this.17:20
*** tesseract has quit IRC17:20
cmurphyi was on the fence about it which is why i asked you :)17:21
kmalloccmurphy: :) I appreciate being asked - it's always better to get extra eyes on something if you're on the fence ^_^17:23
kmalloci need to revisit:
kmallocit would make our code SO much easier to read/17:23
*** magicboiz has quit IRC17:24
kmallochmm. where did I stick that laptop. i can't write this on the desktop atm. it's being ... wonky.17:27
*** mvk has joined #openstack-keystone17:30
openstackgerritayoung proposed openstack/keystone master: Add is_admin_project check to policy for non scoped operations
*** magicboiz has joined #openstack-keystone17:36
*** thorst_ has joined #openstack-keystone17:41
*** notmyname has joined #openstack-keystone17:41
notmynameI though you should see this ;-)17:42
kmallocoh man.17:44
*** thorst has quit IRC17:44
kmallocthat's good :P17:44
*** magicboiz has quit IRC17:44
notmynameI'm not even exactly sure how to comment on it17:44
notmynamebut I'm sure it includes the illuminati, flat earth, and particular metal headwear17:44
notmynameand, of course, keystone being at the center of it all17:45
*** magicboiz has joined #openstack-keystone17:50
*** thorst_ has quit IRC17:51
kmallocI, I kindof want to make an image of it and make that keystone's new mascot :P17:52
*** aselius has joined #openstack-keystone17:55
*** thorst has joined #openstack-keystone18:02
lbragstad.. what...18:02
lbragstadin the world....18:02
lbragstadlol - we don't quite know what a CERN is, but CERN!18:03
lbragstadthat reminds me, we should totally update our wiki page18:06
*** thorst has quit IRC18:07
ayoungcmurphy, thansk for the review on  oslo-context.  I was able to limit my changes to just the middleware file, should make it a little easier for others to review.18:09
ayoungAnd, of course, I need it for the one I really care about:
ayounglbragstad, so...assuming we can get the oslo-context fix in, and I have not broken anything in the rebase, I have a working is_admin_project fix for Keystone, finally.18:13
ayoungThe one thing I need guidance on is how to deal with Tempest18:13
lbragstadi still need to review it18:13
ayoungright now, tempest uses a token with admin role scoped to the default domain18:13
ayoungI grandfathered that in18:13
ayoungits not ideal, but its Okish18:14
ayoungoslo-context patch is going to need a little follow up for code removal, too18:14
ayoungI think common/ can pretty much go away, with anything left merging into common/context.py18:15
*** thorst has joined #openstack-keystone18:15
ayoungso, if we could have the discussion about domain_id:default, role:admin sooner, it will let me move ahead18:16
ayoungkmalloc, lbragstad any preferences?18:16
lbragstadi think my biggest concern is keeping all  the is_admin_project bits abstracted away into oslo.context or oslo.policy18:17
lbragstadinstead of having it permeate into various policy files and whatnot18:18
*** thorst has quit IRC18:20
ayounglbragstad, so you want a single rule that can be encapuslated in oslo policy?18:23
ayounglbragstad, I kinda want that too.  What would it look like?18:23
lbragstadayoung: i worry about landing all this stuff and then having to re-teach people about system scope18:23
lbragstadif there is an opportunity to abstract that into something we can do for them, that might make that whole interaction easier18:24
*** thorst has joined #openstack-keystone18:24
lbragstadbecause we'll also have to think about deployments moving from is_admin_project to system scope eventually18:25
ayounglbragstad, like instead of a policy rule: role:admin and is_admin_project: True (or domain_id: default and is_domain_scoped: True)  we had a magic rule like oslo-context_admin:True?18:25
lbragstadayoung: maybe we can make it even more generic18:25
lbragstadwhat if you just map it to 'system'?18:26
*** panbalag has quit IRC18:26
ayoungsystem: admin18:26
lbragstadtechnically, isn't that what is_admin_project signifies?18:26
lbragstada system-level operation?18:26
openstackgerritMerged openstack/keystone master: Assert default project id is not domain
lbragstadwhat if i have a reader role on the is_admin_project?18:28
lbragstad(it's not really an admin role, but it does let me view all system-level things in the deployment)18:28
ayounglbragstad, so  how about this:  I modify my patch to have a single rule:systemrole_admin  which calculates it. That lets me get it in to Keystone.  Then, I'll modify the nova one to do the same thing.  At the same time, we can start working through the oslo-context code to abstract that way.  Once Oslo has it, it will be a one line change in the policy code, from rule:... to system_role:18:29
ayoungI kindof wanted to introduct that rule in this patch anyway, so this is good justification for doing that18:30
ayoungsystemrole:?  however you want to format it.18:30
lbragstadwe can actually do that with this -
lbragstadwhich allows you to associate scope (e.g. system, project) to policies18:31
lbragstadso it shouldn't need to be in the name18:31
lbragstadif that makes sense18:31
lbragstadinstead it is an immutable attribute of the policy object18:31
lbragstadwhich can be inspected at enforcement time18:31
ayounglbragstad, i think I liked it better when the solution was encapuslated inside of oslo-context18:32
ayoungoslo.policy should really drop the role: rule, as it is the only thing that it does that is not generic18:32
lbragstadoslo.policy is just making the rule objects more flexible so that operations know what level they are suppose to run at18:33
ayoungoh, I am ok with collections, which is what role: is.  We just don't need a one off there18:33
ayoungbut policy can't deal with nesting, which is what I think makes your patch tricky18:34
ayoungreally we want role:  system.admin or role: domain.admin18:35
lbragstadthat patch gives you the ability to say rule = policy.DocumentedRuleDefault(name='identity:create_endpoint', ..., scope_types=['system'])18:35
lbragstadthen the only thing you need in a policy file is the role18:36
lbragstadit helps remove the role scope check from policy.json18:36
lbragstadand encodes it into the operation18:36
ayoungI think we are using the term scope check to mean two things, which might get confusing18:37
*** thorst has quit IRC18:38
ayounguntil today, when we talked scope check, we meant "does the resourece.projectID =="18:38
lbragstadyes - sorry, i'm referring to something slightly different18:38
ayoungbut we didn't have to deal with domain vs project scoping ofr the role assignments18:38
lbragstadright - which is a scope check of the role assignment18:38
* lbragstad is ultra confused now18:39
lbragstadit's comparing the role assignment's scope to the intended scope of the operation being done18:39
lbragstadwhich is a bit different than the scope check we've previously talked about, and that is typically all on the service to do18:39
lbragstad(because it has to inspect information about the thing being acted on, and keystone don't know/shouldn't know any of that information)18:40
*** panbalag has joined #openstack-keystone18:41
*** thorst has joined #openstack-keystone18:41
*** panbalag has left #openstack-keystone18:41
*** thorst has quit IRC18:43
ayounglbragstad, pretty sure you and I see things the same way, just worried about confusing people.  Perhaps modify them: assignment scope versus resource scope?18:45
ayoungresource scope is almost always "do project ids match"18:45
ayoungexcept, of course in Keystone where we have some resources we manage via domain18:45
ayoungand now, with system scoping w have an implied resource scope match18:46
lbragstadassignment scope = is the role assignment made on the right target scope18:46
ayoungall VMs within Nova are withing the system scope, but in different projects18:46
lbragstadwelll - i see that a bit differently18:46
lbragstadi view VMs as being completely project scoped18:46
ayoungOK,  so, lets say we want to encapsulate this whole thing inside olso-context, so we don't need to go and modify every policy file every time18:47
kmallocFWIW, I like the in-oslo.policy encapsulation18:47
kmallocsorry, catching up on the convo now18:47
ayoungand, say we want to support is_admin_project today, and service_role as soon as it is ready18:47
ayoungkmalloc, fire away.  THe sooner we have a consensus the faster we can actually fix things18:48
ayoungand I'm flexible, so long as the approach can be done in a reasonable amount of time18:48
kmalloctl;dr, what lbragstad said. whatever we implement, as long as we don't need to re-teach system context/policies after18:50
kmalloci'm behind it18:50
ayoungkmalloc, did you mean you want the encapsulation in oslo-context?  Or did you really mean oslo-policy18:50
kmalloc^^ simple answer, regardless of where it lands18:50
openstackgerritLance Bragstad proposed openstack/oslo.policy master: Add scope_types to RuleDefault objects
lbragstadso - IMO, it makes sense for oslo-context to maybe know about what scope the token being used in the request has18:53
lbragstadbut it can also pass all that along to oslo.policy since oslo.policy has to know what scope the operation be made requires18:53
ayounglbragstad, what would a policy fragment look like for system admin if landed18:54
lbragstadi attempted to write up what that would look like18:54
lbragstadthe code snippets are super simple, and can be condensed, but i kept them in long form for readability18:55
ayoungso the scope type could never be specified in the policy file itself?18:55
lbragstadit should always be immutable18:55
lbragstadand never configurable in my opinion18:56
lbragstadwhen a service introduces a new api, they should evaluate at what layer that API interacts when the implement the policy for that API18:56
lbragstad(e.g. does this API operate on the system or project levels?)18:56
lbragstadi attempt to highlight that in the card18:58
lbragstador specifically call it out in the examples18:58
ayounglbragstad, 100%18:58
ayoungsplit the scope check from the role check18:58
lbragstadbut also split the assignment check from the role check :)18:59
ayoungand make the services implement the scope check in code18:59
ayoungcan you restate that in the terms we used above?18:59
lbragstadoslo.policy should be performing the role check and the assignment scope check18:59
*** jmlowe has joined #openstack-keystone19:00
ayoungsure...I'd call all of that the role check19:00
lbragstadbut that should allow us to not have to have things like "nova_admin" or "nova_reader" roles19:00
ayoungWhat about overrides?  Using a system scoped token to do project scoped operations?19:00
lbragstadthat shouldn't be possible19:01
lbragstad(e.g. you can't create an instance with a system-scoped token)19:01
ayoungneed to be able to do that.  Its a hard requirement19:02
lbragstadbecause the system isn't really responsible for instances, projects are though19:02
ayounglbragstad, I got caught on that one, too19:02
lbragstadthat kinda defeats the whole purpose of system-scope19:02
lbragstadif it's for backwards compatibility19:02
ayoungadministrators do not rescope  to projects to do cross project work19:02
ayoungthey demand a single token that will work multi-project19:03
ayoungI'd be OK with us dropping the role: admin for everything, and have more targetted roles:19:03
ayoungsystem: hypervisor  vs system:helpdesk for example19:03
ayounghypervisor can do things art rhe cell and hypervisore level, helpdesk can do project scoped operations anywhere19:04
lbragstadso - in that card, that is similar to the admin example i have19:07
lbragstadadmin on the system can create new services, roll out new endpoints, etc...19:07
lbragstadadmin on a project and manage all the instances in that projects, etc...19:08
lbragstadboth are admin roles, but depending on what it is assigned to, you can do different things19:08
lbragstads/admin roles/admin role assignments/19:09
ayounglets put a moratorium on the continued use of the role admin19:16
openstackgerritGage Hugo proposed openstack/keystone master: Migrate jobs to zuulV3
ayounganything you are tempted to call admin, find a new name for it19:16
openstackgerritLance Bragstad proposed openstack/keystone master: Ensure building scope is mutually exclusive
ayounglbragstad, walk me through how  will work and I can +2 . tool19:32
lbragstadayoung: once that lands we will cut a new release of oslo.policy19:33
ayoungnah, walk me through the mechnaisms prior to that19:33
ayoungsay we merged it today.19:33
ayoungWhat would that mean for Keystone and Nova19:34
lbragstadit would mean that we can do something like this19:34
ayoungThe existing policy rules in common/policies would get modified?19:34
ayoungadd in a polic_scope value for each one,19:34
ayoungso take my change here:19:34
lbragstadit's an optional attribute19:34
ayoungwhere I touched those files, those should all get policy_scope system19:35
ayoungall of the user and group ones would get policy_scope project19:36
ayoungwhat about create_domain19:36
lbragstadthat's essentially what i have staged once we get a release of oslo.policy19:36
ayoungyou can do that with a domain scoped token, or with a project scoped token, and it has different constraints19:36
ayoungis the dual scope types covered?19:37
lbragstadas in something can be done on the system and in a project?19:37
lbragstadyeah - scope_types is a list of strings19:38
ayounglbragstad, domain and project in this case.  Cool19:38
ayounglet me give it a firm once over...19:38
lbragstadso we can support that case19:38
lbragstadplease do19:38
ayoung# If the token isn't system-scoped then we're dealing with19:39
ayoung                    # either a domain-scoped token or a project-scoped token.19:39
ayoung                    # From a policy perspective, both are "project" operations.19:39
ayoung                    # Whether or not the project is a domain depends on where19:39
ayoung                    # it sits in the hierarchy.19:39
ayoung                    token_scope = 'project'19:39
ayoungis this right?19:39
lbragstadto my knowledge it is19:39
lbragstada domain is a project at the top of the tree, right?19:39
ayoungyou need a domain scoped token to add a user, but a project scoped token to add projects under HMT19:39
ayoungI'm not really sure I care, but lets not break anything.  hey kmalloc can you look at something for me?19:40
ayounglbragstad, this is probably OK as is, but I would expect to be able to distinguish between project and domain scoped tokens in general19:40
lbragstadso - if you ignore the concept of domains and just consider everything a project19:41
lbragstadugh - this would be easier with a whiteboard19:41
ayounglbragstad, you are pitting me against know how badly I want that to be true19:41
lbragstadlet's say domain-a is the root19:42
lbragstadit has two children, project-a and project-b19:42
lbragstadwhich would make them peers19:42
*** thorst has joined #openstack-keystone19:42
ayounglets say domain default is the root, as that is the case today19:42
lbragstadand domain default has two children19:42
ayoungnow, to add a new domain, I need a system scoped token19:42
ayoungto add a user or group to any of those domains, today I should be using a domain scoped token19:43
ayoungand you are saying that we make that a project scoped token, we just check, in the operation, that the project is-a domain19:43
ayoungand it would be implicitly converted to a project scoped token at the policy check19:44
lbragstadi suppose that would work19:44
ayoungso at the p[olicy check, what oslo context is doing today won't plauy nice19:44
lbragstadso project-a and project-b are peers, right19:44
ayoungif we get a domain scoped tokne, there is no user_proejct_* value set19:44
lbragstadproject-c is a child of project-b19:44
kmallocwhy the hell can i not copy/paste from gerrit19:44
ayoungnah...say at the root19:44
ayoungsay I have a token scoped to default domain with the gawdawful role19:45
ayoungand to creat a user I need to have gawdawful on a project that is-a domain19:45
*** dave-mccowan has joined #openstack-keystone19:45
ayoungthe project ID is not the same as the domain ID, I think, in the case of HMT. but I might be wrong19:45
lbragstadcreate user should be a 'system' level operation or a 'project' level operation19:46
lbragstadif i have a role on project-b that allows me to create users, i should be able to create users in project-b19:46
ayoungusers and groups are the only entities that are currently scoped to domains, not projects...except of course, projects can be scoped to either19:46
lbragstadkeystone should know to enforce things at the project-b level in that case19:46
ayoungyou can't create users in a project19:47
lbragstadif you wanted to support the project-admin case, you could be setting the default_project_id or something like that19:47
lbragstadi'm using it as an example19:47
ayoungso, what we should have had, instead of domains owning both projects and users is that the abstraction for users should be called IdPs19:47
ayoungand..lets assume we made that happen.  Why should we not be able to distinguish between a project scoped, a domain scoped, and and IdP scoped token?19:48
ayoungI'd liketo be able to theoretically have endpoint and service scoped tokens in the future19:48
ayoungwhat is the value of forcing everything not-system into a single buicket, and is that change set in stone?19:48
ayoungCould we adopt it today, and fix it tomorrow?19:48
lbragstadsure - but what you're dealing with is either a tree of projects, or a system tree19:49
lbragstada domain just happens to be a specific node in the project tree19:49
ayoungnot "just"19:49
ayoungit a node with specific semantics19:49
ayoungsystem is "just" a node in the tree...but with very special semantics19:50
ayounglbragstad, don't get me wrong, I like this change, I just want to know the ratioanle for this specific decision, as I think this is going to break things19:51
*** thorst has quit IRC19:51
lbragstadi built a kwarg into enforce that allows for thinsg to be backwards compatible19:51
ayoungwhen we do HMT, and we make a domain act like a project, do we give it the same ID as the domain id?19:51
ayounglbragstad, BTW, can you add me as a member on Trello?19:53
ayounglbragstad, TYVM.  Should I add a card for the oslo-context change?  We didn't realize we were not using it, and it is kindof important19:55
lbragstadayoung: sure - put the context in the card and i can find a place for it19:56
ayounglbragstad, done.  Added to "in progress"19:58
lbragstadcool - thanks19:58
kmalloclbragstad: hmm.19:59
kmalloci think my keystone checkout is broken =/19:59
ayounglbragstad, so do you think system roles will be ready soon? I can back off the is_admin_project work and skip right to using system roles.  I saw that you had a slew of reviews WIP19:59
kmalloclbragstad: why is TenantV2 assignment driver still in the tree?20:00
kmalloclbragstad: did we miss it?20:00
kmallocor did we need to wait?20:00
lbragstadayoung: all the assignment bits are ready for review
lbragstadkmalloc: good question - i though we removed that?20:00
lbragstadayoung: i'm working on a patch now to implement system-scoped tokens20:01
*** itlinux has quit IRC20:01
lbragstadthat's the last bit we need to implement system scope for keystone in queens20:01
ayoungcool.  I'll work on reviewing.20:01
lbragstadthen it's getting the scope_types stuff done in other projects20:01
kmalloci'll work around this for the moment.20:01
kmallocbut... we should look into why that is still there20:02
ayounglbragstad, why are system assignement going into their own table and not the existing assingment table?20:02
lbragstadayoung: the existing assignment table uses Enum20:02
lbragstadwhich happens to be a massive PITA20:02
ayoungso change that20:02
ayounglet me see...20:02
lbragstadthat's what i attempted to do first20:03
ayoungDid I do that?20:04
lbragstadbut changing that while allowing for an online migration was painful20:04
lbragstadso we left if20:04
*** thorst has joined #openstack-keystone20:04
lbragstadand decided to keep system assignments in their own table20:04
lbragstadwhich might be more advantageous in the future if we expand system into its own tree20:04
ayoungI don't think so....if assignements is broken, lets fix that...20:04
lbragstadthe assignment table isn't "broken"20:04
lbragstadit's just really hard to change the schema while doing an online migration20:05
lbragstadit's very hardcoded to the initial implementation20:05
ayoungadd a new column20:05
lbragstadthe type column is part of the primary key20:05
lbragstadso everything in that table needs to have a populated value in `type`20:06
ayoungYou can't add a value to an enum?20:06
lbragstadit's a frozen set20:06
lbragstadwhich requires you to change the schema20:06
ayoung"An ENUM is a string object with a value chosen from a list of permitted values that are enumerated explicitly in the column specification at table creation time. "20:06
lbragstadyeah - so we talked about it at the PTG20:07
lbragstadand figured it might be worth it to just keep system assignments as their own thing20:07
ayounghow about a whole new assignment table, then?20:07
lbragstadone dedicated to system assignment only20:08
lbragstadand not as restrictive20:08
lbragstadso that we can elaborate on it in the future20:08
ayoungno, one that covers all of them, without an enum20:08
lbragstadthen you have to do a migration20:08
ayoungI take it that was discussed as well, and dropped for some reason20:08
ayoungmigration is pretty straight forward, copy from one table to the other20:09
ayoungnot pretty, but, meh20:09
lbragstadadding a new table for system assignment needed to be done regardless, not copying all the data over didn't seem necessary and resulted in a lower bar20:09
lbragstadfor me, personally, i kinda liked the fact they were separate20:09
lbragstadusing Enum of a type when we have control of all the things from the manager and driver seemed a bit over-kill20:10
ayounglet me ask this, is the system assignemtn table capable of handling existing role assignments in the future?20:10
lbragstadexisting role assignment?20:10
ayoungdrop enum, migrate the assignments to the system assignment table?20:10
ayoungin, say, Tango time frame20:10
lbragstadi built the system_assignment table to not have Enum20:10
ayoungIt looks to me like it is20:10
lbragstadwhich means you can come up with new assignment types in the future without requiring a data base migration20:11
ayounglbragstad, so you still have a type column, just not as a nanum, right?20:12
ayoungand the primary key logic is the same20:12
lbragstadi also pulled a bunch of the logic to determine the assignment type into the manager20:12
lbragstadand made the driver moar dumb20:12
lbragstadyeah - primary key is still the same20:12
ayoungOK, lets drive on to success with this, but I think we'll want to merge into a single table in the future20:12
ayoungnot worth holding it up over20:13
ayoungI'm going to rebase to make sure it works, and I'll +220:13
openstackgerritayoung proposed openstack/keystone master: Add a new table for system role assignments
ayoungassuming all is good...20:13
ayoungmost concerned about the migration numbering20:13
lbragstadi don't expect any problems there, we haven't had much churn this release with migrations20:14
ayoungWe are up to 30.  Lets get this in now.20:14
ayounglbragstad, +2 from me on that.20:15
openstackgerritayoung proposed openstack/keystone master: Implement backend logic for system roles
lbragstadfwiw - those patches are all in a linear series20:16
ayoungthe word "grant" is such a henrynashism20:16
lbragstadayoung: also - when you start reviewing the manager logic, just a heads up20:16
ayoungalmost there...20:16
lbragstadi made a bunch of specific APIs for things that are explicit, instead of having one method and a bunch of kwargs20:17
ayoungI get it. And I like that approach...let me see.20:17
ayoungdriver code looks fine20:17
lbragstad^ that helped make the drivers super dumb to be honest20:18
openstackgerritayoung proposed openstack/keystone master: Implement manager logic for user+system roles
ayounglooks fine.  lbragstad you mean like how you hid inherited and type?20:19
lbragstadtype should be set in the manager20:19
lbragstadand just passed to the backend20:19
lbragstadi don't think the backend should be making those types of decisions anyway, that seems like business logic20:20
ayoungexactly the split we shouild be pushing.  COntroll is just web level, business logic in the manager, driver is data layer20:20
openstackgerritayoung proposed openstack/keystone master: Implement manager logic for group+system roles
openstackgerritayoung proposed openstack/keystone master: Add user system grant policies
ayounglbragstad, let me redo my is_admin_project patch so that we have a new rule name, and we can swap out that rule when this stuff gets merged.  Even id we hold off on my patch, we can rebase it on the system, role stuff when required20:24
lbragstadok - i'm going to get my patch up for system scoped tokens20:26
lbragstadtrying to get that done by EOD20:26
ayounglbragstad, the one change I'd like to make is to have the policy for system assignments tightened up ahead of time20:28
ayounginstead of admin_required, lets get a rule ion place for system roles, and use that20:29
lbragstadayoung: let's get the scope_types stuff merged too then?20:29
ayounglbragstad, I'm going to go out on a limb here and +2 that one.  Irealluy don't like that domain is collapsed into project, but I'll defer on this.  I think it will break things, but maybe it will break things in a good way.20:31
ayoungdamnit, he downgraded his +2 to a +120:31
lbragstadayoung: let's get another set of eyes on it then?20:31
ayoungone sec20:31
ayoungmaybe we can get doug to go back up20:31
ayounglbragstad, can we get the types into an actual array as opposed to a comma separated string?20:32
adriantkmalloc, cmurphy: thanks for the review! Just about to run to the office, but will be around on IRC in a few mins if you want to follow up, but I've added comments to the review that should clarify20:32
lbragstadayoung: it is an array20:33
lbragstadit's an array of strings20:33
ayounglbragstad, so what about his comment?20:33
adriantcmurphy, lbragstad: lance was right, I need it for project namespaces (see email thread link in the review)20:33
lbragstadayoung: it's when rendering the documentation20:34
lbragstaddefault.scope_types was being rendered as 'systemproject' when scope_types = ['system', 'project]20:34
lbragstadinstead of 'system, project'20:34
ayounglbragstad,so is there something to fix there?20:34
ayoungah, you got that in the next revision20:35
lbragstadi pushed a new patch set that uses .join()20:35
ayoungwhy did he downgrade  then?20:35
lbragstadayoung: he probably just wanted to make sure concerns were addressed before someone came through and kicked it in20:35
ayoungwould it be presumtuous of me to +@a it then?20:36
ayoungI think I'm the only policy core here20:36
lbragstadayoung: let's just double check with him quick20:36
lbragstador dims20:36
ayounglbragstad, consider my +2 sticky unless something goes radically different, or call me and I'll re-review.20:37
ayoungI'm focused on an internal project for the next 3 days, which is why the current flurry of activity20:37
ayoungI'll be back engaged on Friday20:37
lbragstadcool - i should have a bunch of stuff ready by then20:38
lbragstadwith the system-scope stuff20:38
openstackgerritayoung proposed openstack/keystone master: Add is_admin_project check to policy for non scoped operations
ayounglbragstad, lbragstad OK so ^^ has it down to a single rule.  When we get oslo-context support, we can change that rule: it will be a one line change for policy20:44
lbragstadi can take a look20:44
ayoungfigure out what you want oslo-context to look like, and let me know.  I can work on that.  I'll sync with jamielennox, too20:45
ayoungkmalloc, care to kick that one on through?20:50
*** jdennis has quit IRC20:50
kmallocayoung: sec.20:50
ayoungthat list might need some updating20:50
ayoungdims got it already20:51
kmallocayoung: making a stab and removing @dependency.provides/requires20:51
*** jaosorior has quit IRC20:51
kmallocoh to hell with some of these docstring pep8 things. D400... REALLY?!20:54
kmalloc"must end with a '.'20:54
kmalloc"first line should be in the imperative, "thi" not "this"20:55
kmallocwow. just wow20:55
kmallocworthless docstring linting ftw!20:55
kmalloctotally worthless20:55
*** thorst has quit IRC20:56
ayoungPedantry, thy name is Guido.20:57
kmallocok lets see how bad i screwed it all up with this change.21:02
* kmalloc runs tox -epy2721:02
kmallocit's passing pep8 fwiw21:02
*** thorst has joined #openstack-keystone21:02
*** McClymontS has joined #openstack-keystone21:03
*** McClymontS has quit IRC21:03
kmallocwow our tests are bad and assume a lot of things.21:04
*** itlinux has joined #openstack-keystone21:05
lbragstadkmalloc: we shouldn't need anymore should we?21:06
*** itlinux has quit IRC21:06
*** thorst has quit IRC21:07
lbragstadtechnically, that's only there to authenticate and validate v2.0 tokens, which we removed...21:07
lbragstadand we're not providing any v2.0 token validation compatibility from pike -> queens, right?21:08
lbragstadbecause... that wouldn't make sense?21:08
kmallocin theory we should't21:09
lbragstadwe should support that case?21:09
*** thorst has joined #openstack-keystone21:17
ayoungKILL IT DEEEEAD!21:17
*** nicolasbock has joined #openstack-keystone21:19
*** jdennis has joined #openstack-keystone21:19
SamYaplei dont want to give people false hope... i say we `sed -i '/v2/d' *`21:19
*** thorst has quit IRC21:20
*** thorst has joined #openstack-keystone21:21
openstackgerritayoung proposed openstack/keystone master: Remove dead code for auth_context
cmurphyadriant: responded on the patch, thanks for elaborating but i don't have a better suggestion at the moment :/21:33
*** thorst has quit IRC21:36
adriantcmurphy: that's right we kind of call the vew API additons as versions (3.9 is the current one), but it isn't really microversions.21:38
ayoungthe first path is +52, -39  but the second is +0, -26421:38
ayoungthat is still a net -251 which I consider a win21:39
ayoungwe at 3.9?  Next wone should be 4.021:39
ayoung3.10 is weird21:39
*** pcaruana has quit IRC21:39
adriantayoung: but that will sound confusion since it isn't a true v4 :P21:40
adriantjesus I can't english today21:40
adriantcmurphy: yeah, I don't know what to do. I think kmalloc is right, it is a breaking change since we don't know what/who it might affect, but it's mildly frustrating regardless. :(21:43
cmurphyadriant: yeah :(21:43
adriantsince we are getting rid of v2, can we do v4 now? :P21:43
* adriant is only joking21:43
cmurphywe should start making a list of everything we want to break for v421:44
lbragstadayoung: SamYaple ack - removing it21:45
adriantcmurphy: would be an interesting list to see21:46
*** raildo has quit IRC21:47
SamYaplecmurphy: we should get rid of domains in v421:47
adriantSamYaple: nah, domains are useful. We'd need a different but similar concept instead then.21:48
SamYaplesame-same, but different21:48
SamYaplei just wish there was a way to cleanly correct weird apis inconsistencies *without* carrying the broken old inconsistent way along with the new consistent way21:50
*** thorst has joined #openstack-keystone21:50
SamYaplebut i think thats called a time machine21:50
adriantpretty much21:50
adriantright now I'd love to know why someone thought users should have names of 255 but projects only 6421:51
SamYaplemy legal name is longer than 64 characters21:51
lbragstadimo, a domain should be a root level project, but not called a domain21:51
SamYaplelbragstad: agreed. andi thought there was work towardthat back in mitaka....21:52
adriantlbragstad: so you'd set auth/federated user source on a root project?21:52
cmurphyfrom the thread it seems like they would have preferred everything be 64 but had to make usernames longer because ldap21:52
SamYapleisnt there an is_domain field in the project table?21:52
adriantdomains are projects! But we still have to keep the domain api.21:52
SamYapleyea :/21:53
SamYaplewell v4! good luck selling that one lol21:53
adriantthat api is gone in v4, so you are right, domains go away :P21:53
lbragstadwell - before we even get to that point, i think we should do kmalloc's idea of decoupling auth and making it versionless21:53
*** thorst has quit IRC21:54
SamYaplelbragstad: as in, support what we call v2.0 and v3 and potentially "v4" auth in the same way?21:54
lbragstadSamYaple: just don't have v3 in the url path21:54
lbragstador a version in the url path21:55
lbragstadinstead, it could be discoverable21:55
adriantbut how does that work when your source of users is in part defined by the underlying data model that is related to a given version?21:55
adriantdomain/root project/etc bound to sql or federated source21:55
SamYapleah i see. i thought auth was already discoverable. but maybe thats just middleware abstraction21:55
lbragstadadriant: i'm not sure i understand the question21:56
adriantlbragstad: sorry, lemme try and elaborate.21:57
openstackgerritMerged openstack/oslo.policy master: Add scope_types to RuleDefault objects
adriantlbragstad: how you authenticate, is based on the underlying models. If the backend is sql, it has certain requirements, if it's a a federated source, then it has others, and to control where you authenticate from is also managed by the underlying data models that wouldn't be able to be versionless per say.21:58
ayoungadriant, domains die.  FOr projects, we make projects strictly hierarchical.   FOr users and groups, we use IdP21:59
adriantI could be entirely wrong, or asking the wrong question.21:59
ayoungnah, you are right on22:00
adriantayoung: but that assumes we entirely from the iDP like elements from keystone, which we can't really do22:00
openstackgerritLance Bragstad proposed openstack/keystone master: Remove private methods for v2.0 and v3 tokens
openstackgerritLance Bragstad proposed openstack/keystone master: Teach TokenFormatter how to handle system scope
ayoungadriant, ah, but yes we can22:00
ayoungwe call all local domains IdPs22:00
ayoungand remote Idps get protocols22:00
adriantayoung: so we would make a mini idp as part of keystone and just that 'remotely' like any other?22:02
adriantand just use* that22:02
ayoungadriant, yep22:03
ayoungadriant, which, in essence, we already have22:03
adriantayoung: expect without the duplication of keystone idp users and a linked shadow user for the authorisation component.22:04
adriantbut yeah, I get what you mean22:05
*** rcernin has joined #openstack-keystone22:07
adriantugh, I still need to find time to write up and submit a spec for some MFA bits. lbragstad, spec freeze for keystone just happened or is about to?22:08
adriant says  Dec 04 - Dec 08 so I potentially still have a couple of days :P22:10
*** thorst has joined #openstack-keystone22:10
lbragstadadriant: yep - it happens this week22:11
cmurphyproposal freeze was Oct 20 though :P22:12
adriantcmurphy: blast!22:12
adriantWell, I'll write it up anyway and start prototyping code. If I can convince you potentially otherwise I shall, otherwise we can approve/merge stuff first thing after queens :)22:13
lbragstadcmurphy: aha - yes.. good call22:14
*** thorst has quit IRC22:14
*** spilla has quit IRC22:16
*** phalmos has quit IRC22:22
*** ayoung has quit IRC22:53
*** thorst has joined #openstack-keystone22:53
*** thorst has quit IRC22:58
*** ayoung has joined #openstack-keystone23:03
openstackgerritGage Hugo proposed openstack/keystone master: WIP - Make fernet config and utils generic
*** gmann_afk is now known as gmann23:27
*** thorst has joined #openstack-keystone23:33
*** thorst has quit IRC23:33
*** notmyname has left #openstack-keystone23:41
*** efried is now known as efried_cya_wed23:41
*** thorst has joined #openstack-keystone23:45
openstackgerritColleen Murphy proposed openstack/keystone master: WIP Add application credentials driver
openstackgerritColleen Murphy proposed openstack/keystone master: WIP Add Application Credentials manager
openstackgerritColleen Murphy proposed openstack/keystone master: WIP Add Application Credentials controller
openstackgerritColleen Murphy proposed openstack/keystone master: WIP Add application credential auth plugin
kmallocayoung: oh man i think i have this patchset ready23:48
kmallocayoung: running tests now. need to fix a couple things still, but... it's way better23:49
*** thorst has quit IRC23:49
openstackgerritMorgan Fainberg proposed openstack/keystone master: WIP: Remove Dependency Injection
*** thorst has joined #openstack-keystone23:51
kmalloclbragstad, ayoung: ^ this is way way way way better. it wont past tests (i think 2-3 are failing)23:53
kmallocbut it is the right direction.23:54
*** jmlowe has quit IRC23:55
*** thorst has quit IRC23:55

Generated by 2.15.3 by Marius Gedminas - find it at!