17:01:30 <lbragstad> #startmeeting keystone-office-hours 17:01:31 <openstack> Meeting started Tue Jun 26 17:01:30 2018 UTC and is due to finish in 60 minutes. The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:01:32 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:01:35 <openstack> The meeting name has been set to 'keystone_office_hours' 17:01:48 <kmalloc> if anyone has questions on the RBACEnforcer, i know it's super dense. 17:02:00 <kmalloc> I can speak to it and a lot of the quirks in our policy code. 17:03:35 <lbragstad> awesome 17:03:44 <lbragstad> i'm going to grab lunch quick and i'll be right back 17:03:56 <lbragstad> knikolla: and i were going to try and tag team a few bugs today 17:04:01 <lbragstad> #link https://bugs.launchpad.net/keystone/+bug/1658641 17:04:01 <openstack> Launchpad bug 1658641 in OpenStack Identity (keystone) "Moving/disabling LDAP users break Keystone queries depending on role ID" [Medium,In progress] - Assigned to Kristi Nikolla (knikolla) 17:04:10 <lbragstad> #link https://bugs.launchpad.net/keystone/+bug/1757022 17:04:10 <openstack> Launchpad bug 1757022 in OpenStack Identity (keystone) ""keystone-manage mapping_purge" ignores --type option" [Undecided,In progress] - Assigned to Dai Hanada (dai-hanada) 17:04:18 <lbragstad> #link https://bugs.launchpad.net/keystone/+bug/1775207 17:04:18 <openstack> Launchpad bug 1775207 in OpenStack Identity (keystone) "Fetching all mappings may become too slow" [Undecided,In progress] - Assigned to Pavlo Shchelokovskyy (pshchelo) 17:05:26 * knikolla going for lunch 17:05:29 <kmalloc> lbragstad: i'm going to try and get the "move an API" patch up today. 17:05:45 <kmalloc> so its easier to see how the flask stuff actually shakes out. 17:05:48 <gagehugo> o/ 17:06:07 <lbragstad> that'd help 17:23:17 <kmalloc> lbragstad: yeah, so working on the scaffolding update patches now and then api move will be soon 18:01:24 <pas-ha> lbragstad: hi, re bug 1775207, I noticed you've put an 'office-hours' tag on it - wdym and is my attention required/expected? 18:01:24 <openstack> bug 1775207 in OpenStack Identity (keystone) "Fetching all mappings may become too slow" [Undecided,In progress] https://launchpad.net/bugs/1775207 - Assigned to Pavlo Shchelokovskyy (pshchelo) 18:01:56 <lbragstad> pas-ha: we use the office-hours tag as a way to focus on a specific set of bugs or reviews 18:02:17 <pas-ha> oh, ok, just saw it mentioned in the scrollback :) 18:02:26 <lbragstad> we had a user come through the channel yesterday and we noticed a few reviews related to keystone-manage that could use some attention 18:02:40 <lbragstad> i added the tag to it so that we could hopefully get some eyes on it 18:09:07 <knikolla> lbragstad: i have a meeting now, but will join you in 1 hr or so for the ldap stuff 18:09:27 <lbragstad> sounds good - cleaning up one of the patches now, should be ready for review by then 19:15:47 <openstackgerrit> Lance Bragstad proposed openstack/keystone master: Fix keystone-manage mapping_purge with --type option https://review.openstack.org/554397 19:16:09 <lbragstad> knikolla: ^ those could be a bit more dry - but they're functional 20:03:14 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Add support for enforce_call to set value on flask.g https://review.openstack.org/578189 20:03:14 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Update Scaffolding (flask) for json home documents https://review.openstack.org/578190 20:04:04 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Update Scaffolding (flask) for json home documents https://review.openstack.org/578190 20:10:34 <knikolla> lbragstad: looking 20:28:13 <knikolla> lbragstad: looks good to me. +2 20:29:28 <lbragstad> knikolla: cool - thanks 20:29:39 <lbragstad> i'm a little worried about the duplication 20:30:06 <lbragstad> but i'm open to refactoring it if we can find a better way 20:30:28 <knikolla> i generally like tests to be verbose. 20:31:29 <knikolla> duplication in that case should be fine as it makes it pretty clear what the test is doing. 20:31:36 <lbragstad> that's fair 20:31:37 <knikolla> but that's just my opinion :) 20:36:02 <aning_> Hi lbragstad, I use your fernet-inspector to inspect a fernent-token, the result is this: 20:36:05 <aning_> fernet-inspector -k /opt/cgcs/keystone/fernet-keys gAAAAABbMpejHDDFLNkopYu5_PrFMKo16qidKmOXe5NvctVmja1FxqNBglzJcpma5CqiWG9L7YIVHuXlL29KotzdeHdA50IThiPhzKGREGhpVtKHFoRkGHRRHNK9VRpKSQpj7eTaKBDrRDc61NJ46H1Hh2VARmj1kv3andlwZ9ztHUYvipv86Ng 20:36:05 <aning_> [2, [True, '\xd3]\xb3{\x1c{B\xed\x8e\x9b\xe8\xc1`\x81M`'], 2, [True, '\xe6\x99u\xe0\xf4\xbdI-\x8b\x9bF%J\xbd\\X'], 1530045875.0, ['NP0\xfe\x08TC\xa4\x83\xc2\xc5\xdb\xe4;\x88;']] 20:37:19 <aning_> the Audit id from base64.urlsafe_b64encode('NP0\xfe\x08TC\xa4\x83\xc2\xc5\xdb\xe4;\x88;') is 20:37:31 <aning_> 'TlAw_ghUQ6SDwsXb5DuIOw==' 20:37:57 <aning_> And the UUID from uuid.UUID(bytes='\xd3]\xb3{\x1c{B\xed\x8e\x9b\xe8\xc1`\x81M`').hex is 20:38:11 <aning_> 'd35db37b1c7b42ed8e9be8c160814d60' 20:38:58 <aning_> [True, '\xe6\x99u\xe0\xf4\xbdI-\x8b\x9bF%J\xbd\\X'] in the middel after the second number 2, what is it? 20:39:27 <aning_> and what's Audit id? 20:40:32 <aning_> where are user id and project id hidden in the decoded data? 20:40:39 <lbragstad> https://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/token_formatters.py#n452 20:40:58 <lbragstad> this is going to go into the implementation details a bit 20:41:14 <lbragstad> but keystone users different payload classes to pack up the payload before encrypting it 20:41:24 <lbragstad> which keeps the two things separate 20:41:36 <lbragstad> (building of the payload from the thing that actually does the encryption) ] 20:41:47 <lbragstad> each payload has a version 20:42:05 <lbragstad> which is the first thing in the list when you decrypt a token 20:42:50 <lbragstad> so - in your example, you're dealing with a ProjectScopedPayload because the first element of the list is an integer of 2 20:43:56 <lbragstad> the ProjectScopedPayload returns a tuple which gets used here - https://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/token_formatters.py#n158 20:44:14 <lbragstad> notice that the version is coming from the payload classes that was used to build the payload 20:45:18 <lbragstad> the second integer is a compressed representation of the authentication methods associated with that token 20:45:21 <lbragstad> https://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/token_formatters.py#n464 20:45:36 <knikolla> lbragstad: my brain is fried for now. i'll head home and then work on https://review.openstack.org/#/c/487579/ later tonight 20:45:44 <lbragstad> we do this instead of passing method: ['password', 'token'] 20:45:47 <lbragstad> knikolla: sounds good 20:46:39 <lbragstad> aning_: because using methods: ['password', 'token'] in a token payload bloats it significantly, so we convert the configured authentication methods to a unique integer that can be reinflated at validation time 20:47:24 <lbragstad> see https://git.openstack.org/cgit/openstack/keystone/tree/keystone/auth/plugins/core.py#n46 20:47:33 <lbragstad> and https://git.openstack.org/cgit/openstack/keystone/tree/keystone/auth/plugins/core.py#n63 21:12:03 <aning_> Sorry I was pulled away for while ... these are very valuable information. 21:12:29 <aning_> but jus from a high level, I saw three hex strings 21:12:53 <aning_> The first one is UUID, the last one is Audit ID, what's the middle one? 21:15:08 <aning_> If I guess, it should be password 21:16:21 <lbragstad> this is the payload 21:16:23 <lbragstad> https://git.openstack.org/cgit/openstack/keystone/tree/keystone/token/token_formatters.py#n469 21:16:29 <lbragstad> or the format of the payload 21:16:37 <aning_> or token depends on the integer before it, since that integer is the auth method. 21:17:00 <lbragstad> so version = 2 21:17:12 <lbragstad> b_user_id is [True, '\xd3]\xb3{\x1c{B\xed\x8e\x9b\xe8\xc1`\x81M`'] 21:17:20 <lbragstad> 2 is the methods 21:17:22 <aning_> right, version = 2 in my example. 21:17:35 <lbragstad> b_project_id is [True, '\xe6\x99u\xe0\xf4\xbdI-\x8b\x9bF%J\xbd\\X'] 21:17:49 <lbragstad> expires_at_int is 1530045875.0 21:18:02 <lbragstad> and b_audit_ids is ['NP0\xfe\x08TC\xa4\x83\xc2\xc5\xdb\xe4;\x88;'] 21:18:18 <aning_> Great 21:18:58 <aning_> so audit id contains credentials? 21:19:26 <lbragstad> nope - audit ids are a specific property of a token 21:19:31 <aning_> probably not, since there is no need for credentials in token ... 21:19:35 <lbragstad> right 21:19:52 <aning_> Ok got it 21:19:58 <lbragstad> an audit id is generated whenever you create a token 21:20:13 <lbragstad> we call them audit ids because they help us track which tokens are related 21:20:19 <lbragstad> so - for example 21:20:39 <lbragstad> if you authenticate for a token using your username and password you'll get back a token 21:20:49 <lbragstad> which will have an audit id 21:20:58 <lbragstad> if you use that token to reauthenticate for a new token 21:21:17 <lbragstad> your new token will contain a list of audit ids, one of which will be the audit id of the first token you authenticated for with your password 21:22:18 <lbragstad> since tokens are non-persistent, audit ids help us when a user wants to "delete" a specific token 21:22:52 <lbragstad> we can persist the audit id of the deleted token, and flag it as invalid if we ever attempt to validate a token with that audit (decrypted from the token payload) 21:23:55 <aning_> ok 21:25:20 <lbragstad> that's a lot of details about the internal guts of keystone token system... hopefully it makes sense 21:26:49 <aning_> Yes, it all makes sense ... wouldn't get them anywhere else. Fantastic! 21:28:01 <aning_> Rather complicated, need time to dig and digest. 21:28:17 <kmalloc> lbragstad: i'm trying to avoid a massive rebase/reset the stack https://review.openstack.org/#/c/577586/ 21:28:19 <kmalloc> thats all 21:28:25 <aning_> Thanks a lot 21:28:43 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Update Scaffolding (flask) for json home documents https://review.openstack.org/578190 21:29:16 <lbragstad> aha 21:29:21 <kmalloc> this stack is a bit unweildy as is. 21:29:28 <kmalloc> just because it is a LOT of moving parts. 21:29:31 <lbragstad> yeah 21:29:42 <lbragstad> aning_: no problem 21:29:54 <kmalloc> and keeping my brain in one place at a given time has been hard, touches a lot of really overly complex parts. 21:30:29 <lbragstad> kmalloc: do we need this bit though? https://review.openstack.org/#/c/577586/1/requirements.txt 21:31:05 <lbragstad> shouldn't we be able to get away with just Flask>=1.0.2 21:32:10 <kmalloc> well, we need to adhere to what is in reqirements 21:32:16 <kmalloc> i suck and forgot to remove that part :P 21:32:43 <kmalloc> https://github.com/openstack/requirements/blob/master/global-requirements.txt#L62 21:32:47 <kmalloc> *oops* 21:33:00 <kmalloc> i dunno if the checker will get cranky or not with removing that 21:33:54 <kmalloc> i know this stack is getting deep =/ 21:34:05 <kmalloc> and it's not super easy to follow because of what it touches to begin with 21:34:54 <kmalloc> but fwiw, the "dummy API" will be stood up in https://review.openstack.org/#/c/578190/ [the full end-to-end test] 21:35:00 <lbragstad> ok 21:35:05 <kmalloc> now that I have json_home scaffolding in place. 21:35:39 <kmalloc> fwiw, my brain is fried as hell working on these now =/ testing the RBACEnforcer took 3 days to write the tests. 21:35:56 <lbragstad> yeah... 21:36:07 <lbragstad> the good thing is that most of the stack leading up to that looks good 21:36:11 <lbragstad> at least IMO 21:37:03 <lbragstad> getting those through the gate will give us time to parse the RBACEnforcer change 21:39:03 <kmalloc> the NITs on the 404/418 one, do you want me to fix and rebase or as a side-addendum patch 21:39:16 <lbragstad> i'm not sure i have a solution for it... 21:39:36 <lbragstad> i'm not sure what the fix would be, it was just a concern 21:39:42 <kmalloc> i meant the other nits 21:39:57 <kmalloc> the 418 bit, i can pick another status_code [any] 21:40:14 <kmalloc> i also added the expressive comment to explain this is a testing-only-thing and what it means 21:40:34 <kmalloc> right below your review-comment (the code-comment is expressive that is) 21:40:43 <lbragstad> ahh 21:40:58 <lbragstad> that one is pretty late in the chain 21:41:11 <lbragstad> if you rebase it's only going to affect 4 patches, right? 21:42:21 <kmalloc> yeh, the enforcer patch and the newest ones on top of it 21:42:36 <kmalloc> i am hesitant to rebase the enforcer if people are actively reviewing... 21:42:53 <lbragstad> oh - sure 21:42:54 <kmalloc> but i also realize that is unlikely with the current preceeding patches not fully reviewed 21:43:17 <lbragstad> i'm just about to wrap up my review of the RBACEnforcer patch 21:43:28 <kmalloc> cool. 21:43:56 <kmalloc> i'll add an addendum patch to the 418 one to address the nits and we can swap out the expected_status bit to a different code if we want at anytime 21:44:14 <kmalloc> it's 2 lines to swap to someting else... 4 if you count the comment and the error msg 21:48:04 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Address minor comments to 404 error detection https://review.openstack.org/578216 21:59:57 <lbragstad> #endmeeting