17:20:12 #startmeeting keystone-office-hours 17:20:13 Meeting started Tue Sep 18 17:20:12 2018 UTC and is due to finish in 60 minutes. The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:20:13 oops 17:20:14 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:20:16 The meeting name has been set to 'keystone_office_hours' 17:42:13 Photo and video of damage, destruction caused by Hurricane Florence - The Washington Post (https://www.washingtonpost.com/graphics/2018/national/amp-stories/photo-and-video-of-damage-destruction-caused-by-hurricane-florence/) 17:42:44 I lived in Fayetteville (pictured) for a few years (my first home in NC). Pretty bad 17:42:46 o/ 17:42:55 \o 17:43:15 back from the dr. office yay 17:45:34 lbragstad: no need to backport, that bug (get_domain_id_from_token) was introduced when groups were added 17:45:44 lbragstad: to flask, so ... not long ago 17:45:53 def. post Rocky 17:45:57 ack 17:45:59 thanks! 17:46:55 o/ 17:47:32 lbragstad: and additional bugs/proposed fixes are closed out now that that has merged 17:49:15 ++ cool 17:54:06 Morgan Fainberg proposed openstack/keystone master: Comment out un-runnable tests https://review.openstack.org/603459 17:54:38 lbragstad: ^ that is in lieu of deletion of the tests to allow ayoung to respin them on top of flask auth. 17:59:33 Morgan Fainberg proposed openstack/keystone master: WIP: Convert auth to flask native dispatching https://review.openstack.org/603461 18:02:48 hrybacki: hah, flaskification has only been 67 commits *so far* 18:03:06 we might get it all done in under 100 commits (oh man, so glad we didn't make this an intern project) 18:03:11 lbragstad: ^ 18:05:16 heh, that would be one frazzled intern 18:06:52 kmalloc: so how do I determine if the tags portion of the project API should be a resource or a `ks_flask.ResourceBase` or a `flask_restful.Resource` ? 18:08:01 hrybacki: gut feeling :P 18:08:18 heh, ResourceBase it is 18:08:54 hrybacki: typically a "resource" conforms as follows: A number of concurrent operations (GET/POST/PATCH/DELETE) in a single case, easy prefixing, and not a lot of path substitutions (easier to represent in a mapped resource) 18:09:11 hrybacki: basically, if you need to go through hoops to make resource work, use mapped resource instead 18:09:21 ack 18:12:12 kmalloc ack 18:55:49 hrybacki might be better off doing mapped resource, you have {project_id} & {tag} 18:57:11 gagehugo: ack :) 18:59:29 https://github.com/openstack/keystone/blob/master/keystone/resource/routers.py 19:00:03 the update without {tag value} expects a body, so doing it as non-mapped might be weird 19:01:22 you may need to override wrap_member as well, as tags was written to follow the APIWG spec and it was specific on what should be returned (aka it was different than what keystone does by default) 19:01:54 if you see anything weird let me know and I can help 19:01:59 regarding that 19:06:56 cmurphy, forgive my ignorance, but what do the <> signify in the tables related to assignment? (e.g., <>) 19:07:14 And do I need to include them when I'm inserting values? 19:27:48 I included them and it worked :) 19:28:03 Now I've just got to figure out why keystone-manage bootstrap is exiting 1 with no output. 19:41:49 I just lugged a Dell R910 up to my desk from the parking deck. I am no longer in Army shape -_- 19:47:46 :) 20:10:04 Hi. I need to test my token proposal, but I have not had any OpenStack programming experiment, yet. Where should I start? 20:19:27 hi 20:21:31 Harry Rybacki proposed openstack/keystone master: WIP: Convert projects API to Flask https://review.openstack.org/603451 20:47:57 lbragstad: "Although the trains sound the same, much has changed in keystone since then." 20:48:10 that sentence is... poetic 20:48:38 lol 20:49:04 how'd you find that post so quickly? 20:50:00 lbragstad: was going through planet openstack 20:50:08 and apparently the timing was uncanny as it was the topmost 20:53:07 huh 20:53:28 no kidding 21:25:23 what's people's opinion of medium.com? i'm moving to yet another platform :/ 21:27:15 I've seen some decent articles on there 21:35:35 knikolla i have no idea 21:35:47 i've never used it 21:36:09 https://medium.com/behind-clouds 21:36:12 started a blog 21:36:20 i'll probably be hosting my stuff there moving forward 21:45:18 * lbragstad bookmarks 02:31:43 Vishakha Agarwal proposed openstack/keystone master: Adding test case for MappingEngineTester https://review.openstack.org/603539 03:06:12 Tao Li proposed openstack/keystone master: Use uuidutils instead of uuid.uuid4() https://review.openstack.org/603542 04:10:05 wxy-xiyuan: Hello. Can you pl review this test caes https://review.openstack.org/#/c/603539/ 04:14:27 cmurphy: Pl have a look on https://review.openstack.org/#/c/589378/. Need one more +2 07:01:23 Colleen Murphy proposed openstack/keystone master: Convert legacy functional jobs to Zuul-v3-native https://review.openstack.org/602452 08:01:16 Tao Li proposed openstack/keystone master: Use uuidutils instead of uuid.uuid4() https://review.openstack.org/603542 11:38:46 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: Implement base for pluggable policy drivers https://review.openstack.org/577807 12:30:39 hello 12:32:14 I have an issue with keystone middleware in rocky 12:32:43 I'm trying mistral and I use a memcache server 12:33:26 when installing mistral in a venv, I noticed I don't have python-memcached installed which causes an ImportError in keystonemiddleware 12:34:22 here precisely: https://github.com/openstack/keystonemiddleware/blob/stable/rocky/keystonemiddleware/auth_token/_cache.py#L77 12:35:34 python-memcached seems only required for the tests (according to the METADATA file of the egg) 13:03:13 Colleen Murphy proposed openstack/keystone master: Convert legacy functional jobs to Zuul-v3-native https://review.openstack.org/602452 13:03:54 Hey, knikolla, wondering if you've got any time to help with the keystone/shibboleth federation. I got past the PAOS response error, but am still stuck with a fatal profile exception 13:15:48 Colleen Murphy proposed openstack/keystone master: Convert legacy functional jobs to Zuul-v3-native https://review.openstack.org/602452 13:26:22 cmurphy: tox_envlist: all ? 13:27:21 cmurphy: would tox_envlist: functional be a better choice there? 13:27:23 mordred: hi do you want to help me 13:27:31 i'm following https://docs.openstack.org/devstack/latest/zuul_ci_jobs_migration.html 13:28:11 OH - 13:28:19 that's going to run tox in the tempest repo 13:28:22 *duh* 13:28:27 * mordred isn't always smart 13:29:01 heh okay 13:29:11 because without it this was happening http://logs.openstack.org/52/602452/3/check/keystone-dsvm-functional/5b30bc0/job-output.txt.gz#_2018-09-18_15_11_26_153936 13:29:47 yeah. makes totally more sense now 13:30:34 i still don't know what i'm doing though, just trying to poke it till it works 13:36:15 o/ 13:37:23 lbragstad: do you have advice for pgaxatte ^ i don't remember where we landed on that 13:38:08 on making python-memcache a hard requirement? 13:38:15 yeah 13:38:43 i don't remember a clear outcome :( 13:38:48 but... 13:39:01 I think my initial knee-jerk reaction is to just include it officially 13:39:40 (i thought something with memcache was python three related, though) 13:39:43 Hi guys, is application credential stable in queens release? 13:42:05 aning it is 13:42:36 queens was the initial release of application credentials and they were marked as stable from the beginning 13:42:45 Thx. 13:43:15 What other openstack services support application credentials if any? 13:43:49 Is there any information about that? 13:47:40 application credentials are mostly meant to be used by end users, but any openstack service that uses keystonemiddleware automatically supports using application credentials for service user authentication 13:49:48 cmurphy: but do we have to configure the [keystone_authtoken] section of the openstack service in order for it to use application credential? 13:50:34 aning: yes, there's an example here https://docs.openstack.org/keystone/latest/user/application_credentials.html#using-application-credentials 13:51:25 Is it that once the service is configured with application credential, the service just natively and automatically use it? 13:52:15 cmurphy: yep, I'm readning that document 13:52:55 aning: after creating the application credential and configuring [keystone_authtoken] like that it should just work 13:54:12 cmurphy: ok ... I'm thinking does the service need some (extra) logic to read, understand, and send the application credential to keystone for authentication? 13:55:15 If this is all handled by keystonemiddleware, I could understand the serive doesn't need any change ... 13:55:15 aning: no, the service shouldn't need to do anything special, it's all handled in keystonemiddleware and keystoneauth 13:55:29 cmurphy: yeah, got it. 13:57:37 cmurphy: btw, which table is the application credential hash stored? 13:58:15 credential? 13:58:49 aning: 'application_credential' 13:59:00 'credential' is for something else 14:00:21 cmurphy: so a new table is introdued. 14:00:30 aning: yes 14:01:54 cmurphy: thx 14:02:19 aning: np 14:11:34 that reminds me - i'd like to try and work on an example for using app creds for a single support user 14:46:06 in case anyone wants to double check my work - https://review.openstack.org/#/c/603822/ 14:46:30 ^ those are very similar deadlines to what we used for Queens 14:54:59 hrybacki the audit that gmann did here here is kind of similar to what you were doing (minus adding the granularity) https://review.openstack.org/#/c/547850/ 14:57:13 * hrybacki looks 14:58:32 lbragstad: looking at specs on gerrit can be the worst. I'll review that this afternoon 14:59:07 yep - no problem 15:06:06 cmurphy, lbragstad: reading keystonemiddleware code I don't understand the comment https://github.com/openstack/keystonemiddleware/blob/master/keystonemiddleware/auth_token/_cache.py#L92 15:06:38 it says to rely on oslo_cache to lazily import memcache 15:07:06 but having oslo.cache also in my virtualenv has not added python-memcached as a dependency 15:07:53 also, as soon as there is an "import blah" in the code, shouldn't it make the package blah a hard dependency? 15:08:42 kmalloc: want to comment on that ^ 15:12:34 kmalloc: I'm looking at the API ref to but can't find anything the HEAD endpoints for the v3/projects//tags and ve/projects//tags/ -- should they respond in the same way as a GET in this case? 15:12:39 https://developer.openstack.org/api-ref/identity/v3/#create-project 15:12:53 hrybacki: head uses get unless explicitly defined. 15:13:25 If the behavior of head is fundamentally different than get (some of our APIs) then implement head specifically 15:13:31 cmurphy: answering now 15:13:46 kmalloc++ thanks for confirming. So how are we handling that with flask? /me keeps digging 15:14:05 hrybacki: just implement head when needed. 15:14:15 ah, 'magic' 15:14:34 hrybacki: otherwise just let get work how it is supposed to (same behavior between head and get) 15:15:58 hrybacki: flask automatically removes content on head if it falls through to get (flask restful) 15:17:14 pgaxatte: so, in our code, most of the time you can use different options on the backend. If you never use memcache, there is no reason to install it. 15:17:36 pgaxatte: so, it is not a hard dep. Many folks use other options/no caching. 15:18:16 pgaxatte: if it was pymemcache vs python-memcached we might make it a hard dep. 15:18:24 cmurphy: ^ :) 15:18:35 kmalloc: ty 15:18:43 kmalloc: I understand but this makes mistral, in my case, break horribly 15:19:29 pgaxatte: explicitly add python-memcached in Mistral or just know you need to install it. 15:19:30 kmalloc: so I should install python-memcached separately as a dependency of my architecture 15:19:35 Yeah 15:19:38 ok 15:19:46 I have moving to pymemcache on my backlog 15:19:59 It is pretty far down my list right now (sorry) 15:20:16 I hope I can do it in stien, but am not sure. 15:20:24 should the "import memcache" at least by protected in case of ImportError? 15:21:09 because in my case, mistral throws a 500 and no message, no log, nothing 15:21:47 I took me some time to find the root cause with pdb :D 15:27:12 Not sure, can you point me at where this is happening? I want to know the context before answering. 15:28:21 kmalloc: I need to dig back in mistral to find the exact place this is happening 15:28:56 Because it might indicate Mistral needs it independently. 15:29:25 _MemcacheClientPool should only be instantiated if you have memcached_servers set in keystonemiddleware https://github.com/openstack/keystonemiddleware/blob/master/keystonemiddleware/auth_token/_cache.py#L148 so you have to opt into it and you're assumed to have installed the dependencies if you do that 15:30:02 cmurphy: ahh thats it. 15:57:55 kmalloc: ok so I cannot pinpoint the start of the problem in mistral but the middleware is added here: https://github.com/openstack/mistral/blob/master/mistral/api/access_control.py#L39 15:58:03 this might not be helpful :D 15:58:27 the problem occurs in the process_request of AuthProtocol 15:59:01 Yeah, it's a config bit, for now, install python-memcached as part of the architecture 15:59:29 kmalloc: alright, i'll do that 15:59:35 thanks ;) 16:20:40 Merged openstack/keystone-tempest-plugin master: Import another job from project-config https://review.openstack.org/603281 16:28:51 heads up on broken keystone q->r upgrade path 16:28:52 https://bugs.launchpad.net/keystone/+bug/1793347 16:28:52 Launchpad bug 1793347 in OpenStack Identity (keystone) "keystone upgrade fails q->r oslo.log requirement to low" [Undecided,New] 16:44:20 Merged openstack/keystone master: Use templates for cover and lower-constraints https://review.openstack.org/600690 17:04:09 cmurphy: I'm reading your blog at http://www.gazlene.net/demystifying-keystone-federation.html. Looks like queens support both SAML2.0 WEBSSO and ECP. The setup with keystone and horizon is based on WEBSSO, and the k2k setup is based on ECP. Do I understand right? 17:04:29 lbragstad: ping on launchpad link above fyi 17:34:56 tobias-urdin lbragstad likely https://review.openstack.org/#/c/558901/ didn't get added until 3.37.0 17:41:47 tobias-urdin there is a patch up for review https://review.openstack.org/#/c/599447/ 18:16:38 aning: yes, mostly; keystone as a service provider can work with both websso and ecp, and using horizon depends on websso. when keystone is in a k2k setup it doesn't have websso support, so horizon relies on a kind of modified ecp flow on the backend 18:22:05 cmurphy: Can I setup a keystone as SP with an external Idp (instead of keystone) and use ECP for the user agent to authenticate itself? 18:24:56 cmurphy: the reason is that we may have an application (based on keystone client) that will use federated authentication. 18:25:06 lbragstad can we not bump oslo.log in rocky? 18:25:26 gagehugo sounds like it violates stable policy 18:25:32 gagehugo we need to check with tonyb 18:26:09 sounds like he wants to better understand why it failed 18:26:41 aning: yes that is possible, it relies on the IdP supporting ECP (not all of them do) and the apache SP may need to have ECP turned on (shibboleth has it turned off by default) 18:27:16 lbragstad: i think tonyb is on vacation for a couple of weeks 18:27:54 ack 18:27:57 thanks cmurphy 18:28:14 ah 18:29:27 cmurphy: so still relay on the apache SP mod (such as shiboketh) 18:31:36 aning: yes 18:32:11 cmurphy: the user agent application, does keystoneclient support ECP already? 18:33:08 cmurphy: since the user agent will talk to shiboleth mod in SAML ECP ... 18:34:02 aning: keystoneauth supports it which means keystoneclient and openstackclient also support it 18:34:54 cmurphy: cool. I'll check if there are APIs for ECP Response ... 18:35:41 cmurphy: the support should be in form of APIs right? 18:38:55 cmurphy: or maybe not 18:40:39 aning: hmm not sure what you mean by in the form of APIs, if you pass it the right credentials and parameters it will get you a token which gives you normal access to all the openstack apis 18:41:43 cmurphy: ok IC. same idea, it;s all handled in the background by keystoneauth 18:42:40 right 18:42:50 Thx 18:56:59 hrybacki, kmalloc in a customer presentation at the moment. Not sure if it will go long. Have to wait til its over to switch to talking with you 18:57:24 All good, just ping when done. 18:57:26 ayoung: ack 18:57:31 I am not on the thing yet 18:59:31 ayoung: kmalloc I've got a few hurricane victims that are coming over to get settled -- might not be until after the mtg -- but will have to drop to help them for a bit 18:59:44 kmalloc: wanna join now? I have some questions I can field off of you in the meantime 19:01:55 hrybacki: yeah joining now 19:02:09 btw, no mic/headset so dealing with laptop mic (gross) 19:32:30 cmurphy i ended up just putting most of the feedback for documentation improvements into a single report - https://bugs.launchpad.net/keystone/+bug/1793374 19:32:30 Launchpad bug 1793374 in OpenStack Identity (keystone) "Federated documentations lacks a concise introduction" [Low,Triaged] 19:33:41 lbragstad: okay 19:33:57 also - i was wrong about the document that jdennis authored 19:34:01 lbragstad: i'm just gonna assign it to myself 19:34:24 i thought he wrote something for federation, but it was actually for PCI-DSS 19:34:45 https://github.com/jdennis/documentation/blob/master/openstack/keystone/pci-dss-notes.rst 19:35:05 lbragstad: i thought he wrote all the deep-dive mod_auth_mellon docs 19:35:22 are those already in our docs? 19:36:15 lbragstad: no, they're in mod_auth_mellon's docs https://github.com/Uninett/mod_auth_mellon/commit/ee97812b978cf1cf99bd787d90cf2ace457f475c#diff-670e4c737aee3f2353337e380af89f82 19:36:54 oh - interesting 19:37:09 ... 19:37:10 damn 19:39:42 i wonder if we should just point to that if we don't already 19:40:00 probably 19:40:19 i think the last time i touched our docs all that mellon had was the github readme 19:40:36 =/ 19:41:33 i remember sitting down with asettle, dolphm, and dstanek about 3 years ago, and we were talking pretty much the same thing as whats in https://bugs.launchpad.net/keystone/+bug/1793374 19:41:33 Launchpad bug 1793374 in OpenStack Identity (keystone) "Federated documentations lacks a concise introduction" [Low,Triaged] - Assigned to Colleen Murphy (krinkle) 19:42:46 cmurphy: lbragstad: a deep-dive document would be great! 19:45:54 aning: check out the mod_auth_mellon user guide https://github.com/Uninett/mod_auth_mellon/blob/master/doc/user_guide/mellon_user_guide.adoc it gives a ton of useful background on SAML even if you're not using the mellon SP 19:46:23 cmurphy: thx 20:33:23 Gage Hugo proposed openstack/keystone master: [WIP] Add functional testing gate https://review.openstack.org/531014 20:36:03 Gage Hugo proposed openstack/keystone master: [WIP] Add functional testing gate https://review.openstack.org/531014 20:39:51 Merged openstack/keystone master: Remove unused revoke_by_user_and_project https://review.openstack.org/602216 21:45:30 lbragstad: regarding all the policy and role work, is there an option to sneak an 'auth-only' role into the mix? A role that lets you do pretty much nothing other than maybe change your own password, and exist within a project scope (but with no permissions to any resources in it). 21:46:11 we actually don't protect authentication or password changes with policies 21:46:31 (how do you verify the roles a user has at authentication time) 21:57:42 lbragstad: sorry, just had to run to morning standup 21:57:45 my context is... 21:58:07 right now we have a lot of default policies in openstack. They amount to "user is authed" 21:58:28 which makes it really hard to go through and actually make roles that aren't affected by these default policies 21:59:04 if ALL the policy rules required a role, unless explicitly meant not to, then we can make roles that by default are no-ops 21:59:39 this is useful in a few ways for me, but one... is Swift 22:00:50 Swift ACLs kind of conflict with Keystone roles, and to really use swift ACLs properly with users, or auth creds, you need a user that is auth'd but can't do anything so the ACLs can be assigned to them with the knowledge they can only do Swift things 22:02:02 So a role that explicitly lets you do almost nothing, but can be used to give a user scope within a project, suddenly solves a lot of the Swift ACLs vs Keystone roles issues 22:03:25 does swift fall through if there are roles in the token used to access it's API? 22:03:35 yes 22:03:49 but it needs a project... 22:04:02 well no, it needs an auth'd user 22:04:24 so - this might be a naive question, but can't you use an unscoped token? 22:04:26 but I don't know if an unscoped token is quite enough, but you can do ACLs per project so a no-op role would be useful for limiting that 22:04:48 https://docs.openstack.org/swift/latest/overview_acl.html#keystone-auth-acl-elements 22:05:03 "A token for the user (scoped to any project)" 22:05:14 so looks like not, but I've never actually tested it 22:05:37 so it does require a project... 22:05:56 basically, if "member" lets you access all the containers in a project in Swift, then the ACLs are bypasses 22:06:22 and that's not what you want? 22:06:31 because you want to do more granular things with the ACLs? 22:06:34 yes 22:06:53 I could assign member in a different project... 22:06:54 ok 22:06:57 but that's awful 22:07:29 so - we can't actually use policies for protecting authenticate 22:07:37 authenticate doesn't actually have a policy 22:07:50 that's not what I need though 22:08:02 so... what you could do 22:08:40 we want a user to be auth'd, just don't do the default policy stuff which is 'an authed user' and instead make all the policies actually require explicit roles 22:09:18 we've kind of done that internally, but finding and actually cleaning up all those empty default policies is annoying with some of the older policy files :( 22:09:39 like, we use the member role as an explicit requirement for most API calls. 22:09:50 so - if you have a role called 'noop' and you give it to me on project foo, that doesn't work? 22:10:00 it does in our case 22:10:48 and with the upstream policy work you're doing with the new default roles, (read/write?) you are making most API calls explicit? 22:10:53 the policy I mean 22:11:04 we're going to provide new defaults, yes 22:11:14 and fit each API into one of three camps 22:11:40 so - read operations will need the 'reader' role at the very least 22:11:54 (e.g. getting or listing resources) 22:11:56 cool, I just wanted you to be aware of the usefulness as part of that work of an noop type role 22:12:05 or well, any new role by default would be a noop role 22:12:14 unless they are implied 22:12:22 which they are in our case 22:12:36 admin implies member which implies reader 22:13:21 so if you have the member role on a project, you get the ability to perform list operations 22:14:22 and are you making 'reader' explicitly defined in the new default policy as a requirement? 22:15:12 e.g. for a list "role:read" 22:15:23 which then the implied stuff handles? 22:15:34 correct 22:15:42 it seemed like an appropriate compromise 22:16:08 instead of defining something like "identity:list_users": "role:admin OR role:member OR role:reader" 22:16:24 instead we just do "identity:list_users": "role:reader" 22:16:28 fantastic, then yes, hopefully by the end of that work, the default policies will allow an easy noop still role since there should be far far less empty policies 22:16:50 i guess i'm still a little lost on how the empty policies break that 22:17:15 "identity:list_users": "" < any auth'd user can call this API 22:18:10 vs "identity:list_users": "role:reader" where it must have a specific role 22:18:54 oh! 22:18:55 I know a lot of the various API just assume if you exist as a user in project scope, you can call APIs for resources on that project 22:18:55 ok.. 22:19:28 so you've gone through and actually made those changes by overriding the policies so that I can't access something i'm not supposed to because I have a role called "swift-noop" 22:19:36 yes 22:19:40 damn... 22:19:42 ok 22:20:03 from a keystone perspective, there shouldn't be many empty policies i don't think? 22:20:05 we're having to do that because all the empty rules are kind of too powerful still 22:20:15 nah, Keystone is pretty good 22:20:22 it's all the other services 22:20:25 nova was pretty bad 22:20:31 so many empty rules 22:20:45 But it sounded like you were driving part of the cross project effort 22:20:50 yeah... 22:20:56 it's a long ways away 22:21:15 we need to build out the system-scope and new default roles in keystone so that other services at least have a reference to follow 22:21:19 so I thought you should be aware of the pain we're going through internally trying to fill all the blank rules. 22:21:23 after that we might try and make it a community goal 22:21:52 ok cool 22:23:22 if anyone from catalyst is interested in working on that upstream or pushing it forward, just ask... 22:23:38 we should be able to break thing apart into bite-sized pieces 22:23:54 I'd gladly raise my hand, but I have so many things on my plate right now 22:24:03 But I'll probably raise my hand anyway closer to the time 22:24:05 i hear ya 22:25:11 I've had a post-it on my wall for ages now to try and get this policy work done on our cloud so we can safely do an 'auth-only' role, and part of that was going to be review all our policies and play with patrole 22:26:06 all I want the role to do is be able to auth, update own password, and setup MFA for self, but first I need to make sure we've not left any other policies open :( 22:26:08 it's a pain 22:28:12 lbragstad, kmalloc: I'll try and have a new auth-receipts patch up soonish. 22:28:45 I want to try and get a large part of the MFA work done before the summit 22:29:56 ok 22:30:19 yeah - i guess if you were to do a really limited role you could create something like 'auth-only' 22:30:26 and then have reader imply it 22:31:05 then if there are any APIs that you want them to have access to (which I'm assuming wouldn't be many) you'd just override those to be role:auth-only instead or role:reader 22:32:40 pretty much. 22:33:28 I've yet to play with implied roles much on our cloud, but will be doing something soon for resellers 22:33:45 it's pretty straight forward... 22:34:05 if you have a role assignment that implies another role, we just expand the token reference to include both during token validaiton 22:34:07 validation* 22:34:19 we have a bunch of APIs we can't have resellers seeing so I'll have reseller_member imply member, and then the policy rule will read "role:member AND NOT role:reseller_member" 22:34:30 mmmm 22:34:37 what kind of APIs can't resellers see? 22:34:44 like deployment-specific stuff? 22:34:49 account admin and billing ones 22:34:53 ah 22:35:04 did you happen to see my patch for the credentials work? 22:35:32 no? 22:35:57 we talking app creds or 'credentials' ? 22:36:04 "credentials" 22:36:14 https://review.openstack.org/#/c/594547/ 22:36:24 it is failing two tests... 22:36:25 we are the best at naming 22:36:32 ^ fact 22:36:37 ... 22:36:39 i... 22:36:40 sigh 22:36:52 :P 22:36:57 naming things is hard :P 22:36:58 * kmalloc kills "credentials" with something sharp 22:37:01 there. 22:37:02 fixed. 22:37:05 >.> 22:37:06 <.< 22:37:09 "secrets" ? 22:37:17 i mean... we *did* renaming tenants to projects.... 22:37:29 just rename them to secrets, because that is closer to what they are 22:37:30 "no really, this stuff should have been in barbican.. but we made some bad design decisions" 22:37:35 although barbican might complain 22:37:36 it's like we get bonus points if we overload the term, too 22:37:48 i think that is the official name 22:39:06 self service elements to the credentials APIs would be good, but I'm willing to bet you people will end up using it, and the MFA rules directly and locking themselves out of their own users ;) 22:39:08 but yeah - that patch is a WIP for trying in make the credentials API more self-serviceable 22:39:34 we want to try and that applied consistently across stein 22:39:39 which will mean two things 22:39:48 1.) scope will be honored better than it is today 22:39:51 I say do it, but I'll write some sanity checking APIs in Adjutant for MFA management most likely 22:40:00 2.) policies will incorporate the new default roles 22:40:18 that would make things easier for adjutant, right? 22:40:44 because you can just consume what we've done instead of having to write a layer on top that handles that? 22:40:46 I'd still use the admin level powers really unless I reuse the user token 22:41:02 potentially yeah 22:41:12 I need to read through the patch! 22:41:26 yeah - i'd be curious to know if you have feedback 22:41:39 since it sounds like adjutant has worked around a lot of this stuff 22:42:24 right now we do MFA a little differently since we aren't yet touching auth rules, hence why the MFA code isn't in core Adjutant 22:43:25 but the eventual logic will be: "ask for a totp secret (which adjutant creates as a credential in keystone)" > "pass back a passcode" > "Adjutant validates passcode and adds auth rules for totp to user" 22:43:53 which doesn't exactly need a self service API since the user never touches credentials directly 22:43:57 or auth rules 22:44:47 and turning off totp is pretty much: "pass back a passcode" > "Adjutant validates passcode and removes auth rules for totp from user" 22:44:56 so again, user can't touch credentials directly 22:45:29 gotcha 22:45:40 since that bypasses the need for them to prove they have the totp secret twice (once for auth, again for removal) 22:46:17 but... without Adjutant around, self service is useful, but prone to accidentally doing things a little wrong without hand holding 22:46:33 sure 22:46:47 I can always as part of the Adjutant setup docs include: "update these keystone policies to turn off self service" 22:47:37 we do some interesting stuff in the policies to make sure the user can do that stuff, which might be interesting if you try to shut it off 22:48:04 for example - how these are changing https://review.openstack.org/#/c/594547/10/keystone/common/policies/credential.py 22:48:32 and how we are modifying the business logic of the API to account for it https://review.openstack.org/#/c/594547/10/keystone/api/credentials.py 22:49:34 notice the other half of the policy check strings 22:50:15 we do "user_id:%(target.credential.user_id)s" to make sure the credential can be self-serviceable by its owner 22:51:21 I'd probably turn those off with: "role:admin and system_scope:all" and no 'OR' option 22:51:26 which should work... i think 22:51:58 yeah - that would mean you'd need to be a system admin to do anything with that 22:52:21 and Adjutant acts as one with it's user, so that's ok 22:52:30 although I know I'll need to change that a bit in Adjutant itself 22:52:40 because is still assuming project scope and "admin" 22:55:04 I like the changes. Will mean by default 'owner' can see their own creds, and if not system scoped the API is explicitly written to return only your user creds 22:56:39 not that I have any use for it right now, but we're not doing anything with https://github.com/openstack/keystone/blob/master/keystone/credential/backends/sql.py#L29 ? 02:23:33 fupingxie proposed openstack/keystone master: Do not translate log messages. https://review.openstack.org/603953 03:28:25 wangxiyuan proposed openstack/keystone master: Add hint back https://review.openstack.org/603964 04:05:40 wxy-xiyuan: Hello. Can you pl review this test caes https://review.openstack.org/#/c/603539/ 04:06:40 vishakha, you know how to run just the pep8 tests? 04:08:05 ayoung: use command tox -epep8 04:09:18 vishakha, yeah, get that test case to run clean pep, so it passes check 04:09:54 vishakha, also, your new test fails 04:10:00 SystemExit: Error while parsing rules /tmp/tmpzYxfG1/tmpJ35T2c: No JSON object could be decoded 04:10:39 vishakha, http://logs.openstack.org/39/603539/1/check/openstack-tox-py27/ed2cc94/testr_results.html.gz you can see the test results there. Get the tests to pass and pep8 clean. Most people won 04:10:48 't bother reviewing a broken patch 04:10:49 OK? 04:11:27 ayoung: ok thanks for the response 04:12:17 adriant, I'm sorry I missed that meeting. I'm nort sure I understood correctly, but if you are doing rules like "don't allow if user has role noop" you are making a security hole. With a trust (which all users can create) they can drop any role they have. 04:12:28 and with that. I'm out. 04:13:43 bah, and he's gone before I can respond :P 04:15:05 that isn't the point of the noop role, doing rules with "NOT role:noop" is a silly idea. Instead, just make all your policies require a role. No empty "auth'd only" policies. 04:15:52 so a noop role is really just a role that fulfils only empty policies (which you'd ensure there aren't many or any of them). 04:15:54 but... 04:16:08 lbragstad, cmurphy: on the note of trusts and implied roles... 04:16:26 I assume a trust can't allow you to set a role you don't actually have, but are implied to have? 04:16:45 e.g. I have reseller_member which implies member, can I make a trust for my user with just member? 04:16:49 if so, that's broken 04:20:33 but I'd assume that doesn't work 06:42:05 vishakha: lol, like ayoung said, please let the CI pass first. If you have problem about that, please let me know again. :) 06:59:31 wangxiyuan proposed openstack/keystone master: Add hint back https://review.openstack.org/603964 07:01:50 Vishakha Agarwal proposed openstack/keystone master: Adding test case for MappingEngineTester https://review.openstack.org/603539 07:03:16 Tao Li proposed openstack/keystone master: Use uuidutils instead of uuid.uuid4() https://review.openstack.org/603542 07:03:34 wxy-xiyuan: yes it is not cleared yet. Uploaded a new patch for it. 11:06:59 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: POC: Add Open Policy Agent driver https://review.openstack.org/604038 11:16:57 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: POC: Add Open Policy Agent driver https://review.openstack.org/604038 11:23:47 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: POC: Add Open Policy Agent driver https://review.openstack.org/604038 11:40:30 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: POC: Add Open Policy Agent driver https://review.openstack.org/604038 11:48:06 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: POC: Add Open Policy Agent driver https://review.openstack.org/604038 12:52:05 o/ 14:27:16 o/ 14:27:25 cmurphy: hi, sorry for late reply 14:28:01 cmurphy: applications are now open (since yesterday) and new projects can be submitted any time until October 9th according to the last agenda I have 14:28:21 cmurphy: I will update the info in that link, thanks! 15:02:57 samueldmq o/ 15:45:10 adriant: sounds like something keystone specific. What is the point of noop in Nova? 15:46:02 adriant: if it is just to setup things like mfa, we can do it as a system-scope, and I can modify our enforcer for some calls. 15:46:53 adriant: and a system scope without role, should be sufficient for that. (aka non escalated permissions, but not project scoped) 15:47:16 Generally I don't like an explicit or implicit noop role. 15:48:46 I generally want to down play unscoped tokens anyway with the advent of system scope. 17:20:16 o/ 17:20:39 today is one of those meeting after meeting days :( 17:47:44 knikolla: yup 17:49:35 gagehugo: did you get my nintendo friend request? 17:49:54 yup! 18:50:02 gagehugo: o/ 19:08:26 Could anyone recommend me a good book about RESTfull api? 19:09:54 Lance Bragstad proposed openstack/oslo.policy master: Add docs for developers testing APIs https://review.openstack.org/604192 19:36:28 * raha 19:49:10 Kristi Nikolla proposed openstack/keystone-specs master: [DRAFT] Refreshable Application Credentials https://review.openstack.org/604201 19:52:48 * knikolla considers renewable vs refreshable 21:30:03 Lance Bragstad proposed openstack/keystone master: Implement scope_type checking for credentials https://review.openstack.org/594547 21:30:03 Lance Bragstad proposed openstack/keystone master: Remove obsolete credential policies https://review.openstack.org/597187 21:44:10 zuul's gettin' a workout today 21:44:32 * lbragstad hands zuul a bottle of water and a towel 21:52:26 that credentials patch should pass 100% now 21:52:36 needed to redo a couple tests 21:52:43 but it should be good to review 22:14:25 kmalloc: did you by chance read the email I sent to follow up for Adam? 22:14:46 thanks samueldmq 22:18:31 The use case we have right now that customers are asking for: "I have a backup project, and I want to create a container per person, and give them access only to that container. They need to be able to auth, and scope to that project, but not do anything else in it other than see their own container." To achieve this is part Keystone's auth with ro 22:18:31 les, and part Swift ACLs. But with nova and other services having rules that amount to: "any role on a project lets you access all project resources" that makes it hard. 22:20:04 https://github.com/openstack/nova/blob/master/nova/policies/base.py#L31< is the default style role for Nova, and from memory, most of the other projects do it much the same 22:22:24 and glance: https://github.com/openstack/glance/blob/master/etc/policy.json which I assume then does per project filtering in code (and likely has hardcoded checks for is_admin?). 22:22:58 The issue is that any new roles created, by default when assigned to a project, give a user full access to all project resources. 22:23:12 so what the role is doesn't matter beyond is the role admin 22:25:53 yeah, admin_or_owner is pretty much the norm: https://github.com/openstack/cinder/blob/master/cinder/policies/base.py#L27 01:25:39 Tao Li proposed openstack/keystone master: Use uuidutils instead of uuid.uuid4() https://review.openstack.org/603542 01:47:07 wangxiyuan proposed openstack/keystone master: Update log translation hacking check https://review.openstack.org/604245 02:28:50 Vishakha Agarwal proposed openstack/keystone master: Mapped Groups don't exist breaks WebSSO https://review.openstack.org/597992 02:52:24 Vishakha Agarwal proposed openstack/keystone master: Mapped Groups don't exist breaks WebSSO https://review.openstack.org/597992 02:54:19 Vishakha Agarwal proposed openstack/keystone master: Mapped Groups don't exist breaks WebSSO https://review.openstack.org/597992 02:57:42 Vishakha Agarwal proposed openstack/keystone master: Mapped Groups don't exist breaks WebSSO https://review.openstack.org/597992 03:03:04 wangxiyuan proposed openstack/keystone master: Update log translation hacking check https://review.openstack.org/604245 05:08:16 adriant: part of the default roles and documented rules in code, we can make changes so admin_or_owner can be changed. 05:08:57 kmalloc: oh yeah, it's just going to take time, and require a shift in mindset 05:10:11 yep. 05:11:54 because until we do shift away from the idea of "any role on a project" being the standard, RBAC is pretty limited. Although with the way the policy in code is now will make it somewhat easier for operators to do such changes themselves 05:13:35 but yeah, it will take time. And in the meantime, we'll play with our own deployment's policy and see what works :) 05:33:30 Vishakha Agarwal proposed openstack/python-keystoneclient master: create() call in v3.regions.py is wrong https://review.openstack.org/594921 05:54:52 Juan Antonio Osorio Robles proposed openstack/oslo.policy master: POC: Add Open Policy Agent driver https://review.openstack.org/604038 05:58:43 Tao Li proposed openstack/keystone master: Use uuidutils instead of uuid.uuid4() https://review.openstack.org/603542 08:45:42 Vishakha Agarwal proposed openstack/keystone master: Adding test case for MappingEngineTester https://review.openstack.org/603539 08:55:24 Vishakha Agarwal proposed openstack/keystone master: Adding test case for MappingEngineTester https://review.openstack.org/603539 09:55:55 Vishakha Agarwal proposed openstack/keystone master: Adresses LDAP case-sensitive issue https://review.openstack.org/603345 10:32:57 Vishakha Agarwal proposed openstack/keystone master: Adresses LDAP case-sensitive issue https://review.openstack.org/603345 12:39:06 any ptg recaps I should include in the update email? kmalloc knikolla gagehugo hrybacki 12:53:45 cmurphy: nothing from me. You and Lance do an amazing job every time :) 13:02:07 o/ 13:20:14 awesome summary cmurphy 13:20:44 i just noticed you published it 13:35:25 ayoung proposed openstack/keystone master: Comment out un-runnable tests https://review.openstack.org/603459 13:44:14 lbragstad: this is the unified limits spec in Nova I promised: https://review.openstack.org/#/c/602201 Not finished, but has a lot of detail in there now (probably too much) 13:49:05 johnthetubaguy awesome - thanks! 14:33:30 hrybacki: o/ 14:35:10 kmalloc, Harry is ofline-ish today, helping a friend to get in their house after the Hurricane Florence in the NC coast 14:35:30 but he is responding emails :) 14:35:34 raildo: yeah, not surprised 14:37:01 knikolla, gagehugo hey guys, do you have some time to review this patch? https://review.openstack.org/#/c/597992/ John and me already take a look on it, and we think that it's in a good shape right now 14:41:28 kmalloc: do you think you could write up a proposal for outreachy on the flask test_client and subsystem reorg ideas? i can volunteer to mentor if you don't want to 14:41:47 s/a proposal/proposals 14:42:07 Sure. But it won't be until 1st week of October probably. 14:42:43 okay, the deadline is oct 9 14:50:03 Yeah I can have it by then 14:50:13 o7 14:50:21 It prob will be oct 3 or 4 14:50:50 But def. doable by the 9th 14:51:03 Since I want to take my birthday off work :P 14:51:14 :D 14:53:57 o/ 15:09:45 raildo: o/, reviewed and left a question 15:21:32 knikolla, thank you sir! 15:52:40 Lance Bragstad proposed openstack/keystone-specs master: Repropose JWT specification for Stein https://review.openstack.org/541903 18:37:20 hi all, I have a policy question :) I'm trying to hit the TARGET_DOMAIN portion of the get_domain policy so that I can move this test in tempest https://review.openstack.org/#/c/525244/, but it appears it is not possible http://paste.openstack.org/show/730556/ can someone describe what kind of setup I need to show the default domain with a non-admin user? 18:38:18 here is the policy for reference https://github.com/openstack/keystone/blob/582cab391a10714a4ec8bab1a6cce9b49867f8d4/keystone/common/policies/domain.py#L20 18:42:02 this one specifically? https://github.com/openstack/keystone/blob/582cab391a10714a4ec8bab1a6cce9b49867f8d4/keystone/common/policies/base.py#L21-L23 18:42:45 Yes! 18:45:59 it doesn't look like the target attr information is getting passed into policy check function 18:47:17 sorry i got disconnected :( 18:47:47 actually - gagehugo just modified a bunch of code in the area 18:47:48 https://github.com/openstack/keystone/commit/296f20f0a7e26784b6414ddbe12e0218087a9f51 18:47:51 are you using master? 18:48:42 yes i have the changes as of Monday this week. 18:48:51 commit: c96c7fd03b7afab033bcb31465390f46e56089c5 18:49:25 cool - https://git.openstack.org/cgit/openstack/keystone/commit/keystone?id=c96c7fd03b7afab033bcb31465390f46e56089c5 18:50:44 so you're actually hitting this https://github.com/openstack/keystone/blob/ba459352d8d2b54807a591312bc0b65aa1498b86/keystone/api/domains.py#L78 18:51:24 that call is what's protecting the get_domain API 18:51:29 with policy 18:51:40 and it doesn't look like it's passing in any sort of target informatino 18:51:42 information* 18:52:21 which is an option argument to that method - https://github.com/openstack/keystone/blob/master/keystone/common/rbac_enforcer/enforcer.py#L246-L248 18:52:30 https://github.com/openstack/keystone/blob/master/keystone/common/rbac_enforcer/enforcer.py#L261-L267 18:53:06 so.. that method will actually attempt to populate it - https://github.com/openstack/keystone/blob/master/keystone/common/rbac_enforcer/enforcer.py#L340-L341 18:54:25 which means target_attr will be a dictionary or reference of the domain... 18:54:25 ok so it appears we need to pass the target information so we can check the TARGET_DOMAIN of the token, right? 18:54:39 either that or we need to rewrite the check here 18:54:39 https://github.com/openstack/keystone/blob/ba459352d8d2b54807a591312bc0b65aa1498b86/keystone/common/policies/base.py#L23 18:55:04 because the check string is written like it's assuming target data to be coming from the token and not the domains API 18:55:07 if that makes sense? 18:55:32 but yeah... that policy seems to be inconsistent with the logic in that API... 18:55:36 cc kmalloc gagehugo ^ 18:56:39 i guess you could test that by rewriting the policy 18:58:25 ok I can do that, i'm not following entirely but I think I have enough to get started. I can follow up with gagehugo if I have questions, we're in the same office :) 18:58:37 nice :) 18:58:59 flick a pencil at him for me 18:59:24 lol ok. :P 19:03:22 * gagehugo ducks 19:06:59 ayoung proposed openstack/keystone master: Comment out un-runnable tests https://review.openstack.org/603459 19:18:20 jdennis: does mod_auth_mellon require both the response and the assertion to be signed? 19:22:12 knikolla: could you define response? 19:23:11 jdennis: 19:24:06 which is separate from the signature. 19:24:33 knikolla: response is relative to the party, please be more specific 19:25:36 jdennis: i'm trying to set up keystone as an IdP using ECP. When mellon receives the ECP message it complains that it doesn't have a signature 19:25:42 http://paste.openstack.org/show/jvbKqiFEjdIWY4YstemW/ 19:27:00 knikolla: thanks, that's better, let me check the code ... 19:32:48 knikolla: a quick check of the code implies it does require the assertion to be signed, it would be a huge security risk if assertions were not signed btw. 19:34:45 jdennis: yes, i understand that. the keystone generated assertion does have a signature, i am trying to find out why mellon says it can't find the signature. 19:34:53 assertion for reference http://paste.openstack.org/show/730566/ 19:38:39 knikolla: I'm reviewing the paste ... 19:38:53 thank you 19:45:14 knikolla: I'd have to page in some of my xml signature knowledge but if I'm not mistaken your signature does not point to anything in the message 19:48:22 knikolla: sorry, I misread the xml, forget that :-) 19:53:43 elbragstad: should get_domain not be checking from the token? 19:53:54 sorry been pulled in many directions today 19:54:08 knikolla: hmm... I don't see an obvious problem. I assume you have the IdP's metadata loaded into mellon, yes/no? 19:55:52 jdennis: yes, the first line here says so http://paste.openstack.org/show/jvbKqiFEjdIWY4YstemW/ 19:55:56 metadata here http://paste.openstack.org/show/730567/ 19:56:47 apache config here http://paste.openstack.org/show/730568/ 19:57:19 was playing around with a few settings (including MellonSignatureMethod) 19:57:28 gagehugo maybe it should be? 19:57:33 removing or adding it doesn't change the log output 19:58:29 knikolla: if you send me the soap messge (yes I know it's in the paste), and the IdP metadata as attachments in an email to jdennis@redhat.com I'll try to debug it, but probably not until Monday or Tuesday 19:58:49 knikolla: what version of mellon and lasso are you running? 20:00:59 knikolla: MellonSignatureMethod only changes how mellon signs messages, it's independent of validating signatures 20:01:15 mod_auth_mellon-0.13.1-3.el7_5.x86_64, lasso-2.5.1-2.el7.x86_64 20:01:59 i see 20:02:12 i'll send you an email, thanks a lot. 20:02:38 knikolla: ok great, those are current enough and I'm not aware of any signature bugs in them 20:03:46 knikolla: sometimes lasso reports one error when actually it was a different error that triggered the failure 20:04:24 that doesn't sound fun 20:05:09 kmalloc you use ipdb, right? do you know if there is a way to make it work with stestr? 20:32:02 Merged openstack/keystone master: Mapped Groups don't exist breaks WebSSO https://review.openstack.org/597992 20:37:26 knikolla gagehugo vishakha ^ that should probably have a release note 20:38:04 elbragstad: good point 20:44:44 knikolla: do you know if cmurphy's blog (http://www.gazlene.net/demystifying-keystone-federation.html) will work using keystone as the IdP instead of "http://idp.saml.demo"? 20:46:46 mbeierl: the last section talks about that 20:48:30 for some reason, I cannot get the metadata from the Keystone idP 20:49:42 knikolla: I'll keep plugging at it, thanks 20:52:29 knikolla I can write one real quick unless you are wanting to 20:52:47 gagehugo: go for it 21:00:49 Gage Hugo proposed openstack/keystone master: WIP/DNM Add domain target https://review.openstack.org/604471 21:01:29 trevormc_ ^ 21:09:20 Gage Hugo proposed openstack/keystone master: Add releasenote for bug fix 1789450 https://review.openstack.org/604475 21:09:21 knikolla ^ 21:09:27 elbragstad ^ 21:09:38 nice - thanks 21:34:40 Gage Hugo proposed openstack/keystone master: Add releasenote for bug fix 1789450 https://review.openstack.org/604475 21:35:00 Gage Hugo proposed openstack/keystone master: Add releasenote for bug fix 1789450 https://review.openstack.org/604475 21:35:31 spelling is hard 21:45:34 thanks gagehugo 22:12:17 elbragstad: I use pycharm 22:12:35 elbragstad: and I don't use stestr, I run tests individually if I need to debug 22:12:56 ack 22:12:57 i figured out a workaround ;) 22:13:24 Give me a sec on the other thing (on a phone) 22:13:28 The policy string bit 22:14:35 no worries - i'm about burnt out for the day ( i was trying to figure out tempest testing client stuff so that i can implement system scope) 23:09:44 Ouch 23:09:50 That is some brain fry. For sure. 00:51:18 Im setting up federation on rocky. when my user logs in Im getting NoMatches: No 'keystone.auth.saml2' driver found, looking for 'keystone.auth.plugins.mapped.Mapped' 00:52:22 in my keystone.conf I have a line that says: [auth]\n methods=password,token,saml2\n saml2=keystone.auth.plugins.mapped.Mapped 00:52:58 Im not sure what I have wrong here.. 01:30:17 Ive also noticed in the keystone.log file Im getting Could not load keystone.auth.plugins.mapped.Mapped 01:30:59 but from a cli I can open python and get it imported that way.. 03:11:10 so this has to be a bug. I renamed the auth protocol from saml2 to "mapped" and updated my configs accordingly and now it works 03:12:11 in previous releases you could name the protocol what ever and then in keystone.conf set it to whatever = keystone.auth.plugins.mapped.Mapped and it would work, but now that does not work. 20:13:50 errr: that may have been unintentional behavior (previous release behavior). I'd have to dig through how we changed the loaders, but that sounds familiar (something we fixed) 22:09:47 Merged openstack/keystone master: Add releasenote for bug fix 1789450 https://review.openstack.org/604475 15:23:07 Monty Taylor proposed openstack/keystoneauth master: Cache root urls with and without trailing slashes https://review.openstack.org/604635 15:35:33 mordred: ^ sure. 17:19:40 kmalloc: yah - just what we always wanted 11:50:48 Merged openstack/oslo.limit master: Ignore documentation builds https://review.openstack.org/603167 12:41:12 kmalloc, cmurphy: https://review.openstack.org/#/c/604635/ is green with the testing and ready for review 13:15:26 o/ 13:23:18 o/ 13:23:29 Lance Bragstad proposed openstack/oslo.limit master: Render API reference documentation https://review.openstack.org/600264 13:23:49 Lance Bragstad proposed openstack/oslo.limit master: Add a conceptual overview to docs https://review.openstack.org/600265 13:23:53 Lance Bragstad proposed openstack/oslo.limit master: Allow ProjectClaims to support multiple resources https://review.openstack.org/600266 13:24:11 Lance Bragstad proposed openstack/oslo.limit master: Use openstackdocstheme for documentation https://review.openstack.org/600866 14:04:15 Vishakha Agarwal proposed openstack/keystone master: Adresses LDAP case-sensitive issue https://review.openstack.org/603345 14:14:43 Vishakha Agarwal proposed openstack/oslo.limit master: Make callbacks required for enforcement https://review.openstack.org/604795 14:29:38 o/ 14:31:05 hrybacki morning - do we wanna try and go through the stein board sometime this week? 14:31:44 lbragstad: yes -- I can do tomorrow before the weekly meeting if that works for you? 14:31:51 yessir 14:32:00 * hrybacki goes to send off an invite 14:32:08 my schedule is wide open this week so... 14:34:12 kmalloc: knikolla query for you on the ml http://lists.openstack.org/pipermail/openstack-dev/2018-September/135006.html 14:35:56 I saw, was reading the email, pre-coffee so brain is.... OoooooOOOooooOoooOoo 14:36:15 kmalloc: no rush :) 14:50:13 cmurphy: I have a keystone instance setup as SP and testshib.org as Idp. When I login in with the Idp, Horizon gives an error: The current path, auth/login/default/auth/OS-FEDERATION/websso/saml2, didn't match any of these. 14:50:28 o/ 14:51:06 cmurphy: but I tried "identity/v3/auth/OS-FEDERATION/websso/saml2" directly, it works. 14:52:25 cmurphy: looks like some settings are missed in Horizon, that it doesn't redirect to the right URL 14:53:44 aning: are you using master of horizon? i think I saw something similar last week but haven't had time to report the bug yet 14:54:25 cmurphy: I think so ... I'm using devstack master, so I would think it pulled in the master. 14:56:49 cmurphy: things seem to be working from the point where Apache shibboleth is contact (this is the URL:identity/v3/auth/OS-FEDERATION/websso/saml2) 14:58:18 aning: if it's what i was seeing, it seems to be new on master and rocky is not broken. you can try checking out stable/rocky in /opt/stack/horizon, then you'll also have to run some django commands to reinit horizon and then restart apache http://git.openstack.org/cgit/openstack-dev/devstack/tree/lib/horizon#n154 15:05:10 cmurphy: thx, I'm trying this. Will report back. 15:10:54 Gage Hugo proposed openstack/keystone master: [WIP] Add functional testing gate https://review.openstack.org/531014 15:28:20 mordred: so much for green https://review.openstack.org/604635 15:33:06 cmurphy: ugh. that's a timeout issue in the openstacksdk test suite I've been fighting 15:45:08 cmurphy: hopefully https://review.openstack.org/#/c/604628/ will stop the flapping 17:06:33 Merged openstack/keystone master: Implement Trust Flush via keystone-manage. https://review.openstack.org/589378 17:16:31 kmalloc: you around? 17:27:27 kmalloc: ah ok, well I opened a bug report about it https://bugs.launchpad.net/keystone/+bug/1793845 because in openstack-ansible we used the method I describe in our keystone playbook if we setup federation 17:27:27 Launchpad bug 1793845 in OpenStack Identity (keystone) "Federation Protocol saml2 fails on Rocky" [Undecided,New] 17:28:22 and I feel like it worked still in Queens and changed in Rocky. I know it worked like I described in Pike for sure 17:32:47 gagehugo: should https://github.com/openstack/keystone/blob/master/keystone/resource/controllers.py#L34-L40 be removed as well? 17:51:18 is it just me or does the new chrome update on osx seems *way* too much like safari? 17:55:29 it is very safari-ish 17:56:24 hrybacki: iirc another api was calling that for something 17:57:09 https://github.com/openstack/keystone/blob/master/keystone/auth/controllers.py#L362 17:57:54 so I just left the class in 18:20:29 gagehugo: ah I see. I wonder if we can point that at a newer helper function 18:22:34 Andreas Jaeger proposed openstack/python-keystoneclient master: Use templates for cover and lower-constraints https://review.openstack.org/600692 18:22:35 Andreas Jaeger proposed openstack/python-keystoneclient master: Import legacy keystoneclient-dsvm-functional https://review.openstack.org/604868 18:31:25 hrybacki: yeah, I figured it could be done in a separate change 18:31:39 (or wait until auth gets moved over and just avoid it lol) 18:31:46 >.> 18:43:43 cmurphy: andy idea how to turn Apache Shibboleth Mod ECP on? 18:44:28 cmurphy: I remember that you mentioned ECP on SP is off by default. 18:46:38 its off by default? 18:47:17 errr: to me? 18:47:24 aning: in shibboleth2.xml in the SSO tag i think it's just ECP="true" or something like that 18:48:07 cmurphy: k, trying ... 18:51:22 18:51:22 SAML2 SAML1 ECP 18:51:22 18:55:06 Found this: 18:55:09 18:55:09 SAML2 SAML1 18:55:09 19:17:30 cmurphy: update ... minimum config like this works: 19:17:33 19:17:33 SAML2 SAML1 19:17:33 19:23:45 cmurphy: but the WESSO support for Horizon is broken :( 19:23:50 Harry Rybacki proposed openstack/keystone master: WIP: Convert projects API to Flask https://review.openstack.org/603451 19:24:35 aning: in which release? 19:24:58 errr: I'm using Devstack with master. 19:25:02 ah 19:25:16 havented tested that yet 19:25:30 -ed 19:25:56 I think I may to try another installation with Rocky. 19:26:14 it for sure works in rocky. I just did an install Friday 19:26:44 I was using mellon, but that wont matter at the horizon side of things 19:27:44 errr: what's your SSO section like? 19:28:02 in the horizon config? 19:28:03 errr: never mind, you are not using shibboleth ... 19:28:16 well I use both all the time 19:28:39 if Im working on rhel I have to use mellon, when Im on ubuntu our stuff uses shibboleth 19:28:40 errr: you made both WEBSSO and ECP work at the same time? 19:29:42 so I may have some missunderstanding of what ECP is, but I thought that was something on the IDP side of things, not the SP 19:30:16 errr: so far, I kind of made WEBSSO works with Horizon, and ECP works with openstack CLI, but not wit the same config in shibboleth2.xml 19:30:37 ah so to use cli apps I use a work around which kind of sucks 19:30:53 there is a plugin from pf9 that I use 19:31:17 I don't think it's the client 19:31:39 https://github.com/michaelrice/openrc_maker I made this to get cli working with sso 19:32:57 errr: that's nice. 19:33:06 its ugly but it works 19:33:10 errr: but the openstack CLI does work. 19:33:15 yep 19:34:11 I worked with the pf9 fols to get their plugin into pip so I need to update my code to pip install their plugin rather than pull my fork of it from github 19:34:16 errr: In production, we definitely need WEBSSO with Horizon and ECP with openstack CLI work. 19:34:19 folks* 19:35:03 yeah we have to have web sso and cli for those users too 19:35:10 errr: and both need to work at the same time without any change to configuration. 19:35:43 we dont use sso for service accounts, just users 19:36:18 with the solution I came up with its 1 extra step for people to generate their openrc file before they can start using the cli 19:39:50 errr: What does the openrc maker generate, other than these OS environment varibles? 19:41:24 errr: this is the openstack CLI is used (copied from cmurphy's blog) 19:41:27 $ openstack \ 19:41:27 --os-auth-type v3samlpassword \ 19:41:27 --os-identity-provider testidp \ 19:41:27 --os-identity-provider-url https://idp.testshib.org/idp/profile/SAML2/SOAP/ECP \ 19:41:27 --os-protocol saml2 \ 19:41:28 --os-username myself \ 19:41:30 --os-password myself \ 19:41:33 --os-auth-url http://devstack-sp.wrs.com/identity/v3 \ 19:41:35 --os-project-name demo \ 19:41:37 --os-project-domain-name Default \ 19:41:39 --os-identity-api-version 3 \ 19:41:41 token issue 19:41:55 it just makes a valid openrc file. but it uses v3token instead of v3samlpassword 19:42:31 errr: you are using this with k2k? 19:42:58 I have not tested it with that. We normally use okta as an idp and also adfs 19:43:56 for k2k there may be something else better.. I just havent had that come in yet so I have never set it up 19:50:52 errr: It's just the v3token remind me of k2k, since in k2k federated setup, client get a token from the Idp Keystone, and with that token start the SAML procedure. 20:49:27 Ben Nemec proposed openstack/oslo.limit master: Fix doc grammar/spelling nits https://review.openstack.org/604907 20:57:44 Colleen Murphy proposed openstack/keystone master: Convert legacy functional jobs to Zuul-v3-native https://review.openstack.org/602452 21:28:27 is anyone here familiar with tempest auth clients? 21:35:07 lbragstad, i kinda was, but i'm pretty sure my memory will fail me 21:35:46 rodrigods yeah... it's kinda complicated 21:36:07 i spent most of friday and today trying to find a way to add system-scoping to tempest clients 21:36:18 curious if anyone anyone had pointers https://review.openstack.org/#/c/604909/ 21:36:39 s/anyone anyone/anyone/ 21:37:07 let me take a look 21:37:33 https://review.openstack.org/#/c/604909/1/tempest/api/identity/admin/v3/test_credentials.py is ultimately what i want to do 21:37:54 since it's needed for https://review.openstack.org/#/c/594547/11 to pass 21:41:23 i have 0 memories of them :/ 21:43:34 :) 22:32:12 Merged openstack/oslo.limit master: Render API reference documentation https://review.openstack.org/600264 22:32:13 Merged openstack/oslo.limit master: Add a conceptual overview to docs https://review.openstack.org/600265 23:54:40 Merged openstack/oslo.limit master: Fix doc grammar/spelling nits https://review.openstack.org/604907 02:13:37 lbragstad, kmalloc, gagehugo: I'm reading through release notes again and came across the fact that we allow a user to change their password without first getting token now if they know the original password. We haven't I don't think added any MFA support on top of that. 02:14:31 Because right now you can lock someone out of their account if you know their password, but not their totp code. 02:17:30 We always allowed password change without token. 02:17:53 Oh, but with that change we also allowed it for expired ones? 02:18:00 Correct 02:18:32 kmalloc: should we consider adding MFA based protections on top of that? 02:18:43 Unsure. 02:19:06 Maybe. Let's get the auth receipts first. 02:19:14 oh yeah 02:19:25 We can enhance other APIs independently 02:19:29 this would be follow up 02:19:59 we're just finally doing a keystone upgrade and this is giving me a good reason to read through all the release notes as I go 02:20:12 and thought I'd ask about this 02:24:21 kmalloc: https://review.openstack.org/#/c/404022 but this change dropped the @controller.protected() decorator, i thought that (until that patch) meant it needed a token? 02:24:57 so only in Ocata did we allow a no token method to change password? 02:25:38 adriant: v2 vs. v3. 02:26:35 It was always intended for no token. So maybe in ocata. 02:27:04 Oh so v2 always let you do it without token, but only in Ocata did we make v3 let you? Maybe? 02:27:11 Possibly 02:27:27 Related to pci-dss. 02:28:07 yeah pci-dss related 02:28:15 But needing a token breaks the self-service password change functionality 02:30:06 but no sensible system with MFA support would let you change password if you don't meet all the other auth requirements, so we may need to consider a middle ground were you can supply an auth receipt and the old expired password 02:30:21 or something potentially less silly :/ 02:31:18 I assumed this was mostly in the context of expired passwords, because surely if your password isn't expired you can always get a token 02:37:28 yeah, before that patch the API was protected, and then eventually we remembered to remove the policy: https://github.com/openstack/keystone/commit/77bf1ad0b8991abb6c7ebba608fde27a3fd01c09 05:16:25 Vishakha Agarwal proposed openstack/keystone master: Purge soft-deleted trusts https://review.openstack.org/604970 06:34:29 Vishakha Agarwal proposed openstack/keystone master: Purge soft-deleted trusts https://review.openstack.org/604970 06:54:24 Vishakha Agarwal proposed openstack/keystone master: Adding test case for MappingEngineTester https://review.openstack.org/603539 07:03:03 Merged openstack/keystoneauth master: Cache root urls with and without trailing slashes https://review.openstack.org/604635 09:19:22 Colleen Murphy proposed openstack/keystone master: Convert legacy functional jobs to Zuul-v3-native https://review.openstack.org/602452 09:27:32 Vishakha Agarwal proposed openstack/python-keystoneclient master: create() call in v3.regions.py is wrong https://review.openstack.org/594921 11:24:17 Monty Taylor proposed openstack/keystoneauth master: Reformat Adapter docstring https://review.openstack.org/605042 11:24:19 Monty Taylor proposed openstack/keystoneauth master: Add support for client-side rate limiting https://review.openstack.org/605043 11:25:24 cmurphy, kmalloc, adriant: ^^ I **clearly** need to add tests for that, but I believe that will let us remove the TaskManager from openstacksdk and nodepool and expose the fancy extra goodness we've been doing there to all of the consumers 11:29:05 mordred: that sounds exciting 11:30:33 cmurphy: yah - I'm pretty pleased so far - it only required writing one class that should be in the standard library 11:30:55 but otherwise is shockingly straightforward- unless I'm *completely* missing a logic 11:31:35 * cmurphy looks at it 11:32:10 oh, adding thread locking to a requests lib, totally straightforward 11:33:36 yah. now - writing the unit tests for it, on the other hand... 11:33:49 will probably take the rest of the week 11:34:33 mordred: are you sure this belongs in ksa and not back in openstacksdk? 11:35:23 cmurphy: I've got a patch for it for openstacksdk too ... so it can live either place - but I figured I'd float it over here to see what people think 11:36:05 cmurphy: https://review.openstack.org/#/c/604926 fwiw 11:37:08 i have a little bit of hesitation about this feature creep 11:37:16 interested in how kmalloc feels 11:37:35 yah - and totally fair 11:37:50 I'm good either way - kinda whatever folks think 11:38:19 there isn't an explicit need for it to be down in ksa like there was for discovery stuff 11:41:09 cmurphy: I am perfectly fine with this in KSA if it can be encapsulated nicely 11:41:55 Ineed to look at the code and see how it fits, but it isn't outside of something I see as valuable. 11:42:46 I also might look and say "yeah, this is sdk clearly" 11:43:33 So, ... Let me look at the code today :) 11:43:53 (will be not pre-5am insomnia code review though) 11:44:49 kmalloc: i was wondering why you and mordred appeared at the same time 11:45:48 Have been awake for 4 hours 11:46:00 :O 11:46:08 Lot on my mind and busy couple days this week 11:57:39 kmalloc: I haven't been up for 4 hours - but I have been up for 3 - because NO REASON 11:58:23 Stupid no reason no sleep ;) 11:58:36 * kmalloc shakes fist at no reason. 11:59:43 kmalloc: no reason is the worst isn't it? 12:00:09 Totally. 12:00:26 Makes for less than fun following days 12:00:40 Often... Sleepy, moderately unproductive 14:49:48 hrybacki: mind sending me an invite re the stein board? i would be a bit late but i'd like to join if possible 14:50:58 cmurphy: https://trello.com/invite/b/rj0ECz2c/59eee4dde6cda539a345e91554a92fdc/keystone-stein-roadmap <3 14:53:35 hrybacki: sorry i meant the bluejeans meeting 14:54:03 cmurphy: oh, we couldn't get it to work -- lbragstad is running me through the last few days of PTG on a hangout 14:54:17 https://hangouts.google.com/call/lE9L6UGJeNeqO3WqT4nLAAEE cmurphy 14:56:20 ah i probably don't need to be there then 14:57:10 you're welcome all the same cmurphy 15:00:03 cmurphy: we're about to go over the stein board now if you wanna join 15:00:38 hrybacki: okay i will in about 15 minutes, starting another meeting 15:08:43 o/ 15:24:16 cmurphy: we dropped btw 15:24:58 hrybacki: ya i noticed :) 15:31:22 lbragstad: prob going to miss the meeting today, Dr appointment 15:31:48 A lot fewer appointments starting next week. 15:32:07 Either Thursday or Friday I'll be offline for a large chunk of the day. 15:39:13 kmalloc, sounds good - thanks for the heads up 15:39:44 hrybacki: as of right now can't make that meeting tomorrow. 15:40:24 Might change today, but fwiw, Dr appointment is on the books until 10:30 Pacific (1.5 hours later than the meeting you set) 15:46:45 kmalloc: ohhh, I misread that time 15:47:39 kmalloc: hmm, so you and jaosorior are at odds when it comes to timezones 15:47:49 hrybacki: I will know in about 40 min if the appointment is cancelled for tomorrow. 15:48:06 kmalloc: thank you! 15:48:11 Yeah, Pacific time sucks for coordinating with folks across the smaller pond. 15:50:23 hrybacki: schedule it at a time where kmalloc can make it. I'm not required in that meeting until we actually start talking about deploying that. 15:51:01 jaosorior: ack. Waiting to hear back on this appt. If it's still going on I'll move the meeting to later and update you 16:28:13 lbragstad should we look into suppressing the system scope warnings if it helps infra for the time being? 16:29:51 gagehugo oh - is that causing issues specifically? 16:30:35 looks like they're asking if we can clean up any deprecation warnings, or rather anything that is overly chatty 16:30:50 sure - we can look for ways to suppress it 16:31:02 might just need a line to one of the setup classes in keystone 16:32:16 or are they specifically looking for suppressing it in oslo.policy? 16:37:42 lbragstad, kmalloc sorry I missed the meeting. I am going to try and make 2 changes to Keystone to support Edge: 16:37:53 1. Optional Domain ID on create domain 16:38:10 2. optional "use the same logic as LDAP for Federated Ids"4 16:38:32 what's the usecase for #1? 16:39:03 lbragstad, they go togeteher 16:39:23 if I create the same domain in 2 keystones, and use the same ID, I can get matching userids 16:40:12 its the way the LDAP code works for shadow users, and this means we can keep both LDAP and Federated IDs consistent without having to do Database level synchronization 16:40:52 let me clarify 16:41:09 optional to allow the user to specify the domain ID when creating a domain 16:41:25 it was requested and denied under dolphm years ago, but at the project level 16:41:44 we can punt on projects for the first go round, but for identity synch, it is kindof important 16:41:44 yeah... 16:42:10 this came up recently... 16:42:14 like in Sydney 16:43:00 Its come up a few times 16:43:30 I'd like to be able to run multiple Keystones within a single deployment, and not have to upgrade them in lock-step 16:45:16 are you planning on using tokens between each region? 16:45:37 lbragstad, probably not 16:45:43 more likely K2K 16:45:57 or something like a hub spoke setup 16:46:34 Hub will only know about other keystones 16:47:20 And Federated Identity will match, so the user will have to get a new token when going from hub to spoke, so hub tokens are not valid anywhere but hub keystone 16:47:40 hmmm 16:50:31 lbragstad, tokens between would bring up lots of issues. Fernet means "If I can validate, I can sign" 16:50:59 so spokes would need to call back to hub in order to validate. Might be OK, but means Hub has to be up 16:52:42 what is the motivation for allowing setting id on create then? in the past reusing tokens was the use case and we pushed back hard on that 16:53:39 why isn't basic k2k sufficient? we've been pushing either k2k or galera syncing as the right approach for edge 16:54:30 cmurphy, so galera will work for , say 3 sites 16:54:37 I am looking at deployments with 20 16:54:59 eventually Galera is going to get scaled out 16:55:04 but lets look at K2K 16:55:37 say I have 2 sites (BOS and SFO) and I create a project in BOS. I want to add an SFO user to it 16:56:03 that has to happen completely in the BOS Keystone 16:56:12 but what if I only have access to the SFO keystone? 16:56:56 So, I think K2K will work for some manual cases, but the user Ids then end up being distinct 16:57:03 which for Audit is tough: 16:57:38 I need to got user U from IdP I1 on Keyston K1 is User UU from IdP I1 on Keystone K2 16:58:27 cmurphy, the big thing is the predictable IDs 16:59:15 is auditing the only reason? 16:59:18 cmurphy, now, we could do predictable Ids with Domain Name instead of ID. Just we would have to change the LDAP code now, too, if we want to keep it consistent 16:59:42 cmurphy, without predictable Ids, you can't do anything in a keystone for a user until they log in the first time 16:59:56 no groups or role assignments are possible 17:00:27 we have the mapping api for that 17:00:30 with a predictable ID, you can pre-populate a user (potentially) and their resources are there ahead of time 17:00:40 true 17:01:44 lbragstad: Error: Can't start another meeting, one is in progress. Use #endmeeting first. 17:01:49 Heh 17:02:22 * lbragstad sigh 17:02:26 #endmeeting