16:00:29 #startmeeting keystone 16:00:30 Meeting started Tue Nov 27 16:00:29 2018 UTC and is due to finish in 60 minutes. The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00:31 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 16:00:34 The meeting name has been set to 'keystone' 16:00:34 #link https://etherpad.openstack.org/p/keystone-weekly-meeting 16:00:37 agenda ^ 16:00:59 o/ 16:01:20 o/ 16:01:37 we can give folks a couple minutes 16:02:45 o/ 16:02:52 jdennis: o/ 16:03:33 alright - let's go ahead and get started 16:03:39 #topic announcements 16:04:00 #info we're just over a month away from milestone-2 16:04:13 reminder that milestone-2 is going to mark specification freeze 16:04:26 and feature proposal freeze is only a couple weeks after that 16:04:47 there are still several specifications up for review that we're planning on implementing this release 16:04:53 please take a look if you have time 16:05:21 #link https://review.openstack.org/#/c/599491/ 16:05:27 #link https://review.openstack.org/#/c/541903/ 16:05:54 and a bunch of those ones for the edge/multi-region/federated use cases we talked about in Berlin 16:06:06 also 16:06:07 #info keystoneclient-devstack-functional is failing 16:06:23 frickler brought this to us this morning 16:06:56 it's been failing consistently for some time 16:06:57 mysql password incorrect? 16:07:07 #link http://logs.openstack.org/39/605539/24/check/keystoneclient-devstack-functional/9fff540/job-output.txt.gz#_2018-11-27_04_39_26_939041 16:07:15 yeah - it's strange 16:07:30 I noticed other projects have similar scripts, nearly identical actually 16:07:41 but they their functional jobs aren't failing 16:07:47 but their* 16:08:02 so it might be something with zuul + how we set things up in keystone 16:08:51 anyway - wanted to plug it here in case it piqued anyone' 16:08:56 anyone's interest* 16:09:12 #topic Keystone as an IdP 16:09:17 i'm not sure kmalloc is around 16:09:40 but the plan is to go through all the bits for this in more detail, since it was discussed at length in Berlin 16:10:01 and there are more than a handful of new specs related to it 16:10:14 we'll circle back if kmalloc hops on 16:10:21 #topic default roles and system-scope progress 16:10:39 this is one of the bigger initiatives we're tackling this release 16:10:58 i apologize for all the IRC bot and bug spam recently 16:11:15 but I broke everything out into smaller bug reports, hoping that it will help enable people to pick things up 16:11:28 O/ 16:11:30 so they don't feel pressured into committing to a whole pile of work 16:11:50 they can just pick up a couple things here or there if they have time, but would still be a huge help 16:12:22 ultimately, i created bugs for all keystone policies/apis that aren't currently using the defaults roles work hrybacki did in rocky 16:12:23 #link https://bugs.launchpad.net/keystone/+bugs?field.tag=default-roles 16:12:38 here. 16:12:39 sorry 16:12:52 no worries - i'll wrap up my topic quick and hand the floor over 16:13:05 works for me 16:13:08 no arch diagram that will be next week. 16:13:12 here is what an example fix for the default role bugs looks like 16:13:14 but we can go over stuff otherwise. 16:13:18 #link https://review.openstack.org/#/c/620156/1 16:13:45 it's mostly tests that showcase the behaviors for each scope 16:13:45 there are some subtlties on the implied roles one, I added a comment. Lets us keep a place for those convos, so, I like the smaller bug reports 16:13:54 ++ 16:14:21 i also have other reports dedicated to system-scope gaps 16:14:26 #link https://bugs.launchpad.net/keystone/+bugs?field.tag=system-scope 16:14:34 ideally - they go hand-in-hand 16:15:06 but depending on the API, there isn't a dependency between those two bugs if they affect the same API 16:15:27 just trying to make sure we track how much work it takes to fix all this 16:15:56 does anyone have comments, questions, or concerns about system-scope or default roles work? 16:16:08 or wants to jump in and pick up one or two? 16:16:10 ;) 16:16:11 lbragstad, we are ok with breaking people with these, right? 16:16:28 break people how? 16:17:04 changing roles for APIs will not match the old policies. 16:17:24 yeah - so we have tooling in oslo.policy to handle that for us 16:17:30 and make it graceful for operators 16:17:36 Hopefully in a "it not longer works" way as oppposed to "oops we let something else in" way 16:18:20 my goal is to be explicit with the former 16:18:26 as long as we support the model of: [override-new] > [override-old] > (DEFAULT NEW || DEFAULT OLD) 16:18:37 we should be 100% ok 16:18:45 so it's *really* clear what we support by default from an authorization perspective upstream 16:18:45 and not letting random stuff fall through 16:19:05 just adding additional permissions that operators can opt into 16:19:09 (for transition) 16:19:28 #link https://review.openstack.org/#/c/614195/ should help with that 16:19:38 and then it becomes Override NEw > Default New (eventually) 16:19:43 same with #link https://review.openstack.org/#/c/613635/ 16:19:44 once transition is done 16:19:47 long view. 16:20:51 we'll also need #link https://review.openstack.org/#/c/611443/ 16:21:51 kmalloc those cases might be addressed in https://review.openstack.org/#/c/614195/5/oslo_policy/tests/test_policy.py 16:22:05 lbragstad: i'll check 16:22:14 thanks 16:22:36 i want to be sure 16:22:38 :) 16:22:47 ultimately, everything under keystone.tests.unit.protection.v3 should explicitly test each scope against each default role 16:22:51 i need to leave as soon as the meeting is over btw. 16:23:08 ok - that's about all i had for this 16:23:23 feel free to ping me if you'd like to chip in on a couple of those bugs, or have questions 16:23:36 otherwise, i have fixes for several of them up (i need to update the branch) 16:23:49 #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bug/1788415 16:24:09 and they are all dependent on #link https://review.openstack.org/#/c/605539/ 16:24:18 once ^ merges, I'll rebase them 16:24:52 any other questions? 16:25:16 ... "lance, where do users come from?" :P 16:25:24 Are we going to go into this detail for the other services? 16:25:28 HTTP 403 16:25:36 ayoung how do you mean? 16:25:46 Nova 16:25:48 ayoung: we will provide help as we did for in-code policy 16:25:52 like helping them consume these changes? 16:26:00 Neutron and so on. Bug reports for the APIS? 16:26:13 i'll likely leave that to each project 16:26:15 a way to have the convos about what scope a given API should really have? 16:26:22 the plan is to support and make it a community goal 16:26:28 similar to how we did policy-in-code 16:26:33 right 16:26:36 Yeah...and GLance will get around to it. 16:26:41 glance needs help 16:26:50 it is probably a place we need to explicitly step up 16:26:52 before we can do that we'll need to have it working in keystone first 16:26:53 =/ 16:27:15 but i think the order is Keystone, Nova, Community Goal... and supporting glance 16:27:17 jokke and i had a conversation about how policy-in-code works 16:27:33 i think he's up to speed now 16:27:40 glance will get there 16:27:48 he told me a couple days ago that he's going to poke with the changes locally 16:27:50 https://review.openstack.org/#/c/501360/ Still -1. I'll rework that 16:27:52 they might get policy-in-code and new defaults in short succession 16:28:59 Cool. This really needs to be in across the board to be usable 16:29:06 that is the plan. 16:29:06 i agree 16:29:10 We need a banana test in Tempest 16:29:29 create a "banana" role and assign it. It should not be able to do anything 16:29:32 gmann is working on the system-scope stuff from the tempest side 16:30:04 ayoung: sorryk, but our new super-admin role is named banana 16:30:11 /s 16:31:08 https://lh3.googleusercontent.com/-CjxR7w-1iHA/UvfAaZCCr6I/AAAAAAAAJtk/G-Knih6Ze7M/s400/We%2520Are%2520in%2520a%2520Book%2520Mo%2520Willems%25202.jpg 16:31:31 You guys don't know Elephant and Piggy. yet. 16:31:41 anything else before we move on? 16:32:33 #topic Keystone as an IdP Proxy 16:32:37 kmalloc you're up 16:32:42 ok 16:32:59 quick note: I am working on an architecture diagrame 16:33:11 awesome 16:33:16 it should be done for the next meeting. but with holiday/travel/other things... it's a bit delayed 16:33:17 cc ildikov ^ 16:33:37 it will cover the forward looking goals I see for Keystone, specifically how it works as an IDP and an IDP Proxy 16:33:52 Why do we call it a Proxy? 16:34:02 officially that is the wrong term 16:34:09 broker? 16:34:11 i am trying to reword that to be Broker 16:34:12 yes 16:34:15 because the idea was to shuffle identities between formats 16:34:22 And not just IdP? 16:34:28 Ah 16:34:29 it's some rewiring in my head to keep saying Identity Broker 16:34:43 so if you have a google user 16:34:47 we will be a full featured IDP but also have the ability to broker from one form to another 16:35:00 IDP[s] -> Keystone -> SP[s] 16:35:01 Translation from SAML to OIDC and so on? 16:35:02 you can use keystone to convert whatever google gives you to prove your identity, to something else 16:35:03 yes 16:35:23 ayoung: the SPs will consume whatever they consume, keystone will broker from one form to that form for the SP. 16:35:36 or the *best* form for the SP in the case it supports many 16:35:39 right now we already do that but with SAML ECP. 16:35:50 Adapter 16:35:50 for the k2k pieces. 16:36:02 the industry(ish) term is broker for this 16:36:21 well - we do have a great track record for naming things 16:36:26 https://en.wikipedia.org/wiki/Adapter_pattern 16:36:32 it is an adapter pattern 16:36:35 But...Broker is right 16:36:38 yep. 16:36:42 100% 16:36:49 because we are not making an Adapter, we are converting one to the other 16:36:59 so, the core bits we need from today. 16:37:11 1) Auth will be split from CRUD (backlogged SPEC) 16:37:23 this is so our well-defined endpoints for auth are located at /auth/XXXX 16:37:34 /v3/auth will reamin 16:37:36 remain* 16:37:41 no one will be broken on that front 16:37:55 But...we are going to add additional auth attributes in addition to the original assertion. Specifically, we add the Keystone role assignment data. THey can ignore it, but they can consume it, too, right? 16:38:16 the goal is here 2 fold: let us iterate on crud independant of auth *and* auth can be exposed in isolation from crud for auth to the SPs 16:38:32 ayoung: correct. we will pass through but also allow for applying keystone permissions directly 16:38:45 ayoung: that is the "virtual organization" parts. 16:39:18 Cool. This is going to explode on us, but, I think, in a good way 16:39:32 the second bit we need (2) 16:39:40 is the principal object 16:39:49 this is to replace shadow users. 16:39:56 and be fully featured 16:40:11 hopefully we can just extend shadow users 16:40:14 keystone will maintain a single consistent user object that many AuthN sources can hook onto 16:40:29 it is either "extend and fix shadow users" or "replace shadow users and drop shadow users" 16:40:34 it looks to be about the same amount of work 16:40:43 yeah - i just hope its the first and not the second :) 16:40:44 and i worry how deep / odd shadow users is due to where it left off 16:41:00 Key bits: Users are principals 16:41:07 Groups are groupings of principals 16:41:16 app creds are principals 16:41:35 projects are *most likely* a group of principals 16:41:40 app creds contain a principal and a delegation 16:41:45 ayoung: ++ 16:42:28 i think the next step in that work is to grok the current state of things 16:42:35 the key is normalizing the data structure and making sure we have a clear object that AuthN hooks into, a source of AuthN will be the SQL (password) backend or LDAP backend 16:42:48 and see if we can trace steps that rderose and dstanek were workings towards 16:42:56 these will not implement the entire identity driver anymore, they will be a source of Auth hooked onto the user principal 16:43:35 any questions so far? I can keep moving on the rest of the bits needed 16:43:56 Are we going to support a basi-cauyth mechansim under /auth? 16:44:04 basic-auth 16:44:16 ayoung: the plan would be to be more fully featured on that front 16:44:25 and implement as much as we can directly in python 16:44:34 we may offload to a web server/module 16:44:45 ayoung like #link https://en.wikipedia.org/wiki/Basic_access_authentication ? 16:44:46 but we should implement the functionality 100% in python where possible 16:44:48 Yeah, basic auth would have to be Python 16:45:00 lbragstad: yeah, both basic and digest mode. 16:45:19 and work based on a GET 16:45:23 part of that deal is there will be a UI added to keystone. 16:45:52 we are a standalone IDP, deployers need a way to interact with keystone 16:45:58 in isolation of horizon etc 16:46:05 we will continue support for horizon (of course) 16:46:30 "All My plans are coming together" 16:46:36 BASIC AUTH, SAML, OIDC, Digest+Basic will work 16:46:59 we will also implement support for U2F/FIDO in the ui for Multi-factor Auth 16:47:32 the UI will be something akin to React based (may change the framework) 16:47:47 the goal is to strictly reference the API not be a layer inbetween with more python (e.g. django) 16:48:31 Do we have an HTML renderer for Flask? 16:48:34 there will be discussions about a V4 crud api along the way for supporting the UI because we may want to restructure how the API works for this (breaking changes, but mostly cruft cleanup/re-homing) 16:48:43 ayoung: flask easily supports it 16:48:58 ayoung: we already use it in a couple places, notably in 404 errors 16:49:04 unrouted-404 errors 16:49:13 and some other cases (ec2) 16:49:44 the V4 CRUD api will be discussed one the core bits of Keystone are worked through 16:50:01 once* 16:50:22 that would be in support of the UI. restructuring the API under flask is much faster if we decide to do this. 16:50:42 a couple additional security bits will be needed 16:50:48 Excellent 16:50:49 JWT (for full OIDC support) 16:50:57 yes i classify that as security 16:51:20 The jwt stuff is up for review along with the specification 16:51:28 i want to fully support the timestamp protocol as well for signing when things occur (creation/cadf/tokens) as well 16:52:49 we will need to look at the at-rest data storage in SQL and ensure we are being good at PII, and can support PCI-DSS/NIST recommendations as well as cover GDPR concerns 16:52:55 time check - 7 minutes left 16:52:59 thanks 16:53:31 this, adjutant, and athenz makes me so happy. 16:53:54 kmalloc, you do all your keystone development in containers, right? Do you have a document contributors can follow? Should we get that into keystone/doc/source? 16:53:58 we will finally need to add much much much better autoprovisioning 16:54:18 ayoung: i plan to get that codified into git (the dev docs) 16:54:24 ++ 16:54:26 and yes, i use containers for everything 16:54:29 * lbragstad has ideas for that based on what penick was talking about 16:54:35 lbragstad: exactly 16:54:39 I'll revisit. Its been a year 16:54:43 or more 16:54:47 so, in Stein: I want these things to land 16:54:54 1) Auth support at /auth 16:54:55 fwiw - i was going to wait for recordings to get posted for posting by summary 16:55:03 with the above things aligning with what i need to get done for the MOC, y'all get 60% of my time. 16:55:08 but that's going to be a bit, so i'll just publish today and update later 16:55:12 2) principal work (shadow users rework) 16:55:20 up from 20% 16:55:29 3) Federation support (brokering) changes 16:55:42 4) JWT 16:55:45 (no particular order) 16:56:09 autoprovisioning becomes a #5 if we can 16:56:35 v4 API, UI, Timestamp protocol, those will likely be post Stein 16:56:59 we will also need a LOT of cleanup on our internal SQL store. 16:57:10 oh one last bit we need to clearly work out 16:57:16 E-Tag/Cache-Control 16:57:22 Videos are slow to land this summit. Something is not working right in the process. Used to be up during the week of. 16:57:24 which comes post UI 16:57:51 so i see 3-4 specs in Stein. 16:58:01 still to do. 16:58:10 JWT is almost done, so that is easy 16:58:42 * kmalloc hands the mic back to lbragstad 16:58:45 kmalloc: i can take on some of that during office hours. together with polishing the renewable app creds specs. 16:58:52 cool. 16:58:53 so let's sync up 16:58:56 #topic open discussion 16:58:59 oh yeah refreshable app creds needed too 16:59:00 haha 16:59:05 one minute left if anyone has anything 16:59:07 i am AFK for a few hours post meeting 16:59:11 fyi (knikolla) 16:59:19 not IDP related 16:59:25 we should explore gabbi for functional tests 16:59:35 cdent has done a good chunk of work on it. 16:59:37 it's awesome. 16:59:48 gyee: you're off by an hour :P DST! 16:59:51 alright - let's wrap up 17:00:01 thanks for coming, all 17:00:18 reminder office hours in -keystone 17:00:22 #endmeeting