16:00:29 <lbragstad> #startmeeting keystone
16:00:30 <openstack> Meeting started Tue Nov 27 16:00:29 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:34 <openstack> The meeting name has been set to 'keystone'
16:00:34 <lbragstad> #link https://etherpad.openstack.org/p/keystone-weekly-meeting
16:00:37 <lbragstad> agenda ^
16:00:59 <wxy|> o/
16:01:20 <gagehugo> o/
16:01:37 <lbragstad> we can give folks a couple minutes
16:02:45 <knikolla> o/
16:02:52 <jdennis> jdennis: o/
16:03:33 <lbragstad> alright - let's go ahead and get started
16:03:39 <lbragstad> #topic announcements
16:04:00 <lbragstad> #info we're just over a month away from milestone-2
16:04:13 <lbragstad> reminder that milestone-2 is going to mark specification freeze
16:04:26 <lbragstad> and feature proposal freeze is only a couple weeks after that
16:04:47 <lbragstad> there are still several specifications up for review that we're planning on implementing this release
16:04:53 <lbragstad> please take a look if you have time
16:05:21 <lbragstad> #link https://review.openstack.org/#/c/599491/
16:05:27 <lbragstad> #link https://review.openstack.org/#/c/541903/
16:05:54 <lbragstad> and a bunch of those ones for the edge/multi-region/federated use cases we talked about in Berlin
16:06:06 <lbragstad> also
16:06:07 <lbragstad> #info keystoneclient-devstack-functional is failing
16:06:23 <lbragstad> frickler brought this to us this morning
16:06:56 <lbragstad> it's been failing consistently for some time
16:06:57 <gagehugo> mysql password incorrect?
16:07:07 <lbragstad> #link http://logs.openstack.org/39/605539/24/check/keystoneclient-devstack-functional/9fff540/job-output.txt.gz#_2018-11-27_04_39_26_939041
16:07:15 <lbragstad> yeah - it's strange
16:07:30 <lbragstad> I noticed other projects have similar scripts, nearly identical actually
16:07:41 <lbragstad> but they their functional jobs aren't failing
16:07:47 <lbragstad> but their*
16:08:02 <lbragstad> so it might be something with zuul + how we set things up in keystone
16:08:51 <lbragstad> anyway - wanted to plug it here in case it piqued anyone'
16:08:56 <lbragstad> anyone's interest*
16:09:12 <lbragstad> #topic Keystone as an IdP
16:09:17 <lbragstad> i'm not sure kmalloc is around
16:09:40 <lbragstad> but the plan is to go through all the bits for this in more detail, since it was discussed at length in Berlin
16:10:01 <lbragstad> and there are more than a handful of new specs related to it
16:10:14 <lbragstad> we'll circle back if kmalloc hops on
16:10:21 <lbragstad> #topic default roles and system-scope progress
16:10:39 <lbragstad> this is one of the bigger initiatives we're tackling this release
16:10:58 <lbragstad> i apologize for all the IRC bot and bug spam recently
16:11:15 <lbragstad> but I broke everything out into smaller bug reports, hoping that it will help enable people to pick things up
16:11:28 <kmalloc> O/
16:11:30 <lbragstad> so they don't feel pressured into committing to a whole pile of work
16:11:50 <lbragstad> they can just pick up a couple things here or there if they have time, but would still be a huge help
16:12:22 <lbragstad> ultimately, i created bugs for all keystone policies/apis that aren't currently using the defaults roles work hrybacki did in rocky
16:12:23 <lbragstad> #link https://bugs.launchpad.net/keystone/+bugs?field.tag=default-roles
16:12:38 <kmalloc> here.
16:12:39 <kmalloc> sorry
16:12:52 <lbragstad> no worries - i'll wrap up my topic quick and hand the floor over
16:13:05 <ayoung> works for me
16:13:08 <kmalloc> no arch diagram that will be next week.
16:13:12 <lbragstad> here is what an example fix for the default role bugs looks like
16:13:14 <kmalloc> but we can go over stuff otherwise.
16:13:18 <lbragstad> #link https://review.openstack.org/#/c/620156/1
16:13:45 <lbragstad> it's mostly tests that showcase the behaviors for each scope
16:13:45 <ayoung> there are some subtlties on the implied roles one, I added a comment.  Lets us keep a place for those convos, so, I like the smaller bug reports
16:13:54 <lbragstad> ++
16:14:21 <lbragstad> i also have other reports dedicated to system-scope gaps
16:14:26 <lbragstad> #link https://bugs.launchpad.net/keystone/+bugs?field.tag=system-scope
16:14:34 <lbragstad> ideally - they go hand-in-hand
16:15:06 <lbragstad> but depending on the API, there isn't a dependency between those two bugs if they affect the same API
16:15:27 <lbragstad> just trying to make sure we track how much work it takes to fix all this
16:15:56 <lbragstad> does anyone have comments, questions, or concerns about system-scope or default roles work?
16:16:08 <lbragstad> or wants to jump in and pick up one or two?
16:16:10 <lbragstad> ;)
16:16:11 <ayoung> lbragstad, we are ok with breaking people with these, right?
16:16:28 <lbragstad> break people how?
16:17:04 <ayoung> changing roles for APIs will not match the old policies.
16:17:24 <lbragstad> yeah - so we have tooling in oslo.policy to handle that for us
16:17:30 <lbragstad> and make it graceful for operators
16:17:36 <ayoung> Hopefully in a "it not longer works" way as oppposed to "oops we let something else in" way
16:18:20 <lbragstad> my goal is to be explicit with the former
16:18:26 <kmalloc> as long as we support the model of: [override-new] > [override-old] > (DEFAULT NEW || DEFAULT OLD)
16:18:37 <kmalloc> we should be 100% ok
16:18:45 <lbragstad> so it's *really* clear what we support by default from an authorization perspective upstream
16:18:45 <kmalloc> and not letting random stuff fall through
16:19:05 <kmalloc> just adding additional permissions that operators can opt into
16:19:09 <kmalloc> (for transition)
16:19:28 <lbragstad> #link https://review.openstack.org/#/c/614195/ should help with that
16:19:38 <kmalloc> and then it becomes Override NEw > Default New (eventually)
16:19:43 <lbragstad> same with #link https://review.openstack.org/#/c/613635/
16:19:44 <kmalloc> once transition is done
16:19:47 <kmalloc> long view.
16:20:51 <lbragstad> we'll also need #link https://review.openstack.org/#/c/611443/
16:21:51 <lbragstad> kmalloc those cases might be addressed in https://review.openstack.org/#/c/614195/5/oslo_policy/tests/test_policy.py
16:22:05 <kmalloc> lbragstad: i'll check
16:22:14 <lbragstad> thanks
16:22:36 <kmalloc> i want to be sure
16:22:38 <kmalloc> :)
16:22:47 <lbragstad> ultimately, everything under keystone.tests.unit.protection.v3 should explicitly test each scope against each default role
16:22:51 <kmalloc> i need to leave as soon as the meeting is over btw.
16:23:08 <lbragstad> ok - that's about all i had for this
16:23:23 <lbragstad> feel free to ping me if you'd like to chip in on a couple of those bugs, or have questions
16:23:36 <lbragstad> otherwise, i have fixes for several of them up (i need to update the branch)
16:23:49 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bug/1788415
16:24:09 <lbragstad> and they are all dependent on #link https://review.openstack.org/#/c/605539/
16:24:18 <lbragstad> once ^ merges, I'll rebase them
16:24:52 <lbragstad> any other questions?
16:25:16 <kmalloc> ... "lance, where do users come from?" :P
16:25:24 <ayoung> Are we going to go into this detail for the other services?
16:25:28 <lbragstad> HTTP 403
16:25:36 <lbragstad> ayoung how do you mean?
16:25:46 <ayoung> Nova
16:25:48 <kmalloc> ayoung: we will provide help as we did for in-code policy
16:25:52 <lbragstad> like helping them consume these changes?
16:26:00 <ayoung> Neutron and so on.  Bug reports for the APIS?
16:26:13 <lbragstad> i'll likely leave that to each project
16:26:15 <ayoung> a way to have the convos about what scope a given API should really have?
16:26:22 <kmalloc> the plan is to support and make it a community goal
16:26:28 <kmalloc> similar to how we did policy-in-code
16:26:33 <lbragstad> right
16:26:36 <ayoung> Yeah...and GLance will get around to it.
16:26:41 <kmalloc> glance needs help
16:26:50 <kmalloc> it is probably a place we need to explicitly step up
16:26:52 <lbragstad> before we can do that we'll need to have it working in keystone first
16:26:53 <kmalloc> =/
16:27:15 <kmalloc> but i think the order is Keystone, Nova, Community Goal... and supporting glance
16:27:17 <lbragstad> jokke and i had a conversation about how policy-in-code works
16:27:33 <lbragstad> i think he's up to speed now
16:27:40 <kmalloc> glance will get there
16:27:48 <lbragstad> he told me a couple days ago that he's going to poke with the changes locally
16:27:50 <ayoung> https://review.openstack.org/#/c/501360/  Still -1.  I'll rework that
16:27:52 <kmalloc> they might get policy-in-code and new defaults in short succession
16:28:59 <ayoung> Cool.  This really needs to be in across the board to be usable
16:29:06 <kmalloc> that is the plan.
16:29:06 <lbragstad> i agree
16:29:10 <ayoung> We need a banana test in Tempest
16:29:29 <ayoung> create a "banana" role and assign it.  It should not be able to do anything
16:29:32 <lbragstad> gmann is working on the system-scope stuff from the tempest side
16:30:04 <kmalloc> ayoung: sorryk, but our new super-admin role is named banana
16:30:11 <kmalloc> /s
16:31:08 <ayoung> https://lh3.googleusercontent.com/-CjxR7w-1iHA/UvfAaZCCr6I/AAAAAAAAJtk/G-Knih6Ze7M/s400/We%2520Are%2520in%2520a%2520Book%2520Mo%2520Willems%25202.jpg
16:31:31 <ayoung> You guys don't know Elephant and Piggy.  yet.
16:31:41 <lbragstad> anything else before we move on?
16:32:33 <lbragstad> #topic Keystone as an IdP Proxy
16:32:37 <lbragstad> kmalloc you're up
16:32:42 <kmalloc> ok
16:32:59 <kmalloc> quick note: I am working on an architecture diagrame
16:33:11 <knikolla> awesome
16:33:16 <kmalloc> it should be done for the next meeting. but with holiday/travel/other things... it's a bit delayed
16:33:17 <lbragstad> cc ildikov ^
16:33:37 <kmalloc> it will cover the forward looking goals I see for Keystone, specifically how it works as an IDP and an IDP Proxy
16:33:52 <ayoung> Why do we call it a Proxy?
16:34:02 <kmalloc> officially that is the wrong term
16:34:09 <knikolla> broker?
16:34:11 <kmalloc> i am trying to reword that to be Broker
16:34:12 <kmalloc> yes
16:34:15 <lbragstad> because the idea was to shuffle identities between formats
16:34:22 <ayoung> And not just IdP?
16:34:28 <ayoung> Ah
16:34:29 <kmalloc> it's some rewiring in my head to keep saying Identity Broker
16:34:43 <lbragstad> so if you have a google user
16:34:47 <kmalloc> we will be a full featured IDP but also have the ability to broker from one form to another
16:35:00 <kmalloc> IDP[s] -> Keystone -> SP[s]
16:35:01 <ayoung> Translation from SAML to OIDC and so on?
16:35:02 <lbragstad> you can use keystone to convert whatever google gives you to prove your identity, to something else
16:35:03 <kmalloc> yes
16:35:23 <kmalloc> ayoung: the SPs will consume whatever they consume, keystone will broker from one form to that form for the SP.
16:35:36 <kmalloc> or the *best* form for the SP in the case it supports many
16:35:39 <knikolla> right now we already do that but with SAML ECP.
16:35:50 <ayoung> Adapter
16:35:50 <knikolla> for the k2k pieces.
16:36:02 <kmalloc> the industry(ish) term is broker for this
16:36:21 <lbragstad> well - we do have a great track record for naming things
16:36:26 <ayoung> https://en.wikipedia.org/wiki/Adapter_pattern
16:36:32 <kmalloc> it is an adapter pattern
16:36:35 <ayoung> But...Broker is right
16:36:38 <kmalloc> yep.
16:36:42 <kmalloc> 100%
16:36:49 <ayoung> because we are not making an Adapter, we are converting one to the other
16:36:59 <kmalloc> so, the core bits we need from today.
16:37:11 <kmalloc> 1) Auth will be split from CRUD (backlogged SPEC)
16:37:23 <kmalloc> this is so our well-defined endpoints for auth are located at /auth/XXXX
16:37:34 <kmalloc> /v3/auth will reamin
16:37:36 <kmalloc> remain*
16:37:41 <kmalloc> no one will be broken on that front
16:37:55 <ayoung> But...we are going to add additional auth attributes in addition to the original assertion.  Specifically, we add the Keystone role assignment data.  THey can ignore it, but they can consume it, too, right?
16:38:16 <kmalloc> the goal is here 2 fold: let us iterate on crud independant of auth *and* auth can be exposed in isolation from crud for auth to the SPs
16:38:32 <kmalloc> ayoung: correct. we will pass through but also allow for applying keystone permissions directly
16:38:45 <kmalloc> ayoung: that is the "virtual organization" parts.
16:39:18 <ayoung> Cool.  This is going to explode on us, but, I think, in a good way
16:39:32 <kmalloc> the second bit we need (2)
16:39:40 <kmalloc> is the principal object
16:39:49 <kmalloc> this is to replace shadow users.
16:39:56 <kmalloc> and be fully featured
16:40:11 <lbragstad> hopefully we can just extend shadow users
16:40:14 <kmalloc> keystone will maintain a single consistent user object that many AuthN sources can hook onto
16:40:29 <kmalloc> it is either "extend and fix shadow users" or "replace shadow users and drop shadow users"
16:40:34 <kmalloc> it looks to be about the same amount of work
16:40:43 <lbragstad> yeah - i just hope its the first and not the second :)
16:40:44 <kmalloc> and i worry how deep / odd shadow users is due to where it left off
16:41:00 <kmalloc> Key bits: Users are principals
16:41:07 <kmalloc> Groups are groupings of principals
16:41:16 <kmalloc> app creds are principals
16:41:35 <kmalloc> projects are *most likely* a group of principals
16:41:40 <ayoung> app creds contain a principal and a delegation
16:41:45 <kmalloc> ayoung: ++
16:42:28 <lbragstad> i think the next step in that work is to grok the current state of things
16:42:35 <kmalloc> the key is normalizing the data structure and making sure we have a clear object that AuthN hooks into, a source of AuthN will be the SQL (password) backend or LDAP backend
16:42:48 <lbragstad> and see if we can trace steps that rderose and dstanek were workings towards
16:42:56 <kmalloc> these will not implement the entire identity driver anymore, they will be a source of Auth hooked onto the user principal
16:43:35 <kmalloc> any questions so far? I can keep moving on the rest of the bits needed
16:43:56 <ayoung> Are we going to support a basi-cauyth mechansim under /auth?
16:44:04 <ayoung> basic-auth
16:44:16 <kmalloc> ayoung: the plan would be to be more fully featured on that front
16:44:25 <kmalloc> and implement as much as we can directly in python
16:44:34 <kmalloc> we may offload to a web server/module
16:44:45 <lbragstad> ayoung  like #link https://en.wikipedia.org/wiki/Basic_access_authentication ?
16:44:46 <kmalloc> but we should implement the functionality 100% in python where possible
16:44:48 <ayoung> Yeah, basic auth would have to be Python
16:45:00 <kmalloc> lbragstad: yeah, both basic and digest mode.
16:45:19 <ayoung> and work based on a GET
16:45:23 <kmalloc> part of that deal is there will be a UI added to keystone.
16:45:52 <kmalloc> we are a standalone IDP, deployers need a way to interact with keystone
16:45:58 <kmalloc> in isolation of horizon etc
16:46:05 <kmalloc> we will continue support for horizon (of course)
16:46:30 <ayoung> "All My plans are coming together"
16:46:36 <kmalloc> BASIC AUTH, SAML, OIDC, Digest+Basic will work
16:46:59 <kmalloc> we will also implement support for U2F/FIDO in the ui for Multi-factor Auth
16:47:32 <kmalloc> the UI will be something akin to React based (may change the framework)
16:47:47 <kmalloc> the goal is to strictly reference the API not be a layer inbetween with more python (e.g. django)
16:48:31 <ayoung> Do we have an HTML renderer for Flask?
16:48:34 <kmalloc> there will be discussions about a V4 crud api along the way for supporting the UI because we may want to restructure how the API works for this (breaking changes, but mostly cruft cleanup/re-homing)
16:48:43 <kmalloc> ayoung: flask easily supports it
16:48:58 <kmalloc> ayoung: we already use it in a couple places, notably in 404 errors
16:49:04 <kmalloc> unrouted-404 errors
16:49:13 <kmalloc> and some other cases (ec2)
16:49:44 <kmalloc> the V4 CRUD api will be discussed one the core bits of Keystone are worked through
16:50:01 <kmalloc> once*
16:50:22 <kmalloc> that would be in support of the UI. restructuring the API under flask is much faster if we decide to do this.
16:50:42 <kmalloc> a couple additional security bits will be needed
16:50:48 <ayoung> Excellent
16:50:49 <kmalloc> JWT (for full OIDC support)
16:50:57 <kmalloc> yes i classify that as security
16:51:20 <lbragstad> <shameless-plug>The jwt stuff is up for review along with the specification</shameless-plug>
16:51:28 <kmalloc> i want to fully support the timestamp protocol as well for signing when things occur (creation/cadf/tokens) as well
16:52:49 <kmalloc> we will need to look at the at-rest data storage in SQL and ensure we are being good at PII, and can support PCI-DSS/NIST recommendations as well as cover GDPR concerns
16:52:55 <lbragstad> time check - 7 minutes left
16:52:59 <kmalloc> thanks
16:53:31 <knikolla> this, adjutant, and athenz makes me so happy.
16:53:54 <ayoung> kmalloc, you do all your keystone development in containers, right?  Do you have a document contributors can follow?  Should we get that into keystone/doc/source?
16:53:58 <kmalloc> we will finally need to add much much much better autoprovisioning
16:54:18 <kmalloc> ayoung: i plan to get that codified into git (the dev docs)
16:54:24 <ayoung> ++
16:54:26 <kmalloc> and yes, i use containers for everything
16:54:29 * lbragstad has ideas for that based on what penick was talking about
16:54:35 <kmalloc> lbragstad: exactly
16:54:39 <ayoung> I'll revisit.  Its been a year
16:54:43 <ayoung> or more
16:54:47 <kmalloc> so, in Stein: I want these things to land
16:54:54 <kmalloc> 1) Auth support at /auth
16:54:55 <lbragstad> fwiw - i was going to wait for recordings to get posted for posting by summary
16:55:03 <knikolla> with the above things aligning with what i need to get done for the MOC, y'all get 60% of my time.
16:55:08 <lbragstad> but that's going to be a bit, so i'll just publish today and update later
16:55:12 <kmalloc> 2) principal work (shadow users rework)
16:55:20 <knikolla> up from 20%
16:55:29 <kmalloc> 3) Federation support (brokering) changes
16:55:42 <kmalloc> 4) JWT
16:55:45 <kmalloc> (no particular order)
16:56:09 <kmalloc> autoprovisioning becomes a #5 if we can
16:56:35 <kmalloc> v4 API, UI, Timestamp protocol, those will likely be post Stein
16:56:59 <kmalloc> we will also need a LOT of cleanup on our internal SQL store.
16:57:10 <kmalloc> oh one last bit we need to clearly work out
16:57:16 <kmalloc> E-Tag/Cache-Control
16:57:22 <ayoung> Videos are slow to land this summit.  Something is not working right in the process. Used to be up during the week of.
16:57:24 <kmalloc> which comes post UI
16:57:51 <kmalloc> so i see 3-4 specs in Stein.
16:58:01 <kmalloc> still to do.
16:58:10 <kmalloc> JWT is almost done, so that is easy
16:58:42 * kmalloc hands the mic back to lbragstad
16:58:45 <knikolla> kmalloc: i can take on some of that during office hours. together with polishing the renewable app creds specs.
16:58:52 <kmalloc> cool.
16:58:53 <knikolla> so let's sync up
16:58:56 <lbragstad> #topic open discussion
16:58:59 <kmalloc> oh yeah refreshable app creds needed too
16:59:00 <kmalloc> haha
16:59:05 <lbragstad> one minute left if anyone has anything
16:59:07 <kmalloc> i am AFK for a few hours post meeting
16:59:11 <kmalloc> fyi (knikolla)
16:59:19 <kmalloc> not IDP related
16:59:25 <kmalloc> we should explore gabbi for functional tests
16:59:35 <kmalloc> cdent has done a good chunk of work on it.
16:59:37 <kmalloc> it's awesome.
16:59:48 <kmalloc> gyee: you're off by an hour :P DST!
16:59:51 <lbragstad> alright - let's wrap up
17:00:01 <lbragstad> thanks for coming, all
17:00:18 <lbragstad> reminder office hours in -keystone
17:00:22 <lbragstad> #endmeeting