16:00:11 <lbragstad> #startmeeting keystone
16:00:12 <openstack> Meeting started Tue Jul 24 16:00:11 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:00:15 <openstack> The meeting name has been set to 'keystone'
16:00:21 <lbragstad> #link https://etherpad.openstack.org/p/keystone-weekly-meeting
16:00:23 <lbragstad> agenda ^
16:00:28 <wxy|> o/
16:00:44 <kmalloc> Will be here in 15m or so.  Wrapping up Dr appts
16:00:55 <gagehugo> o/
16:01:05 <knikolla> o/
16:02:50 <lbragstad> #topic release status
16:03:07 <lbragstad> we have several deadlines coming up
16:03:24 <lbragstad> but i think we've wrapped up the initiatives we're pushing for in queens
16:03:27 <lbragstad> s/queens/rocky/
16:04:00 <lbragstad> the last few bits for the strict-two-level enforcement model stuff is going through now
16:04:17 <lbragstad> i plan to have some patches to oslo.limit that consumes some of that stuff
16:04:42 <lbragstad> i know kmalloc has a bunch of reviews up for the flask work yet
16:05:06 <lbragstad> but it looks like we're getting past the ground work and into the actual migration of keystone APIs to flask
16:05:19 <lbragstad> which brings us to
16:05:25 <lbragstad> #topic flaskification status
16:05:37 <ayoung> He might not be here yet
16:05:50 <lbragstad> yeah - he's wrapping something up
16:05:50 <ayoung> can we table and return to it when kmalloc pipes in?
16:06:06 <lbragstad> sure
16:06:14 <lbragstad> #topic Stein PTG Planning
16:06:23 <lbragstad> #link https://etherpad.openstack.org/p/keystone-stein-ptg
16:06:43 <lbragstad> per usual business, i've started an etherpad for collecting ideas for us to talk about in denver at the PTG
16:06:55 <lbragstad> also note
16:06:57 <lbragstad> #link http://lists.openstack.org/pipermail/openstack-dev/2018-July/132390.html
16:07:02 <lbragstad> we're going to be short a day
16:07:16 <lbragstad> we have monday which is intended to be a cross-project day
16:07:30 <lbragstad> (identical to the identity-integration track we had in dublin)
16:07:52 <lbragstad> and then we have Thursday and Friday for keystone-specific discussions that don't require cross-project involvement
16:08:24 <lbragstad> i'm unclear as to why we only have 2 days versus 3... but i haven't received a response from ttx yet on the reasoning
16:08:34 <lbragstad> (i doubt it will be possible to change it at this point)
16:08:45 <lbragstad> more or less just curious as to why that was the case
16:08:49 <ayoung> 2 Days should be enough for Keystone specific topics
16:09:10 <lbragstad> it depends on what comes up...
16:09:29 <knikolla> we can always hang out at the lobby with drinks
16:09:30 <lbragstad> worst case, we try and organize or congregate somewhere if we need more time to work through things
16:09:45 <ayoung> Do we have a general "direction of Keystone" doc we are guiding off of?
16:09:52 <lbragstad> our group is usually pretty small
16:10:06 <lbragstad> ayoung: can you define "direction of Keystone"
16:10:08 <lbragstad> ?
16:10:11 <lbragstad> like, a mission statemetn?
16:10:52 <ayoung> In the past, I've done a page of "A vision for Keystone" to show how I propose we tackle the challenges I saw in the road.  I would see something like that, but more group managed and maintained
16:11:03 <lbragstad> ++
16:11:11 <lbragstad> we actually talked about that in denver last year
16:11:16 <ayoung> INstead of "implementaion of 2 level QUota" level of detail
16:11:31 <lbragstad> hrybacki:  had some action items to work through the notes from the discussion and propose a mission statement
16:11:39 <ayoung> https://adam.younglogic.com/2013/07/a-vision-for-keystone/
16:11:53 <ayoung> 5 short years ago
16:12:24 <lbragstad> i'd love to have a more specific missing statement
16:12:27 <ayoung> maybe start with a list of the biggest challenges?
16:13:07 <lbragstad> IMO - everything always seems like a fire.. and a mission statement would help organize things a bit
16:13:48 <ayoung> Edge/multisite.  INtegration with Kuberenets and the App layer have been on my mind.  Working without tokens.  Better self-service
16:15:10 <knikolla> ayoung, we can work on that in denver, and show the results of descussion in berlin if our talk gets approved.
16:15:31 <ayoung> Start with a list of challenges.  THen prioritize them and see if there is a natural ordering.
16:15:44 * lbragstad tried doing that in Pike
16:15:59 <ayoung> knikolla, I think we want to do this prior to Denver, and just review it there
16:16:00 <lbragstad> the RBAC and unified limit stuff seemed to be the hot items at the time
16:16:26 <knikolla> ayoung, i'd like to get critical mass of core approval
16:16:51 <ayoung> Yeah, I think I misread what you originally wrote
16:17:07 <ayoung> I was thinking of the full list, but you meant the Edge stuff specifically, right?
16:17:44 <knikolla> ayoung, that's my focus at the moment, federation and edge.
16:18:04 <kmalloc> O/
16:18:07 <knikolla> keep the goals focused
16:18:09 <ayoung> lbragstad, want to start a "Challenges for Keystone" doc?
16:18:40 <lbragstad> ayoung: can we just track it in etherpad for the time being until we comb through it a bit?
16:18:47 <ayoung> Like "here are things keystone might potentially tackle in the future"
16:19:03 <lbragstad> https://etherpad.openstack.org/p/keystone-stein-ptg
16:19:06 <ayoung> Etherpad is a good forum, just a specific doc for long term goals
16:19:22 <lbragstad> yeah - i'm not opposed to that one we have consensus on a set of those goals
16:19:52 <lbragstad> sounds like a good thing to work through at the PTG or iteratively until that point
16:20:34 <hrybacki> o/ sorry I'm late
16:21:38 <ayoung> #link https://etherpad.openstack.org/p/keystone-goals-challenges
16:22:56 <ayoung> lbragstad, want to switch back to Flask since we have kmalloc ?
16:23:20 <lbragstad> one minute
16:23:40 <lbragstad> diving into the pain points is useful, but IMO keystone is still missing an overall direction
16:23:52 <lbragstad> at least one that isn't super vague and outdated
16:24:07 <hrybacki> too broad of scope?
16:24:39 <lbragstad> i wouldn't be opposed to starting with that on one end and the pain point at the other and seeing if we can work towards a common ground
16:25:01 <ayoung> lbragstad, hrybacki right.  The docusment I posted is to foster that discussion
16:25:12 <ayoung> start with the challenges we can foresee, order them, etc
16:25:52 <lbragstad> ok - i'll add my $0.02
16:26:04 <lbragstad> but we can move on if there isn't anything pressing
16:26:30 <lbragstad> #topic requirements freeze and other deadlines
16:26:32 <lbragstad> #link http://lists.openstack.org/pipermail/openstack-dev/2018-July/132418.html
16:26:41 <ayoung> lbragstad, I'd say have people update that, and then review and discuss at the start of the Summit Keystone specific sessions
16:26:43 <lbragstad> in case you didn't see the note from sean*
16:27:41 <lbragstad> #info feature freeze, client library freeze, and requirements freeze is Thursday
16:27:56 <lbragstad> we do have some of wxy|'s work going through the gate now
16:28:10 <lbragstad> i see that knikolla has a patch up for the mutable config stuff
16:28:18 <lbragstad> we should try and focus on that prior to Thursday
16:28:25 <kmalloc> :)
16:28:28 <lbragstad> i also have some patches up for client support
16:28:34 <lbragstad> for registered and project limits
16:28:54 <lbragstad> in case anyone is interested in testing those out
16:28:55 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/python-openstackclient+branch:master+topic:bp/unified-limits
16:28:59 <knikolla> yep, mutable config works, though before_request functions fire off only for apis which have migrated to flask
16:29:04 <knikolla> fires*
16:29:26 <kmalloc> which is not terrible
16:29:35 <kmalloc> since we're making rapid moves that way
16:29:41 <lbragstad> sounds like we're going to have to consider landing the mutable config patch after rocky-3 then/
16:29:55 <kmalloc> we can land the patch now tbh.
16:30:10 <lbragstad> but it won't be fully supported until all the flask patches land
16:30:19 <kmalloc> y
16:30:22 <kmalloc> yes
16:30:26 <lbragstad> ok
16:30:41 <knikolla> i need to write a bunch of tests and a release note, which i'll do later today
16:30:54 <lbragstad> thanks knikolla
16:31:06 <knikolla> but it's a simple and small change, so reviewing should be pretty quick
16:31:14 <lbragstad> cool
16:31:25 <lbragstad> we also have requirements freeze this thursday
16:32:57 <lbragstad> if anyone notices anything there that needs to be updated, let me know or propose it
16:33:21 <lbragstad> does anyone have questions regarding upcoming deadlines?
16:33:25 <kmalloc> i'll worry about restplus next release
16:34:19 <lbragstad> let's circle back to the previous topic
16:34:25 <lbragstad> #topic flaskification status
16:34:39 <kmalloc> yay
16:34:54 <kmalloc> it's moving.
16:35:10 <kmalloc> there are a lot of patches, because ultimately no test changes should occur in an api move
16:35:12 <lbragstad> seems like this and the token provider refactor are the only really big changes left to land this release (outside of RC bugs)
16:35:24 <kmalloc> i'll have auth done today [i hope]
16:35:37 <ayoung> kmalloc, a suggestion
16:35:40 <kmalloc> and then will be moving on to catalog, federation, policy, and endpoint_policy
16:35:50 <ayoung> when you move the files to the api directory, do that as a stand along patch
16:36:02 <ayoung> mostly they are the controllers code, and that can live anywhere
16:36:15 <ayoung> it is only in, say trusts, by convention
16:36:21 <kmalloc> except it doesn't load unless it is in .api
16:36:29 <kmalloc> and .api only loads flask-isms
16:36:47 <ayoung> kmalloc, will it load by accident if you move it?
16:36:59 <ayoung> identity/controllers to api/identity.py?
16:37:16 <kmalloc> no it's explicit, but .api should never have non-flask code in it
16:37:41 <kmalloc> by design. we have cases where we do cross-system imports
16:37:46 <ayoung> right, but you can have 2 steps in a migration.  Perhaps it would be cleaner to do this:
16:37:56 <kmalloc> and in .api, that is explicitly not allowed to avoid circular issues
16:38:17 <ayoung> api/identiyt is just a pointer to identity/controllers in the first patch, and then move identity/controllers to api in the second
16:38:17 <kmalloc> i tried that, it added 3-4x the number of patches
16:38:21 <ayoung> it was just hard to review
16:38:25 <ayoung> understood
16:38:28 <kmalloc> yeah =/
16:38:38 <kmalloc> trusts was particularly hard.
16:38:50 <kmalloc> most conversions look more like credentials
16:39:17 <kmalloc> it's why i try and make the conversion just the controller code.
16:39:19 <ayoung> what I ended up doing was loading both files up in unified format in differnt, side by side browser windows, and using comments to confirm I had checked each
16:39:22 <kmalloc> i know it's dense.
16:39:34 <kmalloc> yep, that is what i recommend, it's largely how i have to code it too
16:39:39 <ayoung> that is also a viable technique, just add it as a suggestion to reviewers in those patches, then
16:39:48 <ayoung> fair enough
16:40:05 <kmalloc> :)
16:40:13 <kmalloc> it is why i am tryuing to keep the controller -> .api move just moving that code
16:40:22 <kmalloc> including not touching tests if at all possible
16:40:26 <ayoung> I think Trusts had some non-API type logic in it that should probably move to core, too
16:40:31 <kmalloc> it does.
16:40:32 <ayoung> ++
16:40:34 <kmalloc> auth does too.
16:40:42 <kmalloc> but that is totally out of scope :)
16:40:53 <kmalloc> we have a lot of post flask cleanup to do
16:41:00 <ayoung> So. can api/auth.py call into auth/controllers?
16:41:01 <kmalloc> it will make things better, but it's slow work.
16:41:06 <kmalloc> no.
16:41:19 <kmalloc> api/auth.py may only call code locally *
16:41:32 <kmalloc> there are some exceptions, e.g. in trusts becuase i have to fix wrap_member
16:41:41 <kmalloc> so you can specify "wrap_member, for roles" for example
16:42:03 <ayoung> any helper functions should be able to live in a remote package, though.  We call into core.py, right?
16:42:06 <kmalloc> but ultimately api/<file>.py should not call out to anything except .core
16:42:10 <kmalloc> yes
16:42:19 <kmalloc> controllers goes away, which is why you can't call to it
16:42:24 <kmalloc> [to avoid confusion]
16:42:46 <kmalloc> revision: api/file should not call outside EXCEPT to providers.XXXX
16:42:54 <kmalloc> or to the flask-core code via what is imported
16:43:05 <kmalloc> [common code/providers only]
16:43:21 <kmalloc> you should never need to api/auth -> api/trusts  or api/roles
16:43:27 <ayoung> try to keep the helper functions in the same order at least
16:43:59 <ayoung> and reorder them prior to migration if necessary, or in a separate patch afterwards, so we can spot across more easily.   This is tough to review
16:44:33 <ayoung> it also means that the wrong person will get blamed for lines of code in a git-blame
16:44:51 <kmalloc> right now, i'm going to be blamed forever for all things in .apu
16:44:53 <kmalloc> api*
16:45:00 <kmalloc> i am ok with this.
16:45:08 <kmalloc> but i knew that going in.
16:45:39 <ayoung> anyway, just some suggestions gleaned from the trust review
16:47:00 <ayoung> lbragstad, I'm done
16:47:07 <ayoung> suspect kmalloc is too
16:47:13 <lbragstad> #topic open discussion
16:47:16 * kmalloc is.
16:47:21 <lbragstad> thanks kmalloc ayoung
16:47:45 <lbragstad> i could use some reviews on the token provider refactor (we did a high-bandwidth group review a couple weeks ago)
16:48:12 <kmalloc> i missed it
16:48:22 <kmalloc> but i know most of the code, i can try and get eyes on it.
16:48:27 <lbragstad> thanks
16:48:31 <lbragstad> i appreciate it
16:48:36 <kmalloc> trying to  balance flask work and other code just because it's huge context switches
16:48:42 <lbragstad> right
16:49:03 <ayoung> links?
16:49:03 <lbragstad> that's about all i have
16:49:16 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bug/1778945
16:49:23 <kmalloc> ++
16:49:56 <wxy|> lbragstad: I'm still testing the code. especially the performance impact. Overall it looks good.
16:50:05 * gagehugo will take a look
16:50:12 <kmalloc> lbragstad: hey just think in Stein we can do keystone.X to keystone.subsystem.X [for non-common code] and play test cleanup, with self.test_client ... ^_^
16:50:22 <lbragstad> wxy|: have you noticed anything negative performance-wise?
16:50:38 <wxy|> lbragstad: not yet. keep trying.
16:50:59 <lbragstad> wxy|: thanks for testing that aspect of it
16:51:42 <lbragstad> for the most part, we're doing the same thing just in different places in order to establish better interfaces and boundaries
16:53:41 <lbragstad> if there isn't anything else, we can call it and prep for office hours
16:55:03 <lbragstad> thanks for coming!
16:55:08 <lbragstad> #endmeeting