18:00:11 <lbragstad> #startmeeting keystone
18:00:12 <openstack> Meeting started Tue Jan 16 18:00:11 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:13 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:15 <openstack> The meeting name has been set to 'keystone'
18:00:19 <lbragstad> #link https://etherpad.openstack.org/p/keystone-weekly-meeting
18:00:29 <lbragstad> ping ayoung, breton, cmurphy, dstanek, edmondsw, gagehugo, henrynash, hrybacki, knikolla, lamt, lbragstad, lwanderley, kmalloc, rderose, rodrigods, samueldmq, spilla, aselius, dpar, jdennis, ruan_he
18:00:32 <lbragstad> agenda ^
18:00:38 <knikolla> o/
18:00:39 <lamt`> o/
18:00:41 <gagehugo> o/
18:00:44 <spilla> O/
18:00:48 <kmalloc> bah i need more coffee.
18:01:13 <lbragstad> pretty light agenda today
18:01:24 <lbragstad> so we can give folks a few minutes to show up
18:02:53 <cmurphy> o/
18:03:41 <lbragstad> alright - let's get started
18:04:08 <lbragstad> #topic announcements: non-client library freeze
18:04:23 <lbragstad> just a reminder that this week is going to be non-client library freeze
18:04:42 <lbragstad> so, keystonemiddleware, keystoneauth, oslo libraries, etc...
18:04:59 <lbragstad> if you're looking for things to review, those would be high priority
18:05:12 <lbragstad> #topic announcements: client freeze
18:05:25 <lbragstad> ^ that is going to be next week, which is also going to be feature freeze
18:05:49 <lbragstad> and we do have three big efforts still in flight (excluding the documentation work)
18:05:59 <lbragstad> application credentials
18:06:00 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/application-credentials
18:06:07 <lbragstad> unified limits
18:06:09 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/unified-limits
18:06:14 <lbragstad> and system scope
18:06:16 <lbragstad> #link https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bp/system-scope
18:06:31 <lbragstad> there are a bunch of things to review there, so there's no shortage of reviews
18:06:49 <lbragstad> any questions or concerns regarding what we have in flight?
18:07:49 <lbragstad> cool - moving on then
18:07:56 <lbragstad> #topic core adjustments
18:08:19 <lbragstad> our current core team isn't really reflective of the current state of things
18:08:43 <lbragstad> i've been in touch with a few members and the decision was mutual to remove a couple
18:09:08 <kmalloc> be sure to send the removals to the ML as well :)
18:09:15 <lbragstad> today I'll be removing stevemar and bknudson from keystone core
18:09:21 <lbragstad> and btopol from keystone specs core
18:09:28 <lbragstad> kmalloc: ++
18:10:17 <lbragstad> i don't think any of them are hanging around in IRC, but i personally appreciate everything they've done for the project
18:10:27 <lbragstad> and it'll suck to see them go
18:10:34 <lbragstad> but on a lighter note
18:10:50 <lbragstad> I'm happy to announce that we'll be adding gagehugo to keystone-core!
18:11:02 <cmurphy> yay gagehugo \o/
18:11:05 <lamt`> grats
18:11:09 <jessegler> yay \o/
18:11:29 <gagehugo> thanks everyone
18:11:41 <lbragstad> he's been doing some great work and picking up a lot of slack to keep us moving forward
18:11:58 <lbragstad> which is awesome, thanks for all your hard work gagehugo
18:12:03 <knikolla> congrats! keep up the good work \o/
18:12:16 <gagehugo> happy to help :)
18:12:34 <lbragstad> i'm glad we're making this change prior to the release going into RC
18:12:44 <lbragstad> i think it will make a difference and help with the review queue
18:13:19 <lbragstad> alright - moving on
18:13:26 <lbragstad> #topic Rocky community goals
18:13:40 <lbragstad> just a heads up in case anyone hasn't seen it yet
18:13:46 <lbragstad> #link https://review.openstack.org/#/c/532627/1
18:14:05 <lbragstad> but that's a proposal for one of the community goals - which will impact keystone given our complex history with pagination
18:14:35 <kmalloc> and i'll still -2 pagination changes that require it in keystone
18:14:37 <kmalloc> ftr
18:14:48 <kmalloc> for things that hit ldap things.
18:14:50 <lbragstad> kmalloc: you'll need to weigh in on the community goal
18:15:02 <lbragstad> i highlight it a bit in a comment, but more context would be good
18:15:14 <lbragstad> (i also know we have the pagination discussion scattered everywhere)
18:15:22 <kmalloc> i'm going to let you proxy our concerns to a consistent voice.
18:15:33 <kmalloc> but basically users cannot be paginated.
18:15:45 <kmalloc> and ... i think roles.
18:15:51 <lbragstad> anything that has to read from ldap can't be paginated...
18:15:54 <kmalloc> yep
18:16:35 <kmalloc> otherwise i will say pagination is a terrible design and never works like people expect, but users / ldap backed data is a hard -2, regardless of community goals
18:17:05 <kmalloc> everything else, if someone signs up to write it, i'll at least review it and not -1/-2 because i dislike the design
18:17:08 <lbragstad> i was thinking about this a bit, but we wouldn't be able to work around that technical requirement while keeping keystone stateless
18:17:16 <kmalloc> pretty much.
18:17:41 <lbragstad> ok - i can follow up with mordred about that
18:17:50 <kmalloc> keystone always has "special" cases
18:17:58 <kmalloc> it is part of just how we're designed
18:18:09 <kmalloc> also... pagination is not guaranteed consistent even with SQL because we are stateless
18:18:16 <kmalloc> which is why the design is so bad. but, whatever
18:18:36 <lbragstad> ... so won't that affect every project that uses SQL?
18:18:39 <kmalloc> as long as we communicate "pagination doesn't work like you think" in docs for the things not ldap backed, we're good.
18:18:41 <lbragstad> and not just keystone?
18:18:42 <kmalloc> it will.
18:19:05 <kmalloc> i always use the "how many pages deep in google do you look when looking at items"
18:19:09 <kmalloc> and no one goes beyond page 2
18:19:16 <kmalloc> this is why pagination doesn't work.
18:19:17 <cmurphy> kmalloc: i think you should bring this up on the goal review
18:19:19 <kmalloc> search/filter.
18:19:33 <gagehugo> things get weird after page 2
18:19:38 <lbragstad> i've heard people argue the exact opposite
18:19:51 <kmalloc> but like i said, ldap is the only bit that is a hard -2 and that is because of active cursors and non-deterministic ordering
18:19:58 <lbragstad> that they can't write filters specific enough to get what they want out of the first page
18:20:10 <gagehugo> cmurphy ++
18:20:17 <kmalloc> the rest, i don't have the energy to argue the rest and we should just communicate it in docs
18:20:28 <kmalloc> and expect people to complain it's bad :P
18:20:35 <kmalloc> we already mostly lost the argument.
18:21:38 <lbragstad> what would be required to make it so sql always returns the same result?
18:21:47 <lbragstad> database migrations?
18:21:48 <kmalloc> something MySQL doesn't do well.
18:21:50 <kmalloc> views.
18:22:17 <kmalloc> most things you end up doing is creating a temp view of the data for referencing for a series of requests.
18:22:23 <kmalloc> would eb the oracle method
18:23:05 <kmalloc> then you can have a consistent response for a set of data referenced in the request vs a list that changes behind the scenes if say a project is deletec and shifts the elements on a page
18:23:16 <kmalloc> over a refresh
18:24:01 <kmalloc> so, basically not an argument worth having.
18:24:13 <kmalloc> but LDAP cannot support pagination for keystone
18:24:43 <lbragstad> hmm
18:24:50 <lbragstad> yeah - we need to voice this on the goal
18:25:23 <lbragstad> i can follow up afterwords and do that
18:25:29 <kmalloc> the only thing i recommend commenting on the goal is the user bit.
18:25:40 <lbragstad> that's the hard requirement
18:25:42 <kmalloc> yep.
18:25:57 <lbragstad> because in order to do that, keystone would have to be stateless
18:26:13 <ayoung> Lets make Keystone stateless!
18:26:14 <kmalloc> stateful and communicate cursors across multiple projects
18:26:20 <cmurphy> kmalloc: I really think it would be worth it to comment on the goal with all your concerns, saying it's not an argument worth having means that the argument is never heard
18:26:25 <kmalloc> s/projects/processes
18:26:42 <kmalloc> cmurphy: i've had the argument multip[le times. i am voicing history here
18:26:56 <kmalloc> i really don't want to start down that path again. I want to make sure LDAP backed data is carved out
18:27:19 <lbragstad> well - we will need to voice the concern in the goal before it is accepted
18:27:30 <lbragstad> even if it is, we'll probably have to apply for an exception of some sort
18:28:14 <ayoung> Lets deprecate LDAP, spin it off to its own IdP thing
18:28:48 <cmurphy> ayoung: go for it, i'll review :)
18:29:02 <kmalloc> sure. i'm not signing up to write that code though
18:29:05 <ayoung> cmurphy, code is the easy part
18:31:19 <lbragstad> kmalloc: so are you going to add your concerns to the goal review?
18:31:37 <kmalloc> i can add the ldap bit, but thats really as far as i will go.
18:31:50 <lbragstad> that's fine - thanks for doing that
18:32:08 <ayoung> there was a bug filed recently about LDAP user-list being slow.  I really want to find a way to tell people "you really don't want to do that"
18:32:24 <ayoung> recently by my definition is post Folsom
18:32:43 <lbragstad> that sounds like a PTG topic :)
18:34:00 <lbragstad> #topic open discussion
18:34:08 <lbragstad> floor is open!
18:34:27 <cmurphy> oh i was wondering about the ansible upgrade jobs we have
18:34:35 <cmurphy> i don't think i've ever seen them passing
18:34:40 <cmurphy> who owns those?
18:34:47 <gagehugo> yeah they pass like 1/100 times it seems
18:34:54 <cmurphy> lol not useful
18:34:59 <lbragstad> those were put in place by the openstack ansible folks
18:35:15 <lbragstad> but i think some changes happened around the infra changes that broke some of them
18:35:18 <kmalloc> commented
18:35:29 <lbragstad> cc evrardjp ^
18:35:49 <lbragstad> i can't remember if it was around the time we moved to Zuul or before that
18:35:52 <lbragstad> kmalloc: thank you
18:36:01 <kmalloc> ayoung: yeah "don't list users with LDAP... no really" but i think we should add a proper search API for users
18:36:02 <cmurphy> i feel like it was right before
18:36:10 <kmalloc> rather than telling people to "list and filter"
18:36:17 <kmalloc> it can be a list/filter behind the scenes
18:36:32 <kmalloc> anyway... /me goes back to being the grumpy old core.
18:36:54 <clarkb> lbragstad: it was before we moved to zuul
18:37:01 <clarkb> er moved to zuulv3
18:37:12 <ayoung> kmalloc, the fact that it is a filter should be exoposed to the user, and the assumption of stateful pagination be explicitly expunged,.
18:37:16 <lbragstad> yeah - it was solid at one point
18:37:25 <kmalloc> ayoung: ++ YES.
18:37:31 <lbragstad> but i think something happened there and I want to say it was around that time
18:37:41 <lbragstad> (the time of switching over to zuul)
18:37:52 <gagehugo> yeah it was around zuulv3 time
18:37:52 <ayoung> @cmurphy, link to a common failing example that we can all look at?
18:38:04 <cmurphy> ayoung: sure let me find one
18:38:49 <cmurphy> http://logs.openstack.org/27/524927/21/check/openstack-ansible-keystone-rolling-upgrade/7f25e2b/
18:39:53 <ayoung> task path: /home/zuul/.ansible/roles/os_previous_keystone/tasks/keystone_fernet_keys_create.yml:21
18:40:05 <ayoung> __init__() got an unexpected keyword argument 'scope_types'"
18:40:11 <ayoung> I think that is a lbragstad issue to tackle
18:40:22 <lbragstad> oslo.policy needs to be updated
18:41:49 <lbragstad> it looks like they are testing master with older requirements
18:42:37 <cmurphy> okay it sounds like it's easy to fix, i was wondering if there was a go-to person attending to them
18:42:59 <lbragstad> odyssey4me, evrardjp, or cloudnull
18:43:06 <cmurphy> i think that job was the only thing between us and the rolling-upgrade tag?
18:43:07 <lbragstad> or andymccr
18:43:17 <lbragstad> yeah - with one other stipulation
18:43:30 <lbragstad> which was how the patches from review were applied to the environment
18:44:19 <lbragstad> if patch A is proposed to master and has a Depends-On on patch B which is proposed to a stable branch, and both contain migrations of some sort, then it's possible to brick the migrations because patch B won't be installed in the setup
18:44:26 <lbragstad> of the test environment by OSA
18:44:28 <ayoung> that looks like it is a CLI param for keystone-manage.  Did we drop something?
18:47:59 <gagehugo> possibly?
18:48:11 * lbragstad_ is having internet connection trouble
18:48:40 <mordred> kmalloc, lbragstad_: sorry I missed it earlier - but the pagination goal is "if a collection is paginated, pagination links should be returned"
18:49:11 <mordred> kmalloc, lbragstad_: definitely not "add pagination to all resources" or anything
18:49:20 <gagehugo> ah
18:49:26 <lbragstad_> mordred: ack - that helps
18:49:48 <lbragstad_> sorry if i jumped to assumptions there
18:49:50 <mordred> in which case - it might very well be a no-op :)
18:49:59 <mordred> no - it's a good question for sure!
18:50:14 <kmalloc> mordred: ++
18:50:15 <kmalloc> ok
18:50:35 <kmalloc> mordred: i was worried because we just can't paginate users w/o massive headaches
18:50:47 <kmalloc> i'll recind my -1 and say you updated us
18:51:00 <ayoung> mordred, LDAP and pagination has been a pain point since the mid 1990s
18:51:49 <mordred> ++
18:52:33 <mordred> I'm actually not a big fan of pagination for REST calls across the board - it makes things harder - but I also can somewhat understand the humans who desire such a thing
18:52:58 <mordred> but pagination without pagination links is just the worst
18:54:00 <lbragstad> cool - that clears things up then
18:54:20 <lbragstad> anything else we want to go over before office hours starts in 6 minutes?
18:55:22 <lbragstad> cmurphy: i think i dropped in the middle of the osa rolling upgrade discussion
18:55:54 <cmurphy> lbragstad: oh i don't think you missed much
18:55:58 <cmurphy> if anything
18:56:04 <lbragstad> ok
18:56:39 <cmurphy> after feature freeze i can try to work on them and/or reach out to the osa people about it
18:56:45 <lbragstad> yeah
18:57:04 <lbragstad> they are usually super responsive to those things
18:57:23 <lbragstad> anything else?
18:57:54 <lbragstad> cool - thanks for coming
18:57:57 <lbragstad> #endmeeting