18:57:46 <lbragstad> #startmeeting keystone-office-hours
18:57:47 <openstack> Meeting started Tue Aug 15 18:57:46 2017 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:57:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:57:50 <openstack> The meeting name has been set to 'keystone_office_hours'
18:57:57 <cmurphy> edmondsw: you +1'd it, you're not aware of a use case for a non-sql resource backend?
18:58:00 <kmalloc> #topic lbragstad is cool.
18:58:03 <kmalloc> >.>
18:58:12 <lbragstad> woo
18:58:15 <edmondsw> cmurphy I am not... we use sql in PowerVC
18:58:31 <kmalloc> the resource backend is so highly keystone/openstack specific it is unlikely to be replaced
18:58:33 <kmalloc> imo
18:58:47 <kmalloc> HPE did it sortof, but we put everything in mongo
18:58:59 <kmalloc> which was when there were far less backend options
18:59:07 <edmondsw> kmalloc why?
18:59:18 <kmalloc> because someone decided mongodb was the best choice
18:59:25 <kmalloc> i really don't know, it pre-dated me working at HPE
18:59:37 <kmalloc> this was for the public cloud.
18:59:53 <edmondsw> just seems silly :)
18:59:57 <cmurphy> i could see projects being mapped to like github org memberships or some other corporate resource designator
18:59:59 <kmalloc> and all the resource data was merged into assignment, so it was an all-or-nothing deal for resources
19:00:00 <samueldmq> kmalloc: cmurphy I also don't understand why we aren't following the deprecation process before removing the capability of setting an non-sql backend for that
19:00:14 <cmurphy> samueldmq: well because it doesn't work
19:00:20 <kmalloc> at all
19:00:20 <cmurphy> already
19:00:32 <kmalloc> it's broken as it sits if you use identity SQL at all
19:00:48 <cmurphy> if we'd figured it out ahead of time we could have done a normal deprecation
19:01:11 <kmalloc> i'll plan to implement a test that explodes if there are FKs across subsystems
19:01:19 <kmalloc> but i need to figure out the way to do that in queens
19:01:23 <kmalloc> not in pike-rc
19:01:34 <kmalloc> it's ... not going to be a fun test, but it'll prevent this in the future
19:01:59 <kmalloc> and i want to do a final collapse/kill of the old migration_repo
19:02:03 <lbragstad> kmalloc: well - your patch is allowing FKs between subsystems
19:02:20 <kmalloc> lbragstad: the new test would prevent future cases. resource would be explicitly allowed
19:02:33 <kmalloc> or we allow FKs between systems where possible
19:02:37 <kmalloc> *shrug*
19:02:40 <samueldmq> seems interesting
19:02:48 <samueldmq> kmalloc: btw I left a comment (2) in https://review.openstack.org/#/c/493621
19:02:59 <samueldmq> I am confused with a comment you elft tehre
19:03:17 <lbragstad> the more i think about this the more i want to dedicate time at the ptg to discuss where we allow FKs and where we don't
19:03:24 <lbragstad> but those bits kind of confuse me
19:04:01 <kmalloc> samueldmq: replied
19:04:39 <kmalloc> lbragstad: basically it is simple: if you can configure non-SQL, you cannot have an FK from one sql system to it (meaning cannot go either way)
19:04:59 <kmalloc> if you allow FKs, they will break inserts if the data isn't housed in the same DB
19:05:43 <kmalloc> you cannot insert a user into User table now unless the domain the user belongs to is in the resource table
19:05:57 <samueldmq> kmalloc: well, re-replied, I don't really agree. but I think that might be a separate conversation
19:06:29 <kmalloc> samueldmq: the only reason we ever had a manager was for abstraction/pivot for backends. if there is no pivot, the manager and driver could be collapsed
19:06:48 <samueldmq> kmalloc: the docs says different, https://github.com/openstack/keystone/blob/master/keystone/common/manager.py#L153-L162
19:06:54 <kmalloc> business logic between backends goes in the manager, but if the manager doesn't need to pass data through, i can avoid
19:07:00 <samueldmq> we don't document the pivot for backends as being the only reason
19:07:12 <kmalloc> https://github.com/openstack/keystone/blob/master/keystone/common/manager.py#L160
19:07:25 <kmalloc> if there is no dynamic backend, it doesn't need to pass data through
19:08:16 <samueldmq> well, resource manager is pretty complex already with all the is-domain thing and all (create_project is 40 LOC)
19:08:27 <samueldmq> adding the specific bits to store that into SQL would make it worse
19:08:34 <lbragstad> hrybacki: knikolla and i had a discussion yesterday about preparing a demo for the ptg on global roles
19:08:34 <kmalloc> a lot of that could be simplified if it didn't need to pass to the backend
19:08:52 <lbragstad> hrybacki: notes from that meeting are available here - https://etherpad.openstack.org/p/keystone-global-roles-poc
19:09:08 <kmalloc> samueldmq: a lot of the manager code could lean directly on SQL to do the work.
19:09:22 <samueldmq> kmalloc: ok, it's worth it a try to see if we can save maintainer time
19:09:40 <kmalloc> samueldmq: that is my thought
19:09:44 <samueldmq> I guess that's your point too
19:09:49 * samueldmq nods
19:10:40 <hrybacki> lbragstad: knikolla ack, looking
19:11:18 <samueldmq> kmalloc: test_sql_upgrade.py gets executed against real sql, correct?
19:11:24 <samueldmq> I mean, not sqlite
19:11:45 <hrybacki> decent amount of work before PTG
19:12:09 <lbragstad> hrybacki: yeah
19:12:37 <lbragstad> hrybacki: i'm going to start working on item #1 this week - since i proposed the original patch/implementatin
19:12:41 <hrybacki> but def. worthwhile. I see you have already started
19:12:45 <lbragstad> implementation*
19:16:24 <hrybacki> lbragstad: presently putting out a fire but will re-read the spec and look over the task breakdown so I can figure out how I may best help
19:19:53 <lbragstad> hrybacki: ack
19:20:24 <kmalloc> samueldmq: both
19:21:06 <samueldmq> kmalloc: kk
19:41:11 <knikolla> lbragstad: https://review.openstack.org/#/c/494004/
19:41:36 <knikolla> ^^ removes the uwsgi job
19:41:44 <lbragstad> knikolla: nice - thanks for proposing that
19:43:44 <knikolla> lbragstad: fastest patch i've ever gotten merged.
19:43:55 <lbragstad> :)
19:48:03 <samueldmq> kmalloc: lbragstad I have one more question in https://review.openstack.org/#/c/493259
19:48:25 <samueldmq> inline
19:50:00 <lbragstad> i think that comment is referencing https://review.openstack.org/#/c/493259/10/keystone/common/sql/contract_repo/versions/024_contract_create_created_at_int_columns.py
19:50:47 <gagehugo> lbragstad speaking of the gate, did we ever decide about skipping functional tests for doc changes? https://review.openstack.org/#/c/492630/
19:51:14 <lbragstad> samueldmq: i asked in an earlier patch set - https://review.openstack.org/#/c/493259/7/keystone/identity/backends/sql_model.py
19:51:36 <lbragstad> gagehugo:  i think the consensus was that we should not run functional tests on doc changes to conserve resources
19:51:50 <knikolla> gagehugo: ++
19:52:03 <knikolla> i see jobs being queued a lot these days
19:52:19 <gagehugo> it looks like other projects do have something to skip tests for just docs/api-ref/non-code
19:52:56 <lbragstad> yeah - line 1481
19:53:01 <lbragstad> https://review.openstack.org/#/c/492630/4/zuul/layout.yaml
19:53:04 <lbragstad> that's neat
19:53:47 <lbragstad> and line 1479
19:55:38 <gagehugo> yeah
19:57:02 <gagehugo> if thats ok then I can take a look at proposing something
19:59:53 <samueldmq> lbragstad:  for https://review.openstack.org/#/c/493621/
19:59:59 <samueldmq> we wait a bit on feedback from the ML topic?
20:00:30 <samueldmq> fyi I just approved kmalloc's change fixing bug 1702211
20:00:31 <openstack> bug 1702211 in OpenStack Identity (keystone) "test_password_history_not_enforced_in_admin_reset failed in tempest test" [Medium,In progress] https://launchpad.net/bugs/1702211 - Assigned to Lance Bragstad (lbragstad)
20:00:45 <kmalloc> ok
20:00:50 <kmalloc> i'll wait for it to merge
20:01:02 <kmalloc> lbragstad: want me to push the stable/pike version once master merges?
20:01:10 <lbragstad> kmalloc: yes please
20:01:16 <kmalloc> ok done
20:01:20 <lbragstad> kmalloc: thanks
20:01:45 <lbragstad> samueldmq: yeah - i'd like to have operator feedback before merging it but i also realize that might be unlikely
20:02:13 <kmalloc> lbragstad: looks like you have +2 on stable/pike
20:02:19 <lbragstad> i saw that :)
20:02:21 <kmalloc> lbragstad: so you can +2 the stable/pike of changed_at
20:02:30 <samueldmq> lbragstad: well maybe just wait a bit until we're sure the gate has been 100% fixed?
20:02:49 <lbragstad> samueldmq: the change that is gating now should fix the gate
20:03:09 <kmalloc> samueldmq: ^ folks have run 300+ iterations locally
20:03:11 <samueldmq> lbragstad: exactly, I fully expect it too.
20:03:18 <kmalloc> and prior, it was hitting ~60-70
20:03:21 <kmalloc> without a failure
20:03:26 <lbragstad> cmurphy: ran it up to 2500 without a failure
20:03:32 <kmalloc> i stand corrected
20:03:35 <kmalloc> 2000+
20:03:36 <kmalloc> :)
20:03:49 <kmalloc> but... was it OVER 9000!?!
20:03:50 <lbragstad> i ran it and walked away from my computer saturday and it ran 3500+
20:03:52 <samueldmq> yes, no reason for the gate to be unhappy!
20:03:53 * kmalloc ducks.
20:03:53 <lbragstad> almost
20:04:32 <lbragstad> also - that means there were thousands of revocation events stored in sql
20:04:43 <lbragstad> and keystone will still running way better than it did in the past
20:04:54 <samueldmq> lbragstad: ++
20:05:04 <lbragstad> the database improvement with the revocation table work well
20:05:15 <samueldmq> true
20:05:42 <samueldmq> also, well done with finding and fixing that bug quickly kmalloc and lbragstad
20:05:44 <samueldmq> \o/
20:05:54 <lbragstad> that was all kmalloc
20:06:54 <samueldmq> bugs like that in RC phase are always interesting
20:14:15 <kmalloc> we still need to fix the invalidation race
20:14:16 <kmalloc> but...
20:14:20 <kmalloc> that is much smaller
20:16:17 <samueldmq> kmalloc: what is that invalidation race thing ?
20:21:02 <kmalloc> samueldmq: we do an update after invalidating the cache
20:21:11 <kmalloc> ... always invalidate *after* update
20:21:27 <samueldmq> kmalloc: where is that ? update of what?
20:21:32 <samueldmq> should be a quick fix, right?
20:21:43 <kmalloc> user data i think. i found it in the password change hunt
20:21:49 <kmalloc> for the created_at_int thing
20:22:00 <samueldmq> :(
20:22:10 <kmalloc> sec, i can find it again in a moment
20:22:17 <samueldmq> kmalloc: would be good to get in for rc too, right?
20:22:18 <samueldmq> kk
20:22:51 <kmalloc> https://github.com/openstack/keystone/blob/2164d0550c39a07b9c0dacc6c2167d0018b0c7bd/keystone/identity/core.py#L1092-L1099
20:23:02 <kmalloc> it is always good for RC, but it isn't a blocker
20:23:05 <kmalloc> it can land in queens
20:23:11 <kmalloc> i don't think that has *ever* caused a real issue
20:24:09 <samueldmq> kmalloc: ah got it
20:24:18 <samueldmq> like another process getting in in the meantime
20:24:53 <samueldmq> and re-setting the thing to the old entity before it gets effectively updated
20:25:04 <samueldmq> that's not easy to happen, I agree
20:27:05 <kmalloc> a read on another keystone process for the user will cache the wrong data
20:27:09 <kmalloc> it's not even that theoretical
20:27:18 <kmalloc> thankfully, it's not superduper important.
20:27:25 <kmalloc> because cache expires pretty fast
20:33:35 <lbragstad> well - our bug queue is *almost* back under 100 open bugs
20:34:45 <lbragstad> considering it was up to the 130s at various times throughout the release
20:36:07 <kmalloc> did you count ksa, ksc, ksm too?
20:36:09 <kmalloc> or just server?
20:38:54 <lbragstad> just server
20:51:16 <cmurphy> anything else burning that we should look right now?
20:56:50 <knikolla> lbragstad: might want to get this in too in pike? https://review.openstack.org/#/c/492694/
20:57:44 <lbragstad> knikolla: yeah - that wouldn't be a bad one to include
20:58:42 <knikolla> thanks cmurphy
20:59:22 <lbragstad> knikolla: wooo - look at that shiny +2
20:59:52 <knikolla> lbragstad: \o/
21:13:41 <gagehugo> lbragstad knikolla https://review.openstack.org/#/c/494018/
21:25:10 <gagehugo> knikolla I think we could include coverage in the skip?
21:25:30 <gagehugo> It looks like everything it covers is in keystone/* excluding tests
21:25:37 <gagehugo> so for docs that should be fine
21:26:04 <knikolla> gagehugo: yep. i think so
21:27:04 <gagehugo> oh that test generates a 14M html file
21:27:40 <knikolla> gagehugo: coverage?
21:27:56 <gagehugo> nah the regex checking in that change ^
21:28:06 <gagehugo> I'll add coverage and fix that
21:28:17 <knikolla> yep. it was loading for a while
21:29:07 <knikolla> Regex ^gate-(keystone|tempest|grenade|keystoneclient)-dvsm-.*$ has no matches in job list
21:33:50 <openstackgerrit> Eric Fried proposed openstack/keystoneauth master: Adapter.get_conf_options(deprecated_opts)  https://review.openstack.org/490895
21:38:30 <itlinux> hello all.. is there a way to do ldap nested groups? TY
21:41:44 <gagehugo> knikolla fixed
21:50:15 <cmurphy> efried: thanks
21:50:31 <efried> cmurphy Thank you :)
21:57:05 <mjax> Hey guys, I'm trying to set up domain specific identity driver configurations on keystone, but am getting this error: Failed to load 'keystone.identity.backends.Athens.Identity' using stevedore: No 'keystone.identity' driver found, looking for 'keystone.identity.backends.Athens.Identity' load_driver /opt/stack/keystone/keystone/common/manager.py:76
21:57:21 <mjax> anyone familiar with domain specific config setup?
21:59:17 <lbragstad> mjax: looks like you need to configure the entry point with stevedore
21:59:31 <lbragstad> which is typically done in setup.cfg
22:00:03 <lbragstad> #endmeeting