16:01:07 <kmalloc> #startmeeting keystone
16:01:08 <openstack> Meeting started Tue Aug 27 16:01:07 2019 UTC and is due to finish in 60 minutes.  The chair is kmalloc. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:01:09 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:01:11 <openstack> The meeting name has been set to 'keystone'
16:01:16 <gagehugo> o/
16:01:42 <bnemec> o/
16:01:50 <kmalloc> i'll give a couple minutes for people to arrive.
16:02:02 <kmalloc> it is probably going to be a short meeting.
16:02:05 <vishakha> o/
16:02:10 <kmalloc> or short on planned agenda at least
16:02:36 <lbragstad> o/
16:03:28 <knikolla> o/
16:07:02 <kmalloc> ok
16:07:05 <kmalloc> lets get started
16:07:40 <kmalloc> #topic Request for Review
16:07:57 <kmalloc> KSM v2 removals
16:08:06 <kmalloc> gagehugo: o/
16:08:10 <gagehugo> o/ that's me
16:08:20 <kmalloc> #link https://review.opendev.org/#/c/678386/
16:08:27 <kmalloc> #link https://review.opendev.org/#/c/669706/
16:08:35 <gagehugo> those two for now
16:08:44 <kmalloc> the other links?
16:08:54 <kmalloc> https://review.opendev.org/#/c/677585/ and  https://review.opendev.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bug/1805409 ?
16:08:56 <gagehugo> I mean for ksmv2
16:09:04 <gagehugo> idk about the other links
16:09:08 <vishakha> 0/
16:09:40 <kmalloc> in short we're working to remove the v2 support from keystonemiddleware, gagehugo has been working on getting things passing
16:09:46 <kmalloc> and looks like vishakha is too :)
16:10:05 <vishakha> link https://review.opendev.org/#/c/677585/ , this is colleen's patch for client support app creds. open for review
16:10:13 <kmalloc> vishakha: ahhh
16:10:18 <kmalloc> there we go
16:10:35 <vishakha> if its good to go we can propose build of keystomeclient  so that it is available for openstackclient
16:10:54 <kmalloc> anyway, keystonemiddleware dropping v2 support will ease up the codebase for further cleanup
16:10:58 <kmalloc> ask gagehugo questions
16:11:13 <kmalloc> #link https://review.opendev.org/#/c/677585/
16:11:34 <kmalloc> needs eyes, it's app-cred related
16:11:50 <vishakha> thanks kmalloc
16:11:52 <kmalloc> while cmurphy is on vacation, please feel free to review/comment/approve, etc
16:12:00 <gagehugo> vishakha thanks
16:12:04 <kmalloc> if you have questions both vishakha and I are available to help :)
16:12:20 <kmalloc> and finally
16:12:23 <kmalloc> #link https://review.opendev.org/#/q/status:open+project:openstack/keystone+branch:master+topic:bug/1805409
16:12:44 <kmalloc> lots of policy related work, help to get them passing gate and get these landed would be appreciated
16:12:47 <vishakha> this is few left in policy API in system scopes
16:12:59 * kmalloc nods.
16:13:28 <kmalloc> vishakha: want to continue with the other changes you have in flight?
16:13:41 * kmalloc noticed more review requests by vishakha on the agenda :)
16:13:59 <vishakha> Thats all for policy and association API
16:14:21 <kmalloc> but there is also oauthlib and pdf docs.
16:14:21 <knikolla> i did review a whole bunch yesterday, will be happy to keep reviewing
16:14:43 <vishakha> thanks kmalloc knikolla
16:14:55 <kmalloc> yay knikolla reviews!
16:15:45 <vishakha> for oauthlib I can many test cases failing but I am still digging the reason of failure
16:15:56 * kmalloc nods.
16:16:15 <vishakha> Help will be appreciated :)
16:16:21 <kmalloc> looks like an unfortunate case of the lib changed pretty drastically in weird ways.
16:16:28 <kmalloc> s/weird/subtle
16:17:21 <vishakha> Also for the pdf docs we came across with the errors none of the other project ran into. Also asked the docs team to look inti
16:17:30 * kmalloc nods.
16:17:30 <vishakha> s/inti/into
16:17:47 <kmalloc> that is a bit weird of an error
16:18:14 <vishakha> kmalloc: yes it is
16:20:20 <vishakha> that's it from my side
16:20:29 <kmalloc> ok i think we're on for lbragstad
16:20:32 <kmalloc> lbragstad: o/
16:20:38 <lbragstad> o;
16:20:39 <lbragstad> o/
16:21:01 <gyee> vishakha, what's the weird error. Out of memory?
16:21:01 <lbragstad> ok - so cmurphy|afk kmalloc and i were looking at the test run times for unit tests
16:21:30 <gyee> I know others had hit that error recently, especially with large docs
16:21:41 <lbragstad> and the obvious culprit for increased run times is all the protection testing we've been doing
16:22:05 <kmalloc> gyee: matrix file missing.
16:22:14 <lbragstad> i wanted to throw this on the agenda so that we could discuss additional ideas for how to keep testing policy but in ways that don't take forever or spool out of control
16:22:16 <vishakha> gyee:  https://zuul.opendev.org/t/openstack/build/ec6d51585a3c4b31b42aebe928c73e35/log/job-output.txt#5395
16:22:34 <gyee> k, that's different one then
16:23:17 <lbragstad> initially - we were thinking that reusing some of the common resources would help out - and it certainly would
16:23:30 <lbragstad> but we're ultimately making a lot of API requests, period
16:23:47 <kmalloc> i'm sure that re-using fernet / cred repos would help a lot
16:24:03 <kmalloc> that is both random/urandom and filesystem access
16:24:06 <lbragstad> and some of the bootstrap resources - but the testresources work has kinda hit a wall
16:24:27 <lbragstad> (i'm also not convinced its possible to do what we want with testresources)
16:24:38 <lbragstad> integrating that into our tests has been painful
16:25:02 <lbragstad> but that's the recommended approach over using native structures like setUpClass or setUpModule
16:25:58 <lbragstad> i'm wondering how people feel about graduating the protection tests to formal functional tests...
16:26:15 <kmalloc> i'd be ok with that
16:26:25 <gagehugo> same
16:26:31 <kmalloc> however, i don't think it's going to reduce local testing load
16:26:34 <lbragstad> i haven't planned things out in detail yet
16:26:42 <kmalloc> because we're going to need to run these locally as well as unit tests
16:27:02 <lbragstad> yeah - we'd definitely keep gating on them
16:27:04 <kmalloc> we might be able to at least isolate the protection tests into testresources
16:27:15 <kmalloc> esp. if we're putting them into their own test-run
16:27:17 <lbragstad> but they wouldn't be invoked by ``tox -re py37``
16:27:42 <kmalloc> because we have less concern about API correctness and more of policy behavior
16:27:43 <lbragstad> instead, we could exercise that suite using something like ``tox -e functional`` or ``tox -e protection``
16:27:52 <gyee> we can't mock the target part?
16:28:18 <kmalloc> i'd opt for -e protection, and i would opt for trying to use testresources or similar in the protection-only tests
16:28:32 <kmalloc> we can re-merge them to -epy37 if we want later or keep them isolated
16:28:32 <lbragstad> sure - i think we should still vet that
16:28:45 <kmalloc> we are going to have to rebuild a chunk of them in either case.
16:28:57 <kmalloc> since -eprotection isn't going to be fast
16:29:05 <lbragstad> but this would allow us to slim down the unit test times while we figure out what the deal is with testresources
16:29:17 <kmalloc> but it's not trying to retrofit our entire test suite
16:29:23 <kmalloc> in one go.
16:29:51 <kmalloc> keep in mind it will only add concurrency to gating, it wont, in fact fix test runtimes overall or resource consumption
16:30:08 <kmalloc> unless we also do testresources or more streamlined setup work
16:30:16 <kmalloc> somehow
16:30:32 <lbragstad> at this point - i'd be happy if we could get testresources to handle bootstrap
16:30:36 <lbragstad> and share those resources
16:30:54 <lbragstad> if we can get that done, i'd say we can take the next step and look at other optimizations in how we set things up
16:31:24 <lbragstad> or we look at that now if testresources doesn't actually work for what we're trying to do with it
16:31:25 <kmalloc> ++
16:31:54 * lbragstad thought it was going to be more straight forward to integrate that into keystone's protection tests
16:32:21 <kmalloc> we might need to really just isolate and build a new core test structure up that does testresources from the get go
16:32:40 <kmalloc> it's a big chunk of work, but might be easier than retrofitting it into the current mess
16:32:43 <kmalloc> and history
16:32:47 <lbragstad> yeah...
16:32:58 <lbragstad> kinda damned if you do and damned if you don't...
16:33:04 <kmalloc> yup
16:33:06 <kmalloc> *shrug*
16:33:09 <lbragstad> either way it's going to be a huge refactor
16:33:13 <lbragstad> unfortunately
16:34:02 <kmalloc> anything else?
16:34:13 <lbragstad> do we have general consensus on the approach?
16:34:19 <lbragstad> does anyone see red flags with this?
16:34:23 <kmalloc> anyone want to volunteer (gyee ?) to toss some help over to lbragstad ?
16:34:38 <kmalloc> i don't see any red flags besides "it's a lot of work"
16:34:42 <kmalloc> and that is expected
16:34:45 <lbragstad> again - i'd just be happy to get things isolated this release
16:35:13 <lbragstad> we can have another gate job that exercises protection testing - running it in parallel with our existing unit tests
16:35:19 <lbragstad> (which might help gate run times)
16:35:51 <kmalloc> which is at least a start
16:35:51 <lbragstad> but we're also not forcing people to testing thousands of protection tests if they're not modifying the api layer or policies directly
16:35:51 <gyee> kmalloc, sure, yeah, it'll be a lot of work
16:36:06 <kmalloc> ok
16:36:18 <kmalloc> moving on, resource options (this is me/cmurphy)
16:36:28 <lbragstad> thanks kmalloc and gyee
16:36:36 <kmalloc> some code is posted to add resource options to roles and projects (+domains)
16:36:45 <kmalloc> #link https://review.opendev.org/#/c/678322/
16:36:48 <kmalloc> core addition of code
16:36:54 <kmalloc> #link https://review.opendev.org/#/c/666739/
16:36:58 <kmalloc> immutable option
16:37:06 <kmalloc> #link https://review.opendev.org/#/c/678379/
16:37:11 <kmalloc> tempest change to disable a test
16:37:18 <kmalloc> #link https://review.opendev.org/#/c/678380/
16:37:25 <kmalloc> tempest change to re-enable and fix the test
16:37:27 <kmalloc> eyes would be welcome
16:37:32 <kmalloc> some tests are not 100% done
16:37:44 <kmalloc> and i know there are some issues with gate passing, that is related to LDAP and tempest tests
16:37:50 <kmalloc> i should have these fixed shortly
16:37:57 <kmalloc> but, please review general approach
16:38:20 <kmalloc> due to some complexity with the ORM, we opted to just create new tables for project_options and role_options
16:38:47 <kmalloc> a couple key things, an immutable project cannot have tags created/updated/deleted on it as well as cannot be updated or deletedc
16:39:19 <kmalloc> immutable works similar to chattr command, you can change immutable options and other options in one go, but you may not, for example
16:39:28 <kmalloc> change the project name and the immutable option at the same time
16:39:56 <kmalloc> there is not a lot else to say here, feel free to reach out to me if you have questions
16:40:35 <kmalloc> final topic:
16:40:47 <kmalloc> #topic meeting 2019-09-3
16:41:27 <kmalloc> Unless someone has a critical need, I am planning on skipping the meeting (for everyone). The day before is labor day (US Holiday), this will give us a bit more heads-down time to just keep plugging on items
16:41:38 <kmalloc> s/skipping/cancelling
16:41:46 <kmalloc> i wont send the cancel email until friday
16:42:04 <kmalloc> please let me know if you have a critical item for the next meeting between now and friday
16:42:13 <kmalloc> #topic open discussion
16:42:22 <kmalloc> anyone have anything else?
16:42:43 * gagehugo does not
16:43:25 <kmalloc> ok
16:43:29 <kmalloc> #endmeeting