18:01:25 <dstanek> #startmeeting keystone
18:01:26 <openstack> Meeting started Tue Mar  8 18:01:25 2016 UTC and is due to finish in 60 minutes.  The chair is dstanek. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:29 <openstack> The meeting name has been set to 'keystone'
18:01:34 <dstanek> #topic there are no topics - any late topics that need discussion?
18:01:38 <henrynash> dstanek: I tried to add one thing to the agend (but couldn’t save)...
18:01:39 <dstanek> many of us are bug squashing today...
18:01:58 <dstanek> henrynash: what's the topic?
18:02:19 <henrynash> dstanek: just like clarification of the directory structure of unit tests…
18:02:30 <henrynash> dstanek: now that we are moving these all around
18:02:37 <dstanek> #topic just like clarification of the directory structure of unit tests
18:02:42 <dstanek> henrynash: fire away
18:02:49 <bknudson> the test file should be the same as the code file
18:03:06 <henrynash> bknudson: example?
18:03:28 <bknudson> so if the code is in keystone/resource/backends/sql.py then the test is keystone/tests/unit/resource/backends/test_sql.py
18:03:48 <bknudson> then it's easy to find what test file you need to change
18:04:05 <dstanek> bknudson: I agree that we should be moving that way for all unit tests
18:04:27 <henrynash> bknudson: and the test_backends.py file sists in the unit/resource dir in your example
18:04:28 <ayoung> bknudson, so we are not going to push for v2 vs v3 dir under test/unit?
18:04:32 <ayoung> I'm OK with that
18:04:39 <dstanek> i'm ok with small tests to get us there. like we have, i think, a keystone/tests/unit/assignment/test_backends.py or something like that
18:04:51 <ayoung> henrynash, judgement there
18:05:00 <bknudson> there are integration tests and there are component tests
18:05:01 <ayoung> henrynash, I'd say put it in  keystone/tests/unit/resource/backends/
18:05:06 <bknudson> most of our unit tests should be component tests
18:05:09 <ayoung> test_backend should not be named as such
18:05:33 <dstanek> ayoung: the old test_backends.py is now gone
18:05:41 <ayoung> er..wait, which is the actual executable test file?
18:05:55 <ayoung> test_sql, which calls into the common file, whatever that is now, right?
18:05:58 <bknudson> as in, the tests should test a single component and not cross component boundaries
18:06:13 <henrynash> bknudson: agreed
18:06:33 <dstanek> bknudson: ++
18:06:36 <ayoung> http://git.openstack.org/cgit/openstack/keystone/tree/keystone/tests/unit/identity/test_core.py  right?
18:07:17 <ayoung> gonna make it a pain to refactor the fernet-default code, but still worth it.
18:07:23 <henrynash> bknudson, dstanek: but right now we have test_backends.py (split up it their single components tests) NOT in the , say, resource/backend dir…but i the one above
18:07:29 <bknudson> https://review.openstack.org/#/c/289058/ is my first stab at a component test for backends.
18:07:50 <bknudson> well, test_backends is really cross-component tests and not component tests, so it's wrong to begin with
18:07:51 <marekd> hi!
18:08:26 <henrynash> bknduson: even the split up vesions are wroung, iyho?
18:08:29 <bknudson> test_backends is really testing the manager and not the drivers
18:08:42 <henrynash> bknudson: ahh, different (and accurate) issue
18:08:48 <dstanek> henrynash: only in that they do too much in the tests
18:09:33 <ayoung> henrynash, all good?
18:09:38 <bknudson> so what I'd like to see is like https://review.openstack.org/#/c/289058/ where there's tests for the driver interface, and those run on all the drivers
18:09:47 <henrynash> dstanek: and many of the unit/component dirst have test_core.py as well
18:09:54 <henrynash> what’s that mean to do?
18:10:01 <bknudson> and then test_backends becomes test_manager and tests the manager with a canonical backend (sql, I guess)
18:10:39 <ayoung> if the logic is 100% the same between backeds, it goes in core
18:10:59 <ayoung> test_sql etc just the one offs and setup code
18:11:11 <dstanek> henrynash: my ultimate fantasy would be a patter to map from code.py to test_code.py that always works
18:11:12 <amakarov_> We should use functional tests for backends
18:11:14 <bknudson> I should have said test_core not test_manager since the managers live in core.py
18:11:16 <henrynash> ayoung: ahh, so most of test_backends.py whould end up in test_core.py?
18:11:29 <ayoung> I'd expect so, yes
18:11:35 <henrynash> ayoung: right, got it
18:11:40 <henrynash> dstanek: yep, get it
18:12:08 <ayoung> so... let's talk Newton
18:12:15 <dstanek> henrynash: all good?
18:12:34 <shaleh> \o
18:13:07 <henrynash> dstanek: yep thx
18:13:14 <ayoung> dstanek, I didn't add to the agenda, but we really need to discuss post-mitaka plans
18:13:18 <dstanek> #topic {fig} newton
18:13:24 <ayoung> cool
18:13:26 <dstanek> ayoung: sounds good to me
18:13:42 <bknudson> are fig newtons cookies?
18:13:53 <ayoung> OK,  so, between now and the start of N1 opening up, we need to prioritize the basis for the new features we want to land in Newton
18:14:02 <ayoung> If we wait until the Summit, it won;'t happen
18:14:25 <ayoung> this is based on what I have seen in all releases from Diablo on forward
18:14:35 <amakarov_> ayoung: +1
18:14:37 <gyee> ayoung, what's your hit list?
18:14:53 <ayoung> at this point, onus is on the spec owners to keep them up to date, address issues, and get the specs clean
18:15:04 <ayoung> for me, that means dynaimc RBAC policy
18:15:13 <ayoung> https://review.openstack.org/#/c/279379/
18:15:38 <ayoung> there are a lot of details to work out in getting a policy framework that actually supports fine grained delegation
18:15:39 <bknudson> when does N1 open up?
18:15:39 <gyee> whatever happen to common policy scenarios
18:15:42 <shaleh> ayoung: maybe a list of specs should be gathered like we did when Mitaka started
18:15:46 <ayoung> gyee, that is right up there
18:15:54 <ayoung> I'll link
18:16:13 <ayoung> Common policy https://review.openstack.org/#/c/245629/
18:16:19 <gyee> too many chef in the kitchen for that one I guess
18:16:19 <ayoung> #link https://review.openstack.org/#/c/245629/
18:16:20 <samueldmq> hi, sorry I am late
18:16:21 <bknudson> looks like N1 will open up Apr 4-8
18:16:25 <ayoung> #link https://review.openstack.org/#/c/279379/
18:16:38 <ayoung> gyee, well, I think I want to split common policy into two parts
18:16:49 <dstanek> ayoung: yes, we should be getting specs approved and ready to work on too
18:16:57 <ayoung> one is the top level roles, and the second is the fine grained permissions
18:17:05 <shaleh> ayoung: makes sense
18:17:21 <ayoung> also, I need to wake amakarov_ up, but unified delegation needs to start making progress, too, if we want it in
18:17:27 <ayoung> and they really are related
18:17:44 <amakarov_> I want to come up with something working to the end of March
18:18:23 <ayoung> gyee, I think those three, related efforts, are the burning platform.  I'd like them in, if experimental, in Newton, so they can be supported in Ocata long term
18:18:45 <gyee> ayoung, agreed
18:19:07 <shaleh> plus we have the MFA ground work to lay
18:19:24 <ayoung> On the dynamic RBAC policy, we are going to need a way to store the policy in the Database.  I am leaning towards storing it rule by rule as opposed to blob based
18:19:58 <ayoung> henrynash and I have a RBAC talk accepted at this summit. I'd like to be able to lay out the expected future of RBAC at the end of that talk
18:20:40 <ayoung> is there anything else that people see as major features desired for Newton or later?
18:20:58 <shaleh> what is th status of shadow users?
18:21:01 <rderose> shadow ldap users
18:21:09 <rderose> next
18:21:29 <bknudson> it would be great to have an ldap driver that works with python3
18:21:36 <shaleh> bknudson: ++++
18:21:36 <gyee> yeah, stablization, less moving stuff
18:21:37 <dstanek> we want to finish up shadow users, mfa and better DB upgrades this cycle
18:21:53 <henrynash> I’ll be working with ayoung on some of the RBAC stuff, but will also be (re)propsoing the reseller phase 2 stuff
18:21:56 <shaleh> dstanek: *this?
18:22:23 <dstanek> shaleh: N, i'm already in the future man
18:22:26 <browne> bknudson: i wouldn't mind helping with that
18:22:27 <gyee> v2 retirement party
18:22:33 <shaleh> gyee: ++
18:22:45 <ayoung> bknudson, is LDAP Py3 within our control to affect?
18:23:10 <dstanek> ayoung: morgan had ideas on a replacement library
18:23:11 <gyee> versioned endpoints retirement party
18:23:29 <bknudson> ayoung: I guess we could always for ldappool?
18:23:39 <bknudson> or we just start over with a new driver that uses ldap3 driver
18:23:42 <morgan> ldap3 has a pool built in
18:23:45 <tjcocozz_> https://pypi.python.org/pypi/ldap3
18:23:46 <ayoung> so we ahvea plan, just need to execute on it?
18:23:46 <morgan> fwiw
18:24:09 <shaleh> ldap3 always seemed like the logical choice
18:24:14 <morgan> i'd ideally ask we write an isolate read-only ldap3 driver
18:24:21 <morgan> and deprecate all the old ldap code
18:24:27 <morgan> not try and retrofit things in
18:24:36 <morgan> it's a lot of work to make the current stuff ldap3 working
18:24:38 <bknudson> do we want a spec for ldap3?
18:24:44 <shaleh> morgan: that does not sound too crazy actually
18:24:52 <ayoung> morgan, is an sssd based approach viable?
18:25:07 <bknudson> I think you can already do sssd?
18:25:12 <gyee> ayoung, last I checked, sssd wasn't ready for prime time
18:25:15 <morgan> ayoung: i think it could be separately, but no not as a replacement
18:25:34 <morgan> ayoung: we need to support ldap-current model, sssd would be a different/separate dirver
18:25:39 <bknudson> you could use federation for ldap
18:26:06 <morgan> we aren't deprecating direct ldap access [if we were it would be easier]
18:26:19 <morgan> then SSSD would be more viable.
18:26:38 <ayoung> morgan, did we deprecate writable LDAP ?
18:26:48 <morgan> ayoung: in .. liberty? this cycle? we did
18:26:52 <morgan> i remember we did.
18:27:07 <bknudson> this is where it would be nice to have unit tests for the driver interface
18:27:09 <morgan> it's got a run on it stil. also ldap3 is fully py3 compat
18:27:15 <gyee> SSSD is not read-only LDAP, just to be cleared
18:27:18 <ayoung> OK, so we can yank that in +2 from deprecation.  At that point, yeah, we can pull a lot of the LDAP code out and toss it
18:27:46 <ayoung> gyee, right
18:27:50 <shaleh> ayoung: if we did most of the work now in N, marked it experimental it could go live in O
18:27:51 <morgan> i think ldap3 is the right choice fwiw. an sssd alternative would be good to have too long term
18:27:52 <ayoung> they are two different stages
18:28:12 <ayoung> morgan, SSSD works now, and several releases back
18:28:21 <morgan> some deployments are going to get grumpy if we prescribe "needing" binary SSSD to access ldap
18:28:29 <ayoung> http://adam.younglogic.com/2015/03/key-fed-lookup-redux/
18:28:30 <morgan> since we have not gone on record for that.
18:28:57 <morgan> and have said we are maintaining the current model when we were previously asked
18:29:19 <ayoung> morgan, agreed.  Just want to minimize the amount of effort needed for the existing LDAP code
18:29:37 <ayoung> IE..rewriting it to yank write access is minimal gain
18:29:54 <ayoung> but would be a lovely Masters Thesis project, IMHO
18:29:56 <morgan> existing ldap code can pretty much sit - ldap3 is py3 and isolating the ldap code
18:30:06 <bknudson> it's the tests that are the problem with converting to ldap3
18:30:09 <morgan> since ldap3 has pool capabilities etc
18:30:20 <morgan> and ldap3 is far far more pythonic
18:30:22 <bknudson> and probably config options are going to be different
18:30:32 <henrynash> ayoung: I’m expecting us to have problems trying to pull the writeable LDAP in +2 anyway….I think it will take longer
18:30:33 <morgan> also ldap3 is pure python
18:30:44 <morgan> which is a win
18:30:57 <dstanek> bknudson: we'll need a new fake! or a better way...
18:31:40 <ayoung> (driver selection based on [identity]/driver` option is deprecated and will be removed in the "O" release).'),
18:31:45 <bknudson> if we have real unit tests for identity drivers we don't need a fake ldap like we did, can get away with mock.
18:31:46 <browne> what about mockldap?
18:31:46 <ayoung> that is not the same as writable...
18:31:58 <gyee> I've got this funny feeling that if we are going for this full PCI thingy, writable LDAP may stick around :-)
18:32:11 <dstanek> browne: mockldap is pretty interesting
18:32:59 <bknudson> if mockldap works with ldap3 then that's fine.
18:33:25 <dstanek> who is going to spec this out?
18:33:37 <ayoung> Write support for Identity LDAP backends has been deprecated in the M  release and will be removed in the O release."
18:33:49 <tjcocozz_> mockldap looks like it works with python-ldap
18:34:12 <bknudson> maybe we'll wind up writing our own mockldap3
18:34:16 <gyee> why do we care about mockldap, isn't ldap all func tests?
18:34:25 <ayoung> #link http://git.openstack.org/cgit/openstack/keystone/commit/?id=99a427833b70164e974c0d17b093dfbce6952813
18:34:57 <shaleh> tjcocozz_: yeah, but either a) we help port it to the new lib or b) we come up with a similar layer for ldap3
18:35:14 <shaleh> tjcocozz_: either is better than the current fake we are maintaining
18:35:27 <bknudson> all reviews should be author stevemar, code-review +2 stevemar, workflow+1 stevemar
18:35:34 <tjcocozz_> shaleh, that would be an interesting way of going about it
18:36:03 <dstanek> there is also the real fakeldap project
18:36:21 <morgan> ayoung: i uh.. no. so we're deprecating SQL driver too?
18:36:26 <shaleh> like I said a few minutes ago, it sounds like what we need is an etherpad with viable specs for N
18:36:33 <morgan> ayoung: oh wit nvm.
18:36:35 <morgan> ayoung: i misread
18:36:36 <ayoung> Heh
18:36:38 <morgan> ayoung: nvm...
18:36:40 <morgan> sigh
18:36:47 <bknudson> fakeldap looks like python-ldap, too
18:37:07 <dstanek> bknudson: hmm... i though it was more of a generic server
18:37:15 <dstanek> maybe we'll have some work to do
18:37:28 <ayoung> this would be a great "Hey I want to get into Keystone" project for a new contributor
18:37:52 <dstanek> so do we have a spec or who will write it for ldap?
18:38:19 <bknudson> I wouldn't want to do the work to write up an ldap3 until we've got tests for the driver interface.
18:38:41 <bknudson> because the way the drivers are tested now is just going to mean all the tests run 20 times instead of 6 like they do now
18:39:12 <ayoung> bknudson, can you won the specs on that, and I'll help find coders?
18:39:31 <ayoung> won->own
18:39:37 <bknudson> I can write up a spec
18:40:08 <dstanek> #action bknudson to write up a spec for ldap!
18:40:20 <ayoung> cool. So, any other long term plans for Keystone that we should all be aware of?
18:41:08 <gyee> ayoung, small things, like revocation event optimization
18:41:16 <ayoung> gyee, ah, good point
18:41:19 <ayoung> Fernet default
18:41:48 <ayoung> I had working code for "liveness" checks replacing many of the revocation events
18:42:07 <lbragstad> ayoung patch?
18:42:08 <gyee> better logging to make it easier to monitor
18:42:17 <shaleh> gyee: I was typing that as you were
18:42:19 <ayoung> IE.  instead of revoke by project, just chekc at token validation time if the project exists and is active
18:42:24 <ayoung> lbragstad, sure
18:42:32 <shaleh> make INFO actually usable for operators
18:42:38 <amakarov_> What about adding missing federation features to the os-cli? Is this job done already?
18:42:53 <ayoung> #link https://review.openstack.org/#/c/285134/  Remove unneeded revocation events
18:43:11 <shaleh> amakarov_: all of the pieces exist now. Someone just needs to synthesize an interface
18:43:21 <ayoung> amakarov_, I think it is in the CLI, just not in Puppet
18:43:25 <shaleh> amakarov_: it was on my plan but I am not hacking on federation currently
18:43:36 <ayoung> what is missing from CLI?
18:43:45 <shaleh> ayoung: --use-this-SP
18:43:59 <shaleh> ayoung: --with-this-type-of-auth
18:44:01 <ayoung> shaleh, K2K is not Federation
18:44:05 <ayoung> Heh
18:44:10 <amakarov_> ayoung, shaleh: couple of months ago there was no way to register a service provider for ex.
18:44:22 <marekd> is ksa merged into osc?
18:44:28 <ayoung> OK, so please tag that stuff as K2K, as calling it Federation is really confusing.
18:44:39 <amakarov_> ayoung: got it
18:44:52 <gyee> ayoung, what do you call K2K?
18:45:18 <ayoung> gyee, using  Keystone token in one Keystone via SAML to get a Keystone token in another
18:45:25 <amakarov_> gyee: it's when IdP is a Keystone in another cloud
18:45:27 <ayoung> SP is only there for that case, not basic K2K
18:45:40 <ayoung> right, what he said bettern I did
18:46:23 <gyee> huh?
18:46:30 <shaleh> amakarov_: for K2K, the only bit outside of Keystone API is setting up the SAML bits inside shibboleth or whatever.
18:46:31 <ayoung> gyee, forget it
18:46:46 <amakarov_> shaleh: ack, thank you
18:47:14 <dstanek> #action dstanek to summarize the list of things people mentioned as need in N
18:47:17 <ayoung> So any other major items we need for Newton ?
18:47:21 <dstanek> do we have any other topics?
18:47:46 <gyee> dstanek, that's a bucket-full already!
18:47:55 <dstanek> gyee: overflowing
18:48:03 <dstanek> what else is new?
18:48:16 <bknudson> looks like we can skip the summit since we've got it all taken care of
18:48:31 <dstanek> bknudson: ++
18:48:31 <bknudson> we'll be out by the pool
18:48:33 <shaleh> bknudson: nah, we need to sit together and argue some more.
18:48:33 <samueldmq> hehe
18:48:46 <shaleh> bknudson: come to an agreement. Then have morgan change his mind and derail things.
18:48:52 <gyee> now we finally have time to wait for Franklin BBQ
18:48:55 * shaleh hugs morgan
18:48:58 <dstanek> #open open discussion
18:49:01 <morgan> shaleh: i've removed my -2s btw
18:49:10 <morgan> shaleh: but whatever :P
18:49:16 <shaleh> morgan: for now :-)
18:49:17 <morgan> shaleh: even told gyee
18:49:20 <morgan> shaleh: forever.
18:49:21 <dstanek> morgan: put them back
18:49:25 <morgan> dstanek: nope.
18:49:38 <dstanek> if there's no real work conversation we can call the meeting early
18:49:44 <morgan> dstanek: mostly cause i can't login to launchpad :P
18:49:46 <ayoung> morgan and I are tag teaming to add up to 1 termie
18:49:49 <lbragstad> i have a question
18:49:50 <shaleh> morgan: but, I need your disapproval to give me a reason to keep hacking.
18:49:56 <ayoung> He -2s and I derail conversations in IRC
18:50:08 <lbragstad> thoughts on the validity of https://bugs.launchpad.net/keystone/+bug/1552795 ?
18:50:08 <openstack> Launchpad bug 1552795 in OpenStack Identity (keystone) "enhance notification for user events with user name" [Wishlist,In progress] - Assigned to Lance Bragstad (lbragstad)
18:50:38 <gyee> lbragstad, username is not worldly unique
18:50:41 <bknudson> do we still have 2 kinds of notifications?
18:50:57 <bknudson> cadf and whatever else they're called?
18:50:59 <lbragstad> bknudson yes - kinda
18:51:11 <lbragstad> bknudson you can either have 'basic' or 'cadf' notifications
18:51:20 <bknudson> does cadf have a field for user name?
18:51:29 <gyee> but I am also working on https://bugs.launchpad.net/keystone/+bug/1537963
18:51:29 <openstack> Launchpad bug 1537963 in OpenStack Identity (keystone) "notification not generated for authentication failure with invalid user name" [Wishlist,In progress] - Assigned to Guang Yee (guang-yee)
18:51:39 <lbragstad> cadf notifications just include information about the initiator i believe?
18:51:43 <ayoung> BTW, Audit events from Keystone are totally spoofable
18:51:51 <ayoung> as are any other messages on the bus
18:52:01 <bknudson> you better protect your bus
18:52:10 <ayoung> Does anyone protect the bus?
18:52:19 <gyee> bknudon, non-repudiation is what we need
18:52:25 <gyee> not about the transport
18:52:28 <ayoung> gyee, wrong
18:52:32 <ayoung> it is about the transport, too
18:52:34 <bknudson> if I could spoof messages I'd start booting instances, too
18:52:36 <dstanek> where's Keanu when you need him
18:52:39 <ayoung> what we need is access control
18:53:03 <ayoung> http://adam.younglogic.com/2016/03/what-can-talk-to-what-on-the-openstack-message-broker/
18:53:17 <ayoung> ideally, only Keystone would be able to write to the keystone topic
18:53:45 <ayoung> but writing that as a regex would be frightening
18:54:38 <ayoung> we should at a minimun have a keystone rabbit user on localhost
18:54:42 <gyee> ayoung, still can run into man-in-the-middle
18:54:57 <ayoung> and set up the acl that only the Keystone user can write to the keystone topic
18:55:02 <lbragstad> i'm going to put the patch I was working on for adding usernames to notifications on hold then (and review other bug fixes instead)
18:55:04 <dstanek> there's also message signing
18:55:04 <ayoung> gyee, nah, not if we configure the broker right
18:55:32 <ayoung> broker can enforce that keystone is local and is the only writer, or we need to set up TLS for distrbituted, and use X509 for client auth
18:56:01 <ayoung> gyee, simplest case:  keystone is a localhost user, only keystone user can write to the keystone topic
18:56:05 <dstanek> lbragstad: that sounds like a plan; i want to read over that bug
18:56:10 <ayoung> everything else extends from there
18:56:19 <lbragstad> dstanek i'd be curious to know what your thoughts are
18:56:36 <dstanek> lbragstad: that makes two of us
18:56:50 <ayoung> [{rabbit, [{loopback_users, [guest, keystone]}]}]
18:56:55 <lbragstad> dstanek seems like it would be easy to bloat the notification
18:57:28 <dstanek> lbragstad: yeah, without reading that bug, i'd wonder why that is needed if we have the actual id
18:57:45 <lbragstad> Dmitri has some justifications for it
18:57:58 <samueldmq> 2 mins left
18:58:10 <lbragstad> but we also run into cases where usernames are mutable
18:58:14 <lbragstad> etc...
18:58:22 <dstanek> ok, i'm going to call it in a min and we can take it to our channel
18:58:23 * morgan spoofs ayoung on the bus.
18:58:43 <ayoung> morgan, if only the Keystone user can write to the topic, no spoofs
18:59:06 <dstanek> #endmeeting