18:02:18 <dolphm> #startmeeting keystone 18:02:19 <openstack> Meeting started Tue Jan 15 18:02:18 2013 UTC. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:20 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:20 <gyee> full house today :) 18:02:22 <openstack> The meeting name has been set to 'keystone' 18:02:35 <dolphm> #topic Team Membership Updates 18:02:43 <dolphm> official welcome to henrynash and gyee :) 18:02:51 <henrynash> thank you, kindly 18:03:02 <gyee> look forward to do some serious damage 18:03:02 <ayoung> Good job on code reviews this past week. Much appreciated you two 18:03:14 <dolphm> keep it up :) 18:03:19 <henrynash> :) 18:03:26 <dolphm> #topic Test Coverage 18:03:30 <dolphm> ayoung: any updates? 18:03:34 <ayoung> topol, want to join the weekly Keystone meeting? 18:03:40 <topol> yes 18:03:57 <topol> this is it correct? 18:04:01 <ayoung> dolphm, no major changes in test coverage has happened this week 18:04:09 <ayoung> dolphm, going back one step 18:04:20 <joesavak> jorgew and myself are here too. : ) 18:04:23 <ayoung> topol is going to be helping out on LDAP support 18:04:30 <dolphm> yay! 18:04:41 <ayoung> topol, who is the other dev with you? 18:04:46 <topol> sahdev will be helping too 18:04:51 <topol> say hi sahdev 18:04:53 <spzala> ayoung: hi Adam, I would like to join keystone weekly meeting as well. 18:05:20 <ayoung> cool 18:05:21 <spzala> Hi all! 18:05:26 <dolphm> spzala: /salute 18:05:30 <gyee> welcome 18:05:37 <ayoung> This is starting to feel like a real project. 18:05:39 <topol> Thanks 18:05:41 <spzala> :-) thanks! 18:06:05 <ayoung> so, back to test coverage. Lets open a Jenkins bug to rerun the test coverage results 18:06:16 <dolphm> ayoung: a bug? 18:06:31 <dolphm> ayoung: i.e. we lost coverage on something? 18:06:34 <ayoung> dolphm, wasn't it there before and got removed? 18:06:48 <ayoung> I thought you said last week that we used to run coverage on each Jenkins run 18:06:58 <ayoung> otherwise RFE 18:07:32 <ayoung> Should be as simple as adding the flag to run_tests.sh, but maybe jenkins doesn;t use that 18:07:39 <dolphm> ah that kind of coverage 18:07:52 <ayoung> yeah, unit test coverage 18:08:00 <dolphm> i don't see the charting anymore on jenkins, but we have the infrastructure for it as far as i can tell 18:08:16 <dolphm> keystoneclient too 18:08:25 <dolphm> see tox.ini 18:09:19 <ayoung> dolphm, so, for example, http://logs.openstack.org/18519/5/check/gate-keystone-python27/2010/ 18:09:27 <dolphm> ayoung: want to tackle that? 18:09:34 <ayoung> dolphm, sure 18:09:56 <ayoung> #action ayoung get jenkins to generate unit test coverage report 18:09:57 <dolphm> #action ayoung to trace down jenkins coverage reporting 18:10:06 <dolphm> and do it twice! 18:10:09 <joesavak> ha 18:10:19 <dolphm> #topic critical issues 18:10:49 <dolphm> i know we have a security-related fix in the works, anything else on fire? 18:11:06 <ayoung> http://wiki.openstack.org/Meetings/KeystoneMeeting 18:11:36 <dolphm> ayoung: yeah, criticalness appears to have dissappeared from the agenda 18:11:54 <ayoung> For everyone new, Agenda is ^^ 18:12:05 <dolphm> i noticed yesterday that db_sync no longer supports specifying a migration number -- need to file a bug on taht 18:12:29 <dolphm> #action dolphm to file a bug on lack of db_sync migration number option 18:12:42 <ayoung> dolphm, I don't think it is critical, but the is an issue identified by tempest WRT roles that is coming up as a bug. I'll take it for now 18:13:11 <ayoung> #link http://paste.openstack.org/show/29472/ 18:13:24 <ayoung> jaypipes is opening/assigning 18:13:34 <dolphm> cool 18:13:52 <dolphm> i'll assume that's it for now 18:13:55 <dolphm> #topic "Default" domain migration 18:14:09 <dolphm> so, this is a topic i've only scratched the surface on, but i wanted to give everyone a heads up on what i'm doing 18:14:14 <dolphm> this is probably bp worthy, so... 18:14:22 <dolphm> #action dolphm to write default-domain blueprint 18:14:51 <ayoung> dolphm, how does that tie in with henrynash 's domain blueprint... 18:15:10 <dolphm> anyway, we'll be creating a data migration to create a Default domain (id=default, name=Default) that will contain user/tenant resources on the V2 API 18:15:10 <ayoung> #link https://blueprints.launchpad.net/keystone/+spec/domain-name-spaces 18:15:15 <henrynash> dolphm: I'm already monkeying around with domains, so feel free to assign it to me :-) 18:15:37 <dolphm> i assume it'll use private namespaces once that bp is implemented 18:15:47 <ayoung> +1 18:16:11 <gyee> henrynash, I meant to response to your domain namespace proposal 18:16:12 <dolphm> any operation on the v2 API will automatically work with default domain resources (creating a new tenant on v2 will create a project in v3 in the default domain) 18:16:24 <dolphm> we may also need to special case the domain so that users can't accidentally delete it from the v3 api 18:16:52 <dolphm> and i'd also like to make the default_domain_id a configuration option so that users can override the behavior and delete the domain we create if they'd prefer 18:17:04 <dolphm> any objections? 18:17:08 <dolphm> or questions 18:17:14 <ayoung> dolphm, no. 18:17:20 <ayoung> dolphm, a suggestion 18:17:23 <dolphm> or suggestions 18:17:26 <gyee> +1 on default_domain_id 18:17:30 <henrynash> dolphm: we need to also define what happens when (in v3) a user creates entities without specify a domain (e.ff. users) 18:17:34 <ayoung> we should probably gues the default domain from the DNS domain name. 18:17:39 <ayoung> But that is an install issue 18:18:00 <henrynash> do they go in the default domain? 18:18:04 <dolphm> henrynash: my thoughts-- automatically create the user in the creating user's domain 18:18:12 <dolphm> same for projects 18:18:16 <joesavak> +1 18:18:28 <joesavak> (autlo create the userin the creating user's domain) 18:18:56 <ayoung> why users domain and not "default" domain? 18:19:02 <topol> the same domain you plan to use for v2 tenants or are they separate? 18:19:12 <ayoung> Or, if there is more than one domain, why not force them to specify? 18:19:20 <dolphm> ayoung: as a v3 user, that doesn't seem intuitive to me 18:19:31 <dolphm> ayoung: as a v2 user, i don't care about domains, but the implementation has to 18:19:54 <ayoung> dolphm, so the question is how should the V2 API behave? 18:20:02 <dolphm> ayoung: in terms of..? 18:20:12 <ayoung> if no domain is specified. 18:20:22 <dolphm> ayoung: v2 api isn't domain-aware 18:20:22 <ayoung> V2 create user needs to guess the domain 18:20:29 <ayoung> OK, I can go with "same as user" 18:20:51 <ayoung> for V3, lets make it default if there is only one domain, and must be specified if there is more than one? 18:20:59 <dolphm> ayoung: that's an interesting thought... 18:21:07 <dolphm> ayoung: we can apply that behavior across both API's 18:21:19 <dolphm> ayoung: or, hrm 18:21:55 <dolphm> ayoung: can users not in the default domain authenticate on the v2 api? they absolutely can't with a privately namespaced domain 18:22:02 <ayoung> dolphm, I'm ok with guessing for V2. We don;t want to break existing tools, and matching the existing user seems to make the most sense 18:22:07 <dolphm> ..without using ID's 18:22:18 <joesavak> FYI - jorgew and i have a hard stop at 12:30 and would like to discuss multi-project/token scope 18:22:35 <dolphm> i'll write the BP and ping the list 18:22:50 <henrynash> dolphm: +1 18:22:56 <dolphm> #topic multi-project tokens in V3 18:23:06 <joesavak> dolphm - thank you 18:23:23 <ayoung> joesavak, any change in your thinking from last discussion? 18:23:31 <dolphm> #link https://review.openstack.org/#/c/19014/ 18:23:53 <joesavak> We've updated to include the roles under tokens per your suggestion ayoung 18:24:17 <ayoung> joesavak, +1 18:24:19 <henrynash> so I think the proposal hangs together fine….as long as we are convinced we need the capability…sorry for keeping on asking about that... 18:24:41 <joesavak> henrynash: the rax use case may be unique but it could happen again. We had a swift product runnning under 1 tenant. We acquired a company (mosso) that is the compute product running under a different tenant structure 18:24:41 <jorgew> So we have two major reasons for the feature 18:24:44 <ayoung> henrynash, it should not be a general use thing, but only for specific use cases that span multiple tenants 18:24:59 <jorgew> and joe just mentioned one of them :-) 18:25:13 <joesavak> so to do a back up of a compute instance and to store in a swift instance, we need a token capable of seeing 2 projects 18:25:13 <jorgew> ayoung, +1 18:25:15 <ayoung> the question is how to consume them 18:25:35 <jorgew> The other is to keep in mind that V2 supported the feature at least in the contract 18:25:52 <ayoung> My thinking is that we should make a multi-project token look like a single project token, but provide the data to the application from the middleware 18:26:11 <jorgew> ayoung, I agree. 18:26:14 <ayoung> auth_token will just say "yep, this is OK" 18:26:32 <ayoung> the unmarshalled token data should then be in the context 18:26:41 <jorgew> in fact because a resource can belong in a single project, then and the validate call can check for that the change is minimal 18:26:44 <ayoung> Now, RBAC will come into play 18:27:08 <dolphm> joesavak: v2 supported this feature in the XML contract, but the feature was lost in the JSON interpretation of that contract 18:27:31 <jorgew> dolphm, I think our implementation went ahead and added the support in JSON :-) 18:27:52 <jorgew> +1 on adding support for JSON schema :-) 18:28:00 <dwchadwick> seems the obvious answer, fix JSON 18:28:01 <joesavak> +1 ; ) 18:28:10 <dolphm> ayoung: we don't have a way to provide multiple projects to services underlying auth_token today, i wrote a comment in the review about how to go about supporting that 18:28:45 <ayoung> dolphm, right. 18:29:02 <joesavak> any questions on the use cases presented? 18:29:30 <dolphm> dwchadwick: which will immediately break all clients written against the current implementation; it's too late to go back and match the intent at this point on v2 18:29:34 <dwchadwick> We have an alternative use case based on federated identities 18:29:35 <ayoung> dolphm, my point is that it should be possible to use a multi-project token like a single project token, perhaps based on the ordering of the projects requested. Some projects are more equal than others. 18:29:42 <dolphm> dwchadwick: please share 18:30:16 <dwchadwick> When you present a set of identity attributes from the federated infrastructure, then these are mapped into local tenants/domains/roles 18:30:16 <ayoung> dwchadwick, that uses a different middleware, correct? 18:30:20 <henrynash> joesavak:…so once we different services based on different trees, I think I get the need 18:30:20 <dwchadwick> yes 18:30:42 <dwchadwick> so the client is presented with a set of tenants to choose from 18:30:55 <gyee> ayoung, I am not sure about the ordering 18:31:01 <dwchadwick> currently he picks just one, but can swap to another one later if he wants to 18:31:25 <gyee> depending on ordering is too easy for implementations to screw up 18:31:36 <dwchadwick> so the client has a set of accounts he can use from his single authentication session. Thus it should be possible to support this with keystone tokens 18:31:42 <dolphm> simply migrating a resource from ownership by project A to ownership by project B requires authz on both projects (which is a use case requested on the mailing list recently) 18:32:08 <dwchadwick> that is a different case to the one I am talking about 18:32:15 <dolphm> granted you could implement it with multiple tokens 18:32:25 <ayoung> dolphm, I have to say, I really don't like the _build_user_headers aspect of auth_token middleware. It feels wrong 18:32:45 <ayoung> But, if we must... 18:32:59 <ayoung> X_TENANT_IDS, X_TENANT_NAMES ... 18:33:02 <ayoung> comma separated 18:33:15 <ayoung> X_ROLES...ugh 18:33:16 <dolphm> ayoung: what if the name includes a comma? 18:33:35 <gyee> or in chinese? :) 18:33:47 <dolphm> gyee: right 18:33:58 <ayoung> <lando>This deal is getting worse all the time! </lando> 18:34:05 <dolphm> i think you need to JSON encode a list of strings (w/ unicode support) 18:34:43 <ayoung> dolphm, and, since this is all consumed by the WSGI app after the auth_token middleware, it should all be in the context, and not in custom headers 18:35:34 <gyee> I think various context are built from the headers right now 18:35:38 <dolphm> ayoung: i'm not aware of another way to communicate between middleware layers except through the environment 18:36:20 <gyee> we can stash the entire token access into the env 18:36:27 <ayoung> gyee, yeah 18:36:29 * dolphm runs 18:36:43 <ayoung> gyee, change "we can" to "we should" 18:36:50 <gyee> amen 18:37:02 <joesavak> we are diving quite deep into the implementation for multi-projects/token. Is that needed to agree on https://review.openstack.org/#/c/19014/ ? 18:37:08 <dolphm> then you're expecting underlying services to understand identity api v2, v3, v4, etc 18:37:17 <dolphm> which is not their responsibility 18:38:00 <gyee> we version the tokens 18:38:03 <gyee> might as well :) 18:38:05 <dolphm> joesavak: the only snag i see is that we'll also need to support multi-domain tokens since we're also going down the domain-scoped route 18:38:06 <dwchadwick> You need to move to an extensible design for the API if you can, so that new features can be added without changing the api 18:38:11 <ayoung> dolphm, multi project tokens are breaking the xcontract regadless of who we break it 18:38:15 <jorgew1> dolphm, really it's the middle ware that needs to understand this. Note that a resoruce still exists in a single project 18:38:36 <dwchadwick> the attribute type-value approach solves at least part of this 18:38:58 <dwchadwick> so domains would have just become yet another attribute in the set, and the API would not have needed to change to cater for domains 18:39:15 <joesavak> +1 dwchadwick 18:39:31 <gyee> dwchadwick, what does the attribute for project_ids like? 18:39:45 <dolphm> jorgew1: the implementation spans client + server + middleware though 18:39:49 <dwchadwick> project-ID=11324 18:39:54 <gyee> services still need to be aware of the attributes right? 18:40:02 <dwchadwick> project-ID=456578 18:40:13 <gyee> for multi-project scoping? 18:40:21 <dwchadwick> yes they do, but that then becomes a config issue of the service 18:40:35 <henrynash> so I think we may be getting off in the weeds here…. 18:40:36 <ayoung> dwchadwick, the question is not whether we can solve this in the general case. The question is can we solve this with the existing contract. I'm all for attribute based, but that still means changing the other projects consumption of the tokens 18:40:43 <gyee> how does that different from X-Project-IDs? 18:41:19 <ayoung> We've left the weeds behind. We are in the woods, crossing a stream 18:41:23 <dolphm> ;) 18:41:27 <ayoung> and I think I spotted a deer tick 18:41:27 <joesavak> lol 18:41:45 <dolphm> alright... 18:41:48 <dolphm> #topic SQL Testing 18:41:58 <dolphm> #link https://review.openstack.org/#/c/18519/ 18:42:05 <ayoung> joesavak, I'm OK with the concept of multi-proejct tokens, but I don;'t think the existing projects can consume them yet. Any thing consuming them needs to be a one off. 18:42:08 <dolphm> ayoung: ^ 18:42:24 <ayoung> OK, so the review above again has a test fix 18:42:28 <dolphm> ayoung: don't think = can't :) 18:42:33 <gyee> ayoung, a great topic for the next summit 18:42:36 <joesavak> ayoung - ok. thanks for everyone's attention on it. 18:42:45 <ayoung> https://review.openstack.org/#/c/18519/5/keystone/common/sql/migrate_repo/versions/009_normalize_identity_migration.py 18:42:55 <ayoung> This downgrade was not tested 18:43:10 <dolphm> my only realy issue with https://review.openstack.org/#/c/18519/ is that it's effectively stepping on the toes of tempest, which is already built for this 18:43:14 <ayoung> it probably should have its own unit test, but it gets run as part of the downgrade 18:43:17 <henrynash> dolpm: fyi ,we missed the v3 token discussion topic on the agenda 18:43:36 <ayoung> dolphm, disagree 18:44:01 <jaypipes> ayoung: done: https://bugs.launchpad.net/keystone/+bug/1099966 18:44:03 <uvirtbot> Launchpad bug 1099966 in keystone "Race condition when rapidly deleting and creating tokens" [Undecided,New] 18:44:11 <ayoung> tempest should be feeding us test that we should be working into the unit test framework, but we should be able to run against live DBs as part of development 18:44:27 <dolphm> henrynash: ah, i'll hit that next/last 18:44:29 <ayoung> A developer shouldn't need to know about Tempest to check that Keystone works 18:44:40 <jaypipes> ayoung: lemme know if the information in the bug isn't enough... happy to provide more. 18:44:50 <ayoung> and while sqlite is a good first approximateion, We should be able to test against a RDBMS 18:45:15 <ayoung> jaypipes, would you say it is a critical? 18:45:26 <gyee> I have bad experiences where stuff appeared to work in sqlite but blow up in mysql 18:45:47 <ayoung> gyee, and I don;t trust MySQL. I want PostgrSQL 18:45:56 <gyee> so be able to test against different DBs is definitely useful 18:46:08 <jaypipes> ayoung: meh.. I'm just ignoring the failures in my local tempest env. But it would be nice to fix. 18:46:09 <gyee> ayoung, that's the point 18:46:15 <ayoung> It was essential for LDAP, too 18:46:17 <jaypipes> ayoung: I could give it a go myself. 18:46:24 <jaypipes> ayoung: also, has nothing to do with MySQL. 18:46:38 <dolphm> ayoung: i don't disagree, but the same issue applies to all of openstack... hence, it's a project-wide initiative... ergo, tempest 18:46:40 <ayoung> jaypipes, nah, that is part of the ongoing discussion, not on that bug 18:46:42 <jaypipes> ayoung: and we do run tempest against an env with a PG backend. 18:47:33 <jaypipes> ayoung: https://jenkins.openstack.org/view/Tempest/job/gate-tempest-devstack-vm-postgres/ 18:47:45 <ayoung> dolphm, so the remaining changes in that bug (other than the fix to 009) are just to use teardown instead of assuming a blank database. We could make that conditional on the URL, and assume blank fro sqlite, use teardown otherwise 18:48:00 <gyee> dolphm, somebody's going to do it 18:48:11 <ayoung> jaypipes, we are discussing https://review.openstack.org/#/c/18519/ 18:48:49 <ayoung> dolphm, I think that the same approach (check the URL) probably makes sense for the rest of the unit tests. But that will be a separate commit. It has other issues to work through. 18:48:50 <jaypipes> ayoung: oh, sorry man. 18:49:03 * jaypipes goes back into his hole. 18:49:33 <ayoung> dolphm, should I move https://review.openstack.org/#/c/18519/5/keystone/common/sql/migrate_repo/versions/009_normalize_identity_migration.py into its own review like I did the other fix? 18:50:38 <dolphm> jaypipes: lol your input is appreciated 18:51:26 <dolphm> ayoung: i would say so 18:51:40 <dolphm> ayoung: not that it has backport potential, but it'll still merge faster as a discrete fix 18:51:47 <ayoung> dolphm, OK, will do so. Then I can redo the above for both upgrade test and unit tests. 18:52:51 <dolphm> ayoung: cool 18:52:56 <dolphm> #topic v3 tokens/authn/authz etc. 18:53:12 <dolphm> i didn't add this, but i assume this was the API discussion in gyee's thread? 18:53:12 <henrynash> guang: I added this so you had a chance to discuss 18:53:28 <ayoung> good idea 18:53:34 <gyee> excellent 18:53:53 <ayoung> OK, so I still don't really want to build multiple auth support into Keystone if we get it from Apache for free, but... 18:54:03 <ayoung> I do want to be able to use basic_auth and HTML 18:54:12 <henrynash> gyee: vested interest…since I need to make changes to this for both domain-scoping and name-spaces 18:54:23 <dolphm> ayoung: not everyone runs behind apache 18:54:26 <ayoung> And we need a V3 API for Authentication 18:54:33 <ayoung> dolphm, not yet they don't... 18:54:48 <gyee> ayoung, they said you can get java for free before oracle comes in :) 18:54:51 * ayoung twirls mustache 18:55:50 <ayoung> the use case I can see as most important is 18:56:02 <ayoung> keep ID in Mysql but auth against LDAP 18:56:16 <ayoung> I'd like that to be the first supported 18:56:34 <ayoung> assume that syncing users from LDAP to MySQL is done externally somehow... 18:56:37 <dolphm> ayoung: how does that impact the API? 18:56:49 <ayoung> dolphm, that makes LDAP the first supported Auth Mechanism 18:56:58 <ayoung> instead of Google whateveritis 18:57:02 <gyee> with auth plugin, I don't care where you store LDAP users 18:57:10 <gyee> Keystone can just be a proxy for all I care 18:57:12 <dolphm> gyee: +1 18:57:30 <ayoung> gyee, right. just understand that this is the single most demanded feature right now 18:57:32 <gyee> since we are going to abstract auth, token issuance, and token validation 18:57:36 <dolphm> gyee: have you worked on a python API for authentication plugins? 18:57:58 <gyee> you can effectively have your own backends 18:58:20 <gyee> dolphm, not formally 18:58:30 <gyee> just have a bunch of brain farts so far 18:58:36 <dolphm> gyee: {'auth': {'password'}} x.authenticate(**contents_of_object_x) 18:58:38 <dolphm> whoops 18:59:20 <ayoung> one question: how are we going to decide which auth mechanism to use for a given request? 18:59:26 <gyee> dolphm, I'll send out a proposal for the interfaces this week, assuming we all agree on the direction 18:59:30 <dolphm> gyee: {'auth': {'password': {'username': 'a', 'password': 'b'}} -> password_plugin.authenticate(username='a', password='b') 18:59:31 <ayoung> Domain is the most viable option 18:59:49 <dolphm> something like, **explode the contents of hte request that map to the name of the plugin 19:00:05 <gyee> yep 19:00:14 <dolphm> ayoung: according to gyee's proposal, it depends on the contents of the request 19:00:37 <ayoung> dolphm, that is too wide open 19:00:42 <gyee> mechanism maps to the plugin seem simple enough 19:01:00 <dolphm> plugins[plugin_name].authenticate(**credentials) -> resulting in -> plugins['password'].authenticate(username='a', password='b') 19:01:08 <dolphm> ayoung: what is too wide open for what 19:01:15 <ayoung> dolphm, "based on the request" 19:01:22 <ayoung> that implies a whole policy like rules engine 19:01:45 <ayoung> I'd prefer it to be something more implementable\ 19:01:48 <gyee> I am not seeing any problem 19:02:05 <gyee> if we don't support a given auth mechanism, we return a 401 19:02:09 <ayoung> like list the auth mechanisms in the config file, and then the domain entry has an auth mechanism specified in it 19:02:27 <dolphm> ah, we're over time 19:02:31 <dolphm> switch to #openstack-dev ;) 19:02:34 <dolphm> #endmeeting