18:01:00 #startmeeting keystone 18:01:01 Meeting started Tue Mar 18 18:01:00 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:02 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:05 The meeting name has been set to 'keystone' 18:01:10 * ayoung here 18:01:12 #topic Reminder: Design summit session proposals open until April 20th 18:01:17 #link http://summit.openstack.org/ 18:01:26 this is for the *design* summit, not the conference 18:01:48 dolphm, I assume we are already at the point where we can start merging the proposals 18:02:00 lots of keystone topics already 18:02:08 18 18:02:10 ayoung: i'd like to wait until after the deadline to accept/reject/merge anything so it's clear what we're working with 18:02:16 how many slots do we have? 18:02:19 assuming I can count, which is a big assumption 18:02:22 stevemar, 3 18:02:24 ayoung: otherwise it seems unfair to start shuffling things too early 18:02:24 :) 18:02:39 we had about 8 last time 18:02:45 stevemar: fewer than last time, as we're donating some time to a new cross-project track 18:02:53 stevemar: i don't think i have a final number yet 18:02:54 one days worth, spread over multiple days 18:03:15 our client stuff maybe can be merged with the Oslo common client 18:03:16 boooo... 18:03:37 and then we can make sure anything that is a server side topic at least addresses client side for that same topic 18:03:47 and as a reminder for what the *design* summit is all about if you'd like to propose something: this is for moderated collaborative discussions, not for powerpointy lectures :) 18:04:18 down with ppts 18:04:22 and with prototype demos 18:04:24 i believe the deadline for proposing powerpointy lectures for the conference has passed 18:04:45 gyee: i'd submit that demos are best saved for lightning talks 18:04:52 which you don't need to register for in advance 18:05:02 sounds reasonable 18:05:03 really, the week will be spent doing collaborative development in the dev lounge. the rest is just distractions 18:05:19 gyee: plus if your demo fails, you can sit down and just try again the next day :P 18:05:35 dolphm, we'll do live debugging 18:05:43 Feature freeze for Keystone will be Juno 2. If you want a big feature in to Juno, you need to have it started before the summit 18:05:57 right morganfainberg ? 18:06:15 * ayoung thinking ephemeral tokens 18:06:30 ayoung: oh, yes please 18:06:34 the bigger the feature, the earlier the better 18:06:35 that will be a *fun* discussion 18:06:38 ayoung: and J-2 is happening when? 18:06:48 marekd: probably july, there's no schedule yet 18:07:03 well find out the schedule at the summit right? 18:07:07 or shortly after? 18:07:14 we'll need people doing reviews early, too, then 18:07:44 bknudson: we could do a very early feature proposal freeze, rather than feature freeze 18:08:24 this might be a conversation best saved for the summit, but it's good to give everyone an early warning here 18:08:32 #topic RC1-blocking issues 18:08:50 i wanted to run through what's on our plate for shipping an RC1 18:09:11 my goal is to have fixes for all RC1 issues at least in review by friday 18:09:39 #link https://launchpad.net/keystone/+milestone/icehouse-rc1 18:09:42 so let's run through the list 18:09:43 o/ here sorry 18:09:49 Bug 1291157: idp deletion should trigger token revocation 18:09:59 (no linky bot?) 18:10:01 #link https://bugs.launchpad.net/keystone/+bug/1291157 18:10:06 ayoung, ++ at least work on it 18:10:13 dolphm, bot has been dead for week+ 18:10:22 boo 18:10:49 marekd: ayoung: stevemar: i'm comfortable shipping icehouse with this as a known issue, but i'd be happy to see a fix if anyone wants to pursue 18:11:27 i'm worried we're going to get into api conversations, and it's far too late for that 18:11:33 dolphm, I don't want to do list_tokens_for_idp but doing it in the revocation events makes sense 18:11:36 is it going to require a schema change? 18:11:57 stevemar: does a revocation need to happen on ipd update too? 18:12:13 ayoung: i think it'd just be a matter of adding "OS-FEDERATION:identity_provider" 18:12:16 dstanek, it probably should 18:12:20 bknudson, for revocation events, it could probably be done with the domain ids, assuming they are valid. otherwise, yeah, I'd need to add an IOdP column 18:12:28 ... as an attribute of a revocation event 18:12:49 stevemar: dstanek: why? there's not too much to update, is there? 18:13:03 dolphm, what did we settle on as the relationship between IdPs and (identity) domains? 18:13:08 dolphm: i have to reasoning...it was mentioned in the issue 18:13:20 ayoung: no explicit relationship in icehouse 18:13:24 dolphm, if we use revoke instead of delete, wouldn't we then create a dependency on an optional extension? 18:13:25 dolphm: stevemar dstanek actually you can only disable idp when you PATCH it 18:13:38 thanks marekd 18:13:40 https://github.com/openstack/identity-api/blob/master/openstack-identity-api/v3/src/markdown/identity-api-v3-os-federation-ext.md#update-identity-provider-patch-os-federationidentity_providersidp_id 18:13:46 dolphm, do userids from Federation have a domain_id associated with them in icehouse? 18:13:55 stevemar, nope 18:14:18 stevemar, the event API insulates one from the other 18:14:23 stevemar: yes, but i'd say it's acceptable for extensions to depend on one another 18:14:35 dolphm, ++ 18:14:35 ah, you mean an implicit dep 18:15:05 ayoung, yeah, but even if it was an explicit dep, it wouldn't be horrible (though i think the code doesn't support it) 18:15:23 for now, I would say that an IdP needs to have some sort of domain id. Then we can treat it like a domain disable event 18:15:35 plus won't everything break if there is no domain_id? 18:15:43 ayoung: that's a viable solution, but not really for icehouse 18:15:59 ayoung: there's no domain_id now 18:16:01 dolphm, doesn't any user in Federation have to alreayd live in the Identity backend? 18:16:05 ayoung: no 18:16:13 just groups? 18:16:15 ayoung: nooo? 18:16:18 ayoung: they "live" in the remote IdP are ephemeral in keystone 18:16:19 ayoung: just groups 18:16:25 ayoung: authorization lives in keystone 18:16:44 make IdP ID == domain_id then 18:16:55 ayoung: that's the conversation we can have at the summit 18:16:59 put it in the tokens and get a disable domain event 18:17:06 OK 18:17:45 is anyone completely opposed to waiting until juno to address this? 18:18:10 dolphm, i'm actually more for addressing this in juno 18:18:19 ++ 18:18:22 morganfainberg: i am too 18:18:22 it's scary leaving this bug... there's no workaround? 18:18:35 dolphm, vs squeezing it in under the wire (unless we have a very very good reason not to) 18:18:45 bknudson, so is it just that we delete an idp and tokens aren't revoked? 18:19:01 bknudson: not a simple one 18:19:06 morganfainberg: yes, you delete or disable an idp and tokens aren't revoked 18:19:14 because, as aweful as it is we could do the same thing we do for oauth 18:19:31 morganfainberg: and what's happening there? 18:19:35 oh nvm we can't 18:19:38 we'd need a new index. 18:19:43 bknudson, disable a group would delete the tokens right? 18:19:46 we're not tracking by user. 18:19:56 gyee, which groups 18:19:58 gyee: right, disable a group or a user... or a domain 18:20:02 assuming you keep track of Ide=group mappings 18:20:12 gyee: we dont 18:20:16 IdP user to group mapping I mean 18:20:47 I guess we'd know what groups are referenced by the mapping 18:20:50 gyee: we don't really track that, per se 18:20:52 hm. 18:21:01 bknudson: the mapping is mutable 18:21:05 can we add the IDP id to the token? 18:21:14 morganfainberg: it's already there 18:21:16 changing the mapping should probably invalidate tokens, too 18:21:20 can we make auth_token ask keystone if an IDP is valid on validating the idp? 18:21:21 morganfainberg: and i suspect what the revocation event should be based on 18:21:28 morganfainberg: and later go through all the tokens and match them? 18:21:40 morganfainberg: token -> user -> OS-FEDERATION -> identity_provider -> id 18:21:52 or provide a list of active IDP IDs to auth_token? 18:21:56 internal to keystone this is easy 18:22:05 is IDP valid, nope? reject 18:22:11 loss of IdP is catastrophic, and revoke all tokens 18:22:22 ayoung: lol that's not a bad approach 18:22:35 ayoung, dolphm, for I i think we could expose a list of valid IDP ids 18:22:54 morganfainberg: GET /v3/OS-FEDERATION/identity_providers 18:23:05 dolphm, yeah 18:23:06 dolphm, make auth_token middleware track that 18:23:18 dolphm, just check if token_id is in that list. 18:23:35 it's a low risk / easy solution 18:23:45 and internal to keystone we can check IDP is valid for validation calls 18:23:50 if we don't already 18:23:55 dolphm, it is the sane approach for Icehouse 18:24:04 morganfainberg: ayoung: ++ 18:24:09 i'll add keystoneclient to that bug then 18:24:12 k 18:24:13 morganfainberg, probably a separate middleware sitting behind auth_token 18:24:24 gyee, i don't think we want to do that. 18:24:26 morganfainberg, I'm unwilling to do that 18:24:35 gyee, getting everyone to load another middleware is bad 18:24:50 I thik that a list of valid IdPs as part of the check is getting into the revoke events space 18:24:50 this needs to be in auth_token 18:24:59 I'd rather just add it to revoke events 18:25:30 revised https://bugs.launchpad.net/python-keystoneclient/+bug/1291157 18:25:35 ayoung, but we don't have full revocation events now. 18:25:36 ayoung, and we need a solution for I 18:25:41 morganfainberg, middleware is alreay overloaded, but I like ayoung's idea of having it in revoke events 18:25:51 long term i want it in revocation events 18:26:02 morganfainberg, we have revocation events, and I have a fledgling patch for the client side 18:26:38 ayoung, but we've stated revocation events are expirimental 18:26:38 auth_token middleware refactoring is overdue :) 18:26:51 gyee: it's happening, slowly 18:26:52 morganfainberg, so is Federation in my book 18:27:04 so we need to move on, we have more bugs to discuss :) 18:27:07 Bug 1255321: v3 token requests result in 500 error when run in apache 18:27:08 ayoung, then we need to mark it in the docs 18:27:10 Bug 1291423: revocation events sync slows responses to all authenticated calls 18:27:13 Bug 1291047: Keystone returns traceback for db backend 18:27:20 #link https://bugs.launchpad.net/bugs/1255321 18:27:22 gyee: yes, i was looking at it yesterday and was very tempted to start hacking it up 18:27:23 #link https://bugs.launchpad.net/bugs/1291423 18:27:28 #link https://bugs.launchpad.net/bugs/1291047 18:27:35 https://review.openstack.org/#/c/81166/2 18:27:35 ayoung, will discuss more in -dev after meeting 18:27:43 ayoung: all three of these are assigned to you -- care to share the burden with anyone else? 18:27:56 dolphm, looking 18:28:22 dolphm, I'd be happy to hand them off 18:28:32 I'll keep https://bugs.launchpad.net/keystone/+bug/1291423 18:28:38 anyone want to volunteer to pick any of these up? 18:28:46 which is revocation events, and I think we already have a better approach 18:28:57 the 500 error one is interesting, compressed token should help 18:29:23 gyee, yeah...but I don't think I can slip compressed tokens into the icehouse release 18:29:27 "Keystone returns traceback for db backend" is a regression that should be an easy fix if we can figure out where things went wrong 18:30:12 dolphm: yea, that shouldn't happen with up to date requirements, it was initially filed on bugzilla and i put the same comment there but no response yet 18:30:14 ayoung: so should https://bugs.launchpad.net/keystone/+bug/1255321 be untargeted from rc1? 18:30:30 dolphm, well...maybe 18:30:33 jamielennox: have you tried to reproduce? 18:30:40 dolphm, I have a python33 issue, but assuming I solve that 18:30:49 the path needs to be clear to enabling compressed tokens on the server 18:30:57 I think that means a new config value 18:31:02 ayoung, recommend endpoint filtering to reduce service catalog size, or nocatalog on token request 18:31:04 or, maybe a new token provider 18:31:16 gyee, suspect that none of that is going to make adifference 18:31:21 dolphm: you'd have to get eventlet context switching at just the right time, i guess you could force that - but no 18:31:51 its the service catalog that super sized the PKI token 18:32:09 dolphm, so, assuming I get compressed tokens working, I would probably make the server side be an alternative token provider, that looks just like the PKi provider, but that generates compressed tokens instead 18:32:09 users are going to need the service catalog 18:32:21 and then you would configure that in the config file as the token provider 18:32:40 if that is acceptable, (and I figure out the python33 issue) we can have an RC1 fix 18:32:42 gyee: nocatalog isn't a viable option for everyone 18:32:42 bknudson, not the whole shebang, just the services user is interested in 18:32:46 ayoung: i'm still not sure why this would need to be a provider rather than just a wrap/unwrap middleware option 18:32:58 jamielennox: ++ 18:33:27 jamielennox, OK...so we don't assume that the entire openstack deployment upgrades at once 18:33:35 is there really a use case where the whole thing is needed? 18:33:36 if it doesn't we need to keep from breaking existing clients 18:33:51 beside horizon, where they can get the catalog on a separate API 18:33:59 gyee, doesn't matter 18:34:08 can we make it a ?compressed=true 18:34:12 ? 18:34:13 so much code is written around the service catalog, we can't break that 18:34:19 morganfainberg, no 18:34:21 ayoung: sure that just means you ahve to roll out the client updates before you can switch over the server - but that would be the same as a new provider 18:34:33 morganfainberg: no, i don't think it should be optional at request time 18:34:35 ayoung, or we make it part of the accepts header 18:34:41 morganfainberg, it needs to be a different token format, and then auth token neeeds to handle both 18:34:49 ayoung, hm. 18:34:50 morganfainberg: i don't think it should be optional either -- it should happen transparently for end users 18:35:06 dolphm, oh right derp. sorry was thinking 18:35:08 dolphm, auth_token making the request 18:35:09 dolphm, defaults to uncompressed for Icehouse, compressed for juno 18:35:10 nope. 18:35:35 MII still needs to work 18:35:54 rem,ember, there is Java code out there that talks to openstack....we can't brak this 18:35:57 break 18:36:09 alright, so untargeted Bug 1255321 "v3 token requests result in 500 error when run in apache" from RC1 18:36:12 but we can make the mechanism possible, and publish it 18:36:23 dolphm, wrong call 18:36:29 the right call is to make it optional 18:36:44 just don't try to drop all the baggage at the same time 18:37:29 ayoung: you're advocating for compressed tokens to be a service-side feature, in which case it's too late for icehouse 18:37:39 ... correct me if i'm wrong 18:37:39 Fair enough 18:37:40 ayoung: i don't disagree with the client needing updating and MII etc. My point is that as the logic behind PKI and compressed PKI tokens is exactly the same other than that last compress step it doesn't need to be a new 'provider' 18:38:08 jamielennox, the problem is that the old clients can't deal with the new format 18:38:17 its not just "the final step" 18:38:28 jamielennox: that's my intuition as well, but i haven't tried to make it happen to judge the actual required complexity :) 18:38:35 ayoung: sure 18:38:37 ayoung: auth_token middleware? why do clients care in general? 18:38:49 bknudson: because auth_token has to unpack tokens to validate them offline 18:38:55 bknudson: clients == auth_token here 18:39:02 and compression is part of the unpacking 18:39:06 dolphm: me neither, but i consider the provider to be the controller and compressed or not should be a view of it 18:39:11 right, just making sure it wasn't all clients that care. 18:39:23 bknudson, it will be...in Juno 18:39:50 bknudson: exactly, actual end users won't/shouldn't care 18:40:05 dolphm, OK, so I'll continue on with the compressed token work, but if anyone wants to use it, they will need to run an unsupported, out of tree token provider 18:40:10 alright then, so the other bug up for grabs was https://bugs.launchpad.net/keystone/+bug/1291047 18:40:26 we'll get the token provider into Juno 1, and backport if there is sufficient demand 18:40:27 jamielennox: do you think this should be incomplete? 18:40:44 this is a tough one to duplicate imo 18:41:10 dolphm: i'm wondering from before if I could reproduce with some well place sleeps and a decent load 18:41:23 jamielennox, adding in sleep(0)'s? 18:41:25 it's not using SecurityError somehow, is it? and then the behavior is dependent on debug=true? 18:41:39 morganfainberg: right - something that made eventlet switch to a new thread 18:41:42 jamielennox, it does force context switches 18:42:14 morganfainberg: yea, i *think* you can do it with a call to eventlet as well but i dont know how 18:42:14 how does the context switch cause this error? 18:42:25 i've tried to ignore eventlet as much as possible 18:42:47 * dolphm is confused between this bug and another one with a really similar title... 18:42:47 dstanek, when eventlet switches it's execution point, thread swap, it looks like it's clearing the exc. stack? 18:42:49 dstanek: see linked eventlet bug: https://bitbucket.org/eventlet/eventlet/issue/118/exceptions-are-cleared-during 18:43:10 it's a fairly standard coroutine-like-code hell problem 18:43:35 thankfully eventlet has been mostly decent (lately) about not introducing/causing these 18:43:42 but it shouldn't happen with new greenlet and global-requirements.txt has sufficient greenlet in it 18:44:05 (from my understanding) 18:44:09 dolphm, how is this a security issue? 18:44:40 morganfainberg: leaking internal details unnecessarily 18:44:50 dolphm, ah ok phew, i thought i was missing something else 18:45:05 dolphm, i was concerned you were seeing a rollback issue causing a gap 18:45:22 (this is marked as a private security issue) but i marked it as a dupe of the public bug https://bugs.launchpad.net/keystone/+bug/1293113 18:45:43 it does actually vary a bit from https://bugs.launchpad.net/keystone/+bug/1291047 18:46:31 seems like keystone should trap all exceptions and return a generic 500 error 18:46:43 bknudson, that was what i was thinking would solve this 18:46:46 unless debug=True... but even then doesn't seem necessary if it's in the log 18:47:19 this just seems like we should tighten up what we pass back, make the returns much simpler 18:47:30 bknudson: so basically UnexpectedError should extend SecurityError ? 18:47:35 erm errors to the client 18:47:50 dolphm: that sounds safer to me. 18:47:50 dolphm, ++ 18:48:05 dolphm, ++ 18:48:26 dolphm, I'll take this one if you need. 18:48:31 morganfainberg: sure 18:48:36 dolphm, and chase down any external deps 18:48:50 hopefully it'll be minimal 18:49:26 morganfainberg: it looks like the only gotcha is that UnexpectedError currently expects %(exception)s in the string formatter 18:49:35 dolphm, yeah 18:49:44 dolphm, shouldn't be too bad (might require test massaging) 18:50:05 i'll link it against both the security bug and the regression one 18:50:39 oh wait no the security one is marked dup 18:50:45 haha don't need to. 18:50:46 cool 18:50:56 henrynash: o/ 18:51:02 hi 18:51:04 #link https://bugs.launchpad.net/keystone/+bug/1287219 18:51:17 Bug 1287219: scope of domain admin too broad in v3 policy sample 18:51:19 so fix merged 18:51:21 dolphm, https://review.openstack.org/#/c/81235/ should go in. YorikSar needsd to open a bug to describe the problem he is fixing, but I've been testing the code. It solves some issues that I was seeing, but had not yet been able to reproduce with a server side test 18:51:29 henrynash: you marked this as fixed, but there's no link to a fix as far as i can see? 18:51:43 hmm…ok let me fix thaat :-) 18:52:09 dolphm, the code merged, https://review.openstack.org/#/c/79897/ https://review.openstack.org/#/c/80769/ 18:52:10 ayoung: open a bug and track it against RC1 18:52:14 dolphm, ++ 18:52:14 YorikSar: ^ 18:52:35 sorry #link https://review.openstack.org/#/c/80769/ 18:52:43 #link https://review.openstack.org/#/c/79897/ 18:52:45 dolphm, ayoung: ok 18:53:12 henrynash: morganfainberg: thanks 18:53:23 henrynash: must have been a fluke with the bot 18:53:32 can't trust bots 18:53:32 henrynash: i linked the to the review in the bug 18:53:42 dolphm: thx 18:53:57 and the last one... 18:54:04 #link https://bugs.launchpad.net/keystone/+bug/1273867 18:54:06 I do have one question on https://review.openstack.org/#/c/80769/, see last item of the agenda (cover it then) 18:54:11 Keystone API v3 lists disabled endpoints and services in catalog 18:54:23 dolphm: it's partially fixed... disabled endpoints 18:54:24 henrynash: ack 18:54:27 but not disabled services 18:54:46 bknudson: and i think endpoints are the more important piece, but do you think you'll have time to do the same for services? 18:54:49 (this week) 18:55:10 I can't promise I'd get services done this week 18:55:14 it's crazy around here. 18:55:29 bknudson: this is another one that i'm comfortable shipping with as a known issue, given that we have a solid workaround (disable all the endpoints for your "disabled" services prior to upgrading) 18:55:47 close the bug and open a new one? 18:55:53 ++ 18:55:54 a new one for services? 18:55:55 ++ 18:56:05 that sounds very reasonable 18:56:28 and disable a region? 18:56:47 gyee: /v3/regions has no impact on the catalog, yet 18:57:00 bknudson: so endpoints is complete and we just have to implement services for this one? 18:57:04 gyee: i think that expectation will come, but not yet 18:57:05 dstanek: yes 18:57:05 gyee, separate bug for that too, i'd wager 18:57:22 yeah 18:57:23 bknudson: it's going to be a parallel fix -- probably sharing a lot of code 18:57:24 dstanek: yes, disabled endpoints are not in the catalog anymore... but disabling a service has no effect 18:57:26 dstanek: ^ 18:57:55 for services, I don't think there's a tempest test that doesn't work like there was for endpoints 18:58:13 if you create a new bug you can assign it to me and can work on it 18:58:19 bknudson: good to know 18:58:23 dstanek: will do, thanks! 18:58:25 #topic Should domain_id be immutable by default? 18:58:29 (quickly) 18:58:29 so https://review.openstack.org/#/c/80769/ provides option to made domain_id immutable... 18:58:42 question is, should we really make it immutable by default 18:58:49 would change existing fucntionaliy 18:58:50 henrynash: are you asking about icehouse or juno? 18:58:54 dolphm, icehouse 18:58:56 icehouse 18:58:58 bknudson: is tempest ensuring that disabled endpoints don't appear? 18:59:21 i'm concerned about a security implication (talked w/ henrynash) due to the ability that you could move a user you control into a domain you don't control 18:59:26 dstanek: no, it was trying to set endpoint disabled to an invalid boolean value (by our XML translator) 18:59:45 and i'm not 100% sure we are guarding against admin in domain A with user moved to domain B clearly 18:59:57 dstanek: as far as I know there are no tempest tests for getting a catalog with disabled endpoints... otherwise we wouldn't have the bug 18:59:58 henrynash: what's the logic behind it being mutable in the first place? 19:00:09 morganfainberg: ack; i'm not opposed to making it immutable in icehouse for that reason (it's a security improvement), and i'm very much in favor of making it immutable *by* juno 19:00:11 jamielennox, it shouldn't be. we just didn't make it immutible 19:00:13 it wouldn't seem to work with the way our SQL/LDAP providers work 19:00:15 jamielennox: lsoat in the midst of time 19:00:31 jamielennox: there's not any ( morganfainberg++ ) 19:00:39 dolphm, in juno for sure immutible 19:00:45 i'd prefer to invert the option for I 19:00:54 henrynash: put the patch up to change the default and we'll vote on it :) 19:00:54 if someone _needs_ that, make a less secure deployment 19:01:04 we're over time 19:01:05 #endmeeting