18:06:14 #startmeeting keystone 18:06:15 Meeting started Tue Feb 3 18:06:14 2015 UTC and is due to finish in 60 minutes. The chair is stevemar. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:06:16 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:06:18 The meeting name has been set to 'keystone' 18:06:22 there we go 18:06:23 my bad 18:06:27 #link https://review.openstack.org/#/c/151939/ 18:06:28 what's the topic? 18:06:52 * morganfainberg was off in LP release management zone and missed the alarm for the meeting 18:06:59 idea is to allow a cloud provider to disable the ability of clients to store extra attributes with their SQL entities 18:07:14 o/ 18:07:18 o/ 18:07:24 is somebody asking for it? 18:07:24 #topic Allow disabling of SQL extra attribute storage 18:07:27 right now, people could be storing PII data….and the cloud provdier woulldn’t even know it 18:07:29 o/ 18:07:45 o/ 18:07:56 we already (effectively) allow you to disable it with LDAP (since you have to provide a mapping)…. 18:08:06 ..but for SQL, you can;t turn it off 18:08:18 I'd rather the identity backend went away 18:08:22 use federation 18:08:23 would be “enabled” by default, for backward compatibilty 18:08:40 this is also true for assignments and resources 18:08:47 bknudson: ++ 18:08:56 federation ftw 18:08:58 bknudson: don’t disagree 18:09:06 I will disagree! 18:09:07 henrynash what is the advantage. When would you recommend to a stakeholder to turn it off? 18:09:16 j/k 18:09:31 I get the PII argument but are there others? 18:09:34 unless you know your clients are doing this, we would advsie them to turn it off 18:09:41 henrynash, so we are going to disallow MacDonald to give out toys for kid meat because kids are getting fat? 18:09:43 so, you're mostly talking about extra user attributes? 18:09:44 off by default? 18:09:49 they could be storeing laubounded data sets 18:10:05 gyee only in your state :-) 18:10:07 #link https://review.openstack.org/#/c/151939/ 18:10:23 henrynash: is this for only identity or for all of the models that use extra? 18:10:36 dstanek, i would like to see this for *all* models not just PII on user 18:10:53 morganfainberg: ++ 18:11:07 dstanek: so I think we want to the option for all entities, I’m open if it needs to be a single flag, or one each or identity, assignment etc. 18:11:11 i don't think keystone should act as a cloud providers makeshift redis 18:11:14 morganfainberg: ++ 18:11:17 dstanek, exactly 18:11:18 it's always been something that i hated about our SQL backend, it'd be nice to remove the 'extra' with a view to deprecate it 18:11:19 so anything using extra is to be updated? 18:11:20 seems easy enough and it would help deployers be a bit more secure... hopefully it's implemented such that it's easy to maintain. 18:11:49 no changes to teh SQL models would occur this these proposal…we’d just disable the read/write of it 18:11:50 also stick with a single for now and see if anyone wants in more granular 18:12:10 bknudson: i imagine all the existing code could still run, but if extra != '{}' on create or update then 400? 18:12:17 we’d leave it for out-of-band work for cloud provdier if they want to delete existing stored extra data 18:12:24 what profig gives 'extra' now and what do we offer instead? 18:12:40 s/profig/profit/ 18:12:49 this might mess with our JSONSchema validation code. 18:12:50 dolphm: that was probably my only question…would we silently drop and extra data provdied, or error 18:13:14 error 18:13:15 henrynash, redelegation uses extra now 18:13:18 henrynash: if it's opt in then i think drop is acceptable 18:13:20 henrynash: maybe that's a secondary option, but as a client i'd rather be alerted via 400 18:13:36 if 'extra' is not in the spec, then its a fair game 18:13:39 the config option could be "drop", "error", etc. 18:13:55 bknudson: interesting…. 18:14:09 gyee: extra has never been in the spec as far as I know 18:14:12 i *think* extra was in the spec somewhere. 18:14:13 henrynash: it's also easy to enable this with lbragstad's jsonschema work -- there's an any_other_attribute thing that's enabled somewhere 18:14:15 "i don't think keystone should act as a cloud providers makeshift redis" Great quote by dstanek! 18:14:16 but it wasn't pervasive 18:14:16 henrynash: does this only add a config option or are you thinking that we would actually remove the field using migrations? 18:14:30 morganfainberg, was or is? 18:14:44 henrynash: i don't think extra is in the spec either 18:14:47 dstanek: just the config option….would not remove the SQL data 18:14:49 any spec 18:14:53 henrynash: dolphm https://github.com/openstack/keystone/blob/master/keystone/assignment/schema.py#L39 18:14:54 if it's not in the spec then it's just behavior that we have to preserve. 18:15:04 topol, https://twitter.com/MdrnStm/status/562675595633774592 18:15:12 lbragstad: ++ 18:15:17 bknudson: absolutely…we KNOW peopel do use it in some circumsatnces 18:15:18 bknudson: ++ 18:15:23 lets do this 18:15:29 henrynash: ++ 18:15:34 morganfainberg LOL! Im retweeting 18:15:54 henrynash, metacloud made/makes extensive use of extra values 18:16:10 (cc cburgess ^ not sure if you're still doing that) 18:16:15 morganfainberg: which is why default option is no change 18:16:26 I'm here.. 18:16:32 Yeah we still use extra values. 18:16:44 we could have a config option where you can provide extra jsonschema for each resource type. 18:16:44 We are still using the self service code you are familiar with. 18:16:46 long term, we want people to write out-of-tree extensions that catch notifications to store their own data 18:16:59 henrynash, i'm not seeing a lot of net positive in dropping the extra bits 18:17:38 for a hybrid cloud, if part has extrat atrib8utes enabled and part does not will interoperability be impacted??? 18:17:50 Its probably not the end of the world for us if it goes away. Means we have to write our own migration for that data when we hit the cliff of its going away in the DB. 18:17:52 topol, not really 18:18:06 stevemar: long term, I don’t think we want keystone to be storing the data…we want peopel to extend keystone with tehir own code to store their own data…this is a first step on that path 18:18:06 extra values are *mostly* just redis with a bad name 18:18:11 topol, if its not in the spec, then there's no interoperability concern 18:18:20 I also get concerned that we have all these config options that make their cloud work differently and who knows if anybody uses them. 18:18:21 henrynash, actually i'd go a step further - we don't want people to extend keystone 18:18:22 That being said I would prefer not to have to do extra work but we don't always get what we want. 18:18:29 morganfainberg, good. Was hoping that was the case. But had to ask. even if it involved the QOTD 18:18:39 i'd rather see that data be stored externally and referenced when needed 18:18:54 bknudson, yeah, agree with that 18:18:59 morganfainberg, like roll your own damn IdP :) 18:19:02 morganfainberg: true…we want them to add theit own code that might inteact with keystone (via notifcations) if their data needs to be tied to keystone entiytes 18:19:04 it really isn't keystone's job to store "blob of whatever you shoved in" 18:19:05 default and deploy 18:19:08 the problem is developers wind up coding to the defaults. 18:19:17 morganfainberg: You are killing me here.... :P 18:19:20 sounds like time for another survey! 18:19:35 cburgess, i am trying to make your life exciting! 18:19:44 how many people participated in the previous ones? 18:19:52 breton, exceedingly small numbers 18:19:59 it's not like we can stop deployers from disabling extras already... just edit the code. 18:20:01 the LDAP identity one had the best response. 18:20:23 bknudson, we prefer monkeypatch 18:20:27 so all this proposal does is allow us to stop doing this, but the default is “current fucntionality”….is there any arguement at least about taking at least that first step 18:20:38 so - i think the right answer here is [unfortunately] not getting rid of extras short of a V4 API. 18:20:51 i would like an option to turn those off. 18:21:03 and/or look at moving the data into not being wedged into the model 18:21:28 but we wont ever (again short of API version) be able to rid ourselves of extra since it is an actively used feature 18:21:29 ver ver ver v4! 18:21:31 morganfainberg: v4 api will have the same issues where data would already be stored in extra and you have to choose how to represent it 18:21:47 are there any plans fo v4 actually? 18:21:49 jamielennox, nope. part of the migration would be "extra doesn't exist, use something else" 18:21:53 marekd, no 18:21:56 morganfainberg: ok. 18:21:58 don't start no shit now 18:22:04 and i will go on record here and say not just no, hell no. 18:22:05 v5 ftw 18:22:08 morganfainberg: I agree….we can’t actually take away extra yet] 18:22:08 morganfainberg: a new version is not a migration 18:22:09 but v5 18:22:16 jamielennox, yes it can be 18:22:19 micro versioning! 18:22:23 jamielennox, feature X doesn't exist in new version 18:22:26 jamielennox, don't use it. 18:22:38 lbragstad: v2015.02.03 18:22:38 jamielennox, products do that all the time 18:22:43 morganfainberg: not if v3 has taught us anything, if you don't provide what they want in v4 people will continue to use v3 18:22:51 let's all just forget that v4 ever happened. 18:22:53 jamielennox, but we're not doing v4 18:22:55 we're off topic 18:22:56 we don't get to roll out a new server, v3 and v4 are just views onto the same data 18:23:13 ok, let’s decide on this one…and move on! 18:23:20 henrynash, we can't rid ourselves of extras 18:23:21 who's going to review it? 18:23:22 henrynash, sorry 18:23:24 we can make it better 18:23:30 all we are proposing here is: allow an option to disable storing it 18:23:37 not to remove extra 18:23:38 we can unwedge it from the models and we can allow people to disable it 18:23:42 if you can find 2 cores to review it then go ahead. 18:24:06 henrynash: i'm good with it as an option that we can start pushing people towards 18:24:12 morganfainberg When you said unwedge what does that mean? 18:24:14 but as much as we hate it - it can't go away because it is being used. we might want to offer operators a way to define a schema for it 18:24:37 cburgess, right now extra is a json blob on say "user", make it not be an icky json bobl on the user row 18:24:47 morganfaiberg: you don’t think we can provide an option to disable yet? 18:24:57 henrynash, we can. but just expect no one to ever use it 18:25:00 How would data migrations work for that? 18:25:17 definitely cant take away. But can recommend you don't use 18:25:21 henrynash: do you have customers that would immediately use the feature? 18:25:23 cburgess, same way migration works today. doesn't change the API jsut how we store for more efficient use of storage 18:25:32 You want to just migrate the blob as it to another table and do a reference to it then? 18:25:45 Just trying to predict what a migration would look like for us. 18:25:52 so... one concern was already raised that we actually store real data in extras... 18:25:54 morganfainberg: sounds like a Keystone -> redis bridge - is that really what you want to do? 18:25:54 cburgess: that wouldn't solve henrynash's concern 18:25:55 cburgess, right now - no migration ;) 18:26:00 e.g., something with trusts and also email, etc. 18:26:01 dstanek, shhhh 18:26:27 I don't know what all might be in there 18:26:32 dolphm: not specific names ones…other than customer who are always asking “can I control whether PII info is stored" 18:26:35 bknudson, we should move all attributes we rely on / claim in the spec - as top level columns etc 18:26:39 So during an upgrade the deployers would have to have a script to do the data migration? 18:26:53 I don't want to rely on email. 18:26:53 cburgess, SQL migrate would handle if we did this 18:27:00 Ahh ok sorry was confused. 18:27:16 cburgess, how can we migrate if we don't even know what's in extra? 18:27:31 bknudson, if it isn't part of the spec - it can shove it in extras - it is arbitrary data. 18:27:48 we're approaching the half way marker and there are 3 other topics, we should set time aside for those too 18:28:10 (and I thought this was an easy one!) 18:28:12 henrynash, i think the option is fine 18:28:12 morganfainberg: i'd rather just document a pair of queries - "what extra data is in my db?" and "delete all the extra data in my db" along with the option to prevent keystone from using extra any further 18:28:18 gyee That was kind of my point. If you go changing extras you have to migrate whats there as a blob because its completely unstructured from a keystone perspective today. 18:28:23 we can add comments to the spec. 18:28:38 just expect that no one will in practice use it. meaning there will be an edge case of customer who wants it 18:28:43 but most wont care 18:28:53 please move further comments to the spec 18:29:06 adding options that nobody can use just makes our job harder. 18:29:14 cburgess, which mean its a mess any way you cut it :) 18:29:23 stevemar, next topic 18:29:28 #topic Email as a first class attribute 18:29:28 (like debug=False!) 18:29:34 gyee, cburgess, bknudson, please continue conversation on spec / -keystone channel post meeting 18:29:37 ok, now this one is contentious 18:29:45 ++ 18:30:08 henrynash, short - what is the usecase. 18:30:09 inverse of previous topic: promote PII to first class attribute 18:30:13 some people have multiple email addresses. 18:30:24 bknudson: most* 18:30:30 ++ 18:30:34 dolphm, everyone? 18:30:40 bknudson, dolphm but only one per domain 18:30:44 even my parents have more than one email each 18:30:55 i see the next topic is about a 1st class email, but we may also need to address any other data the Keystone stores in extra 18:31:08 morganfainberg: have requests from customer to “filter by email address”…where tehir users names are NOT email 18:31:18 dstanek: we're on that topic now 18:31:24 use ldap 18:31:37 dolphm: i know i typed that up and topol distracted me :-) 18:31:38 henrynash, i reaaaaaly don't want to make the SQL identity store more featureful unless we need to. 18:31:54 you may have multiple email address, but are you allow more than one when you register with a cloud provider? 18:32:00 and right now we support email (or mention it in specs) a BIT….so we are bit conflicted…we need to push it one way or the other 18:32:03 dstanek, keep your head in the game 18:32:28 i kindof hate that we store any PII in keystone as part of the spec. 18:32:30 henrynash: if it's mentioned in the spec it shouldn't be extra from the sense of disabling extra 18:32:32 henrynash, is the customer using sql or ldap? 18:32:37 jsavak, does RAX allow multiple emails during registration? HPCS does not 18:32:55 stevemar: I don’t actually remember…but I think LDAP 18:33:00 1 email as id per user instance is a common practice... 18:33:04 gyee: sort of? 18:33:11 stevemar: and our LDAP moduel DOES support email address mapping 18:33:23 gyee - no we don't. We have a separate service for "contacts" outside of keystone for address, phone, email, etc 18:33:30 henrynash, we might as well make it consistent between backends. but ugh 18:33:38 jsavak: but there's also sub account or child accounts or whatever 18:33:48 dolphm, subaccount would be a separate account no? 18:33:51 dolphm, but those are per account right? 18:33:55 morganfainberg: and I think some of examples in the API spec show email being returned ! 18:33:58 meaning i email <-> i account 18:34:09 morganfainberg, ++ 18:34:10 morganfainberg: yeah, it's basically separate users in keystone land (someone correct me if i'm wrong) 18:34:11 dolphm - correct - those are additional identities with less authorization to access to the same cloud-account's tenants. 18:34:25 henrynash, sure make it a 1st class column. 18:34:42 consistent between abckends and the spec 18:34:47 * morganfainberg cries a little. 18:34:53 cries?!! 18:35:11 i wish email wasn't in the spec ;) 18:35:15 but wait, are we going to encrypt it? 18:35:16 it's mentioned from the shell when creating users as well 18:35:19 QQ 18:35:26 its PII afterall 18:35:28 gyee, not my problem. 18:35:28 morganfainberg: i don't actually remember that landing 18:35:31 bknudson, oh thats great 18:35:38 gyee, :P 18:35:45 morganfainberg, at least provide a hook to encrypt it? 18:35:55 dolphm, if it's in the spec (API) and supported we need to probably make it 1st order 18:36:04 if it's not in the spec we can make it disappear from examples 18:36:10 gyee: which is why the spec says it should be a) optional, and b) there should be a config option to disable its storage 18:36:10 * morganfainberg checks 18:36:16 a plugin or some sort? 18:36:26 +1 to encrypting stored emails 18:36:39 yep 18:36:42 it's only in examples 18:36:47 we could make it all disappear 18:36:54 henrynash, *optional* first class attribute doesn't sound right 18:37:02 so is the identity / SQL backend now open to any kind of changes? 18:37:02 morganfainberg: which is why I said we are “conflicted a BIT”…since it is not in the spec, but is in the exaples, the client and teh LDAP options 18:37:15 amakarov: if you encrypt the email you can't filter on it 18:37:15 gyee: like description? 18:37:17 I thought we rejected features for identity in the past... 18:37:21 e.g., password policy 18:37:21 If you encrypt email addresses please make that configurable so that "None" is a valid option. 18:37:23 bknudson, if we called it out as an attribute 18:37:34 bknudson, i was going to say we need to fix it 18:37:40 dolphm, ah you're right 18:37:41 in this case i say we remove it from our examples 18:37:54 maybe this is a bug and not a feature? 18:37:56 keystone should not use pii. 18:38:03 bknudson, a doc bug it looks like now 18:38:09 dstanek, and if you not you risk to expose it to some smart guy with a spam cluster :) 18:38:09 i thought it was claimed as an attribute 18:38:11 i was wrong 18:38:20 man this rabbit hole is getting deeper 18:38:22 henrynash, cc ^ 18:38:25 morganfainberg: it is a VERY common user attribute for mapping from LDAP to corproate stores 18:38:35 henrynash, we should *not* be mapping email. 18:38:43 and we should remove it from our examples 18:38:47 but we can't break people 18:38:59 but the right answer is don't put PII in keystone. 18:39:15 soryr to flip, my mistake for thinking one thing about the spec vs reality 18:39:18 dstanek, it still possible to filter by encrypted email: encrypt and filter :) 18:39:26 amakarov, i don't want to layer that in 18:39:30 amakarov, at all 18:39:46 i'd rather go to the operators and say "sorry we can't support this" and take the flames/yelling 18:39:47 morganfainberg, I understand 18:39:54 than need to handle PII properly *in* keystone 18:40:08 amakarov, its call homomorphic encryption I think 18:40:14 morganfainberg, mb as a hook? 18:40:17 if it was part of our spec "aka: user create . email address" as an attribute my stance would be we need it 18:40:18 amakarov, no. 18:40:19 amakarov: only if you are encrypting wrong and not using a salt or we would have to store all of the salts separately 18:40:25 basically allow searches on encrypted fields 18:40:30 if operators shouldn't store PII in keystone then that should be documented somewhere... 18:40:35 morganfaiberg: so I’m actually OK with not stoting in SQL…but it is a very common requirement to map via LDAP….I just don;t think we can say, you can’t do that 18:40:35 bknudson, ++ 18:40:56 henrynash, we should *not* break ldap, but we should document don't store PII in keystone and remove it from all the examples (email) 18:41:14 henrynash, and it should not be made first-order in sql. 18:41:37 morganfainberg: and the client? 18:41:39 henrynash, i would also document that using the email map is a bad idea. email == username makes it a different class of data 18:41:42 does PII include name? 18:42:04 jamielennox, depends on who you ask 18:42:17 jamielennox, i think we can fudge on name if needed - it's grey 18:42:37 henrynash, client should not specifically require/call out email but we can't break compat. 18:42:41 that's why I use a made-up name. 18:42:57 dstanek, agreed, I cant imagine filter with a salt... 18:43:05 bknudson, i KNEW your real name wasn't brant! 18:43:07 bknudson: i wondered about 'brant' 18:43:25 henrynash, same deal as LDAP 18:43:36 bknudson, it's been brent this whole time 18:43:40 henrynash, don't remove it - but it shouldn't be called out as available specifically 18:43:48 or in examples 18:44:01 morganfainberg: ok…so only issue is filtering when using LDAP…I’ll think on that one…but otherwise we clean up examples etc. 18:44:17 henrynash, fix filtering in the filtering fix spec. 18:44:28 next? 18:44:30 henrynash, and we know SQL identity sucks on some filtering 18:44:36 argeed…ok, I yield teh floor! 18:44:41 stevemar, go go next 18:44:46 #topic Barbican backend for Keystone Credential API 18:44:52 \o 18:44:57 so two issues 18:45:04 gyee, arunkant ^ 18:45:14 1) we need to pass the user token to barbican 18:45:22 this is another credential backend, in addition to what we have already? 18:45:30 gyee, yep, known workflow for *things* in OpenStack 18:45:34 2) we need individual user token for migration if we are going to support migration 18:45:48 do we need auth_token in front of keystone then? 18:45:49 gyee, no magic migration. 18:45:53 bknudson, yes, use Barbican 18:46:23 I have no problem with this as long as it's optional... barbican isn't integrated. 18:46:33 bknudson, no, we'll need to change the policy to allow self-management of credentials 18:46:42 right now only admin can access the credential APIs 18:46:45 gyee: does barbican have it's own credential store 18:46:57 like raw for ssh keys and whatever 18:46:59 jamielennox, barbican is a credential store 18:47:02 gyee, magic migration from service -> other service is always painful and if anything should be done via a side-band script not the API 18:47:02 gyee: what if the user doesn't provide a token? (e.g., client cert) 18:47:14 gyee, no "magic" migration needing user tokens. 18:47:24 bknudson, no token, no love 18:47:30 gyee: right - i mean that isn't specific to a certain credential type - you can put anything in our credentials 18:48:01 jamielennox, which is probably not a good thing for our credentials API 18:48:01 morganfainberg, sure, we can call that out in the spec 18:48:02 gyee, you can make a barbican-manage or keystone-to-barbican that can leverage the SQL models etc. but not worth trying to use the REST API for migration. 18:48:29 gyee, bad idea - because once you flip the bit to use the new driver how do you access the old data? or vice-versa 18:48:33 i'm wondering if we can't just turn our credential store off (by config) if barbican is in the deployment 18:48:33 jamielennox, our credential API is very generic right now 18:48:58 bknudson we don't need auth_token in front of keystone, technically auth_context does most of the same stuff w/o the "go talk to keystone" bits. 18:49:08 bknudson, and we do store the token in the token_model 18:49:09 morganfainberg: ++ 18:49:28 nice thing about auth_token is now it gives you an auth plugin. 18:49:37 maybe we could get that in keystone's auth_context. 18:49:45 morganfainberg, we can make keystone credential API "read-only" 18:49:48 or configurable 18:50:06 and users manages their credential using barbican rest API 18:50:08 bknudson, sure. we should retrofit some of that into auth_context (actually we need to break up auth_token into consumable bits that keystone can put into auth_context) 18:50:15 bknudson: we do need to come up with ways of sharing between auth_token and auth_context 18:50:22 gyee, hm, sounds like we don't need a credential api then! 18:50:37 gyee, no magic migration using rest. i'll -2 that really fast. 18:50:38 morganfainberg: that was where i was going 18:50:40 morganfainberg, it's pretty useless right now 18:50:45 stevemar, agreed 18:50:59 so no barbican integration then? 18:51:05 deprecate it and say use barbican? 18:51:13 bknudson, ++ 18:51:16 bknudson: ++ 18:51:19 morganfainberg: but heat uses it in current form 18:51:23 bknudson, uh can we do that? 18:51:39 i think thats the path we want in the end, not sure how feasible that is, especially since it's an incubator project still 18:51:40 Haneef, we can fix that 18:51:46 Haneef: these things never go away instantaneously (or ever) 18:51:46 stevemar, big tent 18:52:00 ok this is a TC issue... i think 18:52:07 we're back to deprecate API 18:52:11 for *service* 18:52:23 erm superseded by *other service* 18:52:31 i think so 18:52:46 this is harder to do right and there are no real guidelines that are clear 18:52:59 ideally we should get heat and other things to use barbican 18:53:00 nova is trying to deprecate their apis. 18:53:04 its easy to *say* deprecating stuff, not so easy to do ;) 18:53:07 e.g., for glance and neutron 18:53:14 then credential api becomes like LDAP assignment 18:53:17 so I don't think it's a new thing. 18:53:22 limited use and we can deprecate 18:53:31 but for now we can't deprecate. 18:53:37 7 minutes left 18:53:48 fact is deployer wants secure storage of credentials 18:53:58 gyee, fact deployer should use barbican 18:54:00 morganfainberg, you have an item to take this up with the tc? 18:54:05 deployer should just put them in a file owned by root 18:54:18 * mordred shuts up 18:54:18 heh 18:54:20 gyee: i'd vote no for the barbican backend, and put work into fixing nova heat etc to move away from the keystone credential store 18:54:39 gyee we should fix the stuff that *used* credential to use barbican as appropriate. 18:54:49 gyee, and get heat moved over to barbican 18:54:50 jamielennox: +1 18:54:52 morganfainberg, there we go 18:54:53 we might be asked to provide a barbican backend to make migration easier. 18:55:02 ha 18:55:13 k, k 18:55:17 5 minutes left, i'm going to next topic 18:55:30 #topic Request for PowerKVM platform CI to comment on Keystone patches 18:55:34 krtaylor, ping 18:55:39 o/ 18:55:42 krtaylor, sorry for the short time 18:55:43