18:00:54 #startmeeting keystone 18:00:55 Meeting started Tue Aug 19 18:00:54 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:56 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:58 The meeting name has been set to 'keystone' 18:00:58 hi 18:01:03 \o/ 18:01:04 #topic Feature proposal freeze August 21st 18:01:19 that's 2 days from now 18:01:24 i suspect this is going to be what most of today's meeting is about, but.... FPF is happening! 18:02:16 it is technically two days from now, but i want to make sure all the blueprinty stuff that we're shipping in juno is in review today/tomorrow so we can make clear cuts on thursday if necessary 18:02:35 right now it only looks like a few things are skating close to the deadline 18:02:50 o/ 18:02:53 which requires things to be "spec approved, code complete, and in review" 18:02:59 o/ 18:03:27 so bp endpoint-policy which has no implementation in review, but has an API review up https://review.openstack.org/#/c/112292/ 18:03:59 dolphm: it looks pretty staightforward 18:04:03 henrynash says that should make it? 18:04:08 bknudson: i hope so! 18:04:14 it does have that going for it 18:04:21 bp non-persistent-tokens has had some updates today - is that still 85/90% going to make it, morganfainberg? 18:04:41 morganfainberg: I had some questions about HEAD ops with no GET on https://review.openstack.org/#/c/112292/3/v3/src/markdown/identity-api-v3-os-endpoint-policy.md 18:04:42 dolphm: so I just posted the backend part of this extension 18:04:44 i'd say we'll be in the 90% range 18:05:01 https://review.openstack.org/115362 18:05:08 dolphm, for change of completion. but it's a load of code 18:05:26 henrynash is awesome, chucks up a patch at 2:04 18:05:39 dolphm, so far like 16 reviews and 1100 lines of change or so 18:05:42 henrynash: that was fast! 18:05:47 henrynash: yay! 18:05:57 henrynash: so we'll expect (1?) more patch? 18:06:17 dolphm: yes. I split it into two - backend and then the controller 18:06:23 henrynash: that works 18:06:26 bknudson, ah i'll take a look 18:06:57 are we still doing separate migrate repos? :-/ 18:07:02 looking at https://review.openstack.org/#/c/115362/ 18:07:30 dolphm: it’s all in the extension, so it has its own migrate repo 18:08:41 dolphm, separate migrate repos lead to fewer conflicts. I'd like to keep them as the norm 18:08:48 alrighty 18:09:02 ayoung, i think we should revisit that concept in K 18:09:14 morganfainberg, sure 18:09:16 ayoung, i think it poses a lot of other associated headaches. but not now 18:09:32 morganfainberg, lets table for now 18:09:36 ayoung, ++ 18:09:43 someone, i'm assuming stuart put composite auth support on the agenda, but that would be outside the named integrated release process https://review.openstack.org/#/c/108384/ 18:09:50 dolphm, wrong section 18:09:59 that was me, was supposed to be one less * 18:10:00 :P 18:10:05 so, as in "please review it" 18:10:10 cool 18:10:46 aand... i'm going to do a separate topic for this, but hopefully it'll be quick 18:10:53 #topic Deprecations in Juno 18:11:04 ooh ooh, token_api 18:11:04 do we have anything? as of this morning, the bp was blank & not started 18:11:19 just tagged a patch to that a few minutes ago 18:11:19 Identity API V2? 18:11:23 morganfainberg: as opposed to token persistence API? 18:11:23 referring to auth plugins by method name 18:11:25 oops, too soon 18:11:36 XML? 18:11:44 do we need to call it out if it's still deprecated? 18:11:50 bknudson: xml support was deprecated in icehouse 18:11:52 dolphm, the internal `token_api` will be deprecated, but persistence is housed under token_provider_api 18:12:03 er...make the class name? morganfainberg what exactly did we deprecate there? 18:12:07 dolphm, the idea is no one should ever use token_api.XXXX 18:12:09 bknudson: i'm going to leave v2 be for the moment - i don't think we totally meet the TC's recommendation on when we can deprecate v2 18:12:22 dolphm: y, nova doesn't work with v3 18:12:27 nor does auth_token middleware 18:12:48 ayoung, the use of .token_api. 18:12:52 bknudson: what's missing in auth_token? 18:13:02 ayoung, that massive chain of stuff i've been shuffling around to get us to non-persistence 18:13:11 ++ 18:13:16 dolphm: it doesn't authenticate using v3... e.g., it doesn't support domains 18:13:30 specifying the user domain and project domain 18:13:32 ayoung, it doesn't change the underlying code, just managers and controlelrs shouldn't use token_api, let token_provider_api call it's .persistence instead 18:13:33 for its token 18:13:34 bknudson: oh from the auth token user's perspective ++ 18:13:35 don't i have a patch up for that? 18:13:52 jamielennox: i believe you do... 18:13:53 jamielennox, yeah, I thought I saw something from you 18:14:33 actually maybe i don't, i had something working in testing but i can't see it 18:14:42 bknudson, nova CLI patch is under review 18:15:13 gyee: the part that I thought I'd get done for nova is to use v3 auth to get the token for its communication with neutron. 18:15:14 bknudson, i need to chase down jogo and resolve the policy,json issues in nova. it's a bit of a headache 18:15:20 bknudson, https://review.openstack.org/#/c/105900/ 18:15:28 bknudson, for nova (and this applies to other projectd) to be v3 friendly 18:15:32 morganfainberg: o/ 18:15:52 I had a spec in nova but I since we're running against the same freeze I told them I wasn't going to get it done for J 18:15:53 jogo, post meeting you around/not busy? [~45mins] 18:15:54 ? 18:16:07 jogo, probably will be quick. 18:16:08 morganfainberg: I think I can squeeze that in 18:16:10 I should be able to work on it and have it ready soon in K 18:16:20 jogo, ok will ping you as soon as we're done 18:16:29 morganfainberg: sounds good 18:16:33 morganfainberg: thanks 18:16:38 bknudson, what else is missing for Nova? 18:16:43 it also depends on a new release of some client libs 18:17:04 gyee: nova needs to be able to use v3 authentication to get a token for its neutronclient stuff 18:17:04 bknudson: How about service user/service tenant in middleware. That is still v2 18:17:34 Haneef: we were just talking about that... jamielennox said he might have been working on it. 18:18:04 bknudson, so as long as nova is passing the token to neutronclient we're fine right? 18:18:15 yep, i'm not sure why it isn't up already but it works 18:18:35 gyee: nova doesn't always pass a token to neutronclient now... it can also pass username + password 18:18:47 gyee: nova uses its own auth to talk to neutron not the users 18:19:02 no idea why, but it means there are a bunch of options to remove 18:19:20 oh, you mean like their own auth plugins? 18:19:43 gyee: no as in the nova service user 18:19:45 gyee: it uses neutronclient which accepts username and password 18:20:04 here was a first stab at the nova work: https://review.openstack.org/#/c/113735/ 18:20:07 it didn't go well 18:20:56 * gyee speechless 18:21:03 we are lacking at least a way to override the catalog from a plugin/session 18:21:10 there was something else as well.... 18:21:26 its ok. no one uses neutron anyway 18:21:39 ayoung: aahhhhhhh 18:21:50 lmao 18:22:04 jamielennox: nova might actually be happier using the service catalog rather than having the endpoint in the config file 18:22:32 but when it's doing token auth then it will need to get the endpoint/catalog from somewhere. 18:22:41 bknudson: i still can't figure out why they are using the nova service user rather than the user token 18:23:00 but yes, i would imagine they would be ok with that bit 18:23:10 jamielennox: it's possible there isn't a case where the nova service user is actually used. 18:23:53 the problem is we still need to honour the old setting if it is present 18:23:57 "This deal is getting worse all the time! " 18:24:58 so, back on topic, it sounds like token_api was the only thing we actually wanted to deprecate as part of bp deprecated-as-of-juno ? :P 18:25:00 Need to beat on Horizon some more to make sure it can support V3. 18:25:11 dolphm, and the auth-plugin thing 18:25:19 ayoung: was that not already deprecated? 18:25:20 the use of .token_api. 18:25:28 not officially 18:25:35 wasn't even inthe docs that you could do it 18:25:58 bknudson, ayoung: on that i did a new version of the endpoint hack: https://review.openstack.org/#/c/90632/ with a WIP follow up review 18:26:09 not sure about it though so i'll talk to you both about it post meeting 18:26:16 how do you deprecate something that's not officially documented? 18:26:25 just saying :) 18:27:10 ayoung: are you volunteering to deprecate that then? 18:27:26 self deprecation 18:27:32 dolphm, a deprecation message is in Log now if it is used 18:27:42 that was morganfainberg 's req to approve the patch 18:27:49 ayoung: well, that's what i meant by "isn't it already deprecated?" 18:28:10 dolphm, ah, misunderstood. 18:28:27 and then there was some conversation here about the federation API 18:28:27 https://review.openstack.org/#/c/107325/6/specs/juno/auth-specific-data.rst 18:28:46 to deprecate both /v3/OS-FEDERATION/projects and /v3/OS-FEDERATION/domains ? 18:29:03 right because jamielennox provides /auth/projects and /auth/domains 18:29:05 i think that's a good idea 18:29:11 works for me. 18:29:16 even though we basically have to support those forever 18:29:36 we'll use JSONHome to point clients to the right URL 18:30:11 so, basically don't advertise both, ever 18:30:11 Hi guys 18:31:01 i'll redo that spec and deprecate the federation resources 18:31:10 i put up a implementation patch as well 18:31:41 oh - spec was merged already 18:32:21 can't change it now. 18:32:48 jamielennox: i already put it on the deprecation spec 18:32:49 deprecation can and should be a separate commit anyway 18:32:56 jamielennox: https://blueprints.launchpad.net/keystone/+spec/deprecated-as-of-juno 18:33:01 ayoung: ++ 18:33:13 jamielennox: follow the /auth/ implementation with a deprecation patch to OS-FEDERATION? 18:33:23 dolphm: yep 18:33:47 alrighty, that's three things then, and one is already done (ayoung's) 18:34:02 and morganfainberg's is deep in review 18:34:16 #topic Email as a first class attribute 18:34:19 #link https://review.openstack.org/#/c/111982/ 18:34:22 henrynash o/ 18:34:39 dolphm: so really a couple of questions here 18:34:52 I thought we just removed the code that even tried to use it (since it would 500) 18:35:15 we seem to sort-of support it… 18:35:25 for what? 18:35:32 …i.e. LDAP mapping will fill in the email attribute 18:35:49 and the keystoneclient supports it explicitly 18:35:49 LDAP mapping can fill in any attribute 18:36:07 https://review.openstack.org/#/c/90296/ 18:36:13 bknudson: we have a specific config variable to control email attribute 18:36:28 henrynash: first of all, do we need to discuss this in the context of juno or kilo? i'd rather have this conversation in terms of kilo 18:36:32 keystoneclient talks about it by name but it just goes into the extra data column 18:36:47 teh real question is….are we already in danger of supporting PII info ? 18:36:52 right, keystone itself doesn't have first class support for email 18:36:54 btw the docs also say that you can filter on email still 18:37:13 so wondered if we should get a security viewpoint on this 18:37:20 henrynash: that's a good argument in favor of not supporting email :P 18:37:24 hanrynash, pii according to which compliance? 18:37:34 gyee: all of the compliances 18:37:40 docs change: https://review.openstack.org/#/c/115330/ 18:37:56 (it's the wadls) 18:38:28 bknudson: all our user entity exampels in the spec also include email 18:38:29 henrynash: email is traditionally in the PII realm 18:38:37 bknudson: +1! 18:39:27 dolphm: ione option is to pull email from teh idenity_api spec in all places 18:39:37 henrynash: i'd be in favor of that 18:39:46 unless there's a strong use case behind the spec 18:39:47 so what's the proposal? move it from extras column to an email column? 18:40:17 bknudson: the current proposal is just to add a filter for email - which i'd argue does require making it a real column, yes 18:40:26 bknudson: no, there’s noneed for that…this was kciked off by looking to impelemt what the spec said taht you can filter on email 18:40:50 dolphm: technicallyit doesn’t required it….but if it gest used a lot, then you need to 18:41:09 but you can stash anything in "extra", PII or not 18:41:09 henrynash: but it's not actually documented as an attribute on the user resource in the spec -- it's just copy pasta in the examples 18:41:16 I'm worried about giving someone the option to do something that's going to be very inefficient 18:41:31 dolphm: agreed…it is not listed as an optional attribute 18:42:00 dolphm: OK…so for Juno…we pull it from the spec…but leave the client and LDAP support as is? 18:42:35 what's the client support? 18:42:46 so the implementation (using extras) would download the entire table and record by record serialize JSON to find the match? 18:42:50 bknudson: just an 'extra' 18:42:52 I think we support it in the user entity class 18:42:56 ? 18:43:38 we support anything that is returned from the server as an attribute on the user object (or any object) 18:43:41 I was wondering if the client requires email for a new user or has filtering by email or something. 18:43:50 henrynash: the client support can be undocumented & pushed into **kwargs 18:43:53 on creation it's just named in the param list but we don't treat it special 18:43:57 maybe 18:43:59 henrynash, but compliance is deployment-specific right, if one needs to encrypt/obfuscate a field, roll their own driver 18:44:53 I would think PII and API are two separate issues 18:44:53 gyee: well yes, but not sure we should ship something that by default risks security defects 18:45:18 henrynash: ++ 18:45:37 The following data clearly classify as PII: Email address, Login name, screen name, nickname, or handle 18:45:50 that's from wikipedia 18:45:52 so we are going to encrypt username now 18:45:58 bknudson: ++ 18:46:13 can we not store PII in keystone? 18:46:13 bknudson: hhmm, user name is PII and we don’t encrypt that 18:46:14 gyee, an argument to why we should go to id-only in the tokens >.> 18:46:47 security is a process, software is a tool 18:47:05 dstanek, while username is PII, you don't need to typeically encrypt it, just prevent it leaking out iirc 18:47:09 just like a knife, it can be both a weapon or tool :) 18:47:41 dstanek, same for some of the other PII items. Some do *need* to be encrypted if stored 18:48:14 so i think it's fair to say we need to have this discussion outside the scope of Juno :) 18:48:14 how do you go about drawing the line though? 18:48:25 dolphm, ++ 18:48:32 #link http://en.wikipedia.org/wiki/Personally_identifiable_information 18:48:39 henrynash: re-propose for kilo and we can continue in the spec review? 18:48:40 morganfainberg: one thing I wondered was wether we had to leave PII info out of teh collection of users returned in list_users calls 18:48:46 I guess I wonder why we would treat email specially... how about allowing filtering on any of the "extras"? 18:48:52 morganfainberg: i don't think we would store anything that needed to be encrypted at rest 18:48:53 dolphm, ++, lets talk more on this later 18:48:54 lbragstad, i think there are some explicitly listed (depending on the compliance standard) that need it 18:48:55 dolphm: OK 18:49:03 morganfainberg: ok 18:49:15 this is the doc we used to use when dealing with PII for PCI compliance - http://csrc.nist.gov/publications/nistpubs/800-122/sp800-122.pdf 18:49:18 bknudson: because i don't want to expose anything more about 'extras' to the API - extras needs to die 18:49:32 lbragstad, what dstanek said, we don't store anything (directly) that needs encryption at rest (i think) 18:49:49 since we're heading towards federation I'd expect us to leave listing users to the provider. 18:49:51 we don't need first class support for an undocumented feature that we're never going to be good at supporting 18:49:51 dolphm, ++ extras dead = good 18:50:07 bknudson: ++ 18:50:31 dolphm: the ability to store additional info with keystone entities is well used, I think 18:50:31 dolphm: i had a patch to delete it a while ago because i hated extras 18:50:36 dolphm: ++ to killing extras 18:50:38 dstanek: lol should have put it up! 18:50:44 henrynash, doesn't mean it shouldn't die :P 18:51:01 dolphm: i doubt it'll be easy to merge now, but if it is i will 18:51:02 henrynash: having it used at all means we need to be careful about removing it :( 18:51:05 henrynash, maybe support it as an extension with a big "DONT DO THIS" warning block :P 18:51:25 morganfainberg, make sure no one waits for you in the parking lot after you kill it :D 18:51:37 gyee, lol 18:51:39 henrynash: what do people use it for? 18:51:46 dstanek: email 18:51:50 i think maybe we need config option for disable_deprecated which turns deprecations into 404s to let people test this stuf 18:51:58 dstanek, metacloud uses it a lot, tracking cost centers etc, 18:51:59 jamielennox: ++ 18:52:01 bknudson: i know that one :-P 18:52:04 fatal_deprecations = True 18:52:21 raise 501's or something when you hit a deprecation 18:52:31 ++ 18:52:38 dolphm: yep, doesn't matter what just so that it fails 18:52:49 dstanek: additioal user info relating to products built on top of OPenStack/Keystone for example 18:52:51 wouuld be useful even to see what the gate was using that it shouldn't 18:53:05 jamielennox: oslo has some sort of support for that from a logging perspective, i forgot what action it took on fatal 18:53:25 alright, last few minutes: 18:53:27 #topic OSprofiler & Keystone integration 18:53:29 boris-42: o/ 18:53:34 dolphm hi there 18:53:36 #link https://review.openstack.org/#/c/103368/ 18:53:41 #link https://review.openstack.org/#/c/114856/ 18:53:55 dolphm I had to rebase python client patch 18:54:05 dolphm let me show the sample of trace that we can get with these patches 18:54:19 #link http://boris-42.github.io/ngk.html 18:54:37 ^ so this is sample of trace that goes through 3 services (nova, glance & keystone) 18:54:45 booting VM operation 18:54:46 boris-42, the assumption is that the middleware would be explicitly added to the pipeline, and not even enabled under normal circumstances, right? 18:54:59 dolphm, jamielennox: IIRC setting fatal_deprecations in the config would raise a strange exception 18:55:08 ayoung it should be enabled by default 18:55:14 boris-42, no it shouldn't 18:55:19 dstanek: it exists though? 18:55:23 security trumps profiling 18:55:36 ayoung what kind of security issues? 18:55:44 boris-42: i mentioned in one of the reviews that i think the middleware can't be enabled by default 18:55:49 * ayoung takes that question as rhetorical 18:55:58 ayoung nope 18:56:01 boris-42, its a great idea 18:56:05 ayoung I really spend half of year 18:56:08 but not on by default 18:56:12 ayoung to make it possible to keep turned on by default 18:56:13 in my view it is optional functionality 18:56:17 yep 18:56:23 optional, and easy to enable 18:56:34 ayoung it is hard to get it enabled in gates 18:56:53 ayoung end if every user will need to turn it on by hand nobody will use it.. 18:57:02 ayoung and I really don't see any big issue 18:57:07 boris, I think you will find that it gets used 18:57:07 ayoung with security 18:57:29 ayoung really what kind of security ? 18:57:41 ayoung only admin can trigger it and only admin can fetch data 18:57:49 boris-42: it's not something that will get deployed by hand, this stuff will get added to puppet so enabling across everything shouldn't be hard 18:58:06 boris-42, I take it you want data from gate? 18:58:12 ayoung yep 18:58:16 ayoung in rally perfromance jobs 18:58:20 ayoung that you have 18:58:24 boris-42, then enable it in devstack 18:58:25 ayoung so profling + benchmark 18:58:33 ayoung it will take about 1 year 18:58:38 the profiling stuff just doesn't happen if the client doesn't request it. 18:58:44 bknudson +1 18:58:50 bknudson it happens only if admin trigger it 18:58:58 bknudson and only for his request 18:59:09 boris-42, what you want is devstack, not the rest of the world 18:59:12 bknudson even if user knows secret key it won't be able to trigger data 18:59:15 i'm also not a fan of it automatically configuring itself - https://review.openstack.org/#/c/103368/10/bin/keystone-all 18:59:37 dstanek hm but I added CONF option 18:59:55 dstanek https://review.openstack.org/#/c/103368/16/keystone/common/profiler.py 19:00:07 it logs all the db queries... that's pretty neat 19:00:23 bknudson you can add in any place 19:00:28 bknudson new points 19:00:40 bknudson e.g. during benchmarking in gates put just any amount of points 19:00:44 bknudson and get them on graph 19:01:09 "statement": "SELECT 1", -- pretty exciting stuff 19:01:24 boris-42: this is a lot of copy/pasting around :( 19:01:31 dolphm ? 19:01:38 boris-42: i left review comments 19:01:46 dolphm about api-paste.ini 19:01:46 dolphm, at time 19:01:52 dolphm I would prefer to keep it in 19:02:01 boris-42: you should be pulling your boiler plate text from osprofiler so you don't have to maintain it everywhere 19:02:03 dolphm can we continue in keystone chat? 19:02:07 yes 19:02:10 ooh, we're over time 19:02:13 i had my head in gerrit 19:02:14 yep 19:02:15 #endmeeting