18:02:37 #startmeeting keystone 18:02:37 Meeting started Tue May 20 18:02:37 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:38 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:40 The meeting name has been set to 'keystone' 18:02:55 welcome back from the summit :) 18:03:10 hopefully everyone besides btopol is decompressed enough for a meeting 18:03:26 #topic identity-specs repo coming soon 18:03:49 so to kick off juno for real, we're starting an identity-specs repo, similar to nova-specs 18:03:51 #link https://review.openstack.org/#/c/94119/ 18:03:57 I'm assuming anything we have in flight now requires an identity-spec 18:03:59 morganfainberg: IIRC, we should have it available this friday? 18:04:02 any bps 18:04:09 bknudson: yes and no... 18:04:19 dolphm, that is the idea. by next week i'll have the basic structure in place 18:04:22 going forward with this eh 18:04:27 we can introduce a hard requirement for -specs starting in j2 18:04:40 dolphm, or well by monday of next week i should have all the supporting structure (like nova has) up for review 18:04:51 or so 18:05:01 so if your feature is in flight, you have some hope of avoiding writing a full blown spec... and thus some encouragement to land your feature early in the juno cycle :) 18:05:28 still, -specs highly appreciated for all blueprint-level changes 18:05:33 dolphm, ++ 18:05:34 #info Mandatory for feature proposals juno-2 onward 18:05:36 I made a first hack of what the content will be based on the nova one 18:05:39 and... 18:05:40 #info Propose a feature spec first, and when it's approved/merged, keystone-drivers will create and manage a corresponding blueprint in LP 18:05:51 so, basically only keystone-drivers will be creating BP's 18:06:02 dolphm, does this mean we will be evicting the current mess of BPs? 18:06:03 how does -specs affect client? 18:06:03 #link https://github.com/admiyo/keystone-specs/ 18:06:06 from LP that is. 18:06:26 feel free to clone and we can resolve differences when the real one gets up and running 18:06:43 morganfainberg: for the most part, that's what i'm thinking (mark them as obsolete?) not sure what the best approach is, or if it should be case by case 18:06:50 dolphm, ++ 18:07:03 dolphm, case-by-case but most should be obsolete 18:07:16 jamielennox: nova's template includes a client impact section, but i'd like to have a discrete dir for client-specific work 18:07:26 BP should not be obsolete if the work will still happen, but maybe someone to indicate "neeeds full spec" 18:07:49 dolphm: ok 18:08:03 ayoung, so we need to figure out the correct status. 18:08:05 jamielennox: but since the release naming doesn't apply to the client, we'll have to create separate versioned directories as we go 18:08:12 does keystone spec have a cross-project impact section? 18:08:18 gyee, it can. 18:08:33 gyee, i'll make sure there is a section for that 18:08:40 morganfainberg, ++ 18:08:49 we could mark them as obsolete and ask for the person who proposed it to implement a full spec. That should weed out old or non-important stuff 18:09:11 nkinder: that's a good diea 18:09:12 morganfainberg: i'd like to simply reference nova's template rather than ever maintaining our own 18:09:14 How about "if it does not have a full spec, it stays "new" and "needs review" means it has a real spec? 18:09:19 +1 to nkinder 's plan 18:09:32 nkinder: that's what i was thinking 18:09:42 nkinder: maybe stick a warning in the whiteboards until j2, and then do that? 18:09:51 hopefully that'll provide sufficient notice to those not here 18:09:56 yeah, sort of a deprecation period 18:10:01 nkinder: ++ 18:10:26 Tag as "review" once there is a real spec, I think....ignore anything that is "new" and weed out any that stay new for past j2? 18:10:33 morganfainberg: do you have a security impact section in the template? 18:10:53 nkinder, haven't built the template, but i belive that nova has that section, and if not, i want it 18:10:55 tell people to move to "drafting" if they are actively working on it 18:11:01 and fair warning... nova's experience with switching to -specs was that the community wasn't particularly good at doing design work up front. we've got some experience with identity-api, but i assume we'll see the same pain. so if you want to land a j2 feature, start working on the spec asap 18:11:04 morganfainberg: yeah, they added one IIRC 18:11:38 #link https://github.com/openstack/nova-specs/blob/master/specs/template.rst nova's spec tempalte 18:11:57 ++ 18:12:07 there is no cross project impact section, but i think that would be valuable for nova as well 18:12:09 gyee: ^ 18:12:19 dolphm, ++ 18:12:20 The security impact portion of the template is pretty decent 18:12:44 there will also be a python check job that will ensure all sections exist 18:12:50 (similar to nova's) 18:12:52 here it is slightly modified for Keystone https://github.com/admiyo/keystone-specs/blob/master/specs/template.rst 18:12:56 dolphm, yes, I would imagine nova have some dependency on others as well 18:13:14 We need to plus up the JSON-SPEC thing... 18:13:24 JSON schema 18:13:27 (oops, sorry I’m late) 18:13:30 how does the assignee stuff work? as things change will a new review need to be submitted? or do we not care? 18:13:34 their API section overlaps our use of identity-api as well 18:14:06 do we have separate spec directory for API as well as client? 18:14:11 ayoung, performance impact, nice! 18:14:14 dstanek: i had the same question for the nova folks -- but i'm not sure they've been doing this long enough to have an answer yet 18:14:29 bknudson, I'd suggest no 18:14:30 bknudson: we'll still have identity-api 18:14:32 bknudson, i am not sure we need it for API, we already have the identity-api repo 18:14:56 the REST API impact section could just link to one or more identity-api reviews :) 18:15:04 dolphm, ++ 18:15:06 ++ 18:15:21 bknudson, instead, under specs, we can have client subdir parallel with juno .... 18:15:21 especially if the identity api docs need updates 18:15:49 the spec should write the identity-api 18:16:21 does this mean we stop creating new blueprints? 18:16:29 pretty much everything you need to update the identity API is in the spec template, so don't approve a bp until the spec is close enough that you could at least submit an API review 18:16:32 stevemar: for everyone but keystone-drivers, yes 18:16:44 stevemar, and only create the BPs for approved specs 18:16:54 stevemar: when a spec is approved and merged, keystone-drivers will create a corresponding BP to track the work 18:16:54 gotcha 18:17:05 ++ 18:17:14 i think most of these details we can work out in review as we get into the process, etc 18:17:14 is there a way to lock down LP so only keystone-drivers can create BPs 18:17:16 ? 18:17:38 stevemar: i asked that question as well, and i think the answer is no :( 18:17:58 stevemar: the bp priority attribute is the only one that is "protected" afaik 18:18:04 the only one we use, anyway 18:18:11 even "Approved" can be set by anyone 18:18:23 lame 18:18:26 dolphm, so we're not really going to fix the clutter in LP 18:18:31 or maybe i'm thinking of Approver 18:18:32 we'll probably have to keep tabs to make sure any new bps coming in get redirected to specs. 18:18:42 stevemar: in the long run, i think we will 18:18:46 * morganfainberg wants storyboard. 18:19:33 why even have blueprints it the info is in git? just to link against? 18:19:50 dstanek, release management 18:20:07 dstanek, all the release management stuff still leverages LP 18:20:26 and we should still be able to tag bp in commits. 18:20:26 morganfainberg: ++ 18:20:59 the agenda is quite long today, so let's move on 18:21:03 #topic Potential Keystone Hackathon dates in San Antonio, TX for Juno 18:21:04 morganfainberg: ah, that sucks. hopefully that will change once everyone adopts this new methodology 18:21:19 #link http://doodle.com/s4g7mm9n7qyu9inh vote for dates here if you're planning on attending 18:21:21 i think this will make it harder for new contributors ... (d'oh topic has changed) 18:22:19 Just to motivate you all: if we make it July 18th, you have to help me celebrate my Birthday 18:22:21 wow, already a clear preference for 9, 10, 11 18:22:32 that was quick 18:22:53 (give a core a button to click on…and they will) 18:23:03 lol 18:23:13 henrynash ++ 18:24:03 If you give a core a button, he is going to want a review to go with it. If you give him a review..... 18:24:12 lol 18:24:20 I've read too many of those books... 18:24:42 the people here with kids get it. 18:25:38 i'm good for either date - i originally thought PyOhio would overlap, but I'm good there 18:25:39 i'll let this run until the end of the day or so, before i pursue event space more aggressively, etc 18:25:47 ++ 18:25:54 Pagination and Filtering? 18:25:56 dstanek, PyOhio? 18:26:14 hoping for http://www.geekdom.com/san-antonio/events or as a backup, rackspace headquarters again 18:26:45 rax headquarters was nice enough 18:26:57 * lbragstad wants to try the slide 18:27:09 nkinder, we are not looking to piggyback the Security group meetup with the keystone one, right, just Barbican? 18:27:13 two geeks enter one geek leaves 18:27:17 oh, this'll also likely be a joint event, overlapping for a day or so with a barbican hackathon 18:27:28 ayoung: ++ 18:27:37 OSSG was talking about piggybacking, but I think it would be too much 18:27:46 we briefly discussed M-Tu-W barbican, W-Th-F keystone in the same space 18:27:59 I also heard August floated. 18:28:13 ayoung: wondering what is the state of pagination is at the moment as a follow-up to 18:28:13 this discussion a few months back - 18:28:14 http://openstack.10931.n7.nabble.com/keystone-Pagination-td17386.html 18:28:22 gyee http://www.pyohio.org/ 18:28:23 OSSG wanted it nearer to the release data IIRC 18:28:25 I was thinking August for some cross-project security stuff around Jamie's common client, but August might be too late in the cycle 18:29:01 clu_, I still think pagination is a UI antipattern 18:29:04 ...that's not really OSSG though, but more of a cross-project developer hackfest around security 18:29:27 30 minutes and 3 topics left :) 18:29:30 #topic Pagination and Filtering 18:29:37 filtering then? 18:29:49 so, there was a summit session on cross-project API consistency, wherein pagination and filtering was a highly relevant topic 18:30:04 dolphm: ahh, didn’t see that one 18:30:10 and the outcome was that we need a TC-blessed API conventions repo/document that spans cross projects 18:30:18 pagination is like a religion around here, extremist at both ends 18:30:29 i threw my name down on a list of people to get that going 18:30:33 dolphm, imagine my surprise 18:30:38 dolphm, the tc bit that is 18:30:53 +++ (to oblivion) dolphm 18:30:59 pagination and filtering of what? 18:31:16 every list api? 18:31:32 bknudson, any place pagination/filtering is implemented 18:31:37 bknudson, it has to be consistent across the board if we are going to do it 18:31:39 or is wanted. 18:31:49 i was in that session, there was lots of talk of needing it - and no actual resolution to how it was to be done 18:31:58 how do we know where it's needed? 18:31:59 having it for some resources, like domains seems silly 18:32:02 are you signing up for another debate? 18:32:04 projects? 18:32:15 role assignments? 18:32:16 same with version discovery, extension discovery, etc 18:32:17 but with regard to pagination specifically, the only voice (that i was aware of) in the room in favor of a specific approach to pagination (jaypipes=marker/limit), agreed that server-specified next/previous links were generally more reliable across various backends (which is the problem that keystone has with ldap, sql, nosql, etc) 18:32:30 we will hopefully not have users and groups and those are the obvious ones 18:32:30 they are really stack-wide initiatives that should be driven by the TC 18:32:33 bknudson: so when I implemented a version of this back in the day (Grizzly?), it pushed pagination into teh backends the way we pushed filtering into teh backends, using the driver_hints concept 18:32:46 dolphm: agreed. 18:33:02 s/more reliable/more implementable/ ? 18:33:19 most likely to not be unimplentium 18:33:35 we could implement it would just be very slow anyways 18:33:52 yeah... 18:34:03 so not viable with any real scale 18:34:31 dolphm: if we do decide to do it, I’m up for doing the implementation, since I attempted it before 18:34:33 prev & next links are the right way to go 18:34:37 but slow is not an API thing, its a backend thing 18:34:57 so deployer have the options for different backends 18:35:21 gyee: the problem with enforcing expectations of a specific approach like marker/limit is that it's exposing implementation details to the client, so an interface problem becomes a performance problem 18:35:51 if you want to be able to scroll to any page in a large table then following next links is going to be slow 18:36:07 dolphm, no where in the API spec mention anything about peformance 18:36:18 or expected performance 18:36:19 dolphm, I like your approach: we return 200 results max. Select 201. If we get 201 "more results, please filter" 18:36:34 it has always been a deployer enhancement 18:36:41 bknudson: even the horizon guys agree that filtering is a better solution that a billion direct links to pages 18:36:45 bknudson: and if you really ARE having to scroll lots an dlots of pages, you’re probably screwed anyway 18:37:08 what type of filtering is in place today? 18:37:31 clu_: I think we’re in good shape with filtering as far as the SQL backend is concrened 18:37:40 that's for GUI access - what about building tooling? i may really want a list of all 1000 of a customer's projects 18:37:46 clu_: quite a bit https://github.com/openstack/identity-api/blob/master/v3/src/markdown/identity-api-v3.md 18:38:11 LDAP in general supportes pretty good filtering, but I have not tested our backend code with it. I have a pretty hefty user-DB I can load it up with and see... 18:38:23 i think filtering is a better solution in most cases, but can't replace everything 18:38:30 we could do approximate filtering with LDAP 18:38:36 I thought jamielennox had a presentation at the summit about performance with LDAP? 18:38:39 dstanek, why would you want that paged? 18:38:42 ayoung, LDAP is supposed to be optimize for lookup, if your LDAP is slow, always blame the configuration :) 18:39:03 gyee, I hear HP is really good at optimizing slow LDAP queries from Keystone. 18:39:07 lbragstad: wasn't me 18:39:18 ayoung: :-) 18:39:27 lbragstad, it was GoDaddy, AD != LDAP 18:39:30 ayoung, I'd blame HP LDAP too 18:39:45 ayoung: paging is usually easier for the server (especially in cases where the process could monopolize the CPU) and it makes caching possible 18:39:52 henrynash, dolphm: wow, did not know that :) nice 18:39:56 ayoung: jamielennox yep 18:40:09 filtering support/performance in LDAP is generally good, but you need to do things like define indexes on the LDAP servers (which is implementation dependent) 18:40:26 20 minutes, 2 topics remaining :) 18:40:34 timeboxing 18:40:44 clu_, to be clear, pagination with LDAP is a bad thing... 18:40:45 clu_: …..and for SQL we pass the filter into the SQL query itself, but for ldap (today) we post-process the filer 18:41:00 LDAP does not return results in the same order for the same query 18:41:14 so you need to maintain a connection...which quickly can overwhelm an LDAP server 18:41:16 SQL doesn't have to return results in the same order 18:41:23 henrynash: but we expose the desired filtering to the ldap driver, which ignores them, correct? 18:41:32 for pagination, its really about API consistency, implementation is a different matter 18:41:34 dolphm: yes 18:42:05 bknudson: you can ensure that by asking sqlalchemy to enforce that 18:42:18 so, it sounds like we need to write a -spec for ldap filtering, and pursue the cross-project api conventions repo (ping me later if you'd like to be involved in that effort) 18:42:24 #topic Split of middleware (auth_token, etc) out of keystoneclient to separate repo 18:42:25 morganfainberg: o/ 18:42:32 dolphm, ++ 18:42:34 o/ 18:42:44 so i think generally speaking there is a desire to split the middleware from ksc. 18:42:44 dolphm: pick me, pick me! 18:42:51 Don't do it! 18:42:59 You will have to change everyones config file 18:43:08 ayoung, there has been more and more requests to make this happen. 18:43:14 morganfainberg: someone also suggested building two packages out of the same repo, with different deps... is that viable? i feel like that's been shot down by infra before 18:43:22 what is the justification for that request? 18:43:23 dolphm, i don't think that works well 18:43:25 or maybe that was when openstack built its own packages 18:43:38 ayoung, limiting extra requirements for the ksc installs. 18:43:45 it can be done via RPM etc, i don't think it works well for pip packages 18:43:53 jamielennox, ++ 18:43:54 dolphm: that's a bad idea and a pain long term 18:43:58 jamielennox: ++ i bet that was it 18:44:05 that is why the repo is called python-keystoneclient...pip required that 18:44:12 isn't there an python-openstacksdk repo already, that's for all the clients right? 18:44:14 dstanek: fair enough, i'm not advocating for it, just echoing a seemingly lazier solution 18:44:25 so there are two things, 1st you are going to have a circular dependency between the two libs 18:44:30 lets discuss why...what do you think we are going to limit by splitting auth token from client: 18:44:34 it would be very confusing me thinks 18:44:36 so keystoneclient, the CRUD part, may eventually end up in there right? 18:44:48 2nd if we are going to make people change configs like that then let's fix it first 18:45:03 ayoung, it is to make it so we can release auth_token without KSC as well as limit deps on both sides. 18:45:12 #link http://git.openstack.org/cgit/openstack/python-keystoneclient/tree/keystoneclient/middleware/auth_token.py#n146 18:45:25 this would be the time to get rid of all those deprecations and just general crap 18:45:33 ayoung, and i think those two arguments are valid alone. security issue in auth_token, release a new one doesn't affect KSC 18:45:46 directly that is. 18:45:53 from oslo.config import cfg is used by the CLI and by ATM, so if we split, and drop the CLI, we drop that one 18:46:20 from keystoneclient.middleware import memcache_crypt 18:46:36 the #1 request is to remove memcache from the dep in ksc 18:46:42 OK, I can see that 18:46:53 i'll be doing that w/ dogpile, but no reason to add dogpile to ksc 18:46:57 so ksc can't depend on auth_token? 18:47:04 bknudson, correct. 18:47:09 because if it does than ksc will still require all the deps 18:47:19 bknudson, dep should be the other way round 18:47:22 so we'd copy-paste auth_token rather than import it? 18:47:31 for backwards-compat 18:47:33 bknudson, ah yeah. 18:48:00 or do we import it without the dep? 18:48:14 how do we do that? 18:48:22 no copy-paste. that bad 18:48:33 we'd have to attempt to import and fail if it couldn't be imported 18:48:35 ayoung, circular imports = bad. 18:48:54 like crossing the streams 18:49:12 so this new repo...python-keystonemiddleware? 18:49:21 just keystonemiddleware i'd assume 18:49:28 does it contain the other middleware? 18:49:32 dolphm, ++ 18:49:33 bknudson, it could. 18:49:34 bknudson: yes 18:49:34 bknudson, yeah, s3 18:49:45 http://git.openstack.org/cgit/openstack/python-keystoneclient/tree/keystoneclient/middleware 18:50:07 morganfainberg: let's write a spec for this to work out what belongs where if we introduce a new repo? 18:50:11 There is an ec2 version, too, which I think is still in Keystone 18:50:13 I think people are still importing s3_token middleware from keystone 18:50:16 dolphm, sounds good 18:50:26 morganfainberg: and by "let's" i mean "you" 18:50:29 morganfainberg, that cross-project section will be long :) 18:50:33 dolphm, ++ 18:50:37 http://git.openstack.org/cgit/openstack/keystone/tree/keystone/middleware/ec2_token.py 18:50:42 #action morganfainberg to write a -spec to introduce keystonemiddleware repo 18:50:51 #topic Add LDAP connection pool 18:51:00 #link https://bugs.launchpad.net/keystone/+bug/1320997 18:51:01 Launchpad bug 1320997 in keystone "Identity Ldap driver connection pooling" [Undecided,New] 18:51:04 gyee: o/ 18:51:18 I thought this was already proposed. 18:51:19 what do you guys think about this https://pypi.python.org/pypi/ldappool/1.0 18:51:42 bknudson, is there a review already? 18:52:02 So long as it is optional, I think we should provide a way to test out pooling. 18:52:05 oh, maybe I just saw the bug 18:52:12 bknudson: ++ i recall connection pooling at some point 18:52:19 bknudson: brand new bug 18:52:28 the LDAP pooling story might be different for Eventlet and Apache. nkinder ? 18:52:38 k, we have some poc code and it seems to give us 30% improvement 18:52:55 my googling is not turning up any prior art 18:53:27 #link https://pypi.python.org/pypi/ldappool/1.0 18:53:30 only issue is that ldappool doesn't support client cert at the moment 18:53:31 ayoung: for using httpd to perform auth up front, sssd would handle pooling 18:53:43 but I don't think 2-way SSL is required 18:53:49 gyee: sounds like an upstream issue that can be solved :) 18:53:55 dolphm, yes 18:53:57 nkinder, what about existing LDAP approach? 18:54:09 ldappool is something that Mozilla developed, which is the only Python implementation that I know of 18:54:22 I can do a pull request for ldappool 18:54:30 nkinder: is it in use? basically zero downloads on pypi 18:54:41 ayoung: We could have httpd use PAM for auth, which will call into sssd and go against LDAP 18:54:48 dolphm: I've never seen it used 18:54:56 only heard about it via google searching 18:55:13 worth evaluating at least 18:55:23 ayoung: you mean support for pooling via our “LDAP driver underneath keystone identity” approach? 18:55:30 henrynash, yeah 18:55:43 ayoung, I can make it configurable 18:55:49 ayoung: which I suspect will contiue to be the standard way for a while 18:56:06 standard = most used 18:56:07 why bother if we can just leverage httpd/sssd? 18:56:41 nkinder, is apache in devstack for keystone already? 18:56:46 I agree with henrynash that the LDAP driver will be the standard way for a while, but the more effort we put into it the less we put towards a better long term approach 18:56:50 nkinder, cuz we are not there yet. But I agree, that is the right long term approach. 18:56:59 gyee: yes, I believe so 18:57:18 gyee: yes 18:57:26 ayoung, nkinder, I totally agree, but unless we standardize apache for keystone, we need something to improve performance right now 18:57:29 nkinder: agreed, it wil be a balance, how much to we “backport” to the old way of doing things, standard dev problem! 18:57:32 i don't think we gate on httpd anywhere, though? 18:57:34 nkinder, could the OS provide us some sort of LDAP connection pool? 18:57:40 gyee: APACHE_ENABLED_SERVICES+=key 18:57:43 nkinder: +++ 18:57:54 dolphm: afaik we do not 18:58:08 we should gate on httpd though 18:58:18 thanks nkinder I was wondering the same 18:58:20 nkinder, morganfainberg is working on it 18:58:23 dstanek: that's going to be super important, as we can't currently gate on a federation configuration otherwise 18:58:30 ayoung: I know 18:58:42 we should gate on httpd, and we should gate on LDAP 18:58:49 nkinder, working on both 18:58:50 nkinder: ++ 18:58:54 gyee, dolphm: I think at some point we are going to have to decide if we mandate apache for keystone or not…I’m a little leary of doing it, but I can see the advantages starting to stack up 18:58:54 morganfainberg: ++ 18:58:58 morganfainberg: link or anything? 18:59:05 * dolphm 2 min 18:59:05 +1 to gating on LDAP 18:59:12 we currently have non voting job 18:59:14 henrynash, I totally agree 18:59:16 dolphm, the LDAP one is not setup yet, working on apache_services now 18:59:21 dolphm, we already have a non-vote job 18:59:31 enfranchise it 18:59:32 dolphm, and compressed tokens should help fix the failures there 18:59:34 henrynash: i don't see any advantages for *not* using httpd, do you? 18:59:49 except for pki tokens being large :) 19:00:00 check-tempest-dsvm-full-apache-services right morganfainberg ? 19:00:02 dolphm, henrynash, if we mandate httpd, we shoudl ensure we can use unicorn/nginx/uwsgi/etc 19:00:03 I am all for apache 19:00:06 dolphm, henrynash, not just apache. 19:00:08 ayoung, yes 19:00:09 dolphm: I think it’s more of a configuration in people’s products and/or deploymetns prolems 19:00:11 how soon can we get it in there? 19:00:17 it's easier to keystone-all than set up apache 19:00:29 morganfainberg: ++ but i'd like to choose a single container for doc purposes 19:00:31 and testign 19:00:32 bknudson, its easier to secure apache than keystone-all, though 19:00:40 s/container/server/ 19:00:43 time! 19:00:54 #endmeeting