18:03:19 #startmeeting keystone 18:03:19 Meeting started Tue Apr 15 18:03:19 2014 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:03:20 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:03:23 The meeting name has been set to 'keystone' 18:03:35 #topic icehouse coming soon 18:03:42 Winter is coming 18:03:44 dolphm, winter is coming? 18:03:54 i hope not 18:03:57 ayoung, damn it beat me to it 18:03:59 we just had winter. let's have some spring for once 18:04:05 hi 18:04:17 In new England you can get all four seasons in one day 18:04:25 the RC seems to have settled down, we haven't had reason to open a new RC window, so our current RC will become stable/icehouse in < 48 hours 18:04:33 ++ 18:04:36 it's going to snow here tomorrow 18:04:37 yay!! 18:04:42 wonderful 18:04:49 it snowed here today :( 18:04:49 +++++ 18:04:59 awesome work everyone! 18:05:16 Time to focuse on client reviews 18:05:33 ayoung: ++ :) 18:05:52 it's snowing there right now 18:05:56 s/there/here/ 18:06:00 there's several client reviews with +2 already -- https://review.openstack.org/#/q/status:open+project:openstack/python-keystoneclient+label:CodeReview%253D2,n,z 18:06:12 fwiw, there's a few icehouse-rc-potential bugs/fixes that we'll need to backport to stable/icehouse rather than milestone-proposed since we don't have an RC window open 18:06:52 #topic Reminder: Design summit session proposals open until April 20th 18:07:00 #link http://summit.openstack.org/ 18:07:09 we're at 24 proposals at the moment, with 8 timeslots available 18:07:33 20 minutes per proposal 18:07:46 so there's going to be a lot of cuts & merges already, but if you're confident the current proposals miss something that's critical to discuss, you still have time to propose it! 18:08:17 I'm assuming the merges will be a topic for next week? 18:08:25 likely 18:08:38 no point in merging now when things can still be proposed 18:09:06 sounds good, I presume we all need to vote on them before next week then? 18:09:14 once the window for proposals is closed, i'll look to produce a draft schedule ASAP 18:09:25 gyee, i dont think there is a vote ... 18:09:36 There is one vote. the PTLs 18:09:40 I thought dolphm votes for all of us 18:09:41 PTL's vote 18:09:44 stevemar, like adding our comments 18:09:46 gyee: there's not really a vote, but your feedback is important :) 18:10:20 dolphm, what say we go through the proposals and add comments like "merge this one with " 18:10:22 dolphm, sure I'll add feedbacks 18:11:02 ayoung: you're welcome to 18:12:42 #topic Oslo Liaison for Keystone 18:12:49 #link http://lists.openstack.org/pipermail/openstack-dev/2014-April/032498.html 18:13:11 i haven't followed this conversation very closely -- who put this on the agenda? 18:13:16 i did 18:13:24 I'm willing to sign up to be the liaison if we need one 18:13:40 +1 for bknudson 18:13:41 they are looking at holding weekly oslo meetings again, but it's at midnight for me 18:13:51 from the wiki, "The liaison should be prepared to assist with writing and reviewing patches in their project as libraries are adopted, and with discussions of API changes to the libraries to make them easier to use within the project." 18:14:06 bknudson and morganfainberg are the ones usually on top of the oslo syncs 18:14:08 so the liason also needs to attend a Friday 1600 UTC weekly meeting? 18:14:14 * topol cant believe bknudson volunteered for this... 18:14:21 liaison* 18:14:22 bknudson, I'm already the policy reviewer 18:14:26 dolphm: i don't think it's a requirement - but i'd suspect it's useful 18:14:39 i'll do it if nobody else wants to or can attend the meetings 18:15:05 I'd be comfortablwe with either of you guys 18:15:18 +1 I second that 18:15:41 who wants it??? 18:15:49 bknudson is the starter and the rest of us are in the bullpen :) 18:15:54 ++ 18:15:58 ++ 18:16:16 should we vote on bknudson vs dstanek? lol 18:16:16 if I'm not the liaison I'll be happy to do the reviews 18:16:37 bknudson, primary, dstanek alternate? 18:16:44 * topol like volunteering to be the caddy for Judge Shmeals in Caddyshack 18:16:45 since bknudson spoke first 18:16:52 Noonan! 18:16:57 ayoung: that sounds good to me 18:17:10 ok, I'll update the wiki page 18:17:28 morganfainberg: btw, you almost got voluntold by stevemar 18:17:41 Heh 18:17:42 i'm learning from topol 18:17:48 rofl 18:17:48 * gyee is looking up voluntold 18:17:59 stevemar, in order to be able to do that you need to be in the volunteers chain of command 18:18:15 nah.. just act like you are 18:18:19 topol: +++ 18:18:27 dolphm, oh dear 18:18:31 topol, kmows it 18:18:35 stevemar, hey no volunteling me for things! :P 18:18:45 morganfainberg, too late 18:18:51 well that was easy 18:18:53 morganfainberg: you just opened the flood gates 18:18:57 ayoung, heh 18:19:24 my first official act as oslo liaison -- please review https://review.openstack.org/#/c/83966/ 18:19:25 bknudson: thank you for your volunteerism! 18:20:05 dstanek is exempt from the request because he already reviewed it 18:20:31 nice, usually i'm the one late to the party 18:20:57 bknudson, thats a biggie compared to most of the oslo syncs 18:21:18 Can we please stop the copy-paste-MADNESS! 18:21:20 I was hoping for the two line change :-) 18:21:28 topol: this is a full sync of everything we use in oslo-incubator 18:21:35 and gets rid of things we don't use 18:21:40 bknudson, gotcha 18:21:55 ayoung, what copy paste madness? 18:21:57 ayoung: I'll bring your concerns to the oslo group... I think that's the plan though 18:22:00 when did we start using threadgroup? 18:22:02 bknudson, ++ on removing things we're not using 18:22:26 ayoung: I think things got brought in due to the crazy deps between projs. 18:22:27 i greatly look forward to the no-more-copy-paste days of oslo 18:22:28 but i think it's a long ways off 18:22:32 it might have come from rpc 18:22:35 bknudson: merge failed 18:22:45 #topic open discussion 18:22:47 bknudson, ugh 18:22:56 dolphm: will work on it... must have picked up another change. 18:22:59 dolphm, LOL at your comment. 18:23:14 i have one: https://bugs.launchpad.net/python-keystoneclient/+bug/1302970 18:23:15 Launchpad bug 1302970 in python-keystoneclient "middleware provides no way to know if a catalog is v2 or v3" [High,Triaged] 18:23:26 keystone/openstack/common/service.py 18:23:33 Has everyone seen this and/or have a good idea how to solve it 18:23:47 (came up again now in channeL) 18:23:56 can auth_token convert to v2? 18:23:56 jamielennox: you see this in nova if you switch auth_token to v3 18:24:03 dolphm: yep 18:24:20 bknudson: sort of, with a bunch of edge cases 18:24:22 there are others that either have the same problem or should do and skip it 18:24:31 that are unsolveable 18:24:33 jamielennox, we have logic in the provider that tests 18:24:35 token provider 18:24:44 that tests the token for v2/v3 ness 18:24:45 could middleware provide a service catalog object? 18:24:52 also, should middleware be requesting no catalog? 18:24:58 and that logic is getting ported to the client in the revoke code 18:25:16 https://review.openstack.org/#/c/81166/9/keystoneclient/contrib/revoke/model.py 18:25:20 bknudson: it could - long term i would like it just to provide a session 18:25:31 bknudson: but so far everything that we pass via auth_token is a string in a header 18:25:34 jamielennox: I like that idea. 18:25:39 agree 18:26:09 bknudson: nocatalog is useful for actually authenticating the auth_token admin user - i don't think so beyond that 18:26:44 I guess there are times where a catalog might be useful -- e.g., nova talks to neutron using the user's token, so it should get the neutron endpoint from that token. 18:27:04 bknudson: right - that's what we've been telling people to do for ages 18:27:07 the idea behind nocatalog is that we would shrink token sizes, while providing the catalog through a separate call, e.g. GET /v3/catalog 18:27:11 bknudson, especially if we want to eventually do things like enforcing endpoint usage based upon catalog 18:27:29 bknudson, but ... i would say there is a fine line between if that is where we want to go. 18:27:39 so we *can* convert the v3 token to a v2 catalog which i guess we probably should for the short term 18:27:42 catalog should be in the token, but it should be a subset of hte endpoints: only those that the token is valid on 18:27:43 so auth_token could still provide a catalog, even if all tokens are generated & validated with nocatalog 18:27:44 but it's not a great solution 18:27:57 ayoung, doesn't solve token size issues. 18:28:17 morganfainberg, different issue 18:28:22 we could have GET /v3/catalog with X-Subject-Token 18:28:23 ayoung: ++ 18:28:29 morganfainberg, token size was postponed due to cutting data out of the catalog 18:28:36 so we get a little breathing room there 18:28:36 ayoung, ah right 18:28:44 I need to close the loop on compression, though 18:28:51 bknudson: that'd work 18:28:52 and tyhat is going to be Juno thing 18:28:53 bknudson: but that doesn't solve how do we provide it to services 18:29:02 ayoung, we could also make the services get a list of endpoints and have the token provide endpoint ids. 18:29:16 ++ 18:29:35 ayoung, rather than encoding all the data (long term) in the token. with a large enough set of cataloged endpoints, we will still (Even with compression) run into issues with the size 18:29:45 morganfainberg: that's intruiging... 18:29:57 and someone will want a bunch of endpoints... especially if we end up doing the service scoped tokens. 18:30:11 morganfainberg, ++ 18:30:13 since it opens the catalog to non-OS things 18:30:30 morganfainberg: ++ 18:30:30 creatre a n "endpoints" collection separate from the service catalog, only has ids in it 18:30:36 but... 18:30:36 ayoung, yep. 18:30:51 morganfainberg so whats the flow on your approach? First ask for what and then get what after that? 18:30:53 morganfainberg: you *could* extend pattern to "tokens only contain ID's" -- you have to GET and cache what all the actual objects are if you care about other attributes 18:30:54 won't work if you want to say "all endpoins for this service" 18:31:00 tokens would be as small as possible 18:31:05 jamielennox: my proposal for that we provide a best-effort v2 catalog if we get a v3 under the normal catalog key, then have a new v3_catalog key with the v3 catalog 18:31:07 dolphm, that was where i was eventually going 18:31:41 ayoung, we could some up with a nomenclature for that. 18:31:54 morganfainberg, I could see something like a collection of services, and each with either a specific endpoint or a wildcard 18:32:04 ayoung, yep. 18:32:04 specific set of endpoints, I guess 18:32:15 jamielennox: and then we actually have to work with the other projects to get their code using the v3_catalog 18:32:17 bknudson: i think we have to do a best-effort v2, i don't want to do a v3_catalog specifically but something where you can tell / use ServiceCatalog.factory() on directly to query 18:32:33 with the move to ephemeral tokens i am expecting token versioning to split from API versions, this would be part of token v4 18:32:34 jamielennox: I like that, too. 18:32:55 bknudson: i'm always trying to stay wary of not having to do this again if we go to a v4 token format 18:33:06 or something similar. 18:33:16 morganfainberg: agree 18:33:31 can we inject a version into the older versions of the tokens? 18:33:32 would it make more sense to solve the catalog problem in oslo then? 18:33:41 ayoung, we already do. 18:33:46 ayoung, or well we are supposed to. 18:33:56 ayoung: what do you mean version? 18:34:00 a version of what? the token itself? 18:34:06 ayoung, i'll make sure we do in the next few patches that do conversion towards ephemeral tokens 18:34:08 gyee: no, this is a keystone issue and should be solved in keystoneclitn 18:34:16 dolphm, correct the token should have the token version embeded in it 18:34:28 dolphm, it allows us (and services) to know what to expect when validating 18:34:41 dolphm, notably we can test and gate on preventing token data creep 18:35:01 dolphm, especially moving towards hyper-small tokens (ids only, yes i'm a fan of this even more now) 18:35:19 morganfainberg: we had that and phased it out 18:35:34 jamielennox, we did? 18:35:35 ID-only tokens would be significantly less compressible 18:35:36 UUIDs 18:35:49 morganfainberg, how do you cache the role grant info if Ids only? 18:35:57 jamielennox, no the token data would have only ids in it 18:36:10 oh endpoint ids in the catalog 18:36:13 dolphm, the idea would be that compressions wouldn't be as important that way. 18:36:28 topol: the assignment relationship could still appear in the token, but the project name and role names would be excluded, and you'd have to GET them separately 18:36:31 jamielennox, correct, and move as much towards id-only for anything in the token data. 18:36:40 morganfainberg: *as* important, but still beneficial! 18:36:58 dolphm, yep. 18:37:00 morganfainberg: i'd be interested to see how far you wouuld want that to go - with roles and other things 18:37:06 dolphm, are the names so long its worth requiring an extra round trip to get those? 18:37:09 jamielennox: why not all the way? 18:37:27 topol, it falls into the category of only putting relevant data in the token 18:37:31 topol: but it's a cacheable round trip 18:37:31 dolphm: i mean what does that entail? not providing username and details just a user_id? 18:37:33 topol, is there a reason to put the name in there? 18:37:41 jamielennox: right 18:38:06 dolphm: that would break on any sort of federated case so user is out but interesting 18:38:29 jamielennox, there is evaluation that needs to be done on any item that is ID only, but as far as we can possibly take it 18:38:30 morganfainberg, not sure. But was hoping that the reduction would be a big reduction in size 18:39:06 userid could work 18:39:09 topol, it would be cutting as much superfluous data out, anything that isn't justifiably needed (ids are needed, but i don't see names as being needed) would be removed. 18:39:25 even in federation, lets see what we come up with in the deconflict session... 18:39:32 topol, so the size reduction should be significant 18:39:41 topol, especially cutting the catalog data out. 18:39:55 we could switch to a binary format 18:39:55 K, but jamielennox concer on breaking federated identity use case still aplies correct? 18:39:56 topol, the long-form catalog data* 18:40:07 topol, not necessarily.... 18:40:24 userid for most federated stuff should be superfluous 18:40:35 bknudson, i think that would be a logical progression, but i don't want to make tokens too difficult to parse in one-fell-swoop 18:40:39 it should be almost all role-assignments 18:40:50 ayoung: federation is the reason i can't have a /user path so the same would apply - you can't start with an id and get user data for federated idps 18:40:57 bknudson, so incremental - strip data down fix formatting and versioning, evaluate binary et al 18:41:04 jamielennox, heh, I wanted to give you a /user path. 18:41:13 jamielennox ++ 18:41:31 ayoung: i know you did - i still want it but it's easier to argue in person 18:41:40 an ID-only token: https://gist.github.com/dolph/10757712 18:42:10 technically you could drop domain scopes as well 18:42:16 dolphm, ++ 18:42:28 that is a much more elegant data set imo than our current token format 18:42:32 dolphm, yeah, they wouldn't be necessary if just using ids 18:42:40 you should add a token version in there 18:42:49 now policy.json has role IDs rather than role names? 18:43:08 bknudson, nope. It could cache the lookup 18:43:23 auth_token? 18:43:25 trimmed a bit more... https://gist.github.com/dolph/10757712 18:43:26 are there any fields in dolphm's example where we the name would be helpful to stakeholder projects? 18:43:32 feh...roles should be names, not ids 18:43:40 so we keep role names 18:43:44 that isn't a big deal. 18:43:45 morganfainberg: versioned 18:43:57 dolphm, ++ 18:44:38 clients will have to do a separate fetch for the endpoints? 18:44:41 if the role names arent so bad its nice to have them. 18:44:51 ayoung: role names are so static that i wouldn't see much benefit to including names over ID's 18:44:57 bknudson, correct you'd need to do a catalog (unprotected) request 18:45:23 bknudson: i'm assuming clients could do a single separate request for a catalog ^ 18:45:33 does the client already implement caching? 18:45:54 do we have a command that takes as input passin the id onlu token and return a verbose token for debuggin pruposes? 18:45:55 dstanek: for some things, but it's not the granular 18:46:08 dstanek: not really 18:46:20 auth_token does some caching, but that's it AFAIK 18:46:25 topol: could, i think that's details 18:46:38 we could start advocating requests-cache if our cache headers are correct 18:47:00 dolphm, ayoung, maybe http://pasteraw.com/c0spwqvdtgthup6kchx8njqpof420cv 18:47:07 then some of the extra lookup requests won't have to happen all of the time 18:47:27 morganfainberg: dreams! 18:47:29 dstanek, we also need to start issuing cache-control headers on our responses imo. 18:47:33 morganfainberg: i like having service_type in the catalog 18:47:37 morganfainberg, love seeing the version attribute 18:47:40 morganfainberg: dstanek: ++ 18:47:43 if you only need one you should only need one 18:47:44 morganfainberg, the catalog naming is a little funky, but yeah 18:47:57 ayoung, sure details to work out 18:48:12 though in which case i'd be interested to see if endpoints could become a dict of lists, rather than list of lists 18:48:15 morganfainberg, instead of catalog it could be services 18:48:24 then it would make sense 18:48:27 ayoung, ++ 18:48:32 morganfainberg: "what if i have two services of the same type in the same region?" 18:49:03 dolphm, compute is a list, you'd have more than one in the list. we support that now, right? 18:49:35 morganfainberg: the problem with indexing by type is that you're implying there's only one service ID of that type - current catalog doesn't have that limitation 18:49:38 we should be able to use keystone as a load balancer 18:49:45 dolphm, oh i see yeah. 18:49:57 (although i sort of wish it did limit by region -> service type -> endpoint interface 18:50:13 bknudson, we should also be able to enforce endpoint usage in the token "sorry your token isn't valid for endpoint XXX go somewhere else" 18:50:16 dolphm: that's backwards for hierarchical regions 18:50:19 i.e. that combination should be unique or you're deployment is super confusing 18:50:22 your* 18:50:35 jamielennox: ooh, that's probably true 18:51:05 morganfainberg: that's kind of what OS-EP-FILTER is doing 18:51:11 all hierarchy resolution for regions should be done on the keystone server before creating the token. 18:51:14 dolphm: maybe that's the point of not putting region in the token though, just id 18:51:16 dolphm, sortof... but not really 18:51:27 dolphm, it doesn't prevent use of an endpoint 18:51:37 dolphm: so when you request the endpoint for a service id you get back the endpoint URL that is appropriate to the region you requested - no more client side region handling 18:51:54 morganfainberg: right; but i think client side enforcement has been proposed somewhere 18:52:00 dolphm, it also means auth_token would need to know it's own id. 18:52:11 morganfainberg: ++ 18:52:13 dolphm, yeah, i think this token format opens us up to better be able to do that though. 18:52:45 morganfainberg, I've wanted auth_token to have a self_id field for a while now 18:53:10 auth_token has self_id of what? 18:53:16 morganfainberg, that also lets auth_token or comparable request the policy file: 18:53:19 its user? 18:53:27 ayoung, ++ 18:53:36 I want GET /policy/endpoint/ or something 18:53:53 and endpoint gets resolved to service 18:53:57 bknudson, it's endpoint id? 18:54:08 ahh. 18:54:10 s/it's/its 18:54:28 yea, i misread that as token has a self_id 18:55:02 a) might not be an endpoint for the service ... b) could have multiple endpoints? 18:55:31 dolphm, are we planing on deprecating the templated catalog? 18:55:38 fwiw, raw token struct (excluding signature) is 630 bytes without compression, 239 bytes with compression 18:55:46 for the ID-only token example 18:55:56 dolphm that is nice 18:56:00 dolphm, that is _way_ better 18:56:08 ooh, that also includes whitespace! 18:56:14 our tokens don't have any 18:56:27 i think it'll need more but yea that's good 18:56:28 just hope that the less info doesnt cause folks to grumble 18:56:42 topol, i think most people treat tokens as opaque blobs 18:56:49 is that compressed to binary or base64 or something? 18:56:57 morganfainberg, good to hear 18:57:04 so 352 -> 239 without excessive whitespace 18:57:09 topol, this also allows us to provide clear external token validators "is this a version X token" 18:57:21 bknudson we need to be able to assign policy files to a)endpoints, b)all endpoint of a services c)something based on regions 18:57:21 dolphm: that's impressive 18:57:37 bknudson, maybe other divisions as well 18:57:54 morganfainberg, I love the versioning!!! 18:58:07 dolphm, we could ditch json and go binary (as bknudson pointed out) might be even better then 18:58:44 morganfainberg, I don't think JSON adds that much once you drop whitespace. I'd not want to get into the parsing business 18:58:47 morganfainberg: that'd be worth prototyping 18:58:56 if you go binary then trying to visually inspect a token in like curl examples become not possible correct? 18:58:57 time! 18:59:02 this is a case where something like a protobuf would be nice. it would allow all languages to grab token data quickly 18:59:09 dolphm, wow meeting went fast. 18:59:15 #endmeeting