18:02:54 #startmeeting keystone 18:02:54 Meeting started Tue Oct 8 18:02:54 2013 UTC and is due to finish in 60 minutes. The chair is dolphm. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:02:55 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:02:57 The meeting name has been set to 'keystone' 18:03:01 #topic Havana 18:03:14 so, i wanted to quickly announce that i suspect we'll have an RC2 18:03:16 bknudson, you ever figure out why the tests are failing? 18:03:26 for you oslo.db patch I mean 18:03:29 dolphm, do we have a page for RC2 yet? 18:03:31 s/you/your/ 18:03:43 we have a low-impact security vulnerability introduced in havana that we should be able to fix fairly easily 18:03:45 ayoung: no 18:03:52 gyee: yes, it's on the meeting agenda 18:03:57 deleting a user at the same time as deleting a role 18:04:05 i'd also like to get the mysql/db2 issues resolved in havana 18:04:24 so we are running the tests in parallel? I had the same concern with jamielennox patch too 18:04:27 i'm not currently aware of anything else that *needs* to land in rc2 18:04:41 #topic Havana release notes 18:04:43 me? which patch? 18:04:53 gyee: is there anyway we can run those tests in parallel to see if it fails? 18:05:02 the tests are run in parallel, but obviously customers can do the same thing. 18:05:06 most of the release notes are filled in, except for the multi-ldap backend issue 18:05:07 https://wiki.openstack.org/wiki/ReleaseNotes/Havana 18:05:09 #link https://wiki.openstack.org/wiki/ReleaseNotes/Havana 18:05:10 jamielennox: gyee is referring to assertRequestHeader 18:05:30 stevemar, jamielennox, yes that 1 18:06:06 that should be ok, the tests are parallel but they are still sequential per-process so last_request() should always work 18:06:07 we've got the best release notes. 18:06:15 our PTL is really doing his job. 18:06:16 bknudson: so far! 18:06:29 i tried to start a bit early 18:06:39 yeah, nice notes! 18:06:39 dolphm, our goal is to keep you doing it so none of us have to 18:06:42 since, you know, we were the first project to hit RC1 and all, we had extra time to spare 18:06:49 ayoung: ++ 18:07:24 extra time is such a foreign concept 18:07:28 :) 18:07:30 #topic keystoneclient 18:07:35 on the topic of no extra time 18:07:49 0.3.3 is sort of hung up on a few code reviews that aren't moving very fast 18:08:05 any one of those would probably warrant an immediate release 18:08:06 so...we've put shardy on the spot making him do a tempest test for a client change 18:08:10 i'm faily certain it's just jamielennox doing reviews :P 18:08:19 so if one merges and the others are still stalled, i'll be cutting a release anyway 18:08:26 the short of it is: we have no way to test client changes against a live server unless we use tempest. 18:08:45 so: new rule...all changes to keystone client that should be tested against a live server should have a tempest test 18:08:57 and...I'll try to get an example one out there shortly 18:09:07 that's not really relevant for https://review.openstack.org/#/c/35403/ 18:09:23 ayoung, would it be a chicken-egg issue? 18:09:36 I presume its a conditional approval 18:09:38 which i think has nearly sufficient unit tests 18:09:45 gyee, that is exactly the term I use to describe it, but we have an approach 18:10:04 in the start of the test, check to see if the client supports the feature,. if not, rais a Skip exception 18:10:12 ooh, i should also point out that the next release of keystoneclient will likely be 0.4.0, not 0.3.3 18:10:25 new features? 18:10:32 due to our new dependency on oslo.config 1.2 18:10:36 dolphm: so regarding the TZ patch should we be looking to make some changes ourselves or wait for original authors? 18:11:08 jamielennox: i'm down for either -- there was a comment by brant that i wasn't comfortable addressing without at least feedback from the original author 18:11:58 #topic oslo.db 18:12:06 bknudson: all yours 18:12:22 there's a bunch of notes on the meeting agenda to get everyone on the same page- https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting#Agenda_for_next_meeting 18:12:23 https://review.openstack.org/#/q/status:open+project:openstack/python-keystoneclient,n,z looks pretty well supported. THere are 6 reviews up there with nothing in the R column for me, and two of them are WIP. 18:12:25 ok, so I'm working on using oslo.db rather than our own 18:12:31 yes, read the notes. 18:12:37 dolphm, before we do that, client issues all resovled? 18:13:25 anyway, using oslo.db seems to be going pretty well, you can see the changes. 18:13:33 ayoung: i just want to make sure the patches in keystoneclient have traction 18:13:42 Base class only has 1 thing it it now which is just forwarding get_session. 18:14:08 dolphm: I think all my comments in https://review.openstack.org/#/c/35403/ should be easy to make. Not asking for a rewrite or anything. 18:14:12 oslo.db reviews: https://review.openstack.org/#/q/status:open+project:openstack/keystone+branch:master+topic:oslo.db,n,z 18:14:33 so, using oslo.db turned up some timing issues. 18:14:46 let me find the code... 18:15:00 ayoung: gyee: https://review.openstack.org/#/c/49655/ 18:15:14 dolphm: re client - if I can figure out what cody is on about i wouldn't mind: https://review.openstack.org/#/c/49106/ being in 0.4 18:15:26 https://github.com/openstack/oslo-incubator/blob/master/openstack/common/db/sqlalchemy/session.py#L672 18:15:33 checkin handler does _thread_yield 18:15:37 dolphm: depends how quickly you wnat to cut it 18:15:37 jamielennox: ping me after the meeting, i can help there 18:15:52 so what can happen is, we're deleting a role 18:16:00 and while deleting a role some other request to delete a user with that role 18:16:05 is that a dupe? couldve sworn I already ACKed that 18:16:08 now delete role gets 404 Not Found 18:16:35 I think we did talk about this, and decided in general it was ok to not find a user 18:16:39 so that operation shouldn't fail 18:16:51 bknudson, that is correct 18:16:55 but I wanted to make sure with others before I go removing that stuff. 18:16:57 in the separation between identity and assignment, i'd say that's fair 18:17:02 in general, things in assignments should not depend on things in identity being there 18:17:06 NotFound: Object not found 2013-10-07 21:31:18.422 | Details: {"error": {"message": "Could not find user, be547582b3c541dda71dc202f9acc153.", "code": 404, "title": "Not Found"}} 18:17:14 for Federated, they are *not* going to be there 18:17:21 dolphm: ok, how about the cases where both in assignment? 18:17:46 I posed the code in assignment delete_grant: https://github.com/openstack/keystone/blob/master/keystone/assignment/backends/sql.py#L169 18:17:49 bknudson: both what? 18:18:06 both in the same backend store? 18:18:07 you could delete a project at the same time as delete a role 18:18:07 bknudson, deleting a role should cascade delete all assignments. 18:18:15 deleting a project the same 18:18:17 https://github.com/openstack/keystone/blob/master/keystone/assignment/backends/sql.py#L179 18:18:33 if that comes in while someone else is attempting to delete...404 is appropriate 18:18:39 its an ACID issue 18:18:54 you tried to delete something that is already gone. 18:19:19 where are we handling the consequences of the project delete, for example? (deleting the role assignments on that project) 18:19:23 for assignment sql backend, could handle by doing the deletes in the same translaction. 18:19:25 in the manager or driver? 18:20:44 bknudson, that would be a good approach 18:21:02 LDAP can't do it all in one transaction, but that is likely to be fairly rare 18:21:23 right, we were going to deprecate LDAP assignments. 18:21:30 bknudson, not any more 18:21:36 CERN specifically asked for them to stay around 18:21:40 bknudson: *were* CERN said please no 18:21:45 said that LDAP was the only thing that would scale like they needed it 18:21:56 really? 18:22:01 joe-savak: i suspect rax would object as well? ^ 18:22:01 morganfainberg, yep 18:22:08 maybe they need no-sql mongodb 18:22:19 (sorry, just got back to my desk, been a hectic week, here now) 18:22:23 dolphm - yes - we use LDAP only for our backend 18:22:43 joe-savak: i mean long term 18:22:45 joe-savak: is keystone configured with read-write? 18:22:51 joe-savak, but you also need multidomain, right? 18:23:10 it is configured with read-write - and we have multi-domain implmeneted as a rax 2.0 extension (moving to v3 next year) 18:23:31 and yes, it will be there long-term as we have 190k+ users and LDAP meets our needs well 18:23:33 joe-savak, is each domain in its own subtree a viable apporach for you guys? 18:23:43 approach 18:23:53 ayoung - let me take that back to our ldap guy. Not sure 18:24:21 ayoung, that's a typical LDAP deployment, domain map to a subtree 18:25:02 gyee, that seems like an accurate assessment 18:25:30 gyee, you know that and I know that, but I'm not sure that RAX knows that 18:25:49 the previous impl had a domain attribute.... 18:25:58 bknudson: so, the oslo.db stuff started because of the db2 handling fix -- is that still separate somewhere? 18:26:04 bknudson: or will that be on top of this patch? 18:26:23 dolphm: the db2 handling fix will not be needed when we switch to oslo.db 18:26:28 bknudson: oh cool 18:26:29 dolphm, as part of the oslo.db code, do we get the start of alembic migrations merged in? 18:26:35 bknudson: what about havana then? 18:26:36 dolphm: so the db2 changes are still there, hoping to be backported to havana. 18:27:02 dolphm: It's just this one https://review.openstack.org/#/c/49272/ 18:27:24 bknudson: we can propose things to milestone-proposed after they land in master 18:27:31 for havana rc2/final 18:27:38 dolphm: I'll do that! 18:27:48 bknudson: /salute 18:28:08 so I'd like https://review.openstack.org/#/c/49272/ to get in before the oslo.db change 18:28:32 which depends on https://review.openstack.org/#/c/49270/ 18:28:39 also, the disconnect fix is in oslo-incubator, so I'd like that to merge first, too. 18:28:50 which also needs to be in milestone-proposed, i think 18:28:51 yes, it turns out that our mysql disconnect handler didn't work at all either. 18:29:03 bknudson, ouch. 18:29:38 i'd love to get that bit merged to both master and milestone-proposed today 18:29:50 morganfainberg: gyee: ayoung: https://review.openstack.org/#/c/49270/ 18:30:00 I'll cherry-pick the changes to milestone-proposed 18:30:42 what is dbapi? 18:30:56 dbapi_conn? 18:30:58 1 line change, 58 lines of test, love these patches! 18:31:25 gyee: the 1 line change was required because there was no test 18:31:50 gyee, yeag...but are the tests now mysql specific? 18:31:59 or is the mysql in the comment just spurious? 18:32:27 ayoung - fyi - all domains are in one subtree today - we are looking to split it out though 18:32:31 errro 2006 is mysql specific iirc 18:32:31 ayoung, can't tell by looking at it 18:32:38 joe-savak, sounds good. I can work wityh you on that 18:32:43 I mean can't tell if they are mysql specific 18:32:51 bknudson, is this test my_sql specific? 18:33:06 this test appears to be mysql specific as it calls sql.mysql_on_checkout 18:33:10 ayoung: dbapi_conn is a database connection, see https://review.openstack.org/#/c/49270/3/keystone/common/sql/core.py 18:33:23 itdoes dbapi_conn.cursor().execute('select 1') and dbapi_conn.OperationalError 18:33:26 ah. 18:33:45 bknudson, "Simulates the dbapi_conn passed to mysql_on_checkout." 18:34:13 ayoung: the test isn't really mysql-specific, but it tests the mysql-specific checkout handler. 18:34:34 morganfainberg: in bknudson's revised implementation that could just be renamed to ping_on_checkout or something 18:34:42 bknudson, does pgsql (or anything else) have the same mechanics? 18:34:50 dolphm, nod. i see that now. 18:34:53 bknudson, what happens if you run a live PostGresql test? 18:34:57 dolphm: morganfainberg: that change is in oslo.db! 18:35:07 * ayoung and morganfainberg on same wavelength 18:35:25 ayoung: the listener isn't registered with postgresql 18:35:53 it's looking for mysql-specific exception 18:35:56 i've never run into 'mysql server has gone away' with postgresql (nor db2 for that matter ;P) 18:36:59 dolphm, heh 18:37:18 dolphm, well...it looks like the live tests are broken anyway 18:37:28 ayoung, i am guessing other DBs would need special handling for the different error cases/codes 18:37:39 NoSuchOptError: no such option: policy_file 18:37:56 that happens because the policy backend registers its own config option, and that is not pulled in by the tests 18:37:58 ayoung: that problem was fixed in other places by importing somthing... 18:38:04 policy 18:38:04 morganfainberg: IF their client libraries expose you to the same issue 18:38:10 dolphm, yeah. 18:38:27 Here's the change in oslo.db: https://review.openstack.org/#/c/48733/ 18:38:46 it's failing Jenkins because of a requirements change that hasn't merged. 18:38:54 bknudson: it seems as though all of IBM is rushing to fix this issue 18:39:23 dolphm: it was found & reported internally so we're trying to fix it. 18:40:35 #topic open discussion 18:41:12 jaimelennox, see my comment on the access key auth patch? 18:41:22 jamielennox 18:41:34 gyee: not as yet 18:42:02 #link https://review.openstack.org/#/c/46771/ 18:42:08 both HP and RAX have access key auth support prior to KSL 18:42:15 so this is not something new 18:42:38 gyee, should that be tied in with the credentials backend? 18:42:39 we are just trying to bring back that capability 18:42:49 ayoung, it is using cred backend 18:42:58 gyee: i've been doing too much crypto - i saw secret keys and started trying to figure out what they were doing 18:43:14 this patch does not use crypto 18:43:23 just straight secret comparison 18:43:28 gyee, are these symetrric keysr or aym? 18:43:29 asym 18:43:35 they are not keys 18:43:49 just really long randomly-generate share secrets 18:43:54 gyee, symetric shared secredts 18:44:02 bleh 18:44:06 * ayoung cannot type 18:44:06 right, not recommended for crypto 18:44:28 gyee, and the driving force behind this is to allow a user to have multiple "passwords" at once 18:44:38 they are designed for non-interactive applications 18:44:48 gyee, fabio, is that the entire spec? or will there be CRUD operations surrounding the access keys? 18:44:56 ayoung, correct, so to make a seamless secret rotation 18:45:01 stevemar, crud should be in the credential backend 18:45:11 ayoung, ++ 18:45:11 gyee, I think I can get behind that 18:45:12 today, you can't rotate password without service disruption 18:45:13 ayoung: correct 18:45:43 gyee: ok, that makes sense 18:45:45 but you can rotate access keys with rolling upgrade 18:45:56 gyee, youi have a comment in there. Once he's updated, and you've reviewed and feel happy with it, ping us and we'll rip it to shreds 18:46:06 ayoung, cool, thanks 18:46:06 but in general I am OK with this 18:46:42 i think definetly remove the decypher remark and i think that there should be a more explicit the 'unscoped token' is related to the example and not that all tokens are unscoped 18:46:48 but otherwise it looks good 18:47:13 probably change key to "shared secret" in the wording 18:47:17 gyee: so the secret is not a secret at all? 18:47:26 as secret as a password. 18:47:27 its a shared secret 18:47:32 "shared" :) 18:47:38 but not in the crypto sense 18:47:50 overloaded terms… *runs and hides* 18:47:52 joe-savak, ++ 18:48:00 yeah, nothing "key" about this, I think 18:48:10 joe-savak: i don't consider passwords to be "secret" 18:48:16 what is yours? 18:48:16 ayoung: +++ 18:48:19 " 18:48:25 joe-savak ++ 18:48:29 "access key" is a bit misleading 18:48:32 speaking of keys, though, the KDS blueprint needs some work. I punted to jamielennox to shepherd it, but we are all kindof involved on this one 18:48:38 joe-savak, ******* 18:48:41 yeah, lets call it shared-secret auth 18:48:47 I am perfectly fine with it 18:48:48 gyee, +1 18:48:50 fabiog: added another comment but it's not that big 18:48:58 joe-savak: if i trusted you i'd be happy to give it to you, which is what makes it a *shared* secret 18:49:05 oauth then 18:49:20 joe-savak: if it was a secret, i wouldn't give it to you even if i trusted you 18:49:21 KDS really needs to be ready to go for I1 in order to have the other teams make use of it in Icehouse. We punted on it in Havana 18:49:32 ayoung, agreed 18:49:53 ayoung: so i was looking through KDS yesterday - other than dolphm's -1 is anyone aware of something 'missing' from KDS? 18:50:03 gyee: doesn't HP call it "api keys" in v2? 18:50:08 I asked dolphm to put together a "style guide to the API docs" link is 18:50:09 (is that HP-IDM?) 18:50:09 jamielennox, the API speC? 18:50:11 dolphm, yes 18:50:19 morganfainberg: either 18:50:23 RAX calls it API keys as well 18:50:23 HP need to pick a better name 18:50:38 morganfainberg: there is a review as well 'Initial KDS' or something 18:50:43 gyee: is HP-IDM broken down into access token + secret key? 18:50:50 jamielennox, hrm. let me look. i think we've pulled it apart and put it back together a bunch in comments so far. 18:51:01 and it's been a couple weeks since i've looked at it 18:51:02 dolphm, it was called access-key prior to KSL I think 18:51:06 dolphm, can you post the link to your review request fopr the style guide? 18:51:10 joe-savak: i think rax's implementation is a standard api key though -- there's no second piece of information 18:51:20 RAX-KSKEY - username & api-key 18:51:26 ayoung: https://review.openstack.org/#/c/49791/ 18:51:40 dolphm, thanks 18:51:55 right - it appears AWS has this id & secret thing - but i can't see what the advantage is over having user_id + shared secret 18:52:14 what I would like to have happen is that we have a very clear set of guideline on API docs, so we don't spend a lot of churn figuring out what goes where 18:52:29 ayoung: is audience not enough? 18:52:30 this is a great start. lets give it a review and make sure it is something we can all understand and support 18:52:40 ayoung: or is it related to "is this a spec or not?" 18:52:48 * annegentle_ catches up 18:52:59 annegentle_, see https://review.openstack.org/#/c/49791/ 18:53:18 we are talking about the identity API, but I would guess that the standard should be OS wide 18:53:51 ayoung: orignally we wanted specs across openstack 18:53:55 we've got both the identity API spec and the Identity API docs... do we need both? 18:53:57 ayoung: that's not really happening tho 18:54:10 annegentle_, we've gotten kudos for our Identiyt API specs...but there has a been a huge amoung of effort to get there 18:54:24 bknudson: as doc coordinator, I have historically ignored spec work because we just don't have time, and we serve users as a higher priority than devs making APIs 18:54:33 ayoung: yes on both 18:54:46 jamielennox: it's totally different in aws 18:54:49 ayoung: but yes, as I indicated, the doc team has to somewhat ignore specs 18:54:57 this has the spec info, too? http://api.openstack.org/api-ref-identity.html 18:54:57 jamielennox: in aws, i think it's oauth2 or a variation thereof 18:54:58 ayoung: you guys were fortunate to have Diane clean up the v2 18:55:10 annegentle_, I'd like to take what dolph wrote for the identity API and make it a stand alone document eventually, but right now I think we most can benefit from the Keystone devs using it as a living doc for what we are doing in identity 18:55:13 bknudson: that documents reality with real response/requests 18:55:32 ayoung: sure, you should make your own priorities as well 18:55:55 bknudson: if it doesn't document reality it's a doc bug 18:56:00 jamielennox: the same two terms are used though, which is why this proposal is confusing 18:56:17 annegentle_, considering how often we have pointed people at the curl examples, I'd like this document to at least reflect the guts of "how do aI talk to Keystone via the web API" 18:56:37 annegentle_, its called "undocumented feature" 18:56:39 not a bug 18:56:52 ayoung: sounds great, Diane Fleming has a blueprint for Icehouse to try to give more real API examples, let me find the link 18:56:58 annegentle_, I used it heavily when writing the few examples I have so far 18:57:05 dolphm: so i assume that it is meant to model some other auth system though, it seems to similar to others to have been made from scratch 18:57:16 annegentle_, http://adam.younglogic.com/2013/09/keystone-v3-api-examples/ 18:57:19 https://wiki.openstack.org/wiki/Blueprint-os-api-docs 18:57:42 ayoung: great, I'll point diane to it for her work 18:58:03 annegentle_, I want to do a follow up with some of the more esoteric ones, like creating trusts 18:59:01 jamielennox: are you referring to gyee's proposal modeling other auth systems? 18:59:21 time is about up -- switching to #openstack-dev! 18:59:32 #endmeeting keystone