Thursday, 2014-03-13

jamielennoxbknudson: so essentially just start putting a *ignore into function definitions?00:00
*** david_lyle_ has quit IRC00:00
jamielennoxbknudson: not really, i don't know how we do that with inspect00:00
bknudsonjamielennox: well, their example has *ignore, and then if ignore: raise TypeError00:00
jamielennoxthe only thing i see that we can do with positional that we can't do with that is required kwargs00:01
bknudsonrequired kwargs? fancy00:01
jamielennox@positional(1) def f(a, b,c)  will end up meaning that b and c MUST be passed and must be kwargs00:01
jamielennox(i don't know if we need that but i thought it was cool)00:02
jamielennoxthat and we would still need to put *ignore in the parameters for all the functions right?00:04
*** nkinder has quit IRC00:04
bknudsonjamielennox: yes, but that matches essentially what you'd have to do with py3.00:04
bknudsonjamielennox: I assume if we had pep 3102 in python2 then we wouldn't implement this decorator?00:05
jamielennoxit would solve the problem enough that i wouldn't worry about it00:05
jamielennoxi haven't used ,*, much so i don't know exactly where it fails00:06
jamielennoxso def a(b, c, *, d): means d is a required kwarg00:07
jamielennoxa(1, 2, 3, 4, e=5) TypeError: a() takes 2 positional arguments but 4 positional arguments (and 1 keyword-only argument) were given00:08
jamielennoxso it's making sure you don't pass extra *args00:08
jamielennoxso yes - if we had pep3102 we wouldn't need the decorator00:09
bknudsonjamielennox: I don't think you could do a(b, c, *, d) ... d would have to be a kwarg -- d=None00:09
bknudsond would have to be optional00:10
jamielennoxbknudson: nope, trying it as i write it here00:10
jamielennox>>> def a(b, c, *, e):00:11
jamielennox...     print("%s, %s, %s" %(b, c, e))00:11
bknudsonjamielennox: weird, so d is a required kwarg in that case?00:11
jamielennox(not sure why i switched from d to e there)00:11
jamielennoxbut yes e is a required kwarg00:11
bknudsonthis is why we should drop python 2 support00:11
jamielennox>>> a(1,2, e=3)  = 1, 2, 300:11
jamielennoxwhich is EXACTLY what i've been wanting for a while00:12
jamielennoxthis is my favourite thing of py3 so far00:12
*** nkinder has joined #openstack-keystone00:12
jamielennoxi think @positional is easier/better than doing *ignore everywhere, but we can ditch in when we go to py3 only00:13
*** browne has quit IRC00:14
*** gokrokve has joined #openstack-keystone00:24
*** gokrokve_ has joined #openstack-keystone00:25
bknudsonnow on to my favorite thing to do... rechecking00:25
*** gokrokve has quit IRC00:29
bknudsonjamielennox: can you explain https://review.openstack.org/#/c/74930/ to me?00:30
bknudsonjamielennox: I don't see how it makes anything easier.00:31
jamielennoxbknudson: so when you used to run get_pki it would generate a new set of keys/certs and then reparse all the .json files to generate new CMS files00:32
jamielennoxby splitting the functions and the generator you can just regenerate the CMS with the same keys - so smaller reviews00:32
jamielennoxeg00:32
*** gordc has joined #openstack-keystone00:32
jamielennoxhttps://review.openstack.org/#/c/74955/00:32
bknudsonthat sounds good, but how to I generate only a few keys?00:32
jamielennoxonly a few keys?00:33
bknudsonjamielennox: how did you regenerate just the 4 files and not all of them?00:33
jamielennoxwith that at moment to just redo the CMS you'd do source gen_pki.sh; gen_sample_cms00:34
bknudsonis that how you did it?00:34
bknudsonhow did you know to do that? is there documentation somewhere?00:34
jamielennoxso that review is the one that ayoung did the split for00:34
*** henrynash has quit IRC00:35
bknudsonwell, maybe it will make sense to someone else and they'll merge it.00:36
morganfainbergjamielennox, ping00:36
morganfainbergjamielennox, how is "enforcement" being assigned to in the inner()?00:36
jamielennoxbknudson: i know it only because i was talking with ayoung when he presented it00:37
jamielennoxi'm not sure if it's worth doing a regenerate.sh script that only redoes the CMS00:37
*** wchrisj has quit IRC00:37
*** devlaps1 has quit IRC00:37
morganfainbergjamielennox, or are you saying that enforcement is global?00:37
jamielennoxmorganfainberg: pulling it up00:38
openstackgerritA change was merged to openstack/keystone: Fixup region description uniqueness  https://review.openstack.org/7915900:38
morganfainbergbecause if "positional_enforcement" is meant to be mutable, it looks wrong to me00:38
jamielennoxmorganfainberg: so not sure what you mean00:39
morganfainbergok00:39
morganfainbergyour comment: It will depend on when you expect the value of positional_enforcement to be set.00:39
jamielennoxso if i define a function with @positional(enforcement=POSITIONAL_EXCEPT) then it will always throw an exception regardless of the positional_enforcement value00:39
morganfainbergi think you're doing it wrong if you're changing positional_enforcement at the module level like that00:39
morganfainbergright00:40
morganfainbergwhich can still be exclusive to the wrapper method00:40
morganfainbergit still wont change at runtime00:40
jamielennoxmy use case for positional_except was applications that want to set POSITIONAL_EXCEPT as the default rther than WARN00:40
morganfainbergok00:41
jamielennoxfor example all keystoneclient unit tests run with POSITIONAL_EXCEPT00:41
morganfainbergwouldn't this be a CONF deal instead of a mutable value?00:41
jamielennoxwe don't have CONF in client00:41
morganfainbergugh,00:41
morganfainbergi really don't like this setup00:42
jamielennoxwell not having CONF makes sense - you shouldn't allow a library access to a global state object00:42
jamielennoxthis was the next best way i could think of doing it00:43
morganfainbergno no i mean i don't like the mutable module thing00:43
jamielennoxalso i copied a lot of the ideas from a google client i found00:43
jamielennoxbknudson: ^ which is why the docs are off00:43
morganfainbergstill doesn't change that it leaves a bad taste in my mouth00:43
morganfainbergtbh, i'd remove the warn.00:43
jamielennoxmorganfainberg: would love to - that would break compatibilty for existing functions00:44
morganfainberghow does py33 handle it?00:44
morganfainberghard exception each time right?00:44
bknudsonjamielennox: the docs haven't been generated for over a year anyways!00:44
jamielennoxbknudson: yea, i've no idea what valid rst is - i just do something that looks like everything else00:45
*** gordc has left #openstack-keystone00:45
jamielennoxmorganfainberg: yea, you can do a hard exception there00:45
morganfainbergi mean python33 doesn't have the soft "Warning" mechanism00:45
morganfainbergwith that syntax00:45
jamielennoxmorganfainberg: no, because it builds it into the language00:45
morganfainbergi'd call this py33 compatibility and still strip the warning00:46
bknudsonhttp://docs.openstack.org/developer/python-keystoneclient/py-modindex.html -- Last updated on Mon Mar 18 19:40:00 201300:46
jamielennoxwe could do only exceptions and then only use the decorator for new code00:46
bknudsonso not quite a year yet but getting close00:46
jamielennoxheh, 5 days00:46
bknudsonwe can have a birthday party for it00:46
jamielennoxmorganfainberg: i figured the warning mode was good for existing functions to at least tell people they are doing it wrong00:47
morganfainbergso explain to me where we're broken with old code and hard exceptions?00:47
morganfainbergi'm just not seeing it atm00:47
morganfainbergso point me to an example :)00:47
jamielennoxdef f(a, b=None)00:47
morganfainbergwhere we use **kwargs?00:47
bknudsonbtw - https://review.openstack.org/#/c/72515/00:48
jamielennoxf(1, 2) is valid, if i wrap a @postiional around it it won't be00:48
jamielennoxmorganfainberg: **kwargs won't be affected00:48
morganfainbergoh i see people _might_ call it as f(1, 2)00:48
morganfainbergnot f(1, b=2)00:48
jamielennoxmorganfainberg: right00:48
jamielennoxand if i want to put a new positional argument in there between a and b i can't because of backwards compat00:49
*** dolphm has quit IRC00:49
morganfainbergjamielennox, ok...00:50
morganfainbergjamielennox, i still don't like it.00:50
jamielennoxits drawing the distinction between keyword arguments and optional positional arguments which python2 can't do00:50
morganfainbergjamielennox, really this mutable module level thing you evaluate everytime feels wrong00:50
morganfainbergalso plural can be evaluated in wrapper00:51
jamielennoxright, so the only reason to do that everytime is the order of operations, if you evaluate the function code before you set the postiional_enforcement variable then you are fixed00:51
morganfainbergjamielennox, it's a NIT.00:51
*** wchrisj has joined #openstack-keystone00:51
jamielennoxmorganfainberg: can, i tend to fall on CPU rather than memory for that sort of stuff but i don't care00:52
morganfainbergthis is bike shedding, i just don't like it :P00:52
morganfainbergpaint it green00:52
*** sudorandom has quit IRC00:53
jamielennoxi have at least two or three places in client where i've got frequently used variables that should be positional tucked at the end of a giant list of kwargs because i'm not allowed to change the order of them00:54
*** dolphm has joined #openstack-keystone00:54
morganfainbergjamielennox, ok i know what i dislike00:54
*** sudorandom has joined #openstack-keystone00:54
morganfainbergactually, i finally put my finger on it00:54
jamielennoxif you call Client(with username=, password=, user_domain_id= and all that crap as positional, you are an idiot, but i still have to support you00:55
morganfainbergit's not relevant so i wont complain about this.00:55
morganfainbergalso 0 arguments is plural00:55
*** wchrisj has quit IRC00:56
morganfainberg;)00:56
morganfainberganother NIT00:56
jamielennoxbut if i can wrap a @positional around that i don't have to00:56
jamielennoxtrue00:56
morganfainbergyah don't worry i know what's bugging me about this now - the mutable module thing00:56
jamielennox you may as well tell me the the problem00:56
morganfainbergi'd invert plural to be '' if arg len() == 1 else 's'00:56
morganfainbergjamielennox, the issue is a python-ism.00:57
jamielennoxi'm looking through bknudson's comments now, if i wrap the decorator into a class rather than a function then i can put the warning as a class level variable00:57
morganfainbergand my aversion to module level mutables, plus that i want switch/case :P00:57
morganfainbergi wouldn't complain if the language had a real switch/case construct00:58
morganfainbergbecause you could do the eval and switch on the result00:58
morganfainbergno a dict is not a more elegant choice (nor more readable)00:58
bknudsonmorganfainberg: no need for switch/case when you've got dynamic dispatch00:58
jamielennoxheh, yea - i'm pretty sure that's never coming00:58
morganfainbergjamielennox, exactly00:58
morganfainbergbknudson, hah00:58
morganfainberganyway00:58
jamielennoxif nothing else it kind of makes sense when you consider the indenting you would have to do00:59
jamielennoxswitch x00:59
jamielennox    case 500:59
jamielennox        start doing stuff00:59
morganfainbergbknudson, i would like the syntactic sugar of a switch-case style syntax even if it;'s not needed00:59
morganfainbergjamielennox, so ignore my complaint on the module mutable thing00:59
*** richm has quit IRC00:59
*** wchrisj has joined #openstack-keystone01:00
jamielennoxmorganfainberg: well i'm going to do the class anyway so it probably makes more sense there01:00
bknudsonswitch(x, (5, lambda: do_something))01:00
morganfainbergbknudson, *shudder*01:00
morganfainbergnooooooo01:00
morganfainbergjamielennox, yeah as a class this makes it easier01:00
morganfainbergjamielennox, and you can just do @property stuff. or some such01:00
bknudsonor just write a factory01:01
jamielennoxnah, property on a class requires metaprogramming and it's just not worth it01:01
morganfainbergjamielennox, does it?01:01
morganfainberg*shrug*01:02
jamielennoxyea, a property is defined on an instance01:02
*** derek_c has joined #openstack-keystone01:02
jamielennoxyou have to do a metaclass to define a class property01:02
jamielennoxthough i do wish they had a different way of doing that01:02
morganfainbergjamielennox, iirc you can do @property and then in __init__ just setattr() over the top of the @property01:03
morganfainbergthe setattr puts the new property in the __dict__01:03
morganfainbergwhich overrides the MRO inheritence01:03
morganfainbergand wont call the defined @property01:03
jamielennoxthat sounds... interesting01:04
* morganfainberg checks this.01:04
jamielennoxnow i have to figure out how to do a class based decorator with arguments - i've always prefered the function approach01:04
morganfainbergjamielennox, doh i was wrong01:05
*** theocean154 has joined #openstack-keystone01:05
morganfainbergjamielennox, now let me figure out how i did this in the past01:05
morganfainbergbecause i _know_ i've done this01:06
bknudsonjamielennox: the class is going to get created and then it's going to be called with the function and it returns the new function01:09
bknudsonso implement __call__(self, f), I guess.01:10
jamielennoxbknudson: yep01:11
morganfainbergjamielennox, oh i see how i did this01:12
jamielennoxbknudson: it just means that you still end up returning a new function rather than returning the class01:12
morganfainbergjamielennox, i used a re-implemented @property that didn't do the @property magic.01:12
*** wchrisj has quit IRC01:12
morganfainbergjamielennox, don't worry about it ignore my complaint01:12
jamielennoxmorganfainberg: i'll show you the class one in a minute, it does look better for the variable01:13
morganfainbergjamielennox, yeah i'm sure it's a bit more straightforward01:13
openstackgerritBrant Knudson proposed a change to openstack/keystone: Remove unnecessary oauth1.Manager constructions  https://review.openstack.org/7921301:14
*** wchrisj has joined #openstack-keystone01:18
*** dstanek has quit IRC01:19
*** marcoemorais has quit IRC01:21
lbragstadbknudson: thanks for submitting a bug01:27
lbragstad+101:27
bknudsonlbragstad: just doing my job01:28
*** nkinder has quit IRC01:29
openstackgerritNathan Kinder proposed a change to openstack/keystone: Filter LDAP dumb member when listing role assignments  https://review.openstack.org/7393501:30
jamielennoxbknudson: how do i generate the module docs again? build_sphinx will only do the docs01:33
jamielennoxah, that's one of your patches that i haven't yet approved isn't it01:33
*** ChanServ sets mode: +o dolphm01:34
bknudsonjamielennox: I submitted a patch to make it the same as keystone...01:34
bknudsonjamielennox: https://review.openstack.org/#/c/72515/01:34
jamielennoxbknudson: yea, i did a pass of them a while ago but i don't think they got merged01:34
bknudsonjamielennox: you know how hard it is to get anything merged into keystoneclient.01:35
jamielennoxyes i do01:35
openstackgerritldbragst proposed a change to openstack/keystone: Allow 'description' in V3 Regions to be optional  https://review.openstack.org/7865801:35
openstackgerritldbragst proposed a change to openstack/keystone: Enforce required parameters for V3 Regions  https://review.openstack.org/7644401:35
*** vhoward- has left #openstack-keystone01:35
*** wchrisj has quit IRC01:36
*** wchrisj has joined #openstack-keystone01:43
*** nkinder has joined #openstack-keystone01:44
openstackgerritJenkins proposed a change to openstack/keystone: Updated from global requirements  https://review.openstack.org/7669101:49
*** amcrn has quit IRC01:56
*** wchrisj has quit IRC02:01
ayoungjamielennox, so to assign a role, I use admin_client.roles.grant(role=role,user=user, project=project) ... we'll ignore that it should be assign. To delete the role assignement, do I do02:05
ayoung            for roles in admin_client.roles.list(user):02:05
ayoung                admin_client.roles.delete(role)02:05
ayoungwell, to delete role assignements, that is?02:06
*** mberlin1 has joined #openstack-keystone02:06
jamielennoxayoung: off the top of my head it looks fine02:06
jamielennoxwe don't have any bulk delete or anything02:06
ayoungjamielennox, but the same command deletes a role definition, no?02:07
jamielennoxoh, sorry delete the assignment02:07
jamielennoxumm02:07
*** mberlin has quit IRC02:08
jamielennoxadmin_client.roles.revoke(role, user)02:08
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Add a positional decorator  https://review.openstack.org/7702602:11
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Start using positional decorator  https://review.openstack.org/7705502:11
*** wchrisj has joined #openstack-keystone02:11
ayoungah, ok...revoke makes more sense. I missed that02:11
jamielennoxmorganfainberg: ^, so i switched the default from WARN to EXCEPT which i think makes sense and i ditched the default altogether02:11
jamielennoxso not more module variable (although it looked much nicer when it was on a class)02:12
morganfainbergjamielennox, ah so must be specified02:12
morganfainbergjamielennox, well specified if you want non-exception behavior?02:12
jamielennoxmorganfainberg: right02:12
jamielennoxmeans we don't get enforcing in tests but that's not a big loss02:12
morganfainbergjamielennox, i like that better, but i was willing to let ya go w/ the module level stuff. it was in the depths of bike shedding whcih is why i backed off.02:13
jamielennoxand from now on we will pretty much always want except behaviour so it makes sense to specify WARN only for the existing things02:13
morganfainbergmakes sense to me02:13
jamielennoxwe can rely on reviewers to make sure you only specify things via kwargs internally, if someone wants to put an override on it later then they can02:14
morganfainbergjamielennox, ++ i like it02:15
ayoungjamielennox, what is the "right" way to request a token using the client.02:21
ayoungjamielennox, this might be artificial02:22
ayoungbut I want to create a bunch of tokens and then do things that revoke them and test the revocation02:22
ayoungis it just02:22
jamielennoxthere's no way to force request something new02:22
jamielennoxbut if you create a client then you can just retrieve it from that02:23
ayoungso create a new client for each02:23
ayounghow do I get the token out of the client?02:23
jamielennoxclient.auth_ref is the accessinfo object02:24
jamielennox.auth_token is just the token02:24
jamielennoxup to 7 failing reverifies02:31
jamielennoxhttps://review.openstack.org/#/c/79474/02:31
ayoungwow.  that is suboptimal02:32
jamielennoxgreat and the log file is actually missing so i can't figure out the error02:33
openstackgerritNathan Kinder proposed a change to openstack/keystone: Filter LDAP dumb member when listing role assignments  https://review.openstack.org/7393502:33
jamielennoxany idea if there is a recheck for unstable where there is no grenade log?02:34
*** zhiyan_ is now known as zhiyan03:06
nkinderjamielennox: I haven't had to deal with rechecks.  Is it just that there was some spurious failure (hopefully test environment related), and you need to tell it to run the tests again?03:08
jamielennoxnkinder: pretty much, it's for failures that aren't your fault essentially03:08
jamielennoxapparently zuul is a bit unstable at the moment so i should just keep reverifying until it works03:08
nkinderjamielennox: I wonder if the patch I submitted a little bit ago is going to encounter that too...03:09
nkindermy luck hasn't been good today, so it will probably hit problems03:09
nkindermy network connection had a 80% packet loss rate for about 5 hours this afternoon...  lots of fun03:10
jamielennoxheh, i was going to say luck of the draw03:10
jamielennoxnkinder: ouch... personal or work?03:10
nkinderjamielennox: personal.  I worked from home today03:11
jamielennoxnkinder: bugger, at least at work someone else will fix it :)03:11
nkinderluckily it was my ISP and not my network equipment03:11
nkindersome outage for my whole area03:12
ayoungjamielennox, when creating   a v3 client, if they only supply username, or only supply project name, we should throw a client side error saying domain is required03:16
jamielennoxyea,03:17
ayoungthe error you get otherwise is "project not found" which is counter-intuitive03:17
jamielennoxi was looking at that for the v3 plugin the other day03:17
ayoungor we should return a better error from POST /auth/tokens03:17
jamielennoxnot sure if it's considered backwards compat03:17
ayoung400  "user.name requires user.domain.name03:18
ayoungit fails either way, just a better error code03:18
jamielennoxok, cool i didn't test it at the time03:19
openstackgerritTim Potter proposed a change to openstack/python-keystoneclient: Convert keyring password to str before pickle.loads()  https://review.openstack.org/8016603:27
*** stevemar has joined #openstack-keystone03:33
*** wchrisj has quit IRC03:48
*** topol has joined #openstack-keystone03:50
*** gokrokve_ has quit IRC03:51
*** stevemar has quit IRC03:53
*** stevemar has joined #openstack-keystone03:55
*** lbragstad1 has joined #openstack-keystone03:57
*** lbragstad has quit IRC03:59
*** wchrisj has joined #openstack-keystone04:00
openstackgerritldbragst proposed a change to openstack/keystone: Enforce required parameters for trusts  https://review.openstack.org/7258204:04
*** lbragstad1 is now known as lbragstad04:07
*** lbragstad is now known as lbragstad__04:16
*** harlowja is now known as harlowja_away04:20
*** harlowja_away is now known as harlowja04:24
*** dstanek has joined #openstack-keystone04:25
ayoungnkinder, jamielennox I was lying in bed, and it struck me that the approach we need for user ids is this:  take one part from LDAP or IdP,  one part from the Keystone (domain id or domain name) and do the SHA256 hash of it.04:26
ayoungthe reason why the hash, aside from fitting into the size field is this:04:27
ayoungwhen I was looking at the revocation events, I realized how much information was leaking04:27
jamielennoxayoung: i'm sure we discussed this once before04:27
ayoungif a user changes their password, anyone looking at the events sees the userid04:27
*** theocean154 has quit IRC04:27
ayoungyeah...I think I needed to let it bake for a while04:27
jamielennoxwhen we were talking about whether we needed a global user_id field and it was rejected in favour of putting @@idp_Id04:28
ayoungjamielennox, it was the revocation events that made me realize the leakage04:28
ayoungI don't think we need to be able to map from the userid back to the identity provider04:28
ayoungwe need to be able to reproduce the ids.04:28
jamielennoxoh? what about change password, change roles whatever04:28
jamielennoxthey are all user_id functions04:29
jamielennoxif you need to store the parts to the hash there is no difference to using a uuid04:29
ayoungso change passwrod can say "you need to provide all the pieces"04:31
ayoungno big deal there04:31
jamielennoxthat's an api change though04:31
ayoungI think that so long as we always go from "what we know" to "what we can do" we are ok.  It is audit that I am worried about04:31
ayoungit might well require API changes...if so, it is an extension04:32
*** wchrisj has quit IRC04:33
jamielennoxif it's an extension it doesn't solve the overall problem though04:33
*** marcoemorais has joined #openstack-keystone04:42
*** marcoemorais1 has joined #openstack-keystone04:44
*** marcoemorais has quit IRC04:47
*** gokrokve has joined #openstack-keystone04:55
*** derek_c has quit IRC05:20
openstackgerritA change was merged to openstack/keystone: Add OS-OAUTH1 to consumers links section  https://review.openstack.org/7303105:36
*** derek_c has joined #openstack-keystone05:46
*** gyee has quit IRC05:51
*** harlowja is now known as harlowja_away05:53
*** gokrokve has quit IRC05:57
*** david-lyle has joined #openstack-keystone05:57
*** dstanek has quit IRC06:01
openstackgerritJenkins proposed a change to openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/7852506:01
*** david-lyle has quit IRC06:02
*** dstanek has joined #openstack-keystone06:03
*** abhishek has joined #openstack-keystone06:12
*** derek_c has quit IRC06:17
*** derek_c has joined #openstack-keystone06:32
*** abhishek has quit IRC06:42
morganfainberghehe07:00
jamielennoxstevemar: what are you trying to do?07:00
morganfainbergblech i need to go sleep =/07:00
stevemarjamielennox, just trying to exercise the auth plugin works07:00
jamielennoxi had session.get_token() there if you have to get it07:00
jamielennoxit should be sufficient to make an authenticated call and have the token added for you07:01
*** topol has quit IRC07:01
stevemaryeah, it does that, but i was hoping to verify the authorizing user and roles/project are all there07:01
jamielennoxif you call get_token() it should force the call for you07:01
stevemarit does, but it only returns back the PKI string07:02
jamielennoxyea07:02
jamielennoxso an auth plugin doesn't have to be keystone based, so you can't assume that it will have an auth_ref07:02
stevemarah i see what you mean07:03
jamielennoxif you do auth_plugin.get_auth_ref() i think it will do what you want07:03
jamielennoxi think fetch_auth_ref does the call and get_ does it if the current one is expired07:03
*** henrynash has joined #openstack-keystone07:04
stevemarjamielennox, wonderful!07:04
*** chandan_kumar has joined #openstack-keystone07:10
*** morganfainberg is now known as morganfainberg_Z07:18
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Add example script for oauth1 functions  https://review.openstack.org/8019307:22
*** stevemar has quit IRC07:33
*** derek_c has quit IRC07:52
*** saju_m has joined #openstack-keystone08:12
*** marekd|away is now known as marekd08:13
marekdmorning all.08:22
*** henrynash has quit IRC08:28
*** andreaf has joined #openstack-keystone08:30
*** henrynash has joined #openstack-keystone08:51
*** petertoft has joined #openstack-keystone08:52
*** leseb has joined #openstack-keystone08:55
*** leseb has quit IRC09:01
*** marcoemorais1 has quit IRC09:08
*** chandan_kumar has quit IRC09:30
*** marcoemorais has joined #openstack-keystone09:35
*** marcoemorais1 has joined #openstack-keystone09:36
*** marcoemorais has quit IRC09:36
*** marcoemorais1 has quit IRC09:40
*** chandan_kumar has joined #openstack-keystone09:46
marekdmhu: ok, let me try reproducing this behaviour...09:47
mhumarekd, I am in a meeting but ping me if I can help09:51
marekdmhu: no problem!09:53
*** henrynash has quit IRC10:20
*** marcoemorais has joined #openstack-keystone10:37
*** marcoemorais has quit IRC10:41
*** topol has joined #openstack-keystone10:53
*** henrynash has joined #openstack-keystone11:01
marekdmhu: replied!11:07
mhumarekd, thanks ! I'll give it a look11:09
marekdmhu: emailing stevemar and dolphm at the moment.11:09
mhumarekd, thanks for the explanation, I can write the bug report11:19
mhubut as you mention, since it is tricky to test the HTTP layer, we might miss other problems ...11:19
marekdmhu: yes :(11:19
mhumarekd, leaving for lunch now, I'll write the bug report when I am back11:20
marekdmhu: that's why i am working on configs for apache/keystone/shibboleth2.xml so one can reuse them and setup 'real' instance.11:21
marekdmhu: don't remember if I mentioned it when we were talking couple of nights ago: https://github.com/zaccone/keystone-federation11:21
marekdmhu: ok, bon app!11:21
marekdmhu: feel free to assign the bug to mysefl.11:22
*** jamielennox is now known as jamielennox|away11:22
*** amcrn has joined #openstack-keystone11:31
*** marcoemorais has joined #openstack-keystone11:38
*** marcoemorais has quit IRC11:42
*** henrynash has quit IRC11:43
*** henrynash has joined #openstack-keystone11:49
*** henrynash has quit IRC12:00
*** Ramakrishna has joined #openstack-keystone12:10
*** leseb has joined #openstack-keystone12:22
*** dims_ has quit IRC12:30
*** dims_ has joined #openstack-keystone12:34
marekdmhu: back from the lunch ?12:35
*** raildo has joined #openstack-keystone12:37
*** marcoemorais has joined #openstack-keystone12:39
*** marcoemorais has quit IRC12:43
*** andreaf has quit IRC12:44
mhumarekd, yes12:47
marekdmhu: cool, fire away the bug report, i shoul be done with the fix in few minutes :-)12:48
mhumarekd, will do12:53
mhumarekd: https://bugs.launchpad.net/keystone/+bug/129198113:05
mhuI tried to assign the bug to you but the search engine doesn't seem to work13:05
*** leseb has quit IRC13:10
*** nkinder has quit IRC13:13
*** browne has joined #openstack-keystone13:17
*** lbragstad__ has quit IRC13:19
marekdmhu: what's the link?13:23
mhumarekd, that's the bug I reported: https://bugs.launchpad.net/keystone/+bug/129198113:23
*** Ramakrishna has quit IRC13:23
marekdmhu: assigned.13:24
*** henrynash has joined #openstack-keystone13:26
*** YorikSar_ has quit IRC13:45
*** lbragstad has joined #openstack-keystone13:47
*** YorikSar has joined #openstack-keystone13:49
*** stevemar has joined #openstack-keystone13:51
*** gtt116 has joined #openstack-keystone13:52
gtt116any body using memcache backend for tokens?13:53
gtt116I found it become slow when a user has lots of token ( about 10,000 tokens)13:54
*** wchrisj has joined #openstack-keystone13:57
*** nkinder has joined #openstack-keystone14:03
stevemargtt116, what keystone release / version are you using?14:04
gtt116The stable/havana14:04
stevemargtt116, also, morganfainberg_Z is your guy for all memcache related queries14:04
*** YorikSar has quit IRC14:04
*** YorikSar has joined #openstack-keystone14:05
stevemargtt116, so you're likely not hitting: https://bugs.launchpad.net/keystone/+bug/1221087 since you're using havana14:06
gtt116stevemar, thanks for the information, but morganfainberg_Z is away now.14:07
gtt116stevemar, actually timezone is not the issue for us, I found a relatives bug here: https://bugs.launchpad.net/keystone/+bug/125112314:08
gtt116stevemar, I merged the patch for the bug, but thing does't go well14:10
*** topol has quit IRC14:11
openstackgerritMarek Denis proposed a change to openstack/keystone: Filter environment before mapping the federation rules.  https://review.openstack.org/8029314:13
marekdstevemar: dolphm mhu: ^^14:14
stevemarthanks marekd14:15
stevemargtt116, the bug wasn't backported to havana14:16
stevemarthe patch*14:16
*** vhoward- has joined #openstack-keystone14:19
gtt116stevemar, the status show "Fix released" in branch havana. Do I miss something?14:19
gtt116stevemar, the backport patch is here: https://review.openstack.org/6614914:20
*** leseb has joined #openstack-keystone14:21
stevemargtt116, my mistake, it didn't come up in my query, but the fix should be in havana14:22
*** saju_m has quit IRC14:22
gtt116stevemar: :), I afraid the fix not work very well in real environment.14:24
*** leseb has quit IRC14:25
gtt116stevemar, I am thinking about can we prune token list when call list_tokens(), rather than each time we create a new token.14:28
marekdmhu: for a quick fix you can try patching your keystone with this diff and carry on with your tests: http://pasteraw.com/q2uq905nsd2uc2n3m279hq1cxr0xqls14:30
marekdmhu: (this is a diff from a patchset i uploaded as a fix to the bug you submitted)14:30
mhumarekd, it's a devstack keystone, so I can point the repo in the gerrit review :)14:31
mhuto restack at will14:31
marekdmhu: a, okay then :-)14:31
marekdmhu: didn't know what you are using actually.14:31
*** flaper87 has joined #openstack-keystone14:31
marekdok, be back in 5 minutes14:31
*** richm has joined #openstack-keystone14:35
marekdmhu: thanks for the comments, answering them!14:35
mhumarekd, happy to hel14:35
mhuhelp14:35
*** gokrokve has joined #openstack-keystone14:37
*** andreaf has joined #openstack-keystone14:37
*** marcoemorais has joined #openstack-keystone14:40
*** marcoemorais has quit IRC14:45
stevemarmhu, if you are at atlanta summit, i owe you like 6 beers14:49
mhustevemar, hold on these 6 beers until the Paris summit :)14:49
marekdmhu, stevemar responded14:52
stevemarmhu, with inflation, it'll be 7 or 8 by that time14:52
marekdhaha, mhu, you should be happy you dont live in switzerland. They now have deflation, so it could be you to buy beer for stevemar D14:54
marekdstevemar: you can help me making commit message more english-pro14:55
*** thedodd has joined #openstack-keystone14:57
ayoungdolphm, I hate this patch https://review.openstack.org/#/c/78068/615:00
ayoungthere has got to be a better way15:00
stevemarmarekd, responded15:02
*** chandan_kumar has quit IRC15:02
stevemarayoung, why the hate?15:02
*** browne has left #openstack-keystone15:02
*** chandan_kumar has joined #openstack-keystone15:03
ayoungstevemar, because, as invasive as it is, it still doesn't solve the problem15:03
*** YorikSar has quit IRC15:03
*** prad_ has joined #openstack-keystone15:03
ayoungand I don't like being forced into a bad solution15:03
stevemarnobody puts ayoung in a corner15:04
ayoungstevemar, um... I think that quote puts me in a far less active role than I am ever willing to take15:04
ayoungI'm the Angry Young Man .  I'm always at home with my back to the wall.15:05
stevemarayoung, i like that it fixes the base_url inconsistencies15:05
ayoungstevemar, Oh, I agree we want to do that....just that host_url doesn15:05
*** YorikSar has joined #openstack-keystone15:05
*** henrynash has quit IRC15:05
ayoung''t do it to the elvel that we want15:05
ayoungand that it is so invasive at the same time15:06
stevemarit is15:06
ayoungworked hard to rip context out of all those locations15:06
*** dstanek has quit IRC15:06
*** gtt116 has left #openstack-keystone15:12
openstackgerritMarek Denis proposed a change to openstack/keystone: Filter environment before mapping the federation rules.  https://review.openstack.org/8029315:13
openstackgerritA change was merged to openstack/keystone: Cleanup backends after each test  https://review.openstack.org/7972615:15
openstackgerritA change was merged to openstack/keystone: Cleanup of instance attrs in core tests  https://review.openstack.org/7972715:16
openstackgerritA change was merged to openstack/keystone: Cleans up test data from limit tests  https://review.openstack.org/7972815:21
openstackgerritA change was merged to openstack/keystone: Cleanup fixture data added to test instances  https://review.openstack.org/7972915:21
openstackgerritA change was merged to openstack/keystone: Cleanup keystoneclient tests  https://review.openstack.org/7973015:21
openstackgerritA change was merged to openstack/keystone: Very minor cleanup to default_fixtures  https://review.openstack.org/8004015:21
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Add example script for oauth1 functions  https://review.openstack.org/8019315:21
bknudsonI thought that if I created a user with default_project_id set to a project then the user would automatically be given a role on the project.15:22
*** topol has joined #openstack-keystone15:23
bknudsonoh, that's only the v2 api.15:33
*** leseb has joined #openstack-keystone15:36
*** daneyon has joined #openstack-keystone15:38
*** david-lyle has joined #openstack-keystone15:44
openstackgerritMarek Denis proposed a change to openstack/keystone: Filter environment before mapping the federation rules.  https://review.openstack.org/8029315:53
*** marekd is now known as marekd|away16:19
*** devlaps has joined #openstack-keystone16:19
ayoungThis is going to be my new -2 response comment.       (╯°□°)╯︵ ┻━┻16:19
flaper87hey guys, what would be the best way to verify from cinder if a project exists ?16:20
flaper87i guess my question is: What's the keystoneclient method that I should be using ?16:21
flaper87:D16:21
flaper87ayoung: LOL at the -2 response16:21
ayoungflaper87, um...that is a good question16:21
ayoungflaper87, we can tell you if it exists, but not if it does not exist16:22
ayoungwhy?  cuz cinder might not have access16:22
ayoungso we return a 404 if you try to get something that you don't have access to.16:22
ayoungflaper87, is that sufficient?  It would be a GET /v3/projects/<projectid>16:23
flaper87ayoung: i think that's sufficient in this case16:26
flaper87thing is that the quota handling is not considering whether the project exists or not16:26
flaper87nor whether the id is correct16:26
flaper87or if it's a project name16:26
ayoungflaper87, you can also do a list name=?  and check the number of results if that works better.  But you should include the domain on the query if you do that16:27
flaper87ayoung: mmh, ok, I'll keep that in mind, although I think doing a get is enough16:28
ayoung++16:28
flaper87ayoung: would you mind pointing me to the right client code ?16:28
ayoungHeh.16:28
ayoungflaper87, I'm writing the example as we speak....16:29
ayoungstart with16:29
flaper87ayoung: awesome, awesome, awesome! thanks a lot!16:29
ayounghttp://www.jamielennox.net/blog/2014/02/24/client-session-objects/16:29
ayoungI have an example in one of my reviews, too16:29
ayounghttps://review.openstack.org/#/c/79096/1/examples/scripts/exercise_v3_regions.py16:30
ayoungthat is for the regions API16:30
ayoungfor a project it would be16:30
*** gokrokve has quit IRC16:30
ayoung admin_client = client.Client  ....16:31
ayoungand then16:31
ayoungadmin_client.projects.get(project_id)16:31
ayoungor16:31
ayoungadmin_client.projects.list(name=proejct_name, domain_name=domain_name)16:31
flaper87ayoung: awesome, thanks!16:32
flaper87and IIRC, the keystone_url should already be in the headers16:32
flaper87right ?16:32
flaper87or at least I remember seeing it somewhere16:32
flaper87:D16:32
ayoungflaper87, hmmm.  You need the url when you create the client, but then it gets the endpoint for the service catalog16:32
*** YorikSar_ has joined #openstack-keystone16:32
ayoungplay around with the client call, find what actually works for you16:33
flaper87ayoung: sounds good! thanks for the hints16:33
ayoungthe regions example might have more params than strictly necessary16:33
ayoungideally, we will use X509s for services to talk to Keystone and make these calls directly, without a token, in the future16:33
*** gyee has joined #openstack-keystone16:33
ayoungyou probably should be using the endpoint in the service catalog that came from the token used to make the request16:34
ayoungbut that is advanced stuff,  requires higher math and imaginary numbers16:34
ayounglike thirty-twelve and such16:34
flaper87ayoung: lol, too much for what I need to do here :D16:35
flaper87let's keep it stupid simple16:35
flaper87:D16:35
*** YorikSar has quit IRC16:36
openstackgerritA change was merged to openstack/keystone: Enforce groups presence for federated authn  https://review.openstack.org/7928416:37
*** gokrokve has joined #openstack-keystone16:37
*** dstanek has joined #openstack-keystone16:52
*** marcoemorais has joined #openstack-keystone16:55
*** afaranha has joined #openstack-keystone16:56
stevemardtroyer, i was expecting some failures, but yeesh https://review.openstack.org/#/c/80320/116:58
dtroyerstevemar: yeah, that's not a trivial change16:59
ayounghttp://tools.ietf.org/html/draft-ietf-scim-core-schema-03    Cross Domain Identity Management....yay!  Er... time to read17:00
*** marcoemorais has quit IRC17:01
*** marcoemorais has joined #openstack-keystone17:02
*** gokrokve has quit IRC17:06
*** morganfainberg_Z is now known as morganfainberg17:09
*** harlowja_away is now known as harlowja17:10
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Do not use keystone.conf.sample in tests  https://review.openstack.org/7952417:12
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Removal of test .conf files  https://review.openstack.org/7952517:12
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Move test .conf files to keystone/tests/config_files  https://review.openstack.org/7952617:12
morganfainbergmornin17:13
*** lbragstad has quit IRC17:17
*** petertoft has quit IRC17:18
*** dstanek has quit IRC17:19
*** lbragstad has joined #openstack-keystone17:23
*** gokrokve has joined #openstack-keystone17:26
morganfainbergyay test cleanup landed (should mean tests require less memory)17:33
*** browne has joined #openstack-keystone17:52
*** gyee has quit IRC17:53
*** YorikSar_ is now known as YorikSar17:55
*** YorikSar has joined #openstack-keystone17:57
*** linuxgeek_ has joined #openstack-keystone18:00
linuxgeek_on a i3 install, the ip in public endpoint url has changed after the node reboot. is there a way i can update the endpoint table in the keystone db?18:02
stevemarmorganfainberg, there was a memcached question here earlier18:03
stevemarmorganfainberg, gtt116 (irc handle) thinks he was hitting https://bugs.launchpad.net/keystone/+bug/1251123 in stable/havana18:04
stevemarjust a heads up18:04
*** flaper87 is now known as flaper87IsNotAwa18:04
*** flaper87IsNotAwa is now known as flaper87DoesNotK18:04
*** flaper87DoesNotK is now known as flaper87WTF18:05
*** flaper87WTF is now known as flaper8718:05
stevemarflaper87 is having some trouble18:05
morganfainbergstevemar, hi18:06
morganfainbergstevemar, let me see18:06
*** zhiyan is now known as zhiyan_18:10
morganfainbergstevemar, responded it looks like this is an issue w/o the fix18:12
morganfainbergstevemar, which yes, you will hit 100% cpu.18:13
morganfainbergwe have independant confirmation the fix solved the majority of the issue(s) because it doesn't have the endless memcache io18:13
morganfainbergstevemar and if they are still having an issue, the answer legitimately is "memcache shouldn't be used for token persistent"18:15
stevemarmorganfainberg, cool it did sound like the fix was probably not being included18:15
morganfainbergit has security ramifications among other things.18:15
morganfainbergyerah18:15
morganfainbergthat was my take18:15
stevemarjust wanted to give you a heads up in case you see pings/emails18:16
morganfainbergyep18:16
morganfainbergi try and circle back on LP bugs and respond.18:16
morganfainbergi really look forward to storyboard, LP is kindof a pain to use.18:16
morganfainbergstevemar, psst https://review.openstack.org/#/c/79524/ :)18:16
morganfainbergsince we got dstanek's test cleanup in18:17
morganfainberg;)18:17
morganfainbergmoar.18:17
* morganfainberg is generating a sample confighttps://review.openstack.org/#/c/80293/ for 18:17
morganfainbergwhoa copy/paste error18:18
stevemarmorganfainberg, i'll quickly add the change for that one18:19
morganfainbergi can't build my venv...18:19
morganfainbergwtf.18:19
morganfainberggreenlet issues18:19
morganfainbergotherwise i'd post a fix w/ the sample config18:19
morganfainbergoh .18:20
morganfainbergoh18:20
morganfainbergnot in a vm18:20
morganfainbergthats why18:20
bknudsonmorganfainberg: you set up a repo recently? for kite or something?18:20
morganfainbergbknudson, jamie was working on getting that together18:21
morganfainberglet me fix the review if it hasn't been fixed18:21
morganfainbergthe repo should go in on Friday18:21
morganfainbergif the review is good18:21
bknudsonmorganfainberg: nkinder was looking at setting up a repo.18:21
morganfainberghttps://review.openstack.org/#/c/73074/18:21
morganfainbergreview is passing, should go in on friday, thats when they do new repos18:21
morganfainbergirc friday that is18:22
stevemarmorganfainberg, it needs a rebase anyway18:23
morganfainbergwant me to do that18:23
morganfainberghave the revierw ready18:23
morganfainbergstevemar, ?18:24
stevemarmorganfainberg, uh, sure, it's conflicting with another federation related bug18:24
morganfainbergyeah i see the files18:24
stevemarbut if you already fixed that, then go ahead18:24
stevemari'm in the process of doing it too, but i think you're a few steps ahead18:25
morganfainbergstevemar, your call18:25
morganfainberglet me do it18:25
stevemarsure18:25
morganfainbergso you can re +218:25
stevemari'll assign the bug back to marek, sounds good18:25
ayounglinuxgeek_, yep.  But that is really why you should probably use hostnames not ipaddress.  To update the endpoint, delete the old and create the new.  You can do that with the CLI and the admin_token18:26
*** vhoward- has left #openstack-keystone18:27
linuxgeek_ayoung, i updated the endpoints with the new ip using the db. checking now if it works. yes i agree hostname is the way to go rather than ip's18:27
ayounglinuxgeek_, good to hear it.18:28
ayoungyou could do it multple ways, but DB is fine if you know what you are doing18:28
ayounglinuxgeek_, I've been messing with the python client API18:29
ayounghttp://paste.openstack.org/show/73446/18:29
ayoungyou'd create a client with18:29
ayoungadmin_client = client.Client(endpoint='http://localhost:35357/v3', token='your-admin-token-goes-here')18:30
morganfainbergstevemar, so it looks like we're supplanting the "BAD_TESTER_ASSERTION" test with the new one?18:30
morganfainbergstevemar, oh wait nvm18:30
morganfainbergderop18:30
ayoungmorganfainberg, I always raise  "BAD_TESTER_ASSERTION"  I guess because I am just a bad tester18:30
*** thiagop has joined #openstack-keystone18:32
*** dstanek has joined #openstack-keystone18:32
morganfainbergayoung, hehe18:33
*** zagranred has joined #openstack-keystone18:33
morganfainbergstevemar, just running tests to make sure i didn't cause issues and i'll be uploading a revised patchset18:33
*** zagranred has quit IRC18:33
stevemarcooliop18:33
morganfainbergayoung, the janitorial / custodial reviews look like they may be good to go (conf cleanup)18:35
morganfainbergayoung, at elast good for review.18:35
ayoungmorganfainberg, I'm still doing unspeakable things with the client18:35
morganfainbergayoung, LOL18:35
morganfainbergayoung, i figured18:35
morganfainbergdid you get the py33 stuff working?18:35
morganfainbergit was universal newlines, right?18:35
ayoungmorganfainberg, but I should have, fairly shortly, a way to verify tokens from the client, using the events and CMS18:36
ayoungI did not get it working yet18:36
ayoungthe py3318:36
morganfainbergayoung, ++18:36
morganfainbergevents and cms verify == cool18:36
ayoungI'm going to get this posted at least enough for a WIP and then go back to the py33/compressed thing18:36
*** amcrn has quit IRC18:37
morganfainbergayoung, sounds good, let me know if you need any help18:38
*** thiagop has quit IRC18:40
*** thiagop has joined #openstack-keystone18:41
*** leseb has quit IRC18:43
*** leseb has joined #openstack-keystone18:43
*** leseb has quit IRC18:48
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Filter environment before mapping the federation rules.  https://review.openstack.org/8029318:48
morganfainbergstevemar, ^18:49
morganfainbergstevemar and i reassigned the LP bug already back18:49
stevemarwoo hoo18:50
*** stevemar is now known as stevemar-afk18:53
morganfainbergdstanek, https://review.openstack.org/#/c/75816/4/keystone/catalog/backends/kvs.py String Freeze issue right? this needs to be held till Juno18:56
bknudsonmorganfainberg: I think that same message was used elsewhere already18:59
morganfainbergbknudson, hmm ok18:59
bknudsonso it'll be in the catalog18:59
morganfainbergbknudson, that was what i was wondering.18:59
dstanekbknudson, morganfainberg: yes it's used in assignment kvs18:59
morganfainbergbknudson, didn't want to -2 if it legitimately wasn't a SF issue18:59
openstackgerritPablo Fernando Cargnelutti proposed a change to openstack/keystone: Moving delete_user and delete_group calls to IdentityManager  https://review.openstack.org/8036819:01
openstackgerritA change was merged to openstack/keystone: Add unit tests for disabled endpoints in catalog  https://review.openstack.org/7737519:08
*** linuxgeek_ has left #openstack-keystone19:08
openstackgerritA change was merged to openstack/keystone: Fix include only enabled endpoints in catalog  https://review.openstack.org/7744119:12
dstanekmorganfainberg: had a quick question on https://review.openstack.org/#/c/79525/3/keystone/tests/core.py19:12
morganfainbergsure19:13
morganfainbergwhoa19:13
morganfainberghow did that one get added.19:13
morganfainbergoh19:14
morganfainbergoh19:14
morganfainbergthere is some issue with the cache fixture not getting setup correctly19:14
morganfainbergif you don't call setup directly you get a nasty exception19:14
morganfainbergit's about the lack of some property let me 2x check that. and see if i can chase down the specifics.19:15
morganfainbergthat might have been a stub that got left in19:15
dstanekmorganfainberg: really? hmmm...that sucks because if we ever add cleanup we'll have to manually call that too19:15
morganfainbergno the cleanup works, it was an issue with _details not being populated19:15
morganfainbergi think it was a workaround not needed sec.19:15
morganfainbergi think it was a stub i had to work around19:17
morganfainbergrunning tests now w/o it and it seems to be passing19:17
ayoungtopol, question for you.  In your proposal for audit, are you going to log username, or just userid?19:21
*** leseb has joined #openstack-keystone19:23
ayoungmorganfainberg, dstanek check me on this:   if we somehow "anonymize" the user ids (henrynash's user id thing)  if we can always get the userid from the username/domainid, then we can probably just record tusername/domain id in audit events.19:24
ayoungso "change password" and any identity API calls that need to find a specific backend would be broken19:24
ayoungbut, hey, its Federation19:24
ayoungthose probably shouldn't be in Keystone's control anyway19:24
ayoungand we could do userid = sha256('username@@domainname")  or something19:25
*** petertoft has joined #openstack-keystone19:25
*** leseb has quit IRC19:25
dstanekayoung: would you log that sha256 or the source data for it?19:31
ayoungdstanek, "log" ?19:31
ayoungmeaning audit log?  I don't think so...least it wouldn't mean much19:31
topolayoung username19:34
ayoungdstanek, the audit log would be expected to have something that could uniquely identify the user.  If userid is generated from username and domain id, but it was not a reversable lookup,  you'd need the source material19:34
ayoungtopol, so thinking of henrynash's problem19:34
ayoungand I think that audit is my major concern19:35
* topol topol needs some context. whic problem19:35
ayoungif we anonymize, we break audit19:35
ayoungids fro multiple LDAP19:35
topolayoung whatw as he going to anonymize???19:35
ayoungand for multiple IdPs in the future as well19:35
ayoungtopol, remember the "ephemeral users"  issue?  Assign each user an ID based on some entry in either a mapping table or something?19:36
topolwe would have to have a an audit record of what the ids were mapped to to keep from breaking19:36
ayoungyeah, but It seems fragile19:36
topolwould nt we do an audit record that captures the mapping???19:36
ayoungtopol, so, say a user came in to two different keystone servers, and got two different Ids?19:37
topolayoung so we need to come up with a solution that hoepfully does not break auditing. I believe henrynash wants to make sure audit still works. Havent check on what he thinks is  a feasible solution19:37
*** devlaps has quit IRC19:38
ayoungOr, if somehow the mapping table gets out of sync19:38
ayoungtopol, this is a topic for the summit.19:38
topolayoung, I agree we need to discuss and vet19:38
ayoungI just had an "ok, we can do this if..." moment last night19:38
*** devlaps has joined #openstack-keystone19:39
topolayoung, yes I was planniong on submitting this as a summit topic (audit + federation).  I can go ahead an get that submitted19:39
topol(joint with martinelli)19:39
topolayoung, make sense?19:40
ayoungso if an audit event records the username, not just the userid, then I think we'll be set.  dolphm was concerned that we were going to "leak" data out of Keystone, such as what users are actively working in there, and that we shouldn't be using anything other than uuids.  But I don;t trust the idea of the mapping table, so I want something that is consistant regardless of whether there is an entyr in the usertable before th19:40
ayounge user visits or not19:40
*** amcrn has joined #openstack-keystone19:41
ayoungsha256 ("%s@@%", [username, domainname])  or something19:41
ayoungit is not perfect19:41
topolayoung for compliance requirments I think it is Ok and required to leak19:41
ayoungwith revocation events, you are going to see one event per, say, password change, and if that is not a tightly controlled call, there will be a lot of data leaking19:41
topolayoung, why?19:42
ayoungso I agree some degree of anonymization is called for.  That was the real epiphany19:42
ayoungtopol, reread it.  I just explained19:42
ayoung you are going to see one event per, say, password change.  event will have the userid in it19:42
ayounganyone can read all of the evetns19:42
ayoungevents19:42
topolthat would be bad19:42
*** devlaps has quit IRC19:43
topolperhaps for certain audit events we filter out the values if leaking too much19:44
topolfor a password change it would just say so and so's password changed. it wouldnt have the values in it19:45
topolayoung19:45
ayoungtopol, so if you change your password, a revocation event is generated with your userid19:46
*** henrynash has joined #openstack-keystone19:46
ayounghenrynash, were your ears buring?19:46
topolyay, henrynash!19:46
ayoungburning19:46
henrynashtopol: hi19:46
topolmy mental telapathy still works19:47
topolayoung, please recap19:47
ayoungtopol, I can do better19:47
*** andreaf has quit IRC19:47
ayoung2014-03-13T19:24:04   on http://eavesdrop.openstack.org/irclogs/%23openstack-keystone/%23openstack-keystone.2014-03-13.log19:48
ayounghenrynash, ^^19:48
topolayoung, you gonna make him read the whole log????19:49
ayoungtopol, nope, but the datatime I posted shows the start of the conversation19:50
*** gokrokve has quit IRC19:50
*** gokrokve has joined #openstack-keystone19:50
henrynashreading…reading…reading...19:51
topolso henrynash net is ayoung is worried about ephemeral ids need for LDAP federation breaking audit support19:51
henrynashtopol: ok19:52
ayounghenrynash, I was saying that it is ok to do userid = sha256( "username@@domainname")  if we don;t break audit19:52
henrynash(fyi, I did post a design summit session on this…."Entity IDs in a multi-backend or Federated World"19:52
ayoungthe problem with audit is it needs to do a backwards mapping from userid to original subject19:52
*** gokrokve has quit IRC19:52
ayounghenrynash, 1 posted one too19:53
henrynashayoung: (to play devil's advocate) so audit *could* of course use the mapping to decode the real user and log that event19:54
henrynash(log that in the event)19:54
ayounghenrynash, what if there are multiple keystone servers in play?19:54
topolhenrynash, good idea19:54
*** eglynn_ has joined #openstack-keystone19:54
ayounghenrynash, so, yeah, the mapping table is essential if there is no other way to map users for, say, audit19:55
henrynashayoung: and the issue tat would cause is….19:55
ayounghenrynash, I'm trying to avoid a data duplication and sync setup with Keystone19:55
eglynn_good evening folks19:56
ayounghenrynash, so...lets treat the mapping table as a last resort19:56
ayoungand...if we make it possible to go from audit events to userid, I think we can do that19:56
henrynashayoung: oh, so I'm with you on that one…my view is that we only allow ourselves to be forced into storing some kind of independent mapping if we prove to ourselves that encoding the info within the identifier itself cannot be the right solution19:56
henrynash:-)19:57
eglynn_I'm wondering whether the --os-cacert keystoneclient option is expected to be propogated?19:57
ayounghenrynash, so,  lets start with sha256("username@@domainname")19:57
topolwhy must we do sha256?19:57
henrynashayoungL but given the output of that, can I get the domain name?19:57
ayoungit will look just like UUIDs do, and won't be backward's mappable, which, to some amount, addresses dolphm 's concern on leaking information19:57
henrynashayoung: I.e. can i reverse it19:58
eglynn_this seems to imply that it allows the cacert to be specified when verifying the service side cert when doing https19:58
eglynn_https://github.com/openstack/python-keystoneclient/blob/master/keystoneclient/shell.py#L18719:58
ayounghenrynash, no, from userid, you would not be able to get domain name19:58
ayoungand I think I am OK with that as it means we can't go from, say, assignments to identity, but we can go the other way19:58
henrynashayoung: errrr, so how does this help the problem we started out to solve19:58
henrynash?19:58
ayounghenrynash, it means the userids are naturally deconflicted between IdPs (LDAP included)19:59
ayoungit means that they look like and act like UUIDs19:59
ayoungand it means we don't need a shadow table19:59
henrynashayoung: the problem we started with was "given this user_id, I need to route it to the right backend"19:59
ayoungso, yeah, you can't change a password with just a sha256 based id19:59
ayounghenrynash, but I don't think that is a real problem19:59
eglynn_folks, anyone know if say "keystone --os-cacert /etc/keystone/ssl/certs/ca.pem endpoint-list" should cause ca.pem to be used to verify the CA for the keystone service's cert?19:59
topolwhy isnt this just another column in the routing table???20:00
ayoungeglynn_, the question is whether that is necessary or sufficient20:00
morganfainbergayoung, do we want anonymous or deconstructible ids? I am all for doing a sha of the combo.20:00
ayoungif should be necessary, not sure if it is sufficient20:00
morganfainbergayoung, sorry reading scrollback20:00
ayoungtopol, "routing"  I think you have neutron on the mind20:00
eglynn_ayoung: sufficient in the sense of allowing the client access?20:01
henrynashayoung: but that IS the problem we have right now….e.g. get_user(user_id)……which backend is that for?20:01
topolif we start out knowing the user name and domain we should be to deconstruct and get back the knowledge we used to have20:01
ayounghenrynash, I am OK for breaking "identity calls work for all backends with just userid"20:01
eglynn_ayoung: ... well the equivalent curl -k option works for me20:01
henrynashayoung: wow, well if you're ok breaking that, then sure anything is possible….I don't know if we can break that20:02
eglynn_i.e. this works: curl -k  /etc/keystone/ssl/certs/ca.pem -i -X POST https://172.16.12.49:35357/v2.0/tokens ...20:02
ayoungeglynn_, I need to try an SSL ified keystone setup and see what happens.20:02
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Removal of test .conf files  https://review.openstack.org/7952520:02
ayounghenrynash, what calls is that really breaking?20:02
topolideally there should be a breadc crumb in the user id to help you work backwards20:02
ayoungreally just stuff to identity20:02
morganfainbergdstanek, removed that one line from the test cleanup you had a question about, looks like it was a stub i needed while working through everything20:02
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Move test .conf files to keystone/tests/config_files  https://review.openstack.org/7952620:02
ayoungtopol, or we could say that , in a federated case, for those apis, you need to provide domainid20:02
henrynashayoung, so authenticate with user_id, get/delete of user, group20:02
eglynn_ayoung: OK, I'm gonna go raise it in LP, it smells bug-like to me20:03
henrynashayoung: also, add/check/remove user from group20:03
topolcould the last for digits of the user id be a code that tells you which domain to use :-)?20:04
henrynashayoung: to name the ones of the top of my head :-)20:04
morganfainbergayoung, we could use uuid5 vs sha25620:04
ayounghenrynash, ok,  so authenticate with userid...for Federated is not an issue20:04
ayoungyou provided a SAML doc20:04
morganfainbergtopol, ^20:04
ayoungfor LDAP....you are going to use username and password20:04
henrynashayoung: it is for multi-backend20:04
ayoungor Kerberos20:04
topolmorganfainberg, i saw that20:05
ayounghenrynash, I guess that someone could cache their userid and pass it back for authentication, but, is anyone really going to do that?20:05
henrynashtopol: so , yes, there are such schemes….ayoung suggested we do something like git does with the first 8 chars20:05
morganfainbergtopol, i don't see a need to use a sha256 but eh.20:05
topolmorganfainberg, I agree20:06
ayoungmorganfainberg, just the currrent "best practice"20:06
ayoungwe can use md5,  IDRC20:06
morganfainbergif we used a UUID (real uuid) uuid5 (takes namespace + other data)20:06
henrynashayoung: I agree that authenticate with  user-Id is the least of our problems20:06
morganfainbergthe downstream consumers can compress the data to 16 bytes if they are concerned about space20:06
morganfainbergwhich was one reason other people like uuids20:06
ayounghenrynash, so we make the assumption that, for multi backend, the data is only going to flow one wa: from identity to assignment. You need to start with the key for identity, which is the username20:07
morganfainberg"real uuid" just means hex 32bytes20:07
ayoungmorganfainberg, it would not be reversable, right?20:07
morganfainbergayoung, no it isn't reversible20:07
ayoungI mean, I can't see how it possibly could be , but I had to ask20:07
morganfainbergayoung, i do have that b64 magic code that gets us 64byte ids20:08
morganfainbergthat is reversible20:08
morganfainbergayoung, not that it's a "good idea"20:08
ayoungmorganfainberg, so I think I'm ok with that scheme so long as audit emest the username and domainname or domainid on events20:08
ayoungemits20:08
morganfainbergayoung, audit could deconstruct the data easily20:08
topolmorganfainberg, thats a critical designcontraint20:09
morganfainbergthe biggest downside is that it _requires_ no more than 32 bytes of "user_id"20:09
morganfainbergor we need 16 bytes or less of domain_id20:09
*** devlaps has joined #openstack-keystone20:09
morganfainbergi was doing uuid magic to use binary form20:09
morganfainbergwe got 48 bytes of data, or we go with the <user_component>@@<partial domain>20:10
morganfainbergand get more space20:10
morganfainbergas a raw string20:10
ayoungmorganfainberg, but that approach was sem reversable, right?  We would actually be able to extract the domain id out of the userid?20:11
*** petertoft has quit IRC20:11
morganfainbergayoung, corred.20:11
morganfainbergayoung, correct* let me find that example i had20:11
ayoungcor, chief!20:11
henrynashso for me it is pretty simple, we need to ask ourselves teh following questions:20:11
ayoungmorganfainberg, so, lets table that for a moment20:12
henrynash1) To we need to hide true identity in IDs20:12
*** leseb has joined #openstack-keystone20:12
ayoungmorganfainberg, cux, while it is cool, it also puts some restrictions on length that we might want to avoid20:12
morganfainbergayoung, i agree20:12
henrynash2) IS it our job to scrunch down the format of an ID so that keystone clients can efficiently store/refer to it?20:12
morganfainbergayoung, the only real limit we have is 64 bytes, and i don't want to go over that.20:12
ayoungmorganfainberg, ++20:12
morganfainberghenrynash, i do like a consistent format for ids20:13
morganfainberghenrynash, regardless of the choice on format20:13
morganfainbergas long as all ids (new, ignoring old) can conform, i'm happy20:13
ayounghenrynash, so 1)  is a design goal.  We don't do that now, but I would like to20:13
ayoung2)  I would say is a short term requirement, otherwise we break other projects DBs20:14
*** devlaps has quit IRC20:14
morganfainbergand 64bytes is the limit, though if we used a raw uuid, clients would be happier since it is smaller20:14
morganfainbergayoung, ++ on 220:14
henrynashmorganfainberg: so if you make that a requirement, then a mapping table is the only answer, since we cannot control the ID format of ReadOnly LDAPs20:14
morganfainbergbut we've had 64 bytes accepted as the maximum, so we can consume up to that for ID (in any form)20:14
ayoungmorganfainberg, so if we do sha2  appraoch, we need such that base64(sha2(useridscheme)) <= 32 chars for current nova approach and <=64 chars for the rest of the world20:15
ayounghenrynash, mapping is required, but not in a table20:15
morganfainbergayoung, nova will get fixed.20:15
morganfainbergayoung, we need to make sure whatever = max of 64bytes20:16
morganfainbergi have a review in to fix nova, so we can press that issue if needed20:16
henrynashayoung: I'm open to how we do it without it in a way that is reversable20:16
ayounghenrynash, so if the userid is calculated from an LDAP portion and a domain portion assigned by keystone, we have a mapping as well, just not one we need to explicitly record20:16
ayoungthe reversable is, I think the part that is negotiable20:16
morganfainbergayoung, if reversible is negotiable, lets go w/ uuid5, 32 bytes, everyone is happy and we can make sure it's 100% consistent20:17
ayoungOK, I've had my say....we are all in sync on the issues, just not the solution yet20:17
henrynashayoung:++ this is a nice summit topic if I ever saw one!20:17
morganfainberghenrynash, ++20:18
morganfainbergayoung, anyway.20:18
ayoungmorganfainberg, uuid5 limits the length of the ids...lets keep it on the table as one of the approaches.20:18
henrynashayoungL agreed20:18
ayounghenrynash, I'll try to have some reading material produced prior to the summit to summarize20:18
eglynn_ayoung: here's a detailed description of that --os-cacert issue https://bugs.launchpad.net/python-keystoneclient/+bug/129220220:18
morganfainbergayoung, uuid5 is really sha1 of uuid + <some string>20:18
henrynashayoung: I was going to write up a BP that described the approaches we have deiscussed20:18
morganfainbergayoung, uuid = domain uuid20:19
ayoungeglynn_, I assigned to jamielennox|away20:19
henrynashayoung: weithour suggesting which is the one to chose20:19
ayounghenrynash, ++20:19
morganfainbergyay! good summit topics20:19
henrynash(nipping off line back later)20:19
ayoung12 summit sessions for keystone proposed20:19
morganfainbergayoung, yeah20:19
ayoungI see 3 that can be combined already20:20
morganfainbergayoung, ++ was about to say similar20:20
ayoungSupport for Hierarchical administrative boundary,  Discoverable and Hierarchical Catalog,  and  RBAC and Policy20:20
morganfainbergi also see one or two that probably should be Dev lounge + irc20:20
openstackgerritPablo Fernando Cargnelutti proposed a change to openstack/keystone: Moving delete_user and delete_group calls to IdentityManager  https://review.openstack.org/8036820:26
*** topol has quit IRC20:27
morganfainbergdstanek, tests are way better now btw20:44
morganfainbergdstanek, glad your changes landed20:44
morganfainbergdstanek, i don't know if you were paying attention when i asked this before, but blinker library20:45
dstanekmorganfainberg: i'm glad they're better for you - i wasn't able to even run them before those changes20:45
dstanekmorganfainberg: yay, blinker :-)20:46
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: Allow hash tokens with sha256  https://review.openstack.org/8039820:46
morganfainberginstead of the notificaiton callbacks, should we use that? In the case of blinker can we ensure that a given callback only is registered a single time? e.g. the version instantied first (e.g. service.py) registers first20:46
morganfainbergand the subsequnt ones don't register, prevents 5 managers from trying to do deletions for the same event20:46
morganfainbergor are we going to need to add a shim layer over the top to help limit the listeners to 120:47
morganfainbergi'm all for in Juno moving to blinker (i actually think the decorator for listening is way cooler)20:47
dstanekmorganfainberg: i think you'd need a small shim20:47
morganfainbergin fact, even if we don't move to blinker, i expect i'll convert over to a decorator in J120:48
dstanekmorganfainberg: typically with blinker you'd have multiple things looking for signals to do some logic20:48
morganfainbergwaaaaaay better than the event callback registration20:48
morganfainbergright.20:48
morganfainbergmaybe we need to "fix" managers to be more of a singleton than they are20:48
dstanekmorganfainberg: maybe i'll whip up a prototype to see what it would look like. I'm intestested now20:49
morganfainberge.g. never call __init__, call .get_manager()20:49
morganfainbergmake __init__ tell people to go away if it's called directly20:49
morganfainbergif we moved to a "singleton"-like model for the managers it would solve that issue implicitly20:49
dstanekmorganfainberg: i think it would be better to do real injection and leave the __init__s as is20:50
dstanekusually your injected would create the instance and all the the depencencies20:50
morganfainbergsure.20:50
dstanekit should controll the graph20:50
morganfainbergi'm now wondering if there is _ever_ a case we should allow multiple of a given manager to be instantiated20:51
morganfainbergregardless of how we handle the injection20:51
morganfainbergdoesn't change your point nor implementation really20:51
dstanekprobably not, but unit tested injected managers would be easier if you could just pass in the deps to it's __init__20:51
dstanekerr...its20:52
morganfainbergdstanek, like i said, wouldn't make a difference on implementation if we made it so managers only were instantiated once20:52
morganfainbergdstanek, that would be icing to a well defined dep injection20:52
openstackgerritBrant Knudson proposed a change to openstack/keystone: Configurable token hash algorithm  https://review.openstack.org/8040120:53
morganfainbergdstanek, lets table making managers more singleton-like until later.20:53
morganfainbergdstanek, but i would love to move to blinker signaling wherever possible20:53
morganfainbergit would be waaaaay cooler for cross-manager notificaiton stuff20:54
dstanekmorganfainberg: i'll get on that after dinner. i've been wanted to dive into our notifications code to really understand it anyway20:54
morganfainbergdstanek, i brought this up because https://review.openstack.org/#/c/80368/ and see my comment on it20:55
*** jamielennox|away is now known as jamielennox20:55
morganfainbergjaypipes, ping - callback stuff20:55
morganfainbergjaypipes, the issue is having a ton of methods you need to add to other managers. each new manager needs to be added to "identity" or some such because it has an action that occurs on user deletion20:56
morganfainbergjaypipes, i would much prefer the dependant manager (e.g. credential needs to act on user delete) define what it's looking for instead of the inverse20:57
morganfainbergjaypipes, identity shouldn't have to care what other managers need to take an action on user_delete, it should just say "hey user X is deleted"20:57
jaypipesmorganfainberg: I'd be fine with that if the firing of the dependent hooks was explicit and not hidden away in some decorator magic.20:57
morganfainbergjaypipes, ^ see the discussion we're had on this just before my ping20:58
dstanekjaypipes: i think firing would look like 'user_deleted_signal.send(user_ref)' and then the signal library would call all registered handlers20:58
morganfainbergand the registered handlers would be configured via a decorator to show they are listening for these events.20:59
morganfainbergiirc20:59
dstanekregistration of the handlers would be very obvious20:59
ayoungOK...I am really starting to think we need a set of live tests for the keystone client.20:59
morganfainbergdstanek, ++20:59
morganfainbergjaypipes, what we have is a decent first pass, but for J we will make it _way_ better20:59
ayoungjamielennox, eglynn_ had a question earlier about SSL and the client.  Read up, but he filed a bug and I assigned to you.21:00
ayoungI suspect it is a documentation issue, but didn't want to say for sure21:00
*** afaranha has quit IRC21:00
dstanekmorganfainberg, jaypipes: i'll take a first pass at what i think it would look like and add you to the review21:00
jaypipesmorganfainberg: how do you deal with correct ordering of callbacks?21:00
ayoungdstanek, morganfainberg is there some way to make testr tests conditional?  Like :  only run these tests if a specific environment variable is set or something?21:01
ayoungI'd love to be able to do21:01
morganfainbergayoung, hmm.  i think that would be in the test cases not testr...21:01
ayoungrun_tests.sh  --backend=mysql21:01
morganfainbergayoung, i don't know though21:01
ayoungmorganfainberg, nah, the whole suite, or class or something21:02
morganfainbergayoung, in setUp()21:02
morganfainbergayoung, look for os.environ['<env>']21:02
ayoungsometihng about the way it does discovery21:02
morganfainbergoh you don't even want skips21:02
ayoungyeah21:02
*** marcoemorais has quit IRC21:02
ayoungskip the whole suite21:02
morganfainbergyou could just make the skip say Skipped because <blah is disabled>21:02
morganfainberg*shrug*21:02
morganfainbergor skipped because not live test21:02
ayoungmorganfainberg, nope21:03
*** marcoemorais has joined #openstack-keystone21:03
ayoungunless I could do that from setuP21:03
ayoungdoesn;t work ofr the live tests21:03
bknudsonyou can skip in setup21:03
morganfainbergayoung, you can21:03
ayoungthey are specified the same way regardless of the database21:03
dstanekayoung: we have a decorator for some tests now to do something similar - don't know what approach it used21:03
bknudsonipv6 tests?21:03
dstanekayoung: i don't think it's reusable though21:03
morganfainbergayoung, if you issue the skiptest (is it a raise method?) if you do the skip in setUp the whole class is skipped21:03
dstanekyeah, i think those are them21:03
morganfainbergjaypipes, i'm trying to think of a way that you have an ordering issue.21:03
morganfainbergjaypipes, if each manager only acts on it's driver, order doesn't really matter21:04
ayoungmorganfainberg, OK...so I could in theory make some tests in the python-keystoneclient that talked to a live server, and would only be run if I said, say "auth_url=...."21:04
dstanekjaypipes: if order matters then there should be 1 callback doing two things instead of trying to order two different callbacks21:04
morganfainbergdstanek, ++21:05
jaypipesmorganfainberg: if I delete credentials before I delete user groups assignments, and things fail in between, is there any issue? that's what I meant by ordering issues...21:05
morganfainbergjaypipes, the delete would occur after the user is deleted, so they aren't relevant in either case21:05
jaypipesmorganfainberg, dstanek: all in all, I just prefer non-magical, non-hidden coding :) I'll wait to see your PoC on this and comment after, ok?21:06
morganfainbergjaypipes, but in the case that something does matter order wise, it can be in a single callback (as dstanek outlined)21:06
morganfainbergjaypipes, ++ sounds good21:06
jaypipescoolio. :) thx guys!21:06
morganfainbergjaypipes, but the fix will be Juno because we need a new requirement addition21:06
morganfainbergjaypipes, the cleanup that is.21:06
dstanekmorganfainberg, jaypipes: sounds good to me, thx21:07
jaypipessure, no worries, I figured as mucjh21:07
morganfainbergayoung, give me a sec, i'll dig up how to skip21:07
ayoungmorganfainberg, thanks....I think that we should redo the _livetests with that like, now21:08
morganfainbergayoung, self.skipTest('<message>')21:08
morganfainbergayoung, should work in .setUp21:08
ayoungmaybe tomorrow, certainly before Ice rc21:08
ayoungschweet21:08
morganfainbergsec21:08
morganfainberg2x checking21:09
dstanekbknudson, ayoung: hmmm...this is what i was thinking about; thought it was a decorator - http://git.openstack.org/cgit/openstack/keystone/tree/keystone/tests/test_ssl.py#n8721:09
morganfainbergayoung, yep./21:09
morganfainbergayoung, :)21:09
bknudsondstanek: maybe testtools provides a decorator, too?21:10
bknudsonbut then you'd have to pass it a function.21:10
morganfainbergbknudson, there is a decorator21:10
morganfainbergbknudson, but just calling self.skipTest(<message>) is sufficient21:10
*** devlaps has joined #openstack-keystone21:10
morganfainbergif you look at how the skip_if_no_ipv6() works21:10
morganfainbergwell it does a raise, but .skipTest also does a raise21:10
*** devlaps has quit IRC21:11
*** devlaps1 has joined #openstack-keystone21:11
morganfainbergskip_if_no_ipv6's raise call is superfluous21:11
morganfainbergayoung, livetests working on env could make things a lot easier for eliminating the last of the config files21:12
*** thiagop has left #openstack-keystone21:12
morganfainbergthen no more config files in tests! yay!21:12
morganfainbergexcept when we're testing config file things specifically21:13
openstackgerritBrant Knudson proposed a change to openstack/keystone: Configurable token hash algorithm  https://review.openstack.org/8040121:13
*** gyee has joined #openstack-keystone21:14
morganfainbergactually... hmm21:14
dstanekit should be easy to write '@skip_if_config("opt.to.look-for", "off-value")'21:14
dstanekyou could also have a skip_if_env that uses an environment variable21:15
morganfainbergskip_if_not_env21:15
morganfainberg:P21:15
morganfainbergi think the backend_ldap.conf is the only one that would be really hard to model out because of the number of variations on ldap configs.21:16
morganfainbergerm live_ldap21:16
ayoungmorganfainberg, I think we could probably keep a config file for LDAP tests.21:17
morganfainbergayoung, do we actually need backend_ldap.conf?21:18
ayoungmorganfainberg, only if we don;t use the defaults21:18
morganfainbergare we ever testing those against a live ldap?21:18
morganfainbergno i mean, do we expect our test suite to run against a live ldap server?21:18
ayoungmorganfainberg, we could easily do it using config values from inside a test21:18
ayoungjust if someone wants to run it against their own setup, it would be nice to allow for a config file21:18
morganfainbergayoung, hm.21:19
ayoungyeah, but that can be round 221:19
morganfainbergi think we don't cleanup the ldap trees properly between tests21:19
*** ayoung is now known as ayoung_dad_mode21:19
ayoung_dad_modemorganfainberg, we do, or at least we did21:19
morganfainbergwe get away with it for fakeldap since we can nuke it in memory21:19
morganfainberghmmm *checks*21:19
*** dstanek is now known as dstanek_afk21:19
dstanek_afki'll be back after dinner21:19
*** petertoft has joined #openstack-keystone21:21
nkindermorganfainberg: I run the tests against live ldap21:31
morganfainbergnkinder, hm...21:31
morganfainbergnkinder, oooh i see how they work21:31
morganfainbergnkinder, i was looking in the wrong place21:31
nkindermorganfainberg: I use backend_liveldap.conf though21:31
morganfainbergyeah21:31
nkindermorganfainberg: I plan to try your recent patches against my live setup, but I can't do any real work like that until I get off of work and go home :P21:32
morganfainbergthey shouldn't impact your live setup for the most part21:32
nkinderI don't expect them too.  The changes all look fine.21:33
morganfainbergi kept the bulk of the ldap settigns out of the config fixture21:33
*** petertoft has quit IRC21:40
*** lbragstad has quit IRC21:54
*** david-lyle has quit IRC21:58
*** gabriel-bezerra has joined #openstack-keystone22:02
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Make LIVE Tests configuable with ENV  https://review.openstack.org/8040922:04
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Make LIVE Tests configurable with ENV  https://review.openstack.org/8040922:04
morganfainbergayoung_dad_mode, ^ when you're back22:05
*** leseb has quit IRC22:07
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Do not use keystone.conf.sample in tests  https://review.openstack.org/7952422:08
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Removal of test .conf files  https://review.openstack.org/7952522:09
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Move test .conf files to keystone/tests/config_files  https://review.openstack.org/7952622:09
*** nkinder has quit IRC22:14
*** leseb has joined #openstack-keystone22:15
*** gabriel-bezerra has quit IRC22:17
morganfainbergbknudson, i'm baffled22:22
morganfainbergbknudson, your catalog disabled tests are breaking in my remove the config files chain22:22
bknudsonmorganfainberg: we all are.22:22
morganfainbergbknudson, and i have no idea why i'm getting numerical ids out of it22:22
morganfainberginstead of the uuidx22:23
bknudsonmorganfainberg: maybe the templated backend?22:23
morganfainbergoh .22:23
bknudsonthat obviously requires a config file.22:23
morganfainberghmm.22:23
morganfainbergoh i see what is going on22:23
morganfainbergbackend_sql doesn't include the config now22:23
morganfainbergthanks22:23
morganfainbergthis was bugging me22:24
bknudsondefault_catalog.templates22:24
morganfainbergyep22:24
morganfainbergthanks22:24
bknudsonmorganfainberg: not so baffled anymore.22:24
morganfainbergbknudson, yep thanks22:26
morganfainbergbknudson, i obviously need more coffee22:26
bknudsonmorganfainberg: catalog.RegionOne.identity.id = 122:26
*** nkinder has joined #openstack-keystone22:26
morganfainbergyep.22:26
morganfainbergsee it now22:26
bknudsonthe templated catalog actually seems like a good idea.22:26
bknudsonthe db lookups for the catalog must be pretty slow22:27
bknudsonthe db lookups could probably be improved22:27
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Removal of test .conf files  https://review.openstack.org/7952522:28
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Move test .conf files to keystone/tests/config_files  https://review.openstack.org/7952622:29
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Make LIVE Tests configurable with ENV  https://review.openstack.org/8040922:29
morganfainbergbknudson, that seems to have solved my issue22:30
morganfainbergbknudson, i think we could be a lot smarter abourt building the catalog in SQL22:30
morganfainbergbknudson, and/or add caching over the top22:31
bknudsonmorganfainberg: it's potentially different for each request...22:31
bknudsonbut it also doesn't change much22:31
morganfainbergbknudson, we could be smart about taking the whole catalog and process it in memory to the "correct" version22:32
morganfainbergbknudson, instead of relying on SQL to do that for us22:32
bknudsonmorganfainberg: essentially turn it into the kvs version.22:32
morganfainbergbknudson, but with a SQL back (e.g. smart about loading from the SQL)22:32
morganfainbergbknudson, but yes.22:32
*** prad_ has quit IRC22:44
*** wchrisj has quit IRC22:46
*** leseb has quit IRC22:50
mfischcan someone explain why the users list returns a tenantId along with the user?22:50
*** eglynn_ has quit IRC22:58
*** dims_ has quit IRC22:58
*** browne has left #openstack-keystone23:00
*** stevemar-afk has quit IRC23:03
openstackgerritA change was merged to openstack/keystone: Remove unnecessary oauth1.Manager constructions  https://review.openstack.org/7921323:05
openstackgerritA change was merged to openstack/keystone: Updated from global requirements  https://review.openstack.org/7669123:05
*** henrynash has quit IRC23:09
*** dims_ has joined #openstack-keystone23:12
openstackgerritBrant Knudson proposed a change to openstack/keystone: Add a space after the hash for block comments  https://review.openstack.org/7811623:13
bknudsondstanek_afk: had to rebase ^23:13
*** thedodd has quit IRC23:21
*** Fin1te has joined #openstack-keystone23:31
*** dstanek_zzz has joined #openstack-keystone23:36
*** dstanek_zzz is now known as dstanek23:37
dstanekbknudson: thanks23:37
*** daneyon_ has joined #openstack-keystone23:37
*** flaper87 is now known as flaper87|afk23:37
morganfainbergi think i just helped chase down a revoke_api API breaking change23:38
morganfainbergas in, a choice made on how revoke_api works breaks previously allowed API calls23:38
bknudsonmorganfainberg: I thought revoke_api was experimental and couldn't cause any problems?23:38
morganfainbergbknudson, well, apparently it loads by default23:38
morganfainbergbknudson, i thought it was not loading23:38
dstanekbknudson: mutables as defaults args are bugs just waiting to happen - it creates a sort of static variable and people don't expect that23:39
bknudsonmorganfainberg: really?23:39
morganfainbergif we fix it's optional-ish nature so it doesn't load, we will solve the issue23:39
*** petertoft has joined #openstack-keystone23:39
bknudsondstanek: that sounds like it could be handy23:39
morganfainbergbut, here is the issue: UUID tokens, on switching projects there is a token delete on the old token23:39
dstanekbknudson: it is sometimes, but usually it just causes bugs23:39
morganfainbergthat delete uses the expiration_time in revoke events to revoke the specific token23:40
morganfainbergall tokens based on that expiration time (from the unscoped login to horizon) are now invalid23:40
*** YorikSar_ has joined #openstack-keystone23:40
morganfainbergto unscoped token -> token 1 -> token 2 -> token 3, if you revoke token 1, all tokens are revoked in the revoke api23:40
morganfainbergsince we key on expiration time23:41
morganfainberghttps://bugs.launchpad.net/horizon/+bug/129109923:41
morganfainbergjust associated this to keystone23:41
*** YorikSar has quit IRC23:41
morganfainbergi'm going to see what i can do to fix this23:42
dstanekbknudson: people don't expect this behavior - http://paste.openstack.org/show/73468/23:42
bknudsonmorganfainberg: revoke_extension isn't loaded by default as far as I can tell.23:42
bknudsonit's not in the paste pipeline23:42
morganfainbergbknudson, well i have a coworker with a clean devstack showing this behavior23:42
morganfainbergbknudson, trying to figure out how it's getting loaded23:43
bknudsonthe bug is that revoke extension is loading by default then23:43
morganfainbergwell it's twofold23:43
morganfainberg1: loading by default, 2: (related) this is broken behavior23:43
morganfainbergbknudson, it is loaded by default23:44
morganfainberglook in service.py23:44
*** petertoft has quit IRC23:44
bknudsonmorganfainberg: it's just not in the paste pipeline?23:44
bknudsongit revert23:44
morganfainbergbknudson, might be a 1 line fix23:45
morganfainbergchecking now23:45
morganfainbergif that doesn't fix, yes, revert is coming23:45
*** dstanek_afk has quit IRC23:45
*** daneyon has quit IRC23:45
morganfainbergbut i'd rather not revert if it's a single line that was missed when we made it optional23:45
bknudsonmorganfainberg: it's @dependency.optional.. so would just remove from service.py and create the manager when the extension is loaded.23:46
morganfainbergyeah that is the th...23:47
bknudsondstanek: you won't be thanking me for rebasing that patch when you see what happened to https://review.openstack.org/#/c/78117/ ...23:51
bknudsonnot sure what happened there.23:51
bknudsonbut now it depends on outdated and is failing to merge...23:51
dstanekbknudson: ha, i'll take a look23:53
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Create a test token generator and use it  https://review.openstack.org/7887723:53
bknudsondstanek: I tried cherry-picking the changes to rebuild the chain but got --23:54
bknudson ! [remote rejected] HEAD -> refs/publish/master/bp/more-code-style-automation (no new changes)23:54
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Create a V3 Token Generator  https://review.openstack.org/7887823:57
dstanekbknudson: that's pretty strange23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!