Friday, 2014-02-21

*** bknudson has quit IRC00:00
*** dstanek has quit IRC00:00
*** bknudson has joined #openstack-keystone00:01
*** dstanek has joined #openstack-keystone00:12
*** ChanServ sets mode: +v dstanek00:12
*** arunkant has quit IRC00:15
*** browne has quit IRC00:23
*** leseb has quit IRC00:36
*** lbragstad1 has joined #openstack-keystone00:47
*** amerine_ has joined #openstack-keystone00:50
*** amerine has quit IRC00:55
*** lbragstad has quit IRC00:55
*** dtroyer has quit IRC00:55
*** Daviey has quit IRC00:55
*** amerine_ is now known as amerine00:56
*** Daviey has joined #openstack-keystone00:57
*** dtroyer has joined #openstack-keystone00:57
*** dtroyer has quit IRC01:02
*** dtroyer has joined #openstack-keystone01:02
*** ayoung-dinner is now known as ayoung01:42
*** leseb has joined #openstack-keystone01:47
*** leseb has quit IRC01:52
ayoungSphinx: "because PEP8 wasn't anal-retentive enough"01:54
*** marcoemorais has quit IRC02:05
*** marcoemorais has joined #openstack-keystone02:05
*** marcoemorais has quit IRC02:05
jamielennoxcan we stick to 6 monthly client releases as well - backwards compat is a PITA02:14
ayoungSure02:21
mfischayoung: I'm working on the bug to limit the LDAP search results and had a question if you're around02:23
ayoungmfisch, just make sure that it works with all LDAP servers02:23
ayoungheh02:23
ayoungmfisch, its just "limit X offset Y"  no?02:23
mfischayoung: that will be a challenge, but I can try AD and FreeIPA02:23
ayoungHeh02:23
mfischayoung: there's a parameter you can pass to search_ext, I have it working, but my issue is the unit test I wrote02:24
ayoungOpenLDAP and you have the trifecta02:24
mfischthe unit test doesn't function02:24
ayoungmfisch, Oy, I am well aware02:24
mfischand I'm at a loss on how one debugs it, pdb isn't working02:24
ayoungah...there I can help02:24
mfischthis is a new unit test I wrote02:24
ayoungpdb is not working because of eventlet02:24
ayoungrun with an env var that tells eventlet not to monkeypatch thread:02:25
ayoungSTANDARD_THREADS=True02:25
ayoungexport that before you run pdb and breakpoints should start working02:25
dstanekmorganfainberg: any reason why https://review.openstack.org/75284 is a bad idea?02:27
morganfainberghmm02:27
morganfainbergnot really sure.02:27
morganfainberglikely it isn't a bad idea02:27
morganfainbergdstanek, nothing sticks out unless pep8 suddenly requires like py33 and something else using venv doesn't02:28
morganfainberg;)02:28
dstanekwe can worry about that in 5-10 years :P02:28
morganfainbergLOL02:29
mfischis there a trick to get pdb working? pdb.set_trace() throws a BdbQuit exception02:29
dstanekmfisch: how are you running the tests?02:29
dstanekmfisch: i'm pretty sure that testr still doesn't support running pdb - try using nosetests (you'll have to pip install it into your virtualenv)02:33
dstaneki swear i added something to make this easier02:33
dstanekmfisch: ah yes, 'tox -e debug'02:34
dstanekmorganfainberg: you around?02:37
morganfainbergdstanek, no :P02:37
dstanekwhat happens when you go here https://review.openstack.org/#/c/72808/13/etc/keystone.conf.sample?02:37
morganfainbergyrah02:37
morganfainbergit's known02:37
morganfainbergmassive review02:37
dstanek:-)02:38
morganfainbergdstanek, infra says gerrit just can't do the in-line comparisons because it's too big a changeset02:38
morganfainbergdstanek, jeblair looked at it02:38
ayounghttps://www.openstack.org/vote-atlanta/Presentation/openstack-security-crunchy-on-the-outside-with-a-chewy-center02:40
ayoungVOTE FOR NATE AND ROB!02:40
ayoungand yes, they stole that quote from me.  Who stole it from The Far Side02:41
morganfainbergdolphm_503. https://bugs.launchpad.net/keystone/+bug/1275615 - incomplte02:43
morganfainbergincomplete*02:43
*** gokrokve has joined #openstack-keystone02:43
morganfainbergcan't repro bug 1275615 , it seems to work every time... but i can't get the alpha ubuntu lts to install in a VM to test that exact scenario02:44
* morganfainberg is sad uvirtbot is not here02:44
dstanekmorganfainberg: it would be nice if the reporter could setup a temp VM that's in a bad state02:47
mfischdstanek: thanks for the hints, my client died but my logger caught them, will give that a try02:47
morganfainbergdstanek, yeah02:47
dstanekmfisch: np02:47
morganfainbergdstanek, i'm inclined to say nogo because well alpha ubuntu?02:47
mfischmorganfainberg: 14.04 won't install for you?02:50
morganfainbergmfisch, nope no in vmware fusion02:50
morganfainbergmfisch, well it installs02:50
morganfainbergit wont ever boot02:51
*** topol has joined #openstack-keystone02:51
morganfainberghangs forever somewhere around the time the gnome logo boots up (or whatever window manager the iso is trying to install) and the VM is locked up, can't escape / go to a console /etc02:51
mfischah, desktop 14.0402:51
morganfainbergmfisch, and server 14.04 isn't exactly easy to get an iso for.02:52
morganfainbergmfisch, at least not that i found.  lots of source, no server iso02:52
morganfainbergmfisch, or not one that would download02:52
mfischlooks like it's moved02:53
morganfainbergmfisch, yeah, *shrug*02:53
morganfainbergmfisch, but pip installed versions of the required "things" and it works.  all things being equal, this should repro even on the older version (13.10)02:53
morganfainbergif it was actually a ipv6 / sqla / etc problem02:54
mfischmorganfainberg: so you can pull a daily although sounds like you've tried enough: http://cdimage.ubuntu.com/ubuntu-server/daily/current/02:54
morganfainbergmfisch ah.02:54
morganfainbergmfisch, yeah i really don't think this is a keystone manage / sqla issue02:54
morganfainbergsqla-migrate02:54
mfischthe missing a2 is odd, will ask a colleague in the morning because now I'm curious02:55
morganfainbergyeah02:56
morganfainberg*shrug* i'm not super worried02:56
morganfainbergthis is an edge-case of an edge-case i think02:57
morganfainbergmfisch, somehow i think if this was a serious problem we'd have more than just a report for Keystone02:58
morganfainbergmfisch, ;)02:58
morganfainbergwe're not doing "magic"02:58
mfischI beg to differ, I had to wave a wand to get ldap functional02:59
mfischalthough those were ldap issues I exorcised and not keystone03:00
ayoungLDAP is magicx03:01
ayoungblackes of majix03:02
morganfainbergayoung, uhm...03:02
ayoungnecoramncy is clean compared to LDAP03:02
morganfainbergayoung, i wont argue03:02
ayoungnecromancy03:02
mfischabsolutely03:02
mfischspeaking of that, ayoung any chance this lands for I?  https://bugs.launchpad.net/keystone/+bug/123148803:02
ayounghttp://adam.younglogic.com/2008/08/openldap-api-is-somewhat-hostile/  was one of my earliest blog posts...and I stand by it03:03
morganfainbergdstanek, https://review.openstack.org/#/c/72026/18/keystone/common/cache/backends/mongo.py am i crazy or line 395 comment, how is meth_kwargs being set on that object.  it seems like it's only ever set on line 261 which is a totally different object class that doesn't propagate03:03
ayoungmfisch, first I need to vent03:03
ayoungAAAAAAAAAAAA03:03
morganfainbergdstanek, i feel.. like i might be going insane03:03
ayoungNothing quite like having a patch not merge because someone submitted a refactoring  cleanup that just/......AAAAAAA!03:03
mfischayoung: I spent an hour last night trying ldap.OPT_SIZELIMIT only to discover it's epically useless03:03
morganfainbergdstanek, i hope he collapses that down into something a bit more straightforward, though, i don't think it's that maintainable03:03
morganfainbergdstanek as is03:03
morganfainbergdstanek, hard to understand the code because of the extra object abstraction03:04
ayoungmfisch, talk with nkinder and richm as they are LDAP Experts.03:04
ayoungThey've been on 389 since it was called something else03:04
morganfainbergayoung, so i just heard you say you're an LDAP expert >.>03:04
morganfainbergayoung, :P03:04
ayoungmorganfainberg, if by expert you mean "someone that knows more than you...."03:04
dstanekmorganfainberg: look on line 29503:04
mfischayoung's blog posts are what comes up when you google for freeIPA/Keystone03:04
morganfainbergdstanek, oh god03:04
* ayoung is a kosher ham03:04
morganfainbergdstanek, i see it now. but GAH03:05
nkinderwhat's up with sizelimits?03:05
dstanekmorganfainberg: he is trying so hard not to listen to me :-)03:05
mfischnkinder: I'm finishing up some work on them, just discovering some annoyances in the process03:05
ayoung-2 and move on dstanek03:05
nkinderif you're talking about LDAP client side sizelimits, it won't override any server limits03:05
mfischnkinder: yes, thats right03:05
morganfainbergdstanek, yeah i think he's going to collapse it down, it just is .. a bit all over now.  it looks like it does a decent implementation just will be hard to maintain03:06
ayoungnkinder, did you vote for your own talk yet?03:06
morganfainbergayoung, -2 thats a feature03:06
ayounghttps://www.openstack.org/vote-atlanta/Presentation/openstack-security-crunchy-on-the-outside-with-a-chewy-center03:06
dstanekmorganfainberg: yeah, generally speaking it look quite goot03:06
dstanekor good03:06
ayoung-2 "Not on my watch"03:06
dstanekayoung: i'm saving the nuke for later03:06
nkinderayoung: I think so...03:06
nkindervoted for a bunch of stuff this morning03:07
ayoungMongo only pawn in game of Keystone03:07
nkinderayoung: writing up a preso on KDS right now actually03:07
morganfainbergnkinder, i would tell you if i voted ot not, but the voting system is the same (personal opinion) trainwreck it was last summit03:07
dstanekmorganfainberg: the test suite worries me a little bit because it's reimplementation of Mongo03:07
nkindermorganfainberg: 3 stars for everything!03:07
mfischthe votes are just opinion right? not binding03:07
morganfainbergdstanek, i think that is because we don't have the mockmongo package in global reqs03:07
dstanekwhen i started reading throught it in my ming i head, "NOT IT!"03:07
morganfainbergnkinder, pretty much.03:07
mfischI thought03:07
*** lbragstad1 has quit IRC03:08
dstanekwow i can't type03:08
morganfainbergnkinder, basically they need an index by company, by topic, and overall index that shows which ones i voted on... cause search is a poor substitute to an at-a-glance overview03:08
mfischsearch by presenter didnt seem to work03:08
dstanekmorganfainberg: when i started reading through it in my mind i heard, "NOT IT"03:08
ayoungCopyright 2014 Hewlett-Packard Development Company, L.P.?  WHat is an LP?  I thought it was kind of record.  Is the Howling Patchderm really an LLP?03:09
nkindermorganfainberg: but it brings up random talks after every vote!  Surely that makes up for a poor search interface.03:09
ayoungnkinder, intentional03:09
morganfainbergnkinder, but apparently that feedback wasn't loud enough (or often enough) from HK03:09
ayoungthey are trying to annoy everyone equally03:09
mfischayoung: that entity is what holds all of HP's patents/IP03:09
morganfainbergayoung, basically it is likely to make me not vote except for talks someone explicitly hands links to me for03:09
ayoungmorganfainberg, that is what I am doing anyway03:09
ayoungdon't want to dilute my votes...maybe I'll go through and One star a few though03:10
nkindermfisch: so are you stuck on anything with LDAP at the moment, or you were just beating your head against the wall yesterday (and have since knocked the wall down)?03:10
morganfainbergayoung, if there was an overview i'd spend time on it, but eh, i'kll be in dev sessions03:10
mfischnkinder: I'm good for now, just need to get my unit test working03:10
dstanekmorganfainberg: really small comment on https://review.openstack.org/#/c/72808/03:10
morganfainbergayoung, maybe i'll 1 start vote everything03:10
ayoungYeah.  and maybe a few that I have a personal stake in like ^^ and the Moc03:10
mfischnkinder: you can get your red pen ready for my review03:10
nkinderheh03:11
morganfainbergdstanek, shouldn't be comma separated it's ENV variable03:11
morganfainbergspace separated03:11
morganfainbergdid i comma separate it?03:11
ayounghttps://review.openstack.org/#/c/69593/  was a mistake03:12
dstanekmorganfainberg: no, there is only one entry in there with instructions to add more03:12
morganfainbergdstanek, there are two entries03:12
ayoungOK, done venting...03:12
morganfainbergin the LIB line03:12
morganfainbergsure i'll specify what to change03:12
*** devlaps has quit IRC03:12
dstanekLIB line?03:13
morganfainbergdstanek, https://github.com/openstack/keystone/blob/master/tools/config/oslo.config.generator.rc#L103:13
morganfainbergdstanek, there are two entries in the ENV variable03:13
morganfainbergkeystone and oslo.messaging03:13
morganfainbergdstanek, it's loaded from .sh files :P03:14
dstanekmorganfainberg: ah, i read that as keystone.03:14
morganfainbergah nope :)03:14
dstaneki may just be too tired for reviews tonight...03:14
dstanekmorganfainberg: maybe a comment in the .rc file would be enough03:15
morganfainbergdstanek, hm.. sure03:15
dstanek'# a space separated list of package that use oslo.config' or something03:15
*** marcoemorais has joined #openstack-keystone03:17
morganfainbergdstanek, http://paste.openstack.org/show/67845/03:17
morganfainbergdstanek, ?03:18
dstanekmorganfainberg: love it! thanks03:19
morganfainbergdstanek, https://review.openstack.org/#/c/75284/ rebase clicky03:20
*** marcoemorais has quit IRC03:21
*** sudorandom has joined #openstack-keystone03:23
mfischdstanek: when I tried tox -e debug, I get an error about a missing debug_helper.sh, any idea what that is?03:24
mfischhmm I think I might know what it is, let me try03:24
*** dolphm_503 is now known as dolphm03:25
morganfainbergdstanek, https://review.openstack.org/#/c/73895/03:25
morganfainbergdstanek, if that goes in, a lot of misc options disappear from the sample config03:25
morganfainbergdstanek, which would be good™03:26
dstanekmorganfainberg: nice, i'll take a look03:26
dstanekmfisch: hmmm...it exists in your tools directory?03:27
*** david-lyle has joined #openstack-keystone03:31
dstanekmfisch: i just tried and it seemed to work ok03:34
*** dolphm is now known as dolphm_50303:34
*** gyee has quit IRC03:40
*** richm has quit IRC03:43
*** harlowja is now known as harlowja_away03:45
mfischdstanek: its working, was my issue03:47
*** dstanek has quit IRC03:50
*** KanagarajM_ has joined #openstack-keystone03:54
*** dstanek has joined #openstack-keystone03:54
*** ChanServ sets mode: +v dstanek03:54
*** dolphm_503 is now known as dolphm04:03
*** dolphm is now known as dolphm_50304:13
*** dstanek has quit IRC04:25
*** dolphm_503 is now known as dolphm04:25
mfischmy test is failing because its hitting the cache and not ldap04:33
*** dolphm is now known as dolphm_50304:35
ayoungmfisch, direct cache frustrations at morganfainberg04:53
mfischwell it was deeper than that, my test needs to live in the _live_ldap test04:53
mfischlooks like that can be run against a real server04:53
*** dstanek has joined #openstack-keystone04:54
*** ChanServ sets mode: +v dstanek04:54
jamielennoxanybody here that vaguely understands what i am trying to do with auth plugins?05:04
jamielennoxayoung, bknudson, dstanek, dtroyer, morganfainberg: ^05:05
*** dstanek has quit IRC05:07
jamielennoxif the problem is the 'vaguely understanding' i can walk you through enough to be a sounding board - it's a fairly conceptual problem05:09
ayoungI understand05:16
ayoungthe question I thought that was still unresolved was "how do we trigger them" jamielennox05:16
jamielennoxayoung: trigger?05:16
jamielennoxayoung: the problem i have at the moment is that i want the session to be responsible for the service catalog05:17
jamielennoxso no more does each client parse it and figure out its base_url and then start making requests05:17
jamielennoxyou just say i want path='/users', service_type="identity", version=(3,0)05:17
ayoungyay!05:18
jamielennoxi also want to keep the auth plugins REALLY basic05:18
ayoungjamielennox, by trigger I meant "try x509" or "kerberos" instead of Password05:18
jamielennoxso if you were to write a plugin that was not for keystone it would work05:19
jamielennoxi want no real required knowledge of a plugin05:19
jamielennoxthe problem i have is when the path contains information that is relative to something in the token05:20
jamielennoxe.g. /users/%(user_id)/roles05:20
ayoungAHHHHHH EVERYTHING HAS CHANGED WITH SQL!05:20
jamielennox(i'm pretty sure that's not a real path)05:21
ayoungthe ones I saw had tenant in there05:21
jamielennoxeverything with SQL? i'm sure i would have heard about that05:21
morganfainbergayoung, ugh, i think i am almost done solving the issues with moving oslo.messaging options to uhm... [oslo_messaging] config group05:21
jamielennoxayoung: it's not a problem coming from the service_catalog because it will do the replace05:21
ayoungjamielennox, trying toi rebase the sql provide for revoke and a load has changed and it is makeing me cranky05:21
morganfainbergayoung, this is an ugly changeset :(05:21
ayoungmorganfainberg, Keystone is ugly to me tongiht05:22
morganfainbergayoung,  21 files changed, 198 insertions(+), 137 deletions(-)05:22
ayoungsounds OK05:22
morganfainbergayoung, i think i have 1 more issue w/ tests to fix.05:22
morganfainbergayoung, but it's been unfun unravling stevedore errors05:22
ayoungjoy05:22
morganfainbergyeah05:22
jamielennoxayoung: keystone is the ugly sister - not pretty but you love her anyway05:23
jamielennox(i'm not sure if that's just wrong)05:23
jamielennoxanyway05:23
ayoungjamielennox, wouldn't know about that.  My sister has her issues, but she ain't ugly05:23
jamielennoxayoung: ...05:24
jamielennoxmorganfainberg: i'm sure ayoung just dropped into conversation that he has a hot sister05:24
ayoungmid forties, two kids...05:24
morganfainbergjamielennox, you know... i think tonight is just a night to let that one sit dude.05:25
jamielennoxanyway what i came up with is to keep the client needing to know anything about it's auth information is that we could just submit a path with replacements in it05:25
morganfainbergjamielennox, it has been an odd day :P05:25
jamielennoxmorganfainberg: completely agree05:26
ayoungjamielennox, my kneejerk reaction is we are trying too hard there05:26
jamielennoxso you can make a request for  /users/%(user_id)/roles and the session will handle it for you05:26
*** dolphm_503 is now known as dolphm05:26
jamielennoxayoung: me too05:26
jamielennoxayoung: at some point here something crosses the line from this is a useful feature, to this is trying to do too much05:27
jamielennox(note that it's not a feature - we need access to this somewhere)05:27
ayoungdolphm_503....   that man Thinks in HTTP05:27
jamielennoxbecause then the problem becomes that i need auth plugins to provide a dictionary so that i can do path % auth_plugin.get_format_kwargs()05:28
ayoungI'll be scare when he changes his nick to dolphm_41005:28
ayoungtoo much, I think...why did you end up there?05:29
jamielennoxthe keys to that dict are essentially part of the stable api, so at which point they might as well be properties on the auth_plugin05:30
jamielennoxso some set of user_id, project_id, and some others are required of every plugin05:30
jamielennoxthat's not unreasonable (i think), but it was part of what i was trying to avoid until i'd fleshed it out a little more05:31
jamielennoxand i don't know, the whole thing just seems to have detoured somewhere into something complex05:32
ayoungso...discovery gets us the top level of the url...hostname:port/v3  for the current keystone.  From there the auth plugin needs to know how to get the token for the url that you are handing it?05:33
*** dstanek has joined #openstack-keystone05:34
*** ChanServ sets mode: +v dstanek05:34
jamielennoxayoung: i ended up there because a large part of this is supposed to be that a client shouldn't care about it's own auth information - other than that it has some05:34
jamielennoxayoung: yes the plugin has done authentication05:34
ayoungright.  and in order to seamlessly authenticate you need to be able to craft the approprate auth request05:34
jamielennoxthe plugin will also handle getting the endpoint because service_catalog is a part of the token05:34
jamielennoxayoung: no authenticate is fine05:35
jamielennoxit's after that in use05:35
ayoungright, so parse the service catalog out of the response and then.  select the approprate endpoint based on the service05:35
ayoungwhy do you need the URL part past there?05:35
jamielennoxthere are some commands (like change my own password) that require knowledge of the user_id of the current authentication context05:35
ayoung /users/%(user_id)/roles  and such?05:35
ayoungah...05:35
ayoungthat is wrong, I think05:36
jamielennoxthere aren't a lot05:36
jamielennox++05:36
*** dolphm is now known as dolphm_50305:36
ayoungthat is beyond the scope of the auth plugin05:36
jamielennoxcompletely agree, anything that requires information from the context should be taken from the token and NOT from the URL05:36
ayoungwhatever calls the auth plugin needs to make that call05:36
ayoungclient should know the User id05:36
jamielennoxi'm not sure of other examples of this yet05:37
jamielennoxayoung: why?05:37
ayoungum...I guess not.  Now that I think about it, they probably only know username, and userid comes in the token/response from auth05:37
ayoungwhich wouldn't be returned in a call into the auth plugin?05:38
jamielennoxayoung: so seperating the client from the authentication05:38
ayounghmmm05:38
jamielennoxwhy does the client need to know the username?05:38
jamielennoxthat's purely authentication information05:38
ayounguserid, this case, but any data necessary to craft the urls05:38
ayoungI meant that username was passed in, say from the environment...disregard for now05:39
ayoungjamielennox, will the auth plugin make the token info queryable to the client?05:39
jamielennoxayoung: right, so what i'm coming to is that certain information like user_id has to leak out from the auth plugins to make our situation work05:39
jamielennoxayoung: yes and no05:39
jamielennoxi don't want to provide defaults for this stuff05:40
jamielennoxthere is a subclass of BaseAuthPlugin called IdentityAuthPlugin05:40
ayoungwhy not cache the body of the token response and make it availalble after authenitaction05:40
jamielennoxIdentityAuthPlugin essentially means that you have authenticated against keystone and then you can query from there05:41
jamielennoxi think it's relatively fair to say if isinstance(plugin, IdentityAuthPlugin): do stuff05:41
jamielennoxi want to allow the case where the token provider is not keystone and may not have all that same information05:42
jamielennoxthough i think as we are discussing there must be certain fields that each plugin must provide - eg user_id05:42
ayoungjamielennox, I think that is wrong05:44
ayoungtoken response is part of our domain model05:44
ayoungdon't try to abstract it away05:44
jamielennoxayoung: no, i think the information that we provide is part of our domain model05:45
jamielennoxayoung: eg user_id, project_id, domain_id etc05:45
ayoungright...all of that is in the token response05:45
jamielennoxeverything that auth_token middleware presents to a service05:45
jamielennoxso long as auth_token middleware and the auth plugin are in sync then i don't think there is any requirement on our token format05:46
ayoungthe body of the response to POST auth/tokens...the data signed inside the token as well ...05:46
jamielennoxfor example we changed from a v2 token to a v3 token05:46
ayoungexcept that we have data from it we need to expose to the end user....05:46
ayoungugh05:46
ayoungdouble ugh05:46
ayoungDIE V2 DIE05:46
jamielennoxlol, ok but there are already rumours of a v4 token05:47
*** dstanek has quit IRC05:47
jamielennoxand we will manage to switch again05:47
ayoungso we would need a client side analogue to the TokenProvider conversion code05:47
ayoungJumping right to v605:47
jamielennoxpretty much yea05:47
jamielennoxwe have a requirement of certain information that will be transmitted in a token05:48
ayoung<whingey_luke>You ask the Impossible.</whingey_luke>05:48
jamielennoxwe have published formats how that happens05:48
jamielennoxthere is nothing i can see that prevents someone coming up with another05:48
jamielennox(gyee got really excited at the idea)05:48
ayoungtoken version....will match the version requested of the session, though, no?05:48
ayounghe would05:48
jamielennoxayoung: not necessarily05:49
jamielennoxwe already allow using v2 tokens on v3 services and vice versa05:49
*** dstanek has joined #openstack-keystone05:49
*** ChanServ sets mode: +v dstanek05:49
ayoungyeah, but if you ask for a v2 session, it is going to get you a v2 token.  You would only get a v3 token on a v3 session.  You could pass in a token from the outside, and then you would need to convert, or use the v2 token to get a v3 token05:52
jamielennoxyou wouldn't need to convert, we support passing an auth_ref to a new client which would work05:53
jamielennoxbut it will be a lot easier with a plugin model05:53
jamielennoxsession = Session(v2.Password(user, pass, tenant))05:53
jamielennoxclient = v3.client.Client(session)05:54
*** dstanek has quit IRC05:54
ayoungif the auth_ref was created with a v2 token, and you passed it to a new client requesting a v3 session, you would need to convert if the rule were:  token should match the version on the session05:55
jamielennoxayoung: we very specifically do not have that rule05:56
ayoungtoken data format, that is...05:56
ayoungyeah, and thus we don't have access to some of the most basic data we need, as you pointed out05:56
jamielennoxfrom what i can see the only thing in client that is needed is user_id05:56
jamielennoxwhich IMO we should have had a /user route that handled operations for the current user05:57
ayoungyeah...05:58
jamielennoxit would almost be a pure redirect from /user -> /users/%(my_id)s05:58
ayoungbut we also don't know the project info if they had default set05:58
jamielennoxhmmm, that would actually be fairly easy05:58
ayoungyeah, that would be05:58
jamielennoxayoung: from what i can see in client there is nothing that requires the project_id you are scoped to to be used as part of the path05:59
jamielennoxi wonder if other services do though05:59
ayoungpretty sure that nova does06:01
jamielennoxthis was how i got to allowing POST /projects/%(project_id)s/images or whatever replacements06:02
ayoungyep06:02
ayoungnova could doa redirect, too06:02
jamielennoxayoung: it's more correct IMO even for rest06:03
jamielennoxanyway - i guess for now i mandate user_id, leave everything else for as required06:03
ayoungjamielennox, how a bout the plugin gives you the token and you pass the token to a helper utility to get user_id...and project id?06:05
*** gokrokve has quit IRC06:05
jamielennoxthis was essentially where i started plugin.get_format_kwargs()06:06
ayoungor it always gives if back to you in v3 format, regardless of the form you got it in.  A V3 helper a-la the provider code?06:06
ayoungok...I need to crash.06:06
jamielennoxayoung: yea, it's nearly beers time here - it must be late06:07
*** gokrokve has joined #openstack-keystone06:12
*** marcoemorais has joined #openstack-keystone06:19
*** topol has quit IRC06:22
*** dolphm_503 is now known as dolphm06:27
*** gokrokve has quit IRC06:30
*** gokrokve has joined #openstack-keystone06:31
*** gokrokve has quit IRC06:35
*** morganfainberg is now known as morganfainberg_Z06:36
*** saju_m has joined #openstack-keystone06:36
*** dolphm is now known as dolphm_50306:36
*** dolphm_503 is now known as dolphm07:27
*** gokrokve has joined #openstack-keystone07:29
*** gokrokve_ has joined #openstack-keystone07:30
*** gokrokve has quit IRC07:33
*** gokrokve_ has quit IRC07:35
*** dolphm is now known as dolphm_50307:37
*** jamielennox is now known as jamielennox|away07:44
*** dstanek has joined #openstack-keystone07:51
*** ChanServ sets mode: +v dstanek07:51
*** dstanek has quit IRC07:55
*** leseb has joined #openstack-keystone08:04
*** marekd|away is now known as marekd08:04
*** dolphm_503 is now known as dolphm08:28
*** saju_m has quit IRC08:31
*** gokrokve has joined #openstack-keystone08:31
*** KanagarajM_ has quit IRC08:32
*** gokrokve has quit IRC08:36
*** dolphm is now known as dolphm_50308:38
*** saju_m has joined #openstack-keystone09:00
*** saju_m has quit IRC09:02
*** bvandenh has quit IRC09:19
*** saju_m has joined #openstack-keystone09:21
*** dolphm_503 is now known as dolphm09:23
*** chandan_kumar has joined #openstack-keystone09:25
*** bvandenh has joined #openstack-keystone09:27
*** gokrokve has joined #openstack-keystone09:29
*** gokrokve has quit IRC09:34
*** marcoemorais has quit IRC09:40
*** marcoemorais has joined #openstack-keystone09:42
*** marcoemorais has quit IRC09:47
*** warpig has left #openstack-keystone09:51
*** chandan_kumar has quit IRC09:51
*** Kanagaraj has joined #openstack-keystone10:00
*** chandan_kumar has joined #openstack-keystone10:05
*** marcoemorais has joined #openstack-keystone10:11
*** KanagarajM_ has joined #openstack-keystone10:13
*** Kanagaraj has quit IRC10:16
*** marcoemorais has quit IRC10:16
*** chandan_kumar has quit IRC10:28
*** gokrokve has joined #openstack-keystone10:29
*** gokrokve_ has joined #openstack-keystone10:31
*** gokrokve has quit IRC10:34
*** gokrokve_ has quit IRC10:36
*** leseb has quit IRC11:00
*** dolphm is now known as dolphm_50311:03
*** dolphm_503 is now known as dolphm11:03
*** leseb has joined #openstack-keystone11:04
*** marcoemorais has joined #openstack-keystone11:12
*** KanagarajM_ has quit IRC11:15
*** KanagarajM_ has joined #openstack-keystone11:15
*** marcoemorais has quit IRC11:17
*** leseb has quit IRC11:23
*** dolphm is now known as dolphm_50311:27
*** gokrokve has joined #openstack-keystone11:29
*** gokrokve has quit IRC11:33
*** KanagarajM_ has quit IRC11:45
*** KanagarajM_ has joined #openstack-keystone11:47
*** dstanek has joined #openstack-keystone11:54
*** ChanServ sets mode: +v dstanek11:54
*** KanagarajM__ has joined #openstack-keystone11:58
*** dstanek has quit IRC11:59
*** leseb has joined #openstack-keystone12:00
*** KanagarajM_ has quit IRC12:00
*** KanagarajM__ has quit IRC12:07
*** marcoemorais has joined #openstack-keystone12:13
*** marcoemorais has quit IRC12:17
*** gokrokve has joined #openstack-keystone12:29
*** gokrokve has quit IRC12:34
*** dolphm_503 is now known as dolphm12:49
*** marcoemorais has joined #openstack-keystone13:14
*** david-lyle has quit IRC13:15
*** marcoemorais has quit IRC13:18
*** leseb has quit IRC13:25
*** gokrokve has joined #openstack-keystone13:29
*** dstanek has joined #openstack-keystone13:36
*** ChanServ sets mode: +v dstanek13:36
*** leseb has joined #openstack-keystone13:41
*** browne has joined #openstack-keystone13:53
*** saju_m has quit IRC14:10
*** leseb has quit IRC14:13
*** nkinder has quit IRC14:13
*** marcoemorais has joined #openstack-keystone14:14
*** marcoemorais has quit IRC14:19
*** leseb has joined #openstack-keystone14:22
*** gokrokve has quit IRC14:37
*** gokrokve has joined #openstack-keystone14:39
*** jagee has joined #openstack-keystone14:46
*** lbragstad has joined #openstack-keystone14:57
*** richm has joined #openstack-keystone15:06
*** nkinder has joined #openstack-keystone15:06
richmlooking at Bug #1282676 Error 500 when trying to set empty description with LDAP15:07
richmI'm having trouble understanding how the mapping of ldap attributes to object properties is supposed to work15:07
richmfor example, in class ProjectApi15:07
richmThere is the attribute_options_names dict15:07
richmafaict, this is is supposed to map the key (the object property name) to the value (the ldap attribute name)15:08
richmwell, not exactly - map the key (the property name) to the value (config file key) that maps to the attribute name.15:10
richmbut some of these are just wrong15:11
richmin the sample config file, anyway15:12
richm# tenant_desc_attribute = desc15:12
richmshould be = description15:12
richmand there is no "enabled" attribute15:13
bknudsonhttp://git.openstack.org/cgit/openstack/keystone/tree/etc/keystone.conf.sample#n107615:13
bknudsonit's #tenant_desc_attribute=description now15:13
richmah, ok15:13
richmI guess I need to do a git pull15:13
dolphmrichm: every hour! ;)15:13
richmnever mind then - I'll shut up now15:14
bknudsonthere's also ignored attributes15:14
dolphmbknudson: have you seen https://bugs.launchpad.net/keystone/+bug/1233365 ?15:14
bknudsontenant_attribute_ignore= -- which allows attributes in the tenant to not be mapped or put in ldap.15:14
bknudsondolphm: have I seen it happen? I think someone mentioned it to me once related to an IBM product.15:15
*** marcoemorais has joined #openstack-keystone15:15
bknudsonbut they weren't using keystone ldap... had their own ldap client impl.15:15
dolphmbknudson: i just meant the bug report, but that works too15:15
dolphmbknudson: based on python-ldap?15:15
bknudsondolphm: yes, and the workaround was essentially to disable referrals15:16
dolphmbknudson: it sounds like there's a patch floating around the ether for that bug, i'm trying to get ahold of it15:16
bknudsondolphm: the proposed fix here of skipping references seems like a good idea.15:16
dolphmbknudson: you mean disable referrals on the AD side?15:16
bknudsonit's assigned to ayoung so was kind of ignoring it.15:16
dolphmbknudson: or on the client side?15:16
dolphmbknudson: ayoung just sits on bugs15:16
bknudsondolphm: on the client side.15:16
dolphmbknudson: that's why i have a "Unassigning due to inactivity." script15:17
*** marcoemorais has quit IRC15:20
ayoungassign it to richm15:21
ayoungdolphm, I hatebugs15:21
richmdo you use a python IDE?  If so, which one?  I've been using pydev with eclipse 4.3.0 and it leaves something to be desired15:21
ayoungrichm, pycharm is better by far15:22
ayoungI just hadn't gotten used to it before learning eclipse, so I always end up back with eclipse15:22
ayoungbut there is a ... special offer?  for a pycharm license for openstack developers.15:22
ayoungIt is built on Intellij Idea15:23
bknudsonpydev works for me... annoying when it adds space after = in arguments.15:23
ayoungthe one thing I find frustrating about pycharm is that it doesn't keep multiple projects open at the same time, so if I am switching between python-keystoneclient and keystone I have to dump one for the other15:24
ayoungbut pycharms' refactoring and code navigation support outstrips pydev15:24
marekdrichm: some of my friends use sublime, you can give it a try.15:24
*** lbragstad has quit IRC15:25
ayoungpycharm has a 30 day trial and then you can search the openstack mailing list for the guy that will hand out the developers licenses15:26
richmpydev is missing a lot of features that I use when using eclipse for C and Java development15:26
richme.g. can't click on a symbol and have it give me the references to that symbol - can't do a call graph - etc.15:27
bknudsonthat's because python doesn't know15:28
bknudsontoo dynamic15:28
marekdayoung: seriously?! wow.15:28
bknudsonthis is why you never write complicated applications in python15:28
*** lbragstad has joined #openstack-keystone15:29
*** david-lyle has joined #openstack-keystone15:29
* richm thinks keystone is complicated, or at least complex15:29
*** bknudson has quit IRC15:29
ayoungbknudson, are you actually a Javaphile?15:30
ayoungbut even python can do better than pydev does. Pycharm is better, its just that i have a huge body of tests in the memory of pydev...and I periodically wipe out my .settings file15:30
ayoungwith a git clean -xdf15:30
ayoungmarekd, yeah...they want to get people using it, and have hitched themselves to the Openstack wagon15:31
marekdayoung: hm, i think i might want to try it, then :-)15:32
ayoungit takes some figuring out, especially with the autogenerated venvs15:32
ayoungmarekd, twas morganfainberg that got me to try it.  He uses it pretty much exclusively, I think15:32
marekdayoung: i see.15:33
*** bknudson has joined #openstack-keystone15:33
ayoungmarekd, when ever someone says "I see" on IRC I read that as:  "Wow, that is the stupidest thing I have heard in a long time."  I know you don't mean it that way.15:34
richmayoung: no, that would be "Wow - cool story bro"15:34
marekdrichm: ++15:35
ayoungrichm, that, too.  "I see implies" a technical dumbassery and an attempt to keep from cursing in a public forum15:35
*** lbragstad has quit IRC15:36
marekdayoung: i see :-) No, i didn't mean it. Is it just your impression or everybodys (so i better stop  using it, at least on IRC)?15:36
ayoungJust mine15:36
* richm likes the term "technical dumbassery"15:38
*** lnxnut has joined #openstack-keystone15:43
*** amcrn has joined #openstack-keystone15:53
*** stevemar has joined #openstack-keystone16:01
*** ChanServ sets mode: +v stevemar16:01
stevemarmarekd, nice refactoring of tests :)16:01
marekdstevemar: thanks :-)16:03
stevemarbknudson, ayoung dolphm morganfainberg_Z https://review.openstack.org/#/c/71353/ is ready for review16:09
ayoungstevemar, we all have PBKRS now16:11
ayoungpost-Brant-Knudson-Review-Syndrome16:11
ayoungits where we don't bother to review something that is close to committing before he has gone through and nickled and dimed it to death16:11
marekdLOL16:12
stevemarayoung, i find that you get used it after a while, and kinda enjoy it16:12
ayoungits flipping AWESOME!16:12
bknudsonI'll make time to review https://review.openstack.org/#/c/71353/ .16:12
ayoungedewata was the same way when we were working together on FreeIPA. He was like a human compiler16:12
bknudsonstevemar: have you run tox -e cover on it?16:12
stevemarbknudson, not yet, more tests are the only thing on my todo list16:13
ayoungstevemar, do you have some sample SAML assertions to use?16:15
ayoungI realize that we are not actually going to parse them16:15
ayoungfeel free to use: http://adam.younglogic.com/resources/adam_example.saml16:16
stevemarayoung, we inject assertions into the context16:16
ayoungattributes....the assertion is the signed file16:16
stevemarayoung, sorry, we inject attributes (as they would look like after being parsed by apache modules) into the context16:17
marekdayoung: this is work for apache.16:17
stevemarmarekd, i noticed you got rid of the prefix16:18
marekdstevemar: not me, somebody removed it from the keystone.conf.sample16:19
marekdstevemar: it was in the master, and everybody here wanted to remove that prefix.16:19
ayoungstevemar, I guess what I am asking is are you using the attributes from "real" SAML files?16:20
bknudsonstevemar: you're asking me to review untested code?16:20
dolphmmarekd: keystone.conf.sample is now dynamically generated16:20
bknudsonwhy should I waste my time on that?16:20
*** browne1 has joined #openstack-keystone16:21
stevemarbknudson, there are tests for it, just didn't run coverage report yet16:22
stevemarbknudson, unless i misunderstood something..16:22
bknudsonstevemar: you said "more tests are the only thing on my todo list"16:23
stevemarmore than half the code is in test16:23
stevemaryes "more"16:23
*** browne has quit IRC16:23
marekdayoung: this is how my environment looked like when I was authenticating to a saml protected simple website. So you can assume, ADFS_* parameters are real there. https://gist.github.com/zaccone/914822d37ac2eea420ce16:23
stevemarnot "start tests"16:23
ayoungawesome16:24
bknudsonI tried to get us a "check experimental" that would run the coverage tests... https://review.openstack.org/#/c/72151/16:24
bknudson-1 by the infra team.16:24
*** lbragstad has joined #openstack-keystone16:26
*** browne has joined #openstack-keystone16:35
*** browne1 has quit IRC16:37
ayoungbknudson, I want an external test repo16:38
ayoungand three flavors of tests:16:38
ayoung1.  unit tests.  run all the time16:38
ayoung2. failing tests;  new test that are known not to pass16:38
ayoung3.  functional tests:  ldap against a live server, or SAML, or Apache HTTPD type stuff16:39
ayounggate would run against 1 and 316:39
ayoungwe also keep a scorecard on the failing tests to record when they start passing16:39
bknudsonayoung: what do you mean by repo? a new git project?16:40
ayoungyeah16:40
ayoungI want to be able to commit tests even when keystone is in code freeze16:40
ayoungalso, it should test both server and client16:40
dolphmayoung: you can contribute tests anytime16:40
dolphmayoung: we don't have a "code" freeze - we have a "feature" freeze16:41
ayoungdolphm  hmmm....OK,  fair enough.16:41
ayoungthere is still a bit of an issue with adding features to the client that need a live server to test16:42
ayoungand the ability to run tests known to fail16:42
ayoungthe "known to fail" thing I guess could also be handled inthe current repo16:42
ayoungits really the client I guess that needs a live keystone server...16:43
bknudsonwrite a test that shows how it fails16:43
ayoungbknudson, a test that "passes"?16:43
ayoungthen when you fix it, it fails?16:43
ayoungbassackwards16:43
bknudsonright, then you know when you fix it.16:43
dolphmayoung: when you fix it, you illustrate by revising the test16:43
ayoungdolphm, that loses one of the primary motivators:  we need to be able to run the set of known-failing tests16:44
bknudsonand also we know what's going to happen if the change is reverted16:44
*** gokrokve has quit IRC16:44
ayoungmaybe instead of "skip" we have a "skip if skipping failures"16:44
*** gokrokve has joined #openstack-keystone16:45
ayoungohh I like ^^16:45
ayoungthat can be done in the current context.  OK16:45
ayoungSO all we needthat we don;t have beyond that is the ability to run a keystone server for client tests.16:45
ayoungno need for a separate repo.  I'm going to write a couple BPs for those things16:45
*** marekd is now known as marekd|away16:46
bknudsonit would be nice to have a fake keystone for keystoneclient that could be unfaked.16:46
ayoungah..actually, we also want the live-tests thing.  But that can be done in the current repo as well...16:46
bknudsonthen all the tests run against real keystone instead16:46
ayoungbknudson, just spin up a keystone the way that we do for thecurrent keystoneclient tests, just from that repo16:46
ayoungmeans you need to have the keystone git repo checked out, or the code available on the python_path16:46
dstanekayoung: we used to use a @needs_work decorator to annotate tests known to be failing16:48
ayoungdstanek, ++16:48
dstanekayoung: maybe i'll replicate the idea here and see what everyone things16:48
ayoungdstanek, so long as we have a switch that can pass through that and actually run the test to see if it now passes16:48
ayoungdstanek, cool16:49
ayoungdstanek, ideally we would run Only the tests decorated that way when checking known failures16:49
*** gokrokve has quit IRC16:49
ayoungand generate a report that shows "these tests were expected to fail but now pass"16:49
*** gokrokve has joined #openstack-keystone16:51
ayoungdstanek, https://blueprints.launchpad.net/keystone/+spec/failing-tests16:53
*** henrynash has quit IRC16:53
ayoungwant me to assign to you?16:54
dstanekayoung: sure16:54
*** gokrokve has quit IRC16:56
*** marcoemorais has joined #openstack-keystone16:58
stevemarbknudson, cover is mostly 90-100% for auth/plugins/saml2 and contrib/federation/*17:00
stevemarbknudson, except for core.. probably because of the abstract driver class17:01
bknudsonstevemar: check if there's any new code added in the review that's important and not covered.17:02
bknudsonthe % isn't what's important.17:02
*** nkinder has quit IRC17:04
*** gokrokve has joined #openstack-keystone17:04
dstanekstevemar: is that branch or line coverage?17:05
*** leseb has quit IRC17:09
*** leseb has joined #openstack-keystone17:09
*** browne has quit IRC17:12
*** henrynash has joined #openstack-keystone17:14
*** leseb has quit IRC17:14
*** henrynash has quit IRC17:15
*** gokrokve has quit IRC17:15
*** gokrokve has joined #openstack-keystone17:15
*** nkinder has joined #openstack-keystone17:20
*** gokrokve has quit IRC17:20
stevemardstanek, not sure? whatever the default is?17:20
dstanekstevemar: line probably17:21
*** achampion has joined #openstack-keystone17:21
achampionIs it possible to use SAML for authentication only, but keep all the authorisation information in keystone, e.g. projects, roles, etc.17:22
bknudsonhere's the latest coverage report: http://logs.openstack.org/a6/a6c3d6d07d217b86e9d2f26e4c216a3011e37c55/post/keystone-coverage/2d5c04f/cover/17:23
ayoungneed to take that report and run a regression on it from commit to commit....17:26
bknudsonayoung: that's what the infra guys suggested.17:26
ayoungis coverage run on gate, or just on check?17:27
bknudsonayoung: it's run in post17:27
bknudsonso we don't get a report until after the change is merged17:27
ayoungah...perfect17:27
ayoungyeah, but we can always grab the latest post to check against...17:27
bknudsonyes, you can get the report from the last merge.17:28
*** gyee has joined #openstack-keystone17:38
dstanekdolphm: this needed a rebase https://review.openstack.org/#/c/75284/17:39
*** gokrokve has joined #openstack-keystone17:40
ayoungbknudson, for example,  I just ran cover on the sql backend for revoke.  I did a wget of the link you posted above, and can do a side-by-side comparison17:46
bknudsonayoung: what comparison do you think we should make? just compare % ?17:47
ayoungnah.....17:47
ayoungthat is too corse17:47
ayoungcoarse17:47
ayoungsome where I didn't cover had the same percentage17:48
ayounglet me run it with a few more lines of contet and I'll post17:48
bknudsoncould compare "missing" count17:48
ayounghttp://paste.openstack.org/show/68085/17:49
ayoungit puts total at the top, which is nice17:50
*** henrynash has joined #openstack-keystone17:53
ayoungbknudson, http://paste.openstack.org/show/68086/  little easier to read17:53
ayoungtotal missing went from 1805  to 183317:54
ayoungbut total coverage stayed at 85%17:54
*** harlowja_away is now known as harlowja17:56
*** YorikSar has quit IRC18:00
*** leseb has joined #openstack-keystone18:10
*** browne has joined #openstack-keystone18:10
*** topol has joined #openstack-keystone18:11
*** henrynash has quit IRC18:14
ayoungbknudson, stevemar do you guys have the ability to test Keystone changes against DB2?18:18
*** henrynash has joined #openstack-keystone18:19
stevemarayoung, i don't. I think bknudson was doing something db2 related at some point in the past, i think enabling support for the community edition (free)?18:20
ayoungstevemar, thanks. topol do you have the ability to test something against DB2?18:21
ayoungguessing henrynash hasn't been dealing with it18:21
topolayoung, bknudson should be able to do this18:21
henrynashayoung: not me, personally, no18:21
ayoungOK.   I copied some of the token cleanup logic that is DB2 specific for the Revocation events.  Would love to have a means to actually run it18:22
ayounghttps://review.openstack.org/#/c/67372/13/keystone/contrib/revoke/backends/sql.py  line 67 bknudson18:23
*** dolphm is now known as dolphm_50318:23
topolayoung, K Let's see what bknudson says. I can try some other folkss but I think he can18:23
ayoungtopol, thanks18:23
*** morganfainberg_Z is now known as morganfainberg18:24
bknudsonayoung: we're working on providing an integrated CI infrastructure for DB218:29
morganfainbergayoung, you can add projects to a pycharm window in preferecnes structure18:29
ayoungbknudson, yeah, I had heard that18:29
morganfainbergayoung, preferences->project structure.18:29
bknudsonwe've got a BVT infrastructure internally that we run already18:29
bknudsonit runs tempest18:29
ayoungmorganfainberg, ohh. lemme test18:29
bknudsonwhich is actually failing right now18:29
ayoungbknudson, need to come up with something like _live_tests for the revoke code that can be run against DB2.18:30
*** devlaps has joined #openstack-keystone18:30
ayoungwouldn't be tempest,  as it is inside Keystone18:30
ayoungbut the idea is the same18:30
bknudsonI've run the live tests against db218:30
bknudsonand we could eventually hook up our CI to keystone...18:30
bknudsonthe DB2 CI work is progressing but seems to be going slowly... need to get it for sqlalchemy project first.18:31
ayoungmorganfainberg, "preferences" is well hidden18:31
*** henrynash has quit IRC18:31
*** henrynash has joined #openstack-keystone18:32
ayoungmorganfainberg, so I would add /opt/stack/python-keystoneclient as an additional content root?18:33
morganfainbergayoung, yep18:34
ayoungmorganfainberg, that looks wrong18:34
morganfainbergayoung, i acutally have an "OpenStack" directory and i checkout all the projects I want to include, and then open OpenStack and add the project roots as sources18:34
ayoungI think that is going to apply the project setting for both, to include the venv setup18:34
morganfainbergayoung, hm. oh oh yeah =/18:34
ayoungthen again, we should be able to have an integrated venv for both18:35
morganfainbergayoung, let me see if i can tell you how to use different venv (might not be doable)18:35
ayounginteresting....18:35
ayoungmorganfainberg, its low priority18:35
ayoungI think the keystone venv includes everything needed for p-kc18:35
ayoungif not...I can pip install it.18:35
morganfainbergayoung, i'm opening pycharm now :P but i'm just 2x checking (takes 10 seconds)18:35
ayoungah...but the kc is installed in the keystone venv.18:36
morganfainbergayoung, yeah only 1 venv, hep there is the limitation18:36
morganfainbergayoung, :( doh18:36
morganfainbergayoung, oh well.18:37
ayoungmorganfainberg, something to think about.  I suspec that we will want to put keystone into the venv for p-kc in order to do the live testing I was ranting about before18:37
*** henrynash has quit IRC18:37
morganfainbergayoung, i saw18:37
morganfainbergayoung, i'm not opposed to that.18:37
ayoungnah, just needs to happen18:38
*** YorikSar has joined #openstack-keystone18:38
morganfainbergayoung, yeah, likely worthwhile to test against a real server vs mocking it all up18:38
morganfainbergayoung, s/likely//18:38
ayoung++18:38
ayoungmorganfainberg, there was talk about trying to run all of devstack inside a single venv.  I wonder if we could start hacking there18:39
morganfainbergayoung, i'd love that18:40
morganfainbergayoung, only issue is libvirt18:40
morganfainbergayoung, you need system-packages because libvirt can't be pip installed (python bindings) last i heard18:40
ayoungI'm sure it isn't the "only"  but it might be the "first biggest"18:40
morganfainbergayoung, everything else 100% could run in the venv18:41
morganfainbergayoung, it's an unfortunate hurdle18:41
ayoung'salright.  We just need one venv for all the python code.  Wouldn't change due the libvirt side of things.18:41
ayoungWe have other deps on native code, too18:41
morganfainbergayoung, i want a precedent to run a venv in production (tbh) and isolate from system python libs18:41
ayoungnah,  LDAP required ldap-devel18:41
*** henrynash has joined #openstack-keystone18:41
morganfainbergayoung, if devstack does it i can make the case on a broader scope18:41
ayoungmorganfainberg, its called PLan 9 from Bell labw18:41
ayounglabs18:41
morganfainbergayoung, hehe18:42
ayoungreally you want a container18:42
ayoungnot just venv18:42
morganfainbergayoung, native code != python bindings built as a side effect of the system lib18:42
morganfainbergayoung, python-ldap while requiring ldap-dev, doesn't require python code outside of the venv18:42
morganfainbergayoung, afaik libvirt python is built (swig?) as a side effect of building libvirt itself18:43
ayoungmorganfainberg, why can't the libvirt bindings be built inside the venv based on the native code18:43
morganfainbergayoung, which is why nova allows system packages in the venv18:43
ayoungjust a build system problem?18:43
morganfainbergayoung, not sure, i think so / swig or something very very build specific18:43
ayoungwe have people on Open Stack that are pretty integral with libvirt.  If it is a real problem I can ambush them next time Im in the office.18:43
morganfainbergayoung, i thought i heard rumblings it would be fixed in the future ™18:43
simoI hope we are not growing an unhealthy "native code" religion in openstack, like ti happend in the java world ?18:44
morganfainbergsimo, nah,18:45
simopheeew18:45
morganfainbergsimo, i want to be able to isloate from system python libs, don't care if it's native18:45
simofor testing ?18:46
morganfainbergsimo, nope, prod.18:46
simoas an option or by default ?18:46
morganfainbergsimo, as an option18:46
simoI see, have fun :)18:46
morganfainbergsimo, the choice is always deployer, i never want to take that away from the deployers18:46
morganfainbergsimo, i just know i run up against conflicts often because <distro> relies on <X> and openstack likes version <z> which is now incompatible18:47
morganfainbergsimo, e.g., don't want to break yum to install openstack ;) (not that it's likely, but similar concerns)18:47
morganfainbergit also means it is easier to test / deploy controlled versions of the python libs, roll a VENV (packaged) and use a control file to determien which venv to activate18:47
* morganfainberg dreams of options that makes his ops teams lives better.18:48
*** henrynash has quit IRC18:48
richmthis is sounding a lot like docker18:49
morganfainbergrichm, ++18:49
morganfainbergabsolutely18:49
morganfainbergbut docker has other impllications in some cases18:49
morganfainbergit's a bit heavier handed than i want to be for dev/qa/etc18:49
morganfainbergrichm, but the way i see it is you can have 4 or 5 solid ways to deploy18:50
morganfainbergrichm, and you pick the "right" one for your use case.18:50
richmsure - what's right for dev is not necessarily what's right for qe or prod18:50
morganfainbergrichm, docker being the most containerized, and the other end is system installed libs18:50
morganfainbergrichm, absolutely.18:50
morganfainbergrichm, and this is why i like keystone folks so much. :) I can talk about this stuff and only every now and again ayoung calls me crazy.18:51
morganfainbergok ok, he probably thinks i'm crazy on a regular basis18:51
*** dolphm_503 is now known as dolphm18:51
morganfainbergdamn it.  now i have a song stuck in my head.18:52
morganfainbergayoung, https://review.openstack.org/#/c/75316/ got part of the way there.  still chasing bugs down.18:53
ayoungdawka  as we call it here in Mass18:53
morganfainbergayoung, but i think that (if we can get it in) will help with the config stuff.18:53
ayoungdawka dawka dawka dawka dawka dawka dawka dawka dawka18:53
ayoungsimo, its like Software collections all over again18:54
*** ayoung is now known as ayoung-lunch18:55
*** henrynash has joined #openstack-keystone19:01
*** marekd|away has quit IRC19:15
*** henrynash has quit IRC19:21
*** marekd|away has joined #openstack-keystone19:23
*** amerine has quit IRC19:27
*** amerine has joined #openstack-keystone19:30
*** henrynash has joined #openstack-keystone19:31
*** henrynash has quit IRC19:36
*** henrynash has joined #openstack-keystone19:55
*** lnxnut has quit IRC19:56
*** lnxnut has joined #openstack-keystone19:57
*** harlowja is now known as harlowja_away19:59
*** lnxnut has quit IRC20:01
*** harlowja_away is now known as harlowja20:08
*** lnxnut has joined #openstack-keystone20:21
*** henrynash has quit IRC20:22
*** YorikSar has quit IRC20:27
*** henrynash has joined #openstack-keystone20:36
achampioncan anyone direct me to any good references of getting keystone working with federated identity with a SAML IdP20:40
dolphmachampion: that's still a work in progress! https://blueprints.launchpad.net/keystone/+spec/saml-id20:41
dolphmachampion: https://review.openstack.org/#/c/71353/20:41
achampionAnd a follow up question, is it possible to use federadeted identity just for authentication but keep authorizattion in keystone (projects, roles, etc)20:41
dolphmachampion: and yes, that's the goal for icehouse20:41
achampiondolphm: thanks I'll take a look at that20:41
achampiondolphm: good to hear, that would meet our requirements20:42
achampiondophm: is there anything I can do to help test this20:43
achampion+l20:43
dolphmmarekd|away: stevemar: ^20:44
dolphmachampion: those two are the ones doing all the heavy lifting ^20:44
dolphmachampion: the biggest help at this point would be reviewing the code, and testing it if you're able20:44
stevemarachampion, yes, authentication would be through the idp, and authnz is kept in keystone20:45
*** YorikSar has joined #openstack-keystone20:48
achampionstevemar: the blueprint mentioned above seems to expect SAML assertions for the authorization with mappings - or I am missing understanding it20:48
stevemarachampion, mappings will translate the saml attributes to keystone entities, but which user/group gets authorization on a specific project/role is still going to be done in keystone20:51
bknudsonkeystone's common.db.sqlalchemy code has gotten way out of date.20:52
morganfainbergbknudson, yeah :(20:52
morganfainbergbknudson, i just tried to do a sync20:53
morganfainbergwow20:53
bknudsonmorganfainberg: I'll work on trying to get keystone working with it again.20:53
morganfainbergbknudson, ok.20:53
morganfainbergbknudson, this is kindof why i don't like the way oslo-incubator works.20:54
morganfainbergbknudson, i'd rather have known targets and releases to work with.20:54
bknudsonmorganfainberg: making backwards incompatible changes doesn't work.20:54
morganfainbergrather than "oh sorry"20:54
bknudsonI think that they're moving the part out to its own library20:54
morganfainbergbknudson, ++ i hope so!20:54
morganfainbergok back to stable fixes I go.20:55
*** marcoemorais has quit IRC20:56
*** marcoemorais has joined #openstack-keystone20:58
achampionstevemar: looking through the BP and etherpad for federation-flows, I'm not sure that it covers what I was asking. Effectively looking for a "local user" (group, project, roles, etc) defined in keystone, but authentication done via SAML with assertions to map to the local user.20:59
morganfainbergachampion, the user/group would come from SAML21:00
morganfainbergachampion, the project/roles etc woiuld be internal to keystone (standard assignment CRUD/work)21:00
morganfainbergachampion, at least that was my understanding of the target  and general direction things are moving21:00
morganfainbergs/was/is21:00
achampionmorganfai: that's what I thought... we have a corporate identity service but the idea of getting them to provide the project/role assertions is probably a no go.21:01
morganfainbergachampion, you shouldn't need to21:01
morganfainbergachampion, just the user informaiton and the SAML assertion should map to the user information needed to provide assignment (project etc)21:02
morganfainbergachampion, so you'd create an assignment (Role Y on project Z) for a user that would match the SAML assertion data.21:02
morganfainbergachampion, though stevemar and marekd|away can provide more specifics21:02
morganfainbergachampion, and stevemar is likely going to show up and say i'm totally wrong21:03
morganfainbergstevemar, *poke*21:03
morganfainberg:P21:03
achampionmorganfai: ok, that would work21:04
bknudsonthe group isn't in saml. you define the group in keystone21:04
morganfainbergbknudson, ah21:04
achampionmorganfai: it was just both use-case 1 and use-case 3 state: 'Keystone creates a user with an expiration of the saml-assertion "NotOnOrAfter" time and provisions the requested domain, roles, etc to that user as passed in the saml assertion.'21:04
bknudsonthe group ids come from the mapping21:04
morganfainbergbknudson, ok, that was it21:04
morganfainbergachampion, ^21:04
morganfainbergbknudson, thanks :)21:05
*** leseb has quit IRC21:05
stevemarachampion, the use-cases are rather out of date21:06
achampionstevemar: ahh, ok.21:06
stevemarachampion, but bknudson and morganfainberg are feeding you correct info21:06
stevemarachampion, we don't expect any role/project stuff in the assertion21:06
morganfainbergstevemar, yay! i understand the mapping stuff (mostly)21:06
morganfainberg>.>21:07
achampionstevemar: great, it sounds like what is being discussed would work... I'll dig some more21:07
achampionso the use isn't ephermal in that case - right?21:07
morganfainbergbknudson, i'm going to resurrect the patchset to move to the config fixture now that it's all in oslo.21:07
achampionuse=user21:07
morganfainbergbknudson, i think i'll aim for that to land after I3 though.21:08
bknudsonmorganfainberg: great. I think dstanek was interested in it too.21:08
morganfainbergbknudson, so we don't make life any worse for BPs and features21:08
stevemarachampion, the user would be, but the group that he's a part of is not.21:08
dstanekbknudson: ?21:08
morganfainbergdstanek, config fixture21:08
dstanekmorganfainberg: yeah, i was excited to see that review21:09
morganfainbergdstanek, it's sync'd into openstack.common.fixture for us now.21:09
morganfainbergdstanek, i'm going to bring back the conf fixture patch once I3 sails21:09
morganfainbergdstanek, so we don't make features more painful21:09
stevemarachampion, if we see an incoming assertion, we grab the mapping rules and try to find a group that the user would be a part of. then issue a token based on the roles that the group has21:09
*** marcoemorais has quit IRC21:10
*** marcoemorais has joined #openstack-keystone21:10
stevemarachampion, the token will have a user field with id and name, but it's just there for auditing/information, the user won't exist in the keystone backend21:10
achampionstevemar: so the groups and roles a user has would have to be part of the assertions - which I think maybe challenged with. I was hoping we can define a local user with their roles, groups, etc. and just use the assertions to identify the local user.21:11
stevemarachampion, nope, i think i just made things worse21:11
achampionstevemar: maybe :)21:12
stevemarachampion, a have to head out now, can you PM me your email, and I promise a long explanation?21:12
*** amcrn has quit IRC21:13
dstaneki'm not sure i case about the order of the args to assertEqual21:14
dstaneks/case/care/21:14
*** stevemar has quit IRC21:21
morganfainbergdstanek, the argument is that there is expected and observed21:38
*** dolphm is now known as dolphm_50321:38
morganfainbergdstanek, and some of the derived methods from assertEqual say "saw this but expected that" and it's unclear from the error21:39
morganfainbergdstanek, functionaly, it doesn't matter21:39
morganfainbergdstanek, but it is more correct with the patches proposed.21:39
dstanekmorganfainberg: the end result is meaningless to me21:39
morganfainbergdstanek, it's shuffling things towards more correct, and i'm ok with that.21:39
dstanekmorganfainberg: what derived methods?21:39
morganfainbergdstanek, we have a couple that fall back on assertEqual i think21:39
bknudsonmorganfainberg: after sync, I get a weird error when tox -e sample_config -- Error importing module keystone.contrib.kds.cli.manage: no such group: database21:40
bknudsonthat part doesn't even use database.21:40
morganfainbergbut in either case, for a new developer, when it says "expected" and "got" inverted it is more confusing21:40
bknudsonmaybe once I get further in the port it will make sense.21:40
morganfainbergbknudson, maybe. it might be that .cli imports something that uses .database?21:40
bknudsonI just wanted to make sure pep8 works21:40
morganfainbergbknudson, ah, yeah just ignore that error then ;)21:41
dstanekmorganfainberg: i'm not sayin i'm against it, but what your saying applies to the derived methods and not assertEqual21:41
morganfainbergdstanek, sure, but if upstream makes the errors clearer (unittest) it's worth being "correct"21:41
*** topol has quit IRC21:42
morganfainbergdstanek, basically, i think this is a noop, i don't care, but if someone wants to correct it, sure21:42
morganfainberggo for it21:42
dstanekmorganfainberg: thier signature is something like assertEqual(first, second) - so i don't think they will change the output21:42
morganfainbergdstanek,     def assertEqual(self, expected, observed, message=''):21:43
morganfainbergdstanek.         """Assert that 'expected' is equal to 'observed'.21:43
morganfainbergat least that is py2721:43
morganfainbergdstanek, https://review.openstack.org/#/c/75521/ OSSA for this21:44
morganfainbergbknudson, ^21:44
dstanekmorganfainberg: i don't see that in case.py21:44
morganfainbergit's public21:44
morganfainbergdstanek, i see it in testcase.py21:44
morganfainbergdstanek, sorry it's testcase not unittest21:45
morganfainbergtesttools.testcase21:45
dstanekmorganfainberg: ah21:45
dstanekyeah so far i have been mostly disappointed by testtool, testr, etc.21:45
morganfainbergtestr is great for seeing the differences in test runs21:46
morganfainberg+XXX / -XXX21:46
morganfainbergbut i can see why it's less friendly21:46
dstanekmorganfainberg: i find i mostly don't care - i'd rather it tell when what i did wrong so i don't have to use nose several times a day21:46
morganfainbergdstanek, *shrug* i think i find both about as useful.21:47
dstanekmorganfainberg: also it would be trivial to write a nose plugin that tells you differences in test runs21:47
morganfainbergdstanek, meaning to say, less useful than i want, but provides me half-way-decent info for debugging21:47
morganfainbergdstanek, now the real challenge... can i backport that fix to grizzly...21:48
* morganfainberg is glad grizzly is going to EOL sooner vs later21:49
morganfainbergit's getting hard to port needed fixes around :P21:50
dstanekmorganfainberg: whoa, looks like there are multiple patches for assertEqual.21:50
morganfainbergdstanek, yeah the guy did one patch per system?21:50
morganfainbergdstanek, rather than a massive one, token, trust, etc21:51
morganfainbergi think that was it21:51
morganfainbergor was it dolph that did it.  honesty didn't pay attention to the committer, just the content21:51
*** henrynash has quit IRC21:56
richmCan someone tell me why assignments/backends/ldap.py RoleApi.update rejects the update if a role by the given name already exists?21:57
dstanekmorganfainberg: what does ttx mean by his comment about your commit being hard to backport?22:01
morganfainberghm?22:01
morganfainbergoh because dogpile.kvs is not backportable22:02
morganfainbergtotally changed structures of the code22:02
ayoung-lunchheh22:02
morganfainbergand until the other fix for CAS update happened, the token backend wasn't fun22:02
ayoung-lunchplease backport a major rewrite from Icehouse to Havana22:02
morganfainbergit's better now22:02
morganfainbergayoung-lunch, sure sure, can we get that for grizzly too?22:02
*** ayoung-lunch is now known as ayoung22:02
ayoungAll the way back to Austin!22:03
morganfainbergayoung, now you're just being unreasonable22:03
ayoungCactus?22:03
morganfainbergayoung, there ya go22:06
morganfainbergayoung, that was keystone v1 right?22:06
ayoungActually, Diablo was where it was incubated and Essex release IIRC22:06
morganfainbergayoung, ok22:06
morganfainbergstill22:06
morganfainberg:P22:06
*** jagee has quit IRC22:08
*** henrynash has joined #openstack-keystone22:10
richmanyone?  This is causing the sole remaining test failure in _ldap_livetest.py22:12
richmwould love to get this passing at 100%22:13
morganfainbergdstanek, https://review.openstack.org/#/c/75526/22:16
morganfainberggrizzly version22:16
dstanekmorganfainberg: strictly speaking i think the code is fine...what other considerations are there for evaluating backports22:19
morganfainbergdolphm_503, ^ bug 126008022:19
morganfainbergdstanek, hm. well if it fixes the issue :P22:19
morganfainbergdstanek, most of the time i wouldn't even be interested in backporting but it was needed22:20
morganfainbergawww uvirbot isn't here :(22:20
* morganfainberg sniffles22:20
dstanekmorganfainberg: the change to pep8 for the sample config caught a sync issue - https://review.openstack.org/#/c/73895/22:20
morganfainbergdstanek, yep.22:21
dstaneki like it22:21
morganfainbergdstanek, yes! So do I :)22:21
*** gokrokve has quit IRC22:22
morganfainbergdstanek, i really do like it that it's forced via pep8.22:22
*** gokrokve has joined #openstack-keystone22:22
morganfainbergdstanek, i also like that it's pretty easy to regenerate the sample22:22
dstanekmorganfainberg: what do you think is easier to read:22:24
dstanek_, a, b = fun()22:24
dstanekor22:24
dstaneka, b = func()[1:]22:24
morganfainbergthe former22:24
*** dolphm_503 is now known as dolphm22:24
dstanekhmmm...really?22:25
morganfainbergthe slices feel ... cooler, but not as quick to proceess at a glance22:25
morganfainbergwhen scanning code, slices always take me longer to go "aha!" than scatter assignment22:25
morganfainbergbut i did most of my early development in perl and C.22:25
morganfainbergthe perl stuff still lingers around22:25
morganfainbergs/scatter assignment/pure scatter assignment22:26
dstaneki have to rename _ to something else - the original author used i22:27
morganfainberghehe yeah22:27
*** gokrokve has quit IRC22:27
dstanekalso pylint doesn't like the unused variable22:27
morganfainbergbut the _, a, b one is ore perl like, the slice is more pythonic22:27
morganfainbergdstanek22:27
dstanekmorganfainberg: all of your warnings down our my pep8 errors :-)22:31
morganfainbergdstanek, LOL just add help strings!22:31
*** achampion has quit IRC22:32
dstanekmorganfainberg: yeah, "TODO: help goes here"22:32
morganfainbergLOL22:32
morganfainbergi'm actually getting some of those filled in now.22:33
dstanekmorganfainberg: is there a changeset on master to reference for those two backports?22:36
morganfainbergdstanek, hmm22:36
morganfainbergdstanek, https://review.openstack.org/#/c/60743/22:36
dstanekmorganfainberg: cool thx22:37
*** leseb has joined #openstack-keystone22:37
dstaneki couldn't find it because the commit message is different22:37
ayoungmorganfainberg, I just rebased all of my changes for Revocation22:39
ayoungAnnnnd a poep 8 error on sql22:40
ayoungWTF22:40
ayoung'tox -esample_config -r22:41
ayoungah....22:41
*** leseb has quit IRC22:41
dstanekayoung: the -r is a trick22:42
ayoungdstanek, knowing that the patch merged that required it is the trick22:42
dstaneki ran the command without thinking and it recreated by venv22:43
dstaneknot a big deal except i used command history to run it again :-(22:43
ayoungthe amount of churn this config file stuff is trigging is a trifle annoying22:44
*** devlaps1 has joined #openstack-keystone22:44
*** devlaps has quit IRC22:45
*** gokrokve has joined #openstack-keystone22:46
*** henrynash has quit IRC22:48
*** leseb has joined #openstack-keystone22:55
dolphmthis should be a fun bit of code for anyone that wants to learn about / play with auth_token https://review.openstack.org/#/c/75529/22:56
dolphmbasically lets you experiment with auth_token without having to stand up a real service to poke at, like nova22:57
*** leseb has quit IRC23:00
lbragstaddolphm: nice23:02
ayoungif our sample config file is going to be autogenerated, it probably should not be checked in to git23:04
ayoungdolphm, that is the coolest thing I have seen today23:05
dolphmayoung: i hear your blog article already23:05
ayoungdolphm, I'll let you write that onw23:06
ayoungone23:06
dolphmayoung: http://dolphmathews.com/23:06
dolphm(not gonna happen)23:06
ayoungI'm working on one explaining why we need to use CMS for Oslo Messaging23:06
*** gokrokve has quit IRC23:08
*** gokrokve has joined #openstack-keystone23:08
*** lbragstad is now known as lbragstad_away23:12
*** gokrokve has quit IRC23:13
bknudsonguess what -- oslo-incubator db is broken.23:22
bknudson:(23:22
*** richm has quit IRC23:25
*** gokrokve has joined #openstack-keystone23:28
dstanekoh, noes - what's wrong with it?23:28
morganfainbergdstanek, https://review.openstack.org/7553723:33
morganfainbergdolphm, ^ dstanek, more help strings23:33
morganfainbergthat should reduce the warnings a lot23:35
dstanekmorganfainberg: ah, yes; that'll help23:38
*** nkinder has quit IRC23:38
morganfainbergdstanek, that one takes it down to ~12 warnings.23:39
*** richm has joined #openstack-keystone23:39
morganfainbergdstanek, i'm not comfortable writing the help strings for some of those23:39
morganfainbergdstanek, or i just don't know what to say "uh.. user id... and things"23:39
*** gokrokve has quit IRC23:40
bknudsondstanek: db_version_control(abs_path, init_version)23:40
bknudsondef db_version_control(engine, abs_path, version=None):23:40
bknudsonapparently it's not tested.23:40
*** gokrokve has joined #openstack-keystone23:40
morganfainbergayoung, you're trying to secure the "most patches in a given review" aren't you? :P23:40
*** gokrokve_ has joined #openstack-keystone23:41
*** leseb has joined #openstack-keystone23:41
*** gokrokve has quit IRC23:44
*** gokrokve_ has quit IRC23:46
*** gokrokve has joined #openstack-keystone23:47
*** dstanek is now known as dstanek_dinner23:48
*** gokrokve has quit IRC23:51

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!