Tuesday, 2014-09-16

*** richm has joined #openstack-keystone00:00
openstackgerritBrant Knudson proposed a change to openstack/keystone: Move unit tests from test_backend_ldap  https://review.openstack.org/11992800:14
*** bknudson has joined #openstack-keystone00:16
openstackgerritBrant Knudson proposed a change to openstack/keystone: Tests raise exception if logging problem  https://review.openstack.org/11994600:19
*** alex_xu has joined #openstack-keystone00:22
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Make keystoneclient use an adapter  https://review.openstack.org/9768100:23
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Allow retrying some failed requests  https://review.openstack.org/11800400:23
jamielennoxbknudson: can you have a look at ^ so we can push it through before release00:25
bknudsonjamielennox: ok00:25
*** ncoghlan has joined #openstack-keystone00:29
*** rodrigods_ has joined #openstack-keystone00:33
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Versioned Endpoint hack for Sessions  https://review.openstack.org/9063200:42
stevemarbknudson, are you cool with "find . -name '*.pyc' -exec rm -f \{} +" to remove pyc files?00:42
bknudsonstevemar: I don't understand why we're removing pyc files.00:43
bknudsonThe command is find and I run it all the time already.00:43
stevemarbknudson, i think I ran into issues once or twice00:43
*** mitz_ has joined #openstack-keystone00:46
jamielennoxbknudson: any particular reason for mock.patch.object over mock.patch?00:52
stevemarjamielennox, for AccessInfoV2, we should make the is_federated property return None right?00:53
stevemarnot NotImplemented00:53
jamielennoxstevemar: for v2 return False, for base return NotImplemented00:53
stevemaroops, right false00:53
jamielennoxor raise NotImplemented00:53
jamielennoxstevemar: not a great fan of if is_federated: return None, can you catch the KeyError and then reraise if not federated?00:55
stevemarjamielennox, i can return true/false00:56
jamielennoxstevemar: i mean for user_domain_name property00:56
jamielennox    return xxxxxxxx00:56
jamielennoxexcept KeyError:00:56
jamielennox    if federated:00:56
jamielennox        return None00:56
jamielennox    raise00:56
jamielennoxdoesn't exclude the case where we might put something in there later, but catches it otherwise00:57
bknudsonjamielennox: object references work better than strings with the editor I use. I can click on it to go to the code, and if the import changes it'll show a problem.00:57
stevemarjamielennox, yeah, we could do that00:57
*** cjellick has quit IRC00:58
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Allow retrying some failed requests  https://review.openstack.org/11800400:59
jamielennoxbknudson: also i completely agree this is horrible, but it's used in a few places so better to just have it here01:00
*** zzzeek has joined #openstack-keystone01:01
*** sigmavirus24_awa is now known as sigmavirus2401:02
*** ncoghlan is now known as ncoghlan_afk01:05
*** zzzeek has quit IRC01:10
*** jasonsb has quit IRC01:11
*** amcrn has quit IRC01:16
stevemarjamielennox, if we use fixtures for keystoneclient for federation tokens, i'm going to have to add quite a few new args to Token's constructor01:16
stevemaryou okay with that? or just want to leave it as a token dump?01:16
stevemaralternatively i could get a regular token and fudge around with it a bit...01:17
*** marcoemorais has quit IRC01:26
*** russo3999 has joined #openstack-keystone01:26
*** gyee has quit IRC01:27
*** miqui has quit IRC01:29
jamielennoxstevemar: the tokens that are created are just a dict01:32
jamielennoxor an object that inherits from dict, create the basics with the fixture and then just add what else you need01:33
jamielennoxalso what else is required?01:33
*** achampio1 has joined #openstack-keystone01:39
*** achampion has quit IRC01:42
jamielennoxbknudson: alright, a new: https://review.openstack.org/#/c/118004 i think if i get a +2 there i can leave the rest01:50
jamielennoxoh and this horrible one: https://review.openstack.org/#/c/90632/01:51
*** rodrigods_ has quit IRC01:56
stevemarjamielennox, alright, i thought you might have wanted helper methods like set_federation() stuff02:01
stevemarbut yeah, I can just get a 'basic' token, and modify it02:01
*** achampion has joined #openstack-keystone02:12
*** achampio1 has quit IRC02:14
*** dims has quit IRC02:29
*** ncoghlan_afk is now known as ncoghlan02:29
*** dims has joined #openstack-keystone02:29
*** dims has quit IRC02:30
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Handle federated tokens.  https://review.openstack.org/12114602:30
*** dims has joined #openstack-keystone02:30
*** dims has quit IRC02:30
stevemarjamielennox, ^02:30
morganfainbergbknudson, i'm not sure how to test the cache poool02:31
*** dims has joined #openstack-keystone02:31
morganfainbergbknudson, been trying to figure out how to do it in a sane way considering it requires eventlet and multiple threads.02:31
morganfainbergbknudson, not something our test suite is particularly good at02:31
*** diegows has quit IRC02:32
jamielennoxstevemar: replied - otherwise i'm good02:35
*** dims has quit IRC02:35
stevemarjamielennox, d'oh!02:36
stevemargood eye02:36
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945202:37
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Handle federated tokens.  https://review.openstack.org/12114602:37
morganfainbergbknudson, ^ this covers your comments but does not cover testing.02:37
stevemarthanks jamie02:38
stevemarmorganfainberg, bknudson ^ if you care to review02:38
*** alex_xu has quit IRC02:40
morganfainberg hm02:41
morganfainbergok fun02:41
morganfainbergthat doesn't work...02:41
stevemarmorganfainberg, who ya talkin to?02:42
jamielennoxthanks all! see you in a few weeks02:44
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945202:46
morganfainbergstevemar, ok so... suggestions02:46
morganfainbergstevemar, i need to figure out how to test that ^02:46
morganfainbergmaybe i can *shudder* spin up a threading.thread in a test case02:47
morganfainbergoh god i feel dirty just considering that02:47
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945202:48
*** junhongl has quit IRC02:49
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945202:51
morganfainbergnow with updated sample config...02:51
morganfainbergi swear i'll get this right one of these times.02:51
*** jamielennox is now known as jamielennox|away02:52
*** ayoung has quit IRC02:53
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945203:00
*** richm has quit IRC03:00
*** jasonsb has joined #openstack-keystone03:00
*** sigmavirus24 is now known as sigmavirus24_awa03:22
*** saipandi has quit IRC03:47
*** rushiagr_away is now known as rushiagr03:56
*** ncoghlan is now known as ncoghlan_afk04:00
*** ncoghlan_afk is now known as ncoghlan04:00
*** ncoghlan is now known as ncoghlan_afk04:01
*** oomichi has joined #openstack-keystone04:16
*** ncoghlan_afk is now known as ncoghlan04:25
*** Sanchit has joined #openstack-keystone04:34
*** Sanchit has quit IRC04:37
*** rushiagr is now known as rushiagr_away04:43
openstackgerritPeter Razumovsky proposed a change to openstack/keystone: Add a simple module to work with filters and DNs to LDAP backend  https://review.openstack.org/11748404:48
openstackgerritNathan Kinder proposed a change to openstack/keystone: Set LDAP certificate trust options for LDAPS and TLS  https://review.openstack.org/12095404:54
*** harlowja_ is now known as harlowja_away04:59
*** stevemar has quit IRC05:03
*** zhiyan_ is now known as zhiyan05:10
*** rushiagr_away is now known as rushiagr05:16
*** RockKuo_Office has joined #openstack-keystone05:46
*** ajayaa has joined #openstack-keystone05:47
*** achampion has quit IRC05:52
*** achampion has joined #openstack-keystone05:55
*** alex_xu has joined #openstack-keystone05:56
openstackgerritOpenStack Proposal Bot proposed a change to openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/12069506:04
*** renlt has joined #openstack-keystone06:07
*** ncoghlan is now known as ncoghlan_afk06:10
*** renlt has quit IRC06:10
*** renlt has joined #openstack-keystone06:11
*** renlt has quit IRC06:12
*** gpanda has joined #openstack-keystone06:12
*** ukalifon1 has joined #openstack-keystone06:14
*** ncoghlan_afk is now known as ncoghlan06:18
*** lufix has joined #openstack-keystone06:37
*** KanagarajM has joined #openstack-keystone06:39
*** henrynash has joined #openstack-keystone06:44
*** henrynash has quit IRC06:45
*** Clabbe has joined #openstack-keystone06:48
ClabbeAnyone know of any issues related to the token handling ? the keystone-manage token_flush have been working for 1.5 days now06:48
ClabbeIs it possible to drop the table ?06:53
*** KanagarajM2 has joined #openstack-keystone07:06
*** KanagarajM has quit IRC07:07
*** BAKfr has joined #openstack-keystone07:16
*** garnav has joined #openstack-keystone07:36
*** wanghong has quit IRC08:04
*** aix has joined #openstack-keystone08:19
*** wanghong has joined #openstack-keystone08:21
*** ncoghlan has quit IRC08:26
*** henrynash has joined #openstack-keystone08:27
openstackgerrithenry-nash proposed a change to openstack/keystone: Ensure identity sql driver supports domain-specific configuration.  https://review.openstack.org/12124608:30
openstackgerritMarek Denis proposed a change to openstack/keystone: Document Keystone2Keystone federation  https://review.openstack.org/12058408:51
BAKfrhi guys09:03
BAKfrI want to add a test in test_v3_identity09:03
BAKfrIn my test method, I should use "self.user" and "self.project" in the IdentityTestCase class, or create new user and project ?09:05
marekdBAKfr: where this self.user comes from? Mind that setUp() is recreated prior to every test.09:06
*** Gippa has joined #openstack-keystone09:11
*** f13o has quit IRC09:12
openstackgerritMarek Denis proposed a change to openstack/python-keystoneclient: Handle federated tokens.  https://review.openstack.org/12114609:13
BAKfrmarekd, it is defined in load_fixtures, and which is called in setUp(). Thanks :)09:14
marekdBAKfr: so if you inherit from that class I think it's even advised to reuse such attributes :-)09:14
*** KanagarajM2 has quit IRC09:23
*** RockKuo_Office has quit IRC09:27
*** Gippa has quit IRC09:32
*** KanagarajM has joined #openstack-keystone09:34
*** lufix has quit IRC09:41
*** lufix has joined #openstack-keystone09:41
*** jasondotstar has quit IRC09:44
*** henrynash has quit IRC09:49
*** Gippa has joined #openstack-keystone09:51
*** KanagarajM2 has joined #openstack-keystone10:00
*** KanagarajM has quit IRC10:02
*** afazekas has joined #openstack-keystone10:03
openstackgerritQin Zhao proposed a change to openstack/python-keystoneclient: Fix the condition expression for ssl_insecure  https://review.openstack.org/11223210:05
*** KanagarajM2 has quit IRC10:06
*** mflobo_ has quit IRC10:07
*** mflobo has joined #openstack-keystone10:08
*** rushiagr is now known as rushiagr_away10:09
*** amakarov_away is now known as amakarov10:10
*** gpanda has quit IRC10:11
marekdwhat is the difference between requirements.txt and text-requirements.txt files?10:21
*** bjornar_ has joined #openstack-keystone10:45
*** dims has joined #openstack-keystone10:56
openstackgerritMarek Denis proposed a change to openstack/keystone: Change pysaml2 comment in test-requrements.txt.  https://review.openstack.org/12180711:02
marekdlbragstad: ^^ i never can add you as a reviewer (Gerrit complains)11:04
*** henrynash has joined #openstack-keystone11:06
*** ukalifon1 has quit IRC11:22
*** henrynash has quit IRC11:23
*** rushiagr_away is now known as rushiagr11:23
*** henrynash has joined #openstack-keystone11:25
*** diegows has joined #openstack-keystone11:29
*** bjornar_ has quit IRC11:42
*** ukalifon has joined #openstack-keystone11:42
*** henrynash has quit IRC11:58
*** henrynash has joined #openstack-keystone12:08
BAKfrIf I want to test a method without the revoke API, can I do "self.assignment_api.revoke_api = None" ?12:11
BAKfrIt seems to work (dependencies are reset between each test), but is it a good practice ?12:11
rodrigodsBAKfr, why would you want to do that?12:16
BAKfrrodrigods, I've a bug which happen only when revoke API is not enabled12:17
*** dims has quit IRC12:18
*** dims has joined #openstack-keystone12:19
rodrigodsBAKfr, hmm that's odd...12:20
rodrigodsBAKfr, can you describe it?12:20
*** richm has joined #openstack-keystone12:21
BAKfrrodrigods, tokens are not revoked when we delete a group role to a project12:22
BAKfrrodrigods, I want to add a test for https://review.openstack.org/#/c/121628/12:22
*** henrynash has quit IRC12:25
*** henrynash has joined #openstack-keystone12:26
rodrigodsBAKfr, the bug is caused by user_id being None, right?12:32
BAKfrrodrigods, the bug is that we revokes the token for user_id instead of user['id']12:34
rodrigodsBAKfr, the _emit is using the wrong variable, but the revoke itself is using the correct one (user['id'])12:36
rodrigodsI thought the issue was caused by the _emit receiving user_id12:36
BAKfrrodrigods, Yes. when revoke_api is enabled, tokens are revoked. The bug happen when revoke_api is not set.12:37
*** vhoward has joined #openstack-keystone12:37
openstackgerritKévin Bernard-Allies proposed a change to openstack/keystone: Revoke the tokens of group members when a group role is revoked  https://review.openstack.org/12162812:41
rodrigodsBAKfr, hmm now I get it... =) yeah... seems legit to disable in a such way than12:43
BAKfrrodrigods, test sent ^12:43
*** sigmavirus24_awa is now known as sigmavirus2412:44
*** KanagarajM has joined #openstack-keystone12:58
*** achampion has quit IRC13:00
*** bjornar has quit IRC13:10
*** topol has joined #openstack-keystone13:11
*** tristanC has joined #openstack-keystone13:12
*** radez_g0n3 is now known as radez13:12
*** bjornar has joined #openstack-keystone13:12
*** bknudson has quit IRC13:16
*** nkinder has quit IRC13:16
*** htruta has joined #openstack-keystone13:21
*** gordc has joined #openstack-keystone13:21
*** joesavak has joined #openstack-keystone13:23
*** henrynash has quit IRC13:25
*** henrynash has joined #openstack-keystone13:32
*** rwsu has quit IRC13:33
ukalifonandreaf: your fix https://review.openstack.org/121562 still doesn't solve the issue for me. Maybe I'm doing something wrong? If I don't set "auth_version = v3" in tempest.conf - I still get "unauthorized" errors and I see calls made to v2.0/tokens instead of /v3/auth/tokens. Actually I see calls to both:13:35
ukalifon2014-09-16 16:24:32,592 1626 INFO     [tempest.common.rest_client] Request (TestKeystoneSanity:test_v3_identity): 201 POST
ukalifon2014-09-16 16:24:32,608 1626 INFO     [tempest.common.rest_client] Request (TestKeystoneSanity:test_v3_identity): 401 POST
*** henrynash has quit IRC13:36
*** ayoung has joined #openstack-keystone13:36
*** ayoung has quit IRC13:45
*** r-daneel__ has joined #openstack-keystone13:48
*** cyeoh has joined #openstack-keystone13:53
*** garnav has quit IRC13:54
*** jasondotstar has joined #openstack-keystone13:55
*** vkmc has joined #openstack-keystone13:55
*** achampion has joined #openstack-keystone13:56
vkmcdolphm, hi there, do have a sec?13:57
*** oomichi has quit IRC13:59
*** nkinder has joined #openstack-keystone14:01
lbragstadmarekd: around?14:01
lbragstadmarekd: use ldbragst14:01
*** henrynash has joined #openstack-keystone14:04
vkmcok well, I wanted to ask you all if someone is interested to mentor an OPW applicant this round https://wiki.openstack.org/wiki/OutreachProgramForWomen14:06
vkmcit doesn't take much time and it's a really nice experience for both mentors and mentees14:07
vkmccurrently there is an applicant interested in Keystone14:07
*** sigmavirus24 is now known as sigmavirus24_awa14:10
vkmcshe wanted to propose a change in Keystone access control model14:12
*** sigmavirus24_awa is now known as sigmavirus2414:12
morganfainbergvkmc, dolphm is likely unavailable at the moment14:13
morganfainbergvkmc, a couple of us are filling in for him.14:13
vkmcthanks morganfainberg14:13
marekdlbragstad: i am here.14:13
*** Tahmina has joined #openstack-keystone14:13
marekdlbragstad: ok14:13
morganfainbergvkmc, though the best bet to get possible volunteers would be to join us for the Keystone IRC meeting at put it on the agenda14:14
lbragstadmarekd: I added myself to it, but for future reference ldbragst is my username14:14
vkmcmorganfainberg, good idea14:14
morganfainbergvkmc, https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting14:14
morganfainbergvkmc, it is later today :)14:14
morganfainbergTahmina, hello14:14
vkmcmorganfainberg, Tahmina is the applicant I was talking talking about :)14:14
Tahminahi morganfainberg14:14
Tahminathanks vkmc14:15
morganfainbergvkmc, Tahmina, welcome I'm sure we can find someone to volunteer.14:15
vkmcTahmina, would you like to join the Keystone meeting today at 18.00UTC?14:15
marekdlbragstad: anyway, what's the procedure for updating requirements.txt ? (see: https://bugs.launchpad.net/keystone/+bug/1369986 )14:15
uvirtbotLaunchpad bug 1369986 in keystone "Federaton extension fails due to missing pysaml2 library" [Undecided,New]14:15
morganfainbergTahmina, and we're always happy to have contributors join us! :)14:16
vkmcTahmina, you can let Keystone folks know about your proposal during the open topic and hopefully somebody would be able to mentor you :)14:16
*** andreaf_ has joined #openstack-keystone14:16
*** andreaf has quit IRC14:16
Tahminaoh thanks vkmc14:16
*** andreaf_ is now known as andreaf14:16
Tahminayes I will join14:16
lbragstadmarekd: we need to check if pysaml2 is available in global requirements14:17
*** andreaf_ has joined #openstack-keystone14:17
vkmcmorganfainberg, thanks :)14:17
morganfainbergvkmc, sure thing.14:17
Tahminabut I don't know where to join14:17
lbragstadmarekd: it looks like it is https://github.com/openstack/requirements/blob/master/global-requirements.txt#L9414:17
lbragstadso we should be able to update our requirements to use it14:17
Tahminathanks morganfainberg14:17
marekdlbragstad: on my way14:17
vkmcTahmina, #openstack-meeting at 18.00UTC14:18
lbragstadmarekd: sweet,14:18
Tahminavkmc: ok thanks14:18
vkmcnice :)14:18
lbragstadmarekd: could you ping me the review when you push?14:18
lbragstadmarekd: I want to add it to the RC1 review list14:18
Tahmina\join #openstack-meeting14:19
rodrigodsmorganfainberg, Tahmina, vkmc, in previous experiences like in GSoC and so on, I think that is worthed to find a nice topic in the project roadmap14:19
marekdlbragstad: yes.14:19
lbragstadmarekd: thanks!14:20
vkmcrodrigods, yeah is usually the best idea... tasks that fit the project's plans14:20
marekdlbragstad: do we need pysaml in both test-requirements and requirements?14:20
marekdlbragstad: or maybe requirements is enough14:20
lbragstadmarekd: is it used in the federation tests?14:20
marekdit is.14:20
lbragstador just in the driver?14:20
marekdit's used in controller14:21
marekdlbragstad: and in tests as well.14:21
morganfainbergTahmina, vkmc, added the topic to the agenda https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting#Agenda_for_next_meeting14:21
*** henrynash has quit IRC14:22
*** vhoward has left #openstack-keystone14:22
vkmcmorganfainberg, awesome! :) we will be there14:22
Tahminavkmc, morganfainberg: thanks a lot14:23
*** stevemar has joined #openstack-keystone14:23
lbragstadmarekd: https://github.com/openstack/keystone/blob/master/test-requirements.txt#L2314:24
lbragstadthat's already in test-requirements.txt14:24
lbragstadmarekd: just add to requirements.txt for now14:24
*** andreaf has quit IRC14:26
morganfainbergYorikSar_, we can't have the cleaner thread14:28
openstackgerritMarek Denis proposed a change to openstack/keystone: Add pysaml2 to requirements.txt.  https://review.openstack.org/12187814:29
marekdlbragstad: for you^^14:29
lbragstadmarekd: thanks!14:29
*** YorikSar_ is now known as YorikSar14:29
YorikSarmorganfainberg: Why?14:29
morganfainbergYorikSar, it's a pattern that we really shouldn't be doing within keystone14:30
*** f13o has joined #openstack-keystone14:30
*** f13o has quit IRC14:30
morganfainbergYorikSar, it is the biggest part of your patchset that was a concern, also my change was mostly to see what it would take to remove it.14:30
YorikSarmorganfainberg: It removed good chunk of functionality as well.14:31
*** rwsu has joined #openstack-keystone14:31
morganfainbergYorikSar, in short we don't want that pattern in keystone. - also my changesets were mostly an experiment to see if i could remove the cleaner thread not a final "has to go this way"14:31
morganfainbergYorikSar i unfortunately meant to convert it to a new change id but missed (sorry)14:31
openstackgerritMarek Denis proposed a change to openstack/keystone: Add pysaml2 to requirements.txt.  https://review.openstack.org/12187814:31
marekdlbragstad: np14:31
YorikSarmorganfainberg: I don't quite see why this thread is so bad from your pov.14:32
morganfainbergYorikSar, because it spins up another thread that must be managed for each pool14:32
YorikSarmorganfainberg: Well, there's no point in having 2 CRs around the same code.14:32
*** david-lyle has joined #openstack-keystone14:32
YorikSarmorganfainberg: It's managed by weakref.14:32
YorikSarmorganfainberg: Thread exists as long as there's at least one reference to the pool.14:33
morganfainbergYorikSar, having 2 CRs is fine, if one is meant to explor an option14:33
morganfainbergYorikSar, we primarily run under apache14:33
morganfainbergYorikSar, which is *single* thread14:33
morganfainbergYorikSar, and we're adding extra overhead for each service that runs auth_token as well.14:33
morganfainbergYorikSar, it's the wrong pattern in this case.14:34
YorikSarmorganfainberg: Really?.. I thought Apache usualy runs multi-process multi-threaded.14:34
morganfainbergYorikSar, keystone under apache is run in mod_wsgi, which ends up being a single thread / process14:34
morganfainbergYorikSar, erm, multiple processes, single thread per14:35
YorikSarmorganfainberg: From mod_wsgi docs: If this option is not defined then the default will be to create 15 threads in each daemon process within the process group.14:36
YorikSarmorganfainberg: It's about threads= option for process group.14:36
YorikSarmorganfainberg: So by default it has multiple threads per process.14:36
morganfainbergYorikSar, oh wait a sec, sorry it's a thread.local works fine in mod_wsgi (sorry i just woke up)14:37
morganfainbergYorikSar, i am *very* concerned about adding in a lot of threading overhead / management / communications.14:37
YorikSarmorganfainberg: There's not much overhead with cleaner thread.14:37
morganfainbergYorikSar, and generally speaking that has been the view when this was discussed in IRC by all involved14:37
YorikSarmorganfainberg: All threading overhead is already there with thread-safety of pool itself. Cleaner just uses existing primitives to create mostly sleeping thread.14:38
YorikSarmorganfainberg: Unfortunatelly I didn't take part in that discussion...14:39
morganfainbergYorikSar, i've tried to ping you on a few occasions about it14:39
morganfainbergYorikSar, :( i wish time differences were such an issue14:40
YorikSarmorganfainberg: Is there any way I can convince people that cleaner thread don't present any significant overhead?14:41
morganfainbergYorikSar, it is mostly a pattern we don't want in Keystone. we don't want extra threads started to "manage" resources14:42
morganfainbergYorikSar, i don't want to have to come back through and rip that out in K or explain to someone why we can't just keep adding threads for things. it adds a lot of complexity for not a lot of benefit14:43
morganfainbergYorikSar, and it makes it a lot harder to understand what is going on in the pool.14:43
*** andreaf has joined #openstack-keystone14:43
YorikSarmorganfainberg: Well, not having tons of unused connections is a benefit...14:43
morganfainbergYorikSar, if it wasn't so late in the cycle (past dep. freeze) we'd convert to pymemcache instead.14:43
morganfainbergYorikSar, we can prevent the explosion of connections without needing a cleaner thread.14:44
YorikSarmorganfainberg: I didn't dig into pymemcache. Does it do connection pooling by itself?14:44
morganfainbergYorikSar, no, but it avoids the thread.local issue completly14:45
morganfainbergour issue is specifically thread.local + eventlet in this case14:45
YorikSarmorganfainberg: It doesn't... dogpile.cache does threadlocal as well.14:45
morganfainbergYorikSar, i've already talked to Mike about that, the plan is to make dogpile use pymemcache instead14:45
morganfainbergYorikSar, but again we're talking about dep-freeze issues.14:46
YorikSarmorganfainberg: ok, back to how to fix this now.14:46
YorikSarmorganfainberg: How do you propose to handle old connections?14:46
morganfainbergYorikSar, we will need to perform the clean up either on accquire or release.14:47
YorikSarmorganfainberg: The approach in you latest patchset won't work...14:47
morganfainbergYorikSar, if i inverted the list for cleanup or separate cleanup it would.14:47
YorikSarmorganfainberg: And if we kill connections only on acquire/release we'll still have lots of connections around when there's no activity..14:48
morganfainbergYorikSar, lets step back, why are we cleaning up connections ever (except when they are legitimately dead)14:48
YorikSarmorganfainberg: But I guess that's not the case for us since there's always some activity.14:49
YorikSarmorganfainberg: To save system resources basicaly.14:49
morganfainbergYorikSar, even if we had no activity, there isn't a big win to cleanup the connections unless we have GC issues.14:49
morganfainbergYorikSar, we're setting a max limit on connections anyway14:49
*** jorge_munoz has joined #openstack-keystone14:49
morganfainbergYorikSar gc/mem leak in the client (aka, using .cas)14:50
YorikSarmorganfainberg: We can't have every Keystone process have hundreds of connections open.14:50
openstackgerritKévin Bernard-Allies proposed a change to openstack/keystone: Revoke the tokens of group members when a group role is revoked  https://review.openstack.org/12162814:50
morganfainbergYorikSar, no, but you'd likely tune the pool for a given keystone install.14:50
YorikSarmorganfainberg: Actually, I think I see that doing cleanup in acquire/release should be good enough...14:51
morganfainbergYorikSar, and i see your point of initial intent though now, sorry about msising it :)14:51
YorikSarmorganfainberg: I thought we shouldn't have maxsize connections always open, but we should be able to open maxsize connections during usage peaks.14:52
morganfainbergYorikSar, so what if we did a .cleanup method that was invoked on a modulus of the time.time() (aka add a delay so we don't cleanup every time)14:52
YorikSarmorganfainberg: I think we can just check if we have anything to cleanup - just as in cleaner thread.14:53
*** ajayaa has quit IRC14:53
morganfainbergYorikSar, oh right should be low enough cost.14:53
*** jsavak has joined #openstack-keystone14:53
YorikSarmorganfainberg: I can implement that later today...14:54
morganfainbergYorikSar, the other question i have is the "always false one"14:54
morganfainbergYorikSar, that is the same logic you used.14:54
YorikSarmorganfainberg: But I'd like to keep separation of logic between abstract pool and specialized memcached pool.14:54
morganfainbergthe only difference is i wrapped it in a ()14:55
morganfainbergfor line wrap14:55
YorikSar           while not self._free_pool and self._acquired >= self._maxsize:14:55
YorikSar                self._no_free.wait()14:55
YorikSarmorganfainberg: That's my code14:55
morganfainbergYorikSar look the line below it14:55
YorikSar_free_pool != _no_free14:55
morganfainbergoh wait14:56
morganfainbergstupid auto-fill :P14:56
*** lufix has quit IRC14:56
*** joesavak has quit IRC14:56
YorikSarmorganfainberg: Yeah. I think the problem is that it's too smart ;)14:57
morganfainbergYorikSar, i'm not opposed to the abstract pool, but it was a lot harder to read when we were dealing with the cleaner14:57
YorikSarmorganfainberg: I use Vim's built-in one :)14:57
morganfainbergYorikSar, let me revert some of this and i'll get it posted up, then you can cleanup/fix as needed and we can move on to the hard part, unit tests14:58
YorikSarmorganfainberg: At least we had all concurrency/threading/scary stuff in one class and all memcached-related stuff in another one.14:58
morganfainbergYorikSar, ++ can't argue there14:58
morganfainbergYorikSar, the hard part is still going to be unit tests.14:59
YorikSarmorganfainberg: I actually think it'd be easier to modify original code to move cleaning logic than do the split in new code...15:00
openstackgerritTristan Cacqueray proposed a change to openstack/keystone: Adds a whitelist for endpoint catalog substitution  https://review.openstack.org/12188915:00
morganfainbergYorikSar, sure. there are some minor stuff i'll pull back15:00
morganfainbergYorikSar, the logger can be a method (for example) instead of a function.15:00
YorikSarmorganfainberg: unittests, yes... Unittests for concurrency code. That's a very funny way to spend a lot of time :)15:00
morganfainbergYorikSar, yeah, but we need them so we don't accidently break it / regress15:01
YorikSarmorganfainberg: Yeah, w/o background thread we don't need it to be external.15:01
morganfainbergYorikSar, ok cool, i'll get this posted up quickly and then we circle back later today?15:01
YorikSarmorganfainberg: Oh, let's just say "never touch this module! it works!" :)15:01
morganfainbergYorikSar, haha i wish.15:01
YorikSarmorganfainberg: Yes, I'll be online in couple hours or so.15:02
morganfainbergYorikSar, and sorry didn't mean to scare you with the changes, i did mean to make it a separate CR for exploration reasons.15:02
morganfainbergYorikSar, just totally missed on it.15:02
morganfainbergYorikSar would have been easier to side-by-side compare15:03
*** ayoung has joined #openstack-keystone15:03
YorikSarmorganfainberg: Yeah.. It felt really bad. Like "Thanks for your code, let's cut it to pieces"15:03
morganfainbergYorikSar, yeah that was *not* my intention15:03
morganfainbergYorikSar, it really was intended to just explore the option(s)15:03
YorikSarmorganfainberg: Yes, I undestand that now. :)15:04
morganfainbergYorikSar, ok good, cause don't want you running off and not continuing with keystone15:04
tristanCHello folks, so https://bugs.launchpad.net/ossa/+bug/1354208 have just been disclosed, I submitted patches there: https://review.openstack.org/#/q/If02327d70d0143d805969fe927898f08eb84c4c2,n,z  . Cores please have a look and approve them!15:04
uvirtbotLaunchpad bug 1354208 in keystone "[OSSA 2014-029] Catalog replacement allows reading config (CVE-2014-3621)" [Medium,In progress]15:04
YorikSarmorganfainberg: I guess I should've found some time to address at least some of comments yesterday.15:04
morganfainbergYorikSar, eh, we're just up againsts a very short window otherwise it would be less of an issue (RC)15:05
morganfainbergYorikSar, things get a little wonky in RC period15:05
YorikSarmorganfainberg: Well... I guess I once again will be disapearing from Keystone after couple patches. I've just moved to another department and will have to pay way more attention to our internal stuff, and will have way less time to give to community...15:06
*** ukalifon has quit IRC15:07
morganfainbergYorikSar, ah well that is a valid reason to disappear15:07
YorikSarmorganfainberg: That was my third-ish attempt to get involved in Keystone.15:07
morganfainbergYorikSar, when you have time come and visit us though :)15:07
YorikSarmorganfainberg: First was LDAP driver that I thought we'll use for internal cloud (which we started to use ~2 years later), 2nd - some more work on LDAP driver, this is a third one :)15:08
YorikSarmorganfainberg: It's like some fate that follows me around: whenever I start doing smth for Keystone, I barely have time to finish one thing and have to runaway; whenever I have intern working on Keystone, someone takes him/her away from me :)15:10
morganfainbergYorikSar, ah well, one of these times you'll get to stick around!15:10
YorikSarmorganfainberg: Unfortunately as Mirantis grows bigger, it becomes harder to find time for comunity in areas I really like...15:11
morganfainbergYorikSar, ah growing organisations15:12
YorikSarmorganfainberg: I remember OpenStack department in Mirantis as 5 guys laughing all day long and trying to make Diablo work as expected :)15:13
*** cjellick has joined #openstack-keystone15:14
morganfainbergYorikSar, how things change15:15
*** sigmavirus24 is now known as sigmavirus24_awa15:17
*** sigmavirus24_awa is now known as sigmavirus2415:19
*** ayoung has quit IRC15:23
*** afazekas has quit IRC15:28
openstackgerritA change was merged to openstack/keystone: Document Keystone2Keystone federation  https://review.openstack.org/12058415:42
*** marcoemorais has joined #openstack-keystone15:42
*** zzzeek has joined #openstack-keystone15:43
*** rushiagr is now known as rushiagr_away15:43
*** ayoung has joined #openstack-keystone15:44
*** wwriverrat has joined #openstack-keystone15:52
*** BAKfr has quit IRC15:55
*** lsmola_ is now known as lsmola15:57
*** lsmola is now known as lsmola______15:58
*** bknudson has joined #openstack-keystone16:00
*** raildo has joined #openstack-keystone16:01
dstanekmorganfainberg: you around?16:02
morganfainbergdstanek, here16:02
dstanekmorganfainberg: quick question about the logic here: https://review.openstack.org/#/c/121628/3/keystone/assignment/core.py16:02
morganfainbergdstanek, sture16:02
dstanekmorganfainberg: why compare user['id'] to user_id? what is the user_id representing in a call to delete grant16:03
morganfainbergdstanek, it's just to save the emitting of an event since user_id already has an event emitted at the end at line 58416:04
morganfainbergdstanek, otherwise in theory we could emit for user_id in each and every item in the list16:04
dstanekmorganfainberg: ah, ok16:04
*** _cjones_ has joined #openstack-keystone16:07
_cjones_morgainfainberg: I was talking with ayoung a few months back. He pointed me to a document regarding keystone terminology tenant vs. project in the like for Openstack > Icehouse. I've searched the google machine but can't find it. Do you know the page I'm talking about?16:08
*** joesavak has joined #openstack-keystone16:08
dstanek_cjones_: a doc talking about why it changed?16:08
_cjones_morganfainberg: Sorry mistyped your name.16:09
_cjones_dstanek: No, just a doc detailing what the new meanings were. IIRC.16:09
* _cjones_ is trying to settle an internal corp. dispute.16:09
dstanek_cjones_: i though they basically mean the same thing - the old things was to call them tenants and the new thing is to call them projects16:10
*** jsavak has quit IRC16:10
_cjones_dstanek: Correct. But there was documenation to that effect. versions > Icehouse the proper thing to do is use "projects">16:10
_cjones_I thought this was keystone specific documentation, but I could be wrong and just in the O/S general manual.16:11
dstanek_cjones_: hmmm, no idea. i just never call them tenants. some of our APIs expect the tenant name in a few places though16:12
*** henrynash has joined #openstack-keystone16:12
raildohenrynash, can you answer me a question?16:16
*** DavidHu_ has joined #openstack-keystone16:17
raildoNow that hirarchical projects is on a branch and  is being reviewed, it still come into Juno, ie, it must be approved until the release of juno-3?16:17
raildoI believe that another keystone-core can also answer :)16:17
henrynashraildo: I think that’s a dolphm question....16:23
*** palendae has quit IRC16:23
raildodolphm, ping?16:23
*** palendae has joined #openstack-keystone16:24
*** gordc has quit IRC16:24
*** palendae has quit IRC16:25
*** palendae has joined #openstack-keystone16:25
henrynashbknudson, lbragstad: I think I have addressed all your collective concerns on https://review.openstack.org/#/c/121246/16:26
lbragstadhenrynash: I was just about to start looking at the latest version16:27
henrynashlbragstad: thx :-)16:27
lbragstadhenrynash: looks good to me so far, I wanted to go through the tests one more time16:27
henrynashlbragstad: sure, good idea16:27
*** amakarov is now known as amakarov_away16:27
*** _cjones_ has quit IRC16:28
*** palendae has quit IRC16:30
*** gordc has joined #openstack-keystone16:30
*** palendae has joined #openstack-keystone16:30
*** marcoemorais has quit IRC16:32
*** marcoemorais has joined #openstack-keystone16:32
ayoungdolphm, can we kill this old-obsolete BP  https://blueprints.launchpad.net/keystone/+spec/revert-multiple-ldap-servers   even though it is set to "Priority:  Not" I've had a couple people ask me if we are reverting the multi-backend code due to seeing that BP.16:45
*** wwriverrat has left #openstack-keystone16:47
henrynashdstanek: quick one…as per your comment in https://review.openstack.org/#/c/121246/12/keystone/identity/backends/sql.py - is it a standard to have 2 lines between the end of imports and any code?  2 lines before a class for sure….?16:53
dstanekhenrynash: yes, http://docs.openstack.org/developer/hacking/#import-order-template16:54
henrynashdstanek: you’re so right....16:55
henrynashdstanek :-)16:55
ayounghenrynash, dumb idea16:55
ayoungI mean, I have a dumb idea16:55
dstanekhenrynash: i didn't -1 because i didn't think it was a huge deal - did you see my question in there?16:55
ayoungand I want to run it past you16:55
ayoungwhat if....16:56
henrynashdtsanek: yes…let me post the answr16:56
ayoungand this is for gyee, really\16:56
ayoungwe allowed any keystone operation to add in the "scope" section of the token request16:56
ayoungand possibly even the password section if we want to be thorought16:56
ayoungand that means:  don'16:56
ayoungt issue me a new token, just perform the whole operation as if I was asking for a token16:57
henrynashayoung: so…a one off token for that operation16:57
ayoungthis makes a lot more sense when using basic/auth or kerberos or something16:57
ayoungyep, a one off non-persisted token16:57
dstanekhenrynash: that's what i thought - the commit message made it sound like there was a behavioral change16:57
henrynashdstanek: ah, ok…I’ll make the limiation a bit clearer...16:58
morganfainbergayoung, marked that BP as superseded16:58
*** sigmavirus24 is now known as sigmavirus24_awa16:59
henrynashdstanek: all this chaneg really does is allow you to give you’re one and only sql backend to a specific domain, rather than have it as a catchall for all domains without a specific config16:59
henrynashdtsanek: but in an “all LDAP’ configuration, that’s exactly what you might want17:00
henrynashayoung: so, yes, if you were using LDAP or something than authenticating each time might make transcations somewhat slow17:02
henrynashayoung: as an aside, haev we given up on service tokens….or are they on the docket for Kilo?17:03
*** harlowja_away is now known as harlowja_17:04
*** antonio__ has joined #openstack-keystone17:16
ayounghenrynash, LDAP can be cached.  I think optimizing LDAP should be its own effort regardless17:19
antonio__Hello, I'm studying keystone for a reliable and scalable deployment of an OpenStack infrastructure. I would like to know if there is something configurable on the Keystone module to scale it up so it does not represent a single point of failure for the whole structure, since every other service uses it as an auth system. Thank you very much!17:19
henrynashayoung: fair comment17:19
*** bjornar_ has joined #openstack-keystone17:19
ayoungantonio__, use Gallerian and multile keystone instances17:19
morganfainbergayoung, antonio__, Galera17:19
ayoungewhat he said17:20
morganfainbergpercona is also viable (based on galera)17:20
ayoungright...make the DB scale, and then run multiple keystone servers pointing at the same DB17:20
morganfainbergthough, don't use limited use trusts on galera / percona17:21
morganfainbergon icehouse that is.17:21
ayoungmorganfainberg, yeah, the whole reference counting thing17:23
ayounghenrynash, so,  you think it makes sense?  Adding in the auth block to other requests?17:23
henrynashayoung: and what’s the key thing this solves…that perhaps a service token would not solve?17:24
ayounghenrynash, it is a service token17:24
ayounghenrynash, it would let an endpoint use X509 or kerberos to auth, and perform everthing it needed in one request17:25
ayounghenrynash, add in a trust....17:25
antonio__ayoung, morgangainberg: thank you for your replies, it'is really very useful17:25
ayoungantonio__, YW17:25
henrynashayoung: so whaat makes it a one-shot “service” token, as opposed to a one-shot regular token?17:26
ayounghenrynash, there is no token17:26
ayoungit is a one-shot-all-in-one request17:26
henrynashayoung: ha!  a subtle plan!17:26
ayounghenrynash, why would we need service tokens?17:26
*** antonio__ has left #openstack-keystone17:26
*** kevinbenton has quit IRC17:27
*** DavidHu has quit IRC17:27
ayounghenrynash, so, service tokens, as I understand them, are dumb17:27
ayoungwhich probably means I don't understand them17:28
ayoungbut if they are being proposed as a proxy for authentication, then they are dangerous17:28
*** richm has quit IRC17:28
ayounghenrynash, I'17:28
ayoungd rather make X509 and Kerberos the default way services authenticate to keystone, then let keystone make the authentication decisions at the point of request17:29
ayoung a token is really a cache of that data, and, while we could always cache, we already do cache with dogpile, so lets let a cache be a cache and not cache out cache.  Capiche?17:29
ayounghenrynash, we could make a separate middleware that does this:17:30
ayounglook for the auth data, call the token-provider with  a flag that says do not persist the token.  The token data gets added to the request, and then that is what is used for the policy check17:31
ayoungthen only run that middleware on internal interfaces17:32
*** kevinbenton has joined #openstack-keystone17:32
henrynashayoung: yes, you could use middleware to fairly easily add support….17:33
*** richm has joined #openstack-keystone17:33
henrynashayoung: in general, this seems like a good idea17:33
ayounghenrynash, let me cook up a sample request for, say, validate token17:33
ayounghenrynash, heh, it would just a be a token request body and a Header with X-Subject-Token: c6758017:36
ayounglet me find something less trivial17:36
ayounggiving a user a role in a project:17:36
ayoungthat is just17:37
ayoungPUT /projects/{project_id}/users/{user_id}/roles/{role_id}17:37
ayoungDamn this well designed REST API.17:37
ayoungOk, create role17:38
ayounghenrynash, morganfainberg http://paste.openstack.org/show/112246/17:39
*** soulxu_ has joined #openstack-keystone17:40
ayounghenrynash, morganfainberg and for kerberos it would be the more straightforward http://paste.openstack.org/show/112248/17:42
*** alex_xu has quit IRC17:43
ayoungwe could actually do it in the existing auth context middleware:  http://git.openstack.org/cgit/openstack/keystone/tree/keystone/middleware/core.py#n24317:44
ayoungthere is code in the auth controller that should be moved into the token provider, but that is no surprise17:45
henrynashayoung: not sure why you would change the api other than include the authentication info….(rather than try and deduce the api paramters from the auth data)17:46
morganfainbergdstanek, henrynash, ayoung, a second +2 / +A on this would be great https://review.openstack.org/#/c/121889/17:46
ayounghenrynash, reall all I need is {scope}17:46
henrynashayoung: which is what I think you are doing in that example17:46
morganfainbergthis has been pre-approved in the bug fwiw.17:46
henrynashayoung: but the role you are adding might not be for you…. (for example)17:46
ayounghenrynash, but you need to convert from, say the kerberos credentials to the user id  etc,17:46
ayoungand need to know which mapping to use17:47
ayoungno, in this case, the role is for "you"17:47
ayoungheh, the "role" example is also potentially confusing, but the alternative is create user, which would probablty be just as confusing17:47
*** aix has quit IRC17:48
ayounghenrynash, the short would be "anthing we can add to a token request we can layer on to any other request and get authentication and authorization for just that request"17:48
dstanekmorganfainberg: i didn't want to +2 because i wrote it17:48
morganfainbergdstanek, ah17:48
henrynashayoung: agreed17:48
morganfainbergdstanek, thanks.17:48
ayoungcreate token would then be  "persist this token you just created"17:48
*** samuelmz has quit IRC17:48
ayoungdstanek, is that a bug fix?17:49
ayoungit smells like a feature to me17:49
ayoungah...now I read the bug17:49
ayoungI see17:49
ayoungmorganfainberg, go ahead and +2 if you like, but let me review it as well17:50
dstanekayoung: yeah. a security bug that got disclosed today17:50
ayoungmorganfainberg, I'll +A  if I am done after you17:50
ayoungdstanek, '(Deprecated)  right up front?17:51
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945217:55
morganfainbergYorikSar, ^ tested that to work as expected in devstack, still needs tests17:55
YorikSarmorganfainberg: Ok, will take a look17:56
morganfainbergYorikSar, oh i'll post a new one in a sec. i need to fix the documentation thing i did before17:56
morganfainbergYorikSar, will post in ~2mins17:56
morganfainbergYorikSar it should be much closer to your original code now but just minus the cleaner thread17:56
openstackgerrithenry-nash proposed a change to openstack/keystone: Ensure identity sql driver supports domain-specific configuration.  https://review.openstack.org/12124617:59
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945218:00
morganfainbergYorikSar, ok there. added doc stuff back in18:00
*** jsavak has joined #openstack-keystone18:00
morganfainbergmeeting time!18:01
*** joesavak has quit IRC18:03
*** samuelmz has joined #openstack-keystone18:07
*** sigmavirus24_awa is now known as sigmavirus2418:07
*** palendae has quit IRC18:08
*** palendae has joined #openstack-keystone18:08
*** joesavak has joined #openstack-keystone18:16
bknudsonmorganfainberg: https://bugs.launchpad.net/python-keystoneclient/+bug/137019218:16
uvirtbotLaunchpad bug 1370192 in python-keystoneclient "Update oslo requirements to final release" [Undecided,New]18:17
morganfainbergbknudson, thanks18:17
*** jsavak has quit IRC18:19
openstackgerritA change was merged to openstack/keystonemiddleware: Use oslo_debug_helper and remove our own version  https://review.openstack.org/12010518:20
marekdbknudson: morganfainberg dstanek ayoung lbragstad : I would appreciate some eyes on this: https://review.openstack.org/#/c/111771/18:26
*** KanagarajM has quit IRC18:26
YorikSarmorganfainberg: Indeed, diff looks way shorter. Fixing bugs...18:31
morganfainbergYorikSar, ++ thanks!18:31
nkinderwhere are the API docs for fetching the token revocation list?18:31
nkinderit doesn't look like it's documented18:31
morganfainbergYorikSar, the _cleaner method i'm not sure about tbh, it feels clunky18:31
morganfainbergnkinder, uhm..18:32
morganfainbergnkinder, i don't think it is documented *wince*18:32
morganfainbergtough i think it shoudl be in identity-api18:32
nkinderthe blueprint says GET /tokens/revoked/ on the admin port, but I get a 404 with that18:33
morganfainbergnkinder, try on port 5000?18:34
nkindermorganfainberg: tried on both with v3.  404 either way18:34
morganfainbergor.. wait, try w/o the trailing /  maybe?18:34
morganfainbergoh v3, v3 is /auth/revoked i think18:34
YorikSarmorganfainberg: It was broken a bit ;)18:34
bknudsonnkinder: there's a v3 extension to get the revoked tokens18:34
YorikSarmorganfainberg: btw, how about calling it in release as well?18:34
morganfainbergYorikSar, i tested it and it worked, but i wasn't sure about it being the right answer18:34
nkinderah, it's v2.0 only18:34
morganfainbergYorikSar, talk about that post meeting18:35
nkinderlet me try auth/revoked with v318:35
YorikSarmorganfainberg: Oh, sure.18:35
bknudsonnkinder: oh, that's events, not the list...18:35
nkindermorganfainberg: no dice with /auth/revoked18:36
rodrigodswhen I want to authenticate a v3 token, where is done the actual checking of user/role/project?18:36
morganfainbergnkinder, let me look at the code there is a router for it, just don't know where it is18:36
morganfainbergnkinder, post meeting i'll find it unless someone beats me to it18:36
nkindermorganfainberg: I can look myself. :)18:36
bknudsonnkinder: the auth_token middleware fetches it, so see what it does18:36
morganfainbergnkinder, of course.18:37
morganfainbergnkinder, :P18:37
*** ukalifon1 has joined #openstack-keystone18:40
rodrigodswhere in the code, I mean =)18:41
ukalifon1nkinder: does the objectSid attribute exist for every record in AD, or how do I add it?18:41
nkinderukalifon1: I think it exists for every record18:42
nkinderukalifon1: I tore down my AD vm, so I'd need about 20 minutes to spin up a new rhos->AD setup to check18:42
ukalifon1nkinder: so how is it that this bug ever existed?18:42
nkinderukalifon1: well, the utf8 code that blows up on it is newer18:42
nkinderukalifon1: newer == icehouse IIRC18:43
nkinderukalifon1: as I mentioned before though, my AD with Icehouse was working18:43
nkinderukalifon1: it might be dependent on what exactly is in the ObjectSID value18:43
ukalifon1nkinder: OK, I still haven't been able to connect keystone to the AD with the IPA trust. Did Rich look at what's failing in his script?18:44
nkinderukalifon1: I'm not sure where you two left off.  I have working scripts that set up AD18:46
nkinderwith RHOS18:46
*** henrynash has quit IRC18:55
*** henrynash has joined #openstack-keystone18:56
morganfainbergYorikSar, ok my only concern about running cleaner on both acquire and release is it could add a lot of extra looping18:57
morganfainbergYorikSar, i'd prefer to only call it in either acquire or release rather than in both18:58
morganfainbergYorikSar in most environments we wont need to reap connections that often.18:58
YorikSarmorganfainberg: I think we should call it only in release then.18:58
morganfainbergYorikSar, works for me18:58
YorikSarmorganfainberg: The later we call it the more connections we catch.18:58
morganfainbergYorikSar, i was thinking about it and i agree, in release makes more sense18:59
YorikSarmorganfainberg: I'm testing my new version, will upload it soon.18:59
morganfainbergYorikSar, cool thanks18:59
*** henrynash has quit IRC19:01
topolhenrynash: This at least enables the (often requested) scenario of service users19:02
topolto be stored in SQL in a predominantly LDAP installation.  <-- THATS AWESOME19:02
YorikSarmorganfainberg: btw, I've removed comments about 'this is for memcached KVS backend' from [memcache] section.19:02
marekdanybody knows what's up with guang? he's been around recently?19:02
YorikSarmorganfainberg: If we are in [memcache] section then we use memcache backedn19:02
morganfainbergYorikSar, sounds good19:03
morganfainbergYorikSar, if you removed that in the config.py file make sure you regenerate the sample config: tox -esample_config19:03
morganfainbergbefore submitting19:03
YorikSarmorganfainberg: Oh, will do.19:03
morganfainbergoh actually. hm19:04
nkindermorganfainberg, bknudson: v3 is /auth/tokens/OS-PKI/revoked19:04
morganfainbergno we might need those comments, or need to add documentation to configuration.rst for the memcache token backend (probably the right answer)19:04
morganfainbergnkinder, ahah19:04
morganfainbergYorikSar, i'll look at adding to the documentation itself instead of in the config file help19:05
bknudsonnkinder: ah. There's a bug out there to document that extension.19:05
nkinderbknudson: yes, writing it on my list19:05
nkinderI'm working on fixing a bug where we blow up calling openssl when we attempt to fetch the revocation list and the configured provider is uuid19:05
nkinderno cert/key == ENOENT19:06
*** gabriel-bezerra has joined #openstack-keystone19:06
bknudsonnkinder: I think I might have a bug for it already?19:07
bknudsonI haven't been working on it19:07
nkinderbknudson: oh, I haven't searched.  I can whip up a fix for it19:07
nkinderbknudson: I'll search LP19:07
bknudsonnkinder: I don't have an external bug for it... just someone complained to me internally19:08
nkinderbknudson: ok, I'll file one then19:08
bknudsonnkinder: and as I was working on it it seemed to require extension discovery so then I worked on JSON Home support19:08
nkinderbknudson: for OS-PKI?19:09
bknudsonnkinder: I've got this change: https://review.openstack.org/#/c/92727/19:09
*** ukalifon1 has quit IRC19:09
bknudsonnkinder: oh, here's the bug: https://bugs.launchpad.net/keystone/+bug/131730219:09
uvirtbotLaunchpad bug 1317302 in keystone "pki_setup shouldn't be required to check revocations" [Wishlist,In progress]19:09
bknudsonnkinder: is this what you think the problem is?19:09
bknudsonthat's the only time auth_token would fetch the revocation list with UUID tokens, is if you've configured auth_token to check the revocation list for uuid tokens.19:10
nkinderbknudson: I thought that UUID simply deletes tokens from the database when revocation happens.  Is that not the case?19:10
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945219:10
YorikSarmorganfainberg: Here it is ^19:11
bknudsonnkinder: no, it marks them as invalid19:11
YorikSarmorganfainberg: I'll add comments on what I've changed19:11
nkinderbknudson: ok, then the token-flush cleans them up?19:11
morganfainbergYorikSar ty19:11
bknudsonnkinder: token-flush will delete them eventually once the expiration time has passed.19:11
morganfainbergYorikSar, ah thanks for using the namedtuple much better!19:12
bknudsonnkinder: the expiration list is a database query: expiration_time > now and invalid19:12
nkinderbknudson: yeah, that makes sense19:13
nkinderbknudson: so you could sign the revocation list even with UUID (and I don't see why you wouldn't want to)19:13
nkinderbknudson: if we have an unsigned revocation list, the client needs to be able to handle that too19:14
bknudsonnkinder: y, it's just when you use UUID tokens you typically don't have to do pki_setup.19:14
bknudsonnkinder: and yes, there's work to do on both sides.19:14
nkinderbknudson: we need to determine if unsigned revocation lists are even a good idea honestly19:15
bknudsonnkinder: also, we've got the audit_id in the tokens now. So rather than have a list of token IDs we can have a list of audit_ids. No need to encrypt in that case.19:15
nkinderbknudson: but signing is valuable19:15
nkinderbknudson: it's something to think on some more19:16
bknudsonnkinder: sure... if you're getting the cert from the server you're not gaining much... if the cert was provided by a more secure transport then signing would provide more security.19:17
bknudsonmaybe the certs should be distributed via your chef or puppet or whatever19:18
openstackgerritYuriy Taraday proposed a change to openstack/keystone: Add a pool of memcached clients  https://review.openstack.org/11945219:21
*** harlowja_ has quit IRC19:22
*** harlowja has joined #openstack-keystone19:22
YorikSarmorganfainberg: Forgot to remove cleanip from acquire()19:23
morganfainbergYorikSar, ah. i see19:23
morganfainbergall good19:23
morganfainbergYorikSar, thanks!19:23
morganfainbergnow... we need to figure out unit tests *ugh*19:23
morganfainbergi'll start trying to come up with those post lunch19:24
YorikSarmorganfainberg: Post new patchset if you come up with smth. I'll look at it tomorrow. I also have some ideas, but it's too late...19:25
morganfainbergYorikSar, sounds good19:26
rodrigodsmorganfainberg, quick question: when I want to authenticate via v3 token, where the user/role/project checking is done? (in the code)19:33
dstanekCan we got one more +2 on https://review.openstack.org/#/c/121889/ ?19:52
*** Gippa has joined #openstack-keystone19:53
*** _nonameentername has quit IRC19:53
*** nonameentername has joined #openstack-keystone19:54
*** Gippa has quit IRC19:56
*** topol has quit IRC19:57
*** afaranha has quit IRC20:01
*** jsavak has joined #openstack-keystone20:02
*** afaranha has joined #openstack-keystone20:03
*** topol has joined #openstack-keystone20:03
*** joesavak has quit IRC20:04
dstanekmorganfainberg: YorikSar: would devstack use the keystone default for max pool size or should that be set higher for a running deployment?20:08
ayoungmarekd, on  https://review.openstack.org/#/c/111771/  .  did you mean to remove the kwargs from the functions  like  _get_unscoped_token  or was that a rebase thing?20:08
*** andreaf has quit IRC20:08
*** andreaf has joined #openstack-keystone20:09
*** Tahmina has quit IRC20:10
*** soulxu__ has joined #openstack-keystone20:15
YorikSardstanek: default should be enough for devstack. It might even be enough for rally runs.20:18
*** soulxu_ has quit IRC20:19
*** jasondotstar has quit IRC20:21
morganfainbergdstanek, probably for a real deployment that needs to be tuned.20:22
dstanekYorikSar: really only 10 concurrent requests using the cache?20:23
dstanekmorganfainberg: yeah, for a real deployment i would expect it to be much, much higher to avoid contention, but i was thinking for devstack deployments. do gate tests run in paralllel or are they mostly serialized?20:25
YorikSardstanek: Yep. I think that can be enough. We don't have hard data about that.20:25
morganfainbergdstanek, we don't test memcache in devstack20:25
dstanekmorganfainberg: ah, ok.20:25
*** wwriverrat has joined #openstack-keystone20:25
YorikSardstanek: For real deployments we're going to use 100 iirc.20:25
morganfainbergdstanek, gate tests run in parallel (i think tempest runs multi-worker)20:25
morganfainbergdstanek, which is why it's important we get unit tests for this. we don't get functional nor integration testing atm20:26
morganfainberg(I'm honestly surprised we haven't had more weird memcache issues)20:26
morganfainbergbut for typical devstack, i *think* we're ok with 10 being the default20:27
morganfainbergwe could move to 100 default and tell people "tune it down if that is too hight"20:27
morganfainbergnot sure which default is more sane in this case20:27
*** radez is now known as radez_g0n320:27
*** htruta has left #openstack-keystone20:28
dstanekmorganfainberg: also is the __getattr__ issue a non-issue?20:30
morganfainbergdstanek, in this case it should be a non-issue. i guess we could add an 'is_callable' check in there somehow20:30
* morganfainberg 2x checks20:31
morganfainbergdstanek, crud. no we need to handle properties20:32
morganfainbergnot just callables20:32
dstanekmorganfainberg: right now dogpile.cache backends are documented to have one property20:33
morganfainbergand in kvs we use more20:33
morganfainbergi've layered a couple extras on20:33
*** rodrigods has quit IRC20:33
morganfainbergok so we need some docs, fix for properties and tests20:34
dstanekmorganfainberg: actually YorikSar answered here: https://review.openstack.org/#/c/119452/5/keystone/common/cache/backends/memcache_pool.py20:34
morganfainbergderp just client20:34
morganfainbergoh yeah that should be fine then20:34
dstanekhmmm....what does dogpile use as the backend?20:34
morganfainbergdogpile's backends are the dopile cache backend object20:35
morganfainbergso class PooledMemcachedBackend(memcached_backend.MemcachedBackend) is a dogpile backend20:35
morganfainberg.client is referencing the memcache lib20:35
dstanekmorganfainberg: sorry, i know next to nothing about the dogpile architecture :-(20:36
morganfainbergdstanek, dogpile has 3 layers (really, ignore the decorators)20:36
dstanekso we pass in an instance of PooledMemcachedBackend for dogpile - then it will call .client when it needs one?20:36
morganfainbergdstanek, Region: This is the base configuration and has the locking, etc, holds the decorators etc20:36
morganfainbergdstanek, Backend: what the region references20:36
morganfainbergdstanek, Storage (e.g. dict, memcached, etc) which is the .client in this case20:37
bknudsondstanek: were you going to work on a change to remove endpoint_substitution_whitelist after https://review.openstack.org/#/c/121889/1/keystone/common/config.py is merged?20:37
morganfainbergdstanek, so yeah the region will say "backend do a 'get' on this key" and the backend grabs a client object then uses that (in this case the proxy'd client object)20:38
morganfainbergin reality when you do a .get / .set you're doing the .get/.set on the dogpile region, which does keymangling, and other logic20:39
dstanekbknudson: yes20:39
*** gabriel-bezerra has quit IRC20:40
dstanekso it looks like there any several properties on the memcache.Client, but I don't think dogpile would be using any of them20:49
dstanekmorganfainberg: do you know dogpile's backend implements all of the memcache client interface? https://bitbucket.org/zzzeek/dogpile.cache/src/1c753914b335b4391bc5847a87b7c52ca81c2bc6/dogpile/cache/backends/memcached.py?at=master20:51
morganfainbergdstanek, it does not support CAS20:51
morganfainbergdstanek, it only supports get/set/delete and the muli variants20:52
morganfainbergdstanek, so it isn't a complete memcache impl.20:52
*** marcoemorais has quit IRC20:52
*** marcoemorais has joined #openstack-keystone20:52
morganfainbergdstanek, and dogpile should only care about the get/set/delete/multi stuff20:53
morganfainberg(for now)20:53
dstanekmorganfainberg: but their backend impl implements those and .client - ours just implements .client20:54
morganfainbergdstanek, our backend is a subclass of that backend20:54
*** marcoemorais1 has joined #openstack-keystone20:54
morganfainbergdstanek, and .get/set/delete call self.client20:54
dstanekmorganfainberg: oh, nm. you're right. i'm getting confused between memcached_pool and _memcached_pool20:55
morganfainbergdstanek, thats why i moved _memcache_pool out of the backends directory20:55
*** jsavak has quit IRC20:56
*** marcoemorais has quit IRC20:58
*** gokrokve has joined #openstack-keystone20:58
*** marcoemorais1 has quit IRC21:02
*** marcoemorais has joined #openstack-keystone21:02
*** amerine has quit IRC21:02
*** marcoemorais has quit IRC21:02
*** amerine has joined #openstack-keystone21:02
*** marcoemorais has joined #openstack-keystone21:03
dstanekmorganfainberg, YorikSar: added a few more comments to https://review.openstack.org/#/c/119452/ not that i understand the dogpile side a little better21:03
morganfainbergdstanek, cool21:03
morganfainbergdstanek, i think the leaking over max is an issue21:06
morganfainbergi can come up with a high load scenario that could in theory continue to leak upwards21:06
morganfainbergand the self._acquired increment/decrement are not protected and could race in some cases.21:06
* morganfainberg hates threading code21:07
*** gabriel-bezerra has joined #openstack-keystone21:07
morganfainbergoh wait hm, += might be ok21:07
morganfainbergdstanek, unless eventlet yeild points wont be hit in that logic, which case the coroutine would be enough of a gate.21:08
morganfainbergayoung, i vote eventlet off the island21:09
morganfainbergdstanek, responded to your comments.21:13
morganfainbergdstanek, either i or YorikSar can address most in the next patch21:13
*** soulxu_ has joined #openstack-keystone21:14
*** rodrigods has joined #openstack-keystone21:16
*** rodrigods has quit IRC21:16
*** rodrigods has joined #openstack-keystone21:16
*** soulxu__ has quit IRC21:17
dstanekmorganfainberg: ok, i'll take a look - the += should be ok, but there is a gap between the check against maxclients and where the client is created21:22
dstanekmorganfainberg: do you know how locks are handled in eventlet?21:22
morganfainbergdstanek, unfortunately no.21:23
dstanekmorganfainberg: i'm going to do some experimenting :-)21:23
morganfainbergbut i don't think a .pop is a yeild point21:23
morganfainbergdstanek, now we might need to increment before we do the create then try/except around the create with a decrement if the create fails21:24
morganfainbergsince the create could have a yeild point deep in it21:24
dstanekdoes creating a connection object introduce an opportunity to yield, especially if it actually does networking stuff to establish a connection21:25
morganfainbergdstanek, looking at that now actually21:25
*** gokrokve has quit IRC21:26
*** gokrokve has joined #openstack-keystone21:27
*** wwriverrat has quit IRC21:27
*** vkmc has quit IRC21:28
*** wwriverrat has joined #openstack-keystone21:28
morganfainbergdstanek, it looks like the actual socket.connect isn't called until a call that actually talks to memcache occurs21:28
morganfainbergdstanek, so there is likely no yield in the create connection method.21:29
dstanekok, my fear is that there is something i don't understand here21:36
*** mflobo has quit IRC21:38
morganfainbergi know eventlet locking is meant to work like normal threading just yeilding for coroutines21:39
*** morganfainberg is now known as CaptainMorgan21:39
*** CaptainMorgan is now known as morganfainberg21:39
*** wwriverrat has left #openstack-keystone21:41
*** rkofman has quit IRC21:44
*** rkofman has joined #openstack-keystone21:45
ayoungmorganfainberg, if by "meant to" you mean "acts nothing like but pretends it does"  then, sure.21:46
*** zzzeek has quit IRC21:47
*** topol has quit IRC21:48
*** stevemar has quit IRC21:49
*** bjornar_ has quit IRC21:54
*** achampion has quit IRC21:55
*** bknudson has quit IRC21:56
openstackgerritA change was merged to openstack/keystone: Change pysaml2 comment in test-requrements.txt.  https://review.openstack.org/12180721:59
openstackgerritA change was merged to openstack/keystone: Revoke the tokens of group members when a group role is revoked  https://review.openstack.org/12162821:59
openstackgerritA change was merged to openstack/keystone: Adds a whitelist for endpoint catalog substitution  https://review.openstack.org/12188921:59
morganfainbergayoung, like i said, i vote it off the island22:00
ayoungmorganfainberg, its been off the island for years, with a book deal and its own reality program22:00
ayoungmaybe a lawsuit for back taxes, too22:00
*** Tahmina has joined #openstack-keystone22:01
ayoungmorganfainberg, I'm having an issue with mox and keystoneclient calls in Django OpenStack Auth.  I'm trying to figure out what I really should be testing here22:02
ayounghttps://review.openstack.org/#/c/121281/1/openstack_auth/backend.py,cm  is the code I changed22:03
ayoungyou can see it is replacing client.Client(big long param list) calls with ones that just accept sessions22:04
morganfainbergayoung, first off sorry, mox, my condolences22:04
ayoungmorganfainberg, meh, it all sucks, just a matter of degree22:04
ayoungbut does it make sense to mox out the session and auth plugin calls?22:04
*** marcoemorais has quit IRC22:05
morganfainbergnot really sure there. it might be the only way to get it done22:05
morganfainbergbut ... honestly, don't know22:05
ayoungI guess we don't really want to run KC code, right?  We just care about running the Django Code22:05
*** marcoemorais has joined #openstack-keystone22:06
ayoungOK...I think I'm going to do this "ugly but passes"  and see what Horizon says22:06
*** marcoemorais has quit IRC22:06
*** marcoemorais has joined #openstack-keystone22:06
ayoungmorganfainberg, I don't know either.  To hell with this.  Let's go bowling.22:07
*** nkinder has quit IRC22:09
*** r-daneel__ has quit IRC22:09
*** soulxu_ has quit IRC22:11
*** soulxu_ has joined #openstack-keystone22:13
*** amerine_ has joined #openstack-keystone22:22
*** amerine has quit IRC22:24
ayoungmorganfainberg, dstanek can you guys shepherd this on through https://review.openstack.org/#/c/106838/  jamielennox|away is out getting all married and stuff and he's  basically blessed off on it anyway22:25
*** david-lyle has quit IRC22:26
*** henrynash has joined #openstack-keystone22:37
*** henrynash has quit IRC22:37
*** r-daneel__ has joined #openstack-keystone22:39
*** cjellick has quit IRC22:43
*** r-daneel__ has quit IRC22:44
*** morgan_remote_ has joined #openstack-keystone22:44
morgan_remote_Ayoung: I'll look over that lac change when I'm back from food / coffee22:44
ayoungmorgan_remote_, thanks22:44
ayoungmorgan_remote_, its pretty straight forward:  it deals with the case where there is no service catalog, since the user has no default project, and did not request a project.  This is the standard Horizon LDAP login case22:45
morgan_remote_I remember the discussion about it and I do like the concept.22:45
*** cjellick has joined #openstack-keystone22:50
*** r1chardj0n3s has joined #openstack-keystone22:52
*** r-daneel__ has joined #openstack-keystone22:56
*** nkinder has joined #openstack-keystone23:01
*** marcoemorais has quit IRC23:05
*** marcoemorais has joined #openstack-keystone23:05
*** dims has quit IRC23:06
*** dims has joined #openstack-keystone23:06
*** marcoemorais has quit IRC23:08
*** marcoemorais has joined #openstack-keystone23:08
*** marcoemorais has quit IRC23:08
*** marcoemorais has joined #openstack-keystone23:09
*** r1chardj0n3s is now known as r1chardj0n3s_afk23:09
*** dims has quit IRC23:11
*** richm has quit IRC23:13
*** cjellick_ has joined #openstack-keystone23:16
*** cjellick has quit IRC23:18
*** diegows has quit IRC23:19
*** achampion has joined #openstack-keystone23:21
*** ayoung has quit IRC23:21
*** sigmavirus24 is now known as sigmavirus24_awa23:22
*** gordc has quit IRC23:26
*** Tahmina has quit IRC23:31
*** r1chardj0n3s_afk is now known as r1chardj0n3s23:32
*** henrynash has joined #openstack-keystone23:34
*** henrynash has quit IRC23:35
*** marcoemorais has quit IRC23:36
*** marcoemorais has joined #openstack-keystone23:36
*** amerine_ has quit IRC23:37
*** amerine has joined #openstack-keystone23:38
*** Guest72739 is now known as mgagne23:42
*** mgagne has joined #openstack-keystone23:42
*** diegows has joined #openstack-keystone23:45
*** ekarlso has quit IRC23:48
*** ekarlso has joined #openstack-keystone23:48
*** rodrigods_ has joined #openstack-keystone23:50
*** cjellick_ has quit IRC23:50

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!