Wednesday, 2018-06-20

*** felipemonteiro has quit IRC00:01
*** felipemonteiro has joined #openstack-keystone00:04
*** zzzeek has quit IRC00:05
*** zzzeek has joined #openstack-keystone00:06
*** blake has quit IRC00:10
*** blake has joined #openstack-keystone00:12
*** felipemonteiro has quit IRC00:14
*** felipemonteiro has joined #openstack-keystone00:17
*** felipemonteiro has quit IRC00:17
*** zzzeek has quit IRC00:27
*** zzzeek has joined #openstack-keystone00:29
*** blake has quit IRC00:32
*** blake has joined #openstack-keystone00:38
*** felipemonteiro has joined #openstack-keystone00:41
*** Dinesh_Bhor has joined #openstack-keystone00:43
*** blake has quit IRC00:45
*** gyee has quit IRC01:02
*** felipemonteiro has quit IRC01:16
*** annp has joined #openstack-keystone01:18
*** felipemonteiro has joined #openstack-keystone01:31
*** mvenesio has quit IRC01:32
*** Dinesh_Bhor has quit IRC01:38
*** Dinesh_Bhor has joined #openstack-keystone01:44
wxylbragstad: sqlite doesn't support change primary key, that's the reason I re-create the table01:51
wxylbragstad: a similar case is here:
*** felipemonteiro has quit IRC01:56
wxykmalloc: lbragstad : If I understand correctly, trigger is used for sync the old data which is newly created during upgrading to the new schema. If drop the triggers, how to deal with this case?02:01
kmallocUse app-level logic to keep the data in sync for a release. Triggers are very hard to debug, are not well tested and could end up causing issues for the small number of cases that use them.02:04
*** germs has quit IRC02:05
kmallocSo, keystone just writes to both places for rocky, and in stien we drop support for the old way. But defer to lbragstad if we are doing that or triggers are acceptable.02:05
*** germs has joined #openstack-keystone02:05
*** germs has quit IRC02:05
*** germs has joined #openstack-keystone02:05
kmallocAnd contract happens in stein then.02:05
wxykmalloc: emm, that's a way, let the code deal with the mix version data02:05
kmallocThat is how I usually handle these cases, easier to test/get right. And not a shot in the dark for say pgsql (very under tested). And not fighting with sqlite to 'test' the code.02:07
kmallocBut again, I am not blocking triggers, I just prefer to not use them. Checking with lbragstad on the way forward is best. I'll roll with what he recommends.02:09
wxykmalloc: Ok, got it.02:10
wxykmalloc: and for schema change, adding PK is not allowed in sqlite. So I tried to re-create the table in my new PS.02:12
kmallocHm. That is annoying.02:12
wxyyeah, I tested the in-place way in my env which Lance wrote here ,all works well, then I upload the PS2 in this way, but the CI tells the sqlite doesn't like it. :(02:13
kmallocI can help write some sqlite specific code.02:14
kmallocIf needed.02:14
kmallocWe have done that in the past a few times, special case for upgrade test. We will get some better tests in gate. Iirc.02:15
*** lifeless_ has quit IRC02:15
*** germs has quit IRC02:16
*** germs has joined #openstack-keystone02:17
*** germs has quit IRC02:17
*** germs has joined #openstack-keystone02:17
*** namnh has joined #openstack-keystone02:17
wxykmalloc: so you prefer to use the in-place way with specific sqlite related code?02:18
kmallocThat is my preference, but I can't really impose my view here as the way forward if the general consensus is "use triggers"02:21
openstackgerritMerged openstack/oslo.policy master: Add examples and clarification around scope_types
wxykmalloc: OK, let's wait for others opinion as well. Thanks for your suggestion.02:22
openstackgerritsunguangning proposed openstack/oslo.policy master: Remove some description from oslo policy
*** zzzeek has quit IRC03:00
*** sonuk has joined #openstack-keystone03:01
*** zzzeek has joined #openstack-keystone03:03
*** lifeless has joined #openstack-keystone03:18
*** links has joined #openstack-keystone03:23
*** hrybacki has quit IRC03:40
*** wlmbasson has quit IRC03:41
*** mnaser has quit IRC03:42
*** hrybacki has joined #openstack-keystone03:42
*** samueldmq has quit IRC03:43
*** gmann has quit IRC03:44
*** wlmbasson has joined #openstack-keystone03:45
*** mnaser has joined #openstack-keystone03:45
*** ykarel has joined #openstack-keystone03:46
*** gmann has joined #openstack-keystone03:46
*** samueldmq has joined #openstack-keystone03:46
*** wolsen has quit IRC03:47
*** zhongjun_ has quit IRC03:48
*** mwhahaha has quit IRC03:48
*** robcresswell has quit IRC03:49
*** lamt has quit IRC03:49
*** jamespage has quit IRC03:49
*** portdirect has quit IRC03:50
*** pas-ha has quit IRC03:50
*** hrybacki has quit IRC03:50
*** yikun has quit IRC03:50
*** wxy has quit IRC03:50
*** hogepodge has quit IRC03:50
*** awestin1 has quit IRC03:50
*** wlmbasson has quit IRC03:50
*** samueldmq has quit IRC03:51
*** tommylikehu has quit IRC03:51
*** NobodyCam has quit IRC03:51
*** ildikov has quit IRC03:51
*** mnaser has quit IRC03:51
*** gmann has quit IRC03:51
*** kmalloc has quit IRC03:51
*** betherly has quit IRC03:52
adriantout of curiosity, are they any upstream tools for testing custom policy?03:58
*** germs has quit IRC04:06
openstackgerritChason Chan proposed openstack/python-keystoneclient master: Update IdentityProviderManager docstring
*** felipemonteiro has joined #openstack-keystone04:20
*** openstack has joined #openstack-keystone04:29
*** ChanServ sets mode: +o openstack04:29
*** markvoelker has quit IRC04:45
*** lifeless_ has joined #openstack-keystone04:54
*** lifeless has quit IRC04:55
*** nicolasbock has joined #openstack-keystone04:56
*** masber has quit IRC04:59
*** felipemonteiro has quit IRC05:04
*** hoonetorg has quit IRC05:09
*** hoonetorg has joined #openstack-keystone05:10
*** zhongjun_ has joined #openstack-keystone05:21
*** ildikov has joined #openstack-keystone05:24
*** wlmbasson has joined #openstack-keystone05:24
*** lamt has joined #openstack-keystone05:25
*** lamt is now known as Guest8154005:26
*** NobodyCam has joined #openstack-keystone05:27
*** mwhahaha has joined #openstack-keystone05:28
*** yikun has joined #openstack-keystone05:29
*** jamespage has joined #openstack-keystone05:29
*** hrybacki has joined #openstack-keystone05:30
*** wxy has joined #openstack-keystone05:32
*** pas-ha has joined #openstack-keystone05:32
*** betherly_ has joined #openstack-keystone05:34
*** awestin1 has joined #openstack-keystone05:34
*** quiquell|off is now known as quiquell|rover05:34
*** kmalloc has joined #openstack-keystone05:40
*** gmann has joined #openstack-keystone05:40
*** mnaser has joined #openstack-keystone05:42
*** mnaser has quit IRC05:51
*** wlmbasson has quit IRC05:51
*** robcresswell has joined #openstack-keystone05:53
*** yikun has quit IRC05:55
*** gmann has quit IRC05:55
*** hrybacki has quit IRC05:55
*** mwhahaha has quit IRC05:56
*** kmalloc has quit IRC05:56
*** Guest81540 has quit IRC05:57
*** jamespage has quit IRC05:57
*** zhongjun_ has quit IRC05:57
*** ildikov has quit IRC05:57
*** awestin1 has quit IRC05:57
*** NobodyCam has quit IRC05:57
*** robcresswell has quit IRC05:58
*** pas-ha has quit IRC05:58
*** wxy has quit IRC05:58
*** betherly_ has quit IRC05:58
*** dims has quit IRC06:09
*** dims has joined #openstack-keystone06:10
*** mnaser has joined #openstack-keystone06:10
*** gmann has joined #openstack-keystone06:12
*** dims has quit IRC06:16
*** dims has joined #openstack-keystone06:17
*** hrybacki has joined #openstack-keystone06:17
*** wlmbasson has joined #openstack-keystone06:17
*** samueldmq has joined #openstack-keystone06:19
*** wxy has joined #openstack-keystone06:20
*** Guest81540 has joined #openstack-keystone06:21
*** yikun has joined #openstack-keystone06:21
*** NobodyCam has joined #openstack-keystone06:21
*** pas-ha has joined #openstack-keystone06:21
*** jamespage has joined #openstack-keystone06:21
*** jamespage has quit IRC06:21
*** jamespage has joined #openstack-keystone06:21
*** portdirect has joined #openstack-keystone06:21
*** NobodyCam has quit IRC06:21
*** NobodyCam has joined #openstack-keystone06:21
*** betherly_ has joined #openstack-keystone06:21
*** jamespage has quit IRC06:21
*** jamespage has joined #openstack-keystone06:21
*** portdirect has quit IRC06:21
*** portdirect has joined #openstack-keystone06:21
*** pas-ha has quit IRC06:22
*** pas-ha has joined #openstack-keystone06:22
*** jamespage has quit IRC06:22
*** jamespage has joined #openstack-keystone06:22
*** zhongjun_ has joined #openstack-keystone06:23
*** kmalloc has joined #openstack-keystone06:23
*** awestin1 has joined #openstack-keystone06:23
*** d34dh0r53 has quit IRC06:24
*** d34dh0r53 has joined #openstack-keystone06:24
*** mwhahaha has joined #openstack-keystone06:25
*** robcresswell has joined #openstack-keystone06:26
*** ildikov has joined #openstack-keystone06:26
*** ykarel_ has joined #openstack-keystone06:33
*** sonuk has quit IRC06:34
*** ykarel has quit IRC06:36
*** ykarel__ has joined #openstack-keystone06:42
*** ykarel_ has quit IRC06:42
*** ykarel__ is now known as ykarel06:43
*** markvoelker has joined #openstack-keystone06:46
*** Dinesh_Bhor has quit IRC06:48
*** Dinesh_Bhor has joined #openstack-keystone06:49
*** tommylikehu has joined #openstack-keystone06:53
*** wolsen has joined #openstack-keystone06:55
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement base for new RBAC Enforcer
*** quiquell|rover is now known as quique|rover|afk07:00
*** hogepodge has joined #openstack-keystone07:00
*** martinus__ has joined #openstack-keystone07:01
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement base for new RBAC Enforcer
*** ispp has joined #openstack-keystone07:03
*** tesseract has joined #openstack-keystone07:04
*** ispp has quit IRC07:18
*** markvoelker has quit IRC07:21
*** amoralej|off is now known as amoralej07:21
openstackgerritMorgan Fainberg proposed openstack/keystone master: Add Flask-RESTful as a requirement
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement scaffolding for Flask-RESTful use
openstackgerritMorgan Fainberg proposed openstack/keystone master: Keystone adheres to public_endpoint opt only
openstackgerritMorgan Fainberg proposed openstack/keystone master: Convert json_home and version discovery to Flask
openstackgerritMorgan Fainberg proposed openstack/keystone master: Add support for before and after request functions
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement base for new RBAC Enforcer
*** tosky has joined #openstack-keystone07:30
*** sonuk has joined #openstack-keystone07:34
*** quique|rover|afk is now known as quiquell|rover07:36
*** AlexeyAbashkin has joined #openstack-keystone07:49
openstackgerritwangxiyuan proposed openstack/keystone master: Strict two level hierarchical limit
*** nicolasbock has quit IRC07:58
*** rcernin has quit IRC08:03
*** jistr is now known as jistr|mtg08:07
*** ispp has joined #openstack-keystone08:08
*** peereb has joined #openstack-keystone08:09
*** pcaruana has joined #openstack-keystone08:11
*** peereb has quit IRC08:14
*** peereb has joined #openstack-keystone08:15
*** peereb has quit IRC08:16
*** peereb has joined #openstack-keystone08:16
*** ykarel_ has joined #openstack-keystone08:17
*** peereb has quit IRC08:17
*** markvoelker has joined #openstack-keystone08:18
*** peereb has joined #openstack-keystone08:18
*** peereb has quit IRC08:19
*** peereb has joined #openstack-keystone08:19
*** ykarel has quit IRC08:20
*** peereb has quit IRC08:20
*** peereb has joined #openstack-keystone08:21
*** peereb has quit IRC08:21
*** pcichy has quit IRC08:29
*** s10 has joined #openstack-keystone08:35
*** nicolasbock has joined #openstack-keystone08:41
*** ykarel_ is now known as ykarel|lunch08:48
*** markvoelker has quit IRC08:52
*** rcernin has joined #openstack-keystone08:54
*** s10 has quit IRC09:04
*** ykarel_ has joined #openstack-keystone09:11
*** jistr|mtg is now known as jistr09:12
*** ykarel|lunch has quit IRC09:13
*** lifeless_ has quit IRC09:27
*** lifeless has joined #openstack-keystone09:29
*** ykarel_ has quit IRC09:35
*** ykarel_ has joined #openstack-keystone09:35
*** deepak_mourya has joined #openstack-keystone09:40
*** aojea_ has joined #openstack-keystone09:42
*** Dinesh_Bhor has quit IRC09:46
*** aojea_ has quit IRC09:47
*** namnh has quit IRC09:49
deepak_mouryahi, in this bug  what exactly we need to do?09:54
openstackLaunchpad bug 1777671 in OpenStack Identity (keystone) "Incorrect use of translation _()" [Medium,Triaged] - Assigned to Deepak Mourya (mourya007)09:54
*** rcernin has quit IRC09:56
*** annp has quit IRC09:58
*** ykarel__ has joined #openstack-keystone09:59
*** ykarel_ has quit IRC10:02
*** ykarel__ is now known as ykarel10:04
*** cristicalin has joined #openstack-keystone10:18
cmurphydeepak_mourya: here's an example of what needs to be fixed:
cmurphythe string is being marked for translation with _() and then being passed to both the LOG and the exception10:26
cmurphybut we don't actually want to have the string for the LOG translated, only for the exception10:26
cmurphyso it should change to something like msg = 'Domain name cannot contain reserved characters.' ; LOG.warning(msg) ; raise exception.Unauthorized(message=_(msg))10:27
deepak_mouryacmurphy: ok got it now10:28
deepak_mouryaThanks for the reply10:28
cmurphyno problem10:28
*** cristicalin has quit IRC10:37
*** markvoelker has joined #openstack-keystone10:48
*** cristicalin has joined #openstack-keystone10:48
*** cristicalin has quit IRC10:53
*** jaosorior has quit IRC10:56
*** belmoreira has joined #openstack-keystone10:57
*** quiquell|rover is now known as quiquell|rover|b11:03
*** quiquell|rover|b is now known as quique|rover|bbl11:03
*** quique|rover|bbl has quit IRC11:09
*** cristicalin has joined #openstack-keystone11:20
*** amoralej is now known as amoralej|out11:20
*** raildo has joined #openstack-keystone11:21
*** markvoelker has quit IRC11:22
*** cristicalin has quit IRC11:25
*** jaosorior has joined #openstack-keystone11:36
*** cristicalin has joined #openstack-keystone11:51
*** cristicalin has quit IRC11:56
*** germs has joined #openstack-keystone12:09
*** sonuk has quit IRC12:09
*** cristicalin has joined #openstack-keystone12:11
*** germs has quit IRC12:14
*** ykarel_ has joined #openstack-keystone12:17
*** markvoelker has joined #openstack-keystone12:19
*** ykarel has quit IRC12:19
*** markvoelker has quit IRC12:22
*** markvoelker has joined #openstack-keystone12:22
*** cristicalin has quit IRC12:29
*** kman has joined #openstack-keystone12:29
*** zhongjun_ has quit IRC12:29
*** kman has quit IRC12:38
*** ykarel_ is now known as ykarel12:50
*** edmondsw has joined #openstack-keystone12:52
*** ispp has quit IRC13:07
*** amoralej|out is now known as amoralej13:07
*** ispp has joined #openstack-keystone13:10
lbragstadthis is a good documentation patch if anyone is interested
*** jaosorior has quit IRC13:42
kmalloccmurphy: :)13:43
*** ykarel has quit IRC13:44
cmurphykmalloc: sup13:44
*** ykarel has joined #openstack-keystone13:44
kmallocgood morning!13:45
cmurphygood afternoon!13:46
kmallocor evening... or whatever it is wherever you are13:46
* kmalloc is pre-coffee.13:46
kmalloclbragstad: i think i've done a reasonable job breaking down @protected and what we're extracting so a proper .enforce_call can be made.
kmalloclbragstad: it's not done, but it's on it's way.13:47
lbragstadsounds good13:48
kmallocayoung, adriant: ^ cc, just because i know you tried to take a stab at diving into @protected as well13:48
lbragstadi can start looking at that today or tomorrow13:48
kmallocthis is very flask-specific.13:48
kmallocyeah just a "hey does this make sense" pass13:48
kmallocis fine, because if it looks better than @protected, i've done something right.13:48
*** PsionTheory has joined #openstack-keystone13:49
lbragstadyeah - that's my main goal13:49
lbragstadif we can remove @protected in favor of something that puts the authorization logic closer to business code13:49
lbragstador makes authorization logic more clean/clear i think that'll be a big win13:50
lbragstadwhich will also be super handy for the default roles + scope types work13:52
kmallocthe docstrings need further expansion too.13:52
kmallocand we can add another wrapper syntactic sugar-style to it on top of enforce_call13:52
kmallocbut i am feeling much better about the enforcer having spent a ton of time diving into @protected and trying to understand the dense craziness.13:53
lbragstadyeah - it's intense13:54
*** ykarel is now known as ykarel|afk13:56
*** david-lyle has joined #openstack-keystone13:57
*** dklyle has quit IRC13:57
*** ispp has quit IRC13:58
*** ispp has joined #openstack-keystone14:01
*** ispp has quit IRC14:01
*** ispp has joined #openstack-keystone14:01
lbragstadgoing back to the database migration discussions we were having yesterday14:07
lbragstadi _think_ we'll need three migrations14:07
lbragstad1. for auto-incrementing primary keys in registered limits14:07
lbragstad2. for auto-incrementing primary keys in limits14:07
lbragstad3. for reducing duplicate data between limit and registered limit tables14:08
lbragstadi think we're at a point with the notes in that we can probably move them to bugs instead14:10
*** germs has joined #openstack-keystone14:10
*** germs has quit IRC14:10
*** germs has joined #openstack-keystone14:10
*** ykarel|afk is now known as ykarel14:13
*** ayoung has quit IRC14:14
*** spilla has joined #openstack-keystone14:15
*** germs has quit IRC14:16
openstackLaunchpad bug 1777892 in OpenStack Identity (keystone) "Reduce duplicate data between unified limit tables" [Medium,Triaged]14:23
*** ayoung has joined #openstack-keystone14:30
openstackLaunchpad bug 1777893 in OpenStack Identity (keystone) "Limit and registered limit tables should auto-increment primary keys" [Medium,Triaged]14:30
lbragstadcmurphy: would i be able to get your eyes on whenever you have a minute?14:40
cmurphylbragstad: looking14:40
lbragstadit should be all squared away per your last set of comments14:41
*** Guest81540 is now known as lamt14:46
cmurphylbragstad: lgtm!14:48
lbragstadthanks cmurphy14:48
*** ispp has quit IRC14:55
*** david-lyle has quit IRC14:56
*** dklyle has joined #openstack-keystone15:01
*** ispp has joined #openstack-keystone15:06
*** felipemonteiro has joined #openstack-keystone15:11
kmalloclbragstad: i also advised wxy to confirm with you the direction we're going, trigger or not15:12
kmalloclbragstad: i will stand behind whichever is the end choice, but i've made my opinion clear15:12
lbragstadsure - it's a big part of the reason why i wanted to write down a couple of the approachs15:13
*** belmoreira has quit IRC15:13
lbragstadi'd like more feedback on it15:13
lbragstadand it's probably easier for people to parse if they have something they can look at15:13
lbragstadbut yeah... it's hard problem15:13
*** felipemonteiro has quit IRC15:14
*** germs has joined #openstack-keystone15:16
*** germs has quit IRC15:16
*** germs has joined #openstack-keystone15:16
*** germs has quit IRC15:16
*** felipemonteiro has joined #openstack-keystone15:17
*** felipemonteiro has quit IRC15:18
*** germs has joined #openstack-keystone15:19
*** germs has quit IRC15:19
*** germs has joined #openstack-keystone15:19
*** PsionTheory has quit IRC15:27
*** felipemonteiro has joined #openstack-keystone15:33
*** felipemonteiro has quit IRC15:34
*** belmoreira has joined #openstack-keystone15:37
*** felipemonteiro has joined #openstack-keystone15:38
*** belmoreira has quit IRC15:38
openstackgerritLance Bragstad proposed openstack/keystone master: Simplify the issue token code path
lbragstadkmalloc: ^15:40
*** links has quit IRC15:41
openstackgerritMorgan Fainberg proposed openstack/keystone master: Add support for before and after request functions
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement base for new RBAC Enforcer
kmalloclbragstad: ok and that should now be passing tests.15:43
kmallocthe enforcer is not done, but it's at least got parity with today15:43
kmallocprobably another hour of coding and then an hour of test writing [might spin the tests up in a followup for the new enforcer] just to keep reviewability (too much code at once is hard)15:44
kmalloclbragstad: i knew flask was going to be a rabbit hole... but FFS :P15:44
kmalloconce the enforcer is ready i'll be able to start moving apis to keystone.api15:45
lbragstadthat last patch i pushed is in merge conflict, but i should have a cleaned up version here in a minute...15:58
kmallocnp, i need to run for some errands, be back around noon15:59
*** r-daneel has joined #openstack-keystone16:00
*** felipemonteiro has quit IRC16:09
*** felipemonteiro has joined #openstack-keystone16:12
*** felipemonteiro has quit IRC16:14
*** ispp has quit IRC16:15
*** felipemonteiro has joined #openstack-keystone16:15
openstackgerritLance Bragstad proposed openstack/keystone master: Introduce new TokenModel object
openstackgerritLance Bragstad proposed openstack/keystone master: Simplify the issue token code path
lbragstadhad to wipe the +2 off of ^ :(16:16
*** ayoung has quit IRC16:17
*** gyee has joined #openstack-keystone16:21
* lbragstad goes for a run16:21
*** ykarel is now known as ykarel|away16:24
*** tesseract has quit IRC16:34
*** felipemonteiro has quit IRC16:38
*** felipemonteiro has joined #openstack-keystone16:40
openstackgerritMerged openstack/keystone master: Api-ref: Refresh the Update APIs for limits
*** felipemonteiro has quit IRC16:54
*** felipemonteiro has joined #openstack-keystone17:01
*** felipemonteiro has quit IRC17:04
*** felipemonteiro has joined #openstack-keystone17:07
*** felipemonteiro has quit IRC17:17
*** AlexeyAbashkin has quit IRC17:23
*** ykarel|away has quit IRC17:25
*** amoralej is now known as amoralej|off17:27
*** rmascena has joined #openstack-keystone17:28
*** raildo has quit IRC17:32
*** links has joined #openstack-keystone17:37
*** links has quit IRC17:39
*** links has joined #openstack-keystone17:39
*** fiddletwix has joined #openstack-keystone17:40
*** SpamapS has quit IRC17:46
*** felipemonteiro has joined #openstack-keystone18:07
*** links has quit IRC18:11
*** links has joined #openstack-keystone18:12
*** links has quit IRC18:14
openstackgerritPavlo Shchelokovskyy proposed openstack/keystone master: Filter by entity_type in get_domain_mapping_list
*** felipemonteiro has quit IRC18:22
*** felipemonteiro has joined #openstack-keystone18:23
*** felipemonteiro has quit IRC18:26
*** felipemonteiro has joined #openstack-keystone18:26
*** itlinux has joined #openstack-keystone18:27
*** germs has quit IRC18:30
*** germs has joined #openstack-keystone18:30
*** germs has quit IRC18:30
*** germs has joined #openstack-keystone18:31
*** germs has quit IRC18:31
*** germs has joined #openstack-keystone18:31
*** r-daneel has quit IRC18:48
*** r-daneel has joined #openstack-keystone18:51
lbragstadi'm noticing something super weird with caching18:51
lbragstadi have a token model handler that serializes token objects to dictionary before caching them18:53
lbragstadand then it deserializes the data back to token model objects on cache hits18:54
lbragstadi can confirm that a token is getting serialized, which means it's getting put in cache18:54
lbragstadbut when it is deserialized, bit'18:54
lbragstadit only executes like halfway through the deserialization18:55
ildikovknikolla: hi18:56
ildikovknikolla: I read through the spec you linked in yesterday quickly for the Devstack plugin and test work18:56
ildikovknikolla: is it tracked anywhere what's done and what's in flight/todo?18:56
*** lifeless has quit IRC19:00
*** aojea_ has joined #openstack-keystone19:03
kmalloclbragstad: back19:04
kmalloclbragstad: this the context cache?19:07
kmalloclbragstad: or the main cache?19:07
kmalloclbragstad: it might need a msgpack deserializer19:08
kmalloclbragstad: can you post what you have and i'll take a look19:09
*** r-daneel_ has joined #openstack-keystone19:11
*** r-daneel has quit IRC19:11
*** r-daneel_ is now known as r-daneel19:11
lbragstadi'll post a wip of what i have19:11
*** aojea_ has quit IRC19:16
lbragstadok - these are the changes i've made
lbragstadthis is the failure with logging -
*** rmascena is now known as raildo19:30
openstackgerritMorgan Fainberg proposed openstack/keystone master: Implement base for new RBAC Enforcer
kmalloclbragstad: ^ fyi, code complete, needs tests.19:34
kmalloclooking at your issue now19:34
lbragstadfyi - this is the test case that it's failing on
kmallocinteresting:     Traceback (most recent call last):19:36
kmalloc      File "keystone/token/", line 170, in _is_valid_token19:36
kmalloc        token_data = token.get('token', token.get('access'))19:36
kmalloc    AttributeError: 'TokenModel' object has no attribute 'get'19:36
lbragstadright - did you see the handler code?19:36
kmallocit's making an assumption you're dealing with a dict.19:36
lbragstadthe authentication code it using the token model19:37
lbragstadthe validation code is using the token reference (the old way)19:37
kmallocbrb, dog needs to not explode inside19:38
lbragstadso - technically the token validation code assuming that's a dictionary is correct (for now)19:38
lbragstadnow worries19:38
*** Deknos has joined #openstack-keystone19:39
kmallocthe log doesn't ever show deserializing19:41
lbragstadweird, right?19:41
kmallocdog wants to play "chase me" instead of "go out"19:41
lbragstadeven though it says it's been serialized and whatnot19:41
kmallocso, nope. not chasing a dog around.19:41
*** Deknos has left #openstack-keystone19:41
lbragstadthat sounds like a fun game19:42
*** lifeless has joined #openstack-keystone19:49
kmallocnear dogsplosion19:56
kmallocok back19:56
*** spilla has quit IRC19:56
kmalloclbragstad: uhm20:06
lbragstadweird, right?20:07
kmalloclbragstad: so, humor me...20:07
kmalloci think you're never hitting a deserialization event20:07
lbragstadi would agree20:07
lbragstadit's never actually getting to that method20:07
lbragstadin _TokenModelHandler20:07
kmallocyou're failing before you hit deserialize20:08
kmallocin         self.get('/auth/tokens', token=admin_token,20:08
kmalloc                 headers={'X-Subject-Token': user_token})20:08
kmallocyou've only requested each token a single time until that point20:08
*** r-daneel has quit IRC20:08
kmallocthe context cache wont deserialize unless you get into "get" token.20:09
lbragstadright - that makes sense20:09
kmallocyou're not getting far enough for the context cache to work, so, caching is not even involved yet20:09
*** r-daneel has joined #openstack-keystone20:09
*** boris_42_ has joined #openstack-keystone20:10
kmalloc    Traceback (most recent call last):20:10
kmalloc      File "keystone/token/", line 170, in _is_valid_token20:10
kmalloc        token_data = token.get('token', token.get('access'))20:10
kmalloc    AttributeError: 'TokenModel' object has no attribute 'get'20:10
kmallocthat is before you get to the deserialize point [somehow]20:10
kmallocUnexpected error or malformed token determining token expiry: <TokenModel (audit_id=YPE8qN_qTTuTdAOC-djoMA, audit_chain_id=[u'YPE8qN_qTTuTdAOC-djoMA']) at 0x7fb9a2c5f610>20:11
lbragstadare we validating a freshly issued token?20:11
kmallocwe haven't validated a token at all20:11
kmallocjust issued20:11
lbragstadright - if what you're saying is true20:11
lbragstadwe haven't issued the user's token back to them yet...20:12
kmalloceven if we had20:12
kmalloccontext cache is memoization20:12
kmallocmeaning it is specific to the validate call20:12
kmallocif you don't call "validate" we aren't caching20:12
lbragstadwe cache tokens on issue20:12
kmallocthen our on-issue cache may be wonky20:13
lbragstadit was a thing amakarov implemented a while back20:13
kmallocyeah... that doesn't look quite right to me20:14
kmalloc            self._validate_token.set(token_data, TOKENS_REGION, token_id)20:14
kmallocyeah that isn't caching anything useful20:14
kmallocit's setting the cache key to the TOKENS_REGION20:14
kmallocwhich is a bogus cache-key20:15
kmallocbasically that code just wastes memory20:15
kmallocboth in memcache and in local context20:15
lbragstadbecause it stuff things in that can't result in hits?20:15
kmallocbecause TOKENS_REGION object isn't a valid cache key20:16
kmallocnothing would ever look that up20:16
kmallocthere is a reason we typically don't use .set()20:16
lbragstadwhat's the method signature for set?20:16
kmallocyou have to generate the cache-key with args that look like what _validate would be called with20:16
lbragstadwhich *should* be the token id20:17
kmalloc.set(self, key, value)20:17
kmallocwhere the key is a mangled set of "method, args, etc" run through a sha120:17
lbragstadoh - it doesn't look like we're doing that...20:17
kmalloclet me confirm, it *may* do some cache-key work20:18
kmallocbut it for-sure doesn't work with TOKENS_REGION20:18
kmallocas the value20:18
lbragstadif you're right about the method signature20:19
lbragstadshouldn't it be20:19
lbragstadself._validate_token.set(TOKEN_REGION,, token)20:19
kmallocso maybe it's right...20:20
kmallocbut... hold on20:21
openstackgerritMerged openstack/python-keystoneclient master: Add support for registered limits
kmallocno, it likely should be  self._validate_token.set(token, self,
kmallocsorry my dogpile foo is a little rusty20:22
kmalloc        if MEMOIZE.should_cache(ret):20:23
kmalloc            self.get_project.set(ret, self, project_id)20:23
kmalloc            self.get_project_by_name.set(ret, self, ret['name'],20:23
kmalloc                                         ret['domain_id'])20:23
kmalloc        return ret20:23
kmallocthat is an example20:23
kmallocswaping TOKEN_REGION for self should fix that20:23
kmallocand get you deserializing and actually getting pre-seeded caches20:23
kmallocright now every single token issued simply caches in the same key20:23
kmallocover and over and over20:24
lbragstadlet me give that a shot quick20:24
lbragstadand that sounds like a bug20:24
kmallocyeah it is a bug20:24
kmallocand proof that this code was never actually tested20:24
lbragstadwhich would pretty much negate the enitre benefit of that feature20:24
kmalloctesting the cache is *hard*20:24
kmallocthere is a reason very few of us tend to write cache code.20:24
kmallocwhich reminds me, i need to unwind the broken config thing soon20:25
kmallocwill do that in a few.20:25
kmallocok let me look at the blame... i think we never had a test implemented for caching code20:27
kmalloci think that needs to be a rule, cache code MUST always have expanded testing20:27
lbragstadwe're still not hitting the deserialization20:27
lbragstadtest changes -
*** aojea has joined #openstack-keystone20:31
lbragstadnew logs -
lbragstadhuh - so it is blowing up in the GET /v3/auth/tokens call on the admin token20:33
openstackgerritMerged openstack/python-keystoneclient master: Add support for project-specific limits
kmallocand it's still not deserializing20:34
kmallocI don't think it's even getting to .validate20:34
kmalloci don't see a "missed" anywhere in your log20:35
lbragstadnope - because it's hitting the cache20:35
lbragstadbut not deserializing20:35
*** martinus__ has quit IRC20:35
*** felipemonteiro has quit IRC20:35
kmallocis it hitting the cache?20:35
lbragstadit has to be20:35
kmallocdo me a favor, lets do some exception debugging.20:36
kmallocadd in an explicit get20:36
kmallocand pprint that20:36
lbragstadwhere do you want that?20:37
kmallocright after the set20:37
kmalloclet's compare the results20:37
kmalloccompared to token/token_data and the return of .get()20:38
lbragstad^ changes20:39
kmallocwell that clearly shows bugs in the deserializing code20:40
lbragstadit's failing because i did something wrong in deserialize20:40
kmallocthat is a start.20:40
lbragstadso - that proves something20:40
lbragstadwhich is that it's getting set in cache20:40
kmallocthe next thing to try is: call ._validate directly and compare .get() and ._validate responses20:40
lbragstadwith self._validate_token.set(token, self,
kmalloconce you have deserialize working20:40
*** pcaruana has quit IRC20:41
kmallocyou should write a test for the handler20:43
lbragstadfixed deserialization20:43
kmallocthat just does serialize/deserialize of a rendered token20:43
kmallocto ensure changes don't break it20:43
kmallociirc i did that with the revoke handler20:43
kmallocand that is just doing .get() then ._validate( ?20:44
lbragstadyeah - it's just calling .get() right after it manually sets the token on the _validate_token() method20:44
kmalloci notice two deserializations now20:45
lbragstadyeah - because the test is authenticating for two tokens20:45
lbragstadthe admin_token and the user_token20:45
kmallocah right.20:45
lbragstadbut the main issue still exists (where TokenModel is somehow getting in the mix in the validate token path)20:45
kmallocok now right below the .get call self._validate(token_id)20:45
kmallocand see if it hits the cache20:46
kmallocwe can also enable cache-debugging (and show the generated keys)20:46
lbragstaddeserialized twice, one for each token20:47
lbragstadso self._validate_token( is working20:47
kmallocthats good news(tm)20:48
kmallocthat clearly means we're not populating bad cache now20:49
kmallocok, but we're still failing.20:49
lbragstadbecause "somehow" validate is getting a TokenModel when it should be getting a dictionary20:49
lbragstadwhich is still blowing my mind...20:50
kmallocand it's def. not cache related [or well, not "context-cache/validate cache"]20:50
kmalloclet me see the whole diff again?20:50
*** raildo has quit IRC20:53
*** markguz has joined #openstack-keystone20:56
lbragstadnote that diff is on top of
markguzhi. Just updated to Ocata from Newton, auth stopped working and seeing this error in the logs TypeError: __call__() got an unexpected keyword argument 'default_config_dirs'20:58
markguzanyone seen that before?20:58
markguzcant find any reference to default_config_dirs in any config files20:58
kmalloclbragstad: so, ._validate is in-fact returning a tokenmodel now20:59
kmalloclbragstad: and you're erroring in .is_valid_token20:59
lbragstadmarkguz: do you have a whole trace?20:59
kmalloclbragstad: fix is_valid token, the pprint for deserialization may just be getting lost in a flush.21:00
kmalloclbragstad: my typical view on caching is also: disable caching and see if it works first21:01
kmalloconce that works, enable caching again21:01
kmallocmarkguz: that sounds like some code mismatch of some sort.21:03
kmallocmarkguz: how was the upgrade performed? [out of curiosity]21:03
kmallocbecause default_config_dirs was an option added somewhere along the line.21:05
kmallocit's like the option is being passed to an older [unaware] version of keystone21:06
markguzkmalloc: only one keystone running21:07
kmallocdid keystone properly shutdown before the upgrade?21:08
kmalloci could see something being weird if some code was still running in mod_wsgi.21:08
kmalloclbragstad: i've never seen that error before.21:08
lbragstadme either21:09
cmurphydefault_config_dirs was added to oslo.confg in ocata so you need to make sure oslo.config is up to date21:09
markguzcmurphy: i need to add that option?21:10
lbragstadjust upgrade oslo.config?21:10
kmallocthat could do it. thanks cmurphy21:10
cmurphymarkguz: no, you need to make sure the oslo.config package is on ocata21:10
lbragstadyeah no kidding, good call cmurphy21:10
cmurphyversion 3.20.0 at least it looks like21:10
markguzyeah that would do it21:10
kmalloclbragstad: yeah might be that oslo.config package was out of date.21:10
kmallocmarkguz: cmurphy swoops in and saves the day. it's her super power :)21:11
kmalloc(well one of them)21:11
markguzyeah that was not updated.  think rdo need to put that in the update page21:11
kmallochehe, or make their keystone package depend on the minimum21:11
markguzkmalloc: yes21:12
kmallocsounds to me like a bad rpm that doesn't know the minimum oslo.config needed21:12
lbragstadthe minimum we define upstream is 3.1421:12
lbragstadat least in stable ocata21:12
kmallocoooh wonderful.21:12
kmallocthat might be a g-r bug then21:13
lbragstadkmalloc: fwiw - that issues goes away when i disable keystone.conf [token] cache_on_issue and keystone.conf [cache] enabled21:13
kmalloclbragstad: ok that is interesting.21:13
kmallocmeans it *is* cache related21:13
kmallocgood to know21:13
kmallocsomehow with cach...21:13
kmallocoh wait a sec.21:13
kmallochooooooollllld the door... hodor!21:14
kmalloci mean...21:14
kmalloclbragstad: you didn;t update the validate pipeline to use the toknemodel did you?21:14
kmallocjust the issue one?21:14
kmalloclbragstad: you're somehow getting a dict back when you don't pre-seed the cache?21:16
kmallocor when you don't cache at all21:16
kmallocvia validate21:16
* lbragstad back in 521:17
*** rledisez has quit IRC21:24
*** lifeless has quit IRC21:46
*** lifeless has joined #openstack-keystone21:47
*** r-daneel has quit IRC21:49
*** nicolasbock has quit IRC21:52
* lbragstad was bombarded21:52
*** itlinux has quit IRC21:52
lbragstadcorrect - only the issue token patch was updated to use the token model object21:53
lbragstadthe validate path still builds a dictionary21:53
kmallocand there is why you're failing.21:53
lbragstadusing all the old way of doing things we're used to21:53
kmallocbecause issue pre-seeds in the cache of the model21:53
kmalloci wonder if the deserialize pprint is just lost in a flush due to the app bailing21:53
lbragstadcheckout the last couple lines of the deserialize method though21:53
lbragstadi'm converting the token model back to a dictionary21:54
kmallocdon't do that.21:54
lbragstad(because i'm doing the token model work in two patches, one for token issuance and one for token validation)21:54
lbragstadi can squash them21:54
lbragstadbut i'm not sure if we're covering up a cache problem?21:55
kmallocdeserialize should rehydrate to the same state21:55
kmallocyou should ensure calls to validate convert -> dict21:55
kmallocif needed21:55
kmallocbasically you need a "if isTokenModeel: token.to_dict()21:55
kmallocfor testing21:56
kmallocif you turn off cache_on_issue21:56
kmallocthe problem also goes away, yah?21:56
kmallocbasically until both issue and validate emit TokenModel you shouldn't lean on cache_on_issue21:56
kmallocit is a recipe for errors.21:57
lbragstadso - smash into
* kmalloc waits for loading...21:57
lbragstadi'd like to make sure cache_on_issue always works21:57
kmallocright, so you have to make sure issue and validate both do tokenmodel21:59
kmallocin a single patch21:59
kmalloc... also, i can't load review.openstack.org21:59
kmallocchanging fundamental format *OR* you need to make a dict-interface for the tokenmodel for compat until everything is converted22:02
kmallocboth are ok22:02
kmalloci probably would do the dict-compat interface22:02
kmalloc[basically, behind the scenes build the token_dict and setup a __getattr__ to reference it]22:02
kmallocand then delete that interface once everything is converted22:03
kmallocmeans for less code change in one swoop22:03
*** spzala has joined #openstack-keystone22:04
*** spzala has quit IRC22:04
lbragstadbut if we do that, we aren't reinflating to a the same thing?22:05
*** edmondsw has quit IRC22:07
lbragstadi see what you mean22:08
*** edmondsw has joined #openstack-keystone22:10
*** rcernin has joined #openstack-keystone22:10
kmallocbasically keep a dict state of the token in all cases on like "tokenmodel.__dictstate" and make .__getitem__ on TokenModel just reference TokenModel.__dictstate.__setitem__22:12
kmallocerm... TokenModel.__dictstate.__getitem__22:14
*** edmondsw has quit IRC22:14
lbragstadthat'd be one option - or we use the big hammer and make issue token and validation token work with TokenModel22:15
kmallocit's ugly but can make it so anything that does Token[<thing>] can work until it's converted to know TokenModel.thing22:15
kmallocit's up to you22:15
kmallocboth will do the job22:15
* lbragstad assess risk22:16
*** r-daneel has joined #openstack-keystone22:34
*** aojea has quit IRC22:36
*** rcernin has quit IRC22:50
*** r-daneel has quit IRC23:15
*** rledisez has joined #openstack-keystone23:17
*** boris_42_ has quit IRC23:19
adriantcmurphy: thanks, will take a look at it!23:24
adriantcmurphy: any clue if Patrole works with older versions of openstack services?23:36
adriantalthough I guess in my case the requirements_authority part is all I need and that's just parsing policy files vs requirements23:39
*** tosky has quit IRC23:42
*** rcernin has joined #openstack-keystone23:43
openstackgerritMerged openstack/keystone master: Clarify scope responses in authentication api ref

Generated by 2.15.3 by Marius Gedminas - find it at!