Thursday, 2018-03-22

*** odyssey4me has quit IRC00:02
*** odyssey4me has joined #openstack-keystone00:03
*** felipemonteiro__ has joined #openstack-keystone00:07
*** oikiki has quit IRC00:07
*** Guest85132 has joined #openstack-keystone00:08
*** Guest85132 has quit IRC00:13
*** Krenair_ has joined #openstack-keystone00:21
*** mvk has joined #openstack-keystone00:27
*** Dinesh_Bhor has joined #openstack-keystone00:36
*** Dinesh_Bhor has quit IRC00:39
*** felipemonteiro__ has quit IRC00:40
*** Dinesh_Bhor has joined #openstack-keystone00:42
*** zhurong has joined #openstack-keystone00:45
*** wes_dillingham has quit IRC00:53
*** wes_dillingham has joined #openstack-keystone00:54
*** Dinesh_Bhor has quit IRC00:59
*** Dinesh_Bhor has joined #openstack-keystone01:16
*** wes_dillingham has quit IRC01:22
*** oikiki has joined #openstack-keystone01:27
*** harlowja has quit IRC01:31
*** Dinesh_Bhor has quit IRC01:35
*** Dinesh_Bhor has joined #openstack-keystone01:36
openstackgerrityangweiwei proposed openstack/keystone master: Fix user email in federated shadow users  https://review.openstack.org/54972301:37
*** hoonetorg has quit IRC01:56
*** hoonetorg has joined #openstack-keystone01:57
lbragstadkmalloc: yeah - that's all handled by mint() now01:58
lbragstadthat new contract would be that whoever implements a new token provider would need to pass back the id and the issued at time01:58
*** zhurong has quit IRC02:02
adriantlbragstad, cmurphy: is it a bug that when you supple a versionless auth url to a v3 plugin in keystoneauth, it can't work out the version? But doing it with generic appears to work.02:09
adriantsupple... supply02:10
adriantthe E and Y buttons aren't even near each other02:10
*** idlemind has quit IRC02:23
*** idlemind has joined #openstack-keystone02:24
*** edmondsw has joined #openstack-keystone02:42
*** oikiki has quit IRC02:47
*** edmondsw has quit IRC02:49
*** germs has quit IRC02:52
*** germs has joined #openstack-keystone02:52
*** germs has quit IRC02:52
*** germs has joined #openstack-keystone02:52
*** germs has quit IRC02:52
*** germs has joined #openstack-keystone02:53
*** germs has quit IRC02:53
*** germs has joined #openstack-keystone02:53
*** zhurong has joined #openstack-keystone02:56
*** dave-mccowan has quit IRC03:11
*** annp has joined #openstack-keystone03:14
*** AlexeyAbashkin has joined #openstack-keystone03:17
*** oikiki has joined #openstack-keystone03:21
*** AlexeyAbashkin has quit IRC03:21
*** harlowja has joined #openstack-keystone03:32
*** oikiki has quit IRC03:35
*** d0ugal_ has joined #openstack-keystone03:38
*** d0ugal has quit IRC03:39
*** harlowja has quit IRC03:54
*** edmondsw has joined #openstack-keystone04:05
*** edmondsw has quit IRC04:10
wxyadriant: I think it works as it is. Can I know that why you think it's a bug? generic plugin is used to support both v2 and v3. Maybe you think we can totally ignore version, because now keystone only support V3 now?04:12
adriantwxy: because with all the other services we now have version discovery built in for the most part, and I assumed with the v3 plugins for keystoneauth, if you supply a versionless url, it will use keystone's version discovery to get the right url for v3.04:14
adriantfor example, we have the catalog entry for keystone versionless04:14
adriantand I was playing with throwing that into keystoneauth v3 plugins as is, but that failed.04:15
adriantwxy: basically, if we encourage versionless urls in the catalogs because version discovery is a thing, then why should the v3 auth plugin require a version in the url?04:18
adriantnot to mention the error message you get from keystoneauth when you do that is 404 because it can't find the token endpoint, which doesn't help you realise what you did wrong.04:19
*** Dinesh_Bhor has quit IRC04:34
*** Dinesh_Bhor has joined #openstack-keystone04:36
*** zhurong has quit IRC04:45
*** felipemonteiro__ has joined #openstack-keystone04:45
*** germs has quit IRC04:49
*** germs has joined #openstack-keystone04:49
*** germs has quit IRC04:49
*** germs has joined #openstack-keystone04:49
*** germs has quit IRC04:54
*** felipemonteiro__ has quit IRC04:57
*** Dinesh__Bhor has joined #openstack-keystone05:03
*** Dinesh_Bhor has quit IRC05:03
*** isssp has joined #openstack-keystone05:03
*** idlemind has quit IRC05:05
*** burned has quit IRC05:06
*** isssp has quit IRC05:09
*** isssp has joined #openstack-keystone05:12
*** gyankum has joined #openstack-keystone05:26
*** jaosorior_ is now known as jaosorior05:26
*** zhurong has joined #openstack-keystone05:34
*** rcernin has quit IRC05:34
*** oikiki has joined #openstack-keystone05:43
*** rcernin has joined #openstack-keystone05:46
*** links has joined #openstack-keystone05:53
*** edmondsw has joined #openstack-keystone05:54
*** edmondsw has quit IRC05:59
*** pcichy has joined #openstack-keystone06:03
*** rcernin has quit IRC06:06
*** rcernin has joined #openstack-keystone06:08
*** Dinesh__Bhor has quit IRC06:12
*** Dinesh__Bhor has joined #openstack-keystone06:16
*** AlexeyAbashkin has joined #openstack-keystone06:16
*** AlexeyAbashkin has quit IRC06:21
*** threestrands has quit IRC06:25
wxyadriant: sorry that just backing now after lunch.06:28
wxyadriant: sounds we can improve the identity plugin to add version automatically in keystoneauth06:30
*** zhurong has quit IRC06:34
adriantwxy: that's what I was thinking. Shouldn't be too hard, but makes life a little nicer06:34
adriantthe code that does that exists, since the generic plugin must do that :P06:34
*** oikiki has quit IRC06:41
*** pcichy has quit IRC06:58
*** masber has quit IRC07:05
*** masber has joined #openstack-keystone07:06
*** deepak_ has quit IRC07:07
*** pcichy has joined #openstack-keystone07:11
*** d0ugal_ has quit IRC07:14
*** zhurong has joined #openstack-keystone07:14
*** d0ugal has joined #openstack-keystone07:14
*** d0ugal has quit IRC07:14
*** d0ugal has joined #openstack-keystone07:14
*** links has quit IRC07:15
*** deepak_ has joined #openstack-keystone07:19
*** belmoreira has joined #openstack-keystone07:22
*** links has joined #openstack-keystone07:23
*** martinus__ has joined #openstack-keystone07:25
*** deepak_ has quit IRC07:25
*** isssp has quit IRC07:25
*** jaosorior has quit IRC07:25
*** hrybacki has quit IRC07:25
*** Anticimex has quit IRC07:25
*** timss has quit IRC07:25
*** jaosorior has joined #openstack-keystone07:26
*** hrybacki has joined #openstack-keystone07:26
*** isssp has joined #openstack-keystone07:26
*** timss has joined #openstack-keystone07:27
*** deepak_ has joined #openstack-keystone07:28
*** gagehugo has quit IRC07:28
*** rcernin has quit IRC07:31
*** markvoelker has quit IRC07:35
*** gagehugo has joined #openstack-keystone07:35
*** edmondsw has joined #openstack-keystone07:42
*** pcaruana has joined #openstack-keystone07:42
*** pcaruana has quit IRC07:44
*** pcaruana has joined #openstack-keystone07:44
*** pcaruana has quit IRC07:45
*** pcaruana has joined #openstack-keystone07:45
*** edmondsw has quit IRC07:46
*** pcaruana has quit IRC07:47
*** pcaruana has joined #openstack-keystone07:47
*** pcaruana has quit IRC07:48
*** pcaruana has joined #openstack-keystone07:48
*** pcaruana has quit IRC07:50
*** pcaruana has joined #openstack-keystone07:50
*** masber has quit IRC07:51
*** pcaruana has quit IRC07:51
*** pcaruana has joined #openstack-keystone07:51
*** pcaruana has quit IRC07:53
*** pcaruana has joined #openstack-keystone07:53
*** pcaruana has quit IRC07:54
*** pcaruana has joined #openstack-keystone07:55
*** AlexeyAbashkin has joined #openstack-keystone07:56
*** pcaruana has quit IRC07:56
*** belmorei_ has joined #openstack-keystone07:57
*** ispp has joined #openstack-keystone07:58
*** asettle has quit IRC07:59
*** belmoreira has quit IRC07:59
*** andymccr has quit IRC07:59
*** isssp has quit IRC08:00
cmurphyadriant: i'm not so sure that's a bug08:01
cmurphyfrom https://docs.openstack.org/keystoneauth/latest/authentication-plugins.html "V3 identity plugins must use an auth_url that points to the root of a V3 identity server URL, i.e.: http://hostname:5000/v3."08:01
adriantcmurphy: can be change that? That wouldn't exactly be a breaking change, and makes life a little nicer08:02
*** belmorei_ has quit IRC08:02
*** belmore__ has joined #openstack-keystone08:02
adriantcan we*08:02
cmurphymaybe08:03
adriantkmalloc: would have a better insight, but I doubt anyone in their right mind is relying on that failing...08:04
cmurphymordred: also a good person to ask about it08:04
*** andymccr has joined #openstack-keystone08:05
*** dstanek has quit IRC08:06
*** asettle has joined #openstack-keystone08:06
*** pcaruana has joined #openstack-keystone08:06
*** asettle is now known as Guest6696908:06
*** pcaruana has quit IRC08:07
*** pcaruana has joined #openstack-keystone08:08
*** pcaruana has quit IRC08:09
*** pcaruana has joined #openstack-keystone08:10
*** pcaruana has quit IRC08:10
*** pcaruana has joined #openstack-keystone08:11
*** tesseract has joined #openstack-keystone08:11
*** masber has joined #openstack-keystone08:11
*** dstanek has joined #openstack-keystone08:11
*** pcaruana has quit IRC08:12
*** pcaruana has joined #openstack-keystone08:13
*** pcaruana has quit IRC08:15
*** pcaruana has joined #openstack-keystone08:15
cmurphyyou should also be able to use the generic plugin and it should do the right thing08:15
*** pcaruana has quit IRC08:16
*** AlexeyAbashkin has quit IRC08:17
*** pcaruana has joined #openstack-keystone08:18
*** masber has quit IRC08:18
*** AlexeyAbashkin has joined #openstack-keystone08:18
*** pcaruana has quit IRC08:20
*** pcaruana has joined #openstack-keystone08:20
*** pcaruana has quit IRC08:21
*** pcaruana has joined #openstack-keystone08:21
*** pcaruana has quit IRC08:22
openstackgerritAndreas Jaeger proposed openstack/keystoneauth master: Remove tox_install.sh and align with constraints consumption  https://review.openstack.org/55083708:23
*** pcaruana has joined #openstack-keystone08:29
*** pcaruana has quit IRC08:30
*** markvoelker has joined #openstack-keystone08:34
*** pcaruana has joined #openstack-keystone08:36
*** pcaruana has quit IRC08:37
*** pcaruana has joined #openstack-keystone08:38
*** pcaruana has quit IRC08:39
*** pcaruana has joined #openstack-keystone08:40
openstackgerritNguyen Hai proposed openstack/keystone master: Follow the new PTI for document build  https://review.openstack.org/55519608:40
*** pcaruana has quit IRC08:40
*** pcaruana has joined #openstack-keystone08:41
*** pcaruana has quit IRC08:41
*** sapd has quit IRC08:44
*** sapd has joined #openstack-keystone08:45
*** masber has joined #openstack-keystone08:55
adriantcmurphy: oh I am, but I would like to use v3 directly :P08:57
adriantjust for clarity as to which auth version I care about08:57
*** pcaruana has joined #openstack-keystone09:02
*** pcaruana has quit IRC09:03
*** pcaruana has joined #openstack-keystone09:03
*** pcaruana has quit IRC09:05
*** zhurong has quit IRC09:16
*** masber has quit IRC09:17
*** Dinesh__Bhor has quit IRC09:25
*** masber has joined #openstack-keystone09:28
*** edmondsw has joined #openstack-keystone09:30
*** edmondsw has quit IRC09:35
*** Guest66969 is now known as asettle09:46
*** masuberu has joined #openstack-keystone10:27
*** masber has quit IRC10:30
*** pcichy has quit IRC10:48
openstackgerritMonty Taylor proposed openstack/keystoneauth master: Remove tox_install.sh and align with constraints consumption  https://review.openstack.org/55083710:55
*** AlexeyAbashkin has quit IRC11:00
mordredcmurphy, adriant: I thnk we should fix v3 to allow versionless url - although I'd argue that if you hand a versionless auth_url to v3 plugin, if it can't get a v3 endpoint through discovery it should be a hard error11:00
*** AlexeyAbashkin has joined #openstack-keystone11:00
mordredcmurphy, adriant: while we're at it, we should do the same thing for v2 - the restriction doens't make any sense, and it undercuts our efforts to get people to use versionless auth urls11:01
*** wes_dillingham has joined #openstack-keystone11:21
*** belmore__ has quit IRC11:23
adriantmordred: ty! ++ for quality of life fixes :P11:31
*** zhurong has joined #openstack-keystone11:39
*** dklyle has quit IRC11:49
*** masuberu has quit IRC11:52
*** masuberu has joined #openstack-keystone11:52
*** odyssey4me has quit IRC12:07
*** odyssey4me has joined #openstack-keystone12:08
*** belmoreira has joined #openstack-keystone12:09
*** edmondsw has joined #openstack-keystone12:15
*** aojea has joined #openstack-keystone12:18
openstackgerritwangxiyuan proposed openstack/keystone master: Clean up token extra code  https://review.openstack.org/55527912:23
*** panbalag has joined #openstack-keystone12:29
*** markvoelker has quit IRC12:34
*** markvoelker has joined #openstack-keystone12:34
*** zhurong has quit IRC13:00
*** idlemind has joined #openstack-keystone13:05
*** dulek has joined #openstack-keystone13:06
dulekHi guys! Any idea of a recent change that was related to certificates? Our Kuryr jobs started to fail because it can't find /opt/stack/data/ca-bundle.pem.13:07
dulekI mean - when connecting to Keystone. :)13:07
lbragstaddulek: nothing on our end afaik13:08
lbragstaddulek: do you have an example failure?13:09
dulekhttp://logs.openstack.org/54/555254/1/experimental/kuryr-kubernetes-tempest-daemon-containerized-lbaasv2/9ecada1/controller/logs/screen-kuryr-kubernetes.txt.gz#_Mar_22_11_50_32_98129913:09
duleklbragstad: ^13:09
duleklbragstad: It's probably because this is being run from inside the container with Kuryr. And this container has no access to /opt/stack/data from the host.13:10
*** NM has joined #openstack-keystone13:10
lbragstadahh13:10
lbragstaddid that work before?13:10
dulekSo seeing a change that triggered this would be helpful to understand how it should be fixed properly.13:10
duleklbragstad: Well… Yes. :)13:10
lbragstadhow long has it been failing?13:10
duleklbragstad: It's a bit hard to tell - that's experimental job, but I'd say that around a week.13:11
lbragstadthere hasn't been a whole lot of activity in the last two weeks, most folks are reviewing specs13:12
lbragstadnothing cert related from keystone-server https://github.com/openstack/keystone/commits/master13:12
duleklbragstad: Okay, thank you. I think this might have been a DevStack change.13:12
lbragstadi just started looking there - https://github.com/openstack-dev/devstack/commits/master13:13
dulek:)13:13
duleklbragstad: Hm, not much activity there since we've fixed our gates. And I've definitely checked experimental back then. That's a bit odd.13:15
duleklbragstad: Anyway thanks again, we'll figure it out ourselves. :)13:15
lbragstaddulek: no worries - let us know if there is anything we can help with13:16
*** germs has joined #openstack-keystone13:33
*** germs has quit IRC13:33
*** germs has joined #openstack-keystone13:33
*** germs has quit IRC13:33
*** germs has joined #openstack-keystone13:34
*** germs has quit IRC13:34
*** germs has joined #openstack-keystone13:34
openstackgerritNguyen Hai proposed openstack/keystone master: Follow the new PTI for document build  https://review.openstack.org/55519613:34
*** aojea has quit IRC13:36
*** aojea has joined #openstack-keystone13:46
*** aojea has quit IRC13:50
*** felipemonteiro__ has joined #openstack-keystone13:55
*** wes_dillingham has quit IRC14:00
*** wes_dillingham has joined #openstack-keystone14:00
*** wes_dillingham has quit IRC14:00
*** wes_dillingham has joined #openstack-keystone14:01
*** wes_dillingham has quit IRC14:01
kmallocadriant: either mordred or I can look into that, but FTR - keystoneauth has an extremely strict contract. We adhere to "The behavior will not change once it is released*"  * == unless there is a major security flaw and there is no other solution14:01
*** wes_dillingham has joined #openstack-keystone14:02
*** wes_dillingham has quit IRC14:02
*** wes_dillingham has joined #openstack-keystone14:02
*** wes_dillingham has quit IRC14:03
*** wes_dillingham has joined #openstack-keystone14:03
*** wes_dillingham has quit IRC14:03
*** felipemonteiro_ has joined #openstack-keystone14:03
*** wes_dillingham has joined #openstack-keystone14:04
*** wes_dillingham has quit IRC14:04
kmallocadriant: the v3 plugin requires a version because you're asking for a versioned plugin and that is the historical behavior14:04
kmallocadriant: it is not a bug. (cmurphy thanks for fielding the question!)14:04
*** wes_dillingham has joined #openstack-keystone14:05
*** wes_dillingham has quit IRC14:05
*** wes_dillingham has joined #openstack-keystone14:06
*** wes_dillingham has quit IRC14:06
*** wes_dillingham has joined #openstack-keystone14:06
*** wes_dillingham has quit IRC14:07
*** wes_dillingham has joined #openstack-keystone14:07
*** wes_dillingham has quit IRC14:07
*** felipemonteiro__ has quit IRC14:07
*** pcichy has joined #openstack-keystone14:07
*** wes_dillingham has joined #openstack-keystone14:08
*** wes_dillingham has quit IRC14:08
*** links has quit IRC14:10
*** itlinux has quit IRC14:11
*** pcichy has quit IRC14:12
*** pcichy has joined #openstack-keystone14:12
*** dklyle has joined #openstack-keystone14:15
*** spilla has joined #openstack-keystone14:17
*** germs_ has joined #openstack-keystone14:18
*** germs has quit IRC14:20
*** r-daneel has joined #openstack-keystone14:27
*** wes_dillingham has joined #openstack-keystone14:40
*** felipemonteiro__ has joined #openstack-keystone14:53
*** felipemonteiro_ has quit IRC14:57
*** gyankum has quit IRC14:58
*** jessegler has joined #openstack-keystone15:04
openstackgerritLance Bragstad proposed openstack/keystone master: WIP: rewrite keystone  https://review.openstack.org/54545015:06
lbragstadpasses unit tests, including python3 ^ '15:06
*** aojea_ has joined #openstack-keystone15:09
lbragstadso - fun fact15:14
*** aojea_ has quit IRC15:14
mordredkmalloc: I contend it's a bug - but I can go along with not fixing it due to strict contract15:15
lbragstadkeystone will break when running in python 3 if CONF.token.cache_on_issue is False15:15
*** dave-mccowan has joined #openstack-keystone15:15
gagehugooof15:15
lbragstadmmmhmm15:16
mordredlbragstad: wow. fun patch15:17
lbragstadi literally went crossed staring at https://github.com/openstack/keystone/blob/master/keystone/auth/plugins/core.py#L63-L9715:17
lbragstadbecause it returned two different things depending on the version of python you run it in15:17
lbragstadso i was like "oh, typing issue somewhere, right?"15:17
lbragstadnope, all types are consistent15:17
lbragstadbut python3 is more precise with rounding than python 2, which automatically floors results15:17
lbragstadhence... integers always being expected...15:18
lbragstadbut the only reason I found it was because i commented out https://github.com/openstack/keystone/blob/master/keystone/token/provider.py#L17315:18
lbragstadwhich cause the token to be rebuilt when validated...15:18
*** germs_ has quit IRC15:20
lbragstadwe obviously short-circuit that code if we stuff the token in the cache on our way out the door when we authenticate the token...15:20
*** germs has joined #openstack-keystone15:20
*** germs has quit IRC15:20
*** germs has joined #openstack-keystone15:20
*** felipemonteiro__ has quit IRC15:21
lbragstadso - if you're running keystone in python3 and disable the configuration option... token validation will break for your deployment15:21
*** felipemonteiro__ has joined #openstack-keystone15:21
gagehugohmm15:21
*** idlemind_ has joined #openstack-keystone15:33
*** idlemind has quit IRC15:33
*** wes_dillingham has quit IRC15:37
*** itlinux has joined #openstack-keystone15:37
*** itlinux has quit IRC15:39
*** itlinux has joined #openstack-keystone15:39
openstackgerritLance Bragstad proposed openstack/keystone master: WIP: Expose python3 bug when cache_on_issue is False  https://review.openstack.org/55533915:40
openstackgerritLance Bragstad proposed openstack/keystone master: Cast division product to int when inflating auth  https://review.openstack.org/55534015:40
lbragstadgagehugo: ^15:40
lbragstadi think it happens with that specific test because it's using more authentication methods15:40
gagehugoack15:40
gagehugointeresting15:40
lbragstadi need to open a bug yet.. but just in case you were curious15:40
lbragstadotherwise we're using relatively nice numbers in the method_map15:41
lbragstadand i think they divide nicely, instead of taking 5 / 4 and getting 1.2515:42
gagehugoyeah15:42
*** pcaruana has joined #openstack-keystone15:42
*** pcaruana has quit IRC15:44
lbragstadmordred: it's a real fun patch, you should review it ;)15:44
lbragstadif getting rid of technical debt is your thing, you'll love it15:45
*** wes_dillingham has joined #openstack-keystone15:46
*** pcaruana has joined #openstack-keystone15:48
kmallocmordred: well, that is the deal with ksa, strict contract (also, the doc explicitly calls out this behavior)15:53
kmalloclbragstad: ugh, the pre-population cache off breaks keystone?! ugh15:53
kmallocoh that is just dumb...15:54
*** Supun has joined #openstack-keystone15:54
lbragstadkmalloc: :)15:57
lbragstadit's one big short circuit15:57
lbragstadand with that.. i'm going to take lunch quick15:58
*** harlowja has joined #openstack-keystone16:04
*** felipemonteiro_ has joined #openstack-keystone16:05
*** felipemonteiro__ has quit IRC16:09
*** harlowja has quit IRC16:09
openstackgerritGage Hugo proposed openstack/keystone master: Add functional testing gate  https://review.openstack.org/53101416:34
*** AlexeyAbashkin has quit IRC16:34
*** masuberu has quit IRC16:36
*** aojea_ has joined #openstack-keystone16:58
*** wes_dillingham has quit IRC16:58
*** gyee has joined #openstack-keystone17:00
*** Supun has quit IRC17:00
*** aojea_ has quit IRC17:02
*** idlemind_ has quit IRC17:02
*** idlemind has joined #openstack-keystone17:03
*** belmoreira has quit IRC17:15
*** jessegler has quit IRC17:18
openstackgerritJohannes Grassler proposed openstack/keystone-specs master: Add whitelist-extension-for-app-creds  https://review.openstack.org/39633117:22
gagehugolbragstad you alright with me submitting a change to governance to add the vmt tag for keystonemiddleware?17:32
*** wes_dillingham has joined #openstack-keystone17:36
*** Supun has joined #openstack-keystone17:39
*** Anticimex has joined #openstack-keystone17:50
lbragstadgagehugo: go for it!17:56
*** felipemonteiro has joined #openstack-keystone17:56
*** felipemonteiro_ has quit IRC17:59
ayounglbragstad, something we don't do, and need to, is address how to clean up all resources in the service catalog when you delete a projects18:11
ayoungI was trying to think through how that would work, and my head hurts18:11
ayounglets assume, for the moment, that we generate a special token for that use case18:11
ayoungit is scoped to the user that performed delete project, and it never expires, but can only be used to delete things in that project18:12
ayoungthat would be the magic case18:12
ayoungand so, we have some workflow that goes to every .... endpoint?18:12
*** boris_42_ has joined #openstack-keystone18:12
ayoungand calls "delete-all-resources-for-project" on each endpoint, validated by that magic token18:13
ayoungso...short of adding Ansible tower into the OpenStack family of services, I have no idea how to make that happen.  Mistral, I guess?18:13
*** harlowja has joined #openstack-keystone18:15
*** AlexeyAbashkin has joined #openstack-keystone18:15
openstackgerritLance Bragstad proposed openstack/keystone master: Fix integer -> method conversion for python3  https://review.openstack.org/55533918:16
lbragstadayoung: that was brought to mistral a while ago, but it was never accepted upstream18:17
*** harlowja_ has joined #openstack-keystone18:17
ayounglbragstad, the problem is keeping track of the set of resources that should be deleted18:17
ayoungand the order...tearing down a network in Neutron has to be done in the right order or you get dependency issues18:18
ayoungideally, the services would implement that themselves:18:18
ayoungdelete all resources in project18:18
ayoungand...I wonder if there is even an ordering there that is necessary18:19
lbragstadi wouldn't be surprised18:19
ayounglike, what happens in Nova if you delete the network in neutron and the volume in cinder18:19
ayoungand the image in glance18:19
*** harlowja has quit IRC18:19
*** AlexeyAbashkin has quit IRC18:20
*** dave-mccowan has quit IRC18:20
ayoungWe could do the workflow outside of Keystone after the project is deleted, but right now it needs a super-power token18:23
ayoungand that I don't like.18:24
*** wes_dillingham has quit IRC18:27
kmallocayoung: it needs to be an API on the service (aka Nova)18:28
kmallocayoung: it is NOT a keystone problem to do that18:28
kmallocayoung: the way openstack is architected, it just isn't viable to make it a keystone problem to do that.18:28
ayoungkmalloc, I think it is a general OpenStack problem to solve, but Keystone is the only service in position to guide the discussion18:28
kmallocayoung: so yeah, mistral, ansible tower, something? iterate over endpoints in keystone and call "clean-up-for-project/<project-id>18:29
kmallocayoung: making a scoped token for a deleted project is a no-go. really, either we do "work outside keystone" or have the services support a cleanup-for-project type API.18:30
ayoungkmalloc, if I had to do it today, I would have a listener get the project delete notification, use an admin scoped token to list all resources of one kind in an end point, delete them, then move on to the next one"18:30
ayoungI would rather be able to do that with a scoped token18:30
kmallocthis really needs to be a service-api for the services.18:30
kmallocit is absurd to have to do a delete-all-instances in nova, if you have say 10000 instances under a project18:31
kmallocthat is a lot of tear down, vs. "hey nova, schedule delete all things for project X"18:31
ayoungwhat if....we generated an app credential and used the Whitelist to limit it to the delete operations?18:31
kmallocand let nova handle scheduling that18:31
ayoungIt would fail validation18:31
*** tesseract has quit IRC18:32
ayoungkmalloc, I hear you, but getting agreement on that API across all the services in OpenStack would be a full time job18:32
ayoungThe fact that we don't even have a clean way to scope/workflow it now is sad18:32
kmalloci'm going to reierate, this isn't a keystone problem, *we cannot fix this* we cna paper over it in some bad ways that don't help much18:32
ayoungwhat if...18:32
ayoungwe disable the project, but allow someone to get tokens scoped to it in order to do the clean up18:33
ayoungthe disable is to prevent new resources from being created18:33
kmallocthis sounds like a terrrrrrible idea but that is somewhat better18:33
ayoungmeanwhile, we can try to drive the long term api changes18:33
kmallocmostly concerns me from a security standpoint18:33
kmallocif a disabled project can get tokens.. it is somewhat of a security issue, but i'm less concerned than scoping to a random project (that may or may not be deleted)18:34
kmallocsince we have a record of the project to base it on18:34
kmalloci am not sure i like the prospect of maintaining auth for disabled projects, but again -- that is less of a red flag (very bad no good) feeling than the scope to deleted projects18:36
*** germs has quit IRC18:36
*** germs has joined #openstack-keystone18:37
*** germs has quit IRC18:37
*** germs has joined #openstack-keystone18:37
*** germs has quit IRC18:39
*** germs has joined #openstack-keystone18:39
*** germs has quit IRC18:39
*** germs has joined #openstack-keystone18:39
*** felipemonteiro has quit IRC18:42
*** felipemonteiro has joined #openstack-keystone18:43
*** Krenair_ is now known as Krenair18:45
*** Krenair has joined #openstack-keystone18:45
*** aojea has joined #openstack-keystone18:46
*** aojea has quit IRC18:50
*** wes_dillingham has joined #openstack-keystone18:51
*** itlinux has quit IRC18:59
*** itlinux has joined #openstack-keystone18:59
*** voelzmo has joined #openstack-keystone19:01
*** aojea has joined #openstack-keystone19:01
*** wes_dillingham has quit IRC19:01
*** r-daneel_ has joined #openstack-keystone19:02
*** r-daneel has quit IRC19:02
*** r-daneel_ is now known as r-daneel19:02
*** voelzmo has quit IRC19:10
*** voelzmo has joined #openstack-keystone19:11
kmallocadriant: one more round of updates for MFA-receipts19:18
kmallocadriant: but you're close. we're at the point where it's just clarifying sections and making sure we're not leaving ourselves open to misinterpretation of the spec during implementation. This is ready for a +2 with the changes i outlined.19:19
*** voelzmo has quit IRC19:22
kmallocjgr: the recent spec for whitelist is looking reaaaaaaally good19:22
kmallocjgr: +1, will be doing more in depth after food.19:22
*** aojea has quit IRC19:23
*** aojea has joined #openstack-keystone19:43
*** devx has joined #openstack-keystone19:44
*** aojea has quit IRC19:45
*** aojea has joined #openstack-keystone19:45
*** aojea has quit IRC19:46
devxhello, I was hopping to get a bit of direction. I'm using AD with keystone and I've encounter what I think it's a bug where I get a Duplicate Entry error. resulting in19:47
devx"An error occurred authenticating", any direction would be appreciated19:47
*** devx is now known as DevX19:49
*** wes_dillingham has joined #openstack-keystone19:49
*** aojea has joined #openstack-keystone19:49
*** r-daneel_ has joined #openstack-keystone19:49
*** r-daneel has quit IRC19:50
*** r-daneel_ is now known as r-daneel19:50
*** panbalag has quit IRC19:58
*** wes_dillingham has quit IRC19:59
*** Supun has quit IRC19:59
lbragstadDevX: how are you authenticating?20:03
DevXvia ldaps20:04
DevXi can share my config if you like to see20:04
DevXthis is the error I'm getting https://gist.github.com/devx/046fa81d1bafb78086d9d20c4831b00320:04
lbragstadsure - i'll take a look20:04
lbragstadoh...20:05
lbragstadhey DevX :)20:05
DevXlet me generate a gist ;)20:05
lbragstadi thought that github profile picture looked familiar20:06
DevXbtw cloudnull says hi.20:06
* lbragstad waves20:06
lbragstadis cloudnull creeping over your shoulder?20:06
DevXhe was20:07
DevXgive me a sec need to sanitize20:07
DevXbtw i've validated ldap connectivity and all that stuff20:07
*** josecastroleon has joined #openstack-keystone20:08
* cloudnull is lurking in the shadows20:08
lbragstadhuh...20:09
lbragstadthe four lines of that trace is interesting20:09
lbragstadit looks like the user you're authenticating with is in fact shadowed in keystone20:09
lbragstadbut it attempts to shadow it anyway?20:09
DevXyeah20:09
DevXhttps://gist.github.com/devx/ccee034943961b60f55aeed37742632520:10
DevXthe ldap stuff has been sanitized but they work.20:10
DevXi've manually tested using ldapsearch20:10
lbragstadis this queens?20:11
DevXTLDR: it was working last week when there was an edir in place alongside AD servers.  However, the edir has now been removed and it's all AD and we are getting this error20:11
DevX:(20:11
DevXnewton20:11
* DevX hides in shame20:11
lbragstadso which one was keystone backed to?20:12
lbragstadedir or ad?20:12
DevXAD20:12
DevXso not sure why the problem started to happen it does not make sense20:12
lbragstadright... that's strange20:13
DevXthe fact that the insert happens lead me to believe that it migth be a bug in Newton not sure if it happens in queens though20:14
* lbragstad tries to find a copy of newton code20:16
DevXI have tested this via openstack cli client and horizion and same error.  Via horizon shows up as a 40920:16
lbragstadwe do seem to have a very clear case in master20:17
lbragstadwe enter here - https://github.com/openstack/keystone/blob/d4f3160334838c592cc8616bba85c13f308468f6/keystone/identity/core.py#L91720:17
lbragstadwe get into this - https://github.com/openstack/keystone/blob/d4f3160334838c592cc8616bba85c13f308468f6/keystone/identity/core.py#L60620:17
*** aojea has quit IRC20:17
lbragstadwhich should be a single user case20:18
lbragstadjudging by the trace20:18
*** aojea has joined #openstack-keystone20:18
DevXi see20:18
lbragstadwe very clearly hit - https://github.com/openstack/keystone/blob/d4f3160334838c592cc8616bba85c13f308468f6/keystone/identity/core.py#L64320:18
lbragstadwhich is the only place we emit the log in your trace20:19
DevXright20:19
lbragstadand we don't bother inserting that public id20:19
lbragstadwe have this in newton https://git.openstack.org/cgit/openstack/keystone/tree/keystone/identity/core.py?h=newton-eol#n59920:21
DevXright20:22
DevXit's just logging it20:22
DevXlet me grab the full trace and update the gist20:27
lbragstadok.. i'm wondering if for some weird reason it's not finding the user after successfully authenticating...20:29
lbragstadhttps://git.openstack.org/cgit/openstack/keystone/tree/keystone/identity/core.py?h=newton-eol#n124920:29
lbragstaddid AD have it's own domain configuration?20:30
DevXyes20:31
openstackgerritGage Hugo proposed openstack/keystone master: Remove the TokenAuth middleware  https://review.openstack.org/50841220:32
*** aojea has quit IRC20:33
*** rarora has quit IRC20:33
*** aojea has joined #openstack-keystone20:34
DevXOk, I've updated the gist with the entire gist. (sorry, I should have done that in the first place)20:38
lbragstadchecking20:39
*** MeltedLux has joined #openstack-keystone20:39
*** boris_42_ has quit IRC20:41
lbragstadthere has to be a mismatch somewhere...20:42
lbragstadhas this user existed since edir was removed?20:42
DevXyes, this happening for all the users20:42
DevXI even ran the mappings purge to ensure it was not not a stale mapping20:43
lbragstadhuh - that was my next suggestion20:43
DevXhave also added users to the AD group20:43
DevXand same effect20:43
DevXthis is affecting all AD users, all local users are ok20:43
DevXat first thought it was a cert issue so i switch to `tls_req_cert = allow`20:45
DevXwhich checks the certs but will continue no matter what20:45
*** AlexeyAbashkin has joined #openstack-keystone20:45
DevXthis LDAP/AD setup was working for a couple of months but broke when the edir was removed20:46
lbragstadthat's really weird...20:47
lbragstadand edir wasn't integrated into keystone at all?20:47
DevXtell me about it20:47
DevXI went as far as setting an incorrect password to verify connectivity20:48
DevXand it failed to connect as expected. aka failed to bind.20:48
lbragstadhmm20:49
DevXI can re-run the clear mappings again and see if that helps20:49
DevXI can try it per domain or per user20:49
lbragstadare you able patch the system?20:49
lbragstadi assume this is using osa in some form or fashion?20:49
DevXcorrect20:49
DevXOSA20:49
DevXi could try to do the domain ID and domain name when i do `keystone-manage mapping_purge --domain-name DOMAIN`20:50
lbragstadhttp://paste.openstack.org/raw/709012/20:52
*** AlexeyAbashkin has quit IRC20:52
lbragstadi was going to see if there is something weird going on that results in that use not getting found20:52
*** edmondsw has quit IRC20:56
*** edmondsw has joined #openstack-keystone20:56
*** aojea has quit IRC20:57
lbragstadbut yeah - you could double check that the mapping purge tooling is actually removing the reference20:57
*** spilla has quit IRC20:57
*** spilla has joined #openstack-keystone20:57
*** spilla has quit IRC20:58
DevXok, let me go do that20:58
*** josecastroleon has quit IRC20:59
*** edmondsw has quit IRC21:01
*** aojea has joined #openstack-keystone21:07
DevXso i purged the mappins and i checked the `id_mapping` table and was empty and it get's populated again when a user checks in.  I'm checking the logs to see if still get the same errors21:07
lbragstadok - cool, does "check in" mean "authenticate"21:08
DevXyes21:10
DevXsame outcome - 'Conflict occurred attempting to store nonlocal_user - Duplicate Entry'21:12
lbragstadwhat in the world21:12
DevXI think i figured it out21:14
DevXhttps://github.com/openstack/keystone/blob/newton-eol/keystone/identity/core.py#L57821:14
*** itlinux has quit IRC21:14
DevXnevermind, i don't think it's that21:15
lbragstadare you able to apply https://gist.github.com/lbragstad/eab71ffc7c8c6c38cbd41bbb7930c82b#file-logging-patch ?21:16
*** AlexeyAbashkin has joined #openstack-keystone21:17
*** AlexeyAbashkin has quit IRC21:21
*** raildo has quit IRC21:24
*** pcichy has quit IRC21:27
*** NM has quit IRC21:32
DevXyes, i'm restarting and testing21:32
lbragstadsweet21:33
DevXadding those extra loggin statements I see the following: Could not find user: u'1366748ea7987e003bb020cbd87c8e89fd64e1e753c0008075a528895d61d5cd'21:51
*** Drankis has quit IRC21:52
*** pcaruana has quit IRC21:53
lbragstadok - so that appears to be the public id21:53
lbragstadhttps://gist.github.com/devx/046fa81d1bafb78086d9d20c4831b003#file-keystone-ad-conflict-nonlocal_user-L2621:53
lbragstadhow is https://git.openstack.org/cgit/openstack/keystone/tree/keystone/identity/core.py?h=newton-eol#n1249 getting hit?21:55
lbragstadthis kinda blows my mind21:55
DevXgive me a sec I'll share the trace21:57
*** martinus__ has quit IRC21:57
DevXlbragstad: I just shared with you a gist22:04
lbragstadchecking22:04
lbragstadand edir wasn't integrated into keystone anyway?22:06
lbragstadthat might be affecting how users are retrieved from AD?22:07
DevXno it was just hitting the LB22:07
DevXI can create a new domain and try it maybe that's the problem22:07
DevXor a later version of keystone22:08
lbragstadthe code looks pretty much the same between newton and queens22:08
openstackgerritGage Hugo proposed openstack/keystone master: Remove the TokenAuth middleware  https://review.openstack.org/50841222:08
lbragstad(the shadow users stuff hasn't changed much since then)22:08
lbragstadbut it's really strange that edir being involved prevented this from breaking22:08
DevXyeah, let me go test some things I'll hit you up tomorrow after I tried a couple of things22:10
lbragstadsounds good22:10
openstackgerritLance Bragstad proposed openstack/keystone master: WIP: rewrite keystone  https://review.openstack.org/54545022:10
*** itlinux has joined #openstack-keystone22:11
adriantkmalloc: just about to go do those last few spec changes22:14
kmallocadriant: nice22:14
adriantkmalloc: and as for Keystone auth, I assumed it wasn't really a bug, but much like mordred I think changing it if we can wouldn't be a bad idea22:14
kmallocstrict contract and behavioral changes = no change22:15
adriant:(22:15
kmallocit's even documented as such22:15
kmallocwe can't change it.22:15
kmallocthat is the rule with ksa22:16
adriantkmalloc, even for something as benign as that?22:16
kmalloccould someone be relying on that behavior on old clouds with v2 and v322:16
kmallocif the answer is yes.22:16
kmallocthen we can't change22:17
kmalloccould someone be realying on that error for any reason, again, if the answer is yes, we can't change it22:17
kmallocKSA's contract is "we will not break you, we will not change behavior unless we have a serious security concern and it is the only way:22:17
kmallocEVEN for behaviors we don't like.22:18
adriantkmalloc: but we can add new behavior in the form of better plugins and such as we go as long as we keep the existing ones?22:18
kmallocas long as we keep the same behavior for existing things, yes22:18
kmallocbut once it's been released in KSA it's permanent22:18
kmallocwe can't remove it.22:18
kmallocso we're very careful about adding to KSA.22:18
adriantk, because I am going to have to do a lot of stuff in KSA for the MFA stuff, but we can do it as new plugins22:19
kmallocnow, you can make old plugins smarter about features like MFA-- or allow KSA itself to handle a receipt for MFA (opt-in)22:20
kmallocbut you can't break old behaviors.22:20
kmallocyou're going to need to work aroudn the old plugins, you probably want the maintain use of the old plugins.22:21
adriantkmalloc: yep, that was my intention, keep the behavior, but add some wrapping, but also add some plugins that you can specify: "auth with these multiple methods"22:21
kmallocyeah a multi-method plugin (somehow) is needed :)22:21
kmallocyou're on the right track :)22:21
adriantthe trick will be not duplicating too much code, and seeing if... we can ideally split the existing plugin logic into generic 'method' definitions that the multi-method plugin can use, while the existing ones only use the single method they used to.22:22
adriantAll without changing the behavior...22:22
adriantI think start with duplication, and then refactor22:23
*** rcernin has joined #openstack-keystone22:34
cmurphyI have a hard time imagining how someone could be relying on that behavior22:35
*** edmondsw has joined #openstack-keystone22:46
adriantcmurphy: me too, but I understand kmalloc's point as much as stuff like that can suck :(22:46
adriantcmurphy: the annoying part is that that error isn't useful, it doesn't actually tell you what failed really22:47
adriantor, it does tell you, but not exactly22:47
*** aojea has quit IRC22:49
cmurphyit just tries to POST directly to /auth/tokens and blows up there right?22:50
adriantyep22:50
openstackgerritAdrian Turjak proposed openstack/keystone-specs master: Add spec for MFA auth receipts  https://review.openstack.org/55367022:50
adriantand throws a 40422:50
cmurphythat seems really unuseful22:50
adriantwhich is accurate just not useful22:50
adriantthings like that, I'm of the opinion: if you rely on that error you're doing something stupid22:51
cmurphysomeone once told me every bugfix is backwards incompatible if someone is relying on the bug22:51
adriantkmalloc, cmurphy, lbragstad: spec updated, all yours22:52
cmurphysweet22:52
adriantI'm a bit terrified because when it is merged, I actually have to implement it :P22:53
*** masber has joined #openstack-keystone22:56
*** felipemonteiro has quit IRC23:03
*** r-daneel has quit IRC23:12
*** masuberu has joined #openstack-keystone23:13
*** AlexeyAbashkin has joined #openstack-keystone23:16
*** masber has quit IRC23:17
*** masber has joined #openstack-keystone23:18
*** masuberu has quit IRC23:18
*** AlexeyAbashkin has quit IRC23:20
*** masber has quit IRC23:29
*** masber has joined #openstack-keystone23:29

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!