| knikolla | keekz: group memberships carried through mapping rules have a time limit and require renewal through periodic re-authentication https://specs.openstack.org/openstack/keystone-specs/specs/keystone/ussuri/expiring-group-memberships.html | 00:55 |
|---|---|---|
| knikolla | it is managed through the [federation] default_authorization_ttl configuration option https://github.com/openstack/keystone/blob/187bcad52219c2c99b572b3d5463dfa7b2128201/keystone/conf/federation.py#L120 | 00:58 |
| knikolla | in your case default_authorization_ttl = 720, so 12 hours | 00:59 |
| opendevreview | Yaguang Tang proposed openstack/keystoneauth master: fix logic issue for getting IDP data with HTTP 400 response https://review.opendev.org/c/openstack/keystoneauth/+/942171 | 01:17 |
| *** mhen_ is now known as mhen | 03:00 | |
| opendevreview | Merged openstack/keystone master: docs: Remove warning about unified limits API https://review.opendev.org/c/openstack/keystone/+/893120 | 08:59 |
| knikolla | gtema: thank you for stepping up to be keystone ptl. amazing work! | 13:42 |
| gtema | thks knikolla, glad to see you here again | 13:42 |
| knikolla | :) would love to hear more about the federation work that you're doing and happy to help a bit here and there | 13:43 |
| knikolla | though unfortunately i don't have anymore work-time allocated to openstack things. | 13:44 |
| gtema | sad. A start of federation work is in my Rust reimplementation of Keystone which is for the beginning necessary as a "library" | 13:45 |
| gtema | Just around an hour ago was reading an article "quantum computers are now the reailty" and think - we need to move very fast to be ready for eventual changes in crypto area | 13:46 |
| gtema | https://www.forbes.com/sites/chuckbrooks/2025/02/22/quantum-computing-has-arrived-we-need-to-prepare-for-its-impact/ - pretty interesting read | 13:46 |
| knikolla | interesting. so you're reimplementing the keystone api? | 13:48 |
| gtema | 1st idea was to have a library for accessing keystone DB, but it immediately evolved into "if we have that anyway it just cost a few loc to add API": https://github.com/gtema/keystone | 13:49 |
| gtema | performance difference is HUGE | 13:49 |
| gtema | it is a very long way to go. Initially I only wanted to have certain parts that are involved in the federation work | 13:50 |
| gtema | btw, also read about next steps https://gtema.github.io/posts/rethinking-openstack-auth-authz/ | 13:50 |
| gtema | I plan to implement that exactly in the new separate component not to break users and allow CSPs deploy that on older versions of OpenStack (sadly the reality shows how old lot of clouds are) | 13:51 |
| knikolla | reading the blog post now, give me a few minutes | 13:55 |
| gtema | sure, no hurry :) | 13:55 |
| knikolla | can you tell me a bit more about the problem "Lack of an easy possibility to easily exchange JWT for an OpenStack token."? | 14:00 |
| gtema | flexibility - every user (customer) must be able to configure own rules, register own provider, define own mapping | 14:01 |
| knikolla | ah, that makes sense. | 14:01 |
| gtema | the use case: github action provides a JWT and we want to exchange it for fernet | 14:01 |
| knikolla | mod_auth_oidc makes it so you can't have per-domain identity providers, and every reconfiguration requires admin and restart | 14:02 |
| gtema | but every domain should be defining own mapping rules for that and grant permissions | 14:02 |
| gtema | precisely - therefore I want to get rid of mod_auth_oidc and merge this into the keystone directly | 14:02 |
| gtema | and sadly while POCing on that figured out that python libs for oidc are in a very bad shape | 14:03 |
| knikolla | i think that makes sense. in our (my employer's) use cases we have put an identity broker in front of keystone that handles the multiple idp part. | 14:03 |
| knikolla | so it's: external idp 1 2 and 3 -> keycloak -> keystone | 14:03 |
| gtema | many are doing exactly that because otherwise it is not possible | 14:03 |
| gtema | issue here is that customers are still not really able to do necessary work themselves unless you grant them into the broker IDP which is also not what everybody wants | 14:04 |
| knikolla | (funny parenthesis, but we (the keystone team at the time of 6-7 years ago) wanted to start work on an identity brokering service that handled just mapping rules to issue tokens, and written in rust https://opendev.org/x/spectroscope ) | 14:05 |
| knikolla | nothing came out of that as that was about the time when most people stopped getting funded to work on openstack on the keystone side | 14:06 |
| gtema | hehe, funny | 14:06 |
| gtema | I am not sure it makes any sense (or any capacity) to implement brokering in openstack. There are lot of tools doing so already | 14:07 |
| gtema | it is just that openstack does not integrate good with them | 14:07 |
| knikolla | that's what the federation plugin sort of does already | 14:08 |
| knikolla | it's just about making it more dynamic and configurable so that authN can be separate from the projects/role assignments piece of keystone. | 14:09 |
| gtema | yupp | 14:09 |
| gtema | the mapping resource is also not flexible enough to address all potential use cases. Basically data model is not fitting well into the authN/Z world of today | 14:10 |
| knikolla | the moment you start dealing with multiple service offerings, from the perspective of a CSP, there is no data model that fits well. | 14:11 |
| knikolla | to your point of "Lack of possibility to integrate external authorization systems." | 14:11 |
| knikolla | i found that what worked best for us is having a centralized service that "pushed" permissions to each of the individual services that we offer to customers (openstack, openshift, etc) | 14:12 |
| knikolla | we used https://coldfront.readthedocs.io/en/latest/ for that | 14:13 |
| knikolla | but again, that doesn't work for the use case of allowing users to handle their own IdPs and users | 14:13 |
| gtema | yeah. I think pretty much every CSP offering more than OS would have some external system where authZ is being managed. We should allow openstack better consume that. | 14:14 |
| gtema | therefore I was playing with role-assignment backend plugin to fetch info from external source (openfga in that particular case). and on the other side OS services would delegate authz decision to OpenPolicyAgent | 14:15 |
| gtema | after I myself came to the idea and prototyped a lot found that it was also already present as an idea in OS just that it never landed | 14:16 |
| gtema | https://www.openstack.org/videos/summits/berlin-2018/dynamic-policy-for-openstack-with-open-policy-agent | 14:16 |
| gtema | I work on implementing this in prod now at my employer | 14:17 |
| knikolla | you'll see that a lot of ideas on how to make keystone and federation better were 80% implemented (because that takes 20% of the effort) during 2018 and 2019, haha. | 14:17 |
| gtema | yeah | 14:17 |
| knikolla | i am myself guilty of a lot of that too | 14:17 |
| knikolla | as you may have seen above from "expiring group membership" or "federated user crud" | 14:17 |
| gtema | yes, those are cool things | 14:18 |
| knikolla | but they didn't go far enough | 14:18 |
| gtema | yeah. I think back those days federation was not really a thing for CSPs, and nowadays implementation does not fit current requirements | 14:19 |
| knikolla | "Implement SCIM support so that customer is able in a self-service manner synchronize domain users from an external IdP" | 14:19 |
| knikolla | This is tricky, because SCIM by default only has users and groups as concepts | 14:20 |
| gtema | I know. For now I have implemented SCIM as a proxy (translating to keystone api) which can be deployed by customer themselves for their domain | 14:20 |
| gtema | another challenge is the auth of the scim itself | 14:21 |
| gtema | but anyway - this is just necessary | 14:21 |
| knikolla | cool stuff, I did something similar. SCIM is exposed as an API through ColdFront (our self-service project/user management service) and it exposes all projects of a user as groups "members of project X" or Y. And that ties into the part of the code in coldfront that pushes the permissions to create users and role assignments through federated user crud api. | 14:23 |
| knikolla | so it's a layer above all the cloud services that we provide. | 14:24 |
| gtema | yeah - we both have cool ideas. Just need to bring them to the keystone ;-) | 14:24 |
| knikolla | in your use case you want keystone to provide authentication to services external to openstack too? | 14:26 |
| gtema | not really | 14:26 |
| gtema | keystone is far away from being an IdP itself. Ideally this would be possible, but I do not think it is worth of effort | 14:26 |
| gtema | I mean if we want to allow external auth in keystone this requires to reimplement keycloak | 14:27 |
| keekz | yeah we're using dex. | 14:27 |
| keekz | so is it possible to keep the groups permanently? does setting default_authorization_ttl=0 mean no groups, or permanent groups? | 14:28 |
| gtema | if we however would allow serving as flexible oauth provider it would probably do the work. Emit JWT that other services would be able to validate | 14:28 |
| knikolla | keekz: 0 disables persisting groups. that means the user can't create application credentials or trusts with the group membership carried through mapping rules. | 14:29 |
| knikolla | i wouldn't reccomend setting it to permanent (or a very high number) because there is no mechanism to keep things in sync | 14:29 |
| knikolla | if a user is deleted from your idp, or they are removed from the group, keystone wouldn't know | 14:29 |
| knikolla | therefore the user would still have access through the application credential | 14:30 |
| knikolla | you can permanently assign roles or groups to the user though using the CLI or API, this only applies for group memberships carried through mapping rules | 14:31 |
| knikolla | gtema: i see your point, but for most of those user cases that is solved by setting an identity broker in front of keystone, for authN. for authZ I think openpolicy agent or something similar would be amazing. | 14:33 |
| gtema | https://github.com/gtema/oslo.policy.opa - I currently work on implementing tests autogeneration | 14:34 |
| gtema | problematic is (drumbeats...) neutron | 14:34 |
| gtema | I have seen repo from 7 years ago trying to do similar in golang, but I do this accessing policies from the code directly what gives much more possibilities | 14:35 |
| knikolla | and that work would potentially make work for me much easier too, since instead of pushing permissions to keystone every time a role assignment changes for our users, I would push it to OPA and in an ideal world, all our services would use that as a source of truth for authZ. | 14:35 |
| knikolla | that leaves keystone to just be a middleware service that is configurable to fetch information for authN and authZ and issue tokens containing it for consumption by openstack services. | 14:36 |
| gtema | very right - this is exactly what in general I am willing to achieve. Focus on authN in Keystone and leave authZ itself for OPA | 14:37 |
| gtema | OPA gives much more flexibility in regards to fine-granular access beyond roles | 14:38 |
| gtema | eventually we could manage opa policies in keystone | 14:38 |
| gtema | to implement the necessary bundling and central control plane for policies of OpenStack | 14:38 |
| gtema | the keystone policy rule... (full message at <https://matrix.org/oftc/media/v1/media/download/AY9bar41Q05sEJLCf2BlxCmdYLm2_4ofp6zpqkg4Ag6ey9FVhVshbz_tOcgn15ywpTtEgiDB9s2q_eKsHTBFH5tCeVhaadwwAG1hdHJpeC5vcmcvT3VRd2p5a0RaeUtBeU9mY3puQWJOdVl3>) | 14:40 |
| gtema | no human is able to decode it reasonably | 14:40 |
| knikolla | hahaha, indeed | 14:41 |
| gtema | and since I parse it myself I suspect humans that developed this rule missed few necessary brackets so that oslo-policy treats it not as human would read | 14:42 |
| gtema | maybe it is not this rule, but I remember seeing something similar where ordering was not matching the text form | 14:42 |
| knikolla | policy in general is something i feel we could have done so much better | 14:44 |
| knikolla | but really struggled to gain any traction as it required broader openstack implementation | 14:45 |
| gtema | and I guess opa would help us solving this now | 14:45 |
| knikolla | I sort of like what application credentials did, in overlaying on top of policy access_rules | 14:46 |
| gtema | yeah. Sadly you can't alter access rules (i.e. if you did a typo) | 14:47 |
| gtema | and also lot of CSPs nowadays complain about need to have concept of service account, which is something similar to app creds but not bound to a single "human" | 14:47 |
| gtema | of course you can create a new "technical user" and get app creds for it, but i.e. in the case of github workload auth you can't do this reasonably | 14:48 |
| gtema | it's too dynamic | 14:49 |
| knikolla | am i correct in assuming you mean letting your users create service accounts? | 14:50 |
| gtema | yes. And in the federation case (the JWT as desribed in my doc) would be sort of dynamic | 14:51 |
| gtema | but one of the issues with tech accounts is that you can't actually have account without password set | 14:51 |
| gtema | so if you want to have service account and create app creds for it you need to have password set | 14:51 |
| gtema | not even with trusts you can bypass this | 14:51 |
| knikolla | if you know the claims that will be present in the JWT coming from github, you can map them to the correct permissions | 14:52 |
| gtema | yes, but in order for rest of openstack to work there must be also user_id | 14:53 |
| knikolla | you can precreate the user https://github.com/nerc-project/coldfront-plugin-cloud/blob/3cc766c46b0541109df5a8fed3c2e675d2b065f7/src/coldfront_plugin_cloud/openstack.py#L277 | 14:54 |
| knikolla | if you know the unique identifier | 14:54 |
| gtema | for the external workload case (github workflow JWT) there are plenty of combinations of repo/workflow/actor/branch ... Of course you can also pre-create users, but there should be also possibility to reduce this (not every combination require dedicated user). | 14:56 |
| gtema | and if we talk about service accounts and app creds it still does not help - you can only create appcreds after loggining-in as the user | 14:57 |
| knikolla | i see your point | 14:59 |
| knikolla | that's not a use case i've had to deal with | 14:59 |
| knikolla | and... uh... i'll try to spend some time next week to put the federated user crud stuff in keystoneclient and openstackclient | 15:00 |
| gtema | you should not do anything in keystoneclient itself - time is ticking for killing it off. We finalize migration in openstackclient to have rely on sdk only | 15:01 |
| knikolla | in sdk then | 15:01 |
| gtema | cool. Honestly the data model of the user is just awesome: 6 tables with mixed relation types (1to1, 1tomany) and this should be all combined in the single operation | 15:05 |
| knikolla | hahaha :) | 15:05 |
| keekz | @gtema the service account / machine user is exactly what i'm trying to do. also through github. i have a github action which runs terraform to do a provisioning test (create network, servers, etc) in each of our environments. and i'm sure my users (other internal teams) would need/want to do the same | 15:05 |
| gtema | keekz - I know. I want to implement auth from GT just like any other cloud - exchange of JWT to keystone token with required privileges (as configured by the user itself) | 15:06 |
| gtema | sadly precisely as we discuss above data model in keystone now is not flexible enough to allow this easily and requires "foundational" changes | 15:07 |
| knikolla | have to head out for now o/ | 15:09 |
| gtema | cy, was nice to chat with you again | 15:17 |
Generated by irclog2html.py 2.17.3 by Marius Gedminas - find it at https://mg.pov.lt/irclog2html/!