17:01:56 <lbragstad> #startmeeting keystone-office-hours
17:01:57 <openstack> Meeting started Tue Jul  3 17:01:56 2018 UTC and is due to finish in 60 minutes.  The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:01 <openstack> The meeting name has been set to 'keystone_office_hours'
17:11:19 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Make keystone.server.flask more interesting for importing  https://review.openstack.org/579928
17:14:17 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Fix keystone.common.rbac_enforcer.__init__.py expoorting  https://review.openstack.org/579930
17:19:15 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Fix keystone.common.rbac_enforcer.__init__.py exporting  https://review.openstack.org/579930
20:15:38 <lbragstad> kmalloc: i worked my way through all flask patches i think
20:42:12 <kmalloc> lbragstad: responded to comments
20:42:26 <kmalloc> lbragstad: specificall the one you -1'd. That is just a double down on policy-in-code
20:42:44 <kmalloc> lbragstad: if a rule isn't defined, we are locked down, it's a safety concern within keystone.
20:43:02 <kmalloc> for security projects (we are one), default closed, open where needed
20:43:18 <lbragstad> i agree about the concern, more or less questioning the backwards compatibility bit?
20:43:36 <kmalloc> i don't think it's backwards incompat
20:43:49 <kmalloc> there is zero reason we have un-accounted for rules with policy-in-code
20:43:55 <kmalloc> or if we do, we should know about it fast
20:45:04 <kmalloc> if we want to reference an action, ensure it is registered
20:45:37 <kmalloc> prior to policy-in-code, the "default open" or "Default closed" was a more reasonable thing to reference
20:46:11 <kmalloc> since the definition of the policy itself was in the policy.json
20:46:32 <kmalloc> so, there was a high likelihood of a non-existant action
20:46:44 <kmalloc> with policy-in-code it is impossible (short of a programming error) for a non-existent action
20:47:07 <kmalloc> (an operator can no longer remove an action from policy.json causing a fallback to the default rule)
20:55:55 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Do not use flask.g imported as g  https://review.openstack.org/579985
20:59:58 <kmalloc> lbragstad, knikolla, wxy: ^
21:11:05 <openstackgerrit> Lance Bragstad proposed openstack/oslo.policy master: Teach Enforcer.enforce to deal with context objects  https://review.openstack.org/578995
21:38:59 <cmurphy> wow this mfa stuff is just completely undocumented http://git.openstack.org/cgit/openstack/keystone/tree/keystone/identity/backends/resource_options.py#n85
21:39:08 <cmurphy> you have to hunt through the code to find out it's there
21:44:04 <lbragstad> yeah - outside of the original patch, i'm not sure the docs were ever amended https://review.openstack.org/#/c/274901/
23:32:36 <adriant> cmurphy: nope, not at all. It doesn't exists outside of the circle of people who know it from the code :P
23:33:27 <adriant> Which honestly is a good thing until we get the auth receipts done, because using it as is was messy anyway
01:19:26 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Delete project limits when deleting project  https://review.openstack.org/538371
01:37:55 <kmalloc> Yeah :(
03:20:16 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Implement enforcement model logic in Manager  https://review.openstack.org/562715
03:20:16 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Expose endpoint to return enforcement model  https://review.openstack.org/562716
03:20:17 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
03:20:17 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
03:20:18 <openstackgerrit> wangxiyuan proposed openstack/keystone master: [WIP] Add show hierarchy filter  https://review.openstack.org/579331
05:22:57 <openstackgerrit> Merged openstack/keystone master: Add registered_limit_id column for limit  https://review.openstack.org/577751
05:36:28 <openstackgerrit> Kristi Nikolla proposed openstack/keystone-tempest-plugin master: Keystone to Keystone tests  https://review.openstack.org/580041
05:38:03 <knikolla> i should be sleeping now, why am i pushing code
06:51:35 <kashyap> lbragstad: Thank you very much
07:30:06 <vigneshwar> Hi hello..
07:30:41 <vigneshwar> i have created a group in keystone.
07:30:53 <vigneshwar> and using that group to rote the keys..
07:31:03 <vigneshwar> but it throws below error..
07:31:08 <vigneshwar> ValueError: Unknown group 'new' in --keystone-group
07:31:19 <vigneshwar> new -- group name i have created..
07:31:36 <vigneshwar> can anyone point me what is the issue..
07:34:28 <wxy> vigneshwar: are you using keystone-manage xx_rotate command?
07:38:13 <wxy> If so, the "--keystone-group" here does not mean the group resource in Keystone, it means the linux group which the keystone process runs with. the same as "--keystone-user"
08:58:57 <vigneshwar> wxy: i tried with the same as user , still it is throwing same issue
09:00:24 <vigneshwar> actually i am just using keystone-manage fernet_setup command,,
09:00:39 <vigneshwar> which is not helpful..returning none..
09:00:58 <vigneshwar> so checked the log , it throws "ValueError: Unknown group 'new' in --keystone-group"
09:08:05 <wxy> vigneshwar: http://paste.openstack.org/show/724962/
09:30:43 <vigneshwar> wxy : thank you very much
09:36:48 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Implement enforcement model logic in Manager  https://review.openstack.org/562715
09:36:49 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Expose endpoint to return enforcement model  https://review.openstack.org/562716
09:36:49 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
09:36:50 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
09:36:50 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
09:36:53 <wxy> vigneshwar: :)
12:45:23 <openstackgerrit> Pavlo Shchelokovskyy proposed openstack/keystone master: Ignore .eggs dir as well  https://review.openstack.org/580154
13:41:59 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
13:42:11 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
13:42:22 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
14:07:02 <openstackgerrit> Stephen Finucane proposed openstack/keystone master: Replace support matrix ext with common library  https://review.openstack.org/527808
18:04:50 <openstackgerrit> Merged openstack/keystone master: Ignore .eggs dir as well  https://review.openstack.org/580154
03:20:46 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
03:20:46 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
03:20:47 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
03:20:47 <openstackgerrit> wangxiyuan proposed openstack/keystone master: [WIP]Update project depth check  https://review.openstack.org/580258
06:55:42 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
06:55:43 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
06:55:43 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
06:55:44 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Update project depth check  https://review.openstack.org/580258
07:27:29 <openstackgerrit> Adrian Turjak proposed openstack/keystone master: Implement auth receipts spec  https://review.openstack.org/572286
09:14:16 <openstackgerrit> Adrian Turjak proposed openstack/keystone master: Implement auth receipts spec  https://review.openstack.org/572286
09:22:57 <openstackgerrit> Adrian Turjak proposed openstack/keystone master: Implement auth receipts spec  https://review.openstack.org/572286
09:33:34 <openstackgerrit> wangxiyuan proposed openstack/keystone master: [WIP]Add project hierarchical tree check when Keystone start  https://review.openstack.org/580331
12:30:26 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project hierarchical tree check when Keystone start  https://review.openstack.org/580331
12:44:00 <knikolla> o/
14:25:07 <gagehugo> o/
14:36:44 <mriedem> riddle me this,
14:36:59 <mriedem> we're debating in #openstack-placement that keystone project/user ids have to be uuids or not
14:37:04 <mriedem> i didn't think they had to be uuids
14:43:30 <ayoung> Anyone interested in co-presenting at the Summit?  I have an idea for a talk. Tentatice title "Pushing Keystone over the Edge" on dealing with the multi-site issues
14:43:43 <ayoung> mriedem, define "have to"
14:44:01 <ayoung> I made them work as DNs back in early LDAP days, but that is yucky
14:44:13 <ayoung> we assign UUIDs or things that look like them to Federated Ids that come in
14:44:26 <ayoung> I wanted them to be sha256 hashes, which are longer
14:45:37 <openstackgerrit> Kristi Nikolla proposed openstack/keystone master: Copy shibboleth logs in v3 functional jobs  https://review.openstack.org/580401
14:46:49 <openstackgerrit> Kristi Nikolla proposed openstack/keystone-tempest-plugin master: Keystone to Keystone tests  https://review.openstack.org/580041
14:51:07 <mriedem> ayoung: is it safe to assume that project and user ids in openstack are UUIDs
14:51:19 <mriedem> or can they be other things based on how the deployment is configured
14:51:32 <mriedem> because a few years ago sdague asserted they don't have to be uuids and some deployments didn't make them uuids
14:51:53 <mriedem> or they encoded domain-specific things in the project id for some deployments
14:55:50 <knikolla> mriedem: for projects, unless they are using their own custom made driver, yes. for users, not. ldap users don't have UUIDs.
14:56:05 <knikolla> bf97c38af9e3a2db2f63190683180b138c57f393a2ebea70287698e1fc427072 | demo
15:04:48 <mriedem> knikolla: ack thanks
15:13:09 <openstackgerrit> Kristi Nikolla proposed openstack/keystonemiddleware master: Document endpoint interface and region behavior  https://review.openstack.org/505396
15:13:29 <openstackgerrit> Kristi Nikolla proposed openstack/keystonemiddleware master: Document endpoint interface and region behavior  https://review.openstack.org/505396
15:18:38 <openstackgerrit> Kristi Nikolla proposed openstack/keystone master: Only upload SP metadata to testshib.org if IDP id is testshib  https://review.openstack.org/545471
15:41:59 <apdibbo> Hi, I am having an issue with Keystone and LDAP, is anyone around who could give me a few  pointers? When active directory users are authenticating through keystone we are getting a 504 timeout. tracing through the logs it looks like it is authenticating against ldap but the clients receive a "ConnectFailure: Unable to establish connection to https://openstack.nubes.rl.ac.uk:5000/v3/auth/tokens: ('Connection aborted.',
15:41:59 <apdibbo> BadStatusLine("''",))"
16:38:36 <openstackgerrit> Stephen Finucane proposed openstack/keystone master: Replace support matrix ext with common library  https://review.openstack.org/527808
18:13:16 <nicodemus_> Hello!
18:13:56 <nicodemus_> I'm trying to configure Keystone as a SP using a third-party IdP
18:15:02 <nicodemus_> but when horizon redirects to the OS-FEDERATION url, keystone logs tht it's "missing entity ID from environment"
18:16:01 <nicodemus_> I'm having trouble understanding exactly how to tell keystone the ID of the entity I want to use... has anyone had such issue?
18:26:32 <cmurphy> nicodemus_: that error could have a lot of different causes but the gist is that horizon is trying to redirect to a keystone federation endpoint but it's failing to go through the apache saml mod which means it's failing to set the right headers in the apache request
18:26:47 <cmurphy> the first thing to check is that remote_id_attribute is set correctly in keystone.conf
18:27:03 <cmurphy> the next thing is to look at the <Location ..> directives in the vhost and make sure they're correct
18:27:26 <cmurphy> and then also check that the OPENSTACK_KEYSTONE_URL in horizon's local_settings.py is correct
18:34:47 <nicodemus_> cmurphy: so when horizon does the redirect to keystone, the request should contain a specific header telling keystone which identity-provider to use? Is that correct?
18:36:36 <cmurphy> nicodemus_: not exactly, when horizon does the redirect to keystone it should be redirecting to one of the paths protected by <Location ...> directives in the apache vhost, and the apache mod will set the needed headers before passing it on to keystone
18:37:51 <nicodemus_> I see. I've configured the remote_id_attribute in keystone.conf as per https://docs.openstack.org/keystone/pike/advanced-topics/federation/federated_identity.html (I'm using mellon, so the attribute is set to MELLON_IDP)
18:39:12 <nicodemus_> but it's unclear the effect that variable would have
18:42:05 <cmurphy> that setting just tells keystone how to process the data that apache is passing to it
18:43:40 <cmurphy> if you're using mellon then you should have some pieces in your keystone vhost that look something like this http://paste.openstack.org/show/725136/
18:44:44 <cmurphy> but the paths need to match the routes you're actually using, for example you may or may not have your keystone using a /identity endpoint and you need to make sure the idp and protocol parts of the path match what you configured
18:47:09 <nicodemus_> I see... so the <Location ...> stanza shouldn't have the MellonSPPrivateKeyFile directives? Those I've configured in the <Location /v3> section
18:49:19 <cmurphy> nicodemus_: no those are correct to have there, for example this has worked for me in the past http://paste.openstack.org/show/725137/
18:49:44 <cmurphy> sorry i just clipped it out because that's usually not the tricky part
18:55:06 <nicodemus_> thanks cmurphy ! That's quite helpful
18:55:33 <nicodemus_> much obliged
18:55:50 <cmurphy> you're welcome, hope you work it out
19:29:55 <nicodemus_> cmurphy: let me ask you yet another question (that might be obvious)
19:30:40 <nicodemus_> in the last paste, there's a <Location...> that goes on the vhost conf on keystone, and another <Location...> that goes on the horizon vhost?
19:31:52 <cmurphy> nicodemus_: no, sorry that comment is misleading, they're all for keystone
19:32:04 <nicodemus_> oh, ok
19:32:44 <nicodemus_> so horizon simply does a redirect, and all the mellon magic happens in keystone
19:33:38 <nicodemus_> do you by any chance know which header would mellon include in the header? I'm trying to validate if mellon is in fact doing something or not
19:36:18 <cmurphy> nicodemus_: I think it will literally be 'MELLON_IDP', and if it's working properly you should be able to see it in the keystone debug logs
19:37:23 <nicodemus_> Got it. Thanks again !!
19:37:37 <cmurphy> np
20:59:31 <openstackgerrit> Kristi Nikolla proposed openstack/keystone master: Copy shibboleth logs in v3 functional jobs  https://review.openstack.org/580401
21:06:25 <nicodemus_> cmurphy: I'm making progress! But still have another doubt regarding the traffic flow
21:07:24 <nicodemus_> I'm being redirected to the SAML host for login, but after using valid credentials there's a "bad request" page waiting for me.
21:07:46 <nicodemus_> Once the SAML host receives the credencials, it is supposed to do a callback to horizon, or to keystone?
21:11:04 <cmurphy> nicodemus_: i have a diagram for you http://www.gazlene.net/demystifying-keystone-federation.html#websso-with-keystone-and-horizon
21:11:13 <cmurphy> it should call back to keystone at that point
21:11:35 <nicodemus_> Wonderful! Many thanks !!
21:11:37 <cmurphy> is the bad request coming from keystone? or horizon? or the idp?
21:11:43 <nicodemus_> It's clear
21:12:03 <nicodemus_> it comes from keystone
21:12:31 <cmurphy> if you turn on insecure_debug = true in keystone.conf it should give you a clear error message of what went wrong
21:16:34 <nicodemus_> I see that I have an encrypted SAML response, but when it's POSTed to keystone on an URL that ends with 'auth/mellon/postResponse' I get the 400 error - bad request. Perhaps Keystone isn't able to decrypt the response?
21:19:07 <cmurphy> it should be able to decrypt it because exchanging the service provider's public key with the identity provider is part of uploading its metadata
21:20:43 <nicodemus_> Perhaps if the user that configures the IdP didn't configure my metadata properly, I should expect a 400 error
21:20:55 <nicodemus_> (I didn't mention that I'm not configuring the IdP)
21:22:15 <cmurphy> you might try setting it up with testshib.org and if you can get that working then you can compare to your idp
21:23:24 <cmurphy> the regular apache logs might have more information on mellon-specific errors if the keystone logs don't have anything
21:23:39 <nicodemus_> certanly, there's an error in the apache logs
21:24:04 <nicodemus_> http://paste.openstack.org/show/725149/
21:27:01 <cmurphy> hmm I've never seen that one, but https://github.com/UNINETT/mod_auth_mellon/issues/112 seems to indicate that something might be wrong in the MellonIdPMetadataFile file
21:27:16 <nicodemus_> Looks like it
21:30:10 <nicodemus_> Do you know if the 'MellonIdP' value in the apache vhost for keystone should be the entityID from the metadata of the IdP?
21:34:04 <cmurphy> I don't think so, I just use "IDP". From the docs I think it's setting the name of the header, as in MELLON_IDP
21:34:40 <cmurphy> it's the name of the header that will have the entityID as its value
22:17:07 <adriant> cmurphy: you still about?
23:03:43 * kmalloc tries to vacation... keeps looking at code
23:07:27 <adriant> kmalloc: I was home sick yesterday... still followed up on code review
23:09:30 <adriant> Trying to keep away from work when stuff is a little time sensitive is hard, and also work/life balance is a thing so many people suck at!
23:40:26 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Flesh out and add testing for flask_RESTful scaffolding  https://review.openstack.org/578190
23:40:36 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Make keystone.server.flask more interesting for importing  https://review.openstack.org/579928
23:40:41 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Fix keystone.common.rbac_enforcer.__init__.py exporting  https://review.openstack.org/579930
23:40:49 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Do not use flask.g imported as g  https://review.openstack.org/579985
00:05:37 <kmalloc> adriant: hm. that is one massive patchset
00:06:04 <adriant> kmalloc: the auth receipts one?
00:06:06 <kmalloc> adriant: not sure if it could have been broken up, but (as guilty as I am of 1000+ line changes right now), that is a lot to review at once.
00:06:07 <kmalloc> yeah
00:06:23 <adriant> I mean, I could break the provider logic and tests into one patch
00:06:33 <kmalloc> that might make it a lot easier to review
00:06:39 <kmalloc> at 2000+ lines of change
00:06:41 <adriant> then add the auth controller logic and tests in another
00:06:47 <adriant> THEN the docs
00:06:49 <kmalloc> yeah and i would prob add the docs in the final one
00:07:07 <kmalloc> if you don't mind, i mean, I'll review it if that is too much work (as is)
00:07:22 <kmalloc> but it also means that if one bit changes you can keep it somewhat isolated.
00:07:44 <kmalloc> i tried to do a lot of that wiht flask code, but sometimes you still end up with 500LOC and 500 lines of test.
00:09:43 <adriant> I don't think it will be too hard to split them...
00:10:01 * adriant says tentatively
00:10:40 <adriant> although the provider patch will still be large
00:10:44 <adriant> since that's the bulk of it
00:11:56 <adriant> yeah, the provider logic (plus conf, plus cli) and tests still looks to be around 1500-2000 LOC
00:13:52 <adriant> docs I can easily split out, and I think doing that is probably not a bad idea, since I can tackle the docs in one or two patch for MFA as a whole, and those aren't exactly going to be a feature freeze issue, so safer to cut those out of the patch that is time sensitive
00:35:06 <adriant> kmalloc: I'll split the docs out, but I'm not sure how else to split the rest nicely when there is a bit of overlap, and most of the controller logic is tiny anyway.'
01:30:37 <openstackgerrit> Adrian Turjak proposed openstack/keystone master: Implement auth receipts spec  https://review.openstack.org/572286
01:30:38 <openstackgerrit> Adrian Turjak proposed openstack/keystone master: [WIP] Add documentation for Auth Receipts and MFA  https://review.openstack.org/580535
01:31:52 <adriant> kmalloc: I've cut the doc changes out at least, but I think I'll leave the giant patch as is other wise :/
01:51:36 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
01:51:36 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
01:51:37 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
01:51:37 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Update project depth check  https://review.openstack.org/580258
01:51:38 <openstackgerrit> wangxiyuan proposed openstack/keystone master: [WIP]Add project hierarchical tree check when Keystone start  https://review.openstack.org/580331
02:41:19 <openstackgerrit> Tuan Do Anh proposed openstack/keystone master: Change "a SQL" to "an SQL"  https://review.openstack.org/579432
02:41:27 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Flesh out and add testing for flask_RESTful scaffolding  https://review.openstack.org/578190
02:43:27 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Make keystone.server.flask more interesting for importing  https://review.openstack.org/579928
02:43:33 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Fix keystone.common.rbac_enforcer.__init__.py exporting  https://review.openstack.org/579930
02:43:39 <openstackgerrit> Morgan Fainberg proposed openstack/keystone master: Do not use flask.g imported as g  https://review.openstack.org/579985
02:43:47 <kmalloc> o
02:43:48 <kmalloc> ok
04:28:02 <openstackgerrit> Kristi Nikolla proposed openstack/keystone master: Fix keystone-manage saml_idp_metadata under python3  https://review.openstack.org/580553
04:29:47 <openstackgerrit> Kristi Nikolla proposed openstack/keystone master: Added keystone identity provider installation to Devstack plugin  https://review.openstack.org/484121
04:30:51 <openstackgerrit> Kristi Nikolla proposed openstack/keystone-tempest-plugin master: Keystone to Keystone tests  https://review.openstack.org/580041
04:32:31 <knikolla> it's too hot to sleep :/
05:48:51 <cmurphy> adriant: sorry I was asleep
06:57:32 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project hierarchical tree check when Keystone start  https://review.openstack.org/580331
09:26:13 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Remove enable config option of trust feature  https://review.openstack.org/580587
09:33:32 <openstackgerrit> Gergely Csatari proposed openstack/keystone master: Clarifications to API & Scenario Tests  https://review.openstack.org/580589
13:42:27 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Remove enable config option of trust feature  https://review.openstack.org/580587
14:00:55 <knikolla> o/
14:44:30 <nicodemus_> Morning!
14:47:16 <nicodemus_> I'm trying to configure Keystone federation, with keystone as an SP and an external IdP. I'm using Mellon for handling the SAML part, but have an error: http://paste.openstack.org/show/725267/
14:47:32 <nicodemus_> Has anyone seen such error before?
16:31:50 <larsks> nicodemus_: have you created an identity provider in keystone?
16:50:07 <nicodemus_> larsks: I did
16:50:23 <nicodemus_> but it turned out that the callback was failing
16:50:28 <larsks> Well, that's everything I know about federation :)
16:51:07 <nicodemus_> rule of thumb, use the same path for the single sign on as in the Mellon endpoint
16:51:35 <larsks> I've recently been tacking openid federation in tripleo and the keystone puppet module.  Mellon is next on my list...
16:51:41 <larsks> s/tacking/tackling/
18:09:24 <kmalloc> adriant: i left a bunch of comments on the receipt patch
18:09:53 <kmalloc> adriant: i skipped reviewing the tests for now. they looked ok-ish, but there are some other changes I'd like to see that may impact the tests.
23:15:55 <openstackgerrit> Gage Hugo proposed openstack/keystone master: Expose random uuid bug in cadf notifications  https://review.openstack.org/580780
01:53:49 <openstackgerrit> Gage Hugo proposed openstack/keystone master: Expose random uuid bug in cadf notifications  https://review.openstack.org/580780
08:14:30 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Remove enable config option of trust feature  https://review.openstack.org/580587
08:50:14 <openstackgerrit> Gergely Csatari proposed openstack/keystone master: Clarifications to API & Scenario Tests  https://review.openstack.org/580589
12:41:36 <hrybacki> o/
12:49:46 <knikolla> o/
12:58:47 <openstackgerrit> Gergely Csatari proposed openstack/keystone master: Clarifications to API & Scenario Tests  https://review.openstack.org/580589
13:03:54 <loicgouarin> Hi, I tried to use kuryr-kubernetes on openstack and I have trouble with keystoneauth1 that I don't understand.
13:04:30 <loicgouarin> I have a config file with the following keystone url  https://keystone.lal.in2p3.fr:5000/v3
13:05:52 <loicgouarin> When kuryr tries to create a keystone clien I have an error which tells me that it is not possible to connect to the url https://keystone-admin.lal.in2p3.fr:35357/v3
13:06:24 <loicgouarin> I don't understand why the url is not unchanged
13:09:03 <loicgouarin> Note that I can create subnet, ... using neutron cli
14:07:39 <gagehugo> o/
14:07:46 <lbragstad> morning
14:12:56 <kmalloc> Mornin
14:17:19 <kmalloc> lbragstad: we need to stop using exception.NotImplemented() for abstract base classes
14:17:42 <lbragstad> kmalloc: and just replace it with pass?
14:17:43 <kmalloc> An http NotImplemented is different than what we are using it for
14:17:45 <kmalloc> No.
14:17:54 <kmalloc> Raise NotImplementedError()
14:18:36 <kmalloc> Http not implemented indicates GET or PUT isn't implemented, NotImplementedError is saying "Python code isn't implemented"
14:18:47 <lbragstad> ahh
14:18:56 <kmalloc> A plain 500 rather than 501
14:19:13 <kmalloc> This is the only case a 500 should be expected in code :)
14:19:16 <lbragstad> it does seem slightly confusing...
14:19:24 <kmalloc> Yeah.
14:19:35 <lbragstad> since the python code is what's implementing the GET/PUT/etc...
14:20:34 <kmalloc> Right, in the cases we don't have a put/post etc, 501 is fine
14:20:35 <lbragstad> would it make a different to someone consuming those APIs?
14:20:55 <kmalloc> But most of these cases we have a put/post/etc and someone failed to write code.
14:21:25 <lbragstad> what's a case where we wouldn't have a PUT/POST/GET/DELETE and should return a 501?
14:21:26 <kmalloc> (except they didn't because abc, but we did it elsewhere and let it bubble up)
14:21:42 <kmalloc> The API spec doesn't implemnt post
14:21:50 <kmalloc> The API is a get/head only.
14:22:04 * lbragstad thought that was always a 404
14:22:19 <lbragstad> but maybe that's wrong
14:22:44 <kmalloc> That is probably wrong.
14:22:55 <lbragstad> it's that how we treat it today?
14:23:00 <kmalloc> Some cases.
14:23:05 <kmalloc> We are inconsistent.
14:23:23 <kmalloc> But the easiest is never raise a 501.
14:23:34 <kmalloc> That is more correct than we do today.
14:23:43 <kmalloc> Esp. in say, read-only backends.
14:24:23 <kmalloc> Read-only backends (catalog) raise 501 on write ops..
14:24:43 <kmalloc> Not a huge deal, just a "hey, this is wrong" and we should be aware of it.
14:24:56 <lbragstad> we should probably write this down in a bug report
14:24:57 <kmalloc> Other things I found when doing flask things.
14:25:14 <lbragstad> i assume flask makes this type of stuff easier to adhere to
14:25:14 <kmalloc> Yeah. On mobile till post coffee. Can write it down after.
14:25:18 <kmalloc> Yep.
14:25:21 <lbragstad> sounds good, thanks
14:25:45 <kmalloc> Flask restful, if we don't implement a get/post/put/whatever method, it 501s.
14:25:53 <kmalloc> Built in. :)
14:28:34 <kmalloc> Also, need to circle up on the policy passthrough, I think we solved the whole reason to support "unknown" rules (passthrough or fail) when we went to in-code.  Someone can no longer remove a line from the policy.json and force a fall-through to the default rule by accident, we fall back on the default in code now.
14:29:34 <kmalloc> Previous to in-code, removing a line from policy.json meant the enforcement action was unknown, and the default "pass/deny" is used. With in-code, an action is never unknown.
14:29:46 <kmalloc> As it has a default registered..
15:09:59 <kimamisa> Hello. I'm facing an issue regarding cache and inherited roles, and I'd like to know if someone already experienced it. I have a role assigned to a user on a domain, with the flag inherited (and also without). When I create a new project in this domain, I expect the role to be assigned on this project, so that when I list the project, I can see the new one created. However, the cache is not disabled, and listing I can't find the new project
15:09:59 <kimamisa> until the cache is expired. I triedwhile disabling the role cache, it works directly. Anyone experienced it ? Is it bug material ??
15:15:38 <lbragstad> kimamisa: it sounds like the project cache needs to be invalidated when the inherited role assignment happens
15:15:49 <lbragstad> kimamisa: does that sound coorect?
15:16:37 <kimamisa> lbragstad: well, the role assignment happened before the project creation in my case
15:17:09 <lbragstad> oh - so the project creation should invalidate the cache then?
15:17:16 <lbragstad> what release are you using?
15:17:38 <kimamisa> lbragstad: I think the role cache should be invalidated when a new project is created AND there are inherited role in the domain
15:18:37 <kimamisa> I'm on queens
15:19:04 <lbragstad> kimamisa: if you'd like to write down that steps you took to recreate in a bug report, you can do that here https://bugs.launchpad.net/keystone/+filebug
15:22:25 <kimamisa> lbragstad: ok. I wanted to check that I wasn't doing anything impossible before reporting a bug. Thanks
15:22:52 <lbragstad> kimamisa: no problem, we can continue to investigate in the bug report
15:42:04 <kimamisa> lbragstad: launchpad found an old bug which points to one of your comments: https://bugs.launchpad.net/keystone/+bug/1780159
15:42:04 <openstack> Launchpad bug 1780159 in OpenStack Identity (keystone) "Some inherited projects missing when listing user's projects" [Undecided,Invalid]
15:44:15 <kimamisa> the bug is exactly what I'm facing. Do you think there is any hope in improving this ?
15:50:48 <lbragstad> kimamisa: hmmmm
15:51:18 <lbragstad> ayoung: is there a reason to not keep https://bugs.launchpad.net/keystone/+bug/1780159 open?
15:51:18 <openstack> Launchpad bug 1780159 in OpenStack Identity (keystone) "Some inherited projects missing when listing user's projects" [Undecided,Invalid]
15:51:38 <ayoung> lbragstad, it was a cache problem
15:51:46 <lbragstad> right
15:52:10 <lbragstad> we don't invalidate the cache in certain inherited role assignment cases
15:52:12 <ayoung> so, cache is going to introduce delay.
15:52:38 <ayoung> ah...you think it should be cache invalidation...ok, keep it open
15:53:03 <ayoung> restored it to the "new" state
15:53:24 <lbragstad> we could go either way with it... but dealing with the invalidation directly is a pattern we have in other places
15:53:46 <lbragstad> kimamisa: in that case, we can reuse that report, can't we?
15:53:57 <kimamisa> yes
15:54:36 <kimamisa> I almost had the same ready !
15:54:49 <lbragstad> cool - setting to medium since the workaround is to set low cache TTL for that specific subsystem
15:55:02 <lbragstad> ayoung: thanks for working that report
16:09:26 <ayoung> Can someone explain K2K to me?
16:09:37 <ayoung> I get SAML.  WHat I don't get is how it keeps assignment data straight
16:09:55 <ayoung> say I have 2 setups,  call em old and new
16:10:22 <ayoung> and I add a project to old.  How does that show up as anything in new without making a direct call to new to create the project?
16:10:58 <openstackgerrit> Merged openstack/keystone-tempest-plugin master: fix tox python3 overrides  https://review.openstack.org/573862
16:11:39 <ayoung> if I want to have a rule that says "anything in old Dom 1 gets mirrored in new Dom 5"  there is nothing that keeps people also from assigning to things in new dom 5.  Fine, I get that
16:12:44 <ayoung> what makes the new Dom 5 project in the first place, or is it just assumed that you will start with some top level sync, like "let old and new each get a set of domains, and we'll explicitly create projects in the remote ones"  so using a domain level assiugnment?
16:31:54 <openstackgerrit> Merged openstack/keystone master: Clarifications to API & Scenario Tests  https://review.openstack.org/580589
16:45:46 <openstackgerrit> Lance Bragstad proposed openstack/oslo.policy master: Teach Enforcer.enforce to deal with context objects  https://review.openstack.org/578995
16:47:05 <openstackgerrit> Lance Bragstad proposed openstack/oslo.policy master: Teach Enforcer.enforce to deal with context objects  https://review.openstack.org/578995
19:24:56 <openstackgerrit> Gage Hugo proposed openstack/keystone master: Add docs for case-insensitivity in keystone  https://review.openstack.org/576640
20:38:45 <lbragstad> kmalloc: might need your eyes on the policy bits here and the @protected stuff https://review.openstack.org/#/c/579330/8/keystone/limit/controllers.py
20:39:02 <lbragstad> context: https://review.openstack.org/#/c/579330/2/keystone/limit/controllers.py
20:39:39 <kmalloc> Headed to the doctor, will look when back.
20:39:44 <lbragstad> ack
20:39:53 <kmalloc> But #1 priority on my list.
20:40:05 <kmalloc> Post non-code things.
20:40:07 <kmalloc> :)
20:42:55 <lbragstad> awesome - thanks
00:17:43 <markguz> anyone know what this error means when syncing config to db in pike :  InvalidDomainConfig: Invalid domain specific configuration: The value of group signing specified in the config should be a dictionary of options.
00:21:22 <markguz> this is the config I;m trying to sync to db: http://paste.openstack.org/show/725379/
01:04:44 <kmalloc> lbragstad: looking now
01:04:50 <kmalloc> lbragstad: just got back from appt...
01:18:00 <kmalloc> lbragstad, wxy: sorry -1, we cannot hard-code role names. The answer is (any option works): 1) Prohibit Filtering on project_id; 2) check service-token only (mostly insufficient as anyone can provide a service token; 3) use callback and make your own cred/target dict that can confirm "project_id" filter is there (pass directly to the enforcer); 4) Wait for oslo_policy and new check types (Stein release); 5) Maybe
01:18:00 <kmalloc> doable via backflips i can add in Flask-RBAC Enforcer.
01:39:27 <adriant> kmalloc: given the refactor work in progress for tokens, I'm tempted to push the auth receipt stuff to Stein until the refactor is done for tokens. Then I can sync up with what we end up with so the receipt and token patterns are the same.
01:41:29 <adriant> None of the other pieces for MFA with receipts would be ready until Stein anyway, so it's not a huge issue, and now that I have something that works with the API side mostly defined I can start the work in keystoneauth + elsewhere so it's ready once auth receipts is merged into master during Stein
01:43:24 <kmalloc> Wfm
01:43:42 <adriant> lbragstad: that work for you? I've started working through all your token provider refactors, and once those are close to done can sync my stuff with them.
01:44:19 <adriant> kmalloc: seems like the most sensible solution rather than me inventing a third pattern for auth receipts when it should really match tokens.
01:45:12 <kmalloc> ++
02:05:49 <wxy> kmalloc: ++ for the policy enforcer way. I tried it in my first patch, but it doesn't work as I thought. So I really like your proposal during last weekly meeting. I want to wait for new oslo_policy, but without "project_id" filter, the "show_hierarchy" filter will be blocked as well (https://review.openstack.org/#/c/579331/8/keystone/limit/core.py@131). So I'd like choose 2) in R, and continue 4) in S.
02:06:18 <kmalloc> Sounds good.
02:07:11 <kmalloc> The only reason I could build the enforcer (custom) is because I've just been working heavily in the enforcer ;)
02:08:12 <kmalloc> I might make this doable with moving to flask.
02:08:36 <kmalloc> FYI, but need to land the bits before I move apis to flask.
02:08:46 <wxy> kmalloc: I'll dig into your flask-enforcer patches today.
02:09:06 <kmalloc> It's mostly ready, just needs some extra +2s
03:18:40 <kmalloc> wxy: we might need a separate api for security reasons if the reason we limit project_id filter is security
03:19:01 <kmalloc> wxy: because anyone could provide a service token, it just wont be useful for other service-only things
03:19:13 <kmalloc> wxy: lets chat w/ lbragstad tomorrow and come to a good solution :)
03:24:59 <openstackgerrit> Adrian Turjak proposed openstack/keystone master: Implement auth receipts spec  https://review.openstack.org/572286
03:25:17 <wxy> kmalloc: then a new policy like "limit_limits_with_project_id: role: admin or service"?. I thought it's a little heave so I dropped this choice when coding. But I'm ok with it if we like it.
03:38:32 <adriant> kmalloc, lbragstad: auth receipts marked as -1 workflow for now then, and I've added myself to all reviews for https://bugs.launchpad.net/keystone/+bug/1778945 so I can keep an eye on how they shape up, will at some stage start work on KeystoneAuth using the existing auth-receipts patch as the API interaction is unlikely to change.
03:38:32 <openstack> Launchpad bug 1778945 in OpenStack Identity (keystone) "Complexity in token provider APIs" [Medium,In progress] - Assigned to Lance Bragstad (lbragstad)
03:39:02 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
03:39:02 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
03:39:03 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
03:39:03 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Update project depth check  https://review.openstack.org/580258
03:39:04 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project hierarchical tree check when Keystone start  https://review.openstack.org/580331
03:39:04 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Filter project_id for list limits  https://review.openstack.org/581177
03:39:42 <kmalloc> adriant: ok
03:40:18 <kmalloc> wxy: yeah, let's chat tomorrow. But that might be the path forward, but I want to check a couple things.
03:40:26 <kmalloc> I have some ideas.
03:40:33 <wxy> kmalloc: cool
04:08:37 <lbragstad> adriant: ok - that sounds fine
04:08:53 <lbragstad> to be fair - the rest of that refactor shouldn't take too long
04:09:07 <lbragstad> it's just a bit of testing concerns and that should be about it
04:25:30 <lbragstad> wxy: did you say you were waiting for a new version of oslo.policy?
05:07:53 <kmalloc> lbragstad: the "does element in Target dict" check I proposed n
05:08:02 <kmalloc> Right now we can't do that at all.
05:12:01 <lbragstad> ah
05:12:56 <kmalloc> it could be done with a callback or custom call to enforce
05:13:04 <kmalloc> it would be wonky/difficult
05:13:13 <kmalloc> flask work makes it easier, but not "easy"
05:13:26 <kmalloc> a "does element exist in target" would make a big difference
06:35:29 <openstackgerrit> Vu Cong Tuan proposed openstack/python-keystoneclient master: Switch to stestr  https://review.openstack.org/581213
07:23:11 <openstackgerrit> Vu Cong Tuan proposed openstack/pycadf master: Switch to stestr  https://review.openstack.org/581228
07:32:53 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Strict two level limit model  https://review.openstack.org/557696
07:32:54 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project_id filter for listing limit  https://review.openstack.org/579330
07:32:54 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add show hierarchy filter  https://review.openstack.org/579331
07:32:55 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Update project depth check  https://review.openstack.org/580258
07:32:55 <openstackgerrit> wangxiyuan proposed openstack/keystone master: Add project hierarchical tree check when Keystone start  https://review.openstack.org/580331
07:37:22 <wxy> lbragstad: yes if oslo.policy can check the request filter in policy way like kmalloc mentioned during last weekly meeting. But we can't just wait, we should find another solution in R.
08:37:23 <openstackgerrit> Vu Cong Tuan proposed openstack/python-keystoneclient master: Switch to stestr  https://review.openstack.org/581213
09:34:14 <openstackgerrit> Merged openstack/keystone master: Implement enforcement model logic in Manager  https://review.openstack.org/562715
10:14:56 <openstackgerrit> Vu Cong Tuan proposed openstack/ldappool master: Switch to stestr  https://review.openstack.org/581307
11:23:28 <openstackgerrit> Vu Cong Tuan proposed openstack/keystone-specs master: Switch to stestr  https://review.openstack.org/581326
11:36:10 <openstackgerrit> Vu Cong Tuan proposed openstack/keystone-specs master: Switch to stestr  https://review.openstack.org/581326
12:29:26 <openstackgerrit> Sami Makki proposed openstack/keystone master: Invalidate 'computed assignments' cache when creating or deleting project.  https://review.openstack.org/581346
14:56:32 <kmalloc> wxy: interesting.
14:57:21 <kmalloc> wxy: so i think the best plan is a service API that is locked to a better rule for filtering by project_id.
14:57:32 <kmalloc> wxy: rather than a list interface.
14:57:44 <kmalloc> wxy: that way we have a separate action that can be controlled.
14:57:59 <kmalloc> wxy: i slept on the whole design and the "change behavior based upon filter" seems weird.
14:58:03 <kmalloc> lbragstad: ^ cc
14:58:49 <kmalloc> if we isolate that API to a separate URL, we have a lot more control over it. and we can do explicit checks.
14:59:58 <kmalloc> wxy: even if we could "check if filter is present" in this case, i'd advocate for a separate API, as it's a different concern/result/set of business logic to be run (different use-case)
15:01:43 <lbragstad> yeah...
15:01:46 <lbragstad> i think i agree
15:01:53 <lbragstad> looking at this while context switching though
15:04:23 <kmalloc> i just wanted to drop that in before meeting today
15:24:07 <wxy|_> kmalloc: lbragstad: so maybe a url like projects/{project_id}/limits to fetch the specified project limits?
15:25:36 <lbragstad> as in the hierarchy?
15:26:29 <kmalloc> wxy|_: that would work, we could also put it under the /limits/ URL prefix somehow.
15:26:49 <kmalloc> but not sure how to adhere to the "rest-ish" style
15:27:10 <lbragstad> yeah...
15:27:28 <kmalloc> i would prefer to keep it in /limits, but seeing as i don't have a good way to do that.
15:27:39 <kmalloc> short of /limits/by-project-id/{projet_id}
15:27:44 <kmalloc> and i don't like that.
15:27:47 <lbragstad> would you want to denote the project with the token scope?
15:28:03 <kmalloc> this is the service user getting limits for a project
15:28:12 <kmalloc> though... i guess we could use x-subject-token here?
15:28:18 <kmalloc> as an alternative
15:28:41 <kmalloc> get it by x-auth-token, or x-subject-token
15:28:53 <kmalloc> but that runs afoul of the same issues with security
15:29:12 <kmalloc> hard to limit (new check_str), but x-subject-token we could at least check for today.
15:29:30 <kmalloc> oh.. wait. no we can't...
15:29:31 <kmalloc> ugh
15:29:51 <lbragstad> so - if you're an admin on a project
15:30:06 <lbragstad> let's say you have project A, B, and C
15:30:21 <lbragstad> and they are in a linear relationship
15:30:33 <lbragstad> (ignore the two-level enforcement model requirement for now)
15:30:54 <kmalloc> we have another problem
15:31:03 <kmalloc> we have a hard-coded role name that is landed in tree
15:31:33 <kmalloc> https://github.com/openstack/keystone/blob/master/keystone/limit/controllers.py#L105
15:31:39 <kmalloc> we need to fix that as well.
15:31:59 <lbragstad> yeah
15:32:03 <wxy|_> kmalloc: ++
15:32:16 <kmalloc> we need a new API, that the services can work from.
15:32:27 <kmalloc> we can't keep this merged together like it is.
15:32:38 <kmalloc> list limits is locked to your current scope.
15:33:15 <kmalloc> if you need external to your current scope (global or other wise) we need APIs that can do that so we can lock them down how we want with correct check strings.
15:33:41 <kmalloc> user vs admin vs cloud-admin uses
15:34:04 <wxy|_> the original purpose for list limits API is that non-admin user in a project can only fetch the limits belong to this project. and admin can fetch all the limits
15:34:24 <kmalloc> right. so lets make a new API for the admin interactions
15:34:31 <kmalloc> either under /project
15:34:36 <kmalloc> or something new under /limits
15:34:39 <lbragstad> well - the good thing about it being experimental is that we can break the functionality that requires elevated authorization out
15:34:46 <kmalloc> lbragstad: ++
15:35:03 <kmalloc> it *sounds* like we need something under /limits
15:35:21 <lbragstad> but... we do have a project API for listing hierarchies already
15:35:31 <kmalloc> we could add per-project limits [admin, filtered to project] under /project/{project_id}/limits
15:35:35 <kmalloc> right.
15:35:42 <kmalloc> that doesn't list/emit limit data
15:35:47 <lbragstad> yeah
15:35:57 <kmalloc> there are 3 concerns here:
15:36:15 <kmalloc> 1) User getting list for current scope ( /v3/limits <-- list)
15:36:24 <kmalloc> 2) admin getting *all* limits (cloud-admin)
15:36:43 <kmalloc> 2a) admin getting *all* hierarchy-limits (project/domain admin)
15:36:58 <kmalloc> 3) project-specific "get limits" [filtered by project]
15:37:11 <kmalloc> i am unsure how this breaks down API, 2a/3 might be one thing.
15:37:28 <kmalloc> but 1 and 2 are clearly distinct actions.
15:37:43 <lbragstad> is 1 expecting a hierarchy?
15:38:00 <lbragstad> or just limits for a single project?
15:38:08 <kmalloc> unsure.
15:38:12 <kmalloc> wxy|_: ^?
15:38:20 <knikolla> o/
15:38:23 <wxy|_> i think just limits
15:38:31 <kmalloc> no hierarchy, just current scope.
15:38:32 <kmalloc> ok
15:38:34 <kmalloc> that is fine.
15:38:37 <wxy|_> hierarchy information should not exposed to common users
15:38:38 <lbragstad> so - as an end user
15:38:53 <kmalloc> yeah, that is what it looks like from the base code impl
15:38:58 <lbragstad> i just need to get limits for a project i'm working on
15:39:10 <lbragstad> i shouldn't be able to query all the hierarchy information
15:39:14 <lbragstad> pertaining to limits
15:39:19 <kmalloc> not with that api
15:39:29 <lbragstad> ok - so we need a separate api for domain/project administrators
15:39:32 <kmalloc> "what is my current set of expected limits"
15:39:47 <lbragstad> to be able to query for the limits of a tree
15:39:59 <lbragstad> which is 2a?
15:40:03 <kmalloc> yep, which an end-user may be granted permission to do.
15:40:04 <kmalloc> yes.
15:40:07 <kmalloc> that is 2a
15:41:37 <lbragstad> so - this really sounds like https://bugs.launchpad.net/keystone/+bug/1750660
15:41:37 <openstack> Launchpad bug 1750660 in OpenStack Identity (keystone) "The v3 project API should account for different scopes" [High,Triaged]
15:41:41 <lbragstad> or very similar to it
15:41:43 <kmalloc> sortof.
15:44:06 <wxy|_> are 2) and 2a) the same API? their response structure is different.
15:44:14 <kmalloc> you have "get my current scope limits" and "get me a hierarchy" ... that is not system vs project scope
15:44:37 <lbragstad> the
15:44:39 <lbragstad> "
15:44:42 <lbragstad> bah...
15:45:00 <kmalloc> get me current scope vs get me all, might be project vs system scope
15:45:11 <lbragstad> the "get me a hierarchy" part should take scope into account (e.g. a domain/project admin asking for the hierarchy versus a system admin)
15:45:18 <kmalloc> ah ++
15:45:20 <kmalloc> yes
15:45:26 <kmalloc> so 2 and 2a *are* the same thing
15:45:29 <lbragstad> but yeah - the other one sounds like a different API
15:45:29 <kmalloc> just scope-specific
15:46:24 <lbragstad> right now we have GET /v3/limits/
15:46:33 <lbragstad> GET /v3/limits *
15:47:22 <lbragstad> and we don't have the hierarchical API, yet...
15:47:24 <lbragstad> right?
15:47:32 <lbragstad> that was something we were going to do with the query parameter?
15:47:43 <lbragstad> query string*
15:47:46 <wxy|_> yeah, show_hierarchy
15:48:29 <kmalloc> overloading via a qp and totally changing the output structure seems like the wrong choice
15:48:40 <kmalloc> a qp should maintain the same output format
15:49:02 <kmalloc> but apply rules to it (e.g. expired_ok, filter by x [where not a security issue])
15:49:30 <kmalloc> if we are emitting a totally different structure (if show hierarchy is embeded in the current structure, that is one thing)
15:49:34 <kmalloc> it should be a different api
15:49:41 <lbragstad> what if we added a query string to the project API that aggregated limit information?
15:50:39 <lbragstad> so - everything we have to build the hierarchy of projects stays in the same spot, but it just populates limit information
15:50:49 <kmalloc> sure
15:51:06 <kmalloc> that is fine, populate the limit data in the "get_hierarchy" data api already
15:51:07 <lbragstad> this is an invite to poke holes :)
15:51:36 <kmalloc> as long as it doesn't meaningfully change the output (limit data may be added in the project output, maybe we don't need a qp for that at all, just do it?)
15:52:08 <kmalloc> s/output/output structure, e.g. no wildly different data struct
15:52:13 <lbragstad> if we add a query parameter, we could signal that it is experimental though, couldn't we?
15:52:22 <kmalloc> ah. opt-in sure
15:52:37 <kmalloc> and when we move away from expirimental, just make it the default?
15:52:39 <lbragstad> just in case we decide we don't like it
15:52:45 <kmalloc> since we ignore un-used qps.
15:52:59 <kmalloc> wfm.
15:53:14 <kmalloc> so that solves the "get me the hierarchy" one.
15:53:17 <kmalloc> we have the end-user one today
15:53:34 <kmalloc> and the heierarchy one solves the system vs project/domain admin scope
15:53:44 <lbragstad> sure - because that should take token scope into account
15:54:01 <kmalloc> do we need a "filtered by project" one e.g. /projects/{project_id}/limits api?
15:54:05 <kmalloc> is there a use for that api?
15:54:06 <lbragstad> and GET /v3/limits should take token scope into account, too
15:54:24 <lbragstad> but it won't return a hierarchy, right?
15:54:53 <kmalloc> it cannot return a hierarchy unless the end user one also does (or we embed the hierarchy info somehow, don't embed it awkwardly imo)
15:55:18 <lbragstad> i'm fine with only supporting one way to get the hierarchy
15:55:46 <lbragstad> do we have a /projects/{project_id}/limits API today, or is it just GET /v3/limits ?
15:56:02 <wxy|_> we only have /v3/limits now.
15:56:09 <lbragstad> ok - cool
15:56:21 <lbragstad> so - that API should take token scope into account (eventually)
15:56:31 <kmalloc> do we need /projects/{project_id}/limits ?
15:56:45 <kmalloc> i don't think we do with the hierarchy bits.
15:56:58 <lbragstad> i'm inclined to say no, but i'm willing to be convinced otherwise
15:57:24 <lbragstad> beacuse GET /v3/limits will take the project from the token context, if present
15:57:31 <lbragstad> and filter the response accordingly
15:57:32 <kmalloc> wfm
15:57:46 <wxy|_> kmalloc: what if a cloud admin want to fetch a specified project's limits?
15:58:10 <lbragstad> with a system scoped token?
15:58:19 <kmalloc> wxy|_: i'm ok with adding filters for systme-scope as long as we don't change the format of the end-user get
15:58:22 <kmalloc> which should be a list anyway
15:58:58 <kmalloc> so end-user list: [{limit, inc. project_id}, {limit, inc. project_id}, {limit, inc. project_id}}
15:59:02 <wxy|_> then we don't need /projects/{project_id}/limits
15:59:26 <kmalloc> system-scope would be [{limit, inc. project_id-1}, {limit, inc. project_id-2} ...]
15:59:44 <kmalloc> and ?project_id=XXX would limit the returned output/filter
16:00:13 <kmalloc> for consistency, we could allow end-users to filter and return an empty list if they filter for a project outside of their current scope
16:00:41 <lbragstad> hmmm
16:01:08 <lbragstad> i'm not sure i'd want to allow someone to get limits for a project without using a token scope though?
16:01:18 <lbragstad> either way - we can pick this up after the meeting
16:01:24 <lbragstad> or raise it in open discussion :)
16:01:59 <kmalloc> lbragstad: i'm not advocating anything of the sort, limits would require a token scope (project, my current scope), (system, all, filterable)
16:02:30 <lbragstad> ok - so the query string would simply be there to be consistent
16:02:41 <kmalloc> but for consistency, filters might need to work on the project-scope as well. just if you filter for a project that isn't your current scope, the returned list is empty
16:02:54 <kmalloc> because you said "filter for project X" and X isn't there
16:02:55 <lbragstad> ok
16:02:57 <kmalloc> :)
16:03:19 <lbragstad> and if they filter by the project they are scoped to, it's a noop
16:03:24 <kmalloc> exactly
16:03:34 <lbragstad> ok - that seems reasonably safe
16:03:34 <kmalloc> keep the filtering behavior VERY consistent
16:03:53 <wxy|_> v3/limits for end users by default, /v3/limits?project_id=xxx works for system-scope only. end user will get empty response if using project_id filter.
16:04:10 <wxy|_> right?
16:04:23 <kmalloc> wxy|_: if project_id differs from the current scope, the list would be empty
16:04:32 <ayoung> Meeting now?
16:04:34 <kmalloc> in the non-system-scope
16:04:35 <kmalloc> adriant: yes
16:04:38 <kmalloc> ayoung: ^
16:04:41 <kmalloc> yes.
16:05:08 <kmalloc> wxy|_: basically in the non-system scope we just do "project_id == filtered_project_id_qp or []"
16:05:52 <kmalloc> in the system-scope we actually filter. behavior should be the same in both scopesso you can't accidently filter for project X and get project Y regardless of project vs system scope
16:49:11 <wxy|_> kmalloc: lbragstad: I add some note here about the filter https://etherpad.openstack.org/p/limits-filter
16:49:37 <wxy|_> if it works, I'll complete the patch tomorrow.
16:49:47 <lbragstad> wxy|_: aweosme
16:50:35 <kmalloc> wxy|_: fantastic!
16:52:31 <tosky> uh, I have a question about service_token_roles_required; recently openstack-ansible-os_nova switched the value to true for nova, and that lead to an error in Sahara tests
16:52:54 <tosky> I put together my findings in the last comment (before recheck) of patchset 7 here: https://review.openstack.org/#/c/569886/
16:53:02 <kmalloc> ayoung: man, python 3.7 is going to be NICE.
16:53:09 <tosky> but I'm not sure if how this can be fixed on the sahara side
16:53:18 <tosky> or if it's just a deployment issue
16:53:29 * kmalloc is somewhat said we don't get to rely on 3.6/3.7 things
16:53:40 <kmalloc> lbragstad: btw, we can drop/should drop the v3-only test
16:53:47 <kmalloc> lbragstad: since... we only have v3 now
16:53:50 <kmalloc> or vice-versa
16:53:56 <kmalloc> drop the non-v3-only version
16:54:14 <lbragstad> ?
16:54:33 <kmalloc> lbragstad: we have a gate job.. v3-only
16:54:38 <kmalloc> or had at least
16:55:17 <kmalloc> lbragstad: ah nvm, looks like it is gone
16:55:19 <kmalloc> woo
17:00:36 <openstack> lbragstad: Error: Can't start another meeting, one is in progress.  Use #endmeeting first.
17:01:15 <kmalloc> lbragstad: did you forget to end the last office hours?
17:01:26 <kmalloc> #endmeeting