18:00:27 #startmeeting keystone 18:00:27 Meeting started Tue Nov 15 18:00:27 2016 UTC and is due to finish in 60 minutes. The chair is lbragstad. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:00:28 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:00:29 i'M HERE 18:00:30 o/ 18:00:31 The meeting name has been set to 'keystone' 18:00:31 o/ 18:00:31 o/ 18:00:35 Hello 18:00:39 o/ 18:00:54 Hello 18:01:04 o/ 18:01:07 o/ 18:01:08 not a whole lot on the agenda - so we'll give it a few minutes 18:01:15 #link https://etherpad.openstack.org/p/keystone-weekly-meeting 18:01:59 hopefully everyone's having a nice Tuesday 18:03:06 alright - let's get going 18:03:08 #topic Announcements 18:03:13 Ocata-1 being released this week 18:03:15 (sorry to be late) 18:03:36 if you have anything to backport to Newton or Mitaka, now is the time to do it 18:03:42 stevemar is going to be rolling new versions soon 18:03:50 o/ 18:03:59 since Ocata 1 is here - spec reviews need to be in full swing 18:04:20 spec proposal freeze is this week 18:04:47 we've merged a few specs but we have several we still need to decide on as a group 18:04:55 o/ <--- here 18:04:57 o/ 18:04:58 (we can spend any free time on that after the meeting) 18:05:27 o/ 18:05:34 o/ 18:05:37 looks like we have a new contributor for the outreachy program (annakoppad) 18:05:37 hello 18:06:04 so if you see them floating around in -keystone, make them feel welcome :) 18:06:19 lbragstad, ++ 18:06:26 me and raildo will be mentoring her 18:06:33 rodrigods awesome 18:06:49 rodrigods any specifics on what she will be working on? 18:06:50 the current project idea is to have LDAP as backend jobs 18:06:59 rodrigods very cool 18:07:07 rodrigods: what do you mean by"backend jobs"? 18:07:19 just a special gate job that checks per backend? 18:07:25 morgan_, running jobs with LDAP as identity backend 18:07:29 rodrigods i assume jobs that test ldap for identity 18:07:50 lbragstad, ++ 18:08:01 rodrigods: ok, yeah makes sense. just being sure i was on the same page 18:08:01 cool 18:08:21 just fyi - stevemar has been using google to track the work being done this release 18:08:29 he wanted me to advertise the link during the meeting 18:08:30 #link https://docs.google.com/spreadsheets/d/156q820cXcEc8Y9YWQgoc_hyOm3AZ2jtMQM3zdDhwGFU/edit?usp=sharing 18:08:50 if you see anything on there that needs to be updated or fixed, let one of us know 18:09:12 Remember that this week we will be continuing our work with the horizon team on Thursdays 18:09:13 * morgan_ glares at laptop that struggles to load that spreadsheet 18:09:23 Thursday at 20:00 UTC in #openstack-meeting-cp 18:09:59 you can add the ical to your calendar - http://eavesdrop.openstack.org/#Keystone/Horizon_Collaboration_Meeting 18:10:00 we also have a weekly policy meeting starting tomorrow 18:10:05 #link http://lists.openstack.org/pipermail/openstack-dev/2016-November/107137.html 18:10:32 I've documented the reason for it in the mailing list thread and looking forward to hosting it tomorrow 18:10:51 the etherpad for the meeting is here - https://etherpad.openstack.org/p/keystone-policy-meeting 18:11:13 feel free to add content before tomorrow - I'm going to go through and formalize the final agenda tonight sometime 18:11:41 So we need to enable LDAP on the devstack instacnce used for the Gate and check jobs first 18:11:59 ayoung: or make it a flag for the new job 18:12:21 ayoung: it would be fine if the enable ldap is just used for the new backend job(s) not every single gate job 18:12:21 morgan_, does not need to be a new job. THere is asubmission for keystone functional tests. SHould be that job 18:12:28 morgan_, ayoung we also need to check if the LDAP thing is working on devstack 18:12:31 properly working 18:12:58 please don't try and enable it for every single job was my point, unless we are using for every single job (even non-keystone) dsvm 18:13:14 ayoung, the way we test different scenarios is with different jobs 18:13:34 we still have the SQL backend scenario 18:13:47 and the keystone tempest plugin tests don't include user/group tests 18:13:56 so we need to run tempest 18:14:45 we can probably continue this specific discussion offline 18:14:51 rodrigods: you're on the right track 18:14:59 #topic Request id chaining in keystoneclient 18:15:02 breton ^ 18:15:11 i expect you'll need to massage some stuff in devstack, but def good direction 18:15:52 morgan_, ++ 18:15:59 is breton around? 18:16:25 i do not see breton being active. he's in the channel 18:16:37 we'll come back to that if he shows up 18:16:44 #topic Mailing list posts that need attention 18:16:53 #link http://lists.openstack.org/pipermail/openstack/2016-November/017934.html 18:17:00 #link http://lists.openstack.org/pipermail/openstack/2016-October/017912.html 18:17:15 stevemar was curious if anyone would be interested in taking point on responding to those 18:17:53 * morgan_ volunteers ayoung for the inheretance one :P (or henrynash_ ) 18:18:10 what did I inherit? 18:18:34 ayoung: second mailing list link, role inheretance use/question 18:18:35 I think I responded to (at least one of) the question son inheritance 18:18:46 but henrynash_ might have it covered 18:18:51 thought I answered that already 18:18:57 domain vs project roles 18:19:02 …and happy to take all things “inheritance” if that would help 18:19:06 domains need to diwe 18:19:08 die 18:19:14 the multi domain one is pretty dense topic fwiw. 18:19:14 henrynash_ can i make that an action for you then? 18:19:36 yep 18:19:37 #action henrynash_ to respond to http://lists.openstack.org/pipermail/openstack/2016-October/017912.html 18:19:49 thanks henrynash_ 18:19:55 morgan_: only two of the three questions are ours and i think they are not terrible... it's the example that makes it look hard 18:20:36 thanks henrynash_ 18:20:40 dstanek: i am guessing it will be more involved in general based upon the example. 18:21:07 anyone feel like tackling multi-domain identity and security groups - http://lists.openstack.org/pipermail/openstack/2016-November/017934.html ? 18:21:54 lbragstad: i can do an initial response....but i'm guessing there will be lots of followup questions 18:22:03 dstanek awesome - that's just fine 18:22:07 dstanek: tag me in if you need extra eyes on it. 18:22:13 #action dstanek to respond to http://lists.openstack.org/pipermail/openstack/2016-November/017934.html 18:22:26 i'll try and keep an eye on it as well. 18:22:27 morgan_: thx 18:22:52 #topic specs to review 18:23:34 alright - here are a handful of specs proposed to Ocata that we need to reach consensus on 18:23:36 #link https://review.openstack.org/#/c/388886/ 18:23:59 Add keystone project properties ^ 18:24:00 dstanek - you were doing a bunch of noodling on that last week 18:24:03 Yeah, that one is still not on my happy list 18:24:16 I did have some alternative thoughts to that approach 18:24:46 ayoung is the alternative documented anywhere? 18:24:58 ayoung: comment on the Review? 18:25:00 lbragstad: i think there are a lot of hole there....big enough to driver a 4x4 pickup truck through 18:25:20 dstanek, ++ 18:25:28 the "properties" can't be owned by the projects 18:25:32 they need to outside. 18:25:33 dstanek yeah - the nova folks seemed to have some valuable feedback there, too 18:25:39 managed separately 18:25:59 dstanek did you get a chance to follow up on the mailing list? 18:25:59 if we had the idea of a project group, and then you enrolled the project in the group, it would be closer 18:26:25 ayoung then associate properties to the group? 18:26:36 lbragstad, I don't think you would need to 18:26:39 lbragstad: i read the responses, but i don't think they are dealing with my concerns. if you like i can reply with my different objects 18:26:46 s/objects/objections/ 18:26:54 any logic based on the projectgroup membership would still be external to Keystone 18:26:58 dstanek cool - that would be helpful 18:26:58 dstanek please do 18:27:12 gagehugo: you know then already :-) 18:27:25 nova folks might have ideas though 18:28:04 most of my issues are around separation of concerns. who can edit what properties 18:28:24 ayoung if you start enrolling projects into groups, what takes care of the properties? 18:29:02 dstanek: agreed. I also don't like the idea that keystone is the generic arbitrary key-value-store location for all of openstack (little restriction on the properties) 18:29:19 morgan_, ++ 18:29:26 ok...let me see if I can make this clear 18:29:36 lbragstad: each group would be similar to a property (dev, projection, billing-code-x) and if only the owner of the group and put projects in it then my issues are gone 18:29:39 agreed, extra has been a PITA i'd prefer to kill it than formalize it 18:29:41 there is a fixed limit on the maximum size (in mysql) we can hold of arbitrary properties. 18:29:44 lets say one reason they want this is to tag a project by rate 18:29:44 in extra 18:29:51 gold, silver, bronze 18:29:57 morgan_: jamielennox: ++ 18:30:03 gold pays 1000/month, bronze is 1 Month 18:30:07 well we had the idea of limiting keys via config, which would limit others in openstack from using it 18:30:10 ah ha... ok 18:30:13 I see what you mean 18:30:17 setting the rate for a project should not be done by an admin of that project 18:30:22 it should be done external to the project 18:30:27 however that is managed. 18:30:37 gagehugo: i had a proposal that respun extras into something that is managed by the cloud admin in a way that didn't run into limits and allowed indexing 18:30:39 anyone doing that sort of managing is maintaining there own databases external to keystone anyway and they can link it to a non-changing project-id there 18:30:52 gagehugo: it was super complex and still not a great design (this was 2-3 cycles ago) 18:30:53 don't make it Cloud admin's responsibility, though 18:30:58 morgan_ ah 18:31:10 just not the project admins responsibilituy 18:31:21 think of it as a resource loan 18:31:24 jamielennox: this honestly feels like CRM tool (salesforce for example) details 18:31:35 billing data should not be in keystone... 18:31:37 I create this "thing" what ever it is, and then I say "this project can use it" 18:31:40 also codifiing allowed keys means you are making fixed custom additions to the keystone api which is another point of difference between cloud implementations 18:31:42 sorry ayoung ^ not jamielennox 18:31:53 morgan_, yeah, I agree..just that is one of the reasons they want it 18:32:00 I am just illustrating the point with that 18:32:02 gagehugo is billing the main use case you have for project properties? 18:32:03 right 18:32:37 no, not the main use, moreso to keep track of a large # of projects 18:32:55 gagehugo: can you explain the usecase for us? 18:32:59 gagehugo: billing is one of your use cases 18:33:04 gagehugo, that is why we have HMT 18:33:05 gagehugo so like - tracking all projects that are used for development? 18:33:11 would it be sufficient to add a tags column to projects? 18:33:35 jamielennox, that has exactly the same problem I just illustrated 18:33:38 you can tag any number of projects with any tag and handle what you care about externally 18:33:43 dstanek billing is one of the tags, there are a bunch 18:33:45 jamielennox: don't make it a static column, make it a table (many tags to one project) 18:33:52 jamielennox: or similar design 18:34:06 jamielennox: static column runs into character limit issues. 18:34:07 ayoung: right but i'm giving one fixed property, not allowing any arbitrary thing to be added to the db 18:34:44 but impl details aside... 18:34:45 morgan_: i wasn't thinking any form of tag management, just as someone who can modify a project you can add an array of strings 18:35:12 morgan_ we have a large # of projects that have various lifecycles/billing/usertypes/etc. and other than using extras or some homebrew extension, we would like this 18:35:17 again though the difference is you provide one thing supported by keystone, not a general key val store 18:35:18 i think someone in the mailing list pointed out the need for making tags or properties indexable 18:35:29 yeah 18:35:33 lbragstad: i could ressurect my previous proposal 18:35:56 it allowed the data to be indexable, but it's complex relational data. 18:36:00 i don't really like it. 18:36:04 morgan_ sure - want to compare it to the existing one? 18:36:16 let me see if i can find it. 18:36:18 morgan_ that might have been the spec we looked at initially 18:36:32 does this break cross cloud compatibility if every cloud has different metadata? 18:36:39 dstanek possibly 18:36:54 dstanek that's exactly what a few of the nova dev brought up as a concern 18:37:01 libragstad tracking projects for dev yes (sandboxes, things that need to stay up for a long time) 18:37:26 https://review.openstack.org/#/c/190532/ 18:37:28 lbragstad* 18:37:43 that could be the basis (it could be re-spun to be more specific to this case) 18:37:43 gagehugo, you understand my prime requirement for removing the -2? 18:38:11 or take bits of that index-design and apply towards the new spec 18:38:33 ayoung kinda, there seems to be a few concerns 18:39:02 biggest one is security. The property, projectgroup, tag, whatever it is, cannot be managed by the project admin. It must be external to the project 18:39:08 it would split the extras into columns that then relationally map to the project(s). 18:39:18 ayoung why is that? 18:39:18 We really should treat projects as resources managed by other projects 18:39:20 and could handle at least searching. 18:39:34 lbragstad, go rereasd my example 18:39:37 ayoung so that a project admin can't modify the billing code to be something cheaper? 18:39:56 lbragstad, so that projects cannot violate the constraints that are placed on them. 18:40:06 ayoung right - got it 18:40:22 morgan_: you spec would take care of the indexing, but would need to changes for authz 18:40:36 dstanek: right. which would need to be expanded upon 18:40:55 dstanek: i abandoned that design because the view was extras shouldn't receive much love since... ick 18:41:52 my primary concern is that there are different tags/properties/whatever that need different authorizations to use; billing-code is cloud admin, environment is domain admin, etc 18:41:52 i really think this cannot be data attached to the project and extras is insufficient 18:42:07 s/projects/resources in keyustone 18:42:21 it really should be outside of Keystone 18:42:26 or we just say this is only for cloud admins....i bring up the authz issues because one of the documented usecases raised the issue 18:42:59 my opinion is at least tags is structured and a standard cross cloud API, but ideally i'd like to kill extras and make people maintain this stuff elsewhere because surely people are already tracking billing and stuff about projects 18:43:19 jamielennox: ++ 18:43:19 Metadata as a service 18:43:44 Somebody else's problem as a service 18:43:47 jamielennox morgan_ ayoung have you all documented your concerns in the spec? 18:44:20 in all seriousness, this is data that likely should not be directly attached to the resource in keystone 18:44:27 the more i think about it the more it seems to be the wrong architecture to actually put the data in keystone. 18:44:28 lbragstad, I've already -2ed it. No one has come back with a sufficient counter argument. We've discussed it here. The spec should not go ahead. 18:44:47 since those resources are mutable and the data is under the purview of domain/project admins to edit 18:45:01 ayoung right - i see that. i want to make sure we're documenting our concerns in the spec though 18:45:03 most of this data belongs in the CRM tool. 18:45:19 it is application specific data and that should live in the application that requires it 18:45:23 the more we talk about this proposal, the more i think it's trying to solve two problems 18:45:24 ayoung: ++ 18:45:33 1.) tracking large numbers of projects 18:45:35 2.) billing 18:46:02 keystone is defintely a bad place to stick any billing information 18:46:25 ok - so if we take billing out of the equation 18:46:39 how do we feel about using something like tags for tracking large numbers of projects 18:46:42 and without the properties being searchable/indexable the problem 1 is missing basic usablility 18:47:04 We could List projecto and pass tags as hints 18:47:14 That would make it searchable 18:47:20 We would be ok with tags if they are searchable 18:47:36 samueldmq: which would need to be something closer to my above linked proposal 18:47:44 since you can't use "extras" effectively for that 18:47:44 lbragstad: yes, those where the original two usecases 18:47:50 why not use hiearchies to organize the projects? 18:48:27 dstanek most of your security concerns were based on #2, right? 18:48:27 rodrigods: because hierarchies are very limited and projects cannot be moved [if i understand the proposal right] 18:48:40 lbragstad: not sure which one exactly 18:48:42 if you need to tag a project (example) with 10 tags 18:48:42 morgan_ k I need to take a look at that too 18:48:48 is there a spec / proposal on a separate service for metadata? I kind of like where that is going 18:49:01 morgan_, hmm right 18:49:09 chrisplo not that I know of 18:49:11 lbragstad: same concerns apply to organization. who does it the cloud admin, domain admin or other? 18:49:36 dstanek: anyone who has write access to ? 18:49:38 in the spec it seemed cloud admin, but in a public cloud it would have to be domain admin or other 18:50:05 morgan_: exactly the problem with the spec when used for cross domain things 18:50:15 tags have potential to cause issues fwiw in data bloat, etc 18:50:20 lbragstad, HMT is designed exactly for scale of projects 18:50:28 so concerns on maximum data held in the store. 18:50:55 but i am not opposed to tags for resources. simple strings that can be used for identification but no added data 18:50:59 converting it into some tag based or other mechanism while peopkle still don't get HMT is adding confusing to a complex system 18:51:29 but there are still the authz, indexable, searchable, etc concerns. 18:51:59 yeah - it would make sense that if we are working with large numbers of projects, the tags need to be indexable 18:52:10 lbragstad yes 18:52:15 ayoung: that's a good point. the origina usecase can be solved with a domain/{dev,staging,prod}/project structure; unless the project needs to live in two different places 18:52:21 tags could be general though, service+uuid could identify specific items, not limited to projects . . . 18:52:24 dstanek, yep 18:52:35 almost out of time..... 18:52:42 chrisplo: that is why i said . not project specific 18:52:55 yep - I urge folks to leave comments on the review 18:53:01 How big is a big set of projects? 18:53:06 sorry, was dual tasking morgan_ 18:53:12 :) no worries 18:53:18 samueldmq: assume 1000s 18:53:21 because i want to make sure we don't lose the context of our concerns https://review.openstack.org/#/c/388886/ 18:53:45 morgan_ k thanks 18:53:50 chrisplo: metadata as a service is really just redis or a thin shim around it :-) 18:53:53 but i wouldn't assume 10s of 1000s 18:54:12 dstanek: ++ 18:54:30 stevemar had a list of other specs to review in the etherpad 18:54:31 dstanek: though that still suffers from non-index/searchable without a lot of shim magic 18:54:40 #link https://review.openstack.org/396634 (lightweight trusts) 18:54:46 #link https://review.openstack.org/396331 (enhancements to trust scopes) 18:54:52 #link https://review.openstack.org/373983 (improved openid connect support) 18:54:57 #link https://review.openstack.org/345705 (user managed TOTP credentials) 18:55:04 #link https://review.openstack.org/#/c/397410/ (federated attributes for users) 18:55:10 #link https://review.openstack.org/#/c/397860/ (native SAML2) 18:55:17 so - we'll need to make a decision on those soon 18:55:22 lbragstad is spec proposal freeze this week ? 18:55:29 oh i think a spec that was lost was the enforced combined auth-plugin... 18:55:29 samueldmq: y 18:55:32 samueldmq yeah - end of this week i htink 18:55:37 who was working on MFA round 2? 18:55:48 manjeets: wasn't that merged? the spec i mean 18:55:49 ++ let's review :) 18:56:19 dstanek morgan_ yeah - looks like it http://specs.openstack.org/openstack/keystone-specs/specs/keystone/ocata/password-totp-plugin.html 18:56:23 morgan_: that was for you ^ 18:56:36 ugh 18:56:42 most of the trust stuff is suspect 18:56:55 that doesn't really solve the problem 18:57:02 https://review.openstack.org/#/c/391624/ is the main thing 18:57:14 so - if you haven't looked at those specs, please do. I assume stevemar will have them on the agenda next week since we didn't get to them today 18:57:20 should have put a discussion on that into the meeting 18:57:20 it's fine, just not... really addressing needing totp+password vs password vs totp for authn 18:57:56 some users may require password+totp, vs just password. which is a valid use-case 18:58:13 * morgan_ grumbles about being busy and unable to review the specs. 18:58:14 does anyone have any last minute things? 18:58:21 quickly 18:58:28 go morgan_ go 18:58:29 if we want to move the agenda back to the wiki 18:58:31 we can 18:58:37 new users can be created again 18:58:40 i like etherpad 18:58:44 but just think about it 18:58:49 we can bug steve next week 18:58:51 morgan_ sounds good, i'll make a note 18:58:56 morgan_ thanks 18:59:03 anything else? 18:59:13 I am fine either way 18:59:30 Let's think about it and take a decision next week? 18:59:36 samueldmq yeah 18:59:47 alright - i'll give everyone one whole minute back 18:59:55 lbragstad: don't do it! 19:00:01 i'm doin' it! 19:00:04 #endmeeting