18:00:50 <ayoung> #startmeeting keystone
18:00:51 <openstack> Meeting started Tue Dec  4 18:00:50 2012 UTC.  The chair is ayoung. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:00:51 <dwchadwick> FYI we have just updated the attribute mapping blueprint
18:00:52 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:00:54 <openstack> The meeting name has been set to 'keystone'
18:01:16 <ayoung> #link http://wiki.openstack.org/Meetings/KeystoneMeeting
18:02:24 <dwchadwick> agenda is fine for me
18:02:26 <ayoung> Robot Rollcall
18:02:35 <ayoung> heckj, you in today?
18:02:38 <heckj> here!~
18:02:41 <heckj> Sorry, was reading bugs
18:02:43 <heckj> :-)
18:02:51 <openstack> heckj: Error: Can't start another meeting, one is in progress.
18:02:54 <ayoung> That is one thing you never need apologise for
18:03:04 <ayoung> heckj, already started
18:03:05 <heckj> Oh - you already did it, sweet!
18:03:07 <dolphm> lol
18:03:12 <dwchadwick> You should apologise for creating bugs :-)
18:03:17 <heckj> heh
18:03:18 <ayoung> dwchadwick, that would be me
18:03:19 <heckj> never
18:03:28 * dolphm apologizes
18:03:29 <ayoung> and I don't mean logging them
18:03:48 <ayoung> #topic High priority bugs or immediate issues?
18:03:50 <heckj> felt bad after henrynash asked me about a triaged bug and realized I hadn't done anything to triage in weeks
18:04:02 <heckj> 8 bugs outstanding to be triaged - working on that today
18:04:25 <dolphm> heckj: aren't you supposed to be on holiday?
18:04:26 <heckj> Haven't made any progress on the memcache thing and repro work - not hard, just haven't been able to dedicate the time as yet
18:04:31 <heckj> Starting tomorrow
18:04:49 * heckj shuts up now and let's Adam run this
18:05:12 <ayoung> heckj, actually, since we were talking Bugs, anything on the new list that is burning?
18:05:22 <ayoung> Or recently triaged?
18:05:34 <heckj> just now going through them
18:05:42 <ayoung> auth_token failure if signing_dir not specified running under upstart.....not a bug
18:05:53 <ayoung> Length of user_name in database too short for X.509 DNs  that is mine
18:06:00 <heckj> The only burning one is the periodic memcache failure due to eventlet race conditions and memcache clients at high concurrency
18:06:08 <heckj> (that I know of)
18:06:31 <ayoung> heckj, OK, so the long term solution for that one is the swift memcache ring, right?
18:06:33 <gyee> swift memcache ring impl seem pretty solid
18:06:56 <heckj> ayoung: yep - Alex Yang was getting that into openstack common - may need a "boost" to do so, then using it from there
18:07:01 <dolphm> heckj: who's working to get the solution into oslo? (i believe that's the blocker on a fix, no?)
18:07:10 <dolphm> heckj: (spoke too soon!)
18:07:21 <heckj> dolphm: fellow (that I don't really know) who commented on the bug
18:07:37 <heckj> dolphm: he opened a blueprint against oslo - looks like he made traction for a while, then disappeared or lost interest
18:08:02 <ayoung> heckj that is https://bugs.launchpad.net/keystone/+bug/1052674  right?
18:08:03 <uvirtbot> Launchpad bug 1052674 in keystone "Keystone auth_token middleware should support Swift global memcache" [High,Triaged]
18:08:09 <gyee> auth_token used to take env['swift.cache']
18:08:19 <gyee> did I file that one?
18:08:39 <heckj> actaully, I think there's a separate bug there - this might be a dupe
18:09:03 <gyee> trivial fix
18:09:05 <gyee> I can do it
18:09:50 <ayoung> #link https://bugs.launchpad.net/keystone/+bug/1052674
18:09:52 <uvirtbot> Launchpad bug 1052674 in keystone "Keystone auth_token middleware should support Swift global memcache" [High,Triaged]
18:09:56 <heckj> #link https://bugs.launchpad.net/keystone/+bug/1020127
18:09:58 <uvirtbot> Launchpad bug 1020127 in keystone "proxy-server Error: Second simultaneous read or write detected" [High,In progress]
18:10:20 <heckj> same intent, different bugs
18:10:41 <ayoung> gyee, can you own this from the Keystone side?
18:10:49 <gyee> sure
18:11:11 <ayoung> OK,  other big issues?
18:11:36 <ayoung> https://bugs.launchpad.net/keystone/+bugs?search=Search&field.status=New
18:12:20 <gyee> btw, I upgrade pep8 to 1.3.3 and it started to flag a bunch of new pep8 errors
18:12:24 <gyee> upgraded
18:12:40 <gyee> in both keystone and keystoneclient
18:12:53 <ayoung> gyee, open ticket.  We'll either fix or we'll ignore
18:13:13 <gyee> question is which version is correct? :)
18:13:28 <gyee> no errors from older versions of pep8
18:13:35 <heckj> trying to figure out how to link https://bugs.launchpad.net/keystone/+bug/1082050 to the Ubuntu packagers crew
18:13:36 <uvirtbot> Launchpad bug 1082050 in keystone "Spelling errors reported on IRC" [Undecided,New]
18:13:50 <gyee> mostly choking on line continuation
18:14:40 <ayoung> OK, lets move on
18:14:55 <ayoung> lets keep open discussion for the end
18:15:03 <ayoung> #topic Groups vs. attribute mappings
18:15:31 <henrynash> ok, so last week we approved the groups bp….and so I am starting implmentation
18:15:45 <ayoung> dwchadwick, I'm guessing which side of this discussion you are going to take...
18:15:52 <henrynash> david & I have had some conversations and Kirsty and I need to have some more
18:15:54 <dwchadwick> Henry raised an issue that the attribute mapping blueprint did not mention tenants so that has been fixed now. A new version was published just before the meeting
18:16:33 <dwchadwick> Kristy is not starting the implementation. We plan to have the first version available by the end of the year
18:16:51 <ayoung> "not"  should be "now"  right?
18:17:00 <dwchadwick> If Henry makes the changes he needs to use this for groups it should make both our works easier
18:17:03 <henrynash> dwchadwick: so have been looking at that…perhaps the thing that is missing is what actually going to use the attributes to enforce permissions
18:17:34 <ayoung> soyeah, that is going to require buy in outside of Keystone
18:17:37 <henrynash> i.e. what changes are needed elsewhere in keystone
18:17:57 <ayoung> We want to use the policy engine to support ABAC
18:18:10 <ayoung> and policy engine is going to live in Oslo
18:18:14 <henrynash> I would think so
18:18:28 <dwchadwick> We also have an open source policy engine that we can use
18:18:29 <ayoung> So I think the right approach is like what we did for PKI
18:18:58 <ayoung> make it work, but disabled by default.  Make is a "seamless change" and switch the default in the H timeframe
18:19:10 <henrynash> ayoung: +1
18:19:27 <ayoung> dwchadwick, is that engine in Python?  It will be a non-starter if it isn't
18:19:38 <dwchadwick> no, its in java
18:19:42 <gyee> haha
18:19:52 <ayoung> dwchadwick, so was Keystone originally
18:19:57 <dwchadwick> but we have a python interface
18:20:01 <heckj> ayoung: excellent idea
18:20:04 <gyee> and it uses xml too? :)
18:20:08 <ayoung> heh
18:20:15 <dwchadwick> yea
18:20:23 * heckj hangs his head
18:20:26 <ayoung> dwchadwick, well, we can use it as the reference implementation,
18:20:40 <dwchadwick> exactly. but back to attribute mapping
18:20:41 <ayoung> same kind of thing was an issue when talking about using Dogtag for certificates
18:21:04 <dwchadwick> Henry, I did not follow your comment
18:21:30 <henrynash> ayoung: I do like the idea of getting the attribute implementation in there, finding where we need to do the hooks, but disabled by defualt
18:21:37 <ayoung> dwchadwick, you mean "what actually going to use the attributes to enforce permissions"
18:21:48 <henrynash> dwchadwick: sorry :-) which one?
18:22:02 <dwchadwick> henry. Ok, now I follow. You are talking about the PDP
18:22:21 <ayoung> dwchadwick, please expand your acronyms
18:22:30 <dwchadwick> But the attribute mapping design was specified so that it can work with the existing RBAC interface
18:22:54 <dwchadwick> I see the use of ABAC by services as independent of attribute mapping
18:23:47 <dwchadwick> PDP - policy decision point. The policy engine that is fed an RBAC or ABAC policy and returns a grant/deny decison
18:24:05 <henrynash> dwchadwick: yes
18:24:23 <dwchadwick> PDP is the term used by Internet RFCs and most published papers on authorisation
18:24:28 <ayoung> dwchadwick, yes, so Keystone doesn't own that.
18:25:07 <dwchadwick> Exactly. Thus attribute mapping is independent of how services make their authz decisions. It only effects how user attributes are converted into roles and tenants
18:26:08 <dwchadwick> Whilst groups do not help with the longer term strategy, attribute mappings do, and can provide most of the functionality that groups need
18:26:41 <ayoung> dwchadwick, so are you saying for V1, we make no changes to the current policy enforcement, and instead only focus on mapping an attribute to a role in a tenant?
18:26:50 <gyee> but adds more complexity?
18:26:56 <dwchadwick> The only thing missing is for the administrator to give the users some group memberships, and then for keystone middleware to pick these up and call the attribute mapping service
18:27:10 <dwchadwick> ayoung - exactly
18:27:48 <dwchadwick> Plus mapping is an optional feature that can switched on or off
18:27:56 <henrynash> what I had in mind was a having a reference groups implementation that is wholly contained withng "current keystone mechanisms", with a prototype that would talk to the attribute mapping component
18:27:59 <ayoung> dwchadwick, OK,  so I think we still need groups.  They are a n additional attribute for a user that will then map to multiple roles.
18:28:22 <henrynash> (sorry I menat prototype interface)
18:28:23 <ayoung> I don't think we have anything yet that can do that for us
18:29:11 <dwchadwick> ayoung. Yes you need something like groups to solve the use cases Henry specified in a non-federated world
18:29:30 <ayoung> dwchadwick, but not in a federated world?
18:29:41 <kwss> ayoung yes, what is missing from our service is the capability for administrators to assign organisational attributes internally
18:29:53 <dwchadwick> But what I am suggesting is a simpler implementation of groups for Henry because the service that deals with assigning tenants and roles to groups is done by the attribute mapping service
18:29:57 <henrynash> that way we can flesh out how the attribute mapping side works, what the admin interfaces for it are etc. etc. and know we can switch over groups to use it when we have those production ready
18:30:34 <dwchadwick> then when we migrate to federated access, the same attribute mapping service is used and the group assignment feature is no longer carried out in Keystone, but in the organisation that acts as the IDP
18:31:11 <henrynash> dchadwick: I'm less worried about typing that Henry has to do, but rather how we incrementally get to where we need to be and still ship production ready code at G and H
18:31:20 <ayoung> henrynash, do you understand what dwchadwick is proposing?
18:31:44 <dwchadwick> So to summarise. Users need to be assigned to groups or organisational attributes. This is done either in Keystone (centralised authn) or in the IDPs (federated login)
18:31:56 <ayoung> right
18:32:07 <henrynash> ayoung: only in part…
18:32:13 <dwchadwick> then these group attributes are mapped into keystone roles and tenants by the attribute mapping service
18:32:15 <gyee> so groups is a must, attribution is option, that right?
18:32:23 <gyee> attribute mapping I mean
18:32:53 <dwchadwick> gyee - groups on their own are useless. This is because services dont understand them
18:33:14 <gyee> they are useful in assigning roles
18:33:18 <henrynash> ayoung: I understand the parts do far proposed, but seems to me that  there are still significant pieces to fill in
18:33:28 <dwchadwick> groups have to be mapped into tenants and roles. So we need the attribute mapping service for this (or something similar)
18:33:41 <ayoung> OK, lets say we have a groups table
18:33:51 <ayoung> then we add an attribute_mapping table
18:33:58 <dwchadwick> gyee - no groups are only useful in assigning roles if you have a service that can do this
18:34:09 <ayoung> and a group assignment table
18:34:40 <Alex-Shulman> just a small comment - groups are also useful for some access control models like ACLs - if someone will wish to plug this in at the storage level
18:34:44 <ayoung> one a user is assigned to a group via an entry in the group assignment table, we add an entry in the attribute_mapping table that would give them the role?
18:34:45 <gyee> dwchadwick, groups is only visible to Keystone
18:35:07 <dwchadwick> yes tables will do this, but if you look at the entity relationship diagram in our blueprint you will see that a simple table is not good enough in general to do many to many mapping
18:35:46 <dwchadwick> To do many to many mapping you need several tables (as per our diagram)
18:36:52 <dwchadwick> you need a group table, an OS set table (which links together tenants and roles) and and a mapping table
18:36:52 <henrynash> dwchadwick: agreed in general - but so far in keystone we have chased static assignments and hence the groups proposal is to extend that staid model….I agree a longer term goal of more flexibility is good
18:37:38 <henrynash> hmm, some words didn't come out right in that sentence :-)
18:37:54 <dwchadwick> henry - but you still have to deal with tenants and roles dont you?
18:38:03 <henrynash> (chosen status assignments……extend static model)
18:38:18 <henrynash> dwchadwick: sure
18:39:06 <dwchadwick> Our model has five new tables I think
18:39:36 <ayoung> dwchadwick, OK, I think we need to go back and review to that design.  Let me see if I can get it up on line
18:39:38 <henrynash> we don't use the sets model of course in leystone today
18:39:40 <dwchadwick> So this is what we are talking about. Providing a set of APIs for setting, getting and deleting the table entries
18:40:03 <dwchadwick> https://docs.google.com/a/kent.ac.uk/document/d/1cObK3P_ic9XSTwJRFsmimG94LDnFbsPbvx_H1aM1FPI/edit#
18:41:56 <dwchadwick> Should we move to the next topic now and continue detailed discussions about attribute mapping via email, as time is running out
18:42:45 <ayoung> Well, I think we covered everything but trusts
18:42:50 <ayoung> #topic Trusts
18:43:06 <ayoung> http://wiki.openstack.org/Keystone/Trusts
18:43:24 <dwchadwick> this seems to be progressing nicely
18:43:43 <dwchadwick> I think some more work is still needed for recursive delegation
18:43:44 <ayoung> I feel pretty good about this spec.  There was one outstanding issue about whether we should allow administrative acces to the trusts.
18:44:02 <ayoung> dwchadwick, agreed, but I think I want to punt on recursive for phase 1
18:44:19 <dwchadwick> You will need either backdoor or front door API access for administrators
18:44:23 <ayoung> Since that is encapsulated inside of Keystone, we can phase it in
18:44:37 <ayoung> dwchadwick, yeah, and I guess there is no harm there, either
18:44:45 <ayoung> it will be either GET or DELETE
18:45:02 <dwchadwick> Yes, you can default delegation depth to zero in the first implementation. Also default validity time to infinity
18:45:54 <ayoung> I don't think I want to allow admins to create Trusts, and no one should be able to modify them
18:46:04 <heckj> ayoung: I like where it's going - want to get vishy input on it too, as it'll impact a number of pieces in core nova as well
18:46:10 <dwchadwick> You can also omit most other policy fields in the first implementation so that delegation is based primarily on the tenant and roles
18:46:12 <heckj> (any notmyname and bcwaldon)
18:46:35 <notmyname> ?
18:46:39 <dwchadwick> ayoung agreed. Admin should only read and delete
18:47:01 <ayoung> heckj, yes, but i suspect that, like most things, they won't pay attention until it is implemented and they try to use it.
18:47:12 * ayoung is slightly jaded from PKI
18:47:29 <dwchadwick> I have to go in five mins. sorry
18:47:31 <heckj> ayoung: fair enough, I'm pushing them to look, read and think about it here
18:47:55 <ayoung> heckj, as well we should.
18:48:07 <ayoung> #topic open discussion
18:48:16 <dwchadwick> once they start to use it, then they will be hooked and you will have plenty of demand for new features :-)
18:48:28 <ayoung> Can you all chime in with who is working on what, if it has not come up in the meeting yet?
18:48:35 <dwchadwick> What about adding IDPs to service catalog
18:48:50 <ayoung> aah, right
18:48:51 <dwchadwick> Kristy has nearly finished this implementation now
18:49:31 <dwchadwick> It wont be needed until federation, but it would be nice to include the code to make sure it is bug free
18:49:33 <ayoung> I don;t see anything contraversial there.
18:49:45 <kwss> yes, i should have a rough implementation finished by the end of tomorrow.
18:49:53 <dwchadwick> excellent
18:50:13 <henrynash> (separate topic): I think we need more clarity on what we mean when we talk about a "tenant" in v3 and beyond
18:50:15 <dwchadwick> but make sure it is smooth before it is sent for QA :-)
18:50:32 <ayoung> kwss, great.  don't be daunted by the thoroughness of dolph's code reviews.
18:50:39 <dwchadwick> tenant is an entity in the ISO standard for clouds
18:51:12 <dwchadwick> It is equivalent to account holder so I am led to believe
18:51:13 <ayoung> henrynash, yes.  The thing is, we misused tenant in Keystone in the past
18:51:30 <henrynash> ayoung: agreed
18:51:31 * heckj is tired of tilting at that windmill
18:51:41 <ayoung> which is why we want to get everyone saying project for a while before we reintroduce the  term in its correct meaning
18:51:50 * vishy has something to add to the meeting when there is a moment
18:51:55 <heckj> we (the project) blew that earlier I'm afraid, makes it difficult to reset to what is reasonable
18:51:58 <henrynash> ayoung: lol
18:52:06 <ayoung> vishy, you have the conch
18:53:11 <dwchadwick> seeing as vishy seems to have dissappeared, what about the release shedule for federation
18:53:49 <dwchadwick> If we can get attribute mapping and idp service in first, then it makes the rest that much less work
18:54:14 <ayoung> dwchadwick, so it really is a question of having it ready for review
18:54:24 <ayoung> IDPs will be G2
18:54:26 <dwchadwick> BTW, there is an OpenStack meeting in London tomorrow, where I am giving a talk about federated Keystone
18:54:51 <ayoung> dwchadwick, where is attribute mapping?  Almost done?  Month or two away?
18:55:12 <heckj> vishy: jump in whenever
18:55:15 <dwchadwick> Ok, so once Kristy has finished IDP service it will go for review, then next will be attribute mapping, and then finally the revised federation making use of the new service APIs.
18:55:41 <dwchadwick> Sorry but I must go now. Take care everyone
18:55:58 <ayoung> dwchadwick, agreed.  So I would guess that G3 is likely, but like PKI, a tech preivew for Grizzly?
18:56:11 <kwss> ayoung, we hope to have an attribute mapping implementation ready by the end of the year
18:56:46 <ayoung> kwss, I would warn you that things slow down in December, but I promise to keep on top of any code reviews
18:57:08 <kwss> ayoung, understood
18:57:16 <ayoung> But that should stil be a go for G2
18:57:54 <ayoung> But even if it isn't  It really matters how far between that and the full Federated impl
18:58:23 <vishy> heckj: sorry, missed the notification
18:58:46 <ayoung> and that is likely to be completed shortly before the end of the Grizzly cycle.  We'll have to decide based on how close and how hard it is to test whether it goes in, and if so, if we caveat it with "something new and shiny, please test."
18:58:49 <vishy> heckj: so we were discussing having a way to map tenant_ids to tenant_names
18:59:10 <vishy> is there an easy way to do it a) for one tenant and b) in bulk for a group of tenants?
18:59:32 <ayoung> vishy, tenant ID generation defaults to uuid, but it doesn't need to be
18:59:50 <ayoung> tenant ID just needs to be unique, so the two could potentially be the same
19:00:17 <ayoung> let me peek at the create_tenant API,
19:00:58 <heckj> vishy: today, no - but we could make an extension API to single or bulk lookup
19:00:59 <Alex-Shulman> also tenants are created by name, but deleted by tenant_id - this is very inconvenient. Can this be changed?
19:01:09 <ayoung> tenant_ref['id'] = tenant_ref.get('id', uuid.uuid4().hex)
19:01:28 <ayoung> so you should be able to create a tenant with a specified ID that you pass it in
19:01:41 <ayoung> TenantController.create_tenant
19:02:01 <Alex-Shulman> can we allow delete by tenant name as well (and implement the look up inside)?
19:02:02 <ayoung> You can't modify the id once it is created
19:02:26 <Alex-Shulman> working with names is easier ...
19:02:34 <vishy> i get push back about making them the same
19:02:49 <vishy> becuase then project names can't change
19:03:02 <vishy> i just need a way to map one to the other for user display
19:03:05 <ayoung> Alex-Shulman, I think we can make that extension.  It is not changing previous behaviour.  It would also entail a CLI change
19:03:07 <vishy> horizon needs it too
19:03:30 <Alex-Shulman> this will be great, thanks
19:04:02 <ayoung> Alex-Shulman, file it as a ticket in track
19:04:11 <Alex-Shulman> ok
19:04:23 <ayoung> er, launchpad
19:04:30 <heckj> ayoung: heh
19:04:49 <ayoung> vishy, by mapping, do you mean just an API to enumerate all the tenants, names and ids?
19:04:49 <heckj> vishy: I'll create a quick BP to make an extension or something for the lookup - will that work to get it started?
19:05:07 <heckj> ayoung: lit "give me a name for this UUID"
19:05:15 <heckj> ayoung: for projects and users I think
19:05:26 * heckj bets cielometer would love that too
19:05:35 * mordred would love it
19:05:43 * mordred loves UUIDs
19:05:48 <ayoung> heckj GET works for userid
19:06:15 <jeblair> mordred: i think the whole infra team loves uuids.  ;)
19:07:20 <kwss> I have to leave. Take care :)
19:07:21 * mordred hands jeblair a UUID, asks what its name is
19:07:35 <henrynash> cheers, everyone
19:09:57 <jeblair> mordred: it's name is "#endmeeting"
19:10:42 <clarkb> ayoung: care toe #endmeeting for us?
19:10:45 <ayoung> #endmeeting