Tuesday, 2014-05-20

jamielennoxwell more that the test shouldn't operate in different ways depending on the version of the library you have installed00:00
rodrigodsmorganfainberg, that was the issue preventing the keystone client tests to pass, right?00:00
jamielennoxso it was fine initially but we shouldn't be testing it any more00:01
morganfainbergjamielennox, ++00:01
morganfainbergrodrigods, yes.00:01
rodrigodscool, have a patch waiting for this fix =)00:01
*** dstanek_zzz is now known as dstanek00:18
dstanekrodrigods: what was your comment?00:20
dstanekrodrigods: ah, about putting the type check back in? we should almost never type check00:21
dstanekbknudson: if you feel strongly about the whitespace i can put it back - it makes more sense to me to have the assertions together in the code since they are checking the same attributes (logical grouping and all that)00:23
bknudsondstanek: I do feel strongly about the whitespace. I don't want there to be changes where whitespace is added and then the next change removes it.00:24
rodrigodsdstanek, was convinced by morganfainberg (thought only about maintaining the original behavior instead of realizing that it was wrong in the first place)00:24
morganfainbergbknudson, i'm fine with that stance, once that is resolved I'll happily +2 as well. (holding based on your -1)00:25
morganfainbergbknudson, that is if you aren't around when change is pushed up.00:25
morganfainberg... or something something ... brain00:26
bknudsonI'm fine with the change once the whitespace issue is fixed. I think the code was just there because it was weird that requests returned a tuple00:27
bknudsonand now requests is less weird00:27
ayoung_dad_modemorganfainberg, ok...I've been thinking about stacked policy distribution.  I think I like this idea of the checksum being distributed in the token.  Basically, the token contains a checksum that tells  auth_token middleware if it needs to fetch any policy files00:35
ayoung_dad_modesomething like:  checksum together all of the policy files for a project.  If you don't have that bundle, ask the server for a refresh.00:35
ayoung_dad_modeServer refresh will also calculate the checksum, so if the token is out of sync, you'll know it.00:36
jamielennoxayoung_dad_mode: in the token? that's a little unusual - because the same could be said for revocations etc00:37
ayoung_dad_modeIn theory, if the policy checksum somehow changes, if any one of the policy files change, then the token can either be treated as revoked, or accepted with no loss of security00:37
ayoung_dad_modejamielennox, the way I see it, the projects affected are going to be scattered, and in a system with a lot of endpoints, doing a blast of updates for every litle change will be chatty00:38
jamielennoxayoung_dad_mode: what if the token is older than the policy change?00:38
ayoung_dad_modebut an indicator telling the service:  you don't have the latest.00:38
ayoung_dad_modejamielennox, I think that is OK, so long as the server returns a response that indicates what the current valid one is00:38
morganfainbergayoung_dad_mode, what if the token and the server are the same but there is a new policy?00:39
ayoung_dad_modemaybe just record the bad checksum to prevent rechecking it00:39
rodrigodsbknudson, just replied your comments at https://review.openstack.org/#/c/91578/00:39
ayoung_dad_modemorganfainberg, we could revoke the token, but that seems harsh00:39
jamielennoxayoung_dad_mode: so we were looking at a daemon for auth_token anyway that could listen for event changes like this00:39
ayoung_dad_modejamielennox, yeah, but if there are say 200 endpoints and thousands of proejcts we probably don't want to be blasting all of that around.00:40
ayoung_dad_modeI'd rather make it pull on demand00:40
bknudsonrodrigods: I don't get why the client API returns HttpNotImplemented -- that's telling the application that the server responded with Not Implemented00:40
bknudsonnot that the python API isn't implemented.00:41
rodrigodsbknudson, i read the exception in another way: "im failing because the HTTP part is not implemented"00:41
rodrigodsso "I didn't even tried to perform the request"00:42
jamielennoxayoung_dad_mode: so i guess the cost is that you need to have calculated the full policy when you issue the token00:43
jamielennoxi assume that's possible given we know the project etc00:44
*** dstanek is now known as dstanek_zzz00:44
ayoung_dad_modejamielennox, yeah.  Ugh...just realized that this would only work with endpoint binding00:45
ayoung_dad_modewith tokens for general use, there is no way to know which service to calculate policy for00:45
bknudsonrodrigods: the exception is HttpNotImplemented(HttpServerError) -- so it's indicating a server error and not a client error00:46
*** hipster has joined #openstack-keystone00:47
*** ayoung_dad_mode is now known as ayoung00:47
rodrigodsbknudson, is there a better exception to be raised there?00:48
ayoungour token mechanism sucks.00:48
bknudsonrodrigods: NotImplemented00:48
bknudsonrodrigods: well, that's not a good one either. Create a new one.00:48
rodrigodsbknudson, ok, about the message, did I understand correclty00:50
bknudsonrodrigods: no, just remove the +. you don't need + to continue a string00:50
rodrigodsbknudson, ok00:51
ayoungI want endpoint binding Dagnabit00:51
*** hipster has quit IRC00:51
rodrigodsbknudson, MethodNotImplemented(ClientException), clear enough?00:53
bknudsonrodrigods: that sounds great!00:53
*** browne has quit IRC01:01
ayoungjamielennox, ok, taking this from the top:  start up a service, and it should have no policy file.  It can query to keystone right away, but the keystone server might not be up yet.  So, it waits until the first request, and fetches the policy file for the service.  But, at this point, it knows it has no policy file for the specified domain or project, so it should ask for this now, too.  So, from this point on, it fetches01:04
ayoungpolicy files based on notifications01:04
ayoungand if it restarts, it has to dump its cache in case it has missed any notifications01:04
*** dstanek_zzz is now known as dstanek01:05
openstackgerritRodrigo Duarte Sousa proposed a change to openstack/python-keystoneclient: Add /role_assignments endpoint support  https://review.openstack.org/9157801:07
rodrigodsbknudson, ready for another round of comments =)01:07
*** stevemar has joined #openstack-keystone01:09
jamielennoxayoung: so i'm still not sure on the concept of whether we should have service policy and overlay with project policy - or just have a project policy that inherits01:10
ayoungjamielennox, you mean, to premerge or process them in series?01:10
ayoungjamielennox, so, I'm assuming the latter: and we can fetch only the policy files we care about.01:12
ayoungjamielennox, maybe I am planning for too big a scale, but the MOC folks got me thinking big01:12
jamielennoxayoung: i need to think about distribution a bit more01:13
jamielennoxayoung: are we still discussing that subsequent policy files should be additive?01:13
ayounglike, a couple hundred companies sharing a datacenter, each with a couple of nova instances,   for different qualities of service01:13
ayoungyeah, for the most part01:13
jamielennoxso that a project can only further restrict the policies of it's parent01:13
ayoung I think an endpoint should be able to completely override the policy file for a service01:13
ayoungsince it could do that anyway01:13
ayoungand then a domain on down to project should probably be only additive...unless the top level one doesn't care01:14
jamielennoxayoung: so i'm more or less in the service scoped tokens exist mindset - and i see that the policies for service scoped vs project scoped are completely different01:14
jamielennoxnot additive or combinatory in any way01:14
ayoungyeah, service scoped are either service or endpoint.  But I could see an endpoint wnated to replace just specific rules...to limit copy/paste errors01:15
*** dstanek is now known as dstanek_zzz01:15
ayoungbut that could actually be handled at the server level, not the policy file processing01:15
jamielennoxayoung: so i think we need to better define what an endpoint is01:15
ayoungIE  PATHC policy/service01:15
jamielennoxin the service -> server -> urls description01:16
jamielennoxcurrently endpoint represents a URL and not a server01:16
ayoungright...well, a set of urls in the case of Keystone, as admin and main and v3 all are linked together in one endpoint01:17
jamielennoxayoung: ok so endpoint = server and we should express that better01:18
ayoungjamielennox, did I throw server in there or did you01:18
jamielennoxi said server so that i didn't re-use the word endpoint01:19
ayoungso, yeah, we are abusing endpoint01:20
ayoungno surprise there, we abuse everything01:20
ayoungjamielennox, Actually, looking here http://en.wikipedia.org/wiki/Service-oriented_architecture  we might be using it correctly, but anyway...01:23
ayounglets assume that server == endpoint for our purposes01:23
ayoungthen a policy bundle is per endpoint, and a project adds to that bundle to get the project specific policy.01:24
ayoungI'm mostly concerned about project policies, as the endpoint and service ones will change infrequently enought that we could rely on notifications for those01:24
*** marcoemorais has quit IRC01:26
jamielennoxdo you really think that a policy will *change* that much?01:27
jamielennoxeven per project01:27
ayoungjamielennox, no one policy will, but in the aggregate, with hierarchical?  Yeah01:27
ayoungIts the hierarchical part that scares me01:27
ayoungit means that changing policy for one top level project will change it for everything underneath01:28
jamielennoxayoung: so if policy is purely more restrictive that's not too bad because you just process every policy as an AND01:31
jamielennoxorder doesn't really matter01:32
jamielennoxso it should be possible to specify multiple policy lines per protected method01:32
ayoungjamielennox, yeah.  But even if a rule is "this is the default, but if there is a follow on rule, ignore me and use that instead"  should work.  It would be more processing, and I don't think it would be that common, but it would allow a reasonable default (the policy file we ship now)  and also a smarter override01:34
ayoungso you don;t need both admin and redhat_admin, just redhat_admin  in some project to, say, add a role to a user01:34
jamielennoxayoung: i think we target the purely more restrictive for now and look at policy enhancements later01:34
*** rodrigods has quit IRC01:34
jamielennoxone of the ARBAC things (that i haven't read yet) is hierarchical roles for that01:35
ayoungjamielennox, even if we implement, we should plan for the other.  Ah, yes ARBAC01:35
jamielennoxi put the email with resources to -dev - but it's a 25 page academic paper01:36
jamielennoxnot sure when i'll get into reading that one01:36
*** dstanek_zzz is now known as dstanek01:37
ayoungjamielennox, hierarchical regions are another wrinkle, too.01:38
openstackgerritDavid Stanek proposed a change to openstack/python-keystoneclient: Fixes an erroneous type check in a test  https://review.openstack.org/9425601:39
jamielennoxayoung: yea, i'm aware - i don't like the way they were dumped on us, i had lots of follow up with 'how do i work with this as a client' that was ignored when i stopped blocking the reviews with it01:39
ayoungjamielennox, and are policies going to follow down the region hierarchy as well as the project hierarchy?01:39
jamielennoxayoung: i don't view regions that way - but i fear i might be way out on my own with regions01:40
ayoungso you could, in theory, have a different base policy for the same project policy, depending on endpoint, etc01:40
jamielennoxas far as i'm concerned regions are just a label that helps you differentiate things01:40
jamielennoxwe have never applied any sort of authentication or logic based on region01:41
ayoungjamielennox, so would policy only be specified at the service or endpoint level, with region completely ignored?01:42
ayoungI would also be OK with two levels of policy;  service OR endpoint and project specific.  If one project specifies a policy file, it supercedes all policy written above it.  You have to have the higher level policy authorization to load a policy for the lower level anyway.01:44
jamielennoxyes to ignoring region (at least for now)01:45
jamielennoxregarding policy i see ONLY two types: service scoped or project scoped depending on your token01:45
jamielennoxpolicies are inheritted in the same way as roles, including when you get to a break (what we considered private projects) you should recreate a policy for it01:46
jamielennoxwe would then obviously have some basic policies that would get defaulted to for those projects01:47
*** hipster has joined #openstack-keystone01:47
jamielennoxpolicies (for now) are additive only01:47
ayoungjamielennox, so overloading policy per endpoint is going to be important.  If only to allow a testable roll-out of a new policy file01:49
jamielennoxso per project per endpoint01:50
jamielennoxok, that's fair01:50
ayoungand the whole different organizations owning different services, too.  But In general, I think we are aligned.  The region thing, though, might be how we split out organizations01:50
jamielennoxbut same rules apply, endpoint policy should be additive to project policy01:51
ayounglike IBM and HP in the same data center.  Each have a set of Nova servers with different QofS.  Each org wants to use their same base policy for Nova.01:51
*** hipster has quit IRC01:51
ayoungRegions is the only grouping mechanism we have.  Without it, you have a window where you bring up a server and it gets the wrong policy01:52
jamielennoxayoung: wait - why is that a problem? those organizations should be seperated by project01:52
jamielennoxwithin their own private project tree01:52
ayoungNo, project is orthoganal01:52
ayoungthe MOC approach is mix-and-match01:52
ayoungI get cinder from here, Nova from there...01:52
jamielennoxoh right the MOC thing01:52
ayoungits ambitious.  But I've learned to listen to Orran01:53
ayoungHe kindof gets this....01:53
jamielennoxyes, as providers that makes sense, just not so much as organizations running their own thing01:53
ayoungGod, there are so many ways to screw this up01:54
jamielennoxi'm still unsure how to stack it01:54
jamielennoxbecause that would imply that an endpoint might want to set a basic policy that should be project independant01:54
ayoungI think we keep playing with it mentally before we write a lick of code.01:55
jamielennoxi don't know if that can work01:55
ayoungWe saw with the domain-id problem that we don't really enforce policy at the right level.  If we were doing it right, it would be enforced at the container.  I should be able to move a user from one domain to another, if I have permissions in both domains.  Same is true of proejcts01:56
ayoungwe don't enforce:  you can move something to this bucket if you have rights on the bucket01:56
jamielennoxmaybe, i'm of the opinion that a role only makes sense within a project as other projects might use other names for different things01:56
jamielennoxhow do you transition roles in that situation/01:57
ayoungjamielennox, not within a service?  Interesting...01:59
jamielennoxayoung: well they only exist within the context of whatever you are scoped to. So a service scoped token may defined roles per service, but a project scoped token will define roles per project and there is no implication that they should ever mean the same tihng02:00
ayoungI kinda want to talk with SELinux Dan about this. I'm guessing he's considered some of these issues himself02:00
ayoungjamielennox, I like the rule that a project can only overwrite what comes from higher in the project hierarchy.  As it  might be the case that a project would want to have different policies based on which endpoint specific policy it was running, so for that we say "use a different subproject"02:04
jamielennoxayoung: only overwrite? or only futher restrict?02:05
ayoungoverwrite for other project policy files02:05
ayounglike, if the role is redhat_admin  at the top, and then it changes to fedora_admin for a resource allocated to fedora02:06
ayoungyou can always make the lower level policy a copy of the higher one there.02:06
ayoungProbably use the same role for endpoints, and even regions02:06
ayoungif we provide a good tool for merging policy off line, we can do "only replace"02:07
jamielennoxsurely we aren't the first people to deal with something like this, is there any prior art?02:07
ayoungthere must be.  I'm afraid to ask David Chadwick, as his responses are more "what is possible" instead of "what is practical"02:08
ayoungwhatever we come up with needs to be well reviewed02:09
ayoungjamielennox, I'm headed into the office tomorrow, and I'm going to kill some trees to read these things without destroying my eyes02:10
ayoungI suspect dpal will have some insight as well.02:10
*** dstanek is now known as dstanek_zzz02:10
ayoungjamielennox, I could also see rules for "what roles can you grant someone"02:10
ayoungIE,  if you are Jr Sys admin, you can grant Member.  If you are Senior Admin you can grant JrAdmin02:11
jamielennoxthat was one of the things mentioned for ARBAC02:11
ayoungyeah, its the meta stuff: creating new projects and assigning roles that is the most likely to create security holes02:12
*** diegows has quit IRC02:14
*** xianghui_afk has joined #openstack-keystone02:15
ayoungjamielennox, ok...I'm more muddled about this than I was when I stared.  Which is probably a good thing.  I suspect the best we'll get out of Juno is "fetch policy for endpoint"02:15
jamielennoxyea, i would prefer to let hierarchical projects actually land before figuring out project based policies02:16
ayoungwe can let _0x44_  write  the stackable  policy mechanism while we deal with delivery.  As it is, we'll still have to deal with notifications and a fetch mechanism02:17
ayoungAnd then we can deal with delivery of stackable in Kilo.02:18
ayoungChris MacGown,  the Oslo guy02:18
*** dstanek_zzz is now known as dstanek02:35
*** amcrn has quit IRC02:37
*** hipster has joined #openstack-keystone02:39
*** ayoung has quit IRC02:44
*** hipster has quit IRC02:44
*** mberlin1 has joined #openstack-keystone02:44
*** hipster_ has joined #openstack-keystone02:46
*** mberlin has quit IRC02:47
*** Guest82130 has quit IRC02:49
openstackgerritayoung proposed a change to openstack/keystone: Compressed Token Provider  https://review.openstack.org/9114502:52
*** marcoemorais has joined #openstack-keystone02:58
*** praneshp_ has quit IRC03:01
*** praneshp_ has joined #openstack-keystone03:04
*** praneshp_ has quit IRC03:05
*** marcoemorais has quit IRC03:14
*** praneshp_ has joined #openstack-keystone03:15
*** praneshp_ has quit IRC03:20
*** marcoemorais has joined #openstack-keystone03:20
*** daneyon has joined #openstack-keystone03:23
*** hipster_ has quit IRC03:23
*** marcoemorais has quit IRC03:26
*** hipster has joined #openstack-keystone03:32
*** jaosorior has joined #openstack-keystone03:42
*** morganfainberg is now known as morganfainberg_Z03:49
*** gyee has quit IRC03:54
*** hipster has quit IRC03:57
*** gokrokve has quit IRC04:03
*** dstanek is now known as dstanek_zzz04:04
*** praneshp_ has joined #openstack-keystone04:05
*** praneshp__ has joined #openstack-keystone04:10
*** praneshp_ has quit IRC04:12
*** gokrokve has joined #openstack-keystone04:19
*** dstanek_zzz is now known as dstanek04:27
*** hipster has joined #openstack-keystone04:27
*** hipster_ has joined #openstack-keystone04:29
*** hipster has quit IRC04:33
*** hipster_ has quit IRC04:33
openstackgerritNathan Kinder proposed a change to openstack/keystone: Adds log message upon token granting  https://review.openstack.org/9397504:39
*** marcoemorais has joined #openstack-keystone04:47
*** marcoemorais1 has joined #openstack-keystone04:49
*** dstanek is now known as dstanek_zzz04:49
*** marcoemorais has quit IRC04:52
*** ajayaa has joined #openstack-keystone05:01
*** hipster has joined #openstack-keystone05:07
*** hipster has quit IRC05:12
*** jamielennox is now known as jamielennox|away05:32
*** hipster has joined #openstack-keystone05:47
*** hipster has quit IRC05:52
*** daneyon has quit IRC05:53
openstackgerritOpenStack Proposal Bot proposed a change to openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/9028806:02
*** gokrokve has quit IRC06:09
*** harlowja is now known as harlowja_away06:28
*** ajayaa has quit IRC06:39
*** gokrokve has joined #openstack-keystone06:40
*** gokrokve_ has joined #openstack-keystone06:42
*** gokrokve has quit IRC06:45
*** xianghui_afk has quit IRC06:47
*** hipster has joined #openstack-keystone06:47
*** gokrokve_ has quit IRC06:47
*** dstanek_zzz is now known as dstanek06:50
*** hipster has quit IRC06:52
*** derek_c has joined #openstack-keystone06:58
*** dstanek is now known as dstanek_zzz07:00
*** xianghui_afk has joined #openstack-keystone07:03
*** BAKfr has joined #openstack-keystone07:04
*** henrynash has joined #openstack-keystone07:10
*** praneshp__ has quit IRC07:11
*** henrynash has quit IRC07:12
*** andreaf has quit IRC07:17
*** leseb has joined #openstack-keystone07:34
*** gokrokve has joined #openstack-keystone07:42
*** leseb has quit IRC07:42
*** leseb has joined #openstack-keystone07:43
openstackgerritMarcos Fermín Lobo proposed a change to openstack/keystone: Unimplemented get roles by group for project list  https://review.openstack.org/7647007:44
*** gokrokve has quit IRC07:47
*** esmute has quit IRC07:52
*** marcoemorais1 has quit IRC07:55
*** xianghui_afk has quit IRC07:57
*** esmute has joined #openstack-keystone07:58
*** stevemar has quit IRC08:01
*** xianghui_afk has joined #openstack-keystone08:09
*** bvandenh has joined #openstack-keystone08:16
*** henrynash has joined #openstack-keystone08:24
*** andreaf has joined #openstack-keystone08:25
*** rwsu has quit IRC08:26
*** mberlin1 has quit IRC08:30
*** mberlin has joined #openstack-keystone08:33
*** gokrokve has joined #openstack-keystone08:42
*** gokrokve has quit IRC08:47
openstackgerritIlya Pekelny proposed a change to openstack/keystone: oslo.db implementation  https://review.openstack.org/7721008:56
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063008:56
*** derek_c has quit IRC09:14
*** xianghui_afk has quit IRC09:24
*** xianghui_afk has joined #openstack-keystone09:38
*** gokrokve has joined #openstack-keystone09:42
*** gokrokve has quit IRC09:47
*** hipster has joined #openstack-keystone09:47
*** hipster has quit IRC09:51
*** rodrigods has joined #openstack-keystone09:59
*** rodrigods has joined #openstack-keystone09:59
openstackgerritKristy Siu proposed a change to openstack/identity-api: Trusted Attributes Policy for External Identity Providers (Federation pt 4)  https://review.openstack.org/6048910:12
*** xianghui_afk has quit IRC10:17
*** rodrigods has quit IRC10:21
*** xianghui_afk has joined #openstack-keystone10:30
openstackgerritKristy Siu proposed a change to openstack/identity-api: Trusted Attributes Policy for External Identity Providers  https://review.openstack.org/6048910:35
*** afazekas has joined #openstack-keystone10:39
*** rodrigods has joined #openstack-keystone10:40
*** gokrokve has joined #openstack-keystone10:42
openstackgerritIlya Pekelny proposed a change to openstack/keystone: oslo.db implementation  https://review.openstack.org/7721010:45
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063010:45
*** gokrokve has quit IRC10:47
*** hipster has joined #openstack-keystone10:47
*** rodrigods has quit IRC10:49
*** hipster has quit IRC10:52
*** xianghui_afk has quit IRC10:59
*** bvandenh has quit IRC11:02
*** diegows has joined #openstack-keystone11:17
*** henrynash has quit IRC11:34
*** henrynash has joined #openstack-keystone11:37
*** gokrokve has joined #openstack-keystone11:42
*** henrynash has joined #openstack-keystone11:43
*** gokrokve has quit IRC11:47
*** lbragstad has quit IRC11:47
openstackgerritRodrigo Duarte Sousa proposed a change to openstack/python-keystoneclient: Add /role_assignments endpoint support  https://review.openstack.org/9157812:00
*** afazekas has quit IRC12:05
*** rodrigods has joined #openstack-keystone12:08
*** raildo has joined #openstack-keystone12:13
*** dhellmann_ is now known as dhellmann12:16
*** raildo has quit IRC12:17
*** afazekas has joined #openstack-keystone12:20
*** JuanManuelOlle has joined #openstack-keystone12:40
*** dstanek_zzz is now known as dstanek12:41
*** gokrokve has joined #openstack-keystone12:42
*** gokrokve has quit IRC12:47
*** lbragstad has joined #openstack-keystone12:50
*** xianghui has joined #openstack-keystone13:06
*** Ju has quit IRC13:09
*** dstanek is now known as dstanekafk13:09
*** dstanekafk is now known as dstanekafk_zzz13:11
*** afazekas has quit IRC13:12
*** bknudson has quit IRC13:17
*** joesavak has joined #openstack-keystone13:17
*** nkinder has quit IRC13:20
*** dstanekafk_zzz is now known as dstanek13:23
*** afazekas has joined #openstack-keystone13:25
*** dhellmann is now known as dhellmann_13:27
*** hipster has joined #openstack-keystone13:29
*** hipster has quit IRC13:32
*** vhoward has left #openstack-keystone13:32
*** bknudson has joined #openstack-keystone13:41
*** jaosorior has quit IRC13:41
*** gokrokve has joined #openstack-keystone13:42
*** andreaf has quit IRC13:45
*** tellesnobrega has joined #openstack-keystone13:46
*** gokrokve has quit IRC13:46
*** erecio_1 has joined #openstack-keystone13:51
*** erecio has quit IRC13:54
openstackgerritChristian Berendt proposed a change to openstack/keystone: fixed several pep8 issues  https://review.openstack.org/9368613:56
*** ayoung has joined #openstack-keystone14:01
*** nkinder has joined #openstack-keystone14:05
*** stevemar has joined #openstack-keystone14:06
*** erecio_2 has joined #openstack-keystone14:10
*** jamielennox|away has quit IRC14:11
ayoungCutting the Gordian Knot.  Both "Domains"  and "Projects"  are instances of "Tenants"  tada!14:12
*** erecio_1 has quit IRC14:14
*** bvandenh has joined #openstack-keystone14:14
dstanekdomains -> tenants -> projects as a nice little hierarchy :-)14:16
*** jamielennox|away has joined #openstack-keystone14:18
ayoungdstanek, nah, domains are the primary tenants14:27
ayoungprojects are the nested tenants14:27
*** dims has joined #openstack-keystone14:28
dstanekwhat was the usecase for domains back in the day?14:28
openstackgerritIlya Pekelny proposed a change to openstack/keystone: oslo.db implementation  https://review.openstack.org/7721014:29
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063014:29
ayoungdstanek, https://wiki.openstack.org/wiki/Domains14:37
*** hipster has joined #openstack-keystone14:39
*** david-lyle has joined #openstack-keystone14:40
*** gokrokve has joined #openstack-keystone14:42
*** gokrokve has quit IRC14:46
*** xianghui has quit IRC14:47
*** ukalifon has joined #openstack-keystone15:08
*** gordc has joined #openstack-keystone15:09
openstackgerritDiane Fleming proposed a change to openstack/identity-api: Clean up files for identity v2.0 reference  https://review.openstack.org/9419415:13
*** browne has joined #openstack-keystone15:15
ayoungukalifon, OK,  so there should be two files, one for Shibboleth, and one for the Configuration for Keystone....and bother stevemar if you need help reproducing...15:15
*** leseb has quit IRC15:16
*** leseb has joined #openstack-keystone15:17
openstackgerritBrant Knudson proposed a change to openstack/keystone: SQL and LDAP fixes for get_roles_for_user_and_project user=group ID  https://review.openstack.org/9439615:19
bknudsonthis is for the security vuln15:20
bknudsonhttps://review.openstack.org/#/c/94397 is for stable/icehouse15:22
*** gokrokve has joined #openstack-keystone15:24
*** browne has quit IRC15:24
*** browne has joined #openstack-keystone15:25
*** daneyon has joined #openstack-keystone15:26
*** andreaf has joined #openstack-keystone15:30
openstackgerritDiane Fleming proposed a change to openstack/identity-api: Clean up files for identity v2.0 reference  https://review.openstack.org/9419415:30
*** ukalifon has quit IRC15:33
*** afazekas has quit IRC15:40
*** praneshp_ has joined #openstack-keystone15:42
nkinderlbragstad: I'm looking into your comment about CADF for https://review.openstack.org/#/c/93975/15:49
nkinderlbragstad: it doesn't look like we actually use CADF anywhere yet (unless I'm missing something)15:49
lbragstadnkinder: yeah, I noticied that too15:50
lbragstadI added topol to the review15:50
lbragstadbecause I though I'd heard him say that was done when we authenticated.15:50
lbragstadbut I couldn't find it15:50
nkinderlbragstad: ok, maybe we should split out adding CADF notifications for successful and failed auth15:51
nkinderlbragstad: even with CADF, we should still log successful issuance, right?15:51
lbragstadnkinder: so we will be sending a cadf notification in both those cases?15:51
lbragstadnkinder: that's what I was unsure about... do we log authentication (possibly sensitive) information in that case?15:51
nkinderlbragstad: that would need to be decided, but from a pure auditing perpective, you would want to know about failed and successful auth15:52
nkinderlbragstad: it just logs the user id15:52
lbragstadnkinder: ++ I agree15:52
lbragstadif it's just the user ID it's probably not so much of a big deal15:52
*** praneshp_ is now known as praneshp15:52
nkinderlbragstad: yeah.  I'll re-trigger the jenkins jobs15:53
lbragstadnkinder: does 'recheck no bug' still work?15:53
lbragstadi thought that was removed15:54
nkinderlbragstad: oh, I didn't hear about that15:54
* lbragstad could be wrong15:54
*** gyee has joined #openstack-keystone15:54
lbragstadnkinder: I guess reverify no bug is gone15:57
nkinderlbragstad: I think you're right.  It didn't retrigger.15:57
lbragstadnkinder: yeah i didn't see it come through the queue15:57
lbragstadI think it was causing issues15:57
*** amcrn has joined #openstack-keystone16:00
*** marcoemorais has joined #openstack-keystone16:01
lbragstadnkinder: so, here https://review.openstack.org/#/c/93975/2/keystone/token/controllers.py do you think we should add a comment inline, just elaborating on the idea of extending the authentication (failure or success) to a cadf implementation?16:03
*** bvandenh has quit IRC16:05
*** BAKfr has quit IRC16:07
*** hipster_ has joined #openstack-keystone16:07
*** cds has joined #openstack-keystone16:08
*** hipster has quit IRC16:11
*** sbfox has joined #openstack-keystone16:15
*** sbfox has quit IRC16:16
*** sbfox has joined #openstack-keystone16:17
*** richm has joined #openstack-keystone16:18
openstackgerritayoung proposed a change to openstack/keystone: multi-backend support for identity  https://review.openstack.org/7421416:21
*** tristanC has joined #openstack-keystone16:23
nkinderlbragstad: yeah, a TODO comment is a good idea16:24
tristanCHello folks! I'm waiting for both 94396 and 94397 to be approved before sending the associated OSSA (those are fixes for the now public bug #1309228)16:24
uvirtbotLaunchpad bug 1309228 in keystone "User gets group auth if same id (CVE-2014-0204)" [High,In progress] https://launchpad.net/bugs/130922816:24
lbragstadnkinder: sounds good, I left my comment16:30
*** rwsu has joined #openstack-keystone16:30
tristanCBoth review received different comments, and as they are the same patch basically, could you update (if required) the master branch please (cc bknudson, dolphm)16:30
*** morganfainberg_Z is now known as morganfainberg16:30
openstackgerritDiane Fleming proposed a change to openstack/identity-api: Clean up files for identity v2.0 reference  https://review.openstack.org/9419416:31
bknudsontristanC: I'll look through the comments and make updates.16:31
bknudsonseems like we could merge what's there as long as there isn't a concern about opening a new security vuln16:31
tristanCbknudson: sadly the stable/icehouse got a couple of check-grenade-dsvm errors (tempest.api.identity.admin.v3.test_certificates.CertificatesV3TestJSON.test_get_ca_certificate NotFound)16:33
nkindertristanC, bknudson: I just added a comment to 9439616:34
tristanCnkinder: thanks!16:34
bknudsontristanC: I think there's something that's causing grenade to fail on stable/icehouse...16:34
*** openstackgerrit has quit IRC16:35
*** openstackgerrit has joined #openstack-keystone16:36
*** sbfox has quit IRC16:39
*** raildo1 has joined #openstack-keystone16:42
bknudsonnkinder: what do you think about me stealing _is_dns_equal(first_dn, second_dn) and putting it into https://review.openstack.org/#/c/94396/16:48
nkinderbknudson: I'm OK with that.  The main purpose of the other patch was to correct the dumb user check.16:50
bknudsonI'll put Sergey Nikitin down as co-author16:51
nkinderbknudson: I think I have a concern about the function though...16:51
openstackgerritguang-yee proposed a change to openstack/keystone: Make sure all the auth plugins agree on the shared identity attributes.  https://review.openstack.org/8494516:51
bknudsonlooks like it doesn't handle multi-valued rdns16:52
nkinderbknudson: the _is_dns_equal() method is only doing case-insensitive comparison of attribute names, not values16:52
bknudsonnkinder: I think that's what we want16:53
bknudsonsince we have no idea what the schema says for comparing attrs anyways16:53
nkinderbknudson: as I explained in my review comments, most (not all) values are compared in a case-insensitive way in LDAP16:53
nkinderbknudson: it will be wrong sometimes, but I suppose it's better to say that equivalent DNs are not equal than to say two different DNs are equal16:54
openstackgerritNathan Kinder proposed a change to openstack/keystone: Adds log message upon token granting  https://review.openstack.org/9397516:57
*** sbfox has joined #openstack-keystone16:58
*** leseb has quit IRC16:58
gyeebknudson, answered your questions there  https://review.openstack.org/8494517:00
*** amcrn has quit IRC17:04
*** radez_g0n3 is now known as radez17:05
morganfainbergnkinder, is that meant to be a cadf event?17:09
morganfainbergnkinder, or is this strictly a "oh hay, token for <X>"17:09
morganfainbergnkinder, also, is INFO the right level?17:09
nkindermorganfainberg: yeah, must have mixed up terminology since it's in notification.py17:09
morganfainbergnkinder, ah. so you want it to be an auditable event not a log17:10
nkindermorganfainberg: yes, the idea is that it us needed for auditing purposes.17:11
morganfainbergok, let me take a look at the notifications, i'll comment where that should hook in. so it will be CADF instead of log17:11
nkindermorganfainberg: CADF might be more appropriate, but this is something that we need to backport on our end too.17:11
morganfainbergto H?17:12
nkindermorganfainberg: yep17:12
morganfainberghm. maybe we do CADF for J, and backport to H would be logging? i'd rather not clutter the logs w/ token data in J (at INFO level) with token issuance info17:13
morganfainbergnkinder, since we have CADF in I and J17:13
morganfainberghuh.. wait a minute...17:14
*** harlowja_away is now known as harlowja17:16
morganfainbergnkinder, we should be emitting CADF notifications on both success and failure to authenticate.17:18
nkindermorganfainberg: agreed17:18
morganfainbergnkinder, no i mean, the code should already be in place for it17:18
nkindermorganfainberg: where is that done?  I didn't see it when I looked.17:18
morganfainbergnkinder, it's from the emit_event decorator on identity_api.authenticate17:21
morganfainbergnkinder, calls into the CADF wrapper http://git.openstack.org/cgit/openstack/keystone/tree/keystone/notifications.py#n22817:21
nkindermorganfainberg: ah, I missed the decorator17:22
morganfainbergnkinder, yeah i had to 2x look :P felt like this was duplicated code.17:23
morganfainbergnkinder, no worries :)17:23
ayoungmorganfainberg, that was my guess:  we want CADF for this, so the stable solution would look different than the Juno solution17:23
morganfainbergayoung, the stable/havana solution, icehouse already has this iirc17:23
ayoung"log upon successful login?"17:23
ayoungyeah, stable/havana  =-= RHOS 4.0 where we need it17:24
morganfainbergayoung, CADF for both success/failure17:24
morganfainbergayoung, stable havana would need t the logging big in this patch17:24
morganfainbergayoung, i don't want to accept this to master though... we have a better system.  is it possible to do logging in stable/havana and not accept the code we already solved another way to master?17:25
*** amcrn has joined #openstack-keystone17:25
ayoungmorganfainberg, yes.   I think that is acceptable.  We need to at least submit a patch upstream before we carry it in RDO, but it does not have to be accepted.  In this case, the patch is minimal enough that it should not cause much angst17:26
nkindermorganfainberg: I agree on not accepting it for master.  CADF is the way.17:26
morganfainbergnkinder, ayoung, ok i'm going to block this one because we have CADF for master then since it is proposed to master, commenting that if stable maintenance will accept it it should go direct to havana17:27
nkindermorganfainberg: +117:27
morganfainbergnkinder, ayoung, does that meet your specific needs as well for backporting in RDO?17:27
ayoungmorganfainberg, ++17:27
ayoungI'll back you on that17:27
*** sbfox has quit IRC17:27
morganfainbergayoung, sounds good.17:28
morganfainberglet me confirm icehouse (i'm fairly certain we have CADF there)17:28
ayoungmorganfainberg, if we want to be pedantic, we could: accept to master,  backport, remove from master, get beaten by ttx.17:28
nkinderayoung: ewww :)17:29
morganfainbergayoung, if it comes to that i'll unblock...but lets not do that .... makes me feel like i need a shower17:29
ayoungmorganfainberg, I think cadf is going to be a killer feature...17:29
ayoungit should allow someone to "unfsck" a Keystone server without having the keys to the kingdom17:30
ayoungI'm thinking of the issues people have had debugging LDAP configs.  With Federation and mapping, its going to be worse17:30
ayoungWith CADF, we could actually have an API for fetching the CADF events for a given scope.....17:31
ayoungso if you have the "auditor" role on a domain you could see all login attempts for that domain.17:31
ayoungNo passwords, but why something failed, like "no role on project"  or "bad password"17:32
morganfainbergayoung, ok https://review.openstack.org/#/c/93975/ put a -2 on that with a comment this is really only needed for stable/havana. if we need to unblock for whatever reason, i'm happy to do that.17:33
ayoungI put a "0  I concur" in there17:33
*** andreaf has quit IRC17:51
*** gokrokve has quit IRC17:52
openstackgerritA change was merged to openstack/python-keystoneclient: Fixes an erroneous type check in a test  https://review.openstack.org/9425617:53
*** gokrokve has joined #openstack-keystone17:54
*** afaranha has quit IRC17:54
browneso is CADF secure?  i ask because ceilometer previously (havana) had no policy enforcement.  so in effect any user could retrieve all audit events from the ceilometer meter.17:55
morganfainbergbrowne, CADF is a notification, ceilometer's lack of security isn't really a factor in that case (you could use another mechanism to store the audit data)17:59
brownemorganfainberg: well in my case i had used a db to store the audit records, which i think is how cadf is typically used.  and then the metering api allows retrieval of those records using any user (not just admins which is what i wanted)18:01
morganfainbergbrowne, sure. that is reasonable. but the ability to remove things isn't a fault of CADF (or even in this case keystone).18:02
*** jamielennox|away is now known as jamielennox18:03
*** packet has joined #openstack-keystone18:04
brownemorganfainberg: sure, i understand its not CADF or keystone at fault.  my point was it was unsafe to use with ceilometer18:04
*** hipster_ has quit IRC18:06
bknudsonlately when I try to start slapd it fails -- *** glibc detected *** /usr/sbin/slapd: double free or corruption (top): 0x00007f8dd34c3df0 ***18:08
bknudsonthis is on ubuntu 12.04 (devstack)18:08
*** leseb has joined #openstack-keystone18:15
openstackgerritA change was merged to openstack/identity-api: Clean up files for identity v2.0 reference  https://review.openstack.org/9419418:17
*** dims has quit IRC18:17
tristanCbknudson: untilthis is reported/resolvedyou can "export MALLOC_CHECK_=0" and it will not crash...18:17
bknudsontristanC: I'll give it a try!18:18
openstackgerritAndreas Jaeger proposed a change to openstack/identity-api: Update to clouddocs-maven-plugin 2.0.2  https://review.openstack.org/9444118:21
tristanCwell I meant it *might* not crash :) Or it will with a traditional segfault...18:22
bknudsonah, it was my own issue with trying to test ssl18:26
*** bvandenh has joined #openstack-keystone18:34
*** JuanManuelOlle has left #openstack-keystone18:38
*** sbfox has joined #openstack-keystone18:42
*** dims has joined #openstack-keystone18:43
*** sbfox1 has joined #openstack-keystone18:43
*** gokrokve has quit IRC18:45
*** sbfox has quit IRC18:46
*** arunkant has joined #openstack-keystone18:51
*** amcrn has quit IRC19:00
gyeeso what's the decision on ldappool, go or no-go?19:01
bknudsongyee: sounds like you've been running with it?19:01
dolphmgyee: evaluate it19:01
bknudsongyee: how big is the change?19:01
gyeebknudson, yes19:01
bknudsongyee: I assume some config options19:02
lbragstadgyee: can you push up something as a WIP?19:02
dolphmgyee: does "yes" mean "big"?19:02
gyeebknudson, pretty trivial change, we just need to wrap the existing connection19:02
gyeenot a big change19:02
gyeeI can push a WIP for you guys to see19:02
bknudsonif we can get it posted I'd rather it was in J119:02
gyeeright now its not configurable, but I can make it optional19:03
*** dims has quit IRC19:03
bknudsonConnectionManager has all sort of options -- size, retry_max, etc.19:03
henrynashdolphm: the mapping table for domain-specif ldap…do you want to do a spec for that, or are we OK with the BP (since we carried this over from IceHouse)19:03
bknudsonI assume we need all that configurable19:03
tristanCMay I ask the status for #1309228 ? OSSA is overdue...19:04
gyeebknudson, yes, I can have all that in a configurable section19:04
dolphmhenrynash: depends on when you think it'll land19:05
dolphmhenrynash: if you can target j1 and feel everyone understands the problem space & proposed solution, skip the -spec19:06
henrynashdolphm: ok, got it…it’s my intention to land it in J119:06
*** erecio_1 has joined #openstack-keystone19:07
ayoungnkinder, I went ahead and  published http://adam.younglogic.com/2014/05/kerberizing-keystone-in-httpd/19:08
dolphmayoung: bknudson: lbragstad: this needs to land ASAP, skip the nits https://review.openstack.org/#/c/94396/19:08
ayoungdolphm, I was looking at that.  Specifically the LDAP side19:08
ayoungthere was a better way to do the DN thing, but it can land after this19:09
ayoungprobably better for a backport if it does.  I'm OK with it if nkinder is19:09
dolphmayoung: ack, and the better way wouldn't be backportable to icehouse anyway19:09
bknudsonI'm not too happy with the dn comparison code either19:09
bknudsonbut if it's not going to cause another vuln we can fix it later19:09
*** erecio_2 has quit IRC19:10
nkinderbknudson: I won't truly be happy with the comparison code until it's truly DN syntax aware, but that's no reason to block this19:10
ayoungbknudson, did you test against liveLDAP?19:11
dstanekyay! i got all of the tests to work with gevent instead of eventlet!19:11
bknudsonayoung: give me a minute and I'll make sure to try it again.19:12
ayoungdstanek, I'm not sure it that makes me want to laugh or cry19:12
ayoungI think cry19:12
dstanekayoung: why is that?19:12
ayoungreplacing one eventloop for another19:13
nkinderayoung: question on your blog post...19:13
ayoungnkinder, fire away19:13
nkinderayoung: it's using kerberos to auth, but Keystone then does lookups via LDAP (as anonymous?)19:13
dstanekayoung: it's all for the scaling baby!19:13
*** bvandenh has quit IRC19:13
ayoungnkinder, yes19:13
nkinderayoung: so it's using REMOTE_USER, then doing a lookup for that user to get groups and figure out roles and all that19:14
dolphmdstanek: does it actually handle more than one request at a time??19:14
lbragstaddolphm: ayoung bknudson nkinder I'm good with https://review.openstack.org/#/c/94396/1 I think those changes can be made later19:14
dstanekdolphm: yes, in my test setup i run (# of cpus +1) processes and each one is asyncio19:15
nkinderayoung: ok, so the next piece would be to use mod_lookup_identity to bypass the LDAP driver19:15
dolphmdstanek: what made you try gevent? and what did it take?19:16
openstackgerritA change was merged to openstack/identity-api: Update to clouddocs-maven-plugin 2.0.2  https://review.openstack.org/9444119:16
dstaneki've always used gevent with flask so i'm familiar with it - it is also making py3 progress19:16
dstanekdolphm: not too much work - totals sunk time to far is about 3 hours19:17
ayoungnkinder, yes.  And the setup I wrote about was based on this one...so it used the LDAP backend to enumerate groups only19:17
dstanekdolphm: i'm going to see if hugh can help me with rally this week after i attempt using it on my own19:18
ayoungbknudson, nkinder what if the site mixes users and groups into the same subtree?  Will if not assignment_dn_norm.endswith(user_tree_dn_norm):  actually work?19:20
ayoungand is that something that people do?19:21
ayoungshouldn't we really check the object class?19:21
nkinderayoung: an objectclass check is the right way of doing things19:21
ayoungnkinder, and I think we have that information at this point, so the check should be based on that, not DN, right?19:22
ayoungdolphm, lets split that patch, and get the SQL portion in.  I think the LDAP portion needs to be rewritten19:23
nkinderayoung: I don't know that we do have the objectclass at that point19:23
ayoungnkinder, we do.  its in the API objects19:24
nkinderayoung: how do we know what objectclass to look for?19:24
*** amcrn has joined #openstack-keystone19:25
*** andreaf has joined #openstack-keystone19:25
ayoungnkinder, it should be comparable to that check19:25
nkinderayoung: ah, user_objectclass from config19:25
ayoungand group_  yep19:26
nkinderayoung: I just -1'd it and recommended an objectclass check19:28
nkinderif sql is what needs to be fixed now, let's just split it19:28
ayoungnkinder, hmmm...his does it without an LDAP fetch19:29
*** gokrokve has joined #openstack-keystone19:29
ayoungnkinder, if we do this blindly, it would be one fetch per assignment.  But maybe that is OK to start19:30
ayoungseems like we should be able to do something like an ldap search filtered by  (objectClass=CONF.user_object_class)19:31
nkinderayoung: it seems like a bit of a mess.  We're looping through the assignments and trying to determine if the id is a user or a group.19:33
*** andreaf has quit IRC19:33
*** andreaf has joined #openstack-keystone19:33
*** hipster has joined #openstack-keystone19:33
*** hipster has quit IRC19:33
*** hipster has joined #openstack-keystone19:34
bknudsonthe only way you'd know the objectclass is to fetch all the member entries19:34
nkinderayoung: if the same id is used for both a user and a group, how can we tell which it applies to?19:34
nkinderI don't think that there is any way to reliably tell which it refers to19:35
bknudsonit does seem like the logic to determine if it's a user or group should be pushed down to get_role_assignments19:35
*** gyee has quit IRC19:36
*** andreaf_ has joined #openstack-keystone19:38
*** andreaf has quit IRC19:39
*** daneyon has quit IRC19:42
*** daneyon has joined #openstack-keystone19:43
*** daneyon has quit IRC19:44
*** daneyon has joined #openstack-keystone19:44
*** daneyon has quit IRC19:48
bknudsonayoung: tried it with openldap and posted some info http://paste.openstack.org/show/81018/19:49
ayoungbknudson, so...I think it is the wrong approach19:50
ayoungbknudson, it really needs to be an objectclass check.  And we might be able to streamline that.19:50
*** marcoemorais has quit IRC19:50
ayoungsomethinkg like  a filter where objectclass=user object class and DN = is in a set of DNs.19:51
ayoungLDAP assignments has been a mess for a while. And its really only used by CERN19:52
ayoungwell, as far as we know...they arethe primary consumers anyway19:52
bknudsonayoung: what's the right approach?19:52
ayoungbknudson, am thinking....my LDAP-fu is weak19:53
ayoungOK,  the first query gives us the set of assignments, with a bunch of member_of values.  Ideally we would do a bulk fetch, filterd by object class.  Let me try it out...19:54
*** erecio_2 has joined #openstack-keystone19:54
bknudsonit's easy enough to split out the SQL change if we can get that in.19:55
*** erecio_1 has quit IRC19:56
ayoungldapsearch -Y gssapi  "(objectClass=posixgroup)"  gets us the object class.  It would be an "anded query"19:57
*** marcoemorais has joined #openstack-keystone19:59
bknudsonby default the attribute is roleOccupant -- so if there was a roleOccupant-of attribute we could search for (&(roleOccupantOf=cn=7d4c172613e446c1a85306f4e993c2ed,cn=0ca027daac884bfcbf82ed15b63b7c3c,ou=Projects,dc=openstack,dc=org)(objectclass=inetorgperson))20:03
bknudsonor (objectclass=groupOfNames) for groups20:03
*** erecio_1 has joined #openstack-keystone20:04
nkinderayoung: is assignment in LDAP done via DN, or via 'id'?20:05
bknudsonwe expect to get DNs back20:05
bknudsonroleOccupant: cn=73114eaf976e4f6480776c3d49213289,ou=Users,dc=openstack,dc=org20:05
bknudsonroleOccupant: cn=suspectid,ou=UserGroups,dc=openstack,dc=org20:05
nkinderbknudson: what if I add an assignment via Keystone?20:05
ayoungbknudson, yes, that is what I am thinking20:06
nkinderbknudson: do I just supply the 'id'?20:06
bknudsonnkinder: that's how keystone does it.20:06
*** erecio_2 has quit IRC20:06
bknudsonnkinder: when you create a role assignment it's by ID20:06
nkinderbknudson: how does it know if I'm referring to a user or a group if the same ID is used for both?20:07
ayoungbknudson,  so we "or" together all of the roleOccupant values then and by objectClass20:07
bknudsonthe code to convert a DN to an ID is by just taking the first value.20:07
bknudsonnkinder: that's the problem!20:07
ayoungso there is no way....20:07
nkinderyes, I think it's fundamentally broken20:07
ayoungwe didn't have groups originally20:07
nkinderid must be unique across users and groups20:07
*** dims has joined #openstack-keystone20:07
bknudsonnkinder: from the example, you can see that they're in different subtrees20:08
ayoungit won't be for LDAP20:08
ayoungNeed josecastro here....20:08
bknudsonbut of course one might lodge their groups under their users if they felt like it20:08
*** amcrn has quit IRC20:08
ayoungbknudson, split out the SQL patch from the LDAP one and submit it separately.  I think LDAP is going to take some more time.  But I am guessing that the SQL side is not really a problem, as IDs are UUIDs20:09
nkinderbknudson: they don't have to be in different subtrees through from an LDAP perspective20:09
bknudsonayoung: the sql side was easy because it just added the assignment type to the query20:09
nkinderbknudson: it is likely a separate tree in most cases though20:09
ayoungwe need to stick the whole DN in the memberOf attribute20:10
bknudsonit is kind of ridiculous to use keystone for assignments considering how you need to set up your schema.20:12
*** marcoemorais has quit IRC20:12
*** marcoemorais has joined #openstack-keystone20:12
bknudsonayoung: have you looked at the sql change? you'd +2 if I split it out?20:12
ayoungbknudson, yes20:14
bknudsonayoung: ok, I'll split it out.20:14
ayoungbknudson, CERN uses it due to replication and scale.20:14
*** ctracey has joined #openstack-keystone20:15
ctraceyquick question - wondering if other have run into this.  Seems keystone hangs during a password update.  This is at least long enough for our front facing LB's to issue a 504 to the client. So we are talking on the order of many seconds.  Anyone else?  This is Havana with mysql token backend.20:16
ayoungwait a seco,  the UserROleAssociation object does hold a DN, not an ID20:17
ctraceythe password gets reset properly, but it looks as if the token used to auth the password update gets destroyed and then everything heads south20:18
ayoungI knew we weren't that dumb....hold on.  bknudson lets see if we can get it with a single query20:18
ctraceyi looked for any outstanding and/or closed bugs but didnt turn anything up...was wondering if someone recollected similar behavior20:18
bknudsonayoung: get what with a single query?20:20
ayoungbknudson, the problem is the call  assignments = self.role.get_role_assignments(proj_dn)20:21
ayoungthat should be different depending on what we want20:21
bknudsonayoung: it returns all the DNs, be they users or groups20:21
openstackgerritSteve Martinelli proposed a change to openstack/keystone: Add openID Connect auth plugin for federation  https://review.openstack.org/6166220:21
ayoungdoesn't have to20:22
bknudsonayoung: how does it know if the DN is for a user or a group?20:22
richmI don't think all LDAP servers support it, but you could use the dereference control to get the objectclasses of all DNs returned by the search20:24
ayoungnkinder, does LDAP have the concept of a join?20:24
richm389 and openldap support, but I suppose AD is the gating factor20:25
ayoungor a nested query:20:25
richmno, you want deref20:25
richmAD supports ASQ, which is very similar20:25
ayoungrichm, OK, so check for the presence of the deref control, and if it exists, use it, and if it does not, do it the slow way?20:26
richmif AD, then use ASQ20:26
ayoungrichm, can you filter the results so that only DNs with objectclasses that match are returned?20:26
bknudsonthat's kind of neat20:27
richmnot with deref20:27
richmnot sure about ASQ20:27
ayoungprobably just a cheap to do 2 LDAP queries as to use the control.20:28
* ayoung still thinks we can do this with one query20:29
bknudsonit's one query per DN20:29
ayoungbknudson, nah, one to fetch the dn, then make a big list, and filter by objectclass20:29
bknudsonwe could fetch all users and filter20:29
ayoungNo we couldn't20:30
bknudsonor all groups, that would be a shorter list20:30
richmso a filter like (|(dn=dn1)(dn=dn2)(....)(dn=dnN)(objectclass=posixGroup))20:30
ayoungrichm, yes20:30
richmnote that dn is not a real attribute20:30
ayoungbut we could potentially run into server limitations20:30
bknudsonldapsearch -LLL -D "cn=Manager,dc=openstack,dc=org" -w "ofs5dac" -s base -b cn=0ddcdc2ee75246999f662a1b09853086,ou=Users,dc=openstack,dc=org "dn=cn=0ddcdc2ee75246999f662a1b09853086,ou=Users,dc=openstack,dc=org"20:30
richmand you can't specify more than one search base in an ldap query20:30
bknudsonthat didn't return anything20:31
*** marcoemorais has quit IRC20:31
ayoungrichm, so you can't | together DN searches?20:31
richmI think you will have to use an ldap extensible filter to query for DN=someDN in a search filter20:31
bknudsonldap's got more extensions than keystone20:32
ayoungjust wait.20:32
*** dstanek is now known as dstanek_zzz20:33
ayoungOK...so is bknudson 's approach right, then?20:34
*** dstanek_zzz is now known as dstanek20:35
ayoungA user and a group with the exact same DN is impossible20:35
bknudsonayoung: don't ask you might not like the answer20:35
ayoungbknudson, class like that is only possible if they are in different subtrees20:36
bknudsonentries can have multiple objectclasses20:36
bknudsonI don't think keystone would work as expected20:36
richman entry can be a user and a group20:36
bknudsonwe don't have to support it20:36
richme.g. a user private group entry could be implemented that way20:36
ayoungrichm, I think that would be OK20:37
ayoungbknudson, is this even a real problem?20:37
openstackgerritA change was merged to openstack/python-keystoneclient: Add mailmap entry  https://review.openstack.org/9266920:37
ayoungI mean, I guess you could have a Nova user and a Nova group, and they would be completely unrelated?20:37
bknudsonayoung: the LDAP part or the SQL part?20:38
ayoungSQL is not areally a problem, unless we have UUID clash20:38
bknudsonayoung: somebody here ran into it.20:38
bknudsonusing LDAP for identity and SQL for assignments20:38
bknudsonr/o LDAP20:39
ayoungbknudson, ah...so SQL assignment fix gets the wors offenders...ok, split that out, and we can discuss the other later.20:39
openstackgerritBrant Knudson proposed a change to openstack/keystone: SQL fix for get_roles_for_user_and_project user=group ID  https://review.openstack.org/9439620:41
openstackgerritBrant Knudson proposed a change to openstack/keystone: LDAP fix for get_roles_for_user_and_project user=group ID  https://review.openstack.org/9447020:41
bknudsonthere they are20:41
bknudsonI'll propose the fix to stable/icehouse when it's +220:42
*** shakamunyi has joined #openstack-keystone20:46
*** browne has quit IRC20:46
*** marcoemorais has joined #openstack-keystone20:50
openstackgerritChristian Berendt proposed a change to openstack/python-keystoneclient: replace string format arguments with function parameters  https://review.openstack.org/9420520:54
*** jsavak has joined #openstack-keystone20:54
*** joesavak has quit IRC20:54
*** marcoemorais has quit IRC21:04
*** jraim has quit IRC21:04
*** jraim has joined #openstack-keystone21:05
bknudsondid we lose the auto-abandon feature?21:07
bknudsonI'd think https://review.openstack.org/#/c/77514/ would be abandoned since jenkins -121:07
morganfainbergbknudson, perhaps21:07
*** browne has joined #openstack-keystone21:11
*** amcrn has joined #openstack-keystone21:13
*** browne has quit IRC21:16
*** erecio_2 has joined #openstack-keystone21:16
*** erecio_1 has quit IRC21:18
*** browne has joined #openstack-keystone21:22
*** daneyon has joined #openstack-keystone21:31
*** andreaf_ has quit IRC21:36
*** daneyon has quit IRC21:36
*** ayoung has quit IRC21:37
*** daneyon_ has joined #openstack-keystone21:40
tristanCbknudson: checking 94396 (the SQL fix), is it normal you change the test_backend_ldap ? Also do I need to reference both new change or only 94470 ?21:48
*** gokrokve has quit IRC21:49
bknudsontristanC: the test in test_backend_ldap overrides the test in test_backend, since the test in test_backend doesn't work for LDAP (since it's not fixed)21:50
tristanCoh ok, just making sure :)21:51
bknudsontristanC: I think you'll want to reference both https://review.openstack.org/#/c/94396/ and https://review.openstack.org/#/c/94470/ for the fix in the announcement21:51
*** amerine_ is now known as amerine21:51
bknudsontristanC: I would also expect there'll be 2 changes for the stable/icehouse fix.21:51
*** ayoung has joined #openstack-keystone21:51
bknudsonor more, since complications keep getting added21:51
tristanCbknudson: alright, thanks for the clarification21:52
*** browne has quit IRC21:52
tristanCwell I guess it will have to wait tomorrow unless we can get all 4 patches approved in the next few hours21:52
bknudsonApparently we'll need to write a function to compare DNs which will require somehow querying the LDAP server for the schema or something21:53
*** jsavak has quit IRC21:55
*** lbragstad has quit IRC21:55
tristanCWell if we could avoid refactoring to much, I would prefer we have the security fix merged before. On the other hand, having a correct "bulletproof" fix is also worthy.21:59
*** harlowja is now known as harlowja_away21:59
bknudsontristanC: people seem happy with the SQL fix.22:00
bknudsonand that's also the one that I'd expect deployers to be more interested in22:00
*** mriedem has joined #openstack-keystone22:00
*** browne has joined #openstack-keystone22:00
mriedempraneshp: re https://review.openstack.org/#/c/93801, looks like an infra issue22:00
mriedemzuul probably went wonky22:00
*** gyee has joined #openstack-keystone22:01
mriedempraneshp: i'll send an email on it to the maintainer22:01
*** dstanek is now known as dstanek_zzz22:01
*** stevemar has quit IRC22:02
morganfainbergdolphm, so what was the end result on {program}-spec vs {project}-spec?22:02
morganfainbergdolphm, i lost track of the meeting22:02
dolphmmorganfainberg: program wins22:02
*** sbfox1 has quit IRC22:03
morganfainbergdolphm, yay22:03
dolphmmorganfainberg: something like 12-222:03
morganfainbergi approve of that22:03
dolphmor 12 to 2 to 122:03
dolphmthere were 3 options22:03
dolphmone of which was 'markwash'22:03
*** nkinder has quit IRC22:03
dolphmwhich ended up getting 2 votes22:03
mriedempraneshp: fyi there is a db2 CI wiki but it's pretty bare right now, i'll get someone to update that with contact info and how to recheck https://wiki.openstack.org/wiki/IBM/DB2-TEST22:03
*** leseb has quit IRC22:04
praneshpmriedem: thanks a lot22:05
*** ChanServ sets mode: +o morganfainberg22:06
*** morganfainberg changes topic to "Juno-1 June 12th! New formalized Identity-spec process for Juno-2 and beyond blueprints."22:06
*** ChanServ sets mode: -o morganfainberg22:07
*** rodrigods_ has joined #openstack-keystone22:07
*** rodrigods has quit IRC22:08
*** harlowja_away is now known as harlowja22:08
bknudsonapparently jdennis has a class that implements DN compare22:09
mriedempraneshp: email sent, thanks for bringing it to my attention22:10
*** bknudson has quit IRC22:10
mriedempraneshp: i also rechecked the patch but not sure it will do anything if there are zuul issues22:11
*** sbfox has joined #openstack-keystone22:12
*** packet has quit IRC22:14
*** nkinder has joined #openstack-keystone22:16
*** mriedem has left #openstack-keystone22:18
*** rodrigods_ has quit IRC22:19
*** browne has quit IRC22:22
*** dstanek_zzz is now known as dstanek22:23
*** gordc has quit IRC22:23
*** rodrigods has joined #openstack-keystone22:30
*** rodrigods has joined #openstack-keystone22:30
*** dstanek is now known as dstanek_zzz22:33
*** dims has quit IRC22:35
rodrigodsbknudson, please review https://review.openstack.org/#/c/91578/ when you have a moment =)22:40
*** david-lyle has quit IRC22:43
*** daneyon_ has quit IRC22:49
gyeejamielennox, you awake?22:52
*** browne has joined #openstack-keystone22:52
jamielennoxgyee: yea i'm here22:53
gyeejamielennox, Bob is working on a patch for keystoneclient Session to add the chunking functionality22:54
*** bknudson has joined #openstack-keystone22:54
gyeelooks like Glance client is using it22:54
jamielennox... what's required?22:54
jamielennoxso if you pass data=gen() it should work now22:55
gyeeright, essentially, just wrap the data into a generator22:55
jamielennoxso what is he prposing on session to help with that?22:55
gyeequestion is do we want that change to be in glanceclient or keystoneclient22:56
jamielennoxis he here? i dont know his nick22:56
gyeebasically inspect the header for Content-Encoding22:56
gyeeif it is chunked, then wrap the data into a generator22:56
jamielennoxumm, can't we fix it higher up than that?22:57
jamielennoxas in make glanceclient actually pass the generator as exected?22:57
gyeeyeah, that's an option, the benefit of doing it in keystoneclient is that the code is shared22:57
bknudsonkeystoneclient should be able to handle chunked transfers too22:57
jamielennoxthen hack the old case to add the content-tye and do the generator from there22:58
jamielennoxbknudson: it does22:58
*** bobt has joined #openstack-keystone22:58
gyeejamielennox, bobt's here22:58
jamielennoxgyee: kind of - i mean in a way you're trying to wrap a low level interface to a high level one22:58
gyeejamkelennox, well, Session is kind of low lever22:59
jamielennoxthe use of a generator is supposed to hide you from things like setting content-type: chunked22:59
bobttransfer-encoding: chunked22:59
jamielennoxyou're doing the reverse, if you detect the content-type: chunked header then you wrap the data into a generator and then make requests re-set the content-type22:59
jamielennoxsorry transfer-encoding22:59
jamielennoxlet me check glance-client22:59
gyeeglanceclient is doing the exact same23:00
gyeetee off on the transfer-encoding header23:00
jamielennoxright, but glanceclient is kindof stupid like that23:00
*** ayoung has quit IRC23:01
gyeeI think they make some assumptions23:01
jamielennoxgyee: bobt: do you know where it is set?23:01
jamielennoxyea, line 310 if content_length is None23:02
gyeelooks like raw is always chunked23:02
gyeeah, for streaming, where content-length is not know ahead of time23:03
gyeejamielennox, we can move that logic to keystoneclient, with the same assumption23:04
*** amcrn_ has joined #openstack-keystone23:04
jamielennoxgyee: well i was wondering can we fix glanceclient's http?23:05
jamielennoxfrom requests:         is_stream = all([23:05
jamielennox            hasattr(data, '__iter__'),23:05
jamielennox            not isinstance(data, (basestring, list, tuple, dict))23:05
jamielennox        ])23:05
gyeejamielennox, we can fix it in either glanceclient or add it to keystoneclient, the benefit of having it in keystoneclient is that it is shared23:06
gyeebut if only glance needs it then we can fix it there instead23:06
*** amcrn has quit IRC23:06
bobtdoes cinder need chunking support or just glance?23:07
jamielennoxgyee right but we're having to share: https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/http.py#L290-L31323:07
jamielennoxthere's more than a simple check there23:07
jamielennoxamongst other things it sets content-type23:08
gyeeright, basically adding raw_request()23:08
jamielennoxi'm wondering if we just make glanceclient create a generator, could we make glanceclient work like requests23:08
jamielennoxbecause i think the notion that i pass a generator and it becomes chunked is easier than23:09
gyeeI suppose we could, by having a glance Session which subclass keystoneclient Session, something like that23:09
jamielennox        if (('body' in kwargs) and (hasattr(kwargs['body'], 'read') and method.lower() in ('post', 'put'))): then it becomes chunked23:09
gyeelike I said, if only glance needs it then there's no benefit of doing it in keystoneclient23:10
gyeejamielennox, how about we do the fix on glance side for now, and if we discover more clients need it later, we add it to keystoneclient?23:11
jamielennoxgyee: yep if we need it we need it - but for now i'd like session to stay as clean as possible23:14
jamielennoxif it's an enhancement then that's great23:14
jamielennoxbut this is a little like carrying other peoples hacks23:14
gyeealrighty then23:14
gyeejamielennox, also are you pushing the append command line args patch for keystoneclient anytime soon?23:15
jamielennoxdoes read() given an iterator?23:15
jamielennoxyea, i've got a session from conf i'm working on now23:15
jamielennoxthen i'll look at auth and session from cli23:15
jamielennoxunfortunately some of that's broken for keystoneclient's shell23:16
*** ayoung has joined #openstack-keystone23:16
gyeelet me check, I think read() have a few options23:16
gyeejamielennox, Response.raw can do streaming23:18
jamielennoxgyee: because one of those checks is hasattr(kwargs['body'], 'read') so at some point the glanceclient is passing around a file object23:18
jamielennoxso it should be fairly easy to identify where you need to replace with a generator23:18
jamielennox(file or other streaming - which i don't know what else can stream)23:18
gyeeits essentially a file-like object23:19
*** dstanek_zzz is now known as dstanek23:19
gyeebknudson, do you know any other clients have the need for chunking functionality? you mentioned we may need this in keystoneclient earlier23:21
*** diegows has quit IRC23:21
jamielennoxgyee: but that would imply they are receiving from another API and directly streaming that onto something else?23:22
bknudsongyee: chunked transfer coding is part of the http 1.1 spec. All clients should be able to do it23:22
bknudsonHTTP 1.1 came out in 1999. seems like we should be able to support it by now23:22
gyeeits only like what 15 years?23:23
bknudsonexactly. I don't think it's going to go away.23:23
bknudsonI was worried at first, but it seems to have caught on23:23
bknudsonwas there a concern about Expects: 100 Continue also?23:24
bknudsonExpect: 100-continue is the header23:25
gyeejamielennox, if Glance is using Swift backend, perhaps they are passing the file object directly into Swift?23:26
gyeelet me check23:26
gyeebknudson, is the 100-continue header used for streaming?23:27
gyeeI don't remember seeing it being used23:28
bknudsongyee: it's used to more efficiently handle an operation that can be aborted by looking at the headers23:28
bknudsonapache will send it if it's used as a proxy23:29
jamielennoxgyee: hmm, maybe23:29
bknudsonit sends the headers for the original request with "Expect: 100-continue", and then it won't send the body unless it gets back the 100 Continue response.23:29
bknudsonI'd expect glance would want to use it if they can23:30
jamielennoxgyee: so https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/utils.py#L293 looks like the only place that does an open like that23:33
jamielennoxwhich is used https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/shell.py#L310 and https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/shell.py#L22923:34
jamielennoxand the equivalent in v2 shell23:34
jamielennoxso i'm not sure if that's a compatability issue we need to maintain23:35
gyeejamielennox, yeah you're right, doesn't make sense to add this to keystoneclient as they are using it in a specific way23:35
jamielennoxso it should be as simple as fixing the glanceclient image.update() to create a generator from a read call23:36
jamielennoxand image.create()23:36
jamielennoxand then interpretting the generator in raw_request as meaning it should be chunked23:37
jamielennoxwhich isn't that simple :)23:37
jamielennoxso bknudson, gyee whilst your here - i need a way to reauthenticate an auth_plugin and at the moment the only interface i have is get_token23:38
jamielennoxso if you get a 401 then refetch a token even if it's still in the valid time range23:38
jamielennoxi think my choices are essentially add force=True to get_token() to always fetch a new token23:39
bknudsonit might have been revoked23:39
jamielennoxbknudson: right23:39
jamielennoxor add an invalidate() method to the plugin, call that, and then expect the next call to get_token will do the right thing23:39
jamielennoxi think that flow is kind of easier - but neither feels great. any opinions?23:40
jamielennoxthis is something that will be handled by the session object so a user should never really care23:40
jamielennoxi should have stuck to my guns originally and had authenticate() and get_token() as seperate methods23:41
bknudsondoes it also fetch if the token is close to expiring?23:41
jamielennoxbknudson: yes23:41
gyeejamielennox, didn't we have that force new token option at one point?23:41
jamielennoxgyee: yes, initially when i had a seperate authenticate() call so i could trigger a force23:41
jamielennoxi was pursuaded that it wasn't needed and let get_token() do everything23:42
bknudsonif I call authenticate() then get_token() returns a different token?23:42
bknudsonand authenticate() fails if my user is disabled?23:42
jamielennoxbknudson: sorry i think i'm confusing the situation with the cross-conversation - there is no authenticate() call at the moment23:42
bknudsonis that how you'd expect it to work, though?23:43
jamielennoxthere is an auth plugin that you can call get_token() on and it should return you something that it feels is valid23:43
jamielennoxwhen the actual auth happened should be irrelevant (cause it might have happened prior to be provided to the session)23:43
bknudsonthe auth plugin would have an invalidate to invalidate its token?23:44
jamielennoxhowever in the case of a 401 i need to make the auth token try again23:44
jamielennoxright so i could call plugin.invalidate() redo the call and the otken would be refetched on the next round23:44
bknudsonthat would make sense to me.23:44
jamielennoxthat feels better to me that putting force= on the get_token23:45
gyeeyeah, force= sounds better23:48
gyeejust need to document the expectation23:48
gyeebknudson, https://review.openstack.org/#/c/94396/2/keystone/tests/test_backend_ldap.py, why is this test in the sql patch?23:51
gyeethere's a separate LDAP patch right?23:52
bknudsongyee: since ldap doesn't handle the situation correctly, the general backend test doesn't work for it23:52
bknudsongyee: so ldap has to override the base test23:52
bknudsongyee: that test shows that ldap doesn't work correctly because the role is returned when it shouldn't be23:53
bknudsongyee: the next patch in the series fixes the LDAP backend to work like the SQL backend so then we can get rid of the overridden test23:53
gyeebknudson, ah, I see what you mean23:54
gyeethe code looks almost identical23:54
dstanek we're eventually going to start using just one port for keystone instead of two right?23:58
bknudsondstanek: yes, 44323:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!