| jamielennox | well more that the test shouldn't operate in different ways depending on the version of the library you have installed | 00:00 |
|---|---|---|
| rodrigods | morganfainberg, that was the issue preventing the keystone client tests to pass, right? | 00:00 |
| jamielennox | so it was fine initially but we shouldn't be testing it any more | 00:01 |
| morganfainberg | jamielennox, ++ | 00:01 |
| morganfainberg | rodrigods, yes. | 00:01 |
| rodrigods | cool, have a patch waiting for this fix =) | 00:01 |
| *** dstanek_zzz is now known as dstanek | 00:18 | |
| dstanek | rodrigods: what was your comment? | 00:20 |
| dstanek | rodrigods: ah, about putting the type check back in? we should almost never type check | 00:21 |
| dstanek | bknudson: if you feel strongly about the whitespace i can put it back - it makes more sense to me to have the assertions together in the code since they are checking the same attributes (logical grouping and all that) | 00:23 |
| bknudson | dstanek: I do feel strongly about the whitespace. I don't want there to be changes where whitespace is added and then the next change removes it. | 00:24 |
| rodrigods | dstanek, was convinced by morganfainberg (thought only about maintaining the original behavior instead of realizing that it was wrong in the first place) | 00:24 |
| morganfainberg | bknudson, i'm fine with that stance, once that is resolved I'll happily +2 as well. (holding based on your -1) | 00:25 |
| morganfainberg | bknudson, that is if you aren't around when change is pushed up. | 00:25 |
| morganfainberg | ... or something something ... brain | 00:26 |
| bknudson | I'm fine with the change once the whitespace issue is fixed. I think the code was just there because it was weird that requests returned a tuple | 00:27 |
| bknudson | and now requests is less weird | 00:27 |
| ayoung_dad_mode | morganfainberg, ok...I've been thinking about stacked policy distribution. I think I like this idea of the checksum being distributed in the token. Basically, the token contains a checksum that tells auth_token middleware if it needs to fetch any policy files | 00:35 |
| ayoung_dad_mode | something like: checksum together all of the policy files for a project. If you don't have that bundle, ask the server for a refresh. | 00:35 |
| ayoung_dad_mode | Server refresh will also calculate the checksum, so if the token is out of sync, you'll know it. | 00:36 |
| jamielennox | ayoung_dad_mode: in the token? that's a little unusual - because the same could be said for revocations etc | 00:37 |
| ayoung_dad_mode | In theory, if the policy checksum somehow changes, if any one of the policy files change, then the token can either be treated as revoked, or accepted with no loss of security | 00:37 |
| ayoung_dad_mode | jamielennox, the way I see it, the projects affected are going to be scattered, and in a system with a lot of endpoints, doing a blast of updates for every litle change will be chatty | 00:38 |
| jamielennox | ayoung_dad_mode: what if the token is older than the policy change? | 00:38 |
| ayoung_dad_mode | but an indicator telling the service: you don't have the latest. | 00:38 |
| ayoung_dad_mode | jamielennox, I think that is OK, so long as the server returns a response that indicates what the current valid one is | 00:38 |
| morganfainberg | ayoung_dad_mode, what if the token and the server are the same but there is a new policy? | 00:39 |
| ayoung_dad_mode | maybe just record the bad checksum to prevent rechecking it | 00:39 |
| rodrigods | bknudson, just replied your comments at https://review.openstack.org/#/c/91578/ | 00:39 |
| ayoung_dad_mode | morganfainberg, we could revoke the token, but that seems harsh | 00:39 |
| jamielennox | ayoung_dad_mode: so we were looking at a daemon for auth_token anyway that could listen for event changes like this | 00:39 |
| ayoung_dad_mode | jamielennox, yeah, but if there are say 200 endpoints and thousands of proejcts we probably don't want to be blasting all of that around. | 00:40 |
| ayoung_dad_mode | I'd rather make it pull on demand | 00:40 |
| bknudson | rodrigods: I don't get why the client API returns HttpNotImplemented -- that's telling the application that the server responded with Not Implemented | 00:40 |
| bknudson | not that the python API isn't implemented. | 00:41 |
| rodrigods | bknudson, i read the exception in another way: "im failing because the HTTP part is not implemented" | 00:41 |
| rodrigods | so "I didn't even tried to perform the request" | 00:42 |
| jamielennox | ayoung_dad_mode: so i guess the cost is that you need to have calculated the full policy when you issue the token | 00:43 |
| jamielennox | i assume that's possible given we know the project etc | 00:44 |
| *** dstanek is now known as dstanek_zzz | 00:44 | |
| ayoung_dad_mode | jamielennox, yeah. Ugh...just realized that this would only work with endpoint binding | 00:45 |
| ayoung_dad_mode | with tokens for general use, there is no way to know which service to calculate policy for | 00:45 |
| bknudson | rodrigods: the exception is HttpNotImplemented(HttpServerError) -- so it's indicating a server error and not a client error | 00:46 |
| *** hipster has joined #openstack-keystone | 00:47 | |
| *** ayoung_dad_mode is now known as ayoung | 00:47 | |
| rodrigods | bknudson, is there a better exception to be raised there? | 00:48 |
| ayoung | our token mechanism sucks. | 00:48 |
| bknudson | rodrigods: NotImplemented | 00:48 |
| bknudson | rodrigods: well, that's not a good one either. Create a new one. | 00:48 |
| rodrigods | bknudson, ok, about the message, did I understand correclty | 00:50 |
| rodrigods | ? | 00:50 |
| bknudson | rodrigods: no, just remove the +. you don't need + to continue a string | 00:50 |
| rodrigods | bknudson, ok | 00:51 |
| ayoung | I want endpoint binding Dagnabit | 00:51 |
| *** hipster has quit IRC | 00:51 | |
| rodrigods | bknudson, MethodNotImplemented(ClientException), clear enough? | 00:53 |
| bknudson | rodrigods: that sounds great! | 00:53 |
| *** browne has quit IRC | 01:01 | |
| ayoung | jamielennox, ok, taking this from the top: start up a service, and it should have no policy file. It can query to keystone right away, but the keystone server might not be up yet. So, it waits until the first request, and fetches the policy file for the service. But, at this point, it knows it has no policy file for the specified domain or project, so it should ask for this now, too. So, from this point on, it fetches | 01:04 |
| ayoung | policy files based on notifications | 01:04 |
| ayoung | and if it restarts, it has to dump its cache in case it has missed any notifications | 01:04 |
| *** dstanek_zzz is now known as dstanek | 01:05 | |
| openstackgerrit | Rodrigo Duarte Sousa proposed a change to openstack/python-keystoneclient: Add /role_assignments endpoint support https://review.openstack.org/91578 | 01:07 |
| rodrigods | bknudson, ready for another round of comments =) | 01:07 |
| *** stevemar has joined #openstack-keystone | 01:09 | |
| jamielennox | ayoung: so i'm still not sure on the concept of whether we should have service policy and overlay with project policy - or just have a project policy that inherits | 01:10 |
| ayoung | jamielennox, you mean, to premerge or process them in series? | 01:10 |
| ayoung | jamielennox, so, I'm assuming the latter: and we can fetch only the policy files we care about. | 01:12 |
| ayoung | jamielennox, maybe I am planning for too big a scale, but the MOC folks got me thinking big | 01:12 |
| jamielennox | ayoung: i need to think about distribution a bit more | 01:13 |
| jamielennox | ayoung: are we still discussing that subsequent policy files should be additive? | 01:13 |
| ayoung | like, a couple hundred companies sharing a datacenter, each with a couple of nova instances, for different qualities of service | 01:13 |
| ayoung | yeah, for the most part | 01:13 |
| jamielennox | so that a project can only further restrict the policies of it's parent | 01:13 |
| ayoung | I think an endpoint should be able to completely override the policy file for a service | 01:13 |
| ayoung | since it could do that anyway | 01:13 |
| ayoung | and then a domain on down to project should probably be only additive...unless the top level one doesn't care | 01:14 |
| jamielennox | ayoung: so i'm more or less in the service scoped tokens exist mindset - and i see that the policies for service scoped vs project scoped are completely different | 01:14 |
| jamielennox | not additive or combinatory in any way | 01:14 |
| ayoung | yeah, service scoped are either service or endpoint. But I could see an endpoint wnated to replace just specific rules...to limit copy/paste errors | 01:15 |
| *** dstanek is now known as dstanek_zzz | 01:15 | |
| ayoung | but that could actually be handled at the server level, not the policy file processing | 01:15 |
| jamielennox | ayoung: so i think we need to better define what an endpoint is | 01:15 |
| ayoung | IE PATHC policy/service | 01:15 |
| ayoung | PATCH | 01:15 |
| jamielennox | in the service -> server -> urls description | 01:16 |
| jamielennox | currently endpoint represents a URL and not a server | 01:16 |
| ayoung | right...well, a set of urls in the case of Keystone, as admin and main and v3 all are linked together in one endpoint | 01:17 |
| jamielennox | ayoung: ok so endpoint = server and we should express that better | 01:18 |
| ayoung | jamielennox, did I throw server in there or did you | 01:18 |
| jamielennox | i said server so that i didn't re-use the word endpoint | 01:19 |
| ayoung | ok | 01:20 |
| ayoung | so, yeah, we are abusing endpoint | 01:20 |
| ayoung | no surprise there, we abuse everything | 01:20 |
| ayoung | jamielennox, Actually, looking here http://en.wikipedia.org/wiki/Service-oriented_architecture we might be using it correctly, but anyway... | 01:23 |
| ayoung | lets assume that server == endpoint for our purposes | 01:23 |
| ayoung | then a policy bundle is per endpoint, and a project adds to that bundle to get the project specific policy. | 01:24 |
| ayoung | I'm mostly concerned about project policies, as the endpoint and service ones will change infrequently enought that we could rely on notifications for those | 01:24 |
| *** marcoemorais has quit IRC | 01:26 | |
| jamielennox | do you really think that a policy will *change* that much? | 01:27 |
| jamielennox | even per project | 01:27 |
| ayoung | jamielennox, no one policy will, but in the aggregate, with hierarchical? Yeah | 01:27 |
| ayoung | Its the hierarchical part that scares me | 01:27 |
| ayoung | it means that changing policy for one top level project will change it for everything underneath | 01:28 |
| jamielennox | ayoung: so if policy is purely more restrictive that's not too bad because you just process every policy as an AND | 01:31 |
| jamielennox | order doesn't really matter | 01:32 |
| jamielennox | so it should be possible to specify multiple policy lines per protected method | 01:32 |
| ayoung | jamielennox, yeah. But even if a rule is "this is the default, but if there is a follow on rule, ignore me and use that instead" should work. It would be more processing, and I don't think it would be that common, but it would allow a reasonable default (the policy file we ship now) and also a smarter override | 01:34 |
| ayoung | so you don;t need both admin and redhat_admin, just redhat_admin in some project to, say, add a role to a user | 01:34 |
| jamielennox | ayoung: i think we target the purely more restrictive for now and look at policy enhancements later | 01:34 |
| *** rodrigods has quit IRC | 01:34 | |
| jamielennox | one of the ARBAC things (that i haven't read yet) is hierarchical roles for that | 01:35 |
| ayoung | jamielennox, even if we implement, we should plan for the other. Ah, yes ARBAC | 01:35 |
| jamielennox | i put the email with resources to -dev - but it's a 25 page academic paper | 01:36 |
| jamielennox | not sure when i'll get into reading that one | 01:36 |
| *** dstanek_zzz is now known as dstanek | 01:37 | |
| ayoung | jamielennox, hierarchical regions are another wrinkle, too. | 01:38 |
| openstackgerrit | David Stanek proposed a change to openstack/python-keystoneclient: Fixes an erroneous type check in a test https://review.openstack.org/94256 | 01:39 |
| jamielennox | ayoung: yea, i'm aware - i don't like the way they were dumped on us, i had lots of follow up with 'how do i work with this as a client' that was ignored when i stopped blocking the reviews with it | 01:39 |
| ayoung | jamielennox, and are policies going to follow down the region hierarchy as well as the project hierarchy? | 01:39 |
| jamielennox | ayoung: i don't view regions that way - but i fear i might be way out on my own with regions | 01:40 |
| ayoung | so you could, in theory, have a different base policy for the same project policy, depending on endpoint, etc | 01:40 |
| jamielennox | as far as i'm concerned regions are just a label that helps you differentiate things | 01:40 |
| jamielennox | we have never applied any sort of authentication or logic based on region | 01:41 |
| ayoung | jamielennox, so would policy only be specified at the service or endpoint level, with region completely ignored? | 01:42 |
| ayoung | I would also be OK with two levels of policy; service OR endpoint and project specific. If one project specifies a policy file, it supercedes all policy written above it. You have to have the higher level policy authorization to load a policy for the lower level anyway. | 01:44 |
| jamielennox | yes to ignoring region (at least for now) | 01:45 |
| jamielennox | regarding policy i see ONLY two types: service scoped or project scoped depending on your token | 01:45 |
| jamielennox | policies are inheritted in the same way as roles, including when you get to a break (what we considered private projects) you should recreate a policy for it | 01:46 |
| jamielennox | we would then obviously have some basic policies that would get defaulted to for those projects | 01:47 |
| *** hipster has joined #openstack-keystone | 01:47 | |
| jamielennox | policies (for now) are additive only | 01:47 |
| ayoung | jamielennox, so overloading policy per endpoint is going to be important. If only to allow a testable roll-out of a new policy file | 01:49 |
| jamielennox | so per project per endpoint | 01:50 |
| jamielennox | ok, that's fair | 01:50 |
| ayoung | and the whole different organizations owning different services, too. But In general, I think we are aligned. The region thing, though, might be how we split out organizations | 01:50 |
| jamielennox | but same rules apply, endpoint policy should be additive to project policy | 01:51 |
| ayoung | like IBM and HP in the same data center. Each have a set of Nova servers with different QofS. Each org wants to use their same base policy for Nova. | 01:51 |
| *** hipster has quit IRC | 01:51 | |
| ayoung | Regions is the only grouping mechanism we have. Without it, you have a window where you bring up a server and it gets the wrong policy | 01:52 |
| jamielennox | ayoung: wait - why is that a problem? those organizations should be seperated by project | 01:52 |
| jamielennox | within their own private project tree | 01:52 |
| ayoung | No, project is orthoganal | 01:52 |
| ayoung | the MOC approach is mix-and-match | 01:52 |
| ayoung | I get cinder from here, Nova from there... | 01:52 |
| jamielennox | oh right the MOC thing | 01:52 |
| ayoung | its ambitious. But I've learned to listen to Orran | 01:53 |
| ayoung | He kindof gets this.... | 01:53 |
| jamielennox | yes, as providers that makes sense, just not so much as organizations running their own thing | 01:53 |
| ayoung | right | 01:53 |
| ayoung | God, there are so many ways to screw this up | 01:54 |
| jamielennox | yea | 01:54 |
| jamielennox | i'm still unsure how to stack it | 01:54 |
| jamielennox | because that would imply that an endpoint might want to set a basic policy that should be project independant | 01:54 |
| ayoung | I think we keep playing with it mentally before we write a lick of code. | 01:55 |
| jamielennox | i don't know if that can work | 01:55 |
| ayoung | We saw with the domain-id problem that we don't really enforce policy at the right level. If we were doing it right, it would be enforced at the container. I should be able to move a user from one domain to another, if I have permissions in both domains. Same is true of proejcts | 01:56 |
| ayoung | we don't enforce: you can move something to this bucket if you have rights on the bucket | 01:56 |
| jamielennox | maybe, i'm of the opinion that a role only makes sense within a project as other projects might use other names for different things | 01:56 |
| jamielennox | how do you transition roles in that situation/ | 01:57 |
| ayoung | jamielennox, not within a service? Interesting... | 01:59 |
| jamielennox | ayoung: well they only exist within the context of whatever you are scoped to. So a service scoped token may defined roles per service, but a project scoped token will define roles per project and there is no implication that they should ever mean the same tihng | 02:00 |
| ayoung | I kinda want to talk with SELinux Dan about this. I'm guessing he's considered some of these issues himself | 02:00 |
| ayoung | jamielennox, I like the rule that a project can only overwrite what comes from higher in the project hierarchy. As it might be the case that a project would want to have different policies based on which endpoint specific policy it was running, so for that we say "use a different subproject" | 02:04 |
| jamielennox | ayoung: only overwrite? or only futher restrict? | 02:05 |
| ayoung | overwrite for other project policy files | 02:05 |
| ayoung | like, if the role is redhat_admin at the top, and then it changes to fedora_admin for a resource allocated to fedora | 02:06 |
| ayoung | you can always make the lower level policy a copy of the higher one there. | 02:06 |
| ayoung | Probably use the same role for endpoints, and even regions | 02:06 |
| ayoung | if we provide a good tool for merging policy off line, we can do "only replace" | 02:07 |
| jamielennox | surely we aren't the first people to deal with something like this, is there any prior art? | 02:07 |
| ayoung | there must be. I'm afraid to ask David Chadwick, as his responses are more "what is possible" instead of "what is practical" | 02:08 |
| ayoung | http://csrc.nist.gov/groups/SNS/rbac/ | 02:09 |
| ayoung | whatever we come up with needs to be well reviewed | 02:09 |
| ayoung | jamielennox, I'm headed into the office tomorrow, and I'm going to kill some trees to read these things without destroying my eyes | 02:10 |
| ayoung | I suspect dpal will have some insight as well. | 02:10 |
| *** dstanek is now known as dstanek_zzz | 02:10 | |
| ayoung | jamielennox, I could also see rules for "what roles can you grant someone" | 02:10 |
| ayoung | IE, if you are Jr Sys admin, you can grant Member. If you are Senior Admin you can grant JrAdmin | 02:11 |
| jamielennox | that was one of the things mentioned for ARBAC | 02:11 |
| ayoung | yeah, its the meta stuff: creating new projects and assigning roles that is the most likely to create security holes | 02:12 |
| *** diegows has quit IRC | 02:14 | |
| *** xianghui_afk has joined #openstack-keystone | 02:15 | |
| ayoung | jamielennox, ok...I'm more muddled about this than I was when I stared. Which is probably a good thing. I suspect the best we'll get out of Juno is "fetch policy for endpoint" | 02:15 |
| jamielennox | yea, i would prefer to let hierarchical projects actually land before figuring out project based policies | 02:16 |
| ayoung | ++ | 02:16 |
| ayoung | we can let _0x44_ write the stackable policy mechanism while we deal with delivery. As it is, we'll still have to deal with notifications and a fetch mechanism | 02:17 |
| ayoung | And then we can deal with delivery of stackable in Kilo. | 02:18 |
| jamielennox | who? | 02:18 |
| ayoung | Chris MacGown, the Oslo guy | 02:18 |
| jamielennox | ok | 02:18 |
| *** dstanek_zzz is now known as dstanek | 02:35 | |
| *** amcrn has quit IRC | 02:37 | |
| *** hipster has joined #openstack-keystone | 02:39 | |
| *** ayoung has quit IRC | 02:44 | |
| *** hipster has quit IRC | 02:44 | |
| *** mberlin1 has joined #openstack-keystone | 02:44 | |
| *** hipster_ has joined #openstack-keystone | 02:46 | |
| *** mberlin has quit IRC | 02:47 | |
| *** Guest82130 has quit IRC | 02:49 | |
| openstackgerrit | ayoung proposed a change to openstack/keystone: Compressed Token Provider https://review.openstack.org/91145 | 02:52 |
| *** marcoemorais has joined #openstack-keystone | 02:58 | |
| *** praneshp_ has quit IRC | 03:01 | |
| *** praneshp_ has joined #openstack-keystone | 03:04 | |
| *** praneshp_ has quit IRC | 03:05 | |
| *** marcoemorais has quit IRC | 03:14 | |
| *** praneshp_ has joined #openstack-keystone | 03:15 | |
| *** praneshp_ has quit IRC | 03:20 | |
| *** marcoemorais has joined #openstack-keystone | 03:20 | |
| *** daneyon has joined #openstack-keystone | 03:23 | |
| *** hipster_ has quit IRC | 03:23 | |
| *** marcoemorais has quit IRC | 03:26 | |
| *** hipster has joined #openstack-keystone | 03:32 | |
| *** jaosorior has joined #openstack-keystone | 03:42 | |
| *** morganfainberg is now known as morganfainberg_Z | 03:49 | |
| *** gyee has quit IRC | 03:54 | |
| *** hipster has quit IRC | 03:57 | |
| *** gokrokve has quit IRC | 04:03 | |
| *** dstanek is now known as dstanek_zzz | 04:04 | |
| *** praneshp_ has joined #openstack-keystone | 04:05 | |
| *** praneshp__ has joined #openstack-keystone | 04:10 | |
| *** praneshp_ has quit IRC | 04:12 | |
| *** gokrokve has joined #openstack-keystone | 04:19 | |
| *** dstanek_zzz is now known as dstanek | 04:27 | |
| *** hipster has joined #openstack-keystone | 04:27 | |
| *** hipster_ has joined #openstack-keystone | 04:29 | |
| *** hipster has quit IRC | 04:33 | |
| *** hipster_ has quit IRC | 04:33 | |
| openstackgerrit | Nathan Kinder proposed a change to openstack/keystone: Adds log message upon token granting https://review.openstack.org/93975 | 04:39 |
| *** marcoemorais has joined #openstack-keystone | 04:47 | |
| *** marcoemorais1 has joined #openstack-keystone | 04:49 | |
| *** dstanek is now known as dstanek_zzz | 04:49 | |
| *** marcoemorais has quit IRC | 04:52 | |
| *** ajayaa has joined #openstack-keystone | 05:01 | |
| *** hipster has joined #openstack-keystone | 05:07 | |
| *** hipster has quit IRC | 05:12 | |
| *** jamielennox is now known as jamielennox|away | 05:32 | |
| *** hipster has joined #openstack-keystone | 05:47 | |
| *** hipster has quit IRC | 05:52 | |
| *** daneyon has quit IRC | 05:53 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/keystone: Imported Translations from Transifex https://review.openstack.org/90288 | 06:02 |
| *** gokrokve has quit IRC | 06:09 | |
| *** harlowja is now known as harlowja_away | 06:28 | |
| *** ajayaa has quit IRC | 06:39 | |
| *** gokrokve has joined #openstack-keystone | 06:40 | |
| *** gokrokve_ has joined #openstack-keystone | 06:42 | |
| *** gokrokve has quit IRC | 06:45 | |
| *** xianghui_afk has quit IRC | 06:47 | |
| *** hipster has joined #openstack-keystone | 06:47 | |
| *** gokrokve_ has quit IRC | 06:47 | |
| *** dstanek_zzz is now known as dstanek | 06:50 | |
| *** hipster has quit IRC | 06:52 | |
| *** derek_c has joined #openstack-keystone | 06:58 | |
| *** dstanek is now known as dstanek_zzz | 07:00 | |
| *** xianghui_afk has joined #openstack-keystone | 07:03 | |
| *** BAKfr has joined #openstack-keystone | 07:04 | |
| *** henrynash has joined #openstack-keystone | 07:10 | |
| *** praneshp__ has quit IRC | 07:11 | |
| *** henrynash has quit IRC | 07:12 | |
| *** andreaf has quit IRC | 07:17 | |
| *** leseb has joined #openstack-keystone | 07:34 | |
| *** gokrokve has joined #openstack-keystone | 07:42 | |
| *** leseb has quit IRC | 07:42 | |
| *** leseb has joined #openstack-keystone | 07:43 | |
| openstackgerrit | Marcos FermÃn Lobo proposed a change to openstack/keystone: Unimplemented get roles by group for project list https://review.openstack.org/76470 | 07:44 |
| *** gokrokve has quit IRC | 07:47 | |
| *** esmute has quit IRC | 07:52 | |
| *** marcoemorais1 has quit IRC | 07:55 | |
| *** xianghui_afk has quit IRC | 07:57 | |
| *** esmute has joined #openstack-keystone | 07:58 | |
| *** stevemar has quit IRC | 08:01 | |
| *** xianghui_afk has joined #openstack-keystone | 08:09 | |
| *** bvandenh has joined #openstack-keystone | 08:16 | |
| *** henrynash has joined #openstack-keystone | 08:24 | |
| *** andreaf has joined #openstack-keystone | 08:25 | |
| *** rwsu has quit IRC | 08:26 | |
| *** mberlin1 has quit IRC | 08:30 | |
| *** mberlin has joined #openstack-keystone | 08:33 | |
| *** gokrokve has joined #openstack-keystone | 08:42 | |
| *** gokrokve has quit IRC | 08:47 | |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: oslo.db implementation https://review.openstack.org/77210 | 08:56 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations. https://review.openstack.org/80630 | 08:56 |
| *** derek_c has quit IRC | 09:14 | |
| *** xianghui_afk has quit IRC | 09:24 | |
| *** xianghui_afk has joined #openstack-keystone | 09:38 | |
| *** gokrokve has joined #openstack-keystone | 09:42 | |
| *** gokrokve has quit IRC | 09:47 | |
| *** hipster has joined #openstack-keystone | 09:47 | |
| *** hipster has quit IRC | 09:51 | |
| *** rodrigods has joined #openstack-keystone | 09:59 | |
| *** rodrigods has joined #openstack-keystone | 09:59 | |
| openstackgerrit | Kristy Siu proposed a change to openstack/identity-api: Trusted Attributes Policy for External Identity Providers (Federation pt 4) https://review.openstack.org/60489 | 10:12 |
| *** xianghui_afk has quit IRC | 10:17 | |
| *** rodrigods has quit IRC | 10:21 | |
| *** xianghui_afk has joined #openstack-keystone | 10:30 | |
| openstackgerrit | Kristy Siu proposed a change to openstack/identity-api: Trusted Attributes Policy for External Identity Providers https://review.openstack.org/60489 | 10:35 |
| *** afazekas has joined #openstack-keystone | 10:39 | |
| *** rodrigods has joined #openstack-keystone | 10:40 | |
| *** gokrokve has joined #openstack-keystone | 10:42 | |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: oslo.db implementation https://review.openstack.org/77210 | 10:45 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations. https://review.openstack.org/80630 | 10:45 |
| *** gokrokve has quit IRC | 10:47 | |
| *** hipster has joined #openstack-keystone | 10:47 | |
| *** rodrigods has quit IRC | 10:49 | |
| *** hipster has quit IRC | 10:52 | |
| *** xianghui_afk has quit IRC | 10:59 | |
| *** bvandenh has quit IRC | 11:02 | |
| *** diegows has joined #openstack-keystone | 11:17 | |
| *** henrynash has quit IRC | 11:34 | |
| *** henrynash has joined #openstack-keystone | 11:37 | |
| *** gokrokve has joined #openstack-keystone | 11:42 | |
| *** henrynash has joined #openstack-keystone | 11:43 | |
| *** gokrokve has quit IRC | 11:47 | |
| *** lbragstad has quit IRC | 11:47 | |
| openstackgerrit | Rodrigo Duarte Sousa proposed a change to openstack/python-keystoneclient: Add /role_assignments endpoint support https://review.openstack.org/91578 | 12:00 |
| *** afazekas has quit IRC | 12:05 | |
| *** rodrigods has joined #openstack-keystone | 12:08 | |
| *** raildo has joined #openstack-keystone | 12:13 | |
| *** dhellmann_ is now known as dhellmann | 12:16 | |
| *** raildo has quit IRC | 12:17 | |
| *** afazekas has joined #openstack-keystone | 12:20 | |
| *** JuanManuelOlle has joined #openstack-keystone | 12:40 | |
| *** dstanek_zzz is now known as dstanek | 12:41 | |
| *** gokrokve has joined #openstack-keystone | 12:42 | |
| *** gokrokve has quit IRC | 12:47 | |
| *** lbragstad has joined #openstack-keystone | 12:50 | |
| *** xianghui has joined #openstack-keystone | 13:06 | |
| *** Ju has quit IRC | 13:09 | |
| *** dstanek is now known as dstanekafk | 13:09 | |
| *** dstanekafk is now known as dstanekafk_zzz | 13:11 | |
| *** afazekas has quit IRC | 13:12 | |
| *** bknudson has quit IRC | 13:17 | |
| *** joesavak has joined #openstack-keystone | 13:17 | |
| *** nkinder has quit IRC | 13:20 | |
| *** dstanekafk_zzz is now known as dstanek | 13:23 | |
| *** afazekas has joined #openstack-keystone | 13:25 | |
| *** dhellmann is now known as dhellmann_ | 13:27 | |
| *** hipster has joined #openstack-keystone | 13:29 | |
| *** hipster has quit IRC | 13:32 | |
| *** vhoward has left #openstack-keystone | 13:32 | |
| *** bknudson has joined #openstack-keystone | 13:41 | |
| *** jaosorior has quit IRC | 13:41 | |
| *** gokrokve has joined #openstack-keystone | 13:42 | |
| *** andreaf has quit IRC | 13:45 | |
| *** tellesnobrega has joined #openstack-keystone | 13:46 | |
| *** gokrokve has quit IRC | 13:46 | |
| *** erecio_1 has joined #openstack-keystone | 13:51 | |
| *** erecio has quit IRC | 13:54 | |
| openstackgerrit | Christian Berendt proposed a change to openstack/keystone: fixed several pep8 issues https://review.openstack.org/93686 | 13:56 |
| *** ayoung has joined #openstack-keystone | 14:01 | |
| *** nkinder has joined #openstack-keystone | 14:05 | |
| *** stevemar has joined #openstack-keystone | 14:06 | |
| *** erecio_2 has joined #openstack-keystone | 14:10 | |
| *** jamielennox|away has quit IRC | 14:11 | |
| ayoung | Cutting the Gordian Knot. Both "Domains" and "Projects" are instances of "Tenants" tada! | 14:12 |
| *** erecio_1 has quit IRC | 14:14 | |
| *** bvandenh has joined #openstack-keystone | 14:14 | |
| dstanek | domains -> tenants -> projects as a nice little hierarchy :-) | 14:16 |
| *** jamielennox|away has joined #openstack-keystone | 14:18 | |
| ayoung | dstanek, nah, domains are the primary tenants | 14:27 |
| ayoung | projects are the nested tenants | 14:27 |
| *** dims has joined #openstack-keystone | 14:28 | |
| dstanek | what was the usecase for domains back in the day? | 14:28 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: oslo.db implementation https://review.openstack.org/77210 | 14:29 |
| openstackgerrit | Ilya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations. https://review.openstack.org/80630 | 14:29 |
| ayoung | dstanek, https://wiki.openstack.org/wiki/Domains | 14:37 |
| *** hipster has joined #openstack-keystone | 14:39 | |
| *** david-lyle has joined #openstack-keystone | 14:40 | |
| *** gokrokve has joined #openstack-keystone | 14:42 | |
| *** gokrokve has quit IRC | 14:46 | |
| *** xianghui has quit IRC | 14:47 | |
| *** ukalifon has joined #openstack-keystone | 15:08 | |
| *** gordc has joined #openstack-keystone | 15:09 | |
| openstackgerrit | Diane Fleming proposed a change to openstack/identity-api: Clean up files for identity v2.0 reference https://review.openstack.org/94194 | 15:13 |
| *** browne has joined #openstack-keystone | 15:15 | |
| ayoung | ukalifon, OK, so there should be two files, one for Shibboleth, and one for the Configuration for Keystone....and bother stevemar if you need help reproducing... | 15:15 |
| *** leseb has quit IRC | 15:16 | |
| *** leseb has joined #openstack-keystone | 15:17 | |
| openstackgerrit | Brant Knudson proposed a change to openstack/keystone: SQL and LDAP fixes for get_roles_for_user_and_project user=group ID https://review.openstack.org/94396 | 15:19 |
| bknudson | this is for the security vuln | 15:20 |
| bknudson | https://review.openstack.org/#/c/94397 is for stable/icehouse | 15:22 |
| *** gokrokve has joined #openstack-keystone | 15:24 | |
| *** browne has quit IRC | 15:24 | |
| *** browne has joined #openstack-keystone | 15:25 | |
| *** daneyon has joined #openstack-keystone | 15:26 | |
| *** andreaf has joined #openstack-keystone | 15:30 | |
| openstackgerrit | Diane Fleming proposed a change to openstack/identity-api: Clean up files for identity v2.0 reference https://review.openstack.org/94194 | 15:30 |
| *** ukalifon has quit IRC | 15:33 | |
| *** afazekas has quit IRC | 15:40 | |
| *** praneshp_ has joined #openstack-keystone | 15:42 | |
| nkinder | lbragstad: I'm looking into your comment about CADF for https://review.openstack.org/#/c/93975/ | 15:49 |
| nkinder | lbragstad: it doesn't look like we actually use CADF anywhere yet (unless I'm missing something) | 15:49 |
| lbragstad | nkinder: yeah, I noticied that too | 15:50 |
| lbragstad | I added topol to the review | 15:50 |
| lbragstad | because I though I'd heard him say that was done when we authenticated. | 15:50 |
| lbragstad | but I couldn't find it | 15:50 |
| nkinder | lbragstad: ok, maybe we should split out adding CADF notifications for successful and failed auth | 15:51 |
| nkinder | lbragstad: even with CADF, we should still log successful issuance, right? | 15:51 |
| lbragstad | nkinder: so we will be sending a cadf notification in both those cases? | 15:51 |
| lbragstad | nkinder: that's what I was unsure about... do we log authentication (possibly sensitive) information in that case? | 15:51 |
| nkinder | lbragstad: that would need to be decided, but from a pure auditing perpective, you would want to know about failed and successful auth | 15:52 |
| nkinder | lbragstad: it just logs the user id | 15:52 |
| lbragstad | nkinder: ++ I agree | 15:52 |
| lbragstad | ok | 15:52 |
| lbragstad | if it's just the user ID it's probably not so much of a big deal | 15:52 |
| *** praneshp_ is now known as praneshp | 15:52 | |
| nkinder | lbragstad: yeah. I'll re-trigger the jenkins jobs | 15:53 |
| lbragstad | nkinder: does 'recheck no bug' still work? | 15:53 |
| lbragstad | i thought that was removed | 15:54 |
| nkinder | lbragstad: oh, I didn't hear about that | 15:54 |
| * lbragstad could be wrong | 15:54 | |
| *** gyee has joined #openstack-keystone | 15:54 | |
| lbragstad | nkinder: I guess reverify no bug is gone | 15:57 |
| nkinder | lbragstad: I think you're right. It didn't retrigger. | 15:57 |
| lbragstad | nkinder: yeah i didn't see it come through the queue | 15:57 |
| lbragstad | I think it was causing issues | 15:57 |
| *** amcrn has joined #openstack-keystone | 16:00 | |
| *** marcoemorais has joined #openstack-keystone | 16:01 | |
| lbragstad | nkinder: so, here https://review.openstack.org/#/c/93975/2/keystone/token/controllers.py do you think we should add a comment inline, just elaborating on the idea of extending the authentication (failure or success) to a cadf implementation? | 16:03 |
| *** bvandenh has quit IRC | 16:05 | |
| *** BAKfr has quit IRC | 16:07 | |
| *** hipster_ has joined #openstack-keystone | 16:07 | |
| *** cds has joined #openstack-keystone | 16:08 | |
| *** hipster has quit IRC | 16:11 | |
| *** sbfox has joined #openstack-keystone | 16:15 | |
| *** sbfox has quit IRC | 16:16 | |
| *** sbfox has joined #openstack-keystone | 16:17 | |
| *** richm has joined #openstack-keystone | 16:18 | |
| openstackgerrit | ayoung proposed a change to openstack/keystone: multi-backend support for identity https://review.openstack.org/74214 | 16:21 |
| *** tristanC has joined #openstack-keystone | 16:23 | |
| nkinder | lbragstad: yeah, a TODO comment is a good idea | 16:24 |
| tristanC | Hello folks! I'm waiting for both 94396 and 94397 to be approved before sending the associated OSSA (those are fixes for the now public bug #1309228) | 16:24 |
| uvirtbot | Launchpad bug 1309228 in keystone "User gets group auth if same id (CVE-2014-0204)" [High,In progress] https://launchpad.net/bugs/1309228 | 16:24 |
| lbragstad | nkinder: sounds good, I left my comment | 16:30 |
| *** rwsu has joined #openstack-keystone | 16:30 | |
| tristanC | Both review received different comments, and as they are the same patch basically, could you update (if required) the master branch please (cc bknudson, dolphm) | 16:30 |
| *** morganfainberg_Z is now known as morganfainberg | 16:30 | |
| morganfainberg | mornin | 16:30 |
| openstackgerrit | Diane Fleming proposed a change to openstack/identity-api: Clean up files for identity v2.0 reference https://review.openstack.org/94194 | 16:31 |
| bknudson | tristanC: I'll look through the comments and make updates. | 16:31 |
| bknudson | seems like we could merge what's there as long as there isn't a concern about opening a new security vuln | 16:31 |
| tristanC | bknudson: sadly the stable/icehouse got a couple of check-grenade-dsvm errors (tempest.api.identity.admin.v3.test_certificates.CertificatesV3TestJSON.test_get_ca_certificate NotFound) | 16:33 |
| nkinder | tristanC, bknudson: I just added a comment to 94396 | 16:34 |
| tristanC | nkinder: thanks! | 16:34 |
| bknudson | tristanC: I think there's something that's causing grenade to fail on stable/icehouse... | 16:34 |
| *** openstackgerrit has quit IRC | 16:35 | |
| *** openstackgerrit has joined #openstack-keystone | 16:36 | |
| *** sbfox has quit IRC | 16:39 | |
| *** raildo1 has joined #openstack-keystone | 16:42 | |
| bknudson | nkinder: what do you think about me stealing _is_dns_equal(first_dn, second_dn) and putting it into https://review.openstack.org/#/c/94396/ | 16:48 |
| bknudson | ? | 16:49 |
| nkinder | bknudson: I'm OK with that. The main purpose of the other patch was to correct the dumb user check. | 16:50 |
| bknudson | I'll put Sergey Nikitin down as co-author | 16:51 |
| nkinder | bknudson: I think I have a concern about the function though... | 16:51 |
| openstackgerrit | guang-yee proposed a change to openstack/keystone: Make sure all the auth plugins agree on the shared identity attributes. https://review.openstack.org/84945 | 16:51 |
| bknudson | looks like it doesn't handle multi-valued rdns | 16:52 |
| nkinder | bknudson: the _is_dns_equal() method is only doing case-insensitive comparison of attribute names, not values | 16:52 |
| bknudson | nkinder: I think that's what we want | 16:53 |
| bknudson | since we have no idea what the schema says for comparing attrs anyways | 16:53 |
| nkinder | bknudson: as I explained in my review comments, most (not all) values are compared in a case-insensitive way in LDAP | 16:53 |
| nkinder | bknudson: it will be wrong sometimes, but I suppose it's better to say that equivalent DNs are not equal than to say two different DNs are equal | 16:54 |
| openstackgerrit | Nathan Kinder proposed a change to openstack/keystone: Adds log message upon token granting https://review.openstack.org/93975 | 16:57 |
| *** sbfox has joined #openstack-keystone | 16:58 | |
| *** leseb has quit IRC | 16:58 | |
| gyee | bknudson, answered your questions there https://review.openstack.org/84945 | 17:00 |
| *** amcrn has quit IRC | 17:04 | |
| *** radez_g0n3 is now known as radez | 17:05 | |
| morganfainberg | nkinder, is that meant to be a cadf event? | 17:09 |
| morganfainberg | nkinder, or is this strictly a "oh hay, token for <X>" | 17:09 |
| morganfainberg | nkinder, also, is INFO the right level? | 17:09 |
| nkinder | morganfainberg: yeah, must have mixed up terminology since it's in notification.py | 17:09 |
| morganfainberg | nkinder, ah. so you want it to be an auditable event not a log | 17:10 |
| morganfainberg | ? | 17:10 |
| nkinder | morganfainberg: yes, the idea is that it us needed for auditing purposes. | 17:11 |
| morganfainberg | ok, let me take a look at the notifications, i'll comment where that should hook in. so it will be CADF instead of log | 17:11 |
| nkinder | morganfainberg: CADF might be more appropriate, but this is something that we need to backport on our end too. | 17:11 |
| morganfainberg | to H? | 17:12 |
| nkinder | morganfainberg: yep | 17:12 |
| morganfainberg | hm. maybe we do CADF for J, and backport to H would be logging? i'd rather not clutter the logs w/ token data in J (at INFO level) with token issuance info | 17:13 |
| morganfainberg | nkinder, since we have CADF in I and J | 17:13 |
| morganfainberg | huh.. wait a minute... | 17:14 |
| *** harlowja_away is now known as harlowja | 17:16 | |
| morganfainberg | nkinder, we should be emitting CADF notifications on both success and failure to authenticate. | 17:18 |
| nkinder | morganfainberg: agreed | 17:18 |
| morganfainberg | nkinder, no i mean, the code should already be in place for it | 17:18 |
| nkinder | morganfainberg: where is that done? I didn't see it when I looked. | 17:18 |
| morganfainberg | nkinder, it's from the emit_event decorator on identity_api.authenticate | 17:21 |
| morganfainberg | nkinder, calls into the CADF wrapper http://git.openstack.org/cgit/openstack/keystone/tree/keystone/notifications.py#n228 | 17:21 |
| morganfainberg | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/identity/core.py#n272 | 17:22 |
| nkinder | morganfainberg: ah, I missed the decorator | 17:22 |
| morganfainberg | nkinder, yeah i had to 2x look :P felt like this was duplicated code. | 17:23 |
| morganfainberg | nkinder, no worries :) | 17:23 |
| ayoung | morganfainberg, that was my guess: we want CADF for this, so the stable solution would look different than the Juno solution | 17:23 |
| morganfainberg | ayoung, the stable/havana solution, icehouse already has this iirc | 17:23 |
| ayoung | "log upon successful login?" | 17:23 |
| ayoung | yeah, stable/havana =-= RHOS 4.0 where we need it | 17:24 |
| morganfainberg | ayoung, CADF for both success/failure | 17:24 |
| morganfainberg | ayoung, stable havana would need t the logging big in this patch | 17:24 |
| ayoung | ++ | 17:24 |
| morganfainberg | ayoung, i don't want to accept this to master though... we have a better system. is it possible to do logging in stable/havana and not accept the code we already solved another way to master? | 17:25 |
| *** amcrn has joined #openstack-keystone | 17:25 | |
| ayoung | morganfainberg, yes. I think that is acceptable. We need to at least submit a patch upstream before we carry it in RDO, but it does not have to be accepted. In this case, the patch is minimal enough that it should not cause much angst | 17:26 |
| nkinder | morganfainberg: I agree on not accepting it for master. CADF is the way. | 17:26 |
| morganfainberg | nkinder, ayoung, ok i'm going to block this one because we have CADF for master then since it is proposed to master, commenting that if stable maintenance will accept it it should go direct to havana | 17:27 |
| nkinder | morganfainberg: +1 | 17:27 |
| morganfainberg | nkinder, ayoung, does that meet your specific needs as well for backporting in RDO? | 17:27 |
| ayoung | morganfainberg, ++ | 17:27 |
| morganfainberg | :) | 17:27 |
| ayoung | I'll back you on that | 17:27 |
| *** sbfox has quit IRC | 17:27 | |
| morganfainberg | ayoung, sounds good. | 17:28 |
| morganfainberg | let me confirm icehouse (i'm fairly certain we have CADF there) | 17:28 |
| ayoung | morganfainberg, if we want to be pedantic, we could: accept to master, backport, remove from master, get beaten by ttx. | 17:28 |
| nkinder | ayoung: ewww :) | 17:29 |
| morganfainberg | ayoung, if it comes to that i'll unblock...but lets not do that .... makes me feel like i need a shower | 17:29 |
| ayoung | heh | 17:29 |
| ayoung | morganfainberg, I think cadf is going to be a killer feature... | 17:29 |
| ayoung | it should allow someone to "unfsck" a Keystone server without having the keys to the kingdom | 17:30 |
| ayoung | I'm thinking of the issues people have had debugging LDAP configs. With Federation and mapping, its going to be worse | 17:30 |
| raildo1 | +1 | 17:30 |
| ayoung | With CADF, we could actually have an API for fetching the CADF events for a given scope..... | 17:31 |
| ayoung | so if you have the "auditor" role on a domain you could see all login attempts for that domain. | 17:31 |
| ayoung | No passwords, but why something failed, like "no role on project" or "bad password" | 17:32 |
| morganfainberg | ayoung, ok https://review.openstack.org/#/c/93975/ put a -2 on that with a comment this is really only needed for stable/havana. if we need to unblock for whatever reason, i'm happy to do that. | 17:33 |
| ayoung | I put a "0 I concur" in there | 17:33 |
| *** andreaf has quit IRC | 17:51 | |
| *** gokrokve has quit IRC | 17:52 | |
| openstackgerrit | A change was merged to openstack/python-keystoneclient: Fixes an erroneous type check in a test https://review.openstack.org/94256 | 17:53 |
| *** gokrokve has joined #openstack-keystone | 17:54 | |
| *** afaranha has quit IRC | 17:54 | |
| browne | so is CADF secure? i ask because ceilometer previously (havana) had no policy enforcement. so in effect any user could retrieve all audit events from the ceilometer meter. | 17:55 |
| morganfainberg | browne, CADF is a notification, ceilometer's lack of security isn't really a factor in that case (you could use another mechanism to store the audit data) | 17:59 |
| browne | morganfainberg: well in my case i had used a db to store the audit records, which i think is how cadf is typically used. and then the metering api allows retrieval of those records using any user (not just admins which is what i wanted) | 18:01 |
| morganfainberg | browne, sure. that is reasonable. but the ability to remove things isn't a fault of CADF (or even in this case keystone). | 18:02 |
| *** jamielennox|away is now known as jamielennox | 18:03 | |
| *** packet has joined #openstack-keystone | 18:04 | |
| browne | morganfainberg: sure, i understand its not CADF or keystone at fault. my point was it was unsafe to use with ceilometer | 18:04 |
| *** hipster_ has quit IRC | 18:06 | |
| bknudson | lately when I try to start slapd it fails -- *** glibc detected *** /usr/sbin/slapd: double free or corruption (top): 0x00007f8dd34c3df0 *** | 18:08 |
| bknudson | this is on ubuntu 12.04 (devstack) | 18:08 |
| *** leseb has joined #openstack-keystone | 18:15 | |
| openstackgerrit | A change was merged to openstack/identity-api: Clean up files for identity v2.0 reference https://review.openstack.org/94194 | 18:17 |
| *** dims has quit IRC | 18:17 | |
| tristanC | bknudson: untilthis is reported/resolvedyou can "export MALLOC_CHECK_=0" and it will not crash... | 18:17 |
| bknudson | tristanC: I'll give it a try! | 18:18 |
| openstackgerrit | Andreas Jaeger proposed a change to openstack/identity-api: Update to clouddocs-maven-plugin 2.0.2 https://review.openstack.org/94441 | 18:21 |
| tristanC | well I meant it *might* not crash :) Or it will with a traditional segfault... | 18:22 |
| bknudson | ah, it was my own issue with trying to test ssl | 18:26 |
| *** bvandenh has joined #openstack-keystone | 18:34 | |
| *** JuanManuelOlle has left #openstack-keystone | 18:38 | |
| *** sbfox has joined #openstack-keystone | 18:42 | |
| *** dims has joined #openstack-keystone | 18:43 | |
| *** sbfox1 has joined #openstack-keystone | 18:43 | |
| *** gokrokve has quit IRC | 18:45 | |
| *** sbfox has quit IRC | 18:46 | |
| *** arunkant has joined #openstack-keystone | 18:51 | |
| *** amcrn has quit IRC | 19:00 | |
| gyee | so what's the decision on ldappool, go or no-go? | 19:01 |
| bknudson | gyee: sounds like you've been running with it? | 19:01 |
| dolphm | gyee: evaluate it | 19:01 |
| bknudson | gyee: how big is the change? | 19:01 |
| gyee | bknudson, yes | 19:01 |
| bknudson | gyee: I assume some config options | 19:02 |
| lbragstad | gyee: can you push up something as a WIP? | 19:02 |
| dolphm | gyee: does "yes" mean "big"? | 19:02 |
| gyee | bknudson, pretty trivial change, we just need to wrap the existing connection | 19:02 |
| gyee | not a big change | 19:02 |
| gyee | I can push a WIP for you guys to see | 19:02 |
| bknudson | if we can get it posted I'd rather it was in J1 | 19:02 |
| gyee | right now its not configurable, but I can make it optional | 19:03 |
| *** dims has quit IRC | 19:03 | |
| bknudson | ConnectionManager has all sort of options -- size, retry_max, etc. | 19:03 |
| henrynash | dolphm: the mapping table for domain-specif ldap…do you want to do a spec for that, or are we OK with the BP (since we carried this over from IceHouse) | 19:03 |
| bknudson | I assume we need all that configurable | 19:03 |
| bknudson | [ldap_pool] | 19:03 |
| tristanC | May I ask the status for #1309228 ? OSSA is overdue... | 19:04 |
| gyee | bknudson, yes, I can have all that in a configurable section | 19:04 |
| dolphm | henrynash: depends on when you think it'll land | 19:05 |
| dolphm | henrynash: if you can target j1 and feel everyone understands the problem space & proposed solution, skip the -spec | 19:06 |
| henrynash | dolphm: ok, got it…it’s my intention to land it in J1 | 19:06 |
| *** erecio_1 has joined #openstack-keystone | 19:07 | |
| ayoung | nkinder, I went ahead and published http://adam.younglogic.com/2014/05/kerberizing-keystone-in-httpd/ | 19:08 |
| dolphm | ayoung: bknudson: lbragstad: this needs to land ASAP, skip the nits https://review.openstack.org/#/c/94396/ | 19:08 |
| ayoung | dolphm, I was looking at that. Specifically the LDAP side | 19:08 |
| ayoung | there was a better way to do the DN thing, but it can land after this | 19:09 |
| ayoung | probably better for a backport if it does. I'm OK with it if nkinder is | 19:09 |
| dolphm | ayoung: ack, and the better way wouldn't be backportable to icehouse anyway | 19:09 |
| bknudson | I'm not too happy with the dn comparison code either | 19:09 |
| bknudson | but if it's not going to cause another vuln we can fix it later | 19:09 |
| *** erecio_2 has quit IRC | 19:10 | |
| nkinder | bknudson: I won't truly be happy with the comparison code until it's truly DN syntax aware, but that's no reason to block this | 19:10 |
| ayoung | bknudson, did you test against liveLDAP? | 19:11 |
| dstanek | yay! i got all of the tests to work with gevent instead of eventlet! | 19:11 |
| bknudson | ayoung: give me a minute and I'll make sure to try it again. | 19:12 |
| ayoung | dstanek, I'm not sure it that makes me want to laugh or cry | 19:12 |
| ayoung | I think cry | 19:12 |
| dstanek | ayoung: why is that? | 19:12 |
| ayoung | replacing one eventloop for another | 19:13 |
| nkinder | ayoung: question on your blog post... | 19:13 |
| ayoung | nkinder, fire away | 19:13 |
| nkinder | ayoung: it's using kerberos to auth, but Keystone then does lookups via LDAP (as anonymous?) | 19:13 |
| dstanek | ayoung: it's all for the scaling baby! | 19:13 |
| *** bvandenh has quit IRC | 19:13 | |
| ayoung | nkinder, yes | 19:13 |
| nkinder | ayoung: so it's using REMOTE_USER, then doing a lookup for that user to get groups and figure out roles and all that | 19:14 |
| ayoung | yes | 19:14 |
| dolphm | dstanek: does it actually handle more than one request at a time?? | 19:14 |
| lbragstad | dolphm: ayoung bknudson nkinder I'm good with https://review.openstack.org/#/c/94396/1 I think those changes can be made later | 19:14 |
| dstanek | dolphm: yes, in my test setup i run (# of cpus +1) processes and each one is asyncio | 19:15 |
| nkinder | ayoung: ok, so the next piece would be to use mod_lookup_identity to bypass the LDAP driver | 19:15 |
| dolphm | dstanek: what made you try gevent? and what did it take? | 19:16 |
| openstackgerrit | A change was merged to openstack/identity-api: Update to clouddocs-maven-plugin 2.0.2 https://review.openstack.org/94441 | 19:16 |
| dstanek | i've always used gevent with flask so i'm familiar with it - it is also making py3 progress | 19:16 |
| dstanek | dolphm: not too much work - totals sunk time to far is about 3 hours | 19:17 |
| ayoung | nkinder, yes. And the setup I wrote about was based on this one...so it used the LDAP backend to enumerate groups only | 19:17 |
| dstanek | dolphm: i'm going to see if hugh can help me with rally this week after i attempt using it on my own | 19:18 |
| ayoung | bknudson, nkinder what if the site mixes users and groups into the same subtree? Will if not assignment_dn_norm.endswith(user_tree_dn_norm): actually work? | 19:20 |
| ayoung | and is that something that people do? | 19:21 |
| ayoung | shouldn't we really check the object class? | 19:21 |
| nkinder | ayoung: an objectclass check is the right way of doing things | 19:21 |
| ayoung | nkinder, and I think we have that information at this point, so the check should be based on that, not DN, right? | 19:22 |
| ayoung | dolphm, lets split that patch, and get the SQL portion in. I think the LDAP portion needs to be rewritten | 19:23 |
| nkinder | ayoung: I don't know that we do have the objectclass at that point | 19:23 |
| ayoung | nkinder, we do. its in the API objects | 19:24 |
| nkinder | ayoung: how do we know what objectclass to look for? | 19:24 |
| *** amcrn has joined #openstack-keystone | 19:25 | |
| ayoung | http://git.openstack.org/cgit/openstack/keystone/tree/keystone/common/ldap/core.py#n703 | 19:25 |
| *** andreaf has joined #openstack-keystone | 19:25 | |
| ayoung | nkinder, it should be comparable to that check | 19:25 |
| nkinder | ayoung: ah, user_objectclass from config | 19:25 |
| ayoung | and group_ yep | 19:26 |
| nkinder | ayoung: I just -1'd it and recommended an objectclass check | 19:28 |
| nkinder | if sql is what needs to be fixed now, let's just split it | 19:28 |
| ayoung | nkinder, hmmm...his does it without an LDAP fetch | 19:29 |
| *** gokrokve has joined #openstack-keystone | 19:29 | |
| ayoung | nkinder, if we do this blindly, it would be one fetch per assignment. But maybe that is OK to start | 19:30 |
| ayoung | seems like we should be able to do something like an ldap search filtered by (objectClass=CONF.user_object_class) | 19:31 |
| nkinder | ayoung: it seems like a bit of a mess. We're looping through the assignments and trying to determine if the id is a user or a group. | 19:33 |
| *** andreaf has quit IRC | 19:33 | |
| *** andreaf has joined #openstack-keystone | 19:33 | |
| *** hipster has joined #openstack-keystone | 19:33 | |
| *** hipster has quit IRC | 19:33 | |
| *** hipster has joined #openstack-keystone | 19:34 | |
| bknudson | the only way you'd know the objectclass is to fetch all the member entries | 19:34 |
| nkinder | ayoung: if the same id is used for both a user and a group, how can we tell which it applies to? | 19:34 |
| nkinder | I don't think that there is any way to reliably tell which it refers to | 19:35 |
| bknudson | it does seem like the logic to determine if it's a user or group should be pushed down to get_role_assignments | 19:35 |
| *** gyee has quit IRC | 19:36 | |
| *** andreaf_ has joined #openstack-keystone | 19:38 | |
| *** andreaf has quit IRC | 19:39 | |
| *** daneyon has quit IRC | 19:42 | |
| *** daneyon has joined #openstack-keystone | 19:43 | |
| *** daneyon has quit IRC | 19:44 | |
| *** daneyon has joined #openstack-keystone | 19:44 | |
| *** daneyon has quit IRC | 19:48 | |
| bknudson | ayoung: tried it with openldap and posted some info http://paste.openstack.org/show/81018/ | 19:49 |
| ayoung | bknudson, so...I think it is the wrong approach | 19:50 |
| ayoung | bknudson, it really needs to be an objectclass check. And we might be able to streamline that. | 19:50 |
| *** marcoemorais has quit IRC | 19:50 | |
| ayoung | somethinkg like a filter where objectclass=user object class and DN = is in a set of DNs. | 19:51 |
| ayoung | LDAP assignments has been a mess for a while. And its really only used by CERN | 19:52 |
| ayoung | well, as far as we know...they arethe primary consumers anyway | 19:52 |
| bknudson | ayoung: what's the right approach? | 19:52 |
| ayoung | bknudson, am thinking....my LDAP-fu is weak | 19:53 |
| ayoung | OK, the first query gives us the set of assignments, with a bunch of member_of values. Ideally we would do a bulk fetch, filterd by object class. Let me try it out... | 19:54 |
| *** erecio_2 has joined #openstack-keystone | 19:54 | |
| bknudson | it's easy enough to split out the SQL change if we can get that in. | 19:55 |
| *** erecio_1 has quit IRC | 19:56 | |
| ayoung | ldapsearch -Y gssapi "(objectClass=posixgroup)" gets us the object class. It would be an "anded query" | 19:57 |
| *** marcoemorais has joined #openstack-keystone | 19:59 | |
| bknudson | by default the attribute is roleOccupant -- so if there was a roleOccupant-of attribute we could search for (&(roleOccupantOf=cn=7d4c172613e446c1a85306f4e993c2ed,cn=0ca027daac884bfcbf82ed15b63b7c3c,ou=Projects,dc=openstack,dc=org)(objectclass=inetorgperson)) | 20:03 |
| bknudson | or (objectclass=groupOfNames) for groups | 20:03 |
| *** erecio_1 has joined #openstack-keystone | 20:04 | |
| nkinder | ayoung: is assignment in LDAP done via DN, or via 'id'? | 20:05 |
| bknudson | we expect to get DNs back | 20:05 |
| bknudson | roleOccupant: cn=73114eaf976e4f6480776c3d49213289,ou=Users,dc=openstack,dc=org | 20:05 |
| bknudson | roleOccupant: cn=suspectid,ou=UserGroups,dc=openstack,dc=org | 20:05 |
| nkinder | bknudson: what if I add an assignment via Keystone? | 20:05 |
| ayoung | bknudson, yes, that is what I am thinking | 20:06 |
| nkinder | bknudson: do I just supply the 'id'? | 20:06 |
| bknudson | nkinder: that's how keystone does it. | 20:06 |
| *** erecio_2 has quit IRC | 20:06 | |
| bknudson | nkinder: when you create a role assignment it's by ID | 20:06 |
| nkinder | bknudson: how does it know if I'm referring to a user or a group if the same ID is used for both? | 20:07 |
| ayoung | bknudson, so we "or" together all of the roleOccupant values then and by objectClass | 20:07 |
| bknudson | the code to convert a DN to an ID is by just taking the first value. | 20:07 |
| ayoung | ARGH | 20:07 |
| bknudson | nkinder: that's the problem! | 20:07 |
| ayoung | so there is no way.... | 20:07 |
| nkinder | yes, I think it's fundamentally broken | 20:07 |
| ayoung | we didn't have groups originally | 20:07 |
| nkinder | id must be unique across users and groups | 20:07 |
| *** dims has joined #openstack-keystone | 20:07 | |
| bknudson | nkinder: from the example, you can see that they're in different subtrees | 20:08 |
| ayoung | it won't be for LDAP | 20:08 |
| ayoung | Need josecastro here.... | 20:08 |
| bknudson | but of course one might lodge their groups under their users if they felt like it | 20:08 |
| *** amcrn has quit IRC | 20:08 | |
| ayoung | bknudson, split out the SQL patch from the LDAP one and submit it separately. I think LDAP is going to take some more time. But I am guessing that the SQL side is not really a problem, as IDs are UUIDs | 20:09 |
| nkinder | bknudson: they don't have to be in different subtrees through from an LDAP perspective | 20:09 |
| bknudson | ayoung: the sql side was easy because it just added the assignment type to the query | 20:09 |
| nkinder | bknudson: it is likely a separate tree in most cases though | 20:09 |
| ayoung | we need to stick the whole DN in the memberOf attribute | 20:10 |
| bknudson | it is kind of ridiculous to use keystone for assignments considering how you need to set up your schema. | 20:12 |
| *** marcoemorais has quit IRC | 20:12 | |
| *** marcoemorais has joined #openstack-keystone | 20:12 | |
| bknudson | ayoung: have you looked at the sql change? you'd +2 if I split it out? | 20:12 |
| ayoung | bknudson, yes | 20:14 |
| bknudson | ayoung: ok, I'll split it out. | 20:14 |
| ayoung | bknudson, CERN uses it due to replication and scale. | 20:14 |
| *** ctracey has joined #openstack-keystone | 20:15 | |
| ctracey | quick question - wondering if other have run into this. Seems keystone hangs during a password update. This is at least long enough for our front facing LB's to issue a 504 to the client. So we are talking on the order of many seconds. Anyone else? This is Havana with mysql token backend. | 20:16 |
| ayoung | wait a seco, the UserROleAssociation object does hold a DN, not an ID | 20:17 |
| ctracey | the password gets reset properly, but it looks as if the token used to auth the password update gets destroyed and then everything heads south | 20:18 |
| ayoung | I knew we weren't that dumb....hold on. bknudson lets see if we can get it with a single query | 20:18 |
| ctracey | i looked for any outstanding and/or closed bugs but didnt turn anything up...was wondering if someone recollected similar behavior | 20:18 |
| bknudson | ayoung: get what with a single query? | 20:20 |
| ayoung | bknudson, the problem is the call assignments = self.role.get_role_assignments(proj_dn) | 20:21 |
| ayoung | that should be different depending on what we want | 20:21 |
| bknudson | ayoung: it returns all the DNs, be they users or groups | 20:21 |
| openstackgerrit | Steve Martinelli proposed a change to openstack/keystone: Add openID Connect auth plugin for federation https://review.openstack.org/61662 | 20:21 |
| ayoung | doesn't have to | 20:22 |
| bknudson | ayoung: how does it know if the DN is for a user or a group? | 20:22 |
| richm | I don't think all LDAP servers support it, but you could use the dereference control to get the objectclasses of all DNs returned by the search | 20:24 |
| ayoung | nkinder, does LDAP have the concept of a join? | 20:24 |
| richm | 389 and openldap support, but I suppose AD is the gating factor | 20:25 |
| ayoung | or a nested query: | 20:25 |
| richm | no, you want deref | 20:25 |
| richm | AD supports ASQ, which is very similar | 20:25 |
| ayoung | richm, OK, so check for the presence of the deref control, and if it exists, use it, and if it does not, do it the slow way? | 20:26 |
| richm | yes | 20:26 |
| richm | if AD, then use ASQ | 20:26 |
| ayoung | richm, can you filter the results so that only DNs with objectclasses that match are returned? | 20:26 |
| bknudson | that's kind of neat | 20:27 |
| richm | not with deref | 20:27 |
| richm | not sure about ASQ | 20:27 |
| ayoung | probably just a cheap to do 2 LDAP queries as to use the control. | 20:28 |
| * ayoung still thinks we can do this with one query | 20:29 | |
| bknudson | it's one query per DN | 20:29 |
| ayoung | bknudson, nah, one to fetch the dn, then make a big list, and filter by objectclass | 20:29 |
| bknudson | we could fetch all users and filter | 20:29 |
| ayoung | No we couldn't | 20:30 |
| bknudson | or all groups, that would be a shorter list | 20:30 |
| ayoung | Nope | 20:30 |
| richm | so a filter like (|(dn=dn1)(dn=dn2)(....)(dn=dnN)(objectclass=posixGroup)) | 20:30 |
| richm | ? | 20:30 |
| ayoung | richm, yes | 20:30 |
| richm | note that dn is not a real attribute | 20:30 |
| ayoung | but we could potentially run into server limitations | 20:30 |
| bknudson | ldapsearch -LLL -D "cn=Manager,dc=openstack,dc=org" -w "ofs5dac" -s base -b cn=0ddcdc2ee75246999f662a1b09853086,ou=Users,dc=openstack,dc=org "dn=cn=0ddcdc2ee75246999f662a1b09853086,ou=Users,dc=openstack,dc=org" | 20:30 |
| richm | and you can't specify more than one search base in an ldap query | 20:30 |
| bknudson | that didn't return anything | 20:31 |
| *** marcoemorais has quit IRC | 20:31 | |
| ayoung | richm, so you can't | together DN searches? | 20:31 |
| richm | I think you will have to use an ldap extensible filter to query for DN=someDN in a search filter | 20:31 |
| bknudson | ldap's got more extensions than keystone | 20:32 |
| richm | indeed | 20:32 |
| ayoung | just wait. | 20:32 |
| *** dstanek is now known as dstanek_zzz | 20:33 | |
| ayoung | OK...so is bknudson 's approach right, then? | 20:34 |
| *** dstanek_zzz is now known as dstanek | 20:35 | |
| ayoung | A user and a group with the exact same DN is impossible | 20:35 |
| bknudson | ayoung: don't ask you might not like the answer | 20:35 |
| ayoung | bknudson, class like that is only possible if they are in different subtrees | 20:36 |
| bknudson | entries can have multiple objectclasses | 20:36 |
| bknudson | I don't think keystone would work as expected | 20:36 |
| richm | an entry can be a user and a group | 20:36 |
| bknudson | we don't have to support it | 20:36 |
| richm | e.g. a user private group entry could be implemented that way | 20:36 |
| ayoung | richm, I think that would be OK | 20:37 |
| richm | ok | 20:37 |
| ayoung | bknudson, is this even a real problem? | 20:37 |
| openstackgerrit | A change was merged to openstack/python-keystoneclient: Add mailmap entry https://review.openstack.org/92669 | 20:37 |
| ayoung | I mean, I guess you could have a Nova user and a Nova group, and they would be completely unrelated? | 20:37 |
| bknudson | ayoung: the LDAP part or the SQL part? | 20:38 |
| ayoung | Both | 20:38 |
| ayoung | SQL is not areally a problem, unless we have UUID clash | 20:38 |
| ayoung | LDAP...meh | 20:38 |
| bknudson | ayoung: somebody here ran into it. | 20:38 |
| bknudson | using LDAP for identity and SQL for assignments | 20:38 |
| bknudson | r/o LDAP | 20:39 |
| ayoung | bknudson, ah...so SQL assignment fix gets the wors offenders...ok, split that out, and we can discuss the other later. | 20:39 |
| openstackgerrit | Brant Knudson proposed a change to openstack/keystone: SQL fix for get_roles_for_user_and_project user=group ID https://review.openstack.org/94396 | 20:41 |
| openstackgerrit | Brant Knudson proposed a change to openstack/keystone: LDAP fix for get_roles_for_user_and_project user=group ID https://review.openstack.org/94470 | 20:41 |
| bknudson | there they are | 20:41 |
| bknudson | I'll propose the fix to stable/icehouse when it's +2 | 20:42 |
| *** shakamunyi has joined #openstack-keystone | 20:46 | |
| *** browne has quit IRC | 20:46 | |
| *** marcoemorais has joined #openstack-keystone | 20:50 | |
| openstackgerrit | Christian Berendt proposed a change to openstack/python-keystoneclient: replace string format arguments with function parameters https://review.openstack.org/94205 | 20:54 |
| *** jsavak has joined #openstack-keystone | 20:54 | |
| *** joesavak has quit IRC | 20:54 | |
| *** marcoemorais has quit IRC | 21:04 | |
| *** jraim has quit IRC | 21:04 | |
| *** jraim has joined #openstack-keystone | 21:05 | |
| bknudson | did we lose the auto-abandon feature? | 21:07 |
| bknudson | I'd think https://review.openstack.org/#/c/77514/ would be abandoned since jenkins -1 | 21:07 |
| morganfainberg | bknudson, perhaps | 21:07 |
| *** browne has joined #openstack-keystone | 21:11 | |
| *** amcrn has joined #openstack-keystone | 21:13 | |
| *** browne has quit IRC | 21:16 | |
| *** erecio_2 has joined #openstack-keystone | 21:16 | |
| *** erecio_1 has quit IRC | 21:18 | |
| *** browne has joined #openstack-keystone | 21:22 | |
| *** daneyon has joined #openstack-keystone | 21:31 | |
| *** andreaf_ has quit IRC | 21:36 | |
| *** daneyon has quit IRC | 21:36 | |
| *** ayoung has quit IRC | 21:37 | |
| *** daneyon_ has joined #openstack-keystone | 21:40 | |
| tristanC | bknudson: checking 94396 (the SQL fix), is it normal you change the test_backend_ldap ? Also do I need to reference both new change or only 94470 ? | 21:48 |
| *** gokrokve has quit IRC | 21:49 | |
| bknudson | tristanC: the test in test_backend_ldap overrides the test in test_backend, since the test in test_backend doesn't work for LDAP (since it's not fixed) | 21:50 |
| tristanC | oh ok, just making sure :) | 21:51 |
| bknudson | tristanC: I think you'll want to reference both https://review.openstack.org/#/c/94396/ and https://review.openstack.org/#/c/94470/ for the fix in the announcement | 21:51 |
| *** amerine_ is now known as amerine | 21:51 | |
| bknudson | tristanC: I would also expect there'll be 2 changes for the stable/icehouse fix. | 21:51 |
| *** ayoung has joined #openstack-keystone | 21:51 | |
| bknudson | or more, since complications keep getting added | 21:51 |
| tristanC | bknudson: alright, thanks for the clarification | 21:52 |
| *** browne has quit IRC | 21:52 | |
| tristanC | well I guess it will have to wait tomorrow unless we can get all 4 patches approved in the next few hours | 21:52 |
| bknudson | Apparently we'll need to write a function to compare DNs which will require somehow querying the LDAP server for the schema or something | 21:53 |
| *** jsavak has quit IRC | 21:55 | |
| *** lbragstad has quit IRC | 21:55 | |
| tristanC | Well if we could avoid refactoring to much, I would prefer we have the security fix merged before. On the other hand, having a correct "bulletproof" fix is also worthy. | 21:59 |
| *** harlowja is now known as harlowja_away | 21:59 | |
| bknudson | tristanC: people seem happy with the SQL fix. | 22:00 |
| bknudson | and that's also the one that I'd expect deployers to be more interested in | 22:00 |
| *** mriedem has joined #openstack-keystone | 22:00 | |
| *** browne has joined #openstack-keystone | 22:00 | |
| mriedem | praneshp: re https://review.openstack.org/#/c/93801, looks like an infra issue | 22:00 |
| mriedem | zuul probably went wonky | 22:00 |
| *** gyee has joined #openstack-keystone | 22:01 | |
| mriedem | praneshp: i'll send an email on it to the maintainer | 22:01 |
| *** dstanek is now known as dstanek_zzz | 22:01 | |
| *** stevemar has quit IRC | 22:02 | |
| morganfainberg | dolphm, so what was the end result on {program}-spec vs {project}-spec? | 22:02 |
| morganfainberg | dolphm, i lost track of the meeting | 22:02 |
| dolphm | morganfainberg: program wins | 22:02 |
| *** sbfox1 has quit IRC | 22:03 | |
| morganfainberg | dolphm, yay | 22:03 |
| dolphm | morganfainberg: something like 12-2 | 22:03 |
| morganfainberg | i approve of that | 22:03 |
| dolphm | or 12 to 2 to 1 | 22:03 |
| dolphm | there were 3 options | 22:03 |
| dolphm | one of which was 'markwash' | 22:03 |
| *** nkinder has quit IRC | 22:03 | |
| dolphm | which ended up getting 2 votes | 22:03 |
| mriedem | praneshp: fyi there is a db2 CI wiki but it's pretty bare right now, i'll get someone to update that with contact info and how to recheck https://wiki.openstack.org/wiki/IBM/DB2-TEST | 22:03 |
| *** leseb has quit IRC | 22:04 | |
| praneshp | mriedem: thanks a lot | 22:05 |
| *** ChanServ sets mode: +o morganfainberg | 22:06 | |
| *** morganfainberg changes topic to "Juno-1 June 12th! New formalized Identity-spec process for Juno-2 and beyond blueprints." | 22:06 | |
| *** ChanServ sets mode: -o morganfainberg | 22:07 | |
| *** rodrigods_ has joined #openstack-keystone | 22:07 | |
| *** rodrigods has quit IRC | 22:08 | |
| *** harlowja_away is now known as harlowja | 22:08 | |
| bknudson | apparently jdennis has a class that implements DN compare | 22:09 |
| mriedem | praneshp: email sent, thanks for bringing it to my attention | 22:10 |
| *** bknudson has quit IRC | 22:10 | |
| mriedem | praneshp: i also rechecked the patch but not sure it will do anything if there are zuul issues | 22:11 |
| *** sbfox has joined #openstack-keystone | 22:12 | |
| *** packet has quit IRC | 22:14 | |
| *** nkinder has joined #openstack-keystone | 22:16 | |
| *** mriedem has left #openstack-keystone | 22:18 | |
| *** rodrigods_ has quit IRC | 22:19 | |
| *** browne has quit IRC | 22:22 | |
| *** dstanek_zzz is now known as dstanek | 22:23 | |
| *** gordc has quit IRC | 22:23 | |
| *** rodrigods has joined #openstack-keystone | 22:30 | |
| *** rodrigods has joined #openstack-keystone | 22:30 | |
| *** dstanek is now known as dstanek_zzz | 22:33 | |
| *** dims has quit IRC | 22:35 | |
| rodrigods | bknudson, please review https://review.openstack.org/#/c/91578/ when you have a moment =) | 22:40 |
| *** david-lyle has quit IRC | 22:43 | |
| *** daneyon_ has quit IRC | 22:49 | |
| gyee | jamielennox, you awake? | 22:52 |
| *** browne has joined #openstack-keystone | 22:52 | |
| jamielennox | gyee: yea i'm here | 22:53 |
| gyee | jamielennox, Bob is working on a patch for keystoneclient Session to add the chunking functionality | 22:54 |
| *** bknudson has joined #openstack-keystone | 22:54 | |
| gyee | looks like Glance client is using it | 22:54 |
| jamielennox | ... what's required? | 22:54 |
| gyee | Glance | 22:54 |
| jamielennox | http://docs.python-requests.org/en/latest/user/advanced/#chunk-encoded-requests | 22:54 |
| jamielennox | so if you pass data=gen() it should work now | 22:55 |
| gyee | right, essentially, just wrap the data into a generator | 22:55 |
| gyee | exactly | 22:55 |
| jamielennox | so what is he prposing on session to help with that? | 22:55 |
| gyee | question is do we want that change to be in glanceclient or keystoneclient | 22:56 |
| jamielennox | is he here? i dont know his nick | 22:56 |
| gyee | basically inspect the header for Content-Encoding | 22:56 |
| gyee | if it is chunked, then wrap the data into a generator | 22:56 |
| jamielennox | oh | 22:57 |
| jamielennox | umm, can't we fix it higher up than that? | 22:57 |
| jamielennox | as in make glanceclient actually pass the generator as exected? | 22:57 |
| gyee | oh | 22:57 |
| gyee | yeah, that's an option, the benefit of doing it in keystoneclient is that the code is shared | 22:57 |
| bknudson | keystoneclient should be able to handle chunked transfers too | 22:57 |
| jamielennox | then hack the old case to add the content-tye and do the generator from there | 22:58 |
| jamielennox | bknudson: it does | 22:58 |
| *** bobt has joined #openstack-keystone | 22:58 | |
| gyee | jamielennox, bobt's here | 22:58 |
| jamielennox | gyee: kind of - i mean in a way you're trying to wrap a low level interface to a high level one | 22:58 |
| gyee | jamkelennox, well, Session is kind of low lever | 22:59 |
| jamielennox | the use of a generator is supposed to hide you from things like setting content-type: chunked | 22:59 |
| gyee | level | 22:59 |
| bobt | transfer-encoding: chunked | 22:59 |
| jamielennox | you're doing the reverse, if you detect the content-type: chunked header then you wrap the data into a generator and then make requests re-set the content-type | 22:59 |
| jamielennox | sorry transfer-encoding | 22:59 |
| jamielennox | let me check glance-client | 22:59 |
| gyee | glanceclient is doing the exact same | 23:00 |
| gyee | tee off on the transfer-encoding header | 23:00 |
| jamielennox | right, but glanceclient is kindof stupid like that | 23:00 |
| *** ayoung has quit IRC | 23:01 | |
| gyee | I think they make some assumptions | 23:01 |
| jamielennox | gyee: bobt: do you know where it is set? | 23:01 |
| bobt | glanceclient/common/http.py | 23:01 |
| gyee | raw_request() | 23:02 |
| jamielennox | yea, line 310 if content_length is None | 23:02 |
| gyee | looks like raw is always chunked | 23:02 |
| gyee | ah, for streaming, where content-length is not know ahead of time | 23:03 |
| gyee | known | 23:03 |
| gyee | jamielennox, we can move that logic to keystoneclient, with the same assumption | 23:04 |
| *** amcrn_ has joined #openstack-keystone | 23:04 | |
| jamielennox | gyee: well i was wondering can we fix glanceclient's http? | 23:05 |
| jamielennox | from requests: is_stream = all([ | 23:05 |
| jamielennox | hasattr(data, '__iter__'), | 23:05 |
| jamielennox | not isinstance(data, (basestring, list, tuple, dict)) | 23:05 |
| jamielennox | ]) | 23:05 |
| gyee | jamielennox, we can fix it in either glanceclient or add it to keystoneclient, the benefit of having it in keystoneclient is that it is shared | 23:06 |
| gyee | but if only glance needs it then we can fix it there instead | 23:06 |
| *** amcrn has quit IRC | 23:06 | |
| bobt | does cinder need chunking support or just glance? | 23:07 |
| jamielennox | gyee right but we're having to share: https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/http.py#L290-L313 | 23:07 |
| jamielennox | there's more than a simple check there | 23:07 |
| jamielennox | amongst other things it sets content-type | 23:08 |
| gyee | right, basically adding raw_request() | 23:08 |
| jamielennox | i'm wondering if we just make glanceclient create a generator, could we make glanceclient work like requests | 23:08 |
| jamielennox | because i think the notion that i pass a generator and it becomes chunked is easier than | 23:09 |
| gyee | I suppose we could, by having a glance Session which subclass keystoneclient Session, something like that | 23:09 |
| jamielennox | if (('body' in kwargs) and (hasattr(kwargs['body'], 'read') and method.lower() in ('post', 'put'))): then it becomes chunked | 23:09 |
| gyee | right | 23:09 |
| gyee | like I said, if only glance needs it then there's no benefit of doing it in keystoneclient | 23:10 |
| gyee | jamielennox, how about we do the fix on glance side for now, and if we discover more clients need it later, we add it to keystoneclient? | 23:11 |
| jamielennox | gyee: yep if we need it we need it - but for now i'd like session to stay as clean as possible | 23:14 |
| jamielennox | if it's an enhancement then that's great | 23:14 |
| jamielennox | but this is a little like carrying other peoples hacks | 23:14 |
| gyee | alrighty then | 23:14 |
| gyee | jamielennox, also are you pushing the append command line args patch for keystoneclient anytime soon? | 23:15 |
| jamielennox | does read() given an iterator? | 23:15 |
| jamielennox | yea, i've got a session from conf i'm working on now | 23:15 |
| jamielennox | then i'll look at auth and session from cli | 23:15 |
| jamielennox | unfortunately some of that's broken for keystoneclient's shell | 23:16 |
| *** ayoung has joined #openstack-keystone | 23:16 | |
| gyee | let me check, I think read() have a few options | 23:16 |
| gyee | jamielennox, Response.raw can do streaming | 23:18 |
| jamielennox | gyee: because one of those checks is hasattr(kwargs['body'], 'read') so at some point the glanceclient is passing around a file object | 23:18 |
| jamielennox | so it should be fairly easy to identify where you need to replace with a generator | 23:18 |
| jamielennox | (file or other streaming - which i don't know what else can stream) | 23:18 |
| gyee | http://docs.python-requests.org/en/latest/api/#requests.Response.raw | 23:19 |
| gyee | its essentially a file-like object | 23:19 |
| *** dstanek_zzz is now known as dstanek | 23:19 | |
| gyee | bknudson, do you know any other clients have the need for chunking functionality? you mentioned we may need this in keystoneclient earlier | 23:21 |
| *** diegows has quit IRC | 23:21 | |
| jamielennox | gyee: but that would imply they are receiving from another API and directly streaming that onto something else? | 23:22 |
| bknudson | gyee: chunked transfer coding is part of the http 1.1 spec. All clients should be able to do it | 23:22 |
| bknudson | HTTP 1.1 came out in 1999. seems like we should be able to support it by now | 23:22 |
| gyee | heh | 23:22 |
| gyee | its only like what 15 years? | 23:23 |
| bknudson | exactly. I don't think it's going to go away. | 23:23 |
| bknudson | I was worried at first, but it seems to have caught on | 23:23 |
| bknudson | http://tools.ietf.org/html/rfc2616#section-10.1.1 | 23:24 |
| bknudson | was there a concern about Expects: 100 Continue also? | 23:24 |
| bknudson | Expect: 100-continue is the header | 23:25 |
| gyee | jamielennox, if Glance is using Swift backend, perhaps they are passing the file object directly into Swift? | 23:26 |
| gyee | let me check | 23:26 |
| gyee | bknudson, is the 100-continue header used for streaming? | 23:27 |
| gyee | I don't remember seeing it being used | 23:28 |
| bknudson | gyee: it's used to more efficiently handle an operation that can be aborted by looking at the headers | 23:28 |
| gyee | oic | 23:29 |
| bknudson | apache will send it if it's used as a proxy | 23:29 |
| jamielennox | gyee: hmm, maybe | 23:29 |
| bknudson | it sends the headers for the original request with "Expect: 100-continue", and then it won't send the body unless it gets back the 100 Continue response. | 23:29 |
| bknudson | I'd expect glance would want to use it if they can | 23:30 |
| jamielennox | gyee: so https://github.com/openstack/python-glanceclient/blob/master/glanceclient/common/utils.py#L293 looks like the only place that does an open like that | 23:33 |
| jamielennox | which is used https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/shell.py#L310 and https://github.com/openstack/python-glanceclient/blob/master/glanceclient/v1/shell.py#L229 | 23:34 |
| jamielennox | and the equivalent in v2 shell | 23:34 |
| jamielennox | so i'm not sure if that's a compatability issue we need to maintain | 23:35 |
| gyee | jamielennox, yeah you're right, doesn't make sense to add this to keystoneclient as they are using it in a specific way | 23:35 |
| jamielennox | so it should be as simple as fixing the glanceclient image.update() to create a generator from a read call | 23:36 |
| jamielennox | and image.create() | 23:36 |
| jamielennox | and then interpretting the generator in raw_request as meaning it should be chunked | 23:37 |
| jamielennox | which isn't that simple :) | 23:37 |
| jamielennox | so bknudson, gyee whilst your here - i need a way to reauthenticate an auth_plugin and at the moment the only interface i have is get_token | 23:38 |
| jamielennox | so if you get a 401 then refetch a token even if it's still in the valid time range | 23:38 |
| jamielennox | i think my choices are essentially add force=True to get_token() to always fetch a new token | 23:39 |
| bknudson | it might have been revoked | 23:39 |
| jamielennox | bknudson: right | 23:39 |
| jamielennox | or add an invalidate() method to the plugin, call that, and then expect the next call to get_token will do the right thing | 23:39 |
| jamielennox | i think that flow is kind of easier - but neither feels great. any opinions? | 23:40 |
| jamielennox | this is something that will be handled by the session object so a user should never really care | 23:40 |
| jamielennox | i should have stuck to my guns originally and had authenticate() and get_token() as seperate methods | 23:41 |
| bknudson | does it also fetch if the token is close to expiring? | 23:41 |
| jamielennox | bknudson: yes | 23:41 |
| gyee | jamielennox, didn't we have that force new token option at one point? | 23:41 |
| jamielennox | gyee: yes, initially when i had a seperate authenticate() call so i could trigger a force | 23:41 |
| jamielennox | i was pursuaded that it wasn't needed and let get_token() do everything | 23:42 |
| bknudson | if I call authenticate() then get_token() returns a different token? | 23:42 |
| bknudson | and authenticate() fails if my user is disabled? | 23:42 |
| jamielennox | bknudson: sorry i think i'm confusing the situation with the cross-conversation - there is no authenticate() call at the moment | 23:42 |
| bknudson | is that how you'd expect it to work, though? | 23:43 |
| jamielennox | there is an auth plugin that you can call get_token() on and it should return you something that it feels is valid | 23:43 |
| jamielennox | when the actual auth happened should be irrelevant (cause it might have happened prior to be provided to the session) | 23:43 |
| bknudson | the auth plugin would have an invalidate to invalidate its token? | 23:44 |
| jamielennox | however in the case of a 401 i need to make the auth token try again | 23:44 |
| jamielennox | right so i could call plugin.invalidate() redo the call and the otken would be refetched on the next round | 23:44 |
| bknudson | that would make sense to me. | 23:44 |
| jamielennox | that feels better to me that putting force= on the get_token | 23:45 |
| gyee | yeah, force= sounds better | 23:48 |
| gyee | just need to document the expectation | 23:48 |
| gyee | bknudson, https://review.openstack.org/#/c/94396/2/keystone/tests/test_backend_ldap.py, why is this test in the sql patch? | 23:51 |
| gyee | there's a separate LDAP patch right? | 23:52 |
| bknudson | gyee: since ldap doesn't handle the situation correctly, the general backend test doesn't work for it | 23:52 |
| bknudson | gyee: so ldap has to override the base test | 23:52 |
| bknudson | gyee: that test shows that ldap doesn't work correctly because the role is returned when it shouldn't be | 23:53 |
| bknudson | gyee: the next patch in the series fixes the LDAP backend to work like the SQL backend so then we can get rid of the overridden test | 23:53 |
| gyee | bknudson, ah, I see what you mean | 23:54 |
| gyee | the code looks almost identical | 23:54 |
| dstanek | we're eventually going to start using just one port for keystone instead of two right? | 23:58 |
| bknudson | dstanek: yes, 443 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!