Friday, 2014-04-25

morganfainbergdstanek, but... i'm not 100% clear still on some of the fallout from this one00:00
morganfainbergsetting up vm now for schema tests00:00
dstanekmorganfainberg: while i agree it seems like a lot of change because alembic doesn't support 5.5 naming convertions yet00:00
dstanekmorganfainberg: when you are testing try to add a new fk and see if you can00:01
morganfainbergdstanek, ok i think i can create a contrived migration for that once i check schemas00:01
morganfainbergmaybe this is because we renamed a column at some point?00:02
dstaneknot sure if this is the same:
*** browne has quit IRC00:06
morganfainbergthats migrating from icehouse, then checking out master -> current (first file), 2nd file is applying migrate 5000:06
*** nkinder has quit IRC00:11
openstackgerritBrant Knudson proposed a change to openstack/keystone: Add localized response test
*** derek_c has quit IRC00:18
morganfainbergdstanek, here is the juno (with collapsed migrate) version00:19
morganfainbergbut i am getting the sinking feeling we'll need one of these for each migration that has a FK00:20
*** lbragstad has joined #openstack-keystone00:27
*** jaosorior has quit IRC00:31
openstackgerritLance Bragstad proposed a change to openstack/keystone: Implement validation on projects
*** derek_c has joined #openstack-keystone00:33
*** browne has joined #openstack-keystone00:34
*** browne has quit IRC00:36
openstackgerritLance Bragstad proposed a change to openstack/keystone: Allow 'description' in V3 Regions to be optional
morganfainbergjamielennox, tagged you to lbragstad's review on validation00:38
morganfainbergjamielennox, since you were working on the validator (descriptor style)00:38
jamielennoxmorganfainberg: looking00:38
jamielennoxmorganfainberg: i don't know if it's going to be worth it00:38
jamielennoxi'm leaving it for now00:38
morganfainbergjamielennox, right.00:39
lbragstadmorganfainberg: jamielennox thanks00:39
jamielennoxspent more or less a whole day and was always so close00:39
morganfainbergjamielennox, lbragstad, i want to make sure we hit a good solution, even if it's rough00:39
jamielennoxbut the combination of what is and isn't a schema and what is and isn't an object just gets confusing00:39
jamielennoxi'm thinking we do the most basic for now00:39
morganfainbergso since you both took different takes on it...00:39
jamielennoxclass Request(dict):00:39
lbragstadyeah.. I agree...00:39
jamielennox    schema = { ... }00:39
jamielennoxlbragstad: did you catch the stuff i was playing with yesterday?00:40
lbragstadjamielennox: I don't think I got the latest review...00:40
lbragstaddo you have a link?00:40
jamielennoxlbragstad: there was no review,00:40
morganfainbergjamielennox, hitting the simplest solution is the best.  i'm thinking we keep looking for something that lets us deserialize and validate in the same mechanism00:40
jamielennoxi couldn't get it to play out how i wanted it to00:40
morganfainbergjamielennox, but i don't know if it exists.00:41
lbragstadjamielennox: what were you trying?00:41
lbragstadI'm jsut going of the nova V3 api validation stuff00:41
morganfainberglbragstad, this will need a BP attached to it (in either case)00:41
* morganfainberg wants to see us be better about tracking associated changes.00:42
morganfainbergbugs or bp etc00:42
jamielennoxlbragstad: if i could get it to work i think it would be really cool00:43
jamielennoxlbragstad: i've just wasted too much time on it already00:43
lbragstadso.. .you're validating types too00:43
lbragstadI don't think what I'm doing is that indepth00:43
jamielennoxlbragstad: so i want to be somewhere between WSME/Sqlalchemy and jsonschema00:43
jamielennoxthe layout of WSME but still able to generate the full schema00:44
jamielennoxmorganfainberg: i got closer than that, i did figure out the problem of type vs data00:44
jamielennoxmorganfainberg: a pure type object, a model object and then a 'relationship' object that provided all the __get__ __set__ etc00:45
jamielennoxrelationship object was built 'magically' by the metaclass00:45
morganfainbergjamielennox, hmmm.00:46
morganfainbergjamielennox, interesting00:46
lbragstadnova accomplishes this by just having a schema template to validate against, which I was thinking we would have for each resource, and then we could validate with a wrapper, like sending a notification:
jamielennoxlbragstad: i was thinking a similar thing00:46
lbragstadso, when someone builds an extension they could leverage the jsonschema validator by adding the appropriate schema for the resource...00:47
jamielennoxbut i've long been of the opinion that we need to move all these dicts we pass around to actual objects00:47
lbragstadjamielennox: if you have ideas and want to hack on what I already have, you're more than welcome to... It's kinda a wip right now..00:47
morganfainbergjamielennox, ++++00:47
lbragstadso like nova00:47
jamielennoxso i want a UserRequestInput object00:47
lbragstadand the migration to object representation00:47
jamielennoxlbragstad: nova objects are way more indepth than that00:48
lbragstad.. I don't think I would mind that...00:48
jamielennoxand actually the only thing i found that nova objects don't do is the representation00:48
jamielennoxyou still access them like a dictionary00:48
jamielennox(or you did last i looked)00:48
lbragstadright, which can make for an easier transition00:48
jamielennoxlbragstad: right they will need to be at least dictionary compatible for a while/ever00:49
jamielennoxbut i just want some protection and implicit validation of using the object and being able to know we are setting an incorrect value00:49
lbragstadon a side note, I think having a consistent validator will help clean up the controller code too00:49
jamielennoxlbragstad: i want that to die so badly00:50
lbragstadjamielennox: agreed..00:50
jamielennoxdolphm: -2ed my last patch in that way00:50
lbragstadwell, I'm hoping that this jsonschema thing is a step in the right direction00:50
jamielennoxyea, i've given up on WSME00:51
jamielennoxi just want to go schema00:51
jamielennoxthen we can actually expose the schema via an API00:51
lbragstadI like it because we can just set a schema for a resource...00:51
jamielennoxi found a while ago - and i want a reason to use it00:51
lbragstadso we have pretty good control over what is a 'valid' attribute or not00:52
jamielennox(not me, hate javascript, but someone...)00:52
lbragstadI think I've heard of that00:52
lbragstadI think my office mate was talking about it00:52
jamielennoxi was thinking a while ago about building a service (non-openstack) where you could install a schema and then do CRUD operations against it00:53
lbragstadthat'd be useful00:54
jamielennoxyou could automate 90% of forms, and a lot of custom services00:54
*** lbragstad is now known as lbragstad__00:54
lbragstad__gotta run quick, jamielennox let me know what you think about the schema stuff if you get a chance, might need to fiddle with the layout a bit still00:55
jamielennoxlbragstad__: ok, i want this to happen and i don't mind so much about the format for now if it's jsonschema based00:55
lbragstad__++ good deal, thank you00:56
morganfainbergi would love something like protobuf (that isn't... protobuf?)01:03
morganfainbergfor each data type.01:04
morganfainbergs/data type/object type/01:04
morganfainbergnot that i'd use it for external communication (sure there are arguments for/against it, but it would be a harder sell)01:05
*** amcrn has quit IRC01:05
*** marcoemorais has quit IRC01:11
*** richm has quit IRC01:11
*** marcoemorais has joined #openstack-keystone01:14
*** marcoemorais has quit IRC01:20
*** derek_c has quit IRC01:24
jamielennoxmorganfainberg: there was another one i just saw01:25
jamielennoxmorganfainberg: messagepack01:27
jamielennoxi don't see any advantage to it over protobuf but it looked interesting01:27
*** nkinder has joined #openstack-keystone01:34
*** praneshp has quit IRC01:35
morganfainbergjamielennox, messagepack i'm not a huge fan of01:35
morganfainbergjamielennox, i've used it before, its FAST but there are utf8 issues01:36
morganfainbergand nasty ones01:36
jamielennoxmorganfainberg: interesting, they don't mention that on their website01:36
*** stevemar has quit IRC01:36
morganfainbergjamielennox, it had a nasty bug related to it, i did a lot of work debugging it for saltstack at one point01:37
morganfainbergit's very unfun.01:37
jamielennoxi don't care really either way - i think compression and validation support what we need01:37
jamielennoxthough it'd be nice if something could do validation and deserialization in one pass01:37
morganfainbergi think the best bet is protobuf for that01:37
morganfainbergusing the json-protobuf module01:37
morganfainbergit does it in one shot iirc.01:37
jamielennoxi was looking at ayoung's PKIZ review the other day01:37
morganfainbergbut... might be a tough sell01:38
jamielennoxwe need a way of making that more pluggable01:38
* ayoung snaps out of his stupor01:38
*** diegows has quit IRC01:38
morganfainbergand you leave a lot on the table if you don't use the binary serialization/deserialize w/ protobuf (no no not advocating using it that way)01:38
*** derek_c has joined #openstack-keystone01:38
jamielennoxayoung: i was going to -1 it and say why is this a seperate format rather than a decompress around an existing format01:38
jamielennoxbut you're doing DER now rather than PEM01:38
morganfainbergi want something that lets us deserialize and validate in one shot.01:39
ayoungmorganfainberg, I kindof like the layerd stack01:39
morganfainbergbut.. the one option i've found i'm not sure i want to really push that hard for (protobuf)01:39
morganfainbergayoung, it has some nice advantages.01:39
ayoungcompression, signature, and serialization are all tricky things. and I like using the best libraray we can for each level.  Allos us to trade up if we want to01:40
*** gokrokve has quit IRC01:40
jamielennoxmorganfainberg: right - because we would still end up validating the token format in the middleware again01:40
dstanekmorganfainberg: sorry, was doing dad stuff01:40
morganfainbergjamielennox, yep. though tokens are a case i could see protobuf being a win01:40
jamielennoxASN1 tokens!01:40
morganfainbergjamielennox, provide the protoc stuff (yes i know it's binary) but it would work across languages01:40
ayoungIs that like ANS1?01:41
dstanekmorganfainberg: you'd have to do that for every fk right?01:41
jamielennoxheh, well supports both now01:41
morganfainbergdstanek, no, i can't tell why some are screwy and some aren't01:41
morganfainbergdstanek, well i know WHY but i don't know how we end up screwy.01:41
morganfainbergit's ... odd01:42
jamielennoxayoung: it's weird how there can be a code review process and no-one notices that stuff01:42
morganfainbergayoung, when the original objectification talk occured (grizzly?) for nova i think there were a couple of us who said protobuf fit the need and provided versioned (layered?) objects01:42
morganfainbergjamielennox, lysdexia?01:43
morganfainbergand ANS1 - ansi?01:43
jamielennoxmorganfainberg: nah, the first and last letters need to be the same01:43
ayoungmorganfainberg, would that do signatures?  Would it do compression?01:44
morganfainbergayoung, well, we probably would need to layer in a signature object, but it could be the same object def for anything needing a sig01:45
ayoungthen all it would really replace is the JSON?01:45
morganfainbergayoung, no i mean if we used protobuf you'd have a "signed" object type, you'd include the signature in that type and that type would include the object that was signed01:46
morganfainbergayoung, which could be ANY(type1, type2, type3) and you get validation on both fronts01:46
morganfainbergayoung, it would replace JSON with binary... and it there is an argument against using binary wire formats ... debugging sucks.01:46
morganfainberga lot01:46
morganfainbergayoung, i think to do native protobuf compression you'd have to use the native RPC code01:47
jamielennoxcommunity: is this something we want to support in client?01:47
jamielennox(it's not mine)01:48
morganfainbergdoesn't V3 remove the admin/public url specifics?01:48
jamielennoxin general i don't mind - but it's making an exception for the admin/public split in v2 which i'm not sure i want to do01:48
jamielennoxmorganfainberg: yes - but currently the client defaults to admin url for everything01:49
morganfainbergjamielennox, i'm also wary of adding anything to v2 - unless it's REALLY needed01:49
morganfainbergjamielennox, v2 should be "frozen" imo.01:49
jamielennoxwhich is something that should be changed - but it was something i was hoping to attack with the session object rather than in the client01:49
ayoungIt seems like we have a good V2/V3 interop story now...or at least you can possibly use them both on the same endpoint, right?01:50
jamielennoxayoung: not even close01:50
jamielennoxayoung: there is a plan i think - but that's not an implemented story01:50
ayoungIf the endpoint says v2.0, can you do a v3 call now, somehow?01:50
jamielennoxyea - with the old client01:50
jamielennoxbut there is no way to know if v3 is supported by the server01:51
jamielennoxthere is no way to take a token you have and move to v301:51
jamielennoxwell there is but it sucks01:51
jamielennoxand i haven't done the v2/v3 hack in the session - so moving to the newer method doesn't help yet01:51
ayoungthere is no automated way, but a savvy user could hit the endpoint with a browser and see01:51
ayoungRight?  I mean, all of the CLI stuff assumed /v2.0, but if you want to explicitly create a V3 session and work with it, you can, or is there something missing?01:53
jamielennoxumm, you can do v3 auth with a v2 client if you use the session object01:53
jamielennoxi don't think anyone is actually doing that yet01:54
jamielennoxalso as i said if you do it that way you don't get the v2/v3 URL hack01:54
jamielennoxi was waiting for the unversioned endpoints in catalog review to go through before i did that01:55
jamielennoxthat's dependent on the discovery changes01:56
ayoungSo priority of review goes to..?01:57
*** derek_c has quit IRC01:59
ayoungjamielennox, that is mostly code reshuffling, right?  What is the actual logic difference?01:59
jamielennoxayoung: it's mostly reshuffling02:00
*** topol has joined #openstack-keystone02:00
jamielennoxayoung: the problem is that the original discover has a dependency on the client object, because it can create clients for you02:00
jamielennoxit means there is a circular dependency to actually using discovery for something within client02:00
ayoungand now...what is the path to using it?02:01
ayoungor is it still there?02:01
jamielennoxi did the original discovery before i had the multi-client plan in place02:01
jamielennoxso from inside the keystoneclient we can import _discovery - everyone else can use discovery as normal02:01
jamielennoxthe new way exposes things like url_for() so you can actually query discovery - the old way all you could really do was create a new client object02:02
ayoungany side effects or API breakages due to this?  cuz it looks pretty straightforward otherwise.02:03
jamielennoxthere shouldn't be any breakages02:03
jamielennox(i'm getting pretty atuned to that now)02:03
jamielennoxit's just adding some new functions to the discovery object02:03
ayoungmorganfainberg, he addressed your concerns?  Want to move this one ahead?  Without v2/v3 interop, we are kindo dead inthe water stuck on V2.02:04
morganfainbergayoung, hmm?02:05
morganfainbergayoung, oh i'm not opposed to the change, i just want to make sure we're clear on why we're changing anything v202:06
morganfainbergayoung, and sure, this is def. a clear benefit02:06
jamielennoxmorganfainberg: new topic02:06
morganfainbergsorry was talking to someone here02:06
ayoungjamielennox, I think you have some rebasing to do02:06
jamielennoxayoung: possibly, i haven't actual chagned that code for a while02:06
ayoung  is the unversioned patch you were talking about?02:06
jamielennoxyea, there are discovery changes embedded in that which was the reason i made the other patch02:07
ayoungthat is the big one, isn't it?02:07
jamielennoxi need to update it02:07
jamielennoxayoung: yea, that will be very useful02:07
jamielennoxit will let us do: as well02:08
ayoungjamielennox, if you get it working, I promise to browbeat the daytime crew into reviewing it02:08
morganfainbergjamielennox, looking at the new patchset now02:08
jamielennoxi was going to do kite stuff today but i'll rebase that now02:09
*** gokrokve has joined #openstack-keystone02:11
ayoungthis is more important than Kite, IMNSHO02:11
*** gokrokve_ has joined #openstack-keystone02:13
*** derek_c has joined #openstack-keystone02:14
morganfainbergjamielennox, i wont block the 81146 but i think the docstrings could use some modifications. I see a lot of narrative-like stuff in them, in one case talking about a past implementation vs current implementation?02:14
morganfainbergjamielennox, i'm looking over the code as well again seeing if anything looks out of place02:15
jamielennoxmorganfainberg: please do, it's been a while since i wrote it02:15
morganfainbergjamielennox, ok do you want a -1 and pointers on the docstrings as well?02:15
*** gokrokve has quit IRC02:15
morganfainbergjamielennox, ok i'll get a bunch of comments up on it so we can be less narrative and more documenting the state of things.02:16
morganfainbergi'll do a second pass for code (or code then docstrings)02:16
jamielennoxmorganfainberg: explaining things is always my problem02:16
jamielennoxwell one of them02:16
ayoungjamielennox, question for you on regions02:17
ayoung  line 3802:17
ayoungI use that in  but its a kwarg so it wasnt' explicit02:17
ayoungam I doing that right?  It is right to make it explicit in the regions patch, right?02:18
jamielennoxayoung: it's just that by making it explicit you remove it from kwargs, but then don't pass it to super02:19
ayounghmmm, ok , so I need to pass it to super, or should I just leave it in the kwargs?02:20
ayoungI guess just pass it is right...02:20
jamielennoxayoung: it's something that's not really handled very well anywhere02:21
jamielennoxin some places it's explicit so that it is readily apparent02:21
jamielennoxin others it's just part of kwargs because it does the same thing anyway02:21
jamielennoxthe best i'm hoping for here is that we get a nice clean new model in -SDK and then i can backport it as part of client 2.002:22
ayoungNone of this has to be pulled out, its really just documentation02:22
ayoungjamielennox, also, in update02:22
ayoungI assume I should change the parameter from parent_region_id to parent_region?02:22
jamielennoxsorry, went through that one fairly quickly, i need to get better at explainning things02:25
ayoungnot a problem, I knew where to find you to ask02:25
*** mberlin1 has joined #openstack-keystone02:26
ayoungjamielennox, but is it right that only the create takes id and the rest take region?02:26
ayoungor should I do the base.getid approach02:27
jamielennoxayoung: yes, because you shouldn't be passing an existing region object to create02:27
jamielennoxthat would be all kinds of messed up02:27
*** mberlin has quit IRC02:27
jamielennoxto clarify (because the or came after) yes, create should use id02:28
jamielennoxi think the parameter should be id=, because of the masking thing you might find some people want it renamed to region_id=02:29
*** derek_c has quit IRC02:30
*** zhiyan_ is now known as zhiyan02:33
ayoungmorganfainberg, ... this patch really needs to be run with the scripts I pulled out of it...because you were insisting on tempest tests...I don't think I can make that happen02:36
openstackgerritayoung proposed a change to openstack/python-keystoneclient: Regions Management
morganfainbergayoung, i am very concerned that if we don't use those scripts in a gate job (could be tempest or otherwise) we're not going to maintain them02:37
morganfainbergayoung, could we .. make a test case that exercises them?02:37
morganfainbergvs. tempest even?02:38
ayoungmorganfainberg, nah...these are examples.  People will be running them and asking us questions about them02:38
ayoungthis will be the code that we point people at, and when the break they say  "I tried that and got a 404"02:38
morganfainbergayoung, ok so we (reviewers) need to run the examples on any change to make sure they aren't broken.02:38
openstackgerritA change was merged to openstack/keystone: Make test_revoke expiry times distinct
ayoungThey are examples.  We can use them to write tempest tests, but that would be secondary02:39
ayoungbut, they are someting we could run to test changes out, yes02:39
morganfainbergour in-tree examples should really be checked that they stay working imo02:39
morganfainbergi get a feeling they are going to bit-rot quickly otherwise. I'm happy to contribute time to help make is so we check them.02:40
morganfainbergs/is/it02:40 about this02:40
morganfainbergand yes the tests can come after.02:40
ayoungwe write them, and then as a follow on, we do tempest tests that call them02:40
morganfainbergjust we need to be sure we do it.02:41
morganfainbergworks for me, i just want to make sure we don't wait too long to get those tests in place02:41
ayoungOK...let me see...the setup and teardown goes back to being its own...region goes back in here, and the revoation event goes back into its own...02:42
morganfainbergmaybe we should get a bug / bp that tagged to j1 for that we can shuffle it down the line, but it puts it on the radar02:42
morganfainbergand keeps us looking at it02:42
morganfainbergfor testing the examples that is02:42
ayoungmorganfainberg, and when tempest rejects them?02:44
morganfainbergayoung, we do the same (less fun) thing we do for keystoneclient tests02:44
morganfainbergmake a testcase that at least tries to keep our examples working02:45
ayoung"the same thing we do every night Pinky..."02:45
morganfainbergayoung, ++02:45
morganfainbergayoung, but... where are we going to get a pair of rubber pants in our size at this time of night?02:45
*** derek_c has joined #openstack-keystone02:45
ayoungYou never cease to amaze me.02:46
jamielennoxmorganfainberg: we need uvirtbot to do those02:46
morganfainberguvirtbot, get on it!02:46
uvirtbotmorganfainberg: Error: "get" is not a valid command.02:46
morganfainbergjamielennox, i agree.02:47
morganfainbergis the python interpreter super smart.. or will "if item in THING and boolean:" end up looking in the "thing" before checking the boolean02:48
jamielennoxi think and is higher precendence02:48
jamielennoxit certainly should be02:48
morganfainbergyeah it should02:48
* morganfainberg doesn't know for sure.02:49
jamielennoxmind you i'd invert that anyway as the bool is cheaper and you wouldn't have that problem02:49
jamielennoxthen again if you care about the cost of order of operations you probably shouldn't be doing python02:49
morganfainbergi was asking because of line 16602:50
morganfainbergit was pure curiosity02:50
morganfainbergnot because i'd ask you to change it02:50
jamielennoxyea should be fine - though it appears i break my own rules02:50
jamielennoxcomment it, because i assume i'm re-rolling it at this point02:51
morganfainberglike i said... that is so much a nit of a potential optimisation i was just curious.02:51
ayounguvirtbot what are we going to do tonight?03:01
uvirtbotayoung: Error: "what" is not a valid command.03:01
ayoungIt was worth a try03:01
*** david-lyle has joined #openstack-keystone03:01
*** david-lyle has quit IRC03:02
ayoungusually with and you want to evaluate the first and ignore the second if the frst is false03:02
ayoungso it should be if boolean and item in thing:03:02
morganfainbergayoung, yeah03:09
morganfainbergayoung in this case it's super nit-picky though,03:09
ayoungmorganfainberg, depends on how often it is executed, and it looks like that logic is in a few places...would be worthwhile to mention it to get it streamlines when you see it03:09
morganfainbergayoung, it also depends on the size of the list/set03:10
ayoungI'd think a boolean would always be faster by several operations even for comparing with an empty list03:10
morganfainbergayoung, i'd agree03:11
ayoungbut if there is even one element in the list, its a string significantly faster03:11
*** marcoemorais has joined #openstack-keystone03:11
morganfainbergjamielennox, ok commented on the review, lots of suggestions03:11
morganfainbergjamielennox, only really one code suggestion, remove the for/else :P03:12
morganfainbergjamielennox, ooooh i need to look at the tests. possibly more incoming03:12
jamielennoxmorganfainberg: what's wrong with for/else03:12
morganfainbergjamielennox, it is very non-intuative even for seasoned python developers03:12
*** marcoemorais has quit IRC03:12
jamielennoxbah, i think in which case seasoned python developers should expand there horizons03:12
morganfainbergjamielennox, it's a code maintainability and future developer comment, it is really a suggestion, but if there isn't a good alternative, this is a case it's more valid than some of the others i've seen03:13
morganfainbergjamielennox, it's just a non-intuative language construct... no other language really does a "if we didn't break out of the loop, run this code"03:13
jamielennoxmorganfainberg: debating on the principal that i like the for/else thing in certain places, i can't remember what i'm doing with it03:13
*** harlowja is now known as harlowja_away03:13
morganfainbergjamielennox, you're doing it in a nested loop03:14
morganfainbergjamielennox, which is why i think it has more merit than other cases i've seen03:14
jamielennoxhmm, yea i don't know if it would be as clean to remove it there03:15
morganfainbergjamielennox, exactly. i'm ok with keeping it if there isn't a good alternative03:15
morganfainbergjamielennox, but it's a strange logic construct03:16
morganfainbergmost people would think "oh if i _don't_ iterate, run this code"03:16
morganfainbergworse is while/else03:16
morganfainbergjamielennox, anyway overall the patchset looks good03:19
jamielennoxwhat should the default behaviour be if someone requests a url for a service but doesn't specify a version/03:28
jamielennoxi want compute service on public interface version 3 is easy03:29
jamielennoxif version is ommitted do we return the most recent?03:29
jamielennoxor just use the URL given in the service catalog03:29
*** Chicago has quit IRC03:29
*** Chicago has joined #openstack-keystone03:34
*** Chicago has joined #openstack-keystone03:34
*** Chicago has quit IRC03:35
morganfainbergjamielennox, hmm.03:50
morganfainbergjamielennox, i think most recent is a sane answer03:50
jamielennoxmorganfainberg: that's what the original patch does03:50
morganfainbergjamielennox you didn't specify, we have to assume you want whatever we think is "best"03:50
jamielennoxbut i don't know if we can03:50
jamielennoxthe existing behaviour is to just use the service catalog03:51
morganfainbergoh backwards compat *doh*03:51
jamielennox- but that's a broken assumption already because the whole point is to move to unversinoed endpoints03:51
morganfainbergcan we make a compat interface that does the same thing as now and new interface "hey guys use this new interface" that does the "get most recent"?03:51
morganfainbergbasically transitional option03:51
morganfainbergif you don't change what you're doing you get current behavior (and a message telling you you should use the new "way")03:52
jamielennoxnot that i can see03:52
jamielennoxwe're into plugins territory03:52
jamielennoxso we're already about 4 layers removed from the actual decision03:52
*** praneshp has joined #openstack-keystone03:52
jamielennoxi don't know if it matters because in the client there should be a layer that says all calls through here are for the identity service03:53
morganfainbergjamielennox, i really want to use the latest version...03:53
jamielennoxand that layer should know to add the correct version03:53
morganfainbergjamielennox, hm. can .. we make that assumption?03:53
*** praneshp_ has joined #openstack-keystone03:53
jamielennoxso it's not something that user's should ever be concerned with i think03:53
morganfainbergjamielennox, hmmm.03:53
jamielennoxnot sure, but it's the best way i can think of to indicate that we are changing behaviour is to add the version= param03:53
morganfainbergjamielennox, might be the best option03:54
*** praneshp has quit IRC03:57
*** praneshp_ is now known as praneshp03:57
*** stevemar has joined #openstack-keystone04:18
*** chandan_kumar has joined #openstack-keystone04:21
*** derek_c has quit IRC04:40
*** bvandenh has joined #openstack-keystone04:57
openstackgerritA change was merged to openstack/keystone: More notification unit tests
*** topol has quit IRC05:26
*** stevemar has quit IRC05:39
*** derek_c has joined #openstack-keystone05:45
*** bvandenh has quit IRC05:48
*** bvandenh has joined #openstack-keystone05:48
openstackgerritOpenStack Proposal Bot proposed a change to openstack/keystone: Imported Translations from Transifex
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Refactor notifications
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Unversioned endpoints in service catalog
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Discovery URL querying functions
dstanekmorganfainberg: thanks06:16
dstaneki wonder why it got confused06:16
morganfainbergdstanek, np, it got lost somehow06:17
morganfainbergdidn't we have a gerrit upgrade?06:17
morganfainbergoh nope not yet06:17
morganfainbergno idea how it got confused.06:17
morganfainbergok i need to do a full review on again06:18
morganfainbergmaybe tomorrow.06:18
morganfainbergdstanek, dogpile update going in06:19
morganfainbergdstanek, means we can fix that cache thing06:19
morganfainbergand the backend property :)06:19
dstanekmorganfainberg: excellent - i think it's in one of the patches for Python 306:23
morganfainbergdstanek, yep06:23
dstaneknow that the oslo fix is in all that stuff is ready to go06:24
dstanekthat'll give us some py3 coverage06:25
morganfainbergit's a start!06:25
morganfainbergi need to go home :P06:26
morganfainbergi was going to go home much earlier but wanted to get some code reviewing done06:27
morganfainbergdid we get blinker added to the global reqs?06:29
morganfainbergdstanek, or were you waiting?06:30
morganfainberganyway. g'night06:31
dstanekmorganfainberg: i was waiting; we can talk about it tomorrow06:32
dstanekhave a good night06:32
*** morganfainberg is now known as morganfainberg_Z06:41
*** ericn has quit IRC06:53
*** rwsu has quit IRC06:57
openstackgerritJuan Antonio Osorio Robles proposed a change to openstack/keystone: No longer allow listing users by email
*** jaosorior has joined #openstack-keystone07:09
openstackgerritA change was merged to openstack/keystone: Explicit foreign key indexes.
openstackgerritA change was merged to openstack/keystone: Compatible server default value in the models.
*** praneshp_ has joined #openstack-keystone07:25
*** praneshp has quit IRC07:27
*** praneshp_ is now known as praneshp07:27
*** gokrokve_ has quit IRC07:31
*** gokrokve has joined #openstack-keystone07:32
*** gokrokve has quit IRC07:36
openstackgerritMarek Denis proposed a change to openstack/identity-api: Add ``user`` object to the mapping rules examples.
openstackgerritMarek Denis proposed a change to openstack/identity-api: Fix federation mapping rules examples.
*** praneshp has quit IRC07:46
*** jamielennox is now known as jamielennox|away07:51
*** gokrokve has joined #openstack-keystone08:02
*** gokrokve has quit IRC08:04
*** gokrokve has joined #openstack-keystone08:04
*** leseb has joined #openstack-keystone08:07
openstackgerritA change was merged to openstack/python-keystoneclient: Enhance tests for auth_token middleware
*** gokrokve has quit IRC08:08
openstackgerritA change was merged to openstack/keystone: Updated from global requirements
openstackgerritA change was merged to openstack/keystone: Fix typo of ANS1 to ASN1
openstackgerritMarek Denis proposed a change to openstack/keystone: Add detailed federation configuration docs
*** derek_c has quit IRC08:35
*** gokrokve has joined #openstack-keystone09:02
*** gokrokve has quit IRC09:07
*** zhiyan is now known as zhiyan_09:14
*** leseb has quit IRC09:26
*** andreaf has joined #openstack-keystone09:27
openstackgerritA change was merged to openstack/python-keystoneclient: Debug log when token found in revocation list
*** leseb has joined #openstack-keystone09:30
*** gokrokve has joined #openstack-keystone10:03
*** gokrokve has quit IRC10:08
openstackgerritA change was merged to openstack/keystone: Added statement for ... if ... else
*** leseb has quit IRC10:30
*** leseb has joined #openstack-keystone10:31
*** leseb has quit IRC10:35
*** waterkinfe has joined #openstack-keystone10:44
*** gokrokve has joined #openstack-keystone11:03
*** gokrokve_ has joined #openstack-keystone11:05
*** vhoward has joined #openstack-keystone11:05
*** gokrokve has quit IRC11:08
*** leseb has joined #openstack-keystone11:08
*** gokrokve_ has quit IRC11:10
*** leseb has quit IRC11:13
openstackgerritOpenStack Proposal Bot proposed a change to openstack/python-keystoneclient: Updated from global requirements
*** leseb has joined #openstack-keystone11:52
*** gokrokve has joined #openstack-keystone12:03
*** erecio has joined #openstack-keystone12:03
*** gokrokve has quit IRC12:07
*** thiagop has joined #openstack-keystone12:37
*** erecio_1 has joined #openstack-keystone12:53
*** erecio has quit IRC12:56
*** dims has quit IRC13:01
*** dims has joined #openstack-keystone13:03
*** gokrokve has joined #openstack-keystone13:03
*** lbragstad__ is now known as lbragstad13:07
*** gokrokve has quit IRC13:08
*** joesavak has joined #openstack-keystone13:10
*** diegows has joined #openstack-keystone13:10
*** waterkinfe has quit IRC13:28
*** toddnni has joined #openstack-keystone13:38
toddnniHi, I would like to ask shortly about Openstack keystone and LDAP integration. We at Cybercom took into use RDO Havana with Ldap identity backend. We are using AD so we needed to change id attribute to sAMAaccountName and I think we bumbed to #1231488. To fix it we added id attribute to identity/backends/ attribute_options_names list and it fixed the problem in our case. So the question is does anyone have up to date13:40
toddnniIt seems that the same bug is in icehouse also.13:41
*** dims has quit IRC13:41
toddnniayoung: I understood that you know something about this?13:42
*** dims has joined #openstack-keystone13:43
ayoungtoddnni, you probably need to change the case of the attribute.  It needs to match exactly13:43
openstackgerritMarek Denis proposed a change to openstack/identity-api: Add ``user`` object to the mapping rules examples.
marekddolphm: ^^ addressed your comments.13:44
*** dims has quit IRC13:53
*** topol has joined #openstack-keystone13:55
*** dims has joined #openstack-keystone13:55
*** leseb has quit IRC13:56
*** leseb has joined #openstack-keystone13:56
openstackgerritLance Bragstad proposed a change to openstack/keystone: Allow 'description' in V3 Regions to be optional
*** gokrokve has joined #openstack-keystone14:04
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token middleware hashes tokens with configurable algorithm
*** thedodd has joined #openstack-keystone14:12
*** chandan_kumar has quit IRC14:13
toddnniayoung: we confrmed that id attribute change had no effect by looking keystone ldap debug output, and the fix I made worked. So I have a fix, but I14:13
*** ayoung is now known as ayoung_meeting14:14
toddnni'm wondering if my way is the right way to fix it14:14
toddnnior if there is a fix or idea how to fix it already14:15
lbragstadjamielennox|away: we wouldn't necessarily have to need something like this using the schema would we?
lbragstadon delete that is14:26
*** stevemar has joined #openstack-keystone14:27
*** erecio_1 has quit IRC14:30
*** tomoiaga has joined #openstack-keystone14:39
tomoiagathe keystone developer docs have examples on how one can use a keystone client through python. However, this doesn't seem to work unless an auth plugin is specified (not mentioned in the docs). Furthermore, the auth param is marked as being optional, but nothing works without it. (keystone v3, examples here: )14:40
ayoung_meetingtoddnni, with you in a bit...working through something with someone else14:41
*** dims has quit IRC14:45
*** dstanek is now known as dstanek_zzz14:45
*** dims has joined #openstack-keystone14:47
*** leseb has quit IRC14:54
*** afaranha has left #openstack-keystone15:01
*** leseb has joined #openstack-keystone15:03
*** zhiyan_ is now known as zhiyan15:05
*** tomoiaga has quit IRC15:05
*** zhiyan is now known as zhiyan_15:07
*** jagee has joined #openstack-keystone15:07
openstackgerritMarek Denis proposed a change to openstack/identity-api: Add ``user`` object to the mapping rules examples.
openstackgerritMarek Denis proposed a change to openstack/identity-api: Fix federation mapping rules examples.
*** erecio_1 has joined #openstack-keystone15:11
*** stevemar has quit IRC15:14
*** gyee has joined #openstack-keystone15:17
*** dstanek_zzz is now known as dstanek15:21
mfischis the admin token in the keystone db somewhere?15:22
mfischa better question is how I can update it15:23
dstanekmfisch: it's in your keystone.conf -
mfischdstanek: so I can just change it there and restart?15:24
mfischI assumed it was more than that but maybe not15:24
*** browne has joined #openstack-keystone15:24
dstanekmfisch: i'm pretty sure that's all you have to do15:24
mfischpuppet was displeased, let me try stuff by hand15:25
mfischdstanek: well it doesn't work that way15:30
mfischdstanek: I mean more is required15:30
dstanekmfisch: are you using OS_TOKEN to specify it for the client?15:31
*** david-lyle_ has joined #openstack-keystone15:32
*** richm has joined #openstack-keystone15:32
*** erecio_1 has quit IRC15:32
mfischdstanek:SERVICE_TOKEN=ABCD SERVICE_ENDPOINT=http://localhost:35357/v2.0/ keystone user-list15:33
*** arunkant has quit IRC15:33
*** david-lyle_ is now known as david-lyle15:34
*** jagee has left #openstack-keystone15:34
*** dstanek is now known as dstanekafk15:35
*** dstanekafk is now known as dstanek15:35
dstanekmfisch: hmmm...i'll give it a try locally and see what happens15:37
dstanekmfisch: i think the SERVICE* environ vars are old style, but i'll check on that in a second15:38
openstackgerritSergey Skripnick proposed a change to openstack/keystone: Add rally job
mfischdstanek: OS_SERVICE or SERVICE* should work15:39
mfischdstanek: and neither do15:39
*** bach has joined #openstack-keystone15:40
boris-42morganfainberg_Z hi15:41
*** bach has quit IRC15:41
*** andreaf has quit IRC15:41
*** bach has joined #openstack-keystone15:41
boris-42morganfainberg_Z take a look pls at rally job for keystone15:41
boris-42morganfainberg_Z as well here infra patch
dstanekboris-42: morganfainberg_Z's probably sleeping15:42
boris-42dstanek yep but he will read this message a bit latter=)15:42
dstanekooh a rally patch, nice15:42
boris-42dstanek yep15:42
dstanekboris-42: how's that been going? i haven't been keeping up15:42
boris-42dstanek let me show sample in rally15:43
*** shakamunyi has joined #openstack-keystone15:43
boris-42dstanek cause in rally we already have this job15:43
boris-42dstanek so
boris-42dstanek there is file called rally-scenarios/your_project.yaml15:43
boris-42dstanek inside it you are specifying any amount of benchmarks with any load15:44
*** gabriel-bezerra has joined #openstack-keystone15:44
boris-42dstanek and it's run in gates by rally against cloud15:44
dstanekthe results.html is pretty neat15:44
boris-42dstanek after that you have check-rally-dsvm-rally job15:44
boris-42and yep you are getting results.html15:45
boris-42with results of all benchmarks (on top there is select )15:45
boris-42dstanek in future (quite soon) we will support plugins15:45
boris-42dstanek so you'll be able to write in keystone source tree plugins for rally15:45
dstanekboris-42: nice - i'd love to make stuff faster, but it's hard to tell what to attack with micro benchmarks15:46
boris-42dstanek by plugins I mean new benchmarks15:46
dstanekthis is going to be a huge help15:46
*** mfisch has quit IRC15:46
boris-42dstanek yep and I am happy to say that project will be fully independent from rally15:46
*** david-lyle has quit IRC15:46
boris-42dstanek so you don't need to make any patches in rally even if you would like to have another benchmark =)15:46
*** david-lyle has joined #openstack-keystone15:46
boris-42dstanek cause waiting every time code reviews is sadness15:47
*** mfisch has joined #openstack-keystone15:49
*** mfisch has quit IRC15:49
*** mfisch has joined #openstack-keystone15:49
*** marekd is now known as marekd|away15:50
*** jaosorior has quit IRC15:51
*** med_ has quit IRC15:53
*** bach has quit IRC15:53
*** medberry has joined #openstack-keystone15:53
*** dims has quit IRC15:53
dstanekmfisch: what does your pipeline look like? you probably don't have the middleware in there15:54
*** bach has joined #openstack-keystone15:55
*** dims has joined #openstack-keystone15:55
dstanekmfisch: admin_token_auth, I believe15:55
mfischdstanek: yes, I do15:56
mfischI'll change the value back and show you15:56
mfischdstanek: and it works fine, so I claim that token is stored somewhere else besides in memory15:57
*** morganfainberg_Z is now known as morganfainberg15:57
dstanekmfisch: i think it is only in the config, but you may have to do some more debugging if it doesn't work for you15:58
mfischdstanek: I'll bet you a beer in Atlanta that I'm right ;)15:58
*** erecio_1 has joined #openstack-keystone15:59
*** marcoemorais has joined #openstack-keystone16:02
dstanekmfisch: shore16:02
dstanekmfisch: i just changed my token in the config file, restarted and i was good to go16:02
mfischdstanek: well I owe you a beer in Atlanta, its not in the DB16:03
mfischdstanek: and I did too, I bet my other issue was HA related, didn't restart all nodes16:03
dstaneki used - keystone --os-token=token --os-endpoint http://localhost:35357/v2.0 endpoint-list16:03
mfischdstanek: yep, you were right, hope you like good beer16:04
*** vhoward has left #openstack-keystone16:06
*** zhiyan_ is now known as zhiyan16:08
morganfainbergbknudson, if you're interested in oslo-meeting its in #openstack-meeting-alt now (not sure if you're interested in joining it)16:09
bknudsonmorganfainberg: I thought it was in openstack-meeting16:09
morganfainbergbknudson, so did I, aparantly it changed16:10
*** ayoung_meeting is now known as ayoung16:11
dimsy there was a request to change the time and the slot was taken16:11
ayoungtoddnni, you get an answer yet?16:11
dstanekmorganfainberg: hmmm...and i was lurking in #openstack-meeting wondering if i had the time wrong16:11
*** david-lyle has quit IRC16:12
ayoungmorganfainberg, do you understand the mapping extension?  I need to test something out, and stevemar is MIA16:12
morganfainbergayoung, i can try and help.16:13
ayoungmorganfainberg, OK, so I have an apache module that, in addition to setting REMOTE_USER givres me groups and email16:13
morganfainbergayoung, though i am expecting a phone call, might become MIA myself at a moment's notice16:13
ayoungI have a Keystone setup with LDAP16:13
ayoungI want to make it do mapping and externalize everything instead16:13
*** zhiyan is now known as zhiyan_16:14
ayoungI assume I can't do that using an auth plugin yet, can I?16:14
morganfainbergayoung, i want to say the saml plugin was the example for that16:15
*** chandan_kumar has joined #openstack-keystone16:15
morganfainbergayoung, yeah looks like the saml2 plugin leverages the federation mapping for an IDP16:15
morganfainbergayoung, you could probably do something similar16:16
ayoungmorganfainberg, right.  So I need to basically munge the SAML and external plugins into a single thing...but it still needs an identity backend for groups16:16
morganfainbergthat is my understanding16:16
ayoungand then I would need 4 mapping rules:  one for REMOTE_USER->userid, REMOTE_USER->username, REMOTE_AEMAIL->email and REMOTE_GROPUS assignments?16:17
morganfainbergsec let me look at that16:18
morganfainbergthe last one is the one i'm not sure on. it sounds right16:18
ayoungmorganfainberg, I've been called to lunch-pickup-duty...we can pick this up after your phone call and my lunch16:20
*** david-lyle has joined #openstack-keystone16:20
morganfainbergsounds good, i'll be off the phone in ~1.5hrs or so16:20
*** leseb_ has joined #openstack-keystone16:23
*** leseb_ has quit IRC16:23
*** leseb has quit IRC16:26
boris-42morganfainberg hi one more time=)16:32
morganfainbergboris-42, hi there, saw your message earlier16:32
boris-42morganfainberg ok nice=)16:32
*** bach has quit IRC16:32
toddnniayoung: no I didn't get. I will get back to this on next week, because of the weekend. Thanks for your attention16:32
*** david-lyle has quit IRC16:32
*** bach has joined #openstack-keystone16:32
morganfainbergboris-42, i'll look over everything (and read the conv you and dstanek had earlier) in more detail when i get to the office in a little bit16:33
morganfainbergboris-42, thanks for getting the work done!16:33
morganfainbergboris-42, (i'll also review the infra patchset)16:33
boris-42morganfainberg after those patches only rally benchmark plugins will be missing16:33
boris-42morganfainberg but they should be quite soon as well16:34
morganfainbergboris-42, ok good to know16:35
*** david-lyle has joined #openstack-keystone16:42
*** harlowja_away is now known as harlowja16:45
*** wchrisj has joined #openstack-keystone16:46
*** wchrisj has left #openstack-keystone16:46
openstackgerritRichard Megginson proposed a change to openstack/keystone: ldap/core deleteTree not always supported
*** amcrn has joined #openstack-keystone16:54
*** chandan_kumar has quit IRC16:54
*** bach has quit IRC16:56
*** ericn has joined #openstack-keystone16:58
*** bada has quit IRC17:02
nkindergyee: I provided some responses to your comments in
nkindergyee: let me know if you have any follow-up questions on them, and I'd be happy to discuss17:06
*** openstackstatus has quit IRC17:07
*** ryanpetrello has joined #openstack-keystone17:08
ryanpetrelloanybody aware of current keystone + devstack issues related to using UUID tokens?17:08
*** openstackstatus has joined #openstack-keystone17:09
ryanpetrellospecifically, if I specify `KEYSTONE_TOKEN_FORMAT=UUID` in my localrc, I see keystone failing with something like this:
ryanpetrello`$keystone-manage pki-setup` and restarting keystone seems to fix things17:10
gyeenkinder, looking17:10
gyeeryanpetrello, that option is deprecated, you need to change the provider if you want UUID token17:11
ryanpetrelloI see17:12
gyeenkinder, I still don't understand this "'(|(objectclass=*)(objectclass=ldapsubentry))'17:12
nkindergyee: are you familiar with operational attributes?17:12
gyeeit matching objectclass=*17:12
gyeethat's the search filter17:12
nkindergyee: it's a similar concept though17:12
nkinderldapsubentry is used to crete operational entries17:12
gyeebut that's a search filter17:13
nkinderthat is, entries that are not returned unless you specify objectclass=ldapsubentry17:13
gyeenkinder, its a or ('|') filter17:13
nkinderobjectclass=* will not return an entry that has "objectclass: ldapsubentry"17:13
nkindergyee: yes, we want all entries17:13
gyeeobjectclass=* or objectclass=ldapsubentry17:13
gyeeso the second part seem redundant17:14
nkindergyee: it wouldn't work with an &17:14
gyeethat's my point, objectclass=* will return everything17:14
nkindergyee: as I stated, "objectclass=*" will not return an entry that has the "ldapsubentry" objectclass in it17:14
nkindergyee: and I'm stating that it will not17:14
nkinderthink of ldapsubentry as hidden entries17:15
nkindereven objectclass=* will not return them17:15
ryanpetrellogyee: any idea when that devstack option was deprecated?17:15
*** erecio_1 has quit IRC17:15
ryanpetrellothere’s no mention of it at ``17:15
openstackgerritLance Bragstad proposed a change to openstack/keystone: Implement validation on Assignment V3 resources
nkindergyee: this is why I say it's like operational attributes.  If you ask for all attributes to be returned with "*", you still won't get operational attributes17:17
*** erecio has joined #openstack-keystone17:17
nkindergyee: this works the same way, as "objectclass=*" won't return operational entries17:17
gyeenkinder, I see, trying to figure out why operation attributes are needed17:18
nkindergyee: ok, so the point of this patch is to do a recursive delete of a tree with children17:19
gyeeright, but we are talking about user or group trees17:19
openstackgerritLance Bragstad proposed a change to openstack/keystone: Implement validation on Assignment V3 resources
nkindergyee: how do you know someone didn't put a subentry in there?17:19
nkindergyee: keystone may not have, but what if someone did that to the ldap server directly?17:20
nkindergyee: the server would return an error in that case, as we'd be tryign to delete a non-leaf17:20
nkindergyee: we need to make sure that we delete all children first, including subentries17:21
*** raildo has joined #openstack-keystone17:21
gyeenkinder, that's fine, I would think if user want keystone to write to their LDAP, they would want to keep their hands off these subtrees17:23
nkindergyee: they should, but if they don't keystone will fail.17:23
gyeenkinder, that's fine then, keep that code17:24
nkindergyee: this is the same thing that a LDAP server will do if it supports the tree delete control17:24
nkindergyee: it's really just implementing the same logic in keystone in case a server doesn't support tree delete17:25
gyeenkinder, I am good with that17:25
nkindergyee: cool.  Just wanted to explain why it is needed.17:25
ayoungnkinder, I need a one-word "method" to describe using external-auth + mod_identity_lookup talking SSSD.17:26
ayoungI was thinking maybe sssd, but not sure that is really clear or the heart of it17:26
nkinderthat seems more confusing than sssd17:27
ayoungthe whole thing is an acronym soup17:27
nkinderis this for a config setting or something?17:27
ayoungI'm trying to remove LDAP from Keystone17:27
ayoungdirect LDAP access17:28
ayoungI spent some time with Jan this morning getting a server up and running where I can get env vars from IPA using mod_id_lookup17:28
ayoungand I think I can use the Current SAML2 code as is17:29
nkinderayoung: I know what you are trying to do, but I'm wondering where the "method" name is being used17:29
ayoungI want to keep Kerberos working, and that is sitting on "external" until I get this working17:29
ayoungI might just call is saml as that lets me use the existing plugin.  There is nothing saml about the plugin17:30
nkinderayoung: what is the name going to be for?  A plug-in name?17:31
ayoungIf I understand it correctly, I should be able to set up some mapping values for REMOTE_USER and REMOTE_GROUPS17:31
ayoungnkinder, yeah.17:31
gyeenkinder, I am just about to +2 the patch, but what about bknudson's comments?17:32
ayoungnkinder, Heh, I was the one that protested against "federated"17:32
gyeenkinder, he mentioned the reverse won't help17:32
nkindergyee: not sure, richm? ^^^17:32
richmI suppose it is theoretically possible that an ldap search may return entries that are not in child -> parent order17:33
richmhowever - every ldap delete tool I have ever seen assumes this order17:33
richmincluding openldap ldapdelete, which is arguably the most ldap compliant tool17:34
gyeeryanpetrello, I see this17:34
gyeeif [[ "$KEYSTONE_TOKEN_FORMAT" = "UUID" ]]; then17:34
gyee        iniset $KEYSTONE_CONF token provider keystone.token.providers.uuid.Provider17:34
gyee    fi17:34
richmanother way to say it - if an ldapsearch does not return entries in child->parent order, then many, many ldap tools will be broken as a result17:34
richmif keystone does run into some ldap server that does not support subtree delete by deleting the entries in the reverse order as returned by ldap search, then there are probably going to be other problems with keystone support for that ldap server17:35
ryanpetrellogyee: and that seems to be working as expected, the keystone.conf that’s generated uses the uuid provider17:36
ryanpetrellobut when keystone starts, it still fails on my aforementioned `unable to load certificate` exception17:36
gyeeryanpetrello, can you file a bug?17:37
gyeerichm, are you aware of any LDAP server which does not support subtree delete?17:38
*** morganfainberg is now known as morganfainberg_Z17:39
gyeenm, openldap17:40
richmopenldap, 389/rhds, SunDS (certain versions), not sure about apache DS, opendj17:41
ryanpetrellogyee: doing so17:41
richmgyee: I think it will be quite uncommon to find an ldap server that does support the subtree delete control, other than AD17:42
nkinderayoung: "external" seems most appropriate17:45
nkinderayoung: keystone doesn't know what is being used really, just that the webserver is providing the info from some external source17:46
ayoungnkinder, yeah.  Actually, I was going enven further and trying to avoid putting any value into "methods"17:47
ryanpetrellogyee: okay, filed:, thanks17:50
uvirtbotLaunchpad bug 1312858 in keystone "Keystone + Devstack fail when KEYSTONE_TOKEN_FORMAT=UUID" [Undecided,New]17:50
ayoung Premature end of script headers: main18:05
*** bach has joined #openstack-keystone18:06
ayoungthat sounds like the "too big service catalog problem", right?18:06
*** praneshp has joined #openstack-keystone18:10
*** morganfainberg_Z is now known as morganfainberg18:12
*** bach has quit IRC18:12
*** bach has joined #openstack-keystone18:15
*** dstanek is now known as dstanek_zzz18:15
morganfainbergayoung, ok i'm in the office now18:15
ayoungmorganfainberg, and I have managed to torque up my system.  Its an old devstack that I was updating18:16
morganfainbergayoung, ah.18:16
ayoungand I need to get the python-keystoneclient in the right state:18:16
ayoung{"error": {"message": "cms_hash_token() got an unexpected keyword argument 'mode'", "code": 400, "title": "Bad Request"}}[ayoung@ayoung530 scripts]$18:16
morganfainbergayoung, ick18:16
ayoungmorganfainberg, nah, just an old client18:16
ayoungdoes devstack pip install the client?18:16
morganfainbergayoung, it should18:17
morganfainbergbut if the version is already satisfied....18:17
*** dstanek_zzz is now known as dstanek18:19
*** gokrokve has quit IRC18:19
*** gokrokve has joined #openstack-keystone18:20
ayoungmorganfainberg, nah, I just need to git fetch and rebase on the RIGHT machine18:20
ayoungnow I get18:20
ayoung[Fri Apr 25 18:14:05.271413 2014] [:error] [pid 3907] [remote]   File "/var/www/cgi-bin/keystone/main", line 51, in <module>18:20
ayoung[Fri Apr 25 18:14:05.271501 2014] [:error] [pid 3907] [remote]     drivers = service.load_backends()18:20
ayoungwhat did we do?18:20
morganfainbergthats odd...18:21
morganfainbergwe need to exercise the mod_wsgi scripts, i think this was the fix to isolate the backend loading18:21
* morganfainberg looks18:22
*** amcrn has quit IRC18:22
morganfainbergayoung, i'll start working on getting apache test in check (perhaps just a simple tempest run that does keystone apache deploy) [non vote to start]18:22
ayoungOh yes18:22
morganfainbergwe need to exercise this stuff on each commit. i'll do that this weekend18:23
ayoungcan we dothat and LDAP in one fell poop?18:23
morganfainbergayoung, hm. sure i can do both.18:24
*** gokrokve has quit IRC18:24
morganfainbergthough i think i'd rather those be separate if we can afford it.18:24
morganfainbergbut i can propose both easily18:25
ayoungmorganfainberg, and we can run the scripts in keystoneclient/examples/scripts!18:25
morganfainbergaha! good plan!18:25
morganfainbergthat'll make everyone happier.18:25
morganfainbergexamples that are maintained... whoa.. is that allowed?18:26
morganfainbergi also want to convert keystone-all and keystone-manage to using the entry-point logic to generate the scripts instead of having separate code for it18:27
morganfainbergso all the cli code is fully in tree vs. in like /bin18:27
ayoungmorganfainberg, wrong error message:18:28
ayoung[Fri Apr 25 18:28:14.775659 2014] [:error] [pid 5075] [remote] mod_wsgi (pid=5075): Exception occurred processing WSGI script '/var/www/cgi-bin/keystone/main'.18:28
ayoung[Fri Apr 25 18:28:14.775801 2014] [:error] [pid 5075] [remote] TypeError: expected byte string object for header value, value of type unicode found18:28
morganfainbergblink wiat what?18:28
morganfainbergerm. something is wonky18:29
ayoungwonder if that is due to my recent cms change?18:30
morganfainbergthat is a very non-helpful message.18:30
morganfainbergthat looks like it.18:30
morganfainbergbut it's odd, in py27 it should still be a byte_str18:30
ayoungutf-8 vs ascii?18:31
ayoungfor encoding?18:31
morganfainbergyeah but...18:32
morganfainberga utf8 encode of ascii text should stay bytestr iirc18:32
morganfainbergmaybe not.18:32
morganfainbergi greatly dislike text vs bytestr in py2718:33
morganfainbergcan we move to py33 yet? :P18:33
ayounglets get the best of both worlds and do BOTH!18:34
ayoungand py2.618:34
morganfainbergayoung, hehe18:37
morganfainberghow far into RHEL7 are we going to need to wait to drop py27 support in OpenStack?18:37
ayoungmorganfainberg, I don't think it is RHEL7 dependent.  I think software collections means we should be able to do it now.18:38
morganfainbergayoung, does 6 support software collections well?18:39
morganfainbergi thought 6 was limited in that regard18:39
morganfainbergoh cool.18:39
morganfainbergwell, prob a cycle or two then.18:39
ayoungYeah, but for Juno it should be a no-brainer18:39
morganfainbergmy guess is K at the earliest18:40
ayoungwhat about Ubuntu?18:40
morganfainbergjuno will move that direction18:40
morganfainberghm. i think ubuntu has py33 runtimes18:40
morganfainberg12.04 and 14.04 are the only ones that really matter as of ~november18:41
morganfainbergright now 13.10 is still relevant18:41
ayoungwe just need a story about how to run on py33 Ubuntu, doesn't need to be default python18:48
ayoungbut we have this Eventlet Anchor18:48
*** praneshp has quit IRC18:49
*** gokrokve has joined #openstack-keystone18:50
morganfainbergthis is such a strange problem...18:54
morganfainbergon os x this regex '.*pot?' doesn't match the .po files18:54
morganfainbergin find18:54
morganfainbergoh sorry .*\.pot?18:54
morganfainbergbut .*\.pot* does match18:55
morganfainbergthe regex doc doesn't indicate the ? shouldn't work18:55
*** thiagop has quit IRC18:58
*** bvandenh has quit IRC19:04
ayoungmorganfainberg, so do I fix the server or do I fix the cms code?19:11
ayoungI'm guessing the server19:11
morganfainbergayoung, well.19:11
ayoungI think in the middleware that sets the token I need to say19:11
morganfainbergit depends on where the issue actually is19:11
ayoungmorganfainberg, I'm guessing that the context type encoding defaults to ascii19:12
ayoungand we need to say "utf-8"19:12
morganfainbergit might make sense to fix this in cms19:12
ayoung "Content-type:19:12
ayoungapplication/xml; charset=utf-8",19:12
morganfainbergwould someone else consuming this code be surprised by this behavior?19:12
morganfainbergah oh then19:13
morganfainbergyeah fix the middleware :)19:13
ayoungor json in our case19:13
morganfainbergas long as we don't surprise people it should be all good19:13
openstackgerritA change was merged to openstack/keystone: Fix the "search for" files for db models
ayoungmorganfainberg, It looks like Headers can't be anything but ASCII19:20
morganfainbergayoung, well then. i think that answers our question.19:20
ayoungmight need to fix this in the client19:20
morganfainbergayoung, HTTP/2.0 amidoingitrite?19:20
morganfainbergayoung, yeah :(19:20
*** ryanpetrello has left #openstack-keystone19:31
*** vhoward has joined #openstack-keystone19:32
bknudsonso auth_token gets a UUID token19:35
bknudsonand with the recent change to check all tokens against revocation list19:35
bknudsonthe server tries to get the revocation list19:35
bknudsonI mean the auth_token middleware tries to get the revocation list19:35
*** joesavak has quit IRC19:35
bknudsonbut I haven't set up the PKI certs because I'm using UUID19:36
bknudsonshould I have to set up the PKI certs if I'm using UUID?19:36
ayoungmorganfainberg, so..converting all of the calls to 'ascii'  made no difference?19:37
morganfainbergayoung, you're somehow decoding something to unicode, hmm.19:37
morganfainbergayoung, this is odd.19:37
ayounghmmm....UUID tokens should never trigger a fetch of the revocation list19:37
ayoungbknudson, bt maybe with the cached check...there is no way to tell the difference?19:38
ayoungyeah... bknudson its due to the cache check.19:38
ayoungmorganfainberg, OK, I can put a debug log statement in the wsgi file19:39
ayoungwhat should I look for?19:39
bknudsonI assume the revocation list is signed with the PKI cert so that's why we need the PKI cert19:39
morganfainbergyou probably need to look at the %r of the headers19:39
morganfainbergbefore we try and decode them19:40
ayoungactually its in
morganfainbergsee which header is ... wonky19:40
morganfainbergayoung, uhm.. do we know what line is raising that exception?19:41
ayoungmorganfainberg, nope19:41
ayoungits in the exchange between wsgi and mod_wsgi19:41
*** derek_c has joined #openstack-keystone19:42
ayoungmod_wsgi is parsing the headers it gets on the outgoing response19:42
morganfainbergoh this is an outgoing header issue?19:42
morganfainbergi was thinking incoming19:42
morganfainberghm. i wonder if this is an issue in our debug middleware19:43
morganfainbergayoung, might be the culprit here19:44
ayoungdoesn't look like it, and It might help me out....let me see19:45
ayoungmorganfainberg, that filter is not in my pipeline...but I'ma add it19:45
morganfainberghopefully that helps out19:46
morganfainbergif not i can try and duplicate you env and see if i can dig anything up here19:46
* morganfainberg is just looking at code atm vs running it19:46
morganfainberglet me know if you want me to set it all up19:46
*** harlowja is now known as harlowja_away19:49
*** shakamunyi has quit IRC19:50
*** marcoemorais has quit IRC19:51
*** browne1 has joined #openstack-keystone19:56
*** browne2 has joined #openstack-keystone20:00
*** browne has quit IRC20:00
*** browne1 has quit IRC20:00
richmtrying to debug some code with python -m pdb - n works, c works, b works, l works, etc.  But s(tep) does not work - it does a continue instead - has anyone seen this before?  How do I get step to actually step and not continue?20:01
ayoungmorganfainberg, its def the token20:02
ayoung+    for header in headers:20:02
ayoung+        LOG.debug('%s:%s ',  header[1], type(header[1]))20:03
ayoungrichm, yep20:03
ayoungwhen you run, you need to turn of eventlets threading20:03
ayoungthis Keystone?  run wiht20:03
richmnova, actually, but I figured it would apply to all openstack projects - I had a feeling it had something to do with monkey patching20:03
nkinderah, that explains why step always works for me.  I have always used --standard-threads20:03
ayoungrichm, not sure if nova support --standard-threads20:04
richmwill find out20:04
ayoungbut if they are now monkey-patching Thread, they will need it20:04
ayoungand monkeypatch of thread is needed for memcached20:04
*** mat-lowery has left #openstack-keystone20:04
ayoungrichm, you can always hack it where they call monkeypatch and modify it to not patch thread if ugly as that sounds20:05
richmno, nova does not support --standard-threads20:08
*** harlowja_away is now known as harlowja20:09
*** praneshp has joined #openstack-keystone20:09
*** marcoemorais has joined #openstack-keystone20:09
*** marcoemorais has quit IRC20:10
nkinderrichm: this might help -
*** marcoemorais has joined #openstack-keystone20:11
*** bach has quit IRC20:17
boris-42morganfainberg hey20:18
boris-42morganfainberg 100 concurrency too much for gates20:19
ayoungmorganfainberg, to add some mystery to this mystery, it only happend for V3 tokens20:20
ayoungV2 works fine, but I don't think v2 puts the token in a header20:20
morganfainbergboris-42, interesting, not surprising though, that is a single-threaded keystone (eventlet)20:21
boris-42morganfainberg so why we don't use in gates by default something more production?20:21
morganfainbergayoung, *blink* what are we doing differently with v320:21
ayoungboris-42, becasue the world is resistant to change.  I've been pushing HTTPD for two years now....20:21
ayoungmorganfainberg, the token comes back not in a header20:22
ayoungbody can be unicode, just not header20:22
morganfainbergayoung, ooooooh20:22
morganfainbergayoung, but still.20:22
boris-42I mean probably APACHE_ENABLED_SERVICES+=keystone,20:22
boris-42^ will help us20:22
nkinderayoung: is there some reason HTTPD couldn't be used in the gates?20:22
morganfainbergboris-42, it's there, there are some things we need to make sure are merged before we do that20:22
morganfainbergboris-42, i think...20:22
morganfainbergayoung, nkinder, i think we were having an issue with token size it might be ok20:23
boris-42morganfainberg for example?20:23
morganfainbergboris-42, ^ the tokens size was an issue.20:23
morganfainbergboris-42, i think we've solved that.20:23
morganfainbergboris-42, this weekend i'll be proposing the changes to make sure we use httpd at (the very least) in a check gate if not make it the default gate deploy option20:24
boris-42morganfainberg okay that will be nice20:24
morganfainbergboris-42, there is likely some minor work that needs to be done, but it should be straightforward to make that the default.20:24
nkinderok, so it's just a real bug/issue and not "thou must use eventlet" thing20:24
ayoungmorganfainberg, just got weird20:24
morganfainbergnkinder, yes. it has been that way for a bit20:24
nkindermorganfainberg: ++ on proposing the changes!20:25
ayoungrunning from HTTPD the type is shown to be unicode, but on my home machine in PyCHarm it shows as str20:25
* ayoung going to start drinking soon20:25
morganfainbergnkinder, i think i'm going to do a non-voting check for a hsort bit before we move to the default, and then ensure we still do a gate on eventlet20:25
nkinderayoung: it IS friday20:25
morganfainbergayoung, ... that is bizzare20:25
morganfainbergayoung, wait... json.loads20:25
morganfainbergayoung, that does unicode magic stuffs20:25
morganfainbergayoung, could that be getting in the way somehow?20:26
nkindermorganfainberg: are you going to be in Atlanta?20:26
morganfainbergnkinder, absolutely!20:26
*** daneyon has joined #openstack-keystone20:26
nkinderawesome!  Are all (or most) of the keystone regulars going?20:26
morganfainbergnkinder, as far as i know, yes.20:26
nkinderI'm looking forward to putting faces to names20:27
morganfainbergnkinder, even some of the folks that missed HK summit.20:27
*** daneyon has quit IRC20:27
morganfainbergnkinder, yeah it's great to meetup in person.20:27
*** daneyon has joined #openstack-keystone20:27
ayoungnkinder, "I'm looking forward to putting faces to name"  I still have no idea what you look like.20:28
nkinderayoung: I'll have to have a conversation with you before letting you know who I am... :)20:28
*** thedodd has quit IRC20:29
*** stevemar has joined #openstack-keystone20:31
*** gokrokve has quit IRC20:31
openstackgerritRichard Megginson proposed a change to openstack/keystone: better handling for empty/None ldap values
marekd|awayayoung: started googling for nkinder, but he didn't publish his photo on linkedin :P20:36
ayoungmarekd|away, nor on the internal Roster, either20:36
nkindermarekd|away: yeah, no particiular reason other than I don't have a decent/recent pic.20:37
nkinderayoung: never used twitter20:38
ayoungmorganfainberg, AHA.  Local keystone debugging not using git version of python-keystoneclient, but rather installed20:38 do I install git version in my tox environ?20:38
ayoungI've got venv activated....20:38
marekd|awayi am guessing the bigger topic for ATL sumit will be 'nkinder hunting :P20:38
*** marekd|away is now known as marekd20:39
nkinderit's like where's waldo20:39
ayoungnah...I know people that know what he looks like that will be there20:39
ayoungso unless he can swear them all to secrecy.20:39
ayoung3 people can keep a secret...if you off two of them.20:39
marekddstanek: Hello! Did you have any chance to take a look at my email concerning apache/wsgi regexes?20:40
marekdayoung: ++20:40
ayoungmorganfainberg, if I do this:  cd /opt/stakc/keystone; . .tox/py27/bin/activate ; cd /opt/stack/python-keystoneclient ;  python ./ install20:41
ayoungdo I get my git version of the client in my repo?20:41
ayounger venv?20:41
ayoungyes, yes I do!20:41
ayoungPcLAxZ5fM=:<type 'unicode'>20:42
*** joesavak has joined #openstack-keystone20:42
openstackgerritDiane Fleming proposed a change to openstack/identity-api: Update KSADM extension for Identity v2.0
ayoungWho is Diane Fleming, and how do we thank her for that?20:43
openstackgerritA change was merged to openstack/python-keystoneclient: Updated from global requirements
openstackgerritA change was merged to openstack/keystone: Add localized response test
openstackgerritA change was merged to openstack/keystone: Removed duplication with list_user_ids_for_project
openstackgerritA change was merged to openstack/keystone: Sync with oslo-incubator 74ae271
*** openstackstatus has quit IRC20:45
*** marcoemorais has quit IRC20:46
dstanekmarekd: i did see it, but i haven't had a chance to replicate and hack on it20:46
dstanekmarekd: i'll do that tonight and get back to you with an answer hopefully20:46
morganfainbergayoung, diane flemming is a doc person i think20:46
*** openstackstatus has joined #openstack-keystone20:46
marekddstanek: hey, take your time. I was just wondering if you have a quick fix for it or not :-)20:47
marekddstanek: and thanks!20:47
dstanekmarekd: i think it will likely be quick once i get it setup - i'll keep you posted20:47
*** marcoemorais has joined #openstack-keystone20:47
marekddstanek: would be cool!20:47
morganfainbergayoung, how the heck does that end up being type unicode?20:48
morganfainbergayoung, oh oh i see20:48
morganfainbergayoung, installed vs non-installed20:48
ayoungmorganfainberg, so even if I encode('ascii') it becomes unicode...need to convert that to a string20:49
boris-42morganfainberg actually good new seems like that we don't have any issues with tenants list command20:50
morganfainbergboris-42, that is good news!20:50
boris-42morganfainberg it works perfect20:50
*** openstackstatus has quit IRC20:50
*** raildo has quit IRC20:50
boris-42morganfainberg but with creating tenants something weird is happening =)20:50
*** openstackstatus has joined #openstack-keystone20:51
morganfainbergboris-42, ok so something we need to get fixed20:52
boris-42morganfainberg yep but it will be nice to get infra guys=)20:52
boris-42morganfainberg to merge patch to add rally job in keystone20:52
boris-42morganfainberg cause I am paling from rally
boris-42morganfainberg so it's impossible to test together with patch20:53
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token allow initial check of revocation list to fail
ayoungmorganfainberg, w00t!20:54
morganfainbergayoung, got it?20:54
morganfainbergayoung, stupid question, does that break py3320:55
morganfainbergayoung, str != bytes in py33 remember20:55
*** gokrokve has joined #openstack-keystone20:56
dhellmannmorganfainberg: that change for the tests loading their plugins fixed my internal packaging build, thanks!20:56
*** jsavak has joined #openstack-keystone20:56
morganfainbergdhellmann, np!20:56
*** ChanServ sets mode: +v openstackstatus20:56
*** marekd is now known as marekd|away20:56
*** rodrigods has quit IRC20:58
-openstackstatus- NOTICE: Gerrit will be unavailable for a few hours starting at 1600 UTC on Monday April 28th for an upgrade. See
*** marcoemorais has quit IRC20:59
*** joesavak has quit IRC20:59
ayoungmorganfainberg, doesn't matter21:00
ayoungthis is in server21:00
*** marcoemorais has joined #openstack-keystone21:01
*** marcoemorais has quit IRC21:02
*** marcoemorais has joined #openstack-keystone21:03
openstackgerritayoung proposed a change to openstack/keystone: Ensure token is a string
*** jaosorior has joined #openstack-keystone21:05
jaosoriorHi, how do I get Jenkins to recheck my patch?21:05
boris-42jaosorior recheck no bug21:05
jaosoriorIt didn't fail in the first run, but now one test failed, for some reason21:05
openstackgerritayoung proposed a change to openstack/keystone: Ensure token is a string
boris-42jaosorior  ^ in review type21:06
jaosoriorthanks man21:06
*** browne2 has quit IRC21:07
*** bach has joined #openstack-keystone21:08
*** bach has quit IRC21:08
*** daneyon has quit IRC21:10
*** derek_c has quit IRC21:11
*** topol has quit IRC21:14
*** erecio has quit IRC21:15
*** thedodd has joined #openstack-keystone21:19
*** derek_c has joined #openstack-keystone21:30
*** dstanek is now known as dstanek_zzz21:45
*** browne has joined #openstack-keystone21:51
*** browne has quit IRC21:51
*** browne has joined #openstack-keystone21:52
*** amcrn has joined #openstack-keystone21:53
*** bach has joined #openstack-keystone21:53
openstackgerritBrant Knudson proposed a change to openstack/python-keystoneclient: auth_token allow initial check of revocation list to fail
*** browne1 has joined #openstack-keystone22:02
*** jsavak has quit IRC22:03
*** browne has quit IRC22:05
*** thedodd has quit IRC22:31
*** gokrokve has quit IRC22:33
*** david-lyle has quit IRC22:36
*** derek_c has quit IRC22:37
*** daneyon has joined #openstack-keystone22:37
*** gokrokve has joined #openstack-keystone22:53
*** marcoemorais has quit IRC22:58
*** marcoemorais has joined #openstack-keystone23:00
*** bach has quit IRC23:06
*** jaosorior has quit IRC23:11
*** derek_c has joined #openstack-keystone23:12
*** bach has joined #openstack-keystone23:13
*** daneyon has quit IRC23:13
*** gokrokve_ has joined #openstack-keystone23:15
*** gokrokve has quit IRC23:18
*** bach has quit IRC23:23
*** leseb has joined #openstack-keystone23:24
openstackgerritayoung proposed a change to openstack/keystone: Ensure token is a string
*** dims has quit IRC23:39
morganfainbergayoung, i'm not sure mod_wsgi is throwing that error. i'm curious it looks awefully pythonic for an apache error23:40
morganfainbergayoung, not saying your fix doesn't solve it, but curious where it's actually coming from23:40
ayoungmorganfainberg, its def mod_wsgi23:40
ayoungthere are two log files23:40
ayoungone is /var/log/httpd/error_log23:40
ayoungthe other is /var/log/keystone/keystone.log23:40
ayoungit comes from the first23:40
morganfainbergi mean the TypeError and the fact it's looking for a byte_string23:41
morganfainbergmod_wsgi isn't mod_python.23:41
ayoungI realize23:41
ayoungbut I am guessing the wsgi marshalls unicode differently than strings23:41
morganfainbergmaybe.. might also be an artifact in deploy23:42
ayoungand mod_wsgi can tell the difference23:42
morganfainbergi think it's probably hiding below the level we interface in keystone. i'm just curious about the error. thats all23:42
ayoung If the error were in Keystone I would see a trace in /var/log/keystone/keystone.log  but that is just fine.  No this is it.23:43
morganfainbergah right23:43
morganfainbergexception would be trapped earlier right23:44
*** dims has joined #openstack-keystone23:51
*** shakamunyi has joined #openstack-keystone23:55

Generated by 2.14.0 by Marius Gedminas - find it at!