Friday, 2014-03-21

*** rwsu has quit IRC00:18
*** joesavak has quit IRC00:26
*** browne has joined #openstack-keystone00:34
*** browne has left #openstack-keystone00:35
*** henrynash has quit IRC00:37
morganfainbergjamielennox, RC is targeted for the 27th iirc00:48
jamielennoxmorganfainberg: yea, i'll fix it anyway the comment is minor00:48
jamielennoxi just heard talk of a string freeze and it would violate that00:48
jamielennoxunfortunately i got so much gerrit email that that -1 got lost, and when i look at my reviews overview page it shows +2 from dolph so i never saw the -100:49
*** wchrisj has quit IRC00:50
*** devlaps has quit IRC00:51
jamielennoxhowever just figured out: https://review.openstack.org/#/q/owner:jamielennox+%28CodeReview-1+OR+CodeReview-2%29+status:open,n,z so now i need to put that somewhere i click often00:51
*** david-lyle has joined #openstack-keystone00:54
*** stevemar has joined #openstack-keystone00:57
*** marcoemorais has quit IRC01:04
openstackgerritJamie Lennox proposed a change to openstack/keystone: Change the default version discovery URLs  https://review.openstack.org/7806801:04
*** wchrisj has joined #openstack-keystone01:05
*** jamielennox has quit IRC01:11
*** ayoung has quit IRC01:16
* gtt116 is away: 我很忙01:20
*** jamielennox has joined #openstack-keystone01:22
*** ayoung has joined #openstack-keystone01:31
*** stevemar has quit IRC01:35
*** ayoung has quit IRC01:41
*** wchrisj has quit IRC01:54
*** gokrokve has joined #openstack-keystone01:55
*** ayoung has joined #openstack-keystone01:57
*** amcrn has quit IRC02:02
*** wchrisj has joined #openstack-keystone02:04
*** wchrisj has quit IRC02:12
*** stevemar has joined #openstack-keystone02:20
*** daneyon has joined #openstack-keystone02:26
*** mberlin1 has joined #openstack-keystone02:54
*** mberlin has quit IRC02:56
openstackgerritJamie Lennox proposed a change to openstack/identity-api: Introduce a path for accessing the current user  https://review.openstack.org/8197303:10
openstackgerritJamie Lennox proposed a change to openstack/identity-api: Add service name to service catalog  https://review.openstack.org/8197503:29
openstackgerritJamie Lennox proposed a change to openstack/identity-api: Allow scoping endpoints to a project or domain  https://review.openstack.org/8197703:47
*** zhiyan_ is now known as zhiyan03:50
openstackgerritJamie Lennox proposed a change to openstack/identity-api: Add service name to service catalog  https://review.openstack.org/8197503:55
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Add request/access token and consumer support for keystoneclient  https://review.openstack.org/8198003:56
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Authenticate via oauth  https://review.openstack.org/8198104:01
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Add example script for oauth1 functions  https://review.openstack.org/8019304:05
stevemarjamielennox, i still need to keep some of oauth logic in _factory right?04:16
stevemarjamielennox, just dump the authConstructor and authMethod into it's own file ?04:16
jamielennoxstevemar: you shouldn't need factory at all04:21
jamielennoxthat's for the older clients and i'm intending to remove it04:21
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Authenticate via oauth  https://review.openstack.org/8198104:30
stevemarjamielennox, cool, done04:30
*** wchrisj has joined #openstack-keystone04:30
openstackgerritSteve Martinelli proposed a change to openstack/python-keystoneclient: Add example script for oauth1 functions  https://review.openstack.org/8019304:32
*** stevemar has quit IRC04:38
*** morganfainberg is now known as morganfainberg_Z04:39
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Remove _factory methods from auth plugins  https://review.openstack.org/8198504:41
*** morganfainberg_Z is now known as morganfainberg04:44
morganfainbergwheee04:45
jamielennoxmorganfainberg: quick nap at the keyboard there04:45
morganfainbergnah, system crashed04:45
morganfainbergand in the process i also lost my backup drive04:45
morganfainberglooks like encrypted backups bit me this time04:46
morganfainberg=/04:46
jamielennoxah, i was coming up with cool scenarios about how maybe your irc is hooked up to a kinect tracker that knows when you passed out04:46
jamielennoxouch04:46
jamielennoxgone?04:46
morganfainbergjamielennox, unrecoverable partition04:46
morganfainbergjamielennox, this just solidifies i need to fix my @home backups04:46
morganfainbergjamielennox, so i have this machine backed up in 2 places.04:46
jamielennoxin theory i definetly understand encrypted backups - in practice i'm way more likely to loose the keys than i am to care about someone else finding my photos04:47
jamielennoxobvoiusly that depends on what you're backing up04:47
morganfainbergi use bad encryption keys04:47
morganfainbergwell bad encryption passphrases04:48
jamielennoxmorganfainberg: heh, the equivalent of hiding your house keys behind a rock04:48
morganfainbergyeah yeah04:49
jamielennoxbut we all still do it04:49
morganfainbergjamielennox, i think i'm going to have bandwidth to do ksc reviews next week04:50
jamielennoxmorganfainberg: sweet!04:50
morganfainbergjamielennox, and yours are the top of my list over there04:50
morganfainbergi'm still trying to duplicate an RC bug.04:50
morganfainbergbut ... eh04:50
jamielennoxi've been trying to find ways of increasing my bullying04:50
morganfainbergthere is some really odd edge case going on here https://bugs.launchpad.net/keystone/+bug/129486204:51
uvirtbotLaunchpad bug 1294862 in keystone "Token expiration time with memcache->kvs->dogpile is wrong" [Medium,New]04:51
morganfainbergbut i'm not finding it easy to duplicate.04:51
morganfainbergi'm getting the feeling we have some clock skew issues there. or a system didn't DST jump04:51
morganfainbergso i need to sync up w/ the reporter, other than that...04:52
jamielennoxnah DST is too obvious04:52
morganfainbergyeah04:52
morganfainbergdamn it.04:52
jamielennoxbut it makes sense with him being UTC+1 that the tokens are expired immediately with a 1 hour timeout04:53
morganfainbergyeah04:53
jamielennoxit just means that something in there is not correctly doing a utc calculation04:53
morganfainbergbut even if the machine didn't jump DST it shouldn't break our setup04:53
jamielennoxno, UTC shouldn't react to DST (in theory)04:53
morganfainbergwe coerce everything to naive (non-offset utc time) values04:53
morganfainbergunless the system thinks it's an hour offf.04:54
morganfainbergand he's getting token data back, so it tells me it isn't memcache being dumb04:54
jamielennoxis this an auth_token failure? where is he seeing the problem?04:54
jamielennoxoh, purely keystone token validatoin04:54
morganfainbergwell04:54
morganfainbergit seems like keystone is working04:55
morganfainbergbut other APIs aren't04:55
morganfainbergso, auth_token04:55
jamielennoxo - right04:55
morganfainbergPKI tokens04:55
jamielennoxmeh - it has been one of those fridays04:55
morganfainbergyeah.04:55
jamielennoxinteresting, we had a bug like that fixed in KSC a while ago04:56
jamielennoxbut it was definitely fixed04:56
morganfainbergyeah04:56
morganfainbergthis is an odd one04:56
jamielennoxso first guess is clock skew between client and server04:56
* morganfainberg taps foot and waits for bug 1295261 to pass gate04:57
uvirtbotLaunchpad bug 1295261 in keystone "test_v3_os_revoke.OSRevokeTests: invalid event issued_before time; Too early" [Critical,In progress] https://launchpad.net/bugs/129526104:57
morganfainbergnothing else seems to be passing without that.04:57
morganfainbergi'm amazed a test ran fast enough to fall between ticks04:58
jamielennoxso i can only guess at that bug, but i would like him to check his clocks first05:00
jamielennoxhe says that expiry is UTC+conf.token.expiration and that is correct05:00
morganfainbergjamielennox, it would make sense05:01
morganfainbergso yeah tomorrow going to have him chase down the last few things05:01
jamielennoxbut if that was an issue in auth_token we would know about it by now05:01
morganfainbergwell, perhaps05:01
morganfainbergwe just moved default token expiry to 360005:01
morganfainbergthis release05:01
morganfainbergand most people don't run UTC+105:02
jamielennoxyea05:02
morganfainbergi'm willing to go the extra distance in this case because of this change.05:02
morganfainbergit could really bite us otherwise05:02
jamielennoxby most people you mean the US right05:02
morganfainbergno i mean, a minority of servers i know run TZ aware05:03
morganfainbergmost people run servers UTC that i know these days05:03
jamielennoxreally? didn't know that05:03
morganfainbergexcept on of my recent companies it's been the standard we've used a lot of places05:04
jamielennoxit would definetly solve some problems05:04
morganfainbergit just doesn't make sense to fight DST stuff twice a year05:04
morganfainbergit's why we go so far out of our way to convert everything to UTC in keystone05:04
morganfainbergactuallty.. all over openstack05:04
jamielennoxalright, commenting on that bug05:05
morganfainbergbut it's much easier to just run servers in UTC05:05
morganfainbergjamielennox, sure thing :)05:05
morganfainbergjamielennox, and forgive his bad bug report, i talked to him before on irc05:05
morganfainbergjamielennox, went over a bunch of stuff and it seems at least based upon the chat a legit edge case05:06
jamielennoxmorganfainberg: oh, ok - so you did do a clock check on his clients and servers05:07
morganfainbergjamielennox, i think i did05:07
morganfainbergjamielennox, feel free to ask05:08
jamielennoxdone, but yea i don't think it's a dogpile issue05:11
jamielennoxunless you're not serializing the time back and forth correctly05:11
*** gokrokve has quit IRC05:12
*** gokrokve has joined #openstack-keystone05:12
jamielennoxwhich might be possible....05:14
jamielennoxmorganfainberg: is it a problem that you use isotime() rather than strtime()?05:14
morganfainbergtimeutiles.isotime?05:14
jamielennoxyea05:15
morganfainbergtimeutils.isotime is strftime05:15
morganfainbergjust makes it ISO860105:15
morganfainberghttps://github.com/openstack/keystone/blob/master/keystone/openstack/common/timeutils.py#L3805:15
jamielennoxjust looking at line 4105:16
morganfainberghmm05:16
morganfainbergi think we'd have bigger issues if that didn't work05:16
jamielennoxjust looking at sql - it lets sqlalchemy do the conversion05:17
morganfainbergSQLAlchemy doesn't convert it to a string05:17
morganfainbergwell... not exactly05:17
*** gokrokve has quit IRC05:17
morganfainbergi guess maybe it's VARCHAR under the hood in sqlite05:18
morganfainbergjamielennox, hmm.05:18
morganfainbergjamielennox, ok the difference between isotime and strtime, iostime will make UTC tz aware explicitly05:19
morganfainbergwell all time tz aware05:19
morganfainbergi think that is much safer05:19
morganfainbergit means we don't worry about one system being UTC and another being UTC+105:20
morganfainbergstrtime will omit TZ info05:20
jamielennoxmorganfainberg: it shouldn't matter though, what is returned from create_token is a datetime object05:20
morganfainbergjamielennox, but we json serialize05:20
morganfainbergjamielennox, PKI tokens are not transmitting datetime objects05:21
morganfainbergnor are validate calls05:21
jamielennoxyea, but i mean that the serialization won't be different between kvs/sql05:21
morganfainbergit would be in the case of auth_token05:21
jamielennoxi was thinking there was some string transition differences there05:21
morganfainbergoh oh yeah05:21
morganfainbergno05:21
morganfainbergshould be 100% the same.05:21
jamielennoxyea, even if the stored form is different from SQL so long as the in and out are correct it won't matter05:22
morganfainbergand crapola. i lost my vagrant file =/05:22
morganfainbergand we already know he's using memcache05:22
morganfainbergso, i'm not worried about the SQL server doing something strange05:22
morganfainberg(not that i really was in either case)05:22
jamielennoxget_token in kvs *may* be different thogh05:23
morganfainberglooking now05:23
morganfainbergbut i think i use the native datetiem object05:23
morganfainbergwhich in the case of memcache just gets pickeled05:23
jamielennoxyou don't really do anything - you just return the ref05:23
jamielennoxline 75 gets returned05:24
morganfainbergyeah then it's native datetime.datetime()05:24
morganfainbergif that didn't work, we'd have larger issues05:24
jamielennoxwhy?05:24
jamielennoxi'm sure you're right just following through05:24
morganfainbergjamielennox, why serialize to str just to deserialize later?05:24
morganfainbergi mean it wouldn't be hard to do it.05:25
jamielennoxno thats ok, i see what i was missing there05:25
jamielennoxyou are letting the dogpile driver do the serialization05:25
morganfainbergyeah.05:25
jamielennoxi saw the expires_str = and thought it was you05:26
morganfainbergjamielennox, yeah expires_str is used for the index only05:27
jamielennoxok, was trying to figure that out cause i can't see where it's used again05:27
morganfainbergif you look at the user_index05:27
morganfainbergthats the relevant part05:27
jamielennoxbut again all this doesn't matter because its a PKI token that won't ever try to retrieve this05:27
morganfainbergexactly05:28
jamielennoxalso there is way to much replicatoin between the drivers that should be done at a higher leve05:28
jamielennoxl05:28
morganfainbergjamielennox, if we can get revoke_api up to snuff, i want these to go away05:28
morganfainbergtargeting L for removal of token persistence05:28
morganfainbergK being default to ephermeral tokens05:29
jamielennoxoh right, yea i was thinking it coudl be done for K05:29
morganfainbergJuno optionally use ephermeral tokens05:29
morganfainbergi wish :(05:29
morganfainbergbut that ship has sailed05:29
morganfainberg*eyes RC barreling down on us*05:29
*** gyee has quit IRC05:30
*** gokrokve has joined #openstack-keystone05:43
*** gokrokve has quit IRC05:56
*** wchrisj has quit IRC06:02
*** gokrokve has joined #openstack-keystone06:04
openstackgerritJenkins proposed a change to openstack/keystone: Imported Translations from Transifex  https://review.openstack.org/7852506:07
*** gokrokve has quit IRC06:10
gtt116morganfainberg, around?06:10
morganfainberggtt116, hi06:11
gtt116morganfainberg, remeber the bug which lots of token can hit the memcache limit?06:12
morganfainberggtt116, yes06:12
gtt116morganfainberg, it is here https://bugs.launchpad.net/bugs/125112306:12
uvirtbotLaunchpad bug 1251123 in keystone/havana "_update_user_list_with_cas causes significant overhead (when using memcached as token store backend)" [Critical,Fix released]06:12
morganfainberggtt116, yep, did you hit the memcache page size limit?06:13
gtt116morganfainberg, nope, this time I hit revocation-list hit memcache item limit06:13
morganfainberggtt116, it's an unfortunate limit of that driver :(06:14
gtt116morganfainberg, from google, memcache item limit is 1MB, right?06:14
morganfainberggtt116, it's why we're looking to deprecate it in lieu of using ephemeral tokens (no stable storage needed)06:14
morganfainberggtt116, by default06:14
gtt116morganfainberg, so I'm thinking about using more keys to store these tokens06:15
gtt116morganfainberg, like revocation_key_0, revocation_key_1 ....06:15
morganfainberggtt116, you can typically increase at start time of memcache, but it increases the slab for all pages (e.g. an element takes more ram)06:16
gtt116morganfainberg, when hit the limit, we increase the key id06:16
morganfainberggtt116, that was one solution i was working on at one point06:16
morganfainberggtt116, but the detection and use of overflow pages was exceptionally complex to do cleanly06:17
gtt116morganfainberg, yep, but if keystone can tolerate the limit sounds good06:17
morganfainberggtt116, ideally i want to move to ephemeral tokens (no need to store them or index them)06:17
morganfainberggtt116, but that wont be for icehouse.06:17
gtt116morganfainberg, so when that finished, the usage of token will change?06:18
morganfainberggtt116, when that is finished you wont need to use SQL or memcache or anything to store tokens06:18
gtt116morganfainberg, cool06:19
morganfainberggtt116, yeah :)06:19
morganfainberggtt116, the feature should be ready for Juno for sure06:19
gtt116morganfainberg, and the user will notice this change?06:19
morganfainberggtt116, the deployer will need to make some config changed, but it should have no impact on the user06:19
morganfainberggtt116, and it'll remove all these issues we have w/ the memcache backend (since it wont be used)06:19
gtt116morganfainberg, lol06:20
gtt116morganfainberg, but the problem is in I, even in havana06:21
gtt116morganfainberg, I think a quick fix may help06:22
morganfainberggtt116, honestly, i recommend no one use memcache for tokens06:22
gtt116morganfainberg, prefer mysql?06:22
morganfainberggtt116, it's not a good storage, if you restart the memcache server you lose your revocation list and if that happens any PKI tokens are now valid again until they expire06:23
morganfainberggtt116, yeah + caching ([cache] section in the keystone.conf) so it still uses memcache.06:23
morganfainberggtt116, you need to flush expired tokens out semi-regularly, but that iseems to have less issues overall06:23
gtt116morganfainberg, yeah, that was the reason why we now using memcache06:24
gtt116morganfainberg, another reason is update the password is really show .. =(06:25
morganfainberggtt116, because of invalidating the tokens.06:25
gtt116morganfainberg, yep06:25
morganfainberggtt116, are you making efforts to re-use tokens?06:26
morganfainberggtt116, or new token for every action?06:26
gtt116morganfainberg, in action, no06:26
gtt116morganfainberg, our users alway create new token if they like06:27
morganfainberggtt116, hm.. oh can't stop silly users from doing silly things06:27
morganfainberggtt116, we had (my company) a customer churning ~million tokens a day06:27
gtt116morganfainberg, excatly! that was the problem06:27
morganfainbergor was it 250k06:27
morganfainbergsomething absurd06:27
gtt116morganfainberg, =(06:28
morganfainberggtt116 are you running keystone under httpd?06:28
morganfainbergor just a single keystone?06:28
gtt116morganfainberg, just a single keystone06:28
morganfainberggtt116, if you do a deployment under mod_wsgi it should help a lot overall06:28
gtt116morganfainberg, but we have two keystone servers behind a haproxy06:29
morganfainberggtt116, ah thats another way to solve it06:29
gtt116morganfainberg, but when keystone eating 100% CPU, our system administrator go crazy..06:30
morganfainberggtt116, hopefully you're not having that happen as much now.06:30
openstackgerritA change was merged to openstack/keystone: explicitly import gettext function  https://review.openstack.org/8190306:30
gtt116morganfainberg, BTW are you using mysql backend?06:30
morganfainberggtt116, yes06:30
morganfainberggtt116 we use mysql backend.06:30
morganfainberggtt116, and we flush un-used tokens once per hour06:31
morganfainbergerm, expired tokens06:31
gtt116morganfainberg, emm..  by cron?06:31
morganfainbergyeah06:31
morganfainbergmaybe we moved it to 15 mins06:31
morganfainbergbut we did something via cron06:31
morganfainbergi'd need to look at the specific interval06:31
morganfainberglet me check on something for you really quickly i might have a solution that could help06:31
gtt116morganfainberg, ok, thanks very much!06:32
morganfainbergbut i want to check before making the recommendation06:32
gtt116morganfainberg, =)06:33
*** daneyon has quit IRC06:34
*** rwsu has joined #openstack-keystone06:35
morganfainbergneed to rebuild my vagrant vm first06:35
morganfainberggtt116, i think we can make some of the revocation logic better (and backport that)06:45
morganfainberggtt116, running my test now.06:45
gtt116morganfainberg, this is a good news06:45
gtt116morganfainberg, compressing the token object?06:47
morganfainberggtt116, nope using a different kvs backend06:47
morganfainberggtt116 instead of using memcache06:47
gtt116morganfainberg, emm.. but we have two keystone servers06:48
morganfainberggtt116, the new KVS system supports more than memcache :)06:48
morganfainbergwould work the same06:48
gtt116morganfainberg, you mean the KVS logic is better than raw memcache backend? right?06:49
morganfainberggtt116, KVS system is based on dogpile.cache06:49
morganfainbergso you can use a number of options such as memcache, redis, (in icehouse mongodb)06:50
gtt116morganfainberg, havana already has KVS system?06:50
morganfainberggtt116, oooh no06:51
morganfainberggtt116, doh!06:51
morganfainberggtt116, sorry :(06:51
gtt116morganfainberg, oh! that is great, let me do some investigation06:51
morganfainberggtt116, KVS is new in icehouse06:51
morganfainberggtt116, but i did just confirm i have a list w/ 100k tokens in it06:52
morganfainberggtt116, it's not speedy, but it doesn't have the same memcache limit issue06:52
gtt116morganfainberg, is it easy backport to havana?06:52
morganfainberggtt116, i'll mull over how to make this better, but honestly it's a tough nut to crack06:52
gtt116morganfainberg, I means in our private repo06:52
morganfainberggtt116 the KVS system? no.06:52
morganfainberggtt116 it is a large restructure of a number of things06:53
openstackgerritA change was merged to openstack/keystone: Comparisons should account for instantaneous test execution  https://review.openstack.org/8187906:53
gtt116morganfainberg, =<06:53
morganfainberggtt116, we are looking at supporting compressed tokens (again juno timeline)06:53
morganfainberggtt116, i'll see if i can figure out something interesting to help you out. might take me a bit to mull on this.06:54
morganfainberggtt116, not sure how much using multiple pages will help, eventually you'll spend a lot of time collecting the pages and putting them into memory06:55
morganfainberggtt116, the real long term solution will be to stop storing tokens06:55
gtt116morganfainberg, yep, yep06:55
gtt116morganfainberg, please let me know if I can do something06:56
morganfainberggtt116, will do.06:56
gtt116morganfainberg, thank you again =)06:56
morganfainberggtt116, sure thing. :)06:56
morganfainberggtt116, i just wish i could say Icehouse had all the fixes... though if we're lucky the ephemeral token work will be easy for you to backport06:57
morganfainberggtt116, to icehouse that is.06:57
morganfainberggtt116, but by then hopefiully you'll be upgraded :)06:57
gtt116morganfainberg, yeah, if it is indeed, we have to upgrade to IceHouse.06:58
morganfainberggtt116, anyway, it's late here.07:01
morganfainberggtt116, i need to go.07:01
gtt116morganfainberg, ok bye :)07:01
gtt116morganfainberg, good night07:01
morganfainbergjust as a quick check, i'm seeing no problem w/ 448000 bytes in a single redis key07:02
morganfainbergerm07:03
morganfainberg~680k07:04
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Don't use a connection pool unless provided  https://review.openstack.org/8200707:05
*** gokrokve has joined #openstack-keystone07:06
morganfainbergyeah redis scales better07:06
morganfainbergup to 2MB in a single key and it def is slowing down, but still is better07:06
morganfainbergthis would be a cpu grind.07:07
morganfainberganyway07:07
morganfainbergnight07:07
*** gokrokve has quit IRC07:10
*** jamielennox is now known as jamielennox|away07:12
*** henrynash has joined #openstack-keystone07:39
openstackgerritA change was merged to openstack/keystone: expires_at should be in a tuple not turned into one  https://review.openstack.org/8189407:42
openstackgerritA change was merged to openstack/keystone: Filter LDAP dumb member when listing role assignments  https://review.openstack.org/7393507:42
*** henrynash has quit IRC07:44
*** gokrokve has joined #openstack-keystone08:05
*** flaper87|afk is now known as flaper8708:07
*** gokrokve has quit IRC08:09
openstackgerritA change was merged to openstack/keystone: Make domain_id immutable by default  https://review.openstack.org/8149908:22
*** gokrokve has joined #openstack-keystone09:05
openstackgerritwanghong proposed a change to openstack/python-keystoneclient: use v3 api to get certificates  https://review.openstack.org/8203209:06
*** bvandenh has quit IRC09:08
*** gokrokve has quit IRC09:10
*** bvandenh has joined #openstack-keystone09:13
*** leseb has joined #openstack-keystone09:13
*** henrynash has joined #openstack-keystone09:17
*** andreaf has joined #openstack-keystone09:18
*** YorikSar_ has joined #openstack-keystone09:29
*** YorikSar has quit IRC09:32
*** morganfainberg is now known as morganfainberg_Z09:48
*** saju_m has joined #openstack-keystone09:55
*** david-lyle has quit IRC10:05
*** gokrokve has joined #openstack-keystone10:06
*** gokrokve has quit IRC10:10
*** YorikSar_ has quit IRC10:30
*** YorikSar has joined #openstack-keystone10:31
*** fabiomorais has quit IRC11:03
*** gokrokve has joined #openstack-keystone11:07
*** gokrokve has quit IRC11:11
*** saju_m has quit IRC11:22
*** leseb has quit IRC11:24
*** leseb has joined #openstack-keystone11:25
*** leseb has quit IRC11:29
*** saju_m has joined #openstack-keystone11:38
*** gokrokve has joined #openstack-keystone12:07
*** gokrokve has quit IRC12:12
*** leseb has joined #openstack-keystone12:25
*** leseb has quit IRC12:29
dstanekdolphm: i've been slow to approve reviews and are not directly bugs targeted at RC1 - is it OK to +A this one https://review.openstack.org/#/c/81720/?12:38
*** dims has quit IRC12:41
*** jagee_ has joined #openstack-keystone12:42
*** dims has joined #openstack-keystone12:48
*** saju_m has quit IRC12:57
dolphmdstanek: that one seems fine, but with the gate failure in the patch it's dependent on, i'm not sure it will/should land?12:59
dolphmdstanek: it looks like a transient failure in https://review.openstack.org/#/c/65870/ - if so, we definitely shouldn't be introducing that risk right nwo13:00
dolphmdstanek: unless something else landed 12:30am and 4:38am today that would have caused that...13:01
dstanekdolphm: https://review.openstack.org/73935 maybe13:01
dolphmdstanek: hmm... definitely suspect13:03
dstaneklooking at the code I can't see how it would cause that failure, but the title make me wonder13:03
ayoungdstanek, I think that "instantiate once and only once" is the correct semantics for managers.  It was not when they were origianlly written.13:04
dstanekayoung: i mostly agree, but in tests i'd really like to create them without DI13:05
dolphmdstanek: i've run morgan's patch rebased onto master a few times, and haven't been able to repro the failure in jenkins (yet)13:05
ayoungdstanek, I don't think that is practicable for all tests.  It should be the case for simple unit tests though13:05
ayoungdstanek, in fact, that might be the litmus test for "uni tests"  if the manager can be directly instatiated13:06
dstanekdolphm: that's the problem with parallel anything :-(13:06
dolphmdstanek: yeah... hence my paranoia landing it now :(13:06
ayoungdumb member only gets tested against fake LDAP,  would highly doubt that would cause an intermittent13:07
dolphmdstanek: i'm just running tox -e py27 test_backend_ldap though13:07
*** saju_m has joined #openstack-keystone13:07
dolphmayoung: it looks like there's a race to me though13:07
ayoungwhere?13:07
dolphmayoung: http://logs.openstack.org/70/65870/8/check/gate-keystone-python27/60bbe04/console.html13:07
dolphmshit13:08
dolphmrepro'd on sixth run13:08
*** gokrokve has joined #openstack-keystone13:08
ayoung "keystone/tests/test_backend_ldap.py", line 1389, in setUp13:09
ayoungdolphm, the fact that the cert stuff is mixed in with the stack trace indicates that we don't do multithreaded logging well13:10
dolphmayoung: ++13:11
dolphmayoung: dstanek: -2'ing the parallel patch until juno :(13:11
dstanekdolphm: that's the right thing to do13:11
ayoung++13:12
dolphmdstanek: i imagine the subsequent patch could be rebased onto master itself?13:12
dstanekwe should rework it to enable running in parallel so that more of us can start using parallel tests daily an look for these issues13:12
dolphmdstanek: the one you wanted to approve13:12
ayoungdolphm, was the stack trace in your last repro in the same place?13:12
dolphmdstanek: i.e. land it now with parallel disabled by default?13:12
*** gokrokve has quit IRC13:13
ayoungdolphm, dstanek yeah, disabled by default13:13
dstanekdolphm: right13:13
dolphmayoung: this is my local log http://paste.openstack.org/raw/73993/13:14
ayoungdolphm, same point13:15
ayoung  File "keystone/common/ldap/core.py", line 240, in get_connection13:15
ayoung    chase_referrals=self.chase_referrals)  is triggering13:15
ayoungUnexpectedMethodCallError: Unexpected method call FakeLdap.__call__('fake://memory', 0, alias_dereferencing=None, chase_referrals=None, tls_cacertdir=None, tls_cacertfile=None, tls_req_cert=2, use_tls=False) -> None13:15
ayoungdolphm, github is not responding to my web request right now, so I can't post the link,  but to me it looks like that is a call to13:16
ayoung            user = self.LDAP_USER13:16
ayoungthat is what I have on line 240...don't think I have any local changes to that patch that won't match master13:16
ayoungdolphm, can you confirm that your line 240 is the same?13:23
*** joesavak has joined #openstack-keystone13:24
ayoungah, no13:24
ayoungjust got a rebase through13:24
openstackgerritDolph Mathews proposed a change to openstack/keystone: update sample conf  https://review.openstack.org/8209013:24
ayoungdolphm, its two lines up...13:24
*** bknudson has joined #openstack-keystone13:25
*** leseb has joined #openstack-keystone13:26
openstackgerritDolph Mathews proposed a change to openstack/keystone: update sample conf  https://review.openstack.org/8209013:26
ayoungdolphm, are you autogenerating those changes?13:26
dolphmayoung: this one - yes. i was having trouble with the config generator before13:27
dolphmayoung: also http://git.openstack.org/cgit/openstack/keystone/tree/keystone13:27
dolphmayoung: instead of github ^13:27
ayoungdolphm, excellent.13:27
dolphmayoung: this patch fixes my own stupid when i was having trouble with generate_sample_conf.sh https://review.openstack.org/#/c/82090/13:28
*** leseb has quit IRC13:30
ayoungdolphm, so the above problem doesn't happen everytime?  Just 1 in 6?13:42
ayoungand just with parallel?13:43
dolphmayoung: i believe so13:43
*** vhoward has joined #openstack-keystone13:43
ayoungdolphm, my understanging then, is that setup is being called on a random test, but the backend for that test has been set to LDAP instead of whatever it was expecting,  and with mock in there.  Mock is expecting one call, and getting another.  This means that the parallel test runs are happening in the same process, and I think that is a mistake13:45
dstaneki've tried at least 6 run without parallel and haven't reprod yet13:45
*** marekd|away is now known as marekd13:52
dolphmanyone know if yuriy has an update in the works for this? https://review.openstack.org/#/c/81235/13:59
dolphmwondering if i should post a revision14:00
*** wchrisj has joined #openstack-keystone14:00
ayoungdolphm, I've got one, I'm tox ing now14:00
dolphmayoung: awesome14:00
dolphmYorikSar: didn't realize you were in the channel! yu<tab> did not produce the success i was expecting :P14:01
openstackgerritBrant Knudson proposed a change to openstack/keystone: Cleanup config.py  https://review.openstack.org/8167114:03
openstackgerritBrant Knudson proposed a change to openstack/keystone: Clean up config help text  https://review.openstack.org/7849714:03
*** leseb has joined #openstack-keystone14:04
dstaneki see no luck yesterday on the token expiration issue - morganfainberg_Z couldn't reproduce on devstack?14:06
*** gokrokve has joined #openstack-keystone14:09
*** gokrokve has quit IRC14:13
dolphmdstanek: https://bugs.launchpad.net/keystone/+bug/1294862 ?14:19
uvirtbotLaunchpad bug 1294862 in keystone "Token expiration time with memcache->kvs->dogpile is wrong" [Medium,Incomplete]14:19
dstanekdolphm: yeah14:19
dstaneklooks like maybe just a deployment issue with clocks reporting the wrong time?14:20
dolphmdstanek: i can't even guess :( hopefully that's the case14:20
dolphmdstanek: if it's the last standing RC-blocker and it's still incomplete, then we'll cut an RC without addressing it14:21
dstaneki'll be out of the office for a few hours starting soon14:22
dolphmdstanek: rubber stamp this one before you go :P https://review.openstack.org/#/c/82090/14:22
*** joesavak has quit IRC14:23
*** joesavak has joined #openstack-keystone14:24
dstanekdone. pretty small diff14:24
*** jsavak has joined #openstack-keystone14:25
dstanekthis there a target for cutting the release?14:26
*** joesavak has quit IRC14:28
dolphmdstanek: ASAP / when it's ready14:31
dolphmdstanek: we'll wait for a good 24 hours or so of no alarming bug reports and then make the cut14:31
dolphmdstanek: here's the overall race to RC-readyness: http://old-wiki.openstack.org/rc/14:32
openstackgerritayoung proposed a change to openstack/keystone: is_revoked check all viable subtrees  https://review.openstack.org/8123514:32
ayoungdolphm, gerrit needs an option:  when diffing two versions of a patch to only show lines from files in either patch14:33
dolphmayoung: i just flip back and forth between two tabs- the last patchset with comments and the latest patchset14:36
dolphmayoung: and sort of manually diff14:36
YorikSardolphm: Yeah, I'm somewhere around.14:36
YorikSardolphm: Did I miss something?14:36
dolphmYorikSar: ayoung just posted a patchset to https://review.openstack.org/#/c/81235/14:36
dolphmayoung: in this case it seems you changed for more than the comments asked for, so there's not much use in comparing against a previous version -- must review from scratch!14:38
*** david-lyle has joined #openstack-keystone14:38
YorikSarayoung: Some time ago I actually did smth like vimdiff <(git diff onepatch~..onepatch) <(git diff otherpatch~..otherpatch)14:40
YorikSarayoung: And it even did get me something useful :)14:40
YorikSardolphm, ayoung: It looks like there's not much changes compared to what we've discussed some time ago.14:41
dolphmYorikSar: i wasn't deeply involved in that conversation though, so i don't have much context to review from14:41
dolphm... for better or worse14:42
*** browne has joined #openstack-keystone14:42
dstanekYorikSar: that's a good idea14:43
openstackgerritA change was merged to openstack/keystone: Do not expose internal data on UnexpectedError  https://review.openstack.org/8138514:43
*** daneyon has joined #openstack-keystone14:44
*** thedodd has joined #openstack-keystone14:45
dstanekalrighty, i'm targeting being back online between 3 and 4 PM EST.14:47
dstanekdolphm: why is that the old wiki?14:47
*** jaosorior has joined #openstack-keystone14:47
*** raildo has joined #openstack-keystone14:47
dstanekdolphm: there looks to be some interesting stuff there still14:47
ayoungYorikSar, you can do that with gitreview , too14:48
ayoungjust that when you upload to the website, sometime you want to do a quick visual and the current UI doens't make it easy14:48
ayoungYorikSar, this is the real diff between your version and my latest https://review.openstack.org/#/c/81235/2..4/keystone/contrib/revoke/model.py14:52
*** zhiyan is now known as zhiyan_14:52
*** dstanek has quit IRC14:53
ayoungYorikSar, yeah.  I think my main objection to the filter was due to the effects I was seeing in the debugger.  I still don't like that looking at the result changes what happens under the covers14:57
ayoungI'm sure there is a cleaner way to say "append only if there is something to append"14:58
ayoungbut all the alternatives seem equally obtuse14:58
*** joesavak has joined #openstack-keystone15:01
*** jsavak has quit IRC15:04
ayoungYorikSar, anyway, +1 it if you approve of the changes,  please15:04
*** saju_m has quit IRC15:05
*** dstanek has joined #openstack-keystone15:07
YorikSarayoung: Here you go.15:10
*** gokrokve has joined #openstack-keystone15:10
*** gokrokve has quit IRC15:14
*** gokrokve has joined #openstack-keystone15:15
ayoungYorikSar, I'm going to make some changes from dolphm 's review15:19
dolphmayoung: if you want to use filter(), then i think you must use lambda x: x is not None?15:19
ayoungdolphm, Seems like the worst of both worlds:  Filter and a function call15:21
ayoungand don't think it is necessary15:21
dolphmayoung: get it correct, then optimize15:21
ayoungfilter(None, ['a', '', True, False, 1, 0, None])  as you point out, removes the Falsy values15:21
ayoungthe nodes we are adding to bundle are lists.  If they are empty, it would actually be correct to remove them15:22
ayoungbut I am pretty sure we never add even an empty list15:22
ayoungactually, the are dictionsaris, but still15:22
ayoungdolphm, agreed on "get it right then optimize" but pretty sure this is right15:24
*** devlaps has joined #openstack-keystone15:29
*** david-lyle has quit IRC15:29
YorikSardolphm: We have concrete data model. All these dicts can contain only other dicts.15:33
dolphmYorikSar: how is that concrete? lol15:34
YorikSardolphm: So replacing it with lambda won't give any profit neither from code readability side nor from performance side.15:34
*** mlemay has joined #openstack-keystone15:34
*** dstanek has quit IRC15:34
dolphmYorikSar: so just use an explicit list comprehension? [x for x in bundle if x is not None]15:34
YorikSardolphm: It's no different from filter with lambda...15:35
dolphmYorikSar: it's explicit and behaves differently15:36
YorikSardolphm: btw, that filter will exclude empty dicts as well and those are of no use to us.15:36
dolphmYorikSar: if you want to write a test that would make filter()'s behavior more useful than something more verbose, then by all means do so15:38
dolphmYorikSar: until then, we need to be in the business of not writing surprising code, especially when it conflicts with the inline comment describing it, because that will only result in bugs in the future15:39
openstackgerritayoung proposed a change to openstack/keystone: is_revoked check all viable subtrees  https://review.openstack.org/8123515:39
YorikSardolphm: Ok. Don't see a point in arguing about it.15:40
ayoungYorikSar, I had actually replaced the filter with that exact code when I was debugging15:46
ayoungso...I can support it.15:46
ayoungYorikSar, this code is dense, and hard for people to understand.  The easier we can make it to seee what is going on, the less likely someone will feel the need to rewrite it once we've moved on to other projects15:47
ayoungYorikSar, I'm making that change and resubmitting.  OK?15:47
YorikSarayoung: Sure.15:47
YorikSarayoung: I just wonder how much will my original code morph in the direction of "slow and not pretty" before it finally can be read by everyone.15:49
ayoungYorikSar, when we get this done, and moved over to the client, and used in place of the current revocation list, the performance benefit will far outweigh any slight slowdown to the native call15:50
ayoungYorikSar, to real ugliness is in appending the None's to the collection.  I suspect that setting those to a local variable and only adding if it is not None would be even more performant15:51
ayoungthe filter(None    syntax is awkward.15:52
YorikSarayoung: I don't have a benchmark in my pocket but I think that doing filter(None, ...) is faster than adding lots of if's.15:53
YorikSarayoung: Just because of less bytecode and more native methods.15:53
ayoungYorikSar, I suspect you are wrong.  Its not the ifs that are slow, it is the append(None)15:53
YorikSarayoung: Ah... I don't want to benchmark it anyway. At least not now.15:54
ayoungYorikSar, http://c2.com/cgi/wiki?PrematureOptimization15:55
YorikSarayoung: I have other, more pleasing thing to benchmark - rootwrap daemon :)15:55
ayoung++15:55
ayoungYorikSar, WTF do we have rootwrap anyways?15:56
YorikSarAlthough Keystone won't get any benefit from that.15:56
ayoungYorikSar, cuz we are smart enough not to use it!15:56
YorikSarayoung: Keystone just doesn't need it ;)15:56
YorikSarayoung: Unfortunatelly all low-level components do.15:57
ayoungYorikSar, rootwrap is one of those things that, each time it comes up, I get shivers and think "If you don't look closely at it, people won't blame you for it."15:57
ayoungYorikSar, I've seen how the Rest of OpenStack treat security.  And it has encouraged me to increase my alcohol consumption.15:58
YorikSarI'm currently migrating to other department and will have more time (all of it) to spend on community tasks. So I hope to start fixing such things like rootwrap (its slowness and insecure use).15:59
ayoungYorikSar, ++  lets talk in Atlanta16:00
YorikSarayoung: I won't be there16:00
ayoungYorikSar, is there a list of what people use rootwrap to do?16:00
ayoung:(16:00
ayoungYorikSar, that is unfortunate16:00
YorikSarayoung: I'm purely remote contributor :)16:00
openstackgerritayoung proposed a change to openstack/keystone: is_revoked check all viable subtrees  https://review.openstack.org/8123516:01
*** gyee has joined #openstack-keystone16:01
*** david-lyle has joined #openstack-keystone16:02
*** dstanek has joined #openstack-keystone16:05
*** marcoemorais has joined #openstack-keystone16:06
YorikSarayoung: It is in every project's filters.d dir.16:07
ayounghttp://git.openstack.org/cgit/openstack/nova/tree/etc/nova/rootwrap.d   YorikSar16:08
YorikSarayoung: And those should be carefully audited.16:08
YorikSarayoung: Yeah... And there's a lot of plain CommandFilter's16:09
ayoungYorikSar, I'm going to stop looking.  Its not something I can affect at this point, and getting worked up about it will do me no good.16:10
*** andreaf has quit IRC16:10
*** dstanek has quit IRC16:18
*** jsavak has joined #openstack-keystone16:29
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Sync test_migrations  https://review.openstack.org/8061816:30
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063016:30
*** mlemay has quit IRC16:32
*** joesavak has quit IRC16:32
*** richm has joined #openstack-keystone16:32
gyeeayoung, jamielennox, is this for the use case where we will not be maintaining a global user_id? https://review.openstack.org/#/c/8197316:33
*** anteaya is now known as cyan16:36
*** cyan is now known as Guest2006416:36
*** Guest20064 is now known as anteaya-who-was-16:37
*** anteaya-who-was- is now known as anteaya16:37
*** gokrokve has quit IRC16:38
*** gokrokve has joined #openstack-keystone16:38
*** jogo is now known as flashgordon16:39
openstackgerritA change was merged to openstack/keystone: update sample conf  https://review.openstack.org/8209016:42
*** gokrokve has quit IRC16:42
*** marcoemorais1 has joined #openstack-keystone16:46
*** marcoemorais1 has quit IRC16:46
*** marcoemorais has quit IRC16:48
*** browne has quit IRC16:48
*** prad has joined #openstack-keystone16:49
*** prad has left #openstack-keystone16:49
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Sync test_migrations  https://review.openstack.org/8061816:54
openstackgerritIlya Pekelny proposed a change to openstack/keystone: Comparision of database models and migrations.  https://review.openstack.org/8063016:54
*** marcoemorais has joined #openstack-keystone16:59
bknudsonI'm going to propose for juno that we don't take sync of modules, only all openstack-incubator16:59
bknudsonoslo-incubator16:59
openstackgerritayoung proposed a change to openstack/keystone: Remember the DN  https://review.openstack.org/4744117:00
dolphmjamielennox|away: ping me when you get in17:00
*** gokrokve has joined #openstack-keystone17:06
*** YorikSar has quit IRC17:06
*** david_lyle_ has joined #openstack-keystone17:07
*** david-lyle has quit IRC17:09
*** gokrokve_ has joined #openstack-keystone17:09
gyeedolphm, do you recall why auth_token middleware store the token information in the headers as oppose to the request environment?17:09
dolphmgyee: you mean why they appear as HTTP headers in the cgi/wsgi environment, rather than as python objects in the cgi/wsgi environment?17:10
gyeeright17:10
gyeethat design was way before I got involved in Keystone17:11
dolphmgyee: for backwards-compatibility with the approach taken by auth_token's predecessor(s) -- so that it could be used as a drop-in replacement17:11
*** gokrokve has quit IRC17:11
*** browne has joined #openstack-keystone17:11
gyeeahh17:11
dolphmgyee: for bonus points, it means that auth can be done by something else entirely -- there's no expectation in auth_token's contract that it actually be implemented in python17:12
gyeedolphm, got it, thanks :)17:12
*** gokrokve has joined #openstack-keystone17:12
dolphmgyee: or it could be in python but not be in the same wsgi pipeline, etc17:13
*** leseb has quit IRC17:13
gyeedolphm, yeah, I was think from that angle17:13
gyeeI mean I wasn't17:13
*** gokrokve_ has quit IRC17:14
gyeedolphm, I am trying to improve the performance of ec2 -> auth_token17:14
gyeetoday ec2 sets the token in X-Auth-Token header, then auth_token validates again17:15
gyeetrying to figure out if we can do this in one go17:15
gyeebut auth_token clears the token info headers everytime17:15
*** leseb has joined #openstack-keystone17:16
gyeeif we don't stash to token info in the headers to solution would be much simpler17:17
gyees/to/the/17:17
*** thedodd has quit IRC17:26
*** leseb has quit IRC17:29
*** leseb has joined #openstack-keystone17:29
*** joesavak has joined #openstack-keystone17:31
*** leseb has quit IRC17:33
*** jsavak has quit IRC17:33
*** saju_m has joined #openstack-keystone17:39
*** marekd is now known as marekd|away17:44
*** YorikSar has joined #openstack-keystone17:45
*** saju_m has quit IRC17:58
*** leseb has joined #openstack-keystone17:59
*** leseb has quit IRC18:05
*** amcrn has joined #openstack-keystone18:09
*** thedodd has joined #openstack-keystone18:10
*** morganfainberg_Z is now known as morganfainberg18:11
morganfainbergdolphm, doh, parallel transient?!18:12
morganfainbergdolphm, ah well18:12
dolphmmorganfainberg: dstanek/ayoung suggested we land it anyway with it disabled18:13
dolphmmorganfainberg: by default18:13
morganfainbergdolphm, thats an easy fix18:13
dolphmmorganfainberg: ++ i'm good with that if you want to rev18:13
ayoungmorganfainberg, yeah, I want people to be able to easily beat on it18:13
*** leseb has joined #openstack-keystone18:13
morganfainbergjust make the echo line return a 1 instead18:13
morganfainbergof 018:13
morganfainbergdolphm, adding that in now18:14
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Add support for parallel testr workers in Keystone  https://review.openstack.org/6587018:16
morganfainbergdolphm, ^ should run 1 worker unless you set TEST_RUN_CONCURRENCY to 0 (1 per core) or N for N workers18:17
*** YorikSar has quit IRC18:17
*** leseb has quit IRC18:17
dolphmmorganfainberg: bknudson: +218:18
dolphmayoung: ^18:18
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: add pooling for cache references  https://review.openstack.org/8107818:18
*** YorikSar has joined #openstack-keystone18:18
ayounggah, read that wrong18:20
ayoungthought it had already made it through check18:21
ayoungjust dropping the +A will stop it, right?18:21
morganfainbergayoung, it wont go to gate untiul check is done18:21
ayoung++18:21
morganfainbergayoung, gate doesn't let that happen anymore18:21
ayoungwhew18:21
morganfainbergayoung, usually i am adverse to pre-empting gate still, but it is useful for getting things moving over night18:22
ayoungmorganfainberg, so...18:22
morganfainbergi'm going to see if i can get that failure to occur (transient) so we can smash it.18:22
morganfainbergonce i reconstruct my VM.18:22
* ayoung needs to add you to adraft18:22
morganfainbergayoung, ok18:22
ayounghttps://review.openstack.org/#/c/79411/18:23
ayoungmorganfainberg, I'm working on that ^^...I'm going to post a nother version that is slightly ahead18:23
ayoungI need this for compressed tokens18:23
morganfainbergayoung, oh yeah18:24
ayoungmorganfainberg, so version 2 shows one that runs in Python33 (at least the unit tests)18:24
morganfainbergayoung, thats a good start!18:25
ayoungI know I need to use six somehow...can you shorten the learning curve for me here?18:25
ayoungthis version fails on 27 with18:25
morganfainberghehe sure18:25
ayoung  File "keystoneclient/common/cms.py", line 109, in cms_verify18:25
ayoung    data = bytes(formatted, 'UTF-8')18:25
ayoungTypeError: str() takes at most 1 argument (2 given)18:25
ayoungI thought it was safe_encode?18:26
*** amcrn has quit IRC18:27
ayoungbut that is, I think, now redundant here18:27
morganfainberghmm.18:28
ayoungand, in fact, I can safely remove it on line 108 now18:28
morganfainbergayoung, realized i don't have py33 on my laptop18:29
ayoungmorganfainberg, heh18:29
morganfainberggive me a sec18:29
morganfainbergsetting it up18:29
morganfainbergthis weekend i'm going to bootcamp up this lapto w/ a native linux boot to help w/ some of this stuff18:30
ayoungthe bytes(formatted,'utf-8')  looks like it is just a str() call in py2718:30
morganfainbergayoung, binary_type in py27 is str()18:30
ayoungso there needs to be a six call to wrap bytes18:30
dolphmayoung: six.text_type?18:30
dolphmayoung: oh i see nevermind18:31
ayoungdolphm, yeah, the opposite18:31
morganfainbergdolphm, that would be the equivalent of unicode in py2718:31
ayoungsomething that produces str on 27 but bytes on 3318:31
dolphmayoung: 27 has bytes() though, right?18:31
dolphmbytearray() ?18:31
morganfainbergdolphm, bytes = str in py2718:32
ayoungdolphm, yeah, it just doesn't take the 'utf-8' param18:32
dolphmayoung: bytearray(formatted, encoding='utf-8') ?18:32
ayounghttp://pythonhosted.org//six/#six.b18:33
ayoungbut that is latin18:33
ayoungdolphm, trying your suggestion18:33
morganfainbergayoung, you might need to do if six.PY318:33
dolphmayoung: http://pythonhosted.org//six/#six.binary_type18:33
morganfainbergayoung, and just handle it differently18:33
morganfainbergdolphm, except encoding works differently (i think) in py33 vs py2718:34
morganfainbergstr().encode vs bytes(str, encoding)18:34
ayoungdolphm, I think that bytearray(formatted, encoding='utf-8')   worked18:35
morganfainbergayoung, be careful, bytearray is mutable18:37
*** dstanek has joined #openstack-keystone18:38
morganfainbergayoung, behavior wise, using if six.PY3: bytes(thing, encoding) else: bytes(thing).encode might be better18:40
bknudsonwhat does bytes with encoding do?18:41
bknudsonvs bytes w/o encoding18:41
morganfainbergbknudson, bytes in py33 needs encoding18:42
morganfainbergbknudson, it's a binary type vs a byte_str18:42
morganfainbergbknudson in py27, bytes = str18:42
bknudsonhttp://docs.python.org/3/library/functions.html#bytes -- encoding is optional18:43
bknudsonoh, if it's a string it needs an encoding18:44
bknudsonstr.encode('utf-8') returns bytes18:45
morganfainbergbknudson, yeah18:46
morganfainbergbknudson, that's what ayoung's running into18:46
*** haneef_ has quit IRC18:47
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Remove extraenous instantiations of managers  https://review.openstack.org/8172018:47
morganfainbergbknudson, should we remove the need to instantiate credential api here: https://review.openstack.org/#/c/81720/3/keystone/tests/test_sql_upgrade.py ?18:48
bknudsonseems like could use str.encode() on either py27 or py33?18:48
morganfainbergbknudson, ++18:49
morganfainbergayoung, ^18:49
morganfainbergbknudson, yeah str.encode() in py33 returns a bytes()18:50
*** mlemay has joined #openstack-keystone18:51
bknudsonmorganfainberg: the creation of credential.Manager() there is ugly but it makes sense why we would have it...18:52
*** amcrn has joined #openstack-keystone18:52
bknudsonmorganfainberg: you don't have managers when you do keystone-manage db_sync.18:52
morganfainbergbknudson, it is, i think we could just check the db structure and do the same query18:52
morganfainbergbknudson, but i'm not opposed to leaving that one as is18:52
ayoungmorganfainberg, can't you @depends that one instead?18:52
bknudsonmorganfainberg: yes, that's what I was thinking... maybe should rewrite the test to not use the manager.18:52
ayoungthat test is also not future proof18:54
ayoungif we futher morph the credentials object, that test will fail18:54
openstackgerritDiane Fleming proposed a change to openstack/identity-api: Clean up naming to match  new conventions  https://review.openstack.org/8107619:00
morganfainbergayoung, correct it should look at DB structure not the manager19:01
morganfainbergayoung, @depends is the wrong approach there19:01
ayoungmorganfainberg, yep...remove the manager check19:01
morganfainbergayoung, ++ was leaning that way but wanted to check w/ someone else (2nd pair of eyes first)19:02
dolphmbknudson: still around?19:11
*** mlemay has quit IRC19:12
openstackgerritDolph Mathews proposed a change to openstack/python-keystoneclient: add functional test for cache pool  https://review.openstack.org/8217219:13
bknudsondolphm: yes, still around19:13
dolphmbknudson: added some functional tests for the cache pool per morgan's suggestion https://review.openstack.org/#/c/82172/1/keystoneclient/tests/test_auth_token_middleware.py19:13
dolphmmorganfainberg: ^19:13
dolphmmorganfainberg: this also needs a re-review https://review.openstack.org/#/c/81235/19:22
*** leseb has joined #openstack-keystone19:24
dolphmmorganfainberg: also left this as incomplete based on jamie's comment https://bugs.launchpad.net/keystone/+bug/1294862 - wasn't sure if you had made progress there?19:25
uvirtbotLaunchpad bug 1294862 in keystone "Token expiration time with memcache->kvs->dogpile is wrong" [Medium,Incomplete]19:25
dolphmmorganfainberg: if a check run fails due to a known transient but otherwise succeeds, recheck *and* +A?19:36
morganfainbergdolphm, i would19:38
dolphmmorganfainberg: doned https://review.openstack.org/#/c/81078/19:38
morganfainbergdolphm, yeah i was going to circle back w/ the guy on irc next week and see if i can chase anything else down19:38
morganfainbergdolphm, he's UTC+1 so, on weekend now19:38
morganfainbergdolphm, but incomplete seems reasonable19:39
morganfainbergdolphm, i also had someone run into more memcache issues w/ token drivers, going to see if i can come up with some magic to help limp us to ephemeral tokens.19:39
morganfainbergdolphm, something trivial we can backport to make quality of life better for them19:40
morganfainbergbut more and more i think ephemeral tokens really is a must have for juno19:40
morganfainbergso in L we can make it the "only way"™19:40
dolphmmorganfainberg: +++19:52
*** saju_m has joined #openstack-keystone20:05
ayoungWow, this "work in both python 27 and 33" thing is a pain20:13
*** marcoemorais has quit IRC20:15
*** marcoemorais has joined #openstack-keystone20:16
*** openstackgerrit has quit IRC20:22
*** openstackgerrit has joined #openstack-keystone20:23
*** Nathan255 has quit IRC20:27
*** marcoemorais has quit IRC20:33
*** saju_m has quit IRC20:35
dolphmjenkins +1 on https://review.openstack.org/#/c/81235/20:38
dolphmmorganfainberg: lbragstad: ^20:38
morganfainbergdolphm, yep reading it over. it's dense code20:38
dolphmmorganfainberg: very20:38
morganfainbergbut i think the list comprehension is better than filter in this case20:38
*** dstanek has quit IRC20:41
*** dstanek has joined #openstack-keystone20:41
* morganfainberg avoids premature optimization comments20:41
*** Nathan255 has joined #openstack-keystone20:42
*** leseb has quit IRC20:43
morganfainbergdolphm, ok so i think this is fine as is, but the whole list comprehension could have been avoided if a simple if is none check was done on tree.get20:44
morganfainbergdolphm, rather than blindly appending and then having to cleanup later.20:44
morganfainbergdolphm, not sure which is faster, but with enough nested for loops, iteration becomes the bottleneck20:45
morganfainbergor make bundle a subclass of list that .append throws away None vs adding it20:45
dolphmmorganfainberg: weren't there multiple places where you'd have to add if-conditions then?20:45
dolphmmorganfainberg: or were those factored back out20:45
morganfainbergdolphm, 220:45
dolphmmorganfainberg: oh that's not bad20:45
dolphmmorganfainberg: test it and share a diff?20:45
morganfainbergdolphm, one in the role_id section and one in the alt_name section20:46
morganfainbergdolphm, give me a sec.20:46
openstackgerritAndreas Jaeger proposed a change to openstack/identity-api: Update to clouddocs-maven-plugin 1.15  https://review.openstack.org/8219620:50
morganfainbergdolphm, building VENV but should have results shortly20:51
*** raildo has quit IRC20:57
openstackgerritA change was merged to openstack/keystone: Add support for parallel testr workers in Keystone  https://review.openstack.org/6587020:59
*** leseb has joined #openstack-keystone21:02
dolphmyay ^21:02
openstackgerritDolph Mathews proposed a change to openstack/keystone: Always include 'enabled' field in service response  https://review.openstack.org/8220521:02
dolphmbknudson: working to get the service side of this moving ^ but ran into a weird issue with the downgrade path in the new migration21:04
dolphmbknudson: if it runs for sqlite, every subsequent test fails with DatabaseAlreadyVersionControlled21:05
morganfainbergayoung, correct me if i'm wrong but does partial matches being none in need to be backdented one more time?21:10
morganfainberghttps://review.openstack.org/#/c/81235/6/keystone/contrib/revoke/model.py21:10
morganfainbergdolphm, ^21:10
morganfainbergit looks like we're bailing out on the first item in _names21:10
morganfainbergif bundle is empty21:10
*** joesavak has quit IRC21:11
morganfainbergor does that imply revoke_map is subsequently also empty?21:11
morganfainbergactually i see that, nvm21:11
morganfainbergpartial matches would be completely empty in that case anyway21:12
morganfainbergdolphm, it turns out you'd need to check for is none in 3 places.21:13
bknudsondolphm: I'll give it a try myself.21:13
morganfainbergdolphm, http://pasteraw.com/bjtnj1wfxgmf1pyj76cyhf3nsxg2fo321:14
*** leseb has quit IRC21:14
ayoungmorganfainberg, line?21:15
dolphmbknudson: found the problem!21:15
morganfainbergayoung, strike my concern, misread earlier in the code21:15
ayoungcool.21:15
*** ayoung is now known as ayoung_dadmode21:15
bknudsondolphm: I got MismatchError: ['id', 'type', 'extra'] != [u'id', u'type', u'extra', u'enabled']: service table21:18
openstackgerritDolph Mathews proposed a change to openstack/keystone: Always include 'enabled' field in service response  https://review.openstack.org/8220521:18
dolphmbknudson: fixed ^ should pass tests now21:19
*** dstanek has quit IRC21:20
*** dstanek has joined #openstack-keystone21:21
dolphmbknudson: oops, let a debuggy statement slip into review - one sec21:21
openstackgerritDolph Mathews proposed a change to openstack/keystone: Always include 'enabled' field in service response  https://review.openstack.org/8220521:21
bknudsonit doesn't look like there's a way to turn off notifications?21:27
*** marcoemorais has joined #openstack-keystone21:27
bknudsonmaybe only by setting the rpc_backend to a fake driver21:27
*** dstanek_afk has joined #openstack-keystone21:31
*** jamielennox|away is now known as jamielennox21:38
*** marcoemorais has quit IRC21:38
*** dstanek has quit IRC21:41
*** dstanek has joined #openstack-keystone21:41
*** dstanek has quit IRC21:42
*** dstanek_afk is now known as dstanek21:42
*** thedodd has quit IRC21:43
jamielennoxdolphm: i'm here for about 5 minutes21:44
dolphmjamielennox: just wanted to give you a heads up that https://review.openstack.org/#/c/81078/ was going in, and that i'll be intended to release 0.7.0 ASAP21:45
morganfainbergdolphm, howabout http://pasteraw.com/mm6v3fohxx8c1jgpleuittx79om2zyk21:45
morganfainbergok ok i'll stop and +2 that review21:45
morganfainbergit eliminates 3 possible code paths, but it uses lambdas to get there21:46
*** marcoemorais has joined #openstack-keystone21:46
jamielennoxdolphm: ahhh, i've got about 5 reviews i wanted in that release21:47
dolphmjamielennox: i'm going to try and wait until monday21:47
morganfainbergnot sure if that is really any more readable or better.21:48
jamielennoxdolphm: this is important for horizon: https://review.openstack.org/#/c/82007/21:48
jamielennoxthis i've been pushing for ages and is the one i need to start getting other clients up to scratch: https://review.openstack.org/#/c/60752/21:48
dolphmjamielennox: the first one could be a minor release anytime21:50
jamielennoxthere's a bunch more, but if we are going to do a client release then i really need people to do some client reviews21:50
jamielennoxi would like: https://review.openstack.org/#/c/77026/21:50
jamielennoxand the follow up so i'm not stuck with a bunch of new mismatchin args/kwargs21:51
morganfainbergjamielennox, i def have the bandwidth to do reviews next week21:51
jamielennoxi can be around again in about 90 minutes to hand hold some reviews, but some of those have been up there for weeks21:52
dolphmjamielennox: we'll probably need to be doing a release before the next keystone meeting, but i put them on the agenda anyway https://wiki.openstack.org/wiki/Meetings/KeystoneMeeting21:53
openstackgerritA change was merged to openstack/python-keystoneclient: add pooling for cache references  https://review.openstack.org/8107821:53
jamielennoxi think this will save us pain in the long run: https://review.openstack.org/#/c/76733/ but i have no justification for it at the moment as it's only my plugins21:55
dstanekjamielennox: is there any way to know which ones are the most important?21:55
jamielennoxdstanek: i've gotta do an airport run now - but i'll be back in about an hour and i'll do a list of the ones that are compatibility issues21:55
jamielennoxbut the couple mentioned above would be good21:56
dstaneki'll start taking a look - i'm in and out of connectivity, but i'll download the patches and take local notes if i'm not connected21:57
jamielennoxok: so the two positional reviews21:57
jamielennoxhttps://review.openstack.org/#/c/77026/ and  https://review.openstack.org/#/c/77055/21:58
*** marcoemorais has quit IRC21:58
openstackgerritDolph Mathews proposed a change to openstack/keystone: Always include 'enabled' field in service response  https://review.openstack.org/8220521:58
jamielennoxarg passing: https://review.openstack.org/#/c/76733/21:58
dolphmdstanek: traveling?21:59
dstanekdolphm: just in the car right now :-)21:59
jamielennoxi think that will do for compat21:59
jamielennoxother's i'd like but those need to happen pre-release22:00
jamielennox /aren't possible post release22:00
*** Nathan255 has quit IRC22:00
*** marcoemorais has joined #openstack-keystone22:01
jamielennoxback later22:02
dolphmi'm about to pass out for now...22:04
*** dstanek has quit IRC22:04
dolphmmorganfainberg: just a reminder that you never +2'd on https://review.openstack.org/#/c/81235/ -- not sure if you're still looking at it22:04
morganfainbergdolphm, eliminating for loops: http://pasteraw.com/mm6v3fohxx8c1jgpleuittx79om2zyk but yeah i hadn't +2'd because i was seeing if we could clean it up easily... i think we'd lose readability22:05
morganfainbergdolphm, any thoughts / concerns before i +2/+A22:07
dolphmmorganfainberg: not for icehouse, i don't think22:07
morganfainberg+2/+a22:08
morganfainbergdone22:08
dolphmmorganfainberg: looking forward to seeing that code in 6 months ;P22:08
morganfainbergbut yeah i think this is a case where using generators is worthwile.22:08
morganfainbergbut lambdas make a lot of people cry22:08
morganfainbergdolphm, LOL22:09
morganfainbergyou can do evil things with itertools >.>22:09
*** bknudson has quit IRC22:09
*** dstanek has joined #openstack-keystone22:14
openstackgerritMorgan Fainberg proposed a change to openstack/keystone: Remove extraenous instantiations of managers  https://review.openstack.org/8172022:19
*** browne has quit IRC22:19
morganfainbergnow will less credential_api manager instantiation22:20
morganfainbergs/will/with22:20
*** daneyon has quit IRC22:22
*** gokrokve has quit IRC22:24
*** gokrokve has joined #openstack-keystone22:24
openstackgerritJenkins proposed a change to openstack/keystone: Updated from global requirements  https://review.openstack.org/8223122:27
*** browne has joined #openstack-keystone22:29
*** gokrokve has quit IRC22:29
*** wchrisj has quit IRC22:34
openstackgerritJenkins proposed a change to openstack/python-keystoneclient: Updated from global requirements  https://review.openstack.org/7969522:34
*** marcoemorais1 has joined #openstack-keystone22:39
*** marcoemorais has quit IRC22:39
*** marcoemorais1 has quit IRC22:45
*** marcoemorais has joined #openstack-keystone22:54
*** browne has left #openstack-keystone22:54
*** dims has quit IRC22:59
*** devlaps has quit IRC22:59
*** marcoemorais1 has joined #openstack-keystone23:03
*** marcoemorais has quit IRC23:03
*** gokrokve has joined #openstack-keystone23:03
*** marcoemorais has joined #openstack-keystone23:06
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Add a positional decorator  https://review.openstack.org/7702623:06
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Start using positional decorator  https://review.openstack.org/7705523:06
*** marcoemorais1 has quit IRC23:07
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Add a positional decorator  https://review.openstack.org/7702623:08
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Start using positional decorator  https://review.openstack.org/7705523:08
*** marcoemorais has quit IRC23:13
*** dims has joined #openstack-keystone23:14
*** gokrokve has quit IRC23:21
*** gokrokve has joined #openstack-keystone23:21
*** gokrokve has quit IRC23:25
openstackgerritJamie Lennox proposed a change to openstack/python-keystoneclient: Allow passing a req.Session object to old client  https://review.openstack.org/8225823:28
*** marcoemorais has joined #openstack-keystone23:28
*** wchrisj has joined #openstack-keystone23:38
ayoung_dadmodedolphm, if I have a string but it was populated with binary data origianlly, it blows up if I do: data = bytearray(uncompressed, encoding='UTF-8')   is there some encoding that implies "NONE"?23:38
*** arunkant has quit IRC23:39
*** marcoemorais has quit IRC23:41
*** marcoemorais has joined #openstack-keystone23:41
*** browne has joined #openstack-keystone23:41
nkinderayoung_dadmode: "dadmode" means no coding....23:42
*** browne has quit IRC23:42
ayoung_dadmodenkinder, 3 Ninjas Playing upstairs says otherwise...I got a bout 20 minutes23:42
*** david_lyle_ has quit IRC23:44
*** browne has joined #openstack-keystone23:45
nkinderayoung_dadmode: just when you think they're playing nicely on their own, all hell will break loose from my experience23:46
ayoung_dadmodenkinder, TV is a very powerful drug, and only is administrered with a prescription in this house.  I told them that they might not get to watch the whole movie due to the late hour...so they are not going to disturb me, hoping that I will forget.23:47
ayoung_dadmodenkinder, I'm working on the compressed token thing...python libraries suck for binary data23:47
ayoung_dadmodeI get back a "string"23:47
ayoung_dadmodehttp://docs.python.org/2/library/zlib.html#zlib.decompress23:48
*** dstanek has quit IRC23:48
*** jagee_ has quit IRC23:54
nkinderayoung_dadmode: can't you make it a bytestring?23:57
jamielennoxayoung_dadmode: headers are ascii anyway right, cant you just decode to bytes23:58
nkinderayoung_dadmode: it's been a while since I've had to mess with that sort of stuff in python, but I remember having to use binascii for a few things23:58
nkinderhttp://docs.python.org/2/library/binascii.html23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!