18:07:05 <henrynash> #startmeeting keystone
18:07:06 <openstack> Meeting started Tue Aug 27 18:07:05 2013 UTC and is due to finish in 60 minutes.  The chair is henrynash. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:07:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:07:09 <openstack> The meeting name has been set to 'keystone'
18:07:16 <bknudson> I'm glad we're not on the marketing committee
18:07:20 <ayoung> #topic Feature Freeze
18:07:31 <henrynash> indeed
18:07:47 <ayoung> Aug 28
18:07:47 <henrynash> who has changes that they are trying to get in
18:07:57 <henrynash> (says me who has 2 or maybe 3)
18:08:01 <morganfainberg> o/
18:08:01 <gyee> pretty much everybody :)
18:08:06 <ayoung> Not me...
18:08:18 <ayoung> I learned my lesson the hard way
18:08:20 <bknudson> do we -2 anything that looks like a new feature?
18:08:31 <ayoung> bknudson, yep
18:08:50 <henrynash> bknudson: if it's not in the bp that was agreed, then yes
18:08:53 <ayoung> it will be allowed to merge as soon as the Havana RC branch is created.  Master becomes Icehouse
18:09:13 <ayoung> #action dolphm publish when we can start Icehouse development
18:09:30 <morganfainberg> ayoung: is meetbot not listening to you?
18:09:35 <ayoung> milestone 3 cute is sept 4, so I assume it will happen around then
18:09:38 <gyee> can somebody review fabio's endpoint filtering extension?
18:09:42 <gyee> that one's real close
18:09:45 <ayoung> morganfainberg, doesn't matter, it will show up in the log
18:09:47 <bknudson> should be able to start icehouse development now, just don't expect it to be merged.
18:09:51 <topol> gyee, url?
18:10:04 <fabiogia> #link https://review.openstack.org/#/c/33118/31
18:10:14 <morganfainberg> gyee: it's on my todo today, it is def. close.
18:10:29 <gyee> thanks guys
18:10:59 <henrynash> so I have two definites that need to go in:
18:11:05 <ayoung> 31?  Bah.  Wake me once he breaks 50.  That when things get interesting
18:11:07 <henrynash> #link https://review.openstack.org/#/c/38308/
18:11:09 <ayoung> right stevemar ?
18:11:10 <gyee> henrynash, I am reviewing them
18:11:28 <henrynash> #link https://review.openstack.org/#/c/43257/
18:12:09 <henrynash> and I need to check with dolphm on #link https://review.openstack.org/#/c/43581/
18:12:14 <ayoung> I think I can Approve endpoitn filtering,
18:12:28 <gyee> ayoung, thank you sir
18:12:54 <morganfainberg> I also have a few that need some hard looks.
18:12:55 <henrynash> ..which is ready, but want to get his view on whether we go ahead with the originally contemplated page sematics, or where we leave that to IceHouse as it was somewhat contentious
18:12:58 <fabiogia> ayoung: thanks
18:13:06 <ayoung> target entities was real close, too.  henrynash is jenkins failure spurious?
18:13:19 <henrynash> ayoung: yes,  think so
18:13:34 <ayoung> fabiogia, understand, I say that because it is an extension, and should be disabled by default.  Low risk.
18:13:50 <henrynash> ayoung: gyee suggested I add a configuration.rst section on policy rules, which I think is a good idea
18:13:50 <ayoung> henrynash's is higher risk, but I've looked at it a few times and know the scope
18:14:05 <henrynash> ayoung: I'll do that after this meeting
18:14:17 <ayoung> filter support is a little riskier
18:14:45 <topol> henrynash +1 on adding a section on policy rules.  Defintely would like a rosetta stone for that
18:15:11 <morganfainberg> ++
18:15:13 <ayoung> I would be motivated to postpone filters to icehouse.  That would make jaypipes sad, but I'm used to doing that to him
18:15:16 <henrynash> ayoung: I think we have to put filtering in - I'll commit to ramping on the testing ahead of RC
18:15:18 <gyee> ayoung, I like the admin domain concept, its a much cleaner deployment pattern
18:15:35 <ayoung> gyee, me too
18:15:41 <topol> gyee +1
18:15:43 <jaypipes> ayoung: lol :) that's fine with me. better to be done right.
18:16:08 <henrynash> ayoung: I'd really like filters in there….there doesn't seem a debate about it…just the testing that we need to do
18:16:09 <jamielennox> what is the admin domain concept? other than what is described by the name
18:16:28 <morganfainberg> ayoung: if we are pushing filtering to Icehouse, i'll want to propose a couple more patchsets for caching around the list methods.
18:16:28 <jamielennox> just an expected deployment pattern
18:16:29 <ayoung> henrynash, it isn't a question of whether the approach is right...it is the risk
18:16:31 <gyee> ayoung, for filters, passing query params to the drivers is a good start
18:17:03 <gyee> we probably will need to make some changes along the way, but at least give us a starting point
18:17:16 <henrynash> ayoung: agreed.  and we have an extra week of testing since we are freezing early
18:17:23 <ayoung> gyee, ... I think we are SQL stupid in OpenStack in general and Keystone specifically.  I suspect that we do way too mcuh sql generation,. and I don't want to open Keystone up for an injection attack
18:17:42 <ayoung> I'd be happier if we forced everything to use sored procedures
18:17:51 <ayoung> heh
18:17:53 <ayoung> stored
18:17:53 <gyee> ayoung, we are doing param binding
18:18:04 <gyee> so sql injection is unlikely
18:18:36 <ayoung> "unlikely"
18:18:42 <ayoung> yeah, lets punt
18:18:48 <henrynash> jaypipes: (fyi, I agree with deferring pagination to IceHouse to make we sure we get the api right, even though that is ready to go)
18:19:01 <henrynash> ayoung: let's punt, what?
18:19:08 <gyee> pagination to IceHouse, fine with me
18:19:10 <topol> punt on what?
18:19:12 <ayoung> henrynash, lets punt filtering to Icehouse.
18:19:15 <gyee> but can we at least get filters in?
18:19:20 <morganfainberg> pagination to icehouse sounds good.
18:19:28 <henrynash> ayoung:  not sure I'm ready to agree
18:19:28 <ayoung> gyee, I won't block it, but I'm not going to approve it.
18:19:30 <jamielennox> agree
18:19:42 <henrynash> ayoungL pagination to IceHouse, yep
18:19:57 <henrynash> ayoung: that's fair
18:20:05 <ayoung> henrynash, filtering is a prereq for pagination, right?
18:20:14 <henrynash> ayoung: yes
18:20:21 <topol> why dont we have enough runway that you feel comfortable with filtering?
18:20:33 <ayoung> now, you make the argument that filtering is not an API change, just an implementation of what is in the spec
18:20:40 <henrynash> ayoung: they are separate patches
18:20:41 <ayoung> I think that breaks the H2 API freeze rule
18:20:58 <henrynash> ayoung: how come?
18:21:08 <gyee> all I care is get the query params into the drivers so I can do my thing
18:21:26 <bknudson> can we split out the part that gets the query params into the drivers?
18:21:31 <ayoung> gyee, you are not going to get them for LDAP anyway.  The best we are going to get is SQL this go round
18:21:58 <ayoung> I'll let y'all make the choice, but I think it is a mistake, and suggest we let filtering land in Icehouse 1
18:22:28 <henrynash> bknduson: well we already do filtering, we just do it at a high-level (controller)…. all this does is let the drives implement that same semantics
18:22:29 <bknudson> what filtering do we allow?
18:22:32 <bknudson> what's in the spec?
18:22:33 <gyee> bknudson, that sound like a good idea
18:22:34 <morganfainberg> i like bknudson's plan.  get the query params down to the driver, but maybe not implment the filtering
18:22:49 <morganfainberg> at the driver.  if someone wants to add that in on their own, they at least have it.
18:23:09 <gyee> fyi, I have some custom (internal) drivers
18:23:24 <bknudson> henrynash: gyee: it sounds we've already got what gyee wants.
18:23:25 <gyee> morganfainberg, yes
18:23:31 <morganfainberg> s/implement/do anything with it yet/
18:23:34 <henrynash> bknudson: yes
18:23:48 <ayoung> so we will break filtering for LDAP  then, since the LDAP drivers don't implement it and we are going to remove it from the controller?
18:23:59 <bknudson> it's not removed from the controller
18:24:08 <henrynash> ayoung: that isn't the way it works
18:24:08 <bknudson> the controller handles what the backend doesn't
18:24:09 <ayoung> So we are going to execute the filter twice?
18:24:15 <henrynash> bknudson: exactly
18:24:36 <henrynash> ayoung: no, the driver removes the filter from the query_dict if it has satisfied it
18:24:41 <bknudson> the controller modifies the query string to remove those that it handles.
18:24:53 <gyee> exactly
18:24:55 <henrynash> bknudson the driver
18:25:08 <bknudson> sorry, should have said driver and not controller there.
18:25:20 <ayoung> I'll step out of the argument:  I've said my piece
18:25:40 <bknudson> henrynash: drivers don't do any filtering at this point?
18:25:59 <henrynash> bkndson: the sql one for assignment and identity does
18:26:30 <bknudson> that's the part that gyee and ayoung are concerned about
18:27:03 <henrynash> bknduson: that was the whole point of this bp, that we started at the last summit, we desoped to pull out just filtering....
18:27:56 <topol> so without the filtering option arent we seeing big problems in production environments with performance?
18:28:09 <ayoung> topol, yeah.
18:28:10 <ayoung> topol, I
18:28:12 <gyee> topol, yeah
18:28:14 <morganfainberg> topol: yep.
18:28:20 <ayoung> ve heard that list users is a big one from horizon
18:28:31 <gyee> we dont' want LDAP to take hours and return you a few thousand entries :)
18:28:31 <ayoung> I am a fan of filtering, just not a 1/2 solution at the last minute
18:28:48 <henrynash> ayoung: why is it 1/2 a solution
18:28:51 <ayoung> gyee, we don't have a filtering patch for LDAP ready to go, and I think that one is the real problem,
18:28:51 <gyee> 1/2 solution > no solution
18:28:58 <ayoung> henrynash, LDAP
18:29:00 <bknudson> do we support partial matches?
18:29:15 <morganfainberg> bknudson: i think that code is there, but not enabled.
18:29:20 <bknudson> I didn't look a the filtering spec so don't know what was expected.
18:29:21 <henrynash> bknduson: I agreed that should go to ice house, there is a separate review for that
18:29:42 <topol> Im confused.  is it viewed as a half solution or just not enough time to test?
18:30:09 <gyee> topol, both
18:30:39 <gyee> I like simo's argument on KDS better, code ain't going to mature to sitting there :)
18:30:39 <henrynash> topol: so the framework is the full solution, then any given driver can whose to satisfy filters to not
18:30:49 <gyee> s/to/by/
18:31:13 <topol> gyee, are you saying lets get it out there so we can kick the tires on it?
18:31:21 <henrynash> topol: today we have the sql assignment and identity drivers that do this, the LDAP one does not (which is OK for compatibility, but you'll get no performance increase for LDAp either)
18:31:21 <bknudson> I don't think we want different backends handling partial match queries differently.
18:31:39 <ayoung> OK...we should move on
18:31:49 <henrynash> bknduson: there is no partial backend support yet enabled
18:31:54 <ayoung> Quotas is on the BP list, and I think it has not had any activitry in a while
18:31:57 <henrynash> ayoung: agreed
18:32:03 <ayoung> https://review.openstack.org/#/c/40568/
18:32:27 <ayoung> oh, wait, 26 Aug...
18:32:37 <morganfainberg> also, he's been working on the API spec.
18:32:49 <ayoung> anyone know Dmitri's IRC handle?
18:32:50 <morganfainberg> which looks close.
18:32:58 <gyee> looks like no love from Jenkins either
18:33:09 <topol> -1s all around
18:33:18 <stevemar> ayoung: nope
18:33:24 <bknudson> it is work in progress
18:34:03 <topol> so it would seem to be on track to be deferred
18:34:04 <topol> , no?
18:34:28 <gyee> topol, unless he can get it done by code freeze
18:34:35 <morganfainberg> topol: it is looking like early Icehouse to me. but he might get it done.
18:34:45 <ayoung> yeah...extension, and would be good to have in, but no other core project  is going to be consuming it in Havana
18:34:57 <ayoung> retarget?
18:35:15 <jamielennox> based on the comments inline i'd suggest push to I
18:35:27 <topol> jamielennox +1
18:35:28 <morganfainberg> ayoung: +1, aim for early icehouse so core projects can work to consume it
18:35:41 <ayoung> henrynash, since meetbot doesn't listen to me, can you try #action ing that?
18:35:56 <stevemar> ayoung: +1, re-target
18:36:06 <ayoung> #action retarget quotas to Icehouse 1
18:36:06 <henrynash> #action defer https://review.openstack.org/#/c/40568/ to I
18:36:15 <ayoung> meh
18:36:24 <ayoung> it might be getting it...
18:36:25 * topol meetbot needs cpr
18:36:31 <henrynash> doesn't listen to me either
18:36:38 <ayoung> topol, maybe, or it might be doing it silently
18:36:46 <ayoung> we'll find out when we read the minutes
18:37:12 <henrynash> #action defer pagination to I
18:37:15 <gyee> somebody doing screen capture right?
18:37:19 <gyee> just in case
18:37:32 <ayoung> henrynash, the only high bugs on there have your name on them,
18:37:33 <stevemar> theres always the meeting log right?
18:37:45 <ayoung> ah, wait
18:37:51 <ayoung> bknudson, you have an AD related one
18:37:53 <henrynash> ayoung: so I'll be on them next
18:38:03 <bknudson> ayoung: https://review.openstack.org/#/c/41515/
18:38:04 <ayoung> #link https://launchpad.net/keystone/+milestone/havana-3
18:38:20 <morganfainberg> #link https://review.openstack.org/#/c/41208/
18:38:32 <ayoung> bknudson, looking now
18:38:35 <morganfainberg> if people are willing to put eyes on caching, i'd appreciate it.
18:38:55 <ayoung> morganfainberg, you got it
18:39:05 <gyee> morganfainberg, me 2
18:39:08 <morganfainberg> and the dependent changesets.
18:39:22 <jamielennox> morganfainberg, will look today
18:39:34 <henrynash> morganfainberg: agreed
18:39:36 <morganfainberg> i expect to have 1 or 2 more small changesets in the next day to add to the chain, but i need to sync w/ henrynash about filtering
18:40:01 <gyee> filtering, you thinking caching filtering?
18:40:32 <morganfainberg> gyee: talked with Dolphm, the get_* methods will be cached and aggressively invalidated as needed
18:40:53 <morganfainberg> gyee: the list_* methods will be cached, but may return slightly stale data (low cache_times, configurable)
18:40:54 <gyee> please don't cache filtering
18:41:11 <stevemar> gyee: what's the deal on endpoint filtering? https://review.openstack.org/#/c/33118/
18:41:13 <gyee> you are going to blow up the cache in the hurry
18:41:26 <stevemar> gyee: is it going in?
18:41:32 <ayoung> stevemar, yeah.
18:41:35 <gyee> stevemar, yes
18:41:48 <morganfainberg> gyee: i can avoid caching filters specifically. i'll talk w/ you after meeting so i'm on the same page.
18:41:54 <ayoung> stevemar, I'm giving it one last look after the meeting and then I'll approve, assuming all is well
18:42:17 <stevemar> ayoung: cool, it needs some +2s if we want it in
18:42:39 <morganfainberg> gyee: it's why i didn't implment for list methods yet.
18:42:57 <gyee> morganfainberg, yeah, list_* will have heavy impact on the cache
18:43:40 <gyee> if you are going to do it, make it optional and configurable
18:44:04 <ayoung> henrynash, I have someone that work on trying to replicate https://bugs.launchpad.net/keystone/+bug/1211445  if you have been unable to
18:44:06 <uvirtbot> Launchpad bug 1211445 in keystone "deleting an unassigned role causes 500" [High,Confirmed]
18:44:58 <henrynash> ayoung: (hey, hello meetbot) that would be good..both Dolphm and I failed to reproduce
18:45:45 <ayoung> henrynash, OK.  I'll reassign to him.
18:46:15 <henrynash> ayoung: although maybe that one ISN"t relate (as suggested) to the other…(which is the one we could not reproduce: https://bugs.launchpad.net/keystone/+bug/1210590
18:46:17 <uvirtbot> Launchpad bug 1210590 in keystone "Split backend crashes with AttributeError" [High,Invalid]
18:47:09 <ayoung> henrynash, I'll have him focus on just the "listing projects..." bug
18:47:16 <ayoung> anyting else high priority?
18:48:07 <jamielennox> ayoung, give me 5 min at the end for client stuff
18:48:09 <ayoung> gyee, filter endpoint based on scope https://blueprints.launchpad.net/keystone/+spec/endpoint-filtering  is covered by fabiogia 's patch right?
18:48:27 <dolphm_> (oops!) getting a new keystone contributor up to speed and just realized what time it was... hope ya'll are being productive ;)
18:48:31 <ayoung> showing as slow progress, but it should be about to get in
18:48:34 <ayoung> dolphm_, very
18:48:48 <stevemar> we were lost without our fearless leader
18:48:50 <ayoung> dolphm_, we voted to postpone quotas to I1.
18:48:57 <henrynash> there are 34 high priority bugs, or which about half have fixes: https://bugs.launchpad.net/keystone/+bugs?search=Search&field.importance=High&field.status=New&field.status=Incomplete&field.status=Confirmed&field.status=Triaged&field.status=In+Progress&field.status=Fix+Committed
18:49:22 <morganfainberg> bugs can be worked on between FF and RC?
18:49:26 <gyee> ayoung, yes
18:49:43 <ayoung> dolphm_, since quotas is still taggedas WIP, doubt it will be finished by tomorrow
18:49:53 <ayoung> morganfainberg, yes
18:49:54 <stevemar> morganfainberg: i would hope so
18:49:58 <dolphm_> ayoung: interesting
18:50:09 <ayoung> morganfainberg, they become blockers
18:50:20 <morganfainberg> ayoung: nod.
18:50:41 <morganfainberg> i expect to have soem time to start working on bugs here once we're past the mad rush to the FF
18:50:46 <ayoung> dolphm_, yeah.  Seems like quotas is close, but I don't think the other projects will be able to consume it, so it makes sense to defer
18:51:17 <ayoung> dolphm_, unless he was all ready to go and just working out minor kinks...do you know Dmitri's IRC handle?  we can ask him
18:52:52 <ayoung> I think we are thorugh the high-priority items for H3?
18:54:50 <ayoung> dolphm_, we spend a good amount of time discussing filters.  My vote is to postpone them, as we will only have SQL and not LDAP, and because it is, as I see it, high risk.  I won't -2 it, but recommend we postpone.  THere are many voices that disagree
18:55:10 <henrynash> dolphm: we agreed that pagination should be deferred, but there wa disagreement on filtering
18:55:47 <jamielennox> alright, so i realize that this will be a bit of a quiet period for client because everyone has a lot of reviews anyway but i want to poke people again to look at client as well (no feature freeze)
18:55:47 <ayoung> morganfainberg, if we cache at the driver level and filter at the controller level, we'll have a workable solution, no?
18:56:01 <morganfainberg> ayoung: i think so.
18:56:09 <morganfainberg> ayoung: it makes the caching easier too.
18:56:14 <jamielennox> particularly https://blueprints.launchpad.net/python-keystoneclient/+spec/auth-plugins which is the base stuff for aababilov's APIClient work
18:56:38 <gyee> jamielennox, +1
18:56:44 <morganfainberg> ayoung: but it doesn't solve the long load really hard hit back end (beyond some basic level) for the list_* methoids, like list_users
18:56:55 <jamielennox> it'd be good if people could at least have a skim through, the first two base patches are up and i think it's a good direction - it will make us much easier to consume
18:56:56 <gyee> jamielennox, that's the same impl as Nova's right?
18:57:07 <jamielennox> gyee, that's the intention
18:57:20 <ayoung> morganfainberg, it will be expensive the first time it is hit, and then cached, and then expensive again when the cache is invalidated, right?
18:57:33 <jamielennox> i want it developed in keystone first but and then we'll try to make it nova compatible - if we have to tweak 1 or 2 things in nova it should be ok
18:57:33 <henrynash> morganfainberg: yeah, we can't have a caching solution the does not allow filtering at the driver level (whether are do that in H or I)
18:57:40 <gyee> stevemar, I am guessing oath will make use of it?
18:58:03 <morganfainberg> ayoung: yes.
18:58:37 <stevemar> gyee: make use of caching?
18:58:46 <henrynash> 2 mins to go....
18:58:50 <gyee> stevemar, no, make use of pluggable auth in the client side
18:59:08 <gyee> the one jamielennox is working on
18:59:14 <jamielennox> gyee, stevemar i haven't looked at that
18:59:16 <stevemar> gyee: oh i'm a bit behind, sounds like it would be a great fit
18:59:20 <ayoung> morganfainberg, I suspect also that the caching will not be used by Apache.
18:59:28 <jamielennox> it sounds like a good fit though
18:59:52 <morganfainberg> ayoung: we can discuss that after the meeting, but not sure on that front.
19:00:05 <ayoung> so this is a a Puppet review, but topol and other HTTPD fans should look at it   https://review.openstack.org/#/c/29059/
19:00:06 <stevemar> jamielennox, i'm knee deep in keystoneclient right now, learning a lot, hopefully i can help you out a bit, soon
19:00:15 <henrynash> ok timesup
19:00:18 <jamielennox> stevemar, sounds good
19:00:28 <topol> ayoung, I will look
19:00:31 <dolphm__> ayoung: thanks! sounds rational.. I'm only on my phone, I'll probably have questions when I get back to my desk
19:00:35 <henrynash> #endmeeting