18:02:26 <heckj> #startmeeting
18:02:27 <openstack> Meeting started Tue May 15 18:02:26 2012 UTC.  The chair is heckj. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:02:28 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:02:56 <heckj> Big topic for today: rafaduran discussing https://bugs.launchpad.net/keystone/+bug/963098 and related blueprint
18:02:58 <uvirtbot> Launchpad bug 963098 in keystone "Keystone isn't acting on consecutive failed logins" [High,Triaged]
18:03:04 <heckj> #topic  https://bugs.launchpad.net/keystone/+bug/963098
18:03:18 <heckj> why don't we start there - rafaduran, you good with that?
18:03:30 <rafaduran> no problem
18:04:03 <rafaduran> as bug https://bugs.launchpad.net/keystone/+bug/963098 reported keystone is not acting on consecutive login fails
18:04:04 <uvirtbot> Launchpad bug 963098 in keystone "Keystone isn't acting on consecutive failed logins" [High,Triaged]
18:04:29 <rafaduran> however I think the real problem is that keyhstone is no acting on any suspicious activity
18:04:42 <ayoung> \O/
18:04:54 <rafaduran> thus I've registered a blueprint for that https://blueprints.launchpad.net/keystone/+spec/improve-keystone-security
18:05:13 <rafaduran> that show how I think keyston should manage this situation
18:05:33 <heckj> #link https://blueprints.launchpad.net/keystone/+spec/improve-keystone-security
18:05:43 <rafaduran> there is also a working draft for this https://review.openstack.org/#/c/7239/
18:06:12 <ayoung> rafaduran, think that link is wrong
18:06:32 <rafaduran> ayoung: which one?
18:06:41 <ayoung> https://review.openstack.org/#/c/7239/
18:06:50 <ayoung> I'm getting a 404
18:06:51 <joesavak> 404 on that
18:06:51 <heckj> https://review.openstack.org/#/c/7239/ is returning a "NotFound"
18:06:57 <joesavak> lol
18:07:01 <rafaduran> not for me
18:07:09 <dolphm> same
18:07:15 <dolphm> (404)
18:07:19 <ayoung> Its too secure
18:07:25 <liemmn> lol
18:07:30 <ayoung> rafaduran, is it draft only?
18:07:38 <rafaduran> ayoung: yes
18:07:48 <ayoung> I think you need to publish or perish
18:08:04 <rafaduran> published
18:08:16 <rafaduran> is working now?
18:08:28 <dolphm> yep
18:08:28 <joesavak> yup
18:08:38 <joesavak> so experation, tolerance, etc are global configs?
18:08:45 <ayoung> #link  https://review.openstack.org/#/c/7239/
18:08:53 <clarkb> you can also add specific people to drafts (if you still want to keep it mostly secret)
18:09:28 <rafaduran> joesavak: no they are specific for a middleware that send emails on 401 errors
18:10:04 <rafaduran> joesavak: that middleware is just an example but it also solves original bug
18:10:11 <joesavak> ok
18:10:51 <dolphm> i assume this all should be disabled out of the box?
18:11:25 <rafaduran> doesn't matter
18:12:09 <ayoung> dolphm, I think that the middle ware must be explicitly added to the pipeline in order to work
18:12:18 <ayoung> but by default it would be left out?
18:12:53 <rafaduran> If you think if should be disabled by default i can disable in etc/keystone.conf.sample
18:13:33 <rafaduran> if everything looks good I will add some tests and updated docs how to enable it
18:13:44 <ayoung> rafaduran, so is it "all or nothing"?
18:13:58 <ayoung> Or can you turn on a limited subset of Functionaliry
18:14:00 <ayoung> ity
18:14:51 <rafaduran> the backends are not activate unless a middleware use them
18:14:55 <dolphm> should be removed from the pipeline out of the box; doc'd as to how to enable it; most config should be commented out with assumed defaults in case it remains commented out; 'admin_emails' needs to be renamed, as 'admin' is very overloaded and the existing usage doesn't apply here
18:15:33 <rafaduran> dolphm: ok
18:16:24 <dolphm> not clear on what the 'secinfo' table is storing -- is it a log in sql?
18:17:31 <rafaduran> not it stores information about requests/responses, so a middleware can query later and thus track suspicious activity
18:17:33 <joesavak> id, response code, request mothod, path, date, response body - look like it's an audit trail
18:17:50 <joesavak> not sure what "extra" is. Just a col to store notes?
18:17:56 <dolphm> rafaduran: it doens't appear to store statistics, are they computed per request then?
18:19:03 <rafaduran> dolphm: yes, e.g: the MailConsecutive401 middleware store information if response is 401 and then query the 401 in a certain period
18:19:56 <rafaduran> dolphm: then if 401 errors are greater than a given number (tolerance) it will send an email
18:21:30 <joesavak> +1 - i like it
18:22:16 <dolphm> so the 'security' driver implements a queryable request/response log; the middleware provides email-based alerts (but doesn't actually throttle, right?)
18:22:43 <joesavak> right - not acting as repose or a rate-limit. Just notificaiton. Back-end could be atom-hopper
18:22:53 <rafaduran> yes
18:23:07 <rafaduran> the rate limit would be the second part
18:23:18 <zykes-> uhm, vishy still getting: /usr/bin/qemu-system-x86_64 -S -M pc-1.0 -no-kvm -m 4096 -smp
18:23:23 <zykes-> whyyyyyy
18:23:37 <rafaduran> but as I mentioned at bp ayoung at last meeting said that it probably conflict http work
18:23:41 <vishy> zykes-: this is the meeting room
18:23:43 <heckj> zykes-: wrong channel… :-)
18:23:44 <dolphm> zykes-: wrong channel?
18:23:50 <joesavak> raf - you may want to look at http://openrepose.org/ for rate-limiting
18:23:50 <rafaduran> so I don't work on that after httpd work is done
18:23:57 <zykes-> ah, wrong chan :)
18:24:26 <ayoung> rafaduran, maybe,  but also seems like you could use either.
18:24:38 <ayoung> It might be possible to Rate limit this way
18:25:05 <rafaduran> ayoung: you mean using middlewares?
18:25:05 <ayoung> But...
18:25:31 <ayoung> if each request is handled by a separate process...it would need to be dealt with at the HTTPD level
18:26:06 <ayoung> so I think, while this might  function OK, it wouldn't actually protect against a DOS or  cracking attempt...
18:26:17 <ayoung> unless it hits a DB table each time?
18:26:51 <rafaduran> yes it queries DB for each request if response is 401
18:27:14 <ayoung> I like the idea of using middlewares for it.  I think it will work ok then against the cracking attempt,  just not against a DOS
18:27:25 <ayoung> but that is still better than nothing
18:27:29 <ayoung> by a long shot
18:27:34 <dolphm> rafaduran: if the response is 401, isn't it too late to rate limit?
18:27:45 <dolphm> the dos has already succeeded at that point
18:28:04 <rafaduran> dolphm: as I said right now I'm not doing rate limiting, just reporting
18:29:07 <joesavak> rate limiting should be split outside of keystone, imo
18:29:14 <dolphm> 'security' is probably too broad of a driver name, then... it's really just very specific monitoring
18:29:22 <liemmn> I think this will increase the network chattiness for the middleware quite a bit... When I think of rate limiting, it is better done at the server, like a rest proxy.  Why can't we configure the server to optionally send notification, rather than involving the middleware?
18:29:28 <dolphm> i.e. there's no additional security
18:29:34 <rafaduran> dolphm: I like monitoring
18:29:46 <dolphm> rafaduran: very specific monitoring
18:30:36 <dolphm> liemmn: +1
18:31:41 <rafaduran> liemmm: I think the idea is keystone itself can handle it, but of course you can provide same thing using others approachs
18:32:06 <dolphm> heckj: thoughts
18:32:07 <dolphm> ?
18:32:38 <liemmn> I am just concerned that we need to keep the middleware "lean and mean"...  A lot of token (and later signature) validation is going through it already...
18:32:43 <heckj> I think having an optional middleware that does ratelimiting is just fine
18:33:12 <heckj> If we also want to enable some simple audit&report mechanism, then I think that might be best as a separate middleware
18:33:41 <joesavak> fyi - rackspace developed this and made rate-limting configurable through a couple API calls. I can share some doc on that if it would help.
18:33:49 <heckj> per liemmn comment's we want to be very careful about keeping the /token API piece performance
18:33:57 <dolphm> heckj: that's what this is - audit & reporting via email
18:34:05 <liemmn> Audt trails have to be secure... It's easier to secure it on the keystone server than on a bunch of service nodes... (IMO)
18:34:30 <joesavak> liemn - +1 - especially since you're putting in the path which could include tokens
18:34:56 <dolphm> liemmn: referring to the 'secinfo' table as the audit trail?
18:36:03 <ayoung> heckj, that is why the PKI Blueprint...the fastest code is that which doesn't have to run....
18:36:12 <liemmn> no... I think that is not the audit trail right?  That's just for checking the rate of failure....  If we want to implement audit trails, I am just saying it should be done on the server side.
18:37:11 <heckj> ayoung: word
18:37:30 <dolphm> ayoung: i'm a fan :)
18:37:47 <liemmn> NOOP is the fastest :)
18:38:13 <liemmn> and bug-free
18:38:15 <ayoung> I updated the PKI Blueprint to make it clearer.  I can talk through it if anyone wants
18:38:16 <dolphm> liemmn: noop takes a cycle :P
18:38:35 <ayoung> if we are done with rafaduran 's code?
18:38:47 <dolphm> ayoung: yeah, we can carry that into code review
18:38:50 <liemmn> dolphm:  I am talking about code ;)
18:38:51 <heckj> rafaduran: did you get the feedback you were after?
18:39:14 <rafaduran> I think so, the reporting seems is not going to work
18:39:29 <rafaduran> but then we are not addressing the original bug
18:40:03 <dolphm> rafaduran: cite the bug in your commit msg, and clean up the Change-Id's plz
18:40:20 <dolphm> i didn't even know this was for a bug
18:40:24 <rafaduran> #link https://bugs.launchpad.net/keystone/+bug/963098
18:40:25 <uvirtbot> Launchpad bug 963098 in keystone "Keystone isn't acting on consecutive failed logins" [High,Triaged]
18:41:00 <rafaduran> I've explained it at the start
18:41:28 <dolphm> rafaduran: sorry, i jumped straight to the review
18:41:36 <rafaduran> np
18:42:37 <ayoung> Am I up?
18:42:37 <rafaduran> I think I will re-think this and try something not using middlewares
18:42:52 <rafaduran> ayoung: yes
18:42:57 <ayoung> OK  updated the write up here: http://wiki.openstack.org/PKI  specifically the section  "Delegation and Scaling"
18:43:09 <heckj> #topic: PKI ness
18:43:15 <dolphm> lol
18:43:16 <heckj> #link http://wiki.openstack.org/PKI
18:43:33 <ayoung> I had to find out from some people smarter than me what the right API to use was
18:43:38 <ayoung> turns out it is called CMS
18:43:44 <ayoung> Crypt Message Syntax
18:43:50 <ayoung> the short of it is this
18:44:09 <ayoung> when you get a token,  there is a part of the response that lists the tenant and roles
18:44:26 <ayoung> so  the data is something like
18:44:58 <ayoung> {user: ayoung,  tenant: coop-city,  role: hallmonitor, groundskeeper}
18:45:20 <ayoung> you then take that date and sign it by encrypting it with a private key
18:45:47 <ayoung> someone else with the corresponding public key can decrypt it and know that it was encrypted by the private key
18:46:07 <ayoung> now this key-pair is going to be from Keystone,  thus validating the name Keystone
18:46:36 <ayoung> and so  when you get a token signed like this,  something like Nova won't have to go back to Keystone in order to authenticate
18:47:05 <ayoung> now,  the expiration date needs to be in there,  and there are details about key management,  but that the TL:DR version
18:47:10 <dolphm> (this is the exciting part)
18:47:38 <ayoung> dolphm, sorry,  that was the exciting part.
18:47:51 <ayoung> the rest is boring details
18:47:52 <dolphm> ayoung: i meant nova not having to go back to keystone :)
18:48:04 <dolphm> ayoung: it's like magic scalability
18:48:39 <ayoung> it will put a little more CPU load on the services,  but I suspect that what is saved in terms of network and SSL back to Keystone will more than make up for it
18:48:45 <dolphm> agree
18:48:49 <dolphm> ayoung: cpu time is cheap
18:49:04 <ayoung> If we really are concerned about CPU time for this,  we've won...
18:49:37 <dolphm> ayoung: so, this all fits with the existing X-Subject-Token header, it'll just be encrypted?
18:49:41 <ayoung> My next task is to get a proof of concept working using the command line tools
18:49:44 <dolphm> err X-Auth-Token**
18:49:44 <ayoung> dolphm, yes
18:49:52 <heckj> ayoung: with the API pieces you've found, is it possible to have multiple signers
18:49:56 <ayoung> I think the X-Auth-Token will just be huge
18:50:10 <ayoung> heckj, yes
18:50:21 <dolphm> ayoung: how much bigger than the data it contains?
18:50:32 <ayoung> heckj, so say there are 3 keystone servers,  each has a different key. That needs to be part of the token
18:50:34 <ayoung> dolphm, 1K
18:50:55 <ayoung> I think that is the nature of using the encryption
18:50:56 <dolphm> ayoung: is there a maximum length on header values?
18:51:06 <ayoung> I think we are OK.  I'll confirm
18:51:16 <ayoung> but I've seen some that are huge
18:51:16 <dolphm> (first google: http://stackoverflow.com/questions/686217/maximum-on-http-header-values )
18:51:50 <heckj> ayoung: I believe you're OK, and the token as it stands today is already a string (albiet much smaller). It makes a nice add-on extra-fit
18:52:04 <ayoung> dolphm, to answer your question "the HTTP spec does not define a limit, however many servers do by default. This means, practically speaking, the lower limit is 8K."
18:52:55 <ayoung> heckj, yes.  The idea is that we should be able to switch out Keystone first, and then the rest.  However, I don't know if /tokens/{id} will allow the super long ones either....
18:53:09 <ayoung> but if we go with the scheme that token auth is in the page as opposed the URL we should be OK
18:53:24 <ayoung> and having tokens in the URL is problematic for other reasons already stated
18:53:41 <heckj> ayoung: yp, with ya
18:54:20 <dolphm> heckj: does this have potential to be core for v3?
18:54:32 <heckj> dolphm: absolutely
18:54:45 <dolphm> heckj: that would be awesome if we can do it
18:55:23 <ayoung> dolphm, I'll try to have a demo for next week,  totally hand jammed,  but should show the flow
18:55:36 <heckj> ayoung: that would be excellent
18:56:02 <heckj> We've got 5 minutes left. Any more questions or feedback for ayoung?
18:56:20 <ayoung> I'd like to use python-nss as the Crypto library,  and it will need to have the CMS API added to it.  CMS is in the native.  Fortunatly,  I've talked with the maintainer and he is more than willing to add it.
18:57:16 <heckj> #topic open discussion
18:57:47 <rafaduran> I've have some patches that need review again...
18:57:47 <heckj> Administrative bits - there's quite a number of reviews pending for Keystone, with lots of new/good work. Please go take a look and put in your thoughts on the code and such
18:57:57 <rafaduran> #link https://review.openstack.org/#/c/6425/
18:58:06 <rafaduran> #link https://review.openstack.org/#/c/7127/
18:58:07 <heckj> #link https://review.openstack.org/#/q/status:open+keystone,n,z
18:58:29 <heckj> I'm a bit behind on bug triage, but will be going through it this week as well
19:00:09 <heckj> And… that's it for this week!
19:00:13 <heckj> #endmeeting