Thursday, 2015-04-23

« Wednesday, 2015-04-22 Index Friday, 2015-04-24 »
*** kebray has quit IRC00:05
*** chadlung has joined #openstack-barbican00:12
*** chadlung has quit IRC00:16
*** gyee has quit IRC00:23
*** zz_dimtruck is now known as dimtruck00:27
openstackgerritArun Kant proposed openstack/barbican: Adding ACL check when new stored key order is submitted  https://review.openstack.org/17655800:38
*** alee_ has quit IRC00:39
*** alee has joined #openstack-barbican00:39
*** rm_you has joined #openstack-barbican01:31
*** rm_you has quit IRC01:31
*** rm_you has joined #openstack-barbican01:31
*** david-lyle has quit IRC01:59
*** dave-mccowan has quit IRC02:24
*** dave-mccowan has joined #openstack-barbican02:25
*** alee_dinner is now known as alee_02:32
*** dimtruck is now known as zz_dimtruck03:08
*** rm_work is now known as rm_work|away04:06
*** rm_you| has joined #openstack-barbican04:07
*** rm_you has quit IRC04:07
*** alee has quit IRC04:08
*** alee_ has quit IRC04:08
*** alee has joined #openstack-barbican04:10
*** alee_ has joined #openstack-barbican04:11
openstackgerritDave McCowan proposed openstack/barbican: [WIP] Add ACL Functional Tests  https://review.openstack.org/17661504:29
*** david-lyle has joined #openstack-barbican04:59
*** alee_ has quit IRC05:17
*** alee has quit IRC05:18
*** rm_you| has quit IRC05:22
*** rm_you has joined #openstack-barbican05:22
*** alee_ has joined #openstack-barbican05:31
*** alee has joined #openstack-barbican05:33
openstackgerritDave McCowan proposed openstack/barbican: [WIP] Add ACL Functional Tests  https://review.openstack.org/17661505:44
*** dave-mccowan has quit IRC05:46
*** woodster_ has quit IRC06:40
openstackgerritMerged openstack/barbican: Fix failure with get on dict that was None  https://review.openstack.org/17610106:53
openstackgerritMerged openstack/python-barbicanclient: Cleaning up Keystone auth tests  https://review.openstack.org/17559707:11
openstackgerritMerged openstack/python-barbicanclient: Adding support for token based authentication  https://review.openstack.org/17559907:26
*** d0ugal has quit IRC09:29
*** d0ugal has joined #openstack-barbican09:29
*** d0ugal is now known as Guest8147209:29
*** Guest81472 is now known as d0ugal209:40
*** d0ugal2 is now known as d0ugal09:47
*** d0ugal has quit IRC09:47
*** d0ugal has joined #openstack-barbican09:47
*** darrenmoffat has quit IRC10:01
*** darrenmoffat has joined #openstack-barbican10:06
*** alkar has joined #openstack-barbican10:16
*** jaosorior has joined #openstack-barbican11:26
*** david-lyle has quit IRC11:29
*** dave-mccowan has joined #openstack-barbican11:53
*** nickrmc84 has quit IRC11:56
*** nickrmc83 has joined #openstack-barbican11:58
openstackgerritDave McCowan proposed openstack/barbican: [WIP] Add ACL Functional Tests  https://review.openstack.org/17661512:01
*** david-lyle has joined #openstack-barbican12:05
*** alee has quit IRC12:21
*** woodster_ has joined #openstack-barbican12:29
*** david-lyle has quit IRC12:31
*** rellerreller has joined #openstack-barbican12:42
*** zz_dimtruck is now known as dimtruck13:03
*** joesavak has joined #openstack-barbican13:23
*** dimtruck is now known as zz_dimtruck13:28
*** alee has joined #openstack-barbican13:41
*** stanzi has joined #openstack-barbican13:58
*** paul_glass has joined #openstack-barbican14:06
*** zz_dimtruck is now known as dimtruck14:40
*** xaeth_afk is now known as xaeth14:45
*** SheenaG has joined #openstack-barbican15:13
openstackgerritSteve Heyman proposed openstack/python-barbicanclient: Updates for CLI testing  https://review.openstack.org/17515015:15
openstackgerritSteve Heyman proposed openstack/python-barbicanclient: Updates for CLI testing  https://review.openstack.org/17515015:20
openstackgerritSteve Heyman proposed openstack/python-barbicanclient: Updates for CLI testing  https://review.openstack.org/17515015:21
*** silos has joined #openstack-barbican15:22
openstackgerritJohn Vrbanac proposed openstack/python-barbicanclient: Fixing misspelling in client docstring  https://review.openstack.org/17681615:24
openstackgerritOpenStack Proposal Bot proposed openstack/barbican: Updated from global requirements  https://review.openstack.org/17682115:27
*** david-lyle has joined #openstack-barbican15:29
*** rm_work|away is now known as rm_work15:29
*** igueths has joined #openstack-barbican15:33
openstackgerritSteve Heyman proposed openstack/python-barbicanclient: Updates for CLI testing  https://review.openstack.org/17515015:35
*** kebray has joined #openstack-barbican15:35
redrobotdave-mccowan I'm backporting https://bugs.launchpad.net/barbican/+bug/1446826 for RC215:37
openstackLaunchpad bug 1446826 in Barbican kilo "Get secret with ACL returns 500 Internal Server Error" [Critical,New] - Assigned to Douglas Mendizábal (dougmendizabal)15:37
*** ccneill has joined #openstack-barbican15:42
*** stanzi has quit IRC15:52
*** stanzi has joined #openstack-barbican15:52
*** tkelsey has joined #openstack-barbican15:55
*** david-lyle has quit IRC15:56
*** ccneill has quit IRC16:00
*** rellerreller has quit IRC16:03
*** gyee has joined #openstack-barbican16:03
*** kebray has quit IRC16:04
*** jsavak has joined #openstack-barbican16:04
*** kebray has joined #openstack-barbican16:04
*** joesavak has quit IRC16:06
*** rm_work is now known as rm_work|away16:11
*** arunkant_ has joined #openstack-barbican16:16
*** jsavak has quit IRC16:36
*** jaosorior has quit IRC16:42
dave-mccowanredrobot, lgtm16:48
dave-mccowanredrobt, quick RBAC question: Does a user in an ACL list need to have some role (like role==other) in the project associated with the secret?16:48
*** igueths has quit IRC16:55
*** igueths has joined #openstack-barbican16:59
*** alkar has quit IRC17:06
*** gyee has quit IRC17:07
*** alkar has joined #openstack-barbican17:07
*** silos has left #openstack-barbican17:08
arunkant_dave-mccowan: Currently No. User does not have to have barbican role. If user is in secret ACL list with read operation, then user can read the secret. There was discussion in weekly meeting to have barbican role requirement with ACL17:14
dave-mccowanarunkant_, thanks.  i'll push up a CR in a few minutes with some test cases.  hopefully you can take a look to see if my assumptions are right.17:17
openstackgerritDave McCowan proposed openstack/barbican: [WIP] Add ACL Functional Tests  https://review.openstack.org/17661517:21
dave-mccowanarunkant_ ^^^, if you look at test_acls.py lines 52 through 86, it details what i'm expecting back from a GET operation for each user, based on different ACLs.  do i have those right?17:23
*** rellerreller has joined #openstack-barbican17:25
*** alkar has quit IRC17:32
openstackgerritJuan Antonio Osorio Robles proposed openstack/barbican: Migrate to oslo_context  https://review.openstack.org/17533817:36
*** jaosorior has joined #openstack-barbican17:36
arunkant_dave-mccowan, in test_acls line# 56 returns 200 and line # 65 return 403 . Not sure why as it just has non-private acl (cannot tell whether auditor is allowed role or not)17:37
*** rm_work|away is now known as rm_work17:37
dave-mccowanarunkat_ auditor has role "audit".  if i understand the spec, they can never read, so should always return 403?17:39
arunkant_dave-mccowan, there are two different policy rules..one for reading secret metadata and then reading actual secret (decrypt ). decrypt does not allow auditor but metadata one does.17:41
arunkant_dave-mccowan, regardless of that..why it will change when we just add non-private acl on secret.17:42
dave-mccowanarunkant,_ i'll make that change: a user with audit role should always get 403 when trying to read a secret payload.  right?17:45
dave-mccowanarunkant_ is that the only issue you see?17:47
*** igueths has quit IRC17:48
arunkant_dave-mccowan, yes..I am assuming admin2, observer2 are users which does not have roles in secret project.17:49
dave-mccowanarunkant_ correct.17:50
*** stanzi has quit IRC17:57
*** stanzi has joined #openstack-barbican17:58
*** stanzi has quit IRC18:02
*** stanzi has joined #openstack-barbican18:02
dave-mccowanarunkant_  should a secret with no acl, have the same accessability as a secret with a non-private acl with no user list?18:03
*** tkelsey has quit IRC18:03
arunkant_dave-mccowan: Yes. Unless its private, ACL just adds additional users who can access it.18:04
*** david-lyle has joined #openstack-barbican18:07
dave-mccowanarunkant_ can a secret be private and add an ACL user?  so then, the creator and the listed users are the only ones with access?18:09
arunkant_alee, woodster_, as discussed yesterday about order acl bug, added review https://review.openstack.org/#/c/176558/ . Its failed at devstack with no obvious reason..trying recheck18:09
arunkant_dave-mccowan, correct18:11
*** gyee has joined #openstack-barbican18:15
*** SheenaG has quit IRC18:16
*** morganfainberg is now known as grebniafnagrom18:24
*** grebniafnagrom is now known as morganfainberg18:24
*** SheenaG has joined #openstack-barbican18:25
*** stanzi has quit IRC18:28
*** stanzi has joined #openstack-barbican18:28
*** gyee has quit IRC18:30
openstackgerritMerged openstack/python-barbicanclient: Fixing misspelling in client docstring  https://review.openstack.org/17681618:31
*** stanzi has quit IRC18:33
*** SheenaG has quit IRC18:35
*** kebray has quit IRC18:37
*** kebray has joined #openstack-barbican18:41
*** SheenaG has joined #openstack-barbican18:41
*** nickrmc83 has quit IRC18:53
*** nickrmc83 has joined #openstack-barbican18:54
*** SheenaG has quit IRC19:03
dave-mccowanhockeynut ping19:16
hockeynutdave-mccowan yessir19:16
dave-mccowanhockeynut when the gate runs functional tests, it runs them twice.  what's the difference between the two runs?19:17
hockeynutonce in parallel and once sequentially19:18
hockeynutlooking for concurrency issues19:18
dave-mccowanhockeynut, interesting.  so, if i have a bunch of failures on second the run, is that usually a barbican problem or a test case problem?19:19
hockeynutyes :-)  if a test fails and its a new test then I'd check to see that the test is good.  if its an existing test then I'd check to see what changed within barbican19:20
dave-mccowanhockeynut, it's both. :-) new tests for a new feature.19:21
dave-mccowanhockeynut, if you want to take a look: https://review.openstack.org/17661519:21
hockeynutI usually run locally and sequentially to see things pass.19:22
hockeynutclicking...19:22
*** ccneill has joined #openstack-barbican19:23
dave-mccowanhockeynut, i have a good handle on the 5 failures in sequential.  it's the 22 failures in parallel that i have this question on.19:23
hockeynutok, will peek at 'em19:23
*** chadlung has joined #openstack-barbican19:25
*** SheenaG has joined #openstack-barbican19:30
dave-mccowanarunkant_ i think there might be a problem with using ACL Lists.  in get_acl_dict_for_user(), the ctxt.user value is a UID (u'e8b7...3e4' in my case), but the acl_users.user_id is in "english" ('observer2' in my case)19:35
*** rellerreller has quit IRC19:43
openstackgerritMerged openstack/barbican: Updated from global requirements  https://review.openstack.org/17682119:44
*** ccneill has quit IRC19:46
*** alkar has joined #openstack-barbican19:48
*** jaosorior has quit IRC19:52
*** alkar has quit IRC19:54
*** stanzi has joined #openstack-barbican20:01
woodster_dave-mccowan: in reality, the user_id will be a unique key to the user in Keystone, not the actual user name. Is that what you are referring to though?20:01
*** tkelsey has joined #openstack-barbican20:01
dave-mccowanwoodster_, when i create a secret ACL like: {'read': {'users': ['observer2'], 'creator-only': False}}   should i be passing in 'observer2' as the user name?20:04
woodster_dave-mccowan: for a mocked unit test, or for a real functional test?20:04
dave-mccowanwoodster_ real deal20:05
woodster_dave-mccowan: if the latter, then you'd need the id of the actual user I believe20:05
woodster_dave-mccowan: so probably a call to keystone to get that user ID is needed, unless redrobot has a slicker way to do that....maybe the user-id can be overriden via keystone config somehow?20:06
*** tkelsey has quit IRC20:06
*** stanzi has quit IRC20:06
*** stanzi has joined #openstack-barbican20:07
redrobotdave-mccowan I haven't had a chance to look at your WIP yet, so you may have already done this20:08
redrobotdave-mccowan but the first thing you'll need to do is add keystone cli commands to create the 6 different users, since each devstack run is going to start with a brandh new database20:08
redrobotdave-mccowan https://github.com/openstack/barbican/blob/master/contrib/devstack/lib/barbican#L194 is the relevant bash function that needs to be edited20:09
dave-mccowanwoodster_  that would make the test case pass. :-)  i guess the API deals with secret IDs and container IDs, so the user ID (instead of user name) makes sense.20:09
dave-mccowanredrobot.  yep, did that.  i'm past that now.20:10
redrobotdave-mccowan so, at creation time, you can parse out the user IDs, and maybe set them as bash environment variables.20:10
redrobotdave-mccowan or you can use keystoneclient to auth and programatically get to the user ids20:11
dave-mccowanredrobot:   woodster_ and I just agreed that I should: {'read': {'users': ['134abdef4367328cdf32'], 'creator-only': False}} instead of {'read': {'users': ['reader_name'], 'creator-only': False}}20:11
*** stanzi has quit IRC20:11
redrobotdave-mccowan yep, I agree as well.20:11
woodster_dave-mccowan: that user ID will change from run to run though, so I don't think you could hard code that if that's what you mean?20:12
woodster_dave-mccowan: well, I'm assuming the ID is generated by the keystone db20:13
dave-mccowanredrobot.  cool.   after i fix that, all tests should pass in the gate when run in serial.  but, several will still fail in parallel.   maybe a critical section failure in the ACL repos?20:13
redrobotwoodster_ it is20:13
woodster_dave-mccowan: it could also be a clean up issue with the parallel tests...hockeynut, tdink wasn't an original issue with parallel testing related to deleting test entities too soon?20:15
tdinkwoodster_: not exactly had to do with the paging tests running in parallel and it would mess with the pref/next ref20:24
tdinkso when secrets were created or deleted those paging tests would freak out sometimes20:24
*** kebray has quit IRC20:26
woodster_arunkant_: I added a comment to your CR please: https://review.openstack.org/#/c/176558/20:28
hockeynutwoodster_ I believe the problem was that a single user was adding/removing secrets and that confused the (parallel) list tests that were counting secrets20:29
woodster_hockeynut: tdink that all make sense then20:31
woodster_dave-mccowan: interesting that the test take a lot longer under the parallel testing mode: http://logs.openstack.org/15/176615/4/check/gate-barbican-devstack-dsvm/ab14725/console.html#_2015-04-23_18_31_57_10420:31
dave-mccowanwoodster_ lots of interesting stuff in these tests. :-)   the new errors are when setting an ACL for a new secret or container.  the message is "already have an ACL"20:34
woodster_dave-mccowan: hockeynut tdink is it easy to test the parallel mode locally? I've only run tox -e functional locally20:34
woodster_dave-mccowan: hockeynut tdink it would be good to get at the server-side errors20:34
hockeynutwoodster_ testr run --parallel --subunit | subunit-trace --no-failure-debug -f20:35
tdinkwoodster_: if i recall properly i couldnt get these errors running tox, had to run testr20:35
tdink....what hockeynut said20:36
dave-mccowanwoodster_ for the server side error, search on HTTPClientError: Existing ACL cannot be updated with POST method.  in the barbican screen in the log.20:36
hockeynutin functionaltests/run_tests.sh you can see how they are run in devstack20:36
hockeynutdave-mccowan so its doing a lookup for a count of acls and getting back a > 0 value?20:40
dave-mccowanhockeynut, that's what i'm reading too.  the traceback points to that line of code, and the exception message matches.20:42
hockeynutbut this doesn't happen locally?20:42
hockeynutsounds like reusing a container id perhaps?20:43
hockeynutcontainer ID on post is 8f3bcc82-7491-4595-9495-6320177eee2b and I only see that once in the log20:44
hockeynutits interesting that the failures are grouped - I wonder if there is a race condition within the acl POST20:46
hockeynutfor a quick+dirty test you could try serializing the POST code - that could help narrow down20:47
*** SheenaG has quit IRC20:48
hockeynutnone of those errors happened during the serialized run, only during the parallel run.20:49
woodster_hmmmm...I see an odd query for the count:20:49
woodster_https://www.irccloud.com/pastebin/hbHkmZJ720:49
woodster_I think that just counts all the container ACL records, not just for the one we are interested in20:50
woodster_arunkant: arunkant_ I think that query needs an optional filter for the container ID20:51
hockeynutdave-mccowan might be good to put out that count in a debug msg so you can see it in the log20:51
woodster_dave-mccowan: I'm pretty sure that query is not correct. Sequential tests will always clear out the container before reating the next one so you'll only have one ACL record. In parallel more than one is likely20:52
dave-mccowanhockeynut, i'll try.  sounds like a good way to "fix" the problem, if it a race condition. :-)20:52
hockeynutputting on my performance guy hat.  NOOOOO!20:53
hockeynuttaking off my performance guy hat now.20:53
dave-mccowanwoodster_  there's an interesting test:  create 3 secrets, then add an ACL to each.20:54
woodster_dave-mccowan: that query needs to accept a container_id, and then do:  if container_id:  query = query.filter(models.ContainerACL.container_id == container_id)20:55
woodster_dave-mccowan: that needs to be put in both the SecretACLRepo and ContainerACLRepo's get_count() methods20:56
woodster_dave-mccowan: then that container_id/secret_id needs to be passed in from the secret and container acl controller get_count() calls20:57
woodster_hockeynut: tdink score +1 for the parallel tests!20:58
hockeynutw00t w00t!20:58
hockeynutcc: every manager ever :-)20:58
woodster_dave-mccowan: can you give all that a try with that CR then?20:58
dave-mccowanwoodster_ will do now20:59
*** kebray has joined #openstack-barbican21:03
*** chadlung has quit IRC21:14
*** SheenaG has joined #openstack-barbican21:20
*** chadlung has joined #openstack-barbican21:26
openstackgerritDave McCowan proposed openstack/barbican: [WIP] Add ACL Functional Tests  https://review.openstack.org/17661521:29
dave-mccowanwoodster_ hockeynut, tdink ^^ test underway21:31
*** alee has quit IRC21:33
*** chadlung has quit IRC21:39
*** gyee has joined #openstack-barbican21:48
*** rm_work is now known as rm_work|away21:55
woodster_dave-mccowan: a watched gate job never finishes22:02
*** xaeth is now known as xaeth_afk22:03
openstackgerritMerged openstack/python-barbicanclient: Cleaning up validate_ref()  https://review.openstack.org/17560522:05
dave-mccowanwoodster_ :-)  it'll give me time to code the other fix too.22:08
*** openstackstatus has quit IRC22:09
arunkant_woodster_, thanks for review, will refactor in next patch22:15
*** paul_glass has quit IRC22:15
*** ccneill has joined #openstack-barbican22:17
*** tkelsey has joined #openstack-barbican22:17
ccneillhey guys, any thoughts on my comments here on the keystone middleware signing_dir question? https://review.openstack.org/#/c/176071/22:18
*** greghaynes has quit IRC22:18
*** tkelsey has quit IRC22:22
redrobotccneill I'm OK with leaving it off, as per my +2.  Juan works out of Finland, so he probably won't see your comment until early AM for us.22:22
ccneillah gotcha22:22
redrobotccneill maybe you can poke woodster_ for a review22:22
* ccneill pokes woodster_22:23
ccneill:)22:23
ccneillyeah, I mean the worst case scenario is that someone has to set their own signing_dir value, which I could call out in the config with a comment if you think that'd help22:23
redrobotccneill yeah, the comment in the linked Nova cr was pretty helpful.22:24
ccneillk I'll go ahead and add that to my commit22:25
*** arunkant_ has quit IRC22:27
openstackgerritCharles Neill proposed openstack/barbican: Removing signing_dir directive from config  https://review.openstack.org/17607122:30
*** alee_ has quit IRC22:30
openstackgerritCharles Neill proposed openstack/barbican: Removing signing_dir directive from config  https://review.openstack.org/17607122:31
*** chadlung has joined #openstack-barbican22:40
*** chadlung has quit IRC22:41
*** chadlung has joined #openstack-barbican22:41
*** alee has joined #openstack-barbican22:43
*** alee_ has joined #openstack-barbican22:44
woodster_ccneill: lgtm!22:45
ccneillshweet22:45
*** kebray has quit IRC22:51
*** arunkant_ has joined #openstack-barbican22:55
arunkant_woodster_, you are right, get_count ACL query need to use id filter. Is there a bug for that. Will fix it soon.23:06
*** chadlung has quit IRC23:07
*** arunkant_ has quit IRC23:18
*** dimtruck is now known as zz_dimtruck23:34
woodster_arunkant: that sounds good23:35
woodster_dave-mccowan: Please see my comments on https://review.openstack.org/#/c/17661523:36
dave-mccowanwoodster_ yep, i've got it working now, but looks like arunkant_ just volunteered to take it23:38
woodster_dave-mccowan: did you want to rebase on that fix for your CR then?23:43
dave-mccowanwoodster_  yea, they need to be separate.  i assume the bug fix will be backported, and the test cases not.23:44
woodster_arunkant: I'd think the full query for count could be:23:45
woodster_https://www.irccloud.com/pastebin/UBSnl4Cs23:45
dave-mccowanwoodster_, arunkant_  this is what i ended up with, getting the models right too:    http://www.fpaste.org/214995/32804142/23:47
woodster_dave-mccowan: yeah that looks good23:47
woodster_dave-mccowan: so I think the only thing missing was passing the id into the get_count calls...so not much else arunkant would need to do then?23:48
woodster_dave-mccowan: arunkant ...assuming your CR landed that is Dave?23:49
dave-mccowanwoodster_, arunkant, yes i've got it done23:50
« Wednesday, 2015-04-22 Index Friday, 2015-04-24 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!