Monday, 2015-04-06

*** paul_glass has quit IRC01:48
*** woodster_ has joined #openstack-barbican02:20
*** kebray has joined #openstack-barbican02:28
*** kebray has quit IRC02:28
*** kebray has joined #openstack-barbican02:37
*** kebray has quit IRC02:38
*** kebray has joined #openstack-barbican02:39
*** chlong has joined #openstack-barbican03:00
*** paul_glass has joined #openstack-barbican03:48
*** paul_glass has quit IRC03:53
*** woodster_ has quit IRC05:40
*** paul_glass has joined #openstack-barbican05:49
*** paul_glass has quit IRC05:53
*** kebray has quit IRC06:34
*** jamielennox|away is now known as jamielennox06:48
*** tkelsey has joined #openstack-barbican07:59
*** tkelsey has quit IRC08:04
*** chlong_ has joined #openstack-barbican08:54
*** chlong has quit IRC08:57
*** tkelsey has joined #openstack-barbican08:58
*** tkelsey has quit IRC10:06
*** darrenmoffat has quit IRC10:19
*** Nirupama has joined #openstack-barbican10:20
*** darrenmoffat has joined #openstack-barbican10:20
*** chlong_ has quit IRC10:41
*** chlong_ has joined #openstack-barbican10:58
*** chlong_ has quit IRC11:05
*** chlong has joined #openstack-barbican11:24
-openstackstatus- NOTICE: gerrit has been restarted to restore event streaming. any change events missed by zuul (between 10:56 and 11:37 utc) will need to be rechecked or have new approval votes set11:38
*** dave-mccowan has joined #openstack-barbican11:38
*** jamielennox is now known as jamielennox|away11:49
*** woodster_ has joined #openstack-barbican12:00
*** Nirupama has quit IRC12:32
*** tkelsey has joined #openstack-barbican12:44
*** tkelsey has quit IRC12:50
*** nkinder has quit IRC13:21
dave-mccowanwoodster_, alee, ping.  shall we talk about validitor calls from orders.py?  john makes a good point in code review comment.13:44
aleewoodster_, dave-mccowan - I was just responding to that.13:46
aleewoodster_, dave-mccowan the initial problem here is that we needed to get the external_project_id so that we could find the containers and secrets13:47
alee(and also validate whether the caller has access to the resources or not)13:47
aleewoodster_, dave-mccowan - as far as I see, that information is not passed to the validators13:48
aleewoodster_, dave-mccowan - in that validator there are two things we need to do ..13:49
*** ametts has joined #openstack-barbican13:49
aleeI'm going to add this to the review - so I dont have to repeat myself ..13:50
woodster_Good point...I think it would be fair to optionally pass the project id into the api.load_body call for validators that need that13:51
dave-mccowanalee, woodster, and validate-stored-key needs external_project_id.  validate-ca-id needs internal project.id.  to validate with ACL, will we also need a user id?  john instinct is right... this section might get beefier, but i don't think the validator call in line 194 can contain it all.13:51
aleedave-mccowan, woodster_ - I agree that the better solution would be pass all these to the validators in the api.load_body call.13:52
*** gitorres has quit IRC13:53
*** david-lyle has quit IRC13:53
*** elmiko has quit IRC13:53
*** reaperhulk has quit IRC13:53
*** d0ugal has quit IRC13:53
*** tdink has quit IRC13:53
*** hockeynut has quit IRC13:53
*** mordred has quit IRC13:53
*** jvrbanac has quit IRC13:53
*** zz_dimtruck has quit IRC13:53
*** jillysciarilly has quit IRC13:53
*** lbragstad has quit IRC13:53
*** jroll has quit IRC13:53
*** eglute has quit IRC13:53
*** darrenmoffat has quit IRC13:53
aleewoodster_, as I have work that depends on parts of this cr, can we workflow this in and have dave correct it in a separate CR?13:55
*** tdink has joined #openstack-barbican13:55
*** hockeynut has joined #openstack-barbican13:55
*** mordred has joined #openstack-barbican13:55
*** jvrbanac has joined #openstack-barbican13:55
*** zz_dimtruck has joined #openstack-barbican13:55
*** jillysciarilly has joined #openstack-barbican13:55
*** lbragstad has joined #openstack-barbican13:55
*** jroll has joined #openstack-barbican13:55
*** eglute has joined #openstack-barbican13:55
*** darrenmoffat has joined #openstack-barbican13:56
*** paul_glass has joined #openstack-barbican13:57
*** gitorres has joined #openstack-barbican13:58
*** david-lyle has joined #openstack-barbican13:58
*** elmiko has joined #openstack-barbican13:58
*** reaperhulk has joined #openstack-barbican13:58
*** d0ugal has joined #openstack-barbican13:58
*** paul_glass has quit IRC14:00
*** paul_glass has joined #openstack-barbican14:01
woodster_alee, that works for me14:02
aleewoodster_, reaperhulk cool14:03
aleewoodster_, dave-mccowan - that is ..14:03
dave-mccowanwoodster_. alee, shall i address the comments in hrefs before or after merge?  either way is fine with me.  impact is on alee.14:05
woodster_dave-mccowan, I'm fine fixing in following CRs14:06
woodster_dave-mccowan, alee, so we are fine workflowing htis CR then?14:07
aleewoodster_, sounds good to me.14:14
aleewoodster_, you need to do it ..14:14
aleeand then dave-mccowan will fix in a follow-on CR14:15
woodster_done, just now14:15
dave-mccowanalee, woodster_, while it's fresh in our minds.  let me ask about moving meta validation logic from orders.py.  that code needs the parsed body that api.load_body() returns to extract the 'meta' parameters.  so, i could move the existing code to api.load_body(), but that seems to just move the uglyness.  i propose: add a new method, meta_validator(), so the new line 194 would be body = api.load_body(pecan.request, validator=se14:16
dave-mccowanlf.type_order_validator, meta_validator=self.type_order_meta_validator).  then all the validation logic can live in validators.py.14:16
aleewoodster_, in Namibia, we also have the concept of "just now" -- it means something entirely different like "I will do X just now" - which could mean anytime within the next 15 mins or the next week or so.14:17
aleewoodster_, you have to distinuguish it with from more immediate timeframes -- like "now now" or "south african now" or "american now"14:19
woodster_dave-mccowan, this still seems like something that could be done in the validators side...i.e. the order validator should know that there is a meta section to validate as well?14:19
*** nkinder has joined #openstack-barbican14:19
aleewoodster_, or in this case, "just now" meaning that which has just passed14:19
woodster_alee, do mean as TODO comments?14:19
aleewoodster_, sorry -- whimsical aside -- reading dave-mccowan question now14:20
*** paul_glass has quit IRC14:21
*** paul_glass has joined #openstack-barbican14:21
aleedave-mccowan, sorry - I'm a little confused -- what code are you trying to move?14:26
aleedave-mccowan, I kinda agree with woodster_ , not sure I see the need for an extra validator argument14:27
dave-mccowanalee, woodster_, yea... i'm with woodster_ too.  all my code should have gone in validators.py.14:27
dave-mccowanalee, woodster_, does that apply to validate_ca_id() too?  that was the example i was following.14:28
*** zz_dimtruck is now known as dimtruck14:29
aleedave-mccowan, most likely yes -- that one is there also because I needed a projct_id I think.14:30
dave-mccowanalee, woodster_ thanks john.  excellent review.  i know what to do.  i'll fix it just now. ;-)14:32
*** igueths has joined #openstack-barbican14:32
aleedave-mccowan, which version of "just now"? :)14:33
openstackgerritMerged openstack/barbican: Implement validators and tests for stored key certificate orders  https://review.openstack.org/16729114:38
*** chlong has quit IRC15:00
*** xaeth_afk is now known as xaeth15:34
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack  https://review.openstack.org/17089615:35
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack  https://review.openstack.org/17089615:43
*** gyee has joined #openstack-barbican15:45
*** ametts has left #openstack-barbican15:46
*** xaeth is now known as xaeth_afk15:46
*** xaeth_afk is now known as xaeth15:55
*** dougwig has left #openstack-barbican16:00
*** xaeth is now known as xaeth_afk16:12
*** xaeth_afk is now known as xaeth16:20
openstackgerritJohn Wood proposed openstack/barbican: Add order_retry_tasks migration per latest model  https://review.openstack.org/16994616:20
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack  https://review.openstack.org/17089616:43
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack  https://review.openstack.org/17089616:47
woodster_jvrbanac please look at my comments on this CR: https://review.openstack.org/#/c/169946/217:06
jvrbanacwoodster_, give me a few17:07
jvrbanacwoodster_, ok17:14
woodster_jvrbanac, I'll set the order the same for this CR though. Also adding guidance for new columns to the sphinx docs.17:16
*** kebray has joined #openstack-barbican17:17
jvrbanacwoodster_, ahh ok17:18
jvrbanacwoodster_, btw, at some point, I'm probably gonna change how we do model registration. I was working with SQLAlchemy over the weekend on a different project and realized there is a much more efficient way of doing it these days.17:19
woodster_jvrbanac, sounds great! Yeah the current approach used hasn't change in over two years :)17:20
*** rm_work is now known as rm_work|away17:29
arunkantredrobot : Can remaining ACL part (4 and 5) be reviewed and possibly merged?17:29
aleereaperhulk, ping17:39
*** paul_glass has quit IRC17:50
*** paul_glass has joined #openstack-barbican17:51
*** jkf has joined #openstack-barbican17:53
*** dave-mccowan has quit IRC18:07
openstackgerritMerged openstack/barbican: Restore worker tasks processing catching exceptions  https://review.openstack.org/16803918:10
*** rm_work|away is now known as rm_work18:24
openstackgerritAdam Harwell proposed openstack/barbican: Use the new Devstack external plugin method  https://review.openstack.org/16788518:25
*** rm_work is now known as rm_work|away18:28
*** rm_work|away is now known as rm_work18:34
*** SheenaG has joined #openstack-barbican18:42
hockeynutjvrbanac ping18:45
jvrbanachockeynut, what's up18:46
hockeynutlooking at cli testing - would like to hear your ideas on how to (or not to) hit it from client tests.18:46
jvrbanachockeynut, to get started, I would probably just create an instance of the Barbican cli class, populate an argv, and pass it into .run()18:52
jvrbanacProbably the fastest way of going about this18:52
hockeynutthx!18:52
jvrbanachockeynut, cliff may have a more prescribed way of doing it, but if I were to just jump into it, that's probably what I would do.18:54
hockeynutcoolness - was looking at a few different ways18:57
jvrbanachockeynut, just don't use subprocesses, I will -2 things with subprocesses ;)19:03
hockeynutLOL I someone got that impression!19:05
hockeynutc/someone/somehow19:06
openstackgerritwerner mendizabal proposed openstack/barbican: Create barbican paste deploy scripts  https://review.openstack.org/17096119:13
*** kebray has quit IRC19:20
aleewoodster_, ping19:35
aleerm_work, ping19:36
aleereaperhulk, pign19:36
aleehockeynut, reaperhulk , redrobot, jvrbanac -- anyone there?19:53
aleewoodster_, ?19:53
jvrbanacalee, kinda what's up?19:53
hockeynuthockeynut is here for 7 more minutes (we have a mtg @ 3CT)19:53
woodster_alee, sounds serious! Maybe I'm not here? :)19:54
aleejvrbanac, hockeynut , woodster_ just trying to get a handle on this secrets encoding issue19:54
Guest48074alee o/19:54
Guest48074derp19:54
Guest48074lost my nick19:54
hockeynutnickless?19:54
aleeof course its a little different on openssl 0.9.8 ..19:55
aleecan ya'll run the following test script and let me know what you see?19:55
*** Guest48074 is now known as redrobot19:55
aleehttp://www.fpaste.org/207729/14283501/19:56
*** kfarr has joined #openstack-barbican19:56
aleeI'm particular interested on systems where we have openssl 0.9.8 (ie. macs for instance)19:56
jvrbanacalee, you want the full output?19:58
aleejvrbanac, sure19:58
aleejvrbanac, just want to see if it matches what I have on my version of openssl19:58
redrobotmeeting is starting now in #openstack-meeting-alt20:00
aleejvrbanac, you got output?20:01
jvrbanacalee, I don't run a Mac, so this is on xubuntu 14.10 with OpenSSL 1.0.1f http://hastebin.com/tiwevawovu.txt20:04
aleejvrbanac, ok - that matches what I see.20:05
aleehockeynut, redrobot  woodster_ reaperhulk -- mac users?20:05
redroboto/20:06
redrobotalee I am, but I use openssl from homebrew20:06
aleeredrobot, version?20:06
woodster_alee, I'll try it out shortly20:07
aleewoodster_, thanks the srcipt clearly shows the issue -- we do some encoding / decoding into base64 when storing / retrieving secrets20:08
aleewoodster_, redrobot when we do this using base64.b64encode/decode we lose all '\n' characters and crypto.loadkey barfs20:08
redrobotalee actually I have an agenda item to talk about that20:09
redrobotalee OpenSSL 1.0.2 22 Jan 201520:09
aleeusing base64.encodestring seems to work for me20:09
aleeredrobot, phooey - then you'll prob see the same as me20:10
woodster_alee, btw we are having our weekly irc meeting now...20:12
aleewoodster_, oh I know -- I'm lurking till something comes up20:13
*** kebray has joined #openstack-barbican20:18
*** igueths has quit IRC20:31
*** dave-mccowan has joined #openstack-barbican20:46
*** rellerreller has joined #openstack-barbican20:52
*** igueths has joined #openstack-barbican20:52
woodster_alee, that test script passes on my machine, but I'm not using the stock mac openssl20:55
aleewoodster_, what do you mean by passes?20:56
aleewoodster_, there are some tests that should fail in there ..20:57
woodster_well, I see this:20:57
woodster_https://www.irccloud.com/pastebin/khxXQSP620:57
aleerellerreller, so -- try this script ..20:58
aleehttp://www.fpaste.org/207729/14283501/20:58
aleewoodster_, yeah thats the last bit20:58
woodster_Barbican meeting, overtime!...20:58
aleewoodster_, but some tests fail before that20:59
aleeyou should see some "not working for  ..."20:59
woodster_alee, do you want to see the rest of the output? My openssl version is > 1.0 btw20:59
aleewoodster_, sure20:59
rellerrelleralee What is the output? They are not the same?21:00
aleewoodster_, rellerreller - the functions in the script are modeled on the ones we use in the code.  rellerreller probably recognizes some of them21:00
rellerrellerYes, they are the ones from translations.21:00
rellerrellerWas that a cut paste?21:00
aleealthough they've had a couple of corrections in them21:00
aleerellerreller, no - run the script21:01
aleerellerreller, its not an exact cut/paste - there are some fixes in there ..21:01
aleerellerreller, woodster_ what the script shows is --21:01
*** pixer has joined #openstack-barbican21:02
alee1. the keys etc. in utils.py need to have a "\n" at the end of each line, or they do not load21:02
alee2. if you encode/decode with b64enocde/decode, then the result has all the '\n' removed, and will not load21:04
woodster_jvrbanac, btw we do have a kilo blueprint for the version resource that wasn't implemented: https://blueprints.launchpad.net/barbican/+spec/fix-version-api21:04
woodster_jvrbanac, ...not sure if that is what you meant during the IRC meeting?21:04
alee3. if you encode/decode with encodestring/decodestrin, it preserves '\n' , and so it does load21:05
alee4. interestingly it preserves the '\n' , but not at the same place21:05
alee5. but the resulting loaded key is the same.21:05
woodster_redrobot, did you want to continue the content-types discussion here?21:05
rellerrelleralee So the issue is that the private, public, and certificates are not returned with the correct formatting for line splits?21:06
aleerellerreller, woodster_ all this means that if we want to continue the manipulations we are currently doing , I think I have a fix that makes things work21:07
aleerellerreller, yup21:07
redrobotwoodster_ o/21:07
rellerrellerI wonder why the tests did not catch this.21:07
alee(and the pem_ related funrtions did not handle line split correctly to begin with21:07
redrobotrellerreller seems to only happen on older versions of OpenSSL21:08
rellerrelleralee We should have just made all of this binary :(21:08
aleerellerreller, 1. the input data did not contain line splits21:08
alee2.  you did not actually try and use it (ie. load_privatkey)21:08
rellerrellerSorry to do this, but my daughter is up.21:08
aleesaved by the bell ..21:09
rellerrelleralee Good points.21:09
rellerrellerI will be online tomorrow. I am back from vacation.21:09
dave-mccowanalee did you try to run the 2 functional tests i put in a CR with your fix?21:09
aleerellerreller, ok by then I'll have a patch21:09
aleedave-mccowan, not yet - but I will21:09
dave-mccowanalee, i ran your script with OpenSSL 0.9.8za 5 Jun 2014.  the last line of output said "woot".  is there other output to check?21:10
kfarralee, there was a functional test that loaded the private and public key, but it used cryptography and not openssl directly  :/21:10
aleekfarr, yes -- cryptography links to a later version of openssl21:11
aleedave-mccowan, thats good to know21:11
aleedave-mccowan, can you post your output?21:11
aleekfarr, some of this didnot appear for me too with later openssl21:11
dave-mccowanalee http://www.fpaste.org/207753/14283547/21:12
kfarralee ah ok21:12
redrobotso, i think that somewhat-related to alee's issues,  some of the functional tests are incorrect.21:12
redrobothttps://github.com/openstack/barbican/blob/master/functionaltests/api/v1/functional/test_secrets.py#L904-L90621:12
redrobot^^ for example, sends "base64" as the encoding with a DER that has not been base64 encoded21:12
aleedave-mccowan, good your output matches mine21:13
aleeredrobot, well part of it is -- I understand your objection - and I think we need to have a discussion of how we want to handle this case.21:14
aleeredrobot, so the question is -- if I want to store pem data -- what should I send in?21:15
redrobotalee that's what I'm trying to figure out... was able to get the boss to task me with sorting this stuff out21:16
dave-mccowanredrobot here are two functional tests (they fail) that use generated keys that demonstrate the current problem. https://review.openstack.org/#/c/169974/21:16
redrobotalee send as DER, but don't set the content-encoding to base6421:17
redrobotI think that when payload_content_encoding is set to base64, barbican should always decode _the entire payload_21:17
aleeredrobot, I think to get some clarity to this, we need some stuff written down.  I suggest we21:18
alee1) set up a google hangout for tommorow21:18
alee2) have you write down how each type of input should be handled.  we are interested in private keys for exmaple in pem/der format, with or without passphrases21:19
aleewhat we want to know is what gets passed in, what gets sent to the plugins, what gets returned from the plugins21:20
aleeand what gets returned from barbican server21:20
redrobotalee agreed.... but I'd rather push the hangout to Wednesday21:21
redrobotalee I think it'll take the rest of today and most of tomorrow to compile all the info21:21
aleealso keep in mind the stored_key case, for which we must be able to load the key in pem format in order to handle the passphrase case21:21
*** rellerreller has quit IRC21:22
aleesure - in the meantime, I'll be creating a fix based on what we have21:22
aleeand so at least we'll have functional tests for all cert types21:22
aleeand then we can make sure they still work if we make changes21:22
*** paul_glass has quit IRC21:24
aleewoodster_, redrobot hockeynut jvrbanac - by the way, acl changes are still out there waiting to be workflowed.  lets get them in please.21:24
*** paul_glass has joined #openstack-barbican21:24
*** kebray has quit IRC21:25
*** paul_glass has quit IRC21:26
*** kfarr has quit IRC21:27
openstackgerritJohn Wood proposed openstack/barbican: Add order_retry_tasks migration per latest model  https://review.openstack.org/16994621:32
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack  https://review.openstack.org/17089621:40
openstackgerritIgor Gueths proposed openstack/barbican: Potential resource exhaustion when registering consumers to containers  https://review.openstack.org/17069321:41
*** pixer has quit IRC21:44
openstackgerritCharles Neill proposed openstack/barbican: Security tests for Consumer resources  https://review.openstack.org/16701821:48
*** SheenaG has quit IRC21:49
openstackgerritCharles Neill proposed openstack/barbican: Security tests for Consumer resources  https://review.openstack.org/16701821:53
*** xaeth is now known as xaeth_afk21:54
openstackgerritJohn Wood proposed openstack/barbican: Add retry server and functional tests to DevStack  https://review.openstack.org/17089621:54
woodster_arunkant, you are oh so close to 100% coverage on your ACL CRs...would you be up for adding the missing unit tests, even as a separate CR?21:58
openstackgerritIgor Gueths proposed openstack/barbican: Potential resource exhaustion when registering consumers to containers  https://review.openstack.org/17069322:07
arunkantwoodster_ , I looked into coverage report, did not find the areas I am missing. I see 100% for acls controller and policy changes. http://logs.openstack.org/07/165207/10/check/barbican-coverage/f9dd194/cover/22:17
arunkantwoodster_ , yes I can add more test in additional CR if I know what coverage is missing22:19
iguethsBtw in case anyone is wondering, Gertty is being screwy hence the broken patchsets.22:22
*** igueths has quit IRC22:26
openstackgerritMerged openstack/barbican: Adding Container ACL controller layer changes (Part 4)  https://review.openstack.org/16520522:32
*** jkf has quit IRC22:48
*** dimtruck is now known as zz_dimtruck22:49
*** nkinder has quit IRC22:51
*** SheenaG has joined #openstack-barbican22:55
*** igueths has joined #openstack-barbican22:56
elmikoredrobot: ping22:56
woodster_arunkant, the 'barbican-coverage' gate is the on to look at...here are the latest logs: http://logs.openstack.org/07/165207/10/check/barbican-coverage/f9dd194/console.html22:56
arunkantwoodster_, does this coverage report only generate coverage report for the changes made, or does it include the uncovered lines which were present before the change ?22:59
woodster_arunkant, it only reports what lines were modified by the CR23:00
woodster_arunkant, it isn't perfect I've found though, but it is worth adding tests for code that you can cover with unit tests23:01
arunkantwoodster_, will look into report again..as I can see only one line which is not covered (mainly because that check condition would not happen so essentially can remove that line).23:03
*** kebray has joined #openstack-barbican23:03
arunkantwoodster_, will add that as additional CR to improve the coverage.23:04
*** jamielennox|away is now known as jamielennox23:07
woodster_arunkant, just to make sure, this is the report I'm referring to:23:07
woodster_https://www.irccloud.com/pastebin/uwGykEFr23:07
woodster_arunkant, you might find that unit tests are covering the lines, in which case just submit the CR and see what the gate reports.23:08
*** chlong has joined #openstack-barbican23:10
arunkantwoodster_, For this part, It reported these 2 lines which are not changed anywhere in ACL..but still reported as missing. Not sure why? http://logs.openstack.org/07/165207/10/check/barbican-coverage/f9dd194/cover/barbican_api_controllers___init__.html#n16123:13
woodster_arunkant, well that gate is designed to slowly improve coverage over time. So it could be that that function was added/used before the gate job was activated, so was never covered by unit tests before. But because you code (or code you touched) is calling that function, not it is pulled into the fold. So major refactoring can lead to a lot of unit tests :)23:19
arunkantwoodster_, Okay. makes sense to me. Thanks. Will add CR to see if I can improve the coverage on whatever is touched in ACL CR.23:22
openstackgerritMerged openstack/barbican: Adding policy layer changes for ACL support (Part 5)  https://review.openstack.org/16520723:22
redrobotelmiko pong23:29
openstackgerritIgor Gueths proposed openstack/barbican: Potential resource exhaustion when registering consumers to containers  https://review.openstack.org/17069323:40
*** kebray has quit IRC23:42
*** igueths has quit IRC23:50
elmikoredrobot: hey, tried running a migration23:51
elmikoredrobot: didn't go so well, maybe you could double check my logic23:52
redrobotelmiko uh oh23:52
elmikoredrobot: https://gist.github.com/elmiko/34c2f064c104d08a3e1f23:52
elmikoredrobot: does my process make sense for an upgrade?23:53
redrobotelmiko yeah... that's pretty much what I would have done to test it. :-\23:54
elmiko=(23:54
redrobotelmiko I'll try to get someone here to try that too.... if it fails for someone else then I think our migration might be broken23:55
elmikoyea, i was worried when it looked like some nasty sql error23:55
elmikoredrobot: k, hopefully it's just my setup ;)23:55
redrobotelmiko I hope so too...   thank you for taking the time to check!23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!