| openstackgerrit | John Wood proposed openstack/barbican: Restore worker tasks processing catching exceptions https://review.openstack.org/168039 | 00:02 |
|---|---|---|
| *** zz_dimtruck is now known as dimtruck | 01:04 | |
| *** alee_ has joined #openstack-barbican | 02:09 | |
| *** kebray has joined #openstack-barbican | 02:10 | |
| *** alee has joined #openstack-barbican | 02:17 | |
| alee | woodster_, ? | 02:17 |
| *** david-lyle has joined #openstack-barbican | 03:13 | |
| openstackgerrit | Ade Lee proposed openstack/barbican: Changes to get remaining cert functional tests working https://review.openstack.org/169600 | 03:18 |
| *** dimtruck is now known as zz_dimtruck | 03:33 | |
| *** zz_dimtruck is now known as dimtruck | 03:36 | |
| *** tkelsey has joined #openstack-barbican | 03:39 | |
| *** tkelsey has quit IRC | 03:44 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Implement validators and tests for stored key certificate orders https://review.openstack.org/167291 | 04:14 |
| *** dimtruck is now known as zz_dimtruck | 04:17 | |
| *** dave-mccowan has quit IRC | 04:42 | |
| *** alee_ has quit IRC | 04:47 | |
| *** alee has quit IRC | 04:47 | |
| *** alee has joined #openstack-barbican | 04:48 | |
| *** alee_ has joined #openstack-barbican | 04:49 | |
| *** tkelsey has joined #openstack-barbican | 05:40 | |
| *** tkelsey has quit IRC | 05:45 | |
| *** kebray has quit IRC | 05:48 | |
| *** kebray has joined #openstack-barbican | 05:49 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/barbican: Imported Translations from Transifex https://review.openstack.org/169634 | 06:11 |
| *** woodster_ has quit IRC | 06:30 | |
| *** everjeje has joined #openstack-barbican | 07:43 | |
| *** kebray has quit IRC | 08:10 | |
| *** tkelsey has joined #openstack-barbican | 08:28 | |
| openstackgerrit | Thomas Herve proposed openstack/python-barbicanclient: Fix order listing on the command line. https://review.openstack.org/169481 | 09:57 |
| *** darrenmoffat has quit IRC | 10:13 | |
| *** darrenmoffat has joined #openstack-barbican | 10:13 | |
| *** dave-mccowan has joined #openstack-barbican | 12:25 | |
| *** atiwari2 has joined #openstack-barbican | 12:28 | |
| *** atiwari1 has quit IRC | 12:31 | |
| *** woodster_ has joined #openstack-barbican | 12:38 | |
| woodster_ | I'm rubbing my eyes and still see it on my phone's gerrit review page...that circa 90's pesky Microsoft paper clip! Aghhhh! | 12:45 |
| dave-mccowan | woodster_ lol april fools | 13:01 |
| woodster_ | dave-mccowan: I had flash backs to the Microsoft Age man, not cool :) | 13:03 |
| *** alee_ has quit IRC | 13:03 | |
| alee | woodster_, dave-mccowan aaargh - its clippy! | 13:04 |
| dave-mccowan | alee, woodster_, "it looks like you're writing a letter" | 13:07 |
| woodster_ | Yeah I remember it being pretty opinionated....if you did something it didn't like it would wag its end at you with a frown, tapping the monitor like a cranky old teacher | 13:07 |
| *** jaosorior has joined #openstack-barbican | 13:11 | |
| alee | woodster_, can you take a look at https://review.openstack.org/#/c/169600/ ? some unit tests are failing under tox but work just fine for me in pycharm. | 13:15 |
| woodster_ | alee: will do | 13:17 |
| alee | dave-mccowan, https://review.openstack.org/#/c/169600/ contains some of your changes right now too so that I could get the tests working. I hope though that your cr will land before mine and I'll just rebase. | 13:17 |
| *** nkinder has quit IRC | 13:17 | |
| dave-mccowan | alee, +1 new patch with those is out for review. https://review.openstack.org/#/c/167291/ | 13:20 |
| alee | woodster_, what I don't understand is that tox seems to be throwing an exception in get_plugin_retrieve_delete() in secret_store.py -- but (and pycharm agrees with me when I do this) I am mock patching that function. | 13:21 |
| *** joesavak has joined #openstack-barbican | 13:26 | |
| -openstackstatus- NOTICE: gerrit has been restarted to restore event streaming. any change events missed by zuul (between 12:48 and 13:28 utc) will need to be rechecked or have new approval votes set | 13:28 | |
| woodster_ | alee, do you mean on the cover gate? The rest of the gates are passing | 13:28 |
| alee | woodster_, https://review.openstack.org/#/c/169600/ ? | 13:29 |
| alee | woodster_, unless clippy fixed it :) | 13:30 |
| woodster_ | alee, ha! wrong CR sorry | 13:30 |
| woodster_ | chellygel, do you remember clippy or is that before your time? | 13:32 |
| dave-mccowan | alee, woodster_ i remember this exception happening last month for a CR where the gate fails, but it works locally. | 13:33 |
| woodster_ | yeah, it's probably a global mem setting issue | 13:34 |
| woodster_ | alee, too many changes in that test file...clippy would be wagging his wiry finger at you :) | 13:35 |
| alee | dave-mccowan, woodster_ well tox fails locally for me too in this case. but pycharm works just fine. | 13:42 |
| alee | woodster_, clippy would also have opened the code files in notepad. | 13:43 |
| woodster_ | alee, ha! Yeah I see where it should be mocked | 13:43 |
| alee | woodster_, yeah - so that works for me in pycharm -- and when I step through in debug mode, I can see it happening. But not in tox. | 13:45 |
| alee | woodster_, one thing that is interesting -- in _config_cert_event_plugin() you mock EVENT_PLUGIN_MANAGER instead of _CertificateEventPluginManager | 13:47 |
| alee | woodster_, should I be mocking _SECRET_STORE instead? | 13:48 |
| woodster_ | alee, yeah I'm looking at that now....I think you should be mocking the return of the get_manager() method at the end of secret_store.py | 13:48 |
| woodster_ | alee, ...to avoid the global check that might be getting set to the real plugin manabge | 13:49 |
| alee | woodster_, trying that now .. | 13:49 |
| woodster_ | alee, 'manager' that is | 13:49 |
| alee | woodster_, that works -- both tox and pycharm like it | 13:52 |
| alee | woodster_, thanks! now to figure out why a couple of functional tests look like they are failing at the gate. | 13:54 |
| chellygel | woodster_, i remember clippy!! | 13:54 |
| alee | I'll resubmit and see what happens | 13:54 |
| woodster_ | alee: Nice! | 13:54 |
| chellygel | woodster_, i was an active computer user at the ripe ol' age of 6. I was very lucky in that my step father was a programmer. I remember good ol' windows 3.0 :D | 13:54 |
| dave-mccowan | alee if it still fails, that exception showed up here too: http://eavesdrop.openstack.org/irclogs/%23openstack-barbican/%23openstack-barbican.2015-03-20.log at 2015-03-20T20:30:21 | 13:56 |
| woodster_ | chellygel: impressive! I had to settle for an abacus at that age...but no clippy at least | 13:56 |
| alee | woodster_, at the ripe ol' age of 6, I don't think we even had computers .. slide rules maybe .. | 13:58 |
| *** ametts has joined #openstack-barbican | 13:58 | |
| openstackgerrit | Ade Lee proposed openstack/barbican: Changes to get remaining cert functional tests working https://review.openstack.org/169600 | 13:59 |
| *** nkinder has joined #openstack-barbican | 14:00 | |
| alee | woodster_, feel free to take a look at the other changes in that CR, so that you're familiar with them when we need to get it all acked | 14:00 |
| *** xaeth_afk is now known as xaeth | 14:00 | |
| chellygel | i could take a workflow on my latest docs change: https://review.openstack.org/#/c/169518/ | 14:01 |
| alee | chellygel, done | 14:02 |
| chellygel | thank you :) | 14:02 |
| dave-mccowan | https://review.openstack.org/167484 also needs a workflow | 14:06 |
| alee | dave-mccowan, done | 14:08 |
| dave-mccowan | alee thanks! | 14:08 |
| *** paul_glass has joined #openstack-barbican | 14:13 | |
| alee | dave-mccowan, reviewed | 14:19 |
| openstackgerrit | Merged openstack/barbican: Adding GET and DELETE for containers quickstart guide https://review.openstack.org/169518 | 14:24 |
| dave-mccowan | alee thanks | 14:25 |
| *** ametts has quit IRC | 14:27 | |
| openstackgerrit | Merged openstack/barbican: Fix string formatting for a secret store exception message https://review.openstack.org/167484 | 14:32 |
| *** dave-mccowan has quit IRC | 14:32 | |
| *** zz_dimtruck is now known as dimtruck | 14:36 | |
| xaeth | d'oh... i never saw the e-mail with updates to the barbican package review. | 14:47 |
| xaeth | le sigh. sorry | 14:47 |
| xaeth | oh wait. ha that was today | 14:47 |
| xaeth | :) | 14:47 |
| xaeth | now i dont feed as bad | 14:47 |
| xaeth | feel | 14:47 |
| jvrbanac | WAT! Why is there a clippy on my gerrit review? | 14:48 |
| chellygel | jvrbanac, dont question clippy, he'll cut you | 14:48 |
| jvrbanac | chellygel, pretty sure MS cut him already | 14:49 |
| chellygel | ba dum tis | 14:49 |
| jvrbanac | lol | 14:49 |
| openstackgerrit | Chelsea Winfree proposed openstack/barbican: Adding docs to index and minor fixes https://review.openstack.org/169785 | 14:54 |
| *** arunkant has quit IRC | 15:16 | |
| *** paul_glass has quit IRC | 15:22 | |
| *** paul_glass has joined #openstack-barbican | 15:23 | |
| *** arunkant has joined #openstack-barbican | 15:29 | |
| *** kebray has joined #openstack-barbican | 15:33 | |
| alee | woodster_, ping | 15:34 |
| woodster_ | alee, hello | 15:35 |
| alee | woodster_, do you have an environement in which the functional tests can be run? | 15:35 |
| alee | woodster_, the reason I ask is because one of the cert functional tests is failing at the gate, but is succeeding for me locally. I think that might be because I have things set up for a dogtag type environment | 15:36 |
| woodster_ | alee, I had one setup :) Some bit rot since then I'm sure. That dockerize keystone CR I have out there adds more info to set this up | 15:37 |
| woodster_ | alee, which CR | 15:37 |
| woodster_ | ? | 15:37 |
| alee | woodster_, https://review.openstack.org/#/c/169600/ | 15:38 |
| alee | woodster_, it passes tox unit tests now, and I can also explain away the coverage failure | 15:38 |
| openstackgerrit | Merged openstack/barbican: Imported Translations from Transifex https://review.openstack.org/169634 | 15:48 |
| woodster_ | alee, I'm running func against my setup now... | 15:51 |
| hockeynut | jvrbanac glad I'm not the only one who saw clippy | 15:54 |
| alee | woodster_, cool thanks | 15:54 |
| xaeth | redrobot, pkgs.fedoraproject.org/cgit/openstack-keystone.git/tree/openstack-keystone.spec#n166 | 15:55 |
| redrobot | xaeth word! thanks for the link, I'll try to take a look at it today. | 15:56 |
| rm_work | alee: any idea if there are stale pyc files? | 16:00 |
| rm_work | alee: we had issues with that the last couple of days because of oslo renames | 16:00 |
| rm_work | but i think barbican may have done those renames a month or so ago, so might be unrelated | 16:01 |
| rm_work | which tests? I can try in my devstack | 16:01 |
| alee | rm_work, thanks -- https://review.openstack.org/#/c/169600/ | 16:02 |
| woodster_ | alee, ugh, boot2docker is giving me fits! :) | 16:02 |
| chellygel | https://review.openstack.org/#/c/169785/ workflow for a +3, -1 change :D | 16:02 |
| openstackgerrit | Adam Harwell proposed openstack/barbican: Use the new Devstack external plugin method https://review.openstack.org/167885 | 16:03 |
| alee | rm_work, in the gate, functional test create_stored_key_order is failing (but succeeding for me locally | 16:03 |
| alee | woodster_, sorry :) | 16:04 |
| rm_work | kk | 16:04 |
| rm_work | woodster_: you work for a cloud company, use a VM :P | 16:04 |
| woodster_ | rm_work, ha, no kidding! | 16:04 |
| rm_work | wget and run: https://gist.githubusercontent.com/rm-you/70f0b779970402e9baba/raw/7b23667064d5ff1777f1d4ce65ebf9e2c601411c/new_barbican_devstack.sh | 16:05 |
| rm_work | success | 16:05 |
| *** dave-mccowan has joined #openstack-barbican | 16:10 | |
| redrobot | chellygel I agree with jvrbanac, we need to get rid of the old api doc | 16:16 |
| chellygel | already on it redrobot :D | 16:16 |
| redrobot | chellygel got plans for lunch? I have to be back at 1pm for the CR hangout. | 16:17 |
| jvrbanac | redrobot, yeah, unfortunately we can't quite get rid of that page yet; however, we can mark it as old and focus on the new format we want people to use | 16:17 |
| redrobot | jvrbanac why can't it be deleted? | 16:17 |
| redrobot | I think having an API page and an "Old API" page is confusing | 16:18 |
| jvrbanac | redrobot, we haven't ported orders and consumers yet. | 16:18 |
| chellygel | i am about to commit this change, so i want a decision before we lunch redrobot | 16:18 |
| alee | dave-mccowan, ping | 16:18 |
| dave-mccowan | alee pong | 16:18 |
| chellygel | and i agree, we can't get rid of it until its 100% done | 16:18 |
| alee | dave-mccowan, you have the functional tests running, right? | 16:18 |
| woodster_ | have you guys pulled over all the content from that old wiki now? | 16:18 |
| chellygel | no woodster_ there's no way | 16:19 |
| redrobot | chellygel jvrbanac any duplicate info? | 16:19 |
| dave-mccowan | alee yes, they all pass with my patch | 16:19 |
| * redrobot looks | 16:19 | |
| * chellygel is also looking | 16:19 | |
| alee | dave-mccowan, can you pull down my patch and see if the tests pass? | 16:19 |
| jvrbanac | brb | 16:19 |
| chellygel | redrobot, turning my tiny cr into a big one | 16:19 |
| chellygel | blame the ptl o/ | 16:20 |
| alee | dave-mccowan, https://review.openstack.org/#/c/169600 | 16:20 |
| dave-mccowan | alee sure. will do now. | 16:20 |
| alee | dave-mccowan, they all pass for me locally - but one fails in the gate. | 16:20 |
| woodster_ | alee, I see this error: | 16:20 |
| woodster_ | https://www.irccloud.com/pastebin/3a5jF0Tu | 16:20 |
| rm_work | sorry alee got delayed by shitty VPN, on it now | 16:20 |
| alee | woodster_, good - thats what happens in the gate. Can you see whats happening in your server log? | 16:21 |
| chellygel | redrobot, some of it can be deleted i can delete all references to secrets and consumers, just a few extra minutes of work | 16:21 |
| alee | woodster_, I can't debug because it doesn't fail for me :/ | 16:21 |
| chellygel | there are some things that are probably not 1:1 | 16:21 |
| chellygel | but thankfully, git is a thing that keeps our history | 16:21 |
| alee | woodster_, my guess is there will be a stack trace | 16:22 |
| redrobot | chellygel yeah, delete all the stuff that has already been added to the new structure. Move the old api.rst under the new structure as orders.rst | 16:22 |
| woodster_ | rm_work, i like the devstack foo btw, nice. Would be good to document that somewhere :) | 16:23 |
| rm_work | woodster_: it's on the wiki ;) | 16:23 |
| rm_work | woodster_: did you actually use it? | 16:24 |
| dave-mccowan | alee i'm seeing what woodster_ is seeing. | 16:27 |
| alee | dave-mccowan, great -- do you see a stack trace? | 16:28 |
| dave-mccowan | alee INFO barbican.openstack.common.policy [-] Can not find policy directory: policy.d | 16:28 |
| alee | dave-mccowan, yeah - thats not relevant I think | 16:29 |
| dave-mccowan | alee http://www.fpaste.org/206005/14279058/ | 16:30 |
| alee | dave-mccowan, aha - interesting that I dont see that -- looking .. | 16:31 |
| alee | dave-mccowan, can you add debug= True to barbican-api.conf? | 16:35 |
| openstackgerrit | Chelsea Winfree proposed openstack/barbican: Adding docs to index and minor fixes https://review.openstack.org/169785 | 16:36 |
| dave-mccowan | alee i have the exception stopped at a breakpoint in pycharm. do you want to share a screen? | 16:36 |
| woodster_ | any nosetests experts out there? passing in path/to/module:TestClass.testmethod doesn't work for me. Only path/to/module works | 16:36 |
| woodster_ | ...for trying to run just one test method | 16:37 |
| dave-mccowan | woodster_ nosetests functionaltests/api/v1/functional/test_certificate_orders.py:CertificatesTestCase.test_create_stored_key_order | 16:37 |
| alee | dave-mccowan, sure | 16:37 |
| *** crc32 has joined #openstack-barbican | 16:37 | |
| *** crc32 has quit IRC | 16:38 | |
| alee | woodster_, dave-mccowan -- so the code that is central to all of this -- and which would be failing in this test is this .. | 16:52 |
| alee | woodster_, dave-mccowan http://fpaste.org/206010/07221142/ | 16:53 |
| alee | not sure why its not failing for me - but it seems to be failing at line 41 | 16:54 |
| alee | this is in certificate_resources.py | 16:56 |
| alee | woodster_, dave-mccowan - now we initially create the data by creating secrets and adding a rsa container | 16:57 |
| alee | woodster_, dave-mccowan so what is initially passed in is this -- barbican.tests.utils.get_private_key() | 16:59 |
| alee | which is a key in PEM format | 16:59 |
| dave-mccowan | alee i have code stopped at a breakpoint in that function now | 16:59 |
| alee | dave-mccowan, yeah -- let me enable java in my browser and hop on | 17:00 |
| dave-mccowan | alee btw private_key looks like binary, not PEM | 17:05 |
| alee | dave-mccowan, trying to figure out how to do that :) | 17:06 |
| alee | dave-mccowan, right - which is what I would expect -- thats why I selected type "ASN1" | 17:06 |
| alee | instead of PEM | 17:06 |
| alee | in the load_privatekey() call | 17:07 |
| *** jkf has joined #openstack-barbican | 17:17 | |
| *** jaosorior has quit IRC | 17:32 | |
| *** ametts has joined #openstack-barbican | 17:43 | |
| alee | reaperhulk, ping | 17:52 |
| *** chadlung has joined #openstack-barbican | 17:52 | |
| alee | reaperhulk, what is crypto.FILETYPE_ASN1 in openssl ? is that binary? | 17:52 |
| alee | dave-mccowan, woodster_ -- so we store a private key in the backend | 17:56 |
| alee | when we get it out -- it comes out of the backend as base64 encoded data -- we then call denormalize_after_decryption() | 17:57 |
| alee | which strips off any pem components and coverts to binary | 17:58 |
| reaperhulk | what is that from? | 17:58 |
| alee | thats what I pass into load_privatekey() | 17:58 |
| reaperhulk | that's pycrypto, but yeah I'd expect that to be the flag stating that it's DER encoded binary ASN1 | 17:58 |
| rm_you | redrobot: wait is this thing via Hangouts or Vidyo? | 17:59 |
| alee | OpenSSL.crypto | 17:59 |
| redrobot | rm_you hangouts | 17:59 |
| rm_you | redrobot: kk | 17:59 |
| reaperhulk | oh, that's pyopenssl, gotcha | 17:59 |
| reaperhulk | lemme take a quick look | 17:59 |
| *** chadlung has quit IRC | 17:59 | |
| reaperhulk | yep, that's the DER encoding flag in the guts of OpenSSL | 17:59 |
| rm_you | redrobot: can't seem to get a LINK out of the calendar invite, can you msg me? | 17:59 |
| reaperhulk | I'm surprised PyOpenSSL is using that...there are ways to avoid it (we don't use it in cryptography's hazmat code) | 18:00 |
| redrobot | rm_you https://plus.google.com/hangouts/_/calendar/ZG91Z21lbmRpemFiYWxAZ21haWwuY29t.9u2jv33g3151n2ar3pfbfsd3tc | 18:00 |
| *** chadlung has joined #openstack-barbican | 18:00 | |
| alee | reaperhulk, need to jump on the google hangout -- but I may need to chat on how to do some code afterwards | 18:00 |
| reaperhulk | okay | 18:01 |
| alee | reaperhulk, basically -- I have code in https://review.openstack.org/#/c/169600/ that s not working | 18:01 |
| alee | reaperhulk, in certificate_resources.py | 18:02 |
| alee | actually its working for me locally but not in the functional tests at the gate (or for anyone else) | 18:02 |
| reaperhulk | what's the data in private_key when it fails? Is it a string in the tests? | 18:03 |
| alee | reaperhulk, load_privatekey() doesn;t like the binary its being passed in .. | 18:03 |
| reaperhulk | is it the current CR revision that's failing? I want to replicate it locally | 18:03 |
| alee | reaperhulk, yes | 18:04 |
| alee | reaperhulk, you should be able to -- I can't , but others can | 18:04 |
| alee | reaperhulk, you need to run the functional tests | 18:05 |
| alee | reaperhulk, the create_stored_key_order fails | 18:05 |
| reaperhulk | okay I'll take a look shortly | 18:05 |
| alee | reaperhulk, thanks | 18:05 |
| *** kfarr has joined #openstack-barbican | 18:08 | |
| dave-mccowan | alee, reaperhulk this is the contents of the private_key string that fails load_privatekey() on my system http://fpaste.org/206084/79132221/ | 18:36 |
| *** jaosorior has joined #openstack-barbican | 18:38 | |
| jvrbanac | thx rm_work | 18:50 |
| *** tkelsey has quit IRC | 18:51 | |
| rm_you | jvrbanac: wait what did i do? :P | 18:53 |
| jvrbanac | rm_you, you muted redrobot :P | 18:54 |
| rm_you | ah, yes :P | 18:54 |
| rm_you | i was surprised it let me do that | 18:54 |
| *** chadlung has quit IRC | 18:58 | |
| *** chadlung has joined #openstack-barbican | 18:58 | |
| *** crc32 has joined #openstack-barbican | 19:14 | |
| *** tkelsey has joined #openstack-barbican | 19:18 | |
| reaperhulk | https://github.com/openstack/barbican/blame/master/barbican/tests/utils.py#L462 I have no idea why this is working for anybody. It is PKCS8 PEM (so you should use FILETYPE_PEM), but it is missing all the line feeds. Every line should have \n on the end. As is it's being implicitly concatenated with no \n and is invalid | 19:18 |
| reaperhulk | ^-- alee that's your problem | 19:19 |
| reaperhulk | the public key and certificate have the same problem | 19:19 |
| reaperhulk | they all just need \n stuck to the end of each string line | 19:20 |
| *** tkelsey has quit IRC | 19:22 | |
| *** ametts has quit IRC | 19:27 | |
| openstackgerrit | Merged openstack/barbican: Adding docs to index and minor fixes https://review.openstack.org/169785 | 19:29 |
| rm_you | yeah I think if that was """ | 19:29 |
| rm_you | it would work | 19:30 |
| rm_you | since that captures newlines, right? | 19:30 |
| rm_you | yes | 19:30 |
| * rm_you just checked | 19:30 | |
| rm_you | I prefer the """ approach because then you don't make it painful to copy/paste in and out :P | 19:31 |
| alee | reaperhulk, looking .. | 19:38 |
| rm_you | reaperhulk should be correct there -- those are malformed as-is | 19:39 |
| alee | rm_you, did you just run the test and it seemed to work? | 19:40 |
| reaperhulk | if you do """ you'll need to textwrap.dedent | 19:41 |
| reaperhulk | (that's how we do it in cryptograpy | 19:41 |
| reaperhulk | cryptography even | 19:41 |
| alee | reaperhulk, when we do a get_secret(), we end up stripping the headers off the secret and returning as binary | 19:43 |
| *** rm_work is now known as rm_work|away | 19:43 | |
| alee | reaperhulk, see denormalize_after_decryption() | 19:44 |
| alee | reaperhulk, in translations.py | 19:44 |
| reaperhulk | okay, will have to look, but those are definitely invalid PEMs | 19:44 |
| reaperhulk | (due to the newlines, but the DER decode should ignore that issue obviously) | 19:44 |
| alee | reaperhulk, yeah - thats good to know --not sure why it was working for me -- but ok. | 19:45 |
| dave-mccowan | alee, reaperhulk if i dump the buffer to file, and run "openssl rsa -inform DER -in private_key.bin -check" on it, i get "rsa key ok". but call load_privatekey() in that file fails on the same buffer. | 19:49 |
| rm_you | reaperhulk: does hacking not like it if you don't dedent manually with """ ? | 19:50 |
| reaperhulk | pep8 shouldn't like it. | 19:50 |
| rm_you | hmm | 19:50 |
| rm_you | it seems to flake8 fine | 19:50 |
| reaperhulk | and if it does, it's ugly. | 19:50 |
| rm_you | it IS ugly | 19:50 |
| rm_you | fair | 19:50 |
| rm_you | i like the textwrap suggestion though, i will check it out | 19:51 |
| alee | dave-mccowan, can you try adding "\n" to the end and see if the test runs ok -- it all seems to work for me just fine. | 19:51 |
| rm_you | second cool python lib thing i've been introduced to this week :P | 19:51 |
| rm_you | that and itertools.ifilter | 19:51 |
| alee | yup | 19:51 |
| dave-mccowan | alee yep. trying it out now. | 19:52 |
| dave-mccowan | alee, reaperhulk same error after adding "\n" | 19:55 |
| reaperhulk | huh | 19:56 |
| dave-mccowan | alee i'll try skipping these hard coded keys altogether and call crypto to get a fresh key pair. | 19:57 |
| alee | dave-mccowan, yup - good idea | 19:58 |
| dave-mccowan | alee, reaperhulk an odd thing is that the PEM dumped from "openssl rsa" does not the same as the PEM from tests/utils.py. so something weird is happening between store and retrieve, even after I added the newlines. | 19:59 |
| reaperhulk | when using openssl rsa it's going to output a PKCS1/Traditional OpenSSL format rather than PKCS8 | 20:02 |
| reaperhulk | (but it can read PKCS8) | 20:03 |
| reaperhulk | that's why it looks different | 20:03 |
| *** kfarr has quit IRC | 20:09 | |
| alee | dave-mccowan, can you dump the data before denormalize_after_decryption() ? | 20:20 |
| alee | dave-mccowan, and try load_privatekey() on that? | 20:20 |
| dave-mccowan | alee sure. | 20:21 |
| alee | using PEM and ASN1 file formats | 20:21 |
| dave-mccowan | alee, reaperhulk there's not a crypto.dump_publickey(). is there some crypto foo to get the public key in PEM format? | 20:22 |
| alee | dave-mccowan, yeah - but its ugly | 20:22 |
| dave-mccowan | alee do we need it to fill a container if we want to test with generated keys? | 20:23 |
| alee | dave-mccowan, I just want to see whats coming out of the get_secret call. | 20:24 |
| alee | and what whether it will work in _generate_csr() | 20:24 |
| dave-mccowan | alee i know, i'm still working on [15:57:55] to generate a container with a new key pair for testing, instead of using the hard coded ones. | 20:25 |
| dave-mccowan | alee, i'll stash this for now and try removing denormalize. | 20:26 |
| alee | dave-mccowan, thanks -- unfortunately I can't debug this myself because it works for me. | 20:27 |
| reaperhulk | alee: version of OpenSSL? | 20:27 |
| reaperhulk | same for you dave | 20:27 |
| reaperhulk | I believe this is an issue with the version of OpenSSL underlying this | 20:27 |
| reaperhulk | I can parse this DER with 1.0.2 (and 1.0.1) but not 0.9.8 | 20:27 |
| alee | openssl-1.0.1e-41.fc20.x86_64 | 20:27 |
| alee | interesting -- | 20:28 |
| alee | dave-mccowan, whats your version of openssl? | 20:28 |
| dave-mccowan | OpenSSL 1.0.2 22 Jan 2015 | 20:28 |
| reaperhulk | dave-mccowan: what OS are you on? | 20:28 |
| dave-mccowan | mac | 20:28 |
| reaperhulk | so that's homebrew openssl | 20:28 |
| reaperhulk | do this for me in your barbican venv | 20:29 |
| reaperhulk | python -c "from cryptography.hazmat.backends.openssl import backend;print(backend.openssl_version_text())" | 20:29 |
| * reaperhulk expects it to say 0.9.8zc | 20:29 | |
| dave-mccowan | OpenSSL 0.9.8za 5 Jun 2014 | 20:30 |
| alee | mine says "OpenSSL 1.0.1e-fips 11 Feb 2013" | 20:30 |
| reaperhulk | yep, so it's definitely the 0.9.8 vs 1.0.x issue | 20:30 |
| alee | woodster_, what do you have? | 20:30 |
| reaperhulk | He'll definitely have 0.9.8 | 20:30 |
| reaperhulk | any mac is going to link against 0.9.8 by default unless you pass some extra flags | 20:31 |
| reaperhulk | Why this key is having trouble under 0.9.8 is an interesting question | 20:31 |
| alee | reaperhulk, so - on your system, what do you have? | 20:31 |
| reaperhulk | I have 0.9.8zc and 1.0.2a, but I link my cryptography explicitly against 1.0.2a :) | 20:31 |
| reaperhulk | (I also rebuild cryptography several times a day, heh) | 20:31 |
| alee | reaperhulk, so -- if I can replace what I have with cryptography calls -- then it will pull in 1.0.2a? | 20:32 |
| alee | reaperhulk, can you confirm that the functional test runs on your system with the right flags? | 20:33 |
| alee | reaperhulk, of course all of this is moot if whats in the gate is 0.9.8 | 20:34 |
| reaperhulk | cryptography will link against 0.9.8 by default on OS X (since that's what Apple ships) | 20:34 |
| reaperhulk | I'm looking into this a bit more now | 20:34 |
| alee | cool :) | 20:36 |
| dave-mccowan | yep, real question is how to make a DER that 0.9.8 likes. | 20:36 |
| reaperhulk | 0.9.8 is perfectly capable of reading it, but you have to pass -nocrypt | 20:36 |
| reaperhulk | I vaguely recall this being an issue with OpenSSL not properly falling back on its parse paths | 20:36 |
| reaperhulk | There's a function inside OpenSSL called d2i_AutoPrivateKey but it had bugs | 20:37 |
| reaperhulk | So much so that we actually wrote our own loaders in cryptography | 20:37 |
| reaperhulk | https://github.com/pyca/cryptography/blob/master/src/cryptography/hazmat/backends/openssl/backend.py#L721 | 20:37 |
| reaperhulk | note the comment :) | 20:37 |
| reaperhulk | So you are running afoul of this exact bug in OpenSSL ade | 20:38 |
| reaperhulk | Arguably I should go patch this in pyOpenSSL as well, but that won't resolve your problem right now... | 20:38 |
| reaperhulk | so where is this getting decoded into DER? | 20:41 |
| *** chadlung has quit IRC | 20:43 | |
| *** rm_work|away is now known as rm_work | 20:47 | |
| reaperhulk | (If possible, let's just not do DER PKCS8) | 20:50 |
| reaperhulk | at least not until we can switch to cryptography, which will handle this pain for you | 20:50 |
| reaperhulk | but that's going to be a liberty thing (hopefully) | 20:50 |
| *** rm_work is now known as rm_work|away | 20:53 | |
| alee | sorry -- trying to deal with a flat tire issue | 20:56 |
| alee | (grandma has flat tire on other side of town) | 20:57 |
| alee | reaperhulk, dave-mccowan -- as part of the whole content types thing we standardized on pkcs8 | 20:58 |
| alee | now that said -- what we should expect to get back from the plugins is PEM | 20:59 |
| alee | dave-mccowan, reaperhulk - which is converted to DER in the denormalize function | 21:00 |
| alee | dave-mccowan, can you try adding a parameter to get_secret() that tells it not to call denormalize()? | 21:00 |
| alee | dave-mccowan, and then load using PEM? | 21:02 |
| dave-mccowan | alee ok | 21:04 |
| *** nkinder has quit IRC | 21:11 | |
| *** jaosorior has quit IRC | 21:12 | |
| openstackgerrit | Thomas Dinkjian proposed openstack/barbican: Updating Orders functional tests to new naming convention https://review.openstack.org/168485 | 21:12 |
| dave-mccowan | alee coded. now different error, debugging now. | 21:12 |
| jamielennox | can someone tell me what's causing http://logs.openstack.org/50/148450/3/check/gate-python-barbicanclient-devstack-dsvm/7d30cfc/console.html.gz#_2015-03-20_07_20_03_243 | 21:12 |
| jamielennox | https://review.openstack.org/#/c/148450/ | 21:14 |
| dave-mccowan | alee hmmm. code is failing in same place, this time parsing PEM format. one odd thing is the \ns disappeared somehow, which may be causing error. | 21:18 |
| *** kebray has quit IRC | 21:18 | |
| redrobot | jamielennox it appears that the exception raised for a 404 in the new Adapter class you're introducing does not include an http_status attribute, which is what the functional test is looking for | 21:19 |
| alee | dave-mccowan, the \n's are in your code? | 21:19 |
| dave-mccowan | reaperhulk, alee do i need to add \n or \\n ? | 21:19 |
| redrobot | jamielennox I don't think it's a useful test though, so you could probably just delete this assert https://github.com/openstack/python-barbicanclient/blob/master/functionaltests/client/v1/functional/test_secrets.py#L142 | 21:19 |
| alee | maybe remove the \n's and see what it does? | 21:19 |
| dave-mccowan | alee is the code expecting only base64 characters? | 21:20 |
| redrobot | jamielennox I think the fact that an exception is raised is good enough. We're planning on revisiting the exceptions being thrown for 4xx and 5xx errors | 21:20 |
| jamielennox | redrobot: i'm interested in what that exception object is though, the standard exceptions don't have a http_status https://github.com/openstack/python-barbicanclient/blob/master/barbicanclient/client.py#L34-L57 | 21:20 |
| alee | dave-mccowan, so whats coming out of the plugin is base64 encoded | 21:20 |
| alee | dave-mccowan, if you dump it -- can you see base 64 + headers> | 21:21 |
| alee | ? | 21:21 |
| redrobot | jamielennox it's likely a keystoneclient exception... there's a few places where we're not catching the underlying exceptions and raising a Barbicanclient exception. | 21:21 |
| jamielennox | redrobot: hmmm, that's weird... i wonder why that's happening... | 21:22 |
| redrobot | jamielennox we've filed this bug to track the work being done to prevent 3rd party exceptions from bubbling up like that https://bugs.launchpad.net/python-barbicanclient/+bug/1431514 | 21:22 |
| openstack | Launchpad bug 1431514 in python-barbicanclient "client shouldn't return http errors directly" [Medium,Confirmed] | 21:22 |
| jamielennox | redrobot: well hopefully soon we can standardize the http errors across all the clients | 21:22 |
| dave-mccowan | alee when i dump it, it is base64 + headers. but the newline characters have been removed. | 21:23 |
| alee | dave-mccowan, maybe \\n ? | 21:24 |
| *** kebray has joined #openstack-barbican | 21:25 | |
| jamielennox | redrobot: do you know how the keystone error would bubble up like that? | 21:25 |
| jamielennox | https://github.com/openstack/python-barbicanclient/blob/master/functionaltests/client/v1/functional/test_secrets.py#L142 is the test | 21:26 |
| jamielennox | and _delete is doing it's own status_code checking: https://github.com/openstack/python-barbicanclient/blob/master/barbicanclient/client.py#L103 | 21:26 |
| alee | dave-mccowan, maybe it makes sense at this point to take the '\n' thing out of the equation and just generate keys? | 21:27 |
| jamielennox | redrobot: oh - it looks like it's not getting to that point at all, it's coming from the session directly | 21:27 |
| alee | dave-mccowan, there is code that does that in the test_certificate_resources.py | 21:27 |
| redrobot | jamielennox yeah, the _session.delete() should be done in a try/except | 21:28 |
| alee | dave-mccowan, in fact, I suspect that if we do that, this whole issue might just go away | 21:29 |
| jamielennox | redrobot: that or if you pass raise_exc=False like in https://review.openstack.org/#/c/148450/3/barbicanclient/client.py L86 then request() will return the response object with >400 status codes and the existing error handling will work | 21:29 |
| alee | irrespective of whether we skip the denormalization or not | 21:29 |
| alee | dave-mccowan, though tbh - we may end up skipping denormalization in any case to support the passphrase case | 21:31 |
| dave-mccowan | alee i have that mostly coded, just need a way to get the public key from pkey | 21:31 |
| redrobot | jamielennox your CR makes that change right? Looks like it's definitely a bad assert then. | 21:31 |
| alee | dave-mccowan, try using pycrypto instead .. | 21:31 |
| alee | let me point you to the right test .. | 21:31 |
| jamielennox | redrobot: well i don't want to mix the two changes, i'll fix it so that the behaviour is the same as now, and i'll leave a note as to how to fix it if you want to handle those errors yourself. | 21:32 |
| openstackgerrit | Jamie Lennox proposed openstack/python-barbicanclient: Use the ksc Adapter instead of custom HTTPClient https://review.openstack.org/148450 | 21:32 |
| alee | dave-mccowan, test_should_return_for_pycrypto_stored_key_without_passphrase() | 21:32 |
| alee | in test_certificate_resources.py | 21:33 |
| alee | dave-mccowan, right now -- the public key does not really matter | 21:33 |
| alee | we just need and get the private key | 21:34 |
| redrobot | jamielennox sounds good | 21:34 |
| dave-mccowan | alee yea, i think the container validators gripe if it's not there though. | 21:34 |
| alee | dave-mccowan, sur but any value is ok | 21:34 |
| dave-mccowan | alee pycrypto everywhere and the world is a better place? | 21:35 |
| alee | dave-mccowan, you can send "public key" | 21:35 |
| dave-mccowan | alee OK... same error. still wind up with "PEM format" with no newlines. | 21:41 |
| *** chadlung has joined #openstack-barbican | 21:44 | |
| alee | dave-mccowan, interesting -- the test_should_return_for_openssl_stored_key_ca_id_passed_in() unit test works .. | 21:45 |
| alee | dave-mccowan, so that says something changes when we actually store and then retrieve | 21:47 |
| *** bdpayne has joined #openstack-barbican | 21:48 | |
| *** chadlung has quit IRC | 21:49 | |
| dave-mccowan | alee that would be a good functional test. does original == retrieved. for the rsa key case, at least the newlines are stripped. | 21:49 |
| alee | dave-mccowan, well I'm looking at the test_secrets() functional test | 21:49 |
| alee | and that seems to pass too | 21:49 |
| alee | using the utils.get_private_key() | 21:50 |
| dave-mccowan | alee so bug is specific to rsa container ? | 21:50 |
| alee | dave-mccowan, I think maybe when we store the private key , we are not passing in the right content types | 21:51 |
| alee | payload_content_type': 'application/octet-stream', | 21:51 |
| alee | 'payload_content_encoding': 'base64', | 21:51 |
| alee | we need to pass in the right content type and encoding | 21:51 |
| alee | dave-mccowan, I think that may be the problem and why when we retrieve, we get the wrong thing | 21:53 |
| alee | dave-mccowan, see the tests in there for private and public key | 21:53 |
| dave-mccowan | alee what should the encoding be? | 21:57 |
| alee | dave-mccowan, ok -- let me look at the secret functional tests | 21:57 |
| dave-mccowan | alee i followed that example. i guess base64 is not right, because PEM is base64 + headers + and newlines. | 21:59 |
| alee | dave-mccowan, actually it looks like you followed the example .. | 22:00 |
| alee | dave-mccowan, in _generate_csr() | 22:00 |
| alee | there is a call to get_secret() | 22:00 |
| alee | and a content type in there -- | 22:00 |
| alee | maybe that content type should be 'application/octet-stream' ? | 22:01 |
| dave-mccowan | alee change from application/pkcs8 to application/octet-stream? | 22:02 |
| alee | yup | 22:03 |
| dave-mccowan | alee no difference | 22:04 |
| alee | dave-mccowan, sorry - still trying to deal with tire thing | 22:10 |
| alee | diagnosing flat tire across town over phone is like trying to diagnose openssl code issue remotely :) | 22:10 |
| dave-mccowan | alee lol | 22:11 |
| alee | dave-mccowan, will have to get back to you tommorow morning or later tonight | 22:11 |
| dave-mccowan | alee a break sounds like a good idea. let's hope for epiphanies. | 22:11 |
| alee | ok | 22:12 |
| alee | thanks | 22:12 |
| *** paul_glass has quit IRC | 22:14 | |
| *** xaeth is now known as xaeth_afk | 22:17 | |
| *** bdpayne has quit IRC | 22:19 | |
| *** kebray has quit IRC | 22:20 | |
| *** chadlung has joined #openstack-barbican | 22:25 | |
| *** dimtruck is now known as zz_dimtruck | 22:25 | |
| *** chadlung has quit IRC | 22:25 | |
| *** dave-mccowan has quit IRC | 22:28 | |
| *** tkelsey has joined #openstack-barbican | 22:28 | |
| woodster_ | reaperhulk or jvrbanac, were you able to autogenerate the alembic migration that adds FK indices, or was that hand generated? | 22:31 |
| *** tkelsey has quit IRC | 22:32 | |
| *** nkinder has joined #openstack-barbican | 22:34 | |
| reaperhulk | we autogenerated it and then tweaked by hand a bit | 22:35 |
| woodster_ | for some reason even though the FK is indexed, the generated alembic file didn't have the constraint generated: https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/aa2cf96a1d5_add_orderretrytask.py | 22:37 |
| *** dave-mccowan has joined #openstack-barbican | 22:40 | |
| *** kebray has joined #openstack-barbican | 23:02 | |
| *** chlong has joined #openstack-barbican | 23:07 | |
| *** kebray has quit IRC | 23:08 | |
| *** kebray has joined #openstack-barbican | 23:09 | |
| openstackgerrit | John Wood proposed openstack/barbican: Add order_retry_tasks migration per latest model https://review.openstack.org/169946 | 23:16 |
| *** joesavak has quit IRC | 23:21 | |
| *** jkf has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!