| *** kebray has joined #openstack-barbican | 00:05 | |
| *** dave-mccowan has joined #openstack-barbican | 00:23 | |
| *** dave-mccowan has quit IRC | 00:29 | |
| *** SheenaG has quit IRC | 00:31 | |
| *** SheenaG has joined #openstack-barbican | 00:32 | |
| *** gyee has quit IRC | 00:35 | |
| *** SheenaG has quit IRC | 00:41 | |
| *** SheenaG has joined #openstack-barbican | 00:41 | |
| *** dimtruck is now known as zz_dimtruck | 00:46 | |
| *** SheenaG has quit IRC | 01:11 | |
| *** zz_dimtruck is now known as dimtruck | 01:25 | |
| *** kebray has quit IRC | 01:36 | |
| *** SheenaG has joined #openstack-barbican | 02:08 | |
| elmiko | anyone around? | 02:10 |
|---|---|---|
| *** igueths has quit IRC | 02:10 | |
| *** rm_work|away is now known as rm_work | 02:52 | |
| *** rm_work is now known as rm_work|away | 03:05 | |
| *** dave-mccowan has joined #openstack-barbican | 03:14 | |
| *** xaeth_afk is now known as xaeth | 03:19 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix expectations of order certificate test cases https://review.openstack.org/173163 | 03:42 |
| *** xaeth is now known as xaeth_afk | 03:45 | |
| *** SheenaG has left #openstack-barbican | 03:51 | |
| *** rm_work|away is now known as rm_work | 03:52 | |
| *** xaeth_afk is now known as xaeth | 04:03 | |
| *** woodster_ has quit IRC | 04:10 | |
| *** rm_you| has joined #openstack-barbican | 04:32 | |
| *** rm_you has quit IRC | 04:35 | |
| *** alee_afk has quit IRC | 04:36 | |
| *** alee_afk has joined #openstack-barbican | 04:36 | |
| *** woodster_ has joined #openstack-barbican | 04:53 | |
| *** xaeth is now known as xaeth_afk | 05:01 | |
| openstackgerrit | Chelsea Winfree proposed openstack/python-barbicanclient: Adding raw and clif formatted payload https://review.openstack.org/173172 | 05:12 |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/barbican: Imported Translations from Transifex https://review.openstack.org/173180 | 06:07 |
| *** dimtruck is now known as zz_dimtruck | 06:23 | |
| *** dave-mccowan has quit IRC | 06:58 | |
| *** jamielennox is now known as jamielennox|away | 07:02 | |
| *** woodster_ has quit IRC | 07:20 | |
| *** chlong has quit IRC | 07:27 | |
| *** tkelsey has joined #openstack-barbican | 08:19 | |
| *** zz_dimtruck is now known as dimtruck | 08:52 | |
| *** jaosorior has joined #openstack-barbican | 08:53 | |
| *** dimtruck is now known as zz_dimtruck | 09:02 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/barbican: Enable alternate error message for OpenSSL 1.0.2 https://review.openstack.org/172844 | 10:46 |
| *** zz_dimtruck is now known as dimtruck | 11:12 | |
| openstackgerrit | Juan Antonio Osorio Robles proposed openstack/barbican: Readability-related changes to secret store functions https://review.openstack.org/172378 | 11:27 |
| *** dimtruck is now known as zz_dimtruck | 11:35 | |
| *** woodster_ has joined #openstack-barbican | 12:15 | |
| *** rellerreller has joined #openstack-barbican | 12:34 | |
| *** rellerreller has quit IRC | 13:04 | |
| *** zz_dimtruck is now known as dimtruck | 13:10 | |
| elmiko | redrobot: hey, i ran through the migrations with a true mysql server last night. it did not go so well | 13:19 |
| *** nkinder has joined #openstack-barbican | 13:39 | |
| *** alee has joined #openstack-barbican | 13:45 | |
| *** joesavak has joined #openstack-barbican | 13:50 | |
| *** stanzi has joined #openstack-barbican | 13:50 | |
| *** dimtruck is now known as zz_dimtruck | 14:01 | |
| *** zz_dimtruck is now known as dimtruck | 14:13 | |
| *** paul_glass has joined #openstack-barbican | 14:15 | |
| *** xaeth_afk is now known as xaeth | 14:23 | |
| openstackgerrit | Merged openstack/barbican: Imported Translations from Transifex https://review.openstack.org/173180 | 14:34 |
| *** stanzi has quit IRC | 14:42 | |
| *** stanzi has joined #openstack-barbican | 14:42 | |
| *** stanzi_ has joined #openstack-barbican | 14:46 | |
| *** stanzi has quit IRC | 14:47 | |
| *** dave-mccowan has joined #openstack-barbican | 14:51 | |
| *** xaeth is now known as xaeth_afk | 14:51 | |
| *** nkinder has quit IRC | 14:58 | |
| *** xaeth_afk is now known as xaeth | 14:59 | |
| *** rellerreller has joined #openstack-barbican | 15:04 | |
| *** joesavak has quit IRC | 15:04 | |
| *** darrenmoffat has quit IRC | 15:12 | |
| *** darrenmoffat has joined #openstack-barbican | 15:13 | |
| *** stanzi_ has quit IRC | 15:16 | |
| *** stanzi has joined #openstack-barbican | 15:17 | |
| elmiko | redrobot: you around? | 15:17 |
| *** xaeth is now known as xaeth_afk | 15:19 | |
| *** xaeth_afk is now known as xaeth | 15:19 | |
| *** stanzi has quit IRC | 15:20 | |
| *** stanzi has joined #openstack-barbican | 15:20 | |
| redrobot | elmiko o/ | 15:26 |
| elmiko | redrobot: hey, ran into a bunch of weird issues with mysql | 15:26 |
| elmiko | seems that true mysql is a little more strict than maria | 15:26 |
| elmiko | i wanted to confirm something though, | 15:27 |
| elmiko | in stable/juno, the db schema gets create when a call comes in? is there another way to create the schema? | 15:27 |
| redrobot | elmiko so, I think there's a config option that controls whether the API attempts to build the schema or not | 15:30 |
| elmiko | oh right, the auto create field | 15:31 |
| *** SheenaG has joined #openstack-barbican | 15:31 | |
| elmiko | redrobot: so, the big issues i am running into with mysql are that drop_constraints and alter_column seem to want type information | 15:31 |
| elmiko | the alembic docs mention this for the drop_constraints method | 15:31 |
| elmiko | i've fixed a couple of migrations on a local branch, but now i'm running into a new sticking point | 15:32 |
| elmiko | redrobot: i think it's probably worthwhile to have someone else take a look at this too. i'm concerned that some of my fixes are a little too hacky. | 15:34 |
| elmiko | i'm at the edge of my sql knowledge, and there are some odd constraint keys that are getting creating automagically that i'm not sure about | 15:35 |
| redrobot | elmiko sure... let me ping woodster_ and see if he has time to poke at that stuff | 15:36 |
| redrobot | elmiko he's way better at db stuff than I am | 15:36 |
| elmiko | redrobot: ok, cool. i can pass on what i've found to him. maybe it's a little more simple than i imagine | 15:36 |
| dave-mccowan | redrobot, ping. | 15:38 |
| redrobot | dave-mccowan pong | 15:39 |
| dave-mccowan | redrobot, how's it going with fixing the b64 encoding? let me know if you can help. i have a fix for #1443008, but the test case needs all the b64 stuff straight to pass. | 15:40 |
| redrobot | dave-mccowan almost done... I had to touch a bunch of stuff and fix a lot of functional tests. | 15:41 |
| redrobot | dave-mccowan I just have the stored-key case left. | 15:41 |
| *** gyee has joined #openstack-barbican | 15:42 | |
| dave-mccowan | redrobot, awesome! i've become very familiar with that code. just let me know if you'd like any early review or extra testing. | 15:42 |
| redrobot | dave-mccowan I can push a WIP right now if you want to take a look at it | 15:43 |
| dave-mccowan | redrobot, sure. | 15:43 |
| alee | redrobot, dave-mccowan - let me know when ya'll think you have something ready to work together, and I'll test it against dogtag | 15:44 |
| woodster_ | elmiko: redrobot mentioned you were dealing with mysql migrations? | 15:44 |
| elmiko | woodster_: yea, running into a bunch of weird issue | 15:46 |
| elmiko | woodster_: https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/795737bb3c3_change_tenants_to_projects.py | 15:46 |
| elmiko | that migration is where my problems started | 15:46 |
| *** stanzi has quit IRC | 15:46 | |
| woodster_ | elmiko: I'm wondering if the 'InnoDB' lines in the models.py module are breaking things for mysql.... | 15:46 |
| openstackgerrit | Douglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 15:46 |
| redrobot | dave-mccowan alee ^^ it's going to fail the stored-key case | 15:47 |
| *** stanzi has joined #openstack-barbican | 15:47 | |
| elmiko | woodster_: hmm, not sure | 15:47 |
| *** stanzi_ has joined #openstack-barbican | 15:48 | |
| rellerreller | redrobot I am also interested when you are done. I'm glad others are seeing how much fine content types are! | 15:48 |
| elmiko | woodster_: what other options could i try? | 15:49 |
| elmiko | woodster_: seems like the main complaint i'm getting to start with is that drop_constraints and alter_column want to know the types in mysql | 15:49 |
| woodster_ | elmiko, well, there are sqlalchemy calls that can be made for migrations as well (instead of alembic op)...maybe try to compare to some other version files | 15:49 |
| redrobot | rellerreller haha yeah, some of that stuff is like a rube goldberg machine | 15:50 |
| elmiko | woodster_: fixing the type information definitely helped | 15:50 |
| redrobot | rellerreller I want to refactor it, but I'm trying to keep this CR small, so I'll probably come back around after everything is working and simplify a lot of it. | 15:50 |
| elmiko | woodster_: but i'm getting a little turned around about the constraints because it seems there are hidden constraint columns that are being created and i'm having to remove them by name | 15:51 |
| *** dave-mccowan has quit IRC | 15:51 | |
| *** stanzi has quit IRC | 15:51 | |
| *** stanzi_ has quit IRC | 15:51 | |
| *** stanzi has joined #openstack-barbican | 15:51 | |
| woodster_ | elmiko: check out 1a7cf79559e3_new_secret_and_container_acl_tables.py | 15:54 |
| woodster_ | elmiko: it uses the sqlalchemy foo | 15:56 |
| elmiko | woodster_: yea, i looked at that one. i'll study further | 15:56 |
| *** dave-mccowan has joined #openstack-barbican | 15:57 | |
| woodster_ | elmiko: please let me know what you discover. I'm in various sprint planning meetings today but will be able to help out later this afternoon | 16:01 |
| woodster_ | elmiko: I'll be on IRC throughout though if you discover something | 16:01 |
| rellerreller | redrobot I put some comments on https://review.openstack.org/173396 | 16:02 |
| *** kebray has joined #openstack-barbican | 16:02 | |
| elmiko | woodster_: ack, i'll try to collect my results | 16:02 |
| redrobot | rellerreller thanks! | 16:03 |
| *** joesavak has joined #openstack-barbican | 16:04 | |
| *** kebray has quit IRC | 16:06 | |
| *** joesavak has quit IRC | 16:11 | |
| rellerreller | aa | 16:13 |
| *** stanzi has quit IRC | 16:20 | |
| *** stanzi has joined #openstack-barbican | 16:21 | |
| *** stanzi has quit IRC | 16:26 | |
| *** xaeth is now known as xaeth_afk | 16:31 | |
| *** xaeth_afk is now known as xaeth | 16:38 | |
| *** joesavak has joined #openstack-barbican | 16:43 | |
| *** stanzi has joined #openstack-barbican | 16:43 | |
| dave-mccowan | redrobot, w00t. Ran 18 tests in 3.775s OK | 16:46 |
| *** rellerreller has quit IRC | 16:51 | |
| *** dimtruck is now known as zz_dimtruck | 16:51 | |
| *** stanzi has quit IRC | 16:53 | |
| *** stanzi has joined #openstack-barbican | 16:53 | |
| *** stanzi_ has joined #openstack-barbican | 16:54 | |
| *** stanzi has quit IRC | 16:54 | |
| elmiko | woodster_: so yea, looks like the issues i'm having with mysql are involving the extra columns that are created with respect to the constraints. | 16:56 |
| *** xaeth is now known as xaeth_afk | 16:58 | |
| *** zz_dimtruck is now known as dimtruck | 16:58 | |
| *** joesavak has quit IRC | 17:05 | |
| *** stanzi_ has quit IRC | 17:06 | |
| *** stanzi has joined #openstack-barbican | 17:07 | |
| *** kebray has joined #openstack-barbican | 17:08 | |
| *** stanzi has quit IRC | 17:11 | |
| *** joesavak has joined #openstack-barbican | 17:14 | |
| *** SheenaG has quit IRC | 17:14 | |
| *** alee is now known as alee_lunch | 17:17 | |
| *** rellerreller has joined #openstack-barbican | 17:31 | |
| *** tkelsey has quit IRC | 17:33 | |
| *** SheenaG has joined #openstack-barbican | 17:37 | |
| *** tkelsey has joined #openstack-barbican | 17:37 | |
| *** stanzi has joined #openstack-barbican | 17:37 | |
| *** tkelsey has quit IRC | 17:43 | |
| *** stanzi has quit IRC | 17:46 | |
| *** jaosorior has quit IRC | 17:52 | |
| redrobot | dave-mccowan hey I think one of the rsa smoke tests is incorrect | 17:58 |
| redrobot | dave-mccowan https://github.com/openstack/barbican/blob/master/functionaltests/api/v1/smoke/test_rsa.py#L835 | 17:58 |
| redrobot | dave-mccowan the CSR has newlines in it, so it can't be just included in the request | 17:58 |
| dave-mccowan | redrobot, yes. i've got a patch that fixes those issues. | 17:58 |
| *** stanzi has joined #openstack-barbican | 17:58 | |
| dave-mccowan | redrobot, waiting for +2s | 17:58 |
| *** stanzi has quit IRC | 17:59 | |
| redrobot | dave-mccowan link? | 17:59 |
| redrobot | dave-mccowan are we going to base64(CMC) then? The functional test right now is doing json.dumps(CMC) | 17:59 |
| *** stanzi has joined #openstack-barbican | 17:59 | |
| dave-mccowan | redrobot https://review.openstack.org/#/c/173163/ | 18:00 |
| dave-mccowan | redrobot, also https://review.openstack.org/#/c/172819/ | 18:01 |
| dave-mccowan | redrobot, i have a delta from your patch that now has all 18 tests working. i need to figure out how to upload the good parts. | 18:01 |
| dave-mccowan | redrobot, yes it should be base64(cmc). | 18:03 |
| dave-mccowan | redrobot, what git commands can i use to upload my patchset to your CR? | 18:05 |
| *** jamielennox|away is now known as jamielennox | 18:05 | |
| redrobot | dave-mccowan yours depends on mine? | 18:06 |
| dave-mccowan | i got the stored-key case working on a branch that i fetched from your CR. | 18:06 |
| dave-mccowan | test_rsa.py works (but it breaks 20 unit tests, and 10 functional tests, so more work to do) | 18:07 |
| redrobot | oh I see... well hmmm.... so the problem is that mine won't pass the gate | 18:07 |
| redrobot | so we'd have to squash it into a single commit. | 18:07 |
| redrobot | want to add the changes to my CR? | 18:08 |
| *** alee_lunch is now known as alee | 18:08 | |
| redrobot | just git commit --amend, and it'll show both of us as authors. | 18:08 |
| dave-mccowan | i must have check it out wrong... it's trying to amend to another CR | 18:09 |
| *** nkinder has joined #openstack-barbican | 18:13 | |
| redrobot | dave-mccowan git review -d 173396 | 18:15 |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 18:19 |
| dave-mccowan | redrobot, if you agree with those changes, we can split fixing unit and functional tests. | 18:20 |
| redrobot | dave-mccowan looking | 18:25 |
| *** xaeth_afk is now known as xaeth | 18:26 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix functional test test_rsa_order_certificate_from_csr https://review.openstack.org/173452 | 18:33 |
| woodster_ | elmiko: so the MySQL column issue happened even with sqlalchemy correct? | 18:36 |
| *** stanzi has quit IRC | 18:37 | |
| *** stanzi has joined #openstack-barbican | 18:37 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix generating a CSR with an encrypted private key https://review.openstack.org/173457 | 18:39 |
| *** stanzi has quit IRC | 18:41 | |
| *** stanzi has joined #openstack-barbican | 18:42 | |
| woodster_ | Can cores review these two small CRs?: https://review.openstack.org/171868 and https://review.openstack.org/169946 | 18:46 |
| elmiko | woodster_: well, i'm sorting through this | 18:46 |
| elmiko | woodster_: it looks like there are some constraints that get automatically named depending on the db impl | 18:47 |
| elmiko | woodster_: these needs to be deleted for mysql, but first their names need to be determined | 18:47 |
| elmiko | i'm still understanding what these extra constraints are though | 18:48 |
| *** stanzi has quit IRC | 18:48 | |
| elmiko | woodster_: also, this migration is problematic https://github.com/openstack/barbican/blob/master/barbican/model/migration/alembic_migrations/versions/1c0f328bfce0_fixing_composite_primary_keys_and_.py#L39 | 18:49 |
| *** stanzi has joined #openstack-barbican | 18:49 | |
| elmiko | that highlighted line is using a column name that is postgresql specific as far as i can tell | 18:49 |
| elmiko | woodster_: like, that column preferred_certificate_authorities_project_id_key is the name generated for postgres by sqlalchemy, but as near as i can tell it would be preferred_certificate_authorities_project_ibfk_1 for mysql | 18:51 |
| *** stanzi has quit IRC | 18:51 | |
| elmiko | so... basically, fun times ;) | 18:51 |
| *** stanzi has joined #openstack-barbican | 18:51 | |
| redrobot | dave-mccowand looks good. I'm going to remove the skip for one of the tests | 18:54 |
| redrobot | dave-mccowan stored key is still failing for me though | 18:55 |
| dave-mccowan | redrobot, i think i missed i file when moving between branches. | 18:55 |
| woodster_ | elmiko: so maybe part of the issue is using alembic's op instead of sqlalchemy. Maybe we could then genetically name our constraints and sqlalchemy maps to database specific names? | 18:57 |
| elmiko | woodster_: i think that would help, but i'm still not quite understanding why these extra constraints only cause an issue with mysql | 18:58 |
| *** paul_glass has quit IRC | 18:58 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 19:00 |
| dave-mccowan | redrobot, i missed my changes to translations.py ^^ | 19:00 |
| *** paul_glass has joined #openstack-barbican | 19:01 | |
| hockeynut | tdink jvrbanac would y'all be so kind as to take a look at https://review.openstack.org/#/c/172604/ and hit me with your comments? | 19:02 |
| elmiko | woodster_: the annoying thing is that for postgresql the op.drop_constraints call seems to get rid of the associated constraints fields. but for mysql they need to be removed by name. | 19:10 |
| *** dave-mccowan has quit IRC | 19:10 | |
| elmiko | woodster_: take a gander at https://gist.github.com/elmiko/9badea7384aaec355a06 | 19:14 |
| elmiko | on mysql, the tenant_secret_ibfk_1 foreign key constraint needs to be deleted before the _tenant_secret_uc unique can be dropped | 19:14 |
| elmiko | on postrgresl this doesn't seem to matter | 19:15 |
| elmiko | likewise, in the 1c0f328bfce0 migration, the drop_constraint needs extra work for mysql | 19:16 |
| woodster_ | elmiko: my hope is that we can use sqlalchemy only, and give the constraint a name like 'tenant_secret_uc' and then sqlalchemy translates to the db specific naming under the hood | 19:17 |
| woodster_ | elmiko: so we would need to name all constraints though | 19:18 |
| elmiko | woodster_: it seems the actual unique constraints do have names assigned in sqlalchemy, alembic seems to complain about the implied, and unnamed, foreign key constraints | 19:18 |
| elmiko | woodster_: yea, i think so | 19:18 |
| elmiko | woodster_: some of them are named though | 19:19 |
| woodster_ | elmiko: these names would need to be synced with those in models.py too | 19:20 |
| elmiko | woodster_: that makes sense, this all just smells really fishy because the error i get when removing the named constraint (_tenant_secret_uc) is a complaint about it being used in another constraint (tenant_secret_ibfk_1) | 19:22 |
| woodster_ | elmiko: yeah that is odd | 19:23 |
| woodster_ | elmiko: my hunch is we need consistent and unique constraints everywhere | 19:24 |
| woodster_ | elmiko: ....or else things get out of whack | 19:24 |
| elmiko | yea | 19:24 |
| redrobot | dave's not here man | 19:30 |
| elmiko | woodster_: i *think* if you look at the models.ProjectSecret that the project_id field which contains a ForeignKey constraint in the schema, that it would need a name | 19:33 |
| *** dave-mccowan has joined #openstack-barbican | 19:33 | |
| elmiko | woodster_: or, it would be need to be added as a constraint separately | 19:33 |
| elmiko | woodster_: then, they could be dropped consistently | 19:33 |
| woodster_ | elmiko: I think that's the case. My guess is other openstack projects are doing this too | 19:35 |
| elmiko | woodster_: i'm looking at the sahara base to see what it does, but there aren't any migrations with explicitly named constraints being dropped | 19:36 |
| elmiko | woodster_: i think the rename is just tricky | 19:36 |
| elmiko | woodster_: i'll keep playing around with this though | 19:36 |
| *** dave-mccowan has quit IRC | 19:37 | |
| *** SheenaG has quit IRC | 19:39 | |
| *** rellerreller has quit IRC | 19:40 | |
| redrobot | does anyone have time to review https://review.openstack.org/#/c/172714/3 ? | 19:45 |
| redrobot | I need it to rebase my work on top of the openssl 1.0.2 fix | 19:45 |
| redrobot | woodster_ you got time to workflow? https://review.openstack.org/#/c/172714/3 | 19:46 |
| redrobot | woodster_ should be a quick review | 19:47 |
| *** SheenaG has joined #openstack-barbican | 20:01 | |
| openstackgerrit | Douglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 20:03 |
| *** dimtruck is now known as zz_dimtruck | 20:04 | |
| *** stanzi has quit IRC | 20:07 | |
| *** stanzi has joined #openstack-barbican | 20:08 | |
| *** zz_dimtruck is now known as dimtruck | 20:09 | |
| *** stanzi has quit IRC | 20:12 | |
| *** igueths has joined #openstack-barbican | 20:14 | |
| *** dave-mccowan has joined #openstack-barbican | 20:14 | |
| openstackgerrit | Merged openstack/barbican: Sign CSRs issued in SnakeOilCA tests https://review.openstack.org/172714 | 20:16 |
| openstackgerrit | Merged openstack/barbican: Enable alternate error message for OpenSSL 1.0.2 https://review.openstack.org/172844 | 20:17 |
| igueths | Hey all, could I get some reviews on https://review.openstack.org/170693? | 20:19 |
| redrobot | anyone here familiar with pykmip? | 20:20 |
| *** tkelsey has joined #openstack-barbican | 20:20 | |
| *** crc32 has joined #openstack-barbican | 20:22 | |
| *** tkelsey has quit IRC | 20:24 | |
| redrobot | alee ping? | 20:29 |
| redrobot | kfarr around? | 20:29 |
| redrobot | nope | 20:29 |
| elmiko | woodster_: think i'm making some progress | 20:34 |
| woodster_ | elmiko: hey, I'm back from meetings now...good timing then? ;) | 20:35 |
| elmiko | hehe yea | 20:35 |
| woodster_ | elmiko: is naming all the constraints the answer? | 20:35 |
| elmiko | woodster_: looks like it | 20:35 |
| elmiko | woodster_: it's working for both postgresql and mysql, now i'm worried that the fk constraints aren't getting replaced after the rename | 20:36 |
| woodster_ | elmiko: so hopefully no need to replace the alembic op calls with sqlalchemy ones then? | 20:36 |
| elmiko | woodster_: hopefully /me fingers-crossed | 20:36 |
| elmiko | woodster_: if i get this patch working maybe i can share it and we can figure out how best to put it up for review? | 20:36 |
| woodster_ | elmiko: that sounds good. So I imagine a few of those version files will need to be tweaked, maybe the models.py as well? | 20:37 |
| elmiko | woodster_: yup, both | 20:38 |
| *** igueths has quit IRC | 20:42 | |
| woodster_ | elmiko: wow, so yeah maybe better to separate those into separate smaller CRs then | 20:43 |
| elmiko | woodster_: it's gonna be complicated... | 20:43 |
| *** igueths has joined #openstack-barbican | 20:44 | |
| woodster_ | elmiko: can you just do it one entity/model at a time...so each one just making part of the overall migration work correctly | 20:44 |
| elmiko | woodster_: logically, i think we could do a review for the model changes + associated migrations, then the additional migration fixes | 20:44 |
| elmiko | woodster_: i think so | 20:44 |
| redrobot | elmiko woodster_ will this involve editing existing migrations? | 20:45 |
| elmiko | redrobot: unfortunately yea | 20:45 |
| redrobot | elmiko yikes... well, that'll be fun to sort out for our deployment. | 20:45 |
| elmiko | redrobot: it also means that existing stable/juno -> kilo migration is probably not realistic | 20:45 |
| woodster_ | redrobot, yeah so it turns out postgres is more forgiving than mysql | 20:45 |
| redrobot | woodster_ think we can get away with rebuilding PROD? | 20:45 |
| alee | redrobot, > | 20:46 |
| alee | ? | 20:46 |
| alee | whats up? | 20:46 |
| woodster_ | redrobot, do you mean our internal prod? | 20:46 |
| elmiko | redrobot: we can make it work but it will mean more introspection of db columns to make the migration work well | 20:46 |
| redrobot | alee was wondering if you knew something about KMIP, but I think I guessed correctly | 20:46 |
| alee | dave-mccowan, redrobot - hows the bug fixes goign? | 20:46 |
| redrobot | woodster_ yeah... if we edit existing migrations, then our current deployment won't match what would result from the new migrations | 20:47 |
| alee | dave-mccowan, redrobot you guys have something reaady for me to test? | 20:47 |
| redrobot | alee almost, https://review.openstack.org/#/c/173396/ is passing all the functional tests, but I still need to fix one more unit test | 20:47 |
| elmiko | redrobot, woodster_, i think there are 2 issues here. 1. existing deployments that need migration, 2. fixing the migrations/models for future stuff | 20:48 |
| alee | redrobot, ok - this is a merge of both yours and dave-mccowan changes? | 20:48 |
| redrobot | alee yep | 20:48 |
| woodster_ | redrobot, elmiko So it seems to me that the changes just tighten up the naming of constraints and so forth which shouldn't affect current schema unless a new name is added that wasn't there before? | 20:48 |
| alee | redrobot, ok cool - I'll test and review it late tonight | 20:48 |
| dave-mccowan | alee, redrobot to get all the tests to pass, i have 4 more CRs outstanding too. we'll need all 5 together. | 20:49 |
| elmiko | woodster_: that sounds about right | 20:49 |
| alee | 4 more .. | 20:49 |
| elmiko | woodster_: also, the anonymous foreignkey constraints need to be named to prevent the bifurcation between mysql postgresql impls in the future | 20:49 |
| elmiko | woodster_, redrobot, so... it might be necessary to do this the hard way if we need to run the migrations on juno deployments. sounds like that might be a goal here? | 20:50 |
| dave-mccowan | alee, redrobot. the 5th patch is a collaboration. the guts is all there, and needs review, but there's about 30 test cases that need to be updated with expections of our clarified b64 design. i've started fixing common/ and plugin/ unit test failures. | 20:50 |
| redrobot | elmiko yes, the main goal is to fix Juno -> Kilo. I don't think we guarantee unreleased migrations. | 20:51 |
| redrobot | dave-mccowan are you building on my CR? | 20:51 |
| redrobot | dave-mccowan I had to rebase to include the openssl changes from jaosorior | 20:52 |
| elmiko | redrobot: ok, with that said. i'll look more at "the hard way" ;) | 20:52 |
| alee | dave-mccowan, redrobot so is there a defined set of patches I need to apply (in order)? | 20:52 |
| redrobot | dave-mccowan alee I've been working with the assumption that others will be depending on my CR | 20:52 |
| dave-mccowan | redrobot, the other four are stand-alone. all future work i'm doing in your CR | 20:52 |
| redrobot | dave-mccowan I'm working on getting my CR to pass the gate, so you can work off of it, instead of sharing it | 20:53 |
| alee | dave-mccowan, so I can apply redrobot patch and then your others in any order? | 20:53 |
| dave-mccowan | alee, redrobot you'd probably have merge conflicts, but functionally any order is OK. hopefully we can merge the other 4 today. they're all small. | 20:54 |
| dave-mccowan | redrobot, i've just been working in your CR, since it needs help to pass the gate. is that the way to do it? or is there a more correct way? | 20:55 |
| woodster_ | elmiko, is the 'hard way' having try/excepts around blocks that might fail in mysql but pass in postgres? | 20:56 |
| redrobot | dave-mccowan we're starting to step on each other's toes. Your last patch erased a bunch of stuff I had done. | 20:56 |
| elmiko | woodster_: that or checking to see if the columns exist, but yea | 20:56 |
| elmiko | woodster_: it's doable | 20:56 |
| redrobot | dave-mccowan I'm -> <- this close to having a passing gate | 20:56 |
| redrobot | dave-mccowan as soon as I push the next patch you can rebase your work on top of it and send it as a dependent cr | 20:57 |
| elmiko | woodster_: then the next part will be figuring out what to do going forward | 20:57 |
| dave-mccowan | redrobot, in that case we're either duplicating effort or coding to a different design. :-) | 20:57 |
| woodster_ | redrobot, that looks like a PTL trying to squeeze something out | 20:57 |
| elmiko | lol | 20:57 |
| dave-mccowan | redrobot, OK. i'll wait for your patch, and then rebase. | 20:58 |
| woodster_ | elmiko: are you using the db migration tool for this work? You can easily move between versions that way | 20:58 |
| elmiko | woodster_: i am, but i have errors when creating the fresh juno db | 20:59 |
| elmiko | (with the migration tool, that is) | 21:00 |
| woodster_ | elmiko: hmm, I recall we were going to add a time-zero migration, that forklifts all the tables. Are you just loading the juno release though and then syncing that to a fresh database? | 21:00 |
| elmiko | woodster_: yea | 21:00 |
| elmiko | woodster_: although now that you suggest it, i could just roll back and forth between the good and bad migrations with the tool? | 21:01 |
| woodster_ | elmiko: and just doing that sync fails for mysql | 21:01 |
| elmiko | woodster_: here's my workflow | 21:01 |
| elmiko | 1. loadup stable/juno and let barbican auto create the db | 21:02 |
| woodster_ | elmiko, forgot the ? there. Yes, once the db is stood up at a given version, you can use the tool to upgrade and downgrade between versions (or blocks of versions) | 21:02 |
| elmiko | 2. switch to master and run the db-migrate to head | 21:02 |
| elmiko | woodster_: should i be able to run the db-migrate on a fresh db? | 21:02 |
| woodster_ | elmiko: yep 1 and 2 look good, but if you are testing a fix to an individual version, you could upgrade/downgrade -v <version before one I'm fixing> | 21:04 |
| woodster_ | elmiko: and then upgrade -v <version to test> once you've updated that script file | 21:04 |
| elmiko | woodster_: cool! i'll give that a try | 21:05 |
| woodster_ | elmiko: so upgrade if you are currently below the version to fix, or downgrade if above it | 21:05 |
| elmiko | woodster_: what if the upgrade fails, does it leave the db in an unrecoverable state? | 21:05 |
| woodster_ | elmiko: but to stand a fresh db up with alembic alone would require a version file that has all the things in it | 21:05 |
| elmiko | woodster_: yea, thought so | 21:06 |
| woodster_ | elmiko: if it fails the database is (should be) left in the current version | 21:06 |
| elmiko | woodster_: ack, thanks =) | 21:06 |
| *** joesavak has quit IRC | 21:10 | |
| woodster_ | elmiko: I figure once the dust settles on what you are doing, this page can be updated with guidance for future devs: http://docs.openstack.org/developer/barbican/contribute/database_migrations.html | 21:11 |
| elmiko | woodster_: lol, that's a mouthful ;) | 21:11 |
| woodster_ | elmiko: I also added a liberty note to check up on the grenade process...we really need to get migration gates working to keep us out of this mess | 21:11 |
| elmiko | woodster_: yea, it would also be nice to make the barbican-db-migrate work off of the installed version rather than adding the current path | 21:12 |
| elmiko | woodster_: in sahara we have migrations tested, maybe i can read up on some of that | 21:12 |
| *** stanzi has joined #openstack-barbican | 21:13 | |
| woodster_ | elmiko: grenade is the project used for migrations, but projects have been having to check project specific stuff into it and tempest to make the gate work. Both tempest and grenade are moving away from that approach to instead call back into projects to configure and run tests, but that isn't supposed to be ready until the Liberty time frame. | 21:13 |
| woodster_ | elmiko: do you mean having to specify the db URL as an argument vs getting it from teh installed config file? | 21:14 |
| elmiko | woodster_: ack on grenade. i can see value in specifying the dburl, i meant more that it looks like db-migrate wants to grab the migrations from the local directory rather than from the installed location. | 21:16 |
| elmiko | woodster_: but having it use the barbican-api.conf db connection would be a nice convenience feature | 21:17 |
| openstackgerrit | Douglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 21:20 |
| *** alee has quit IRC | 21:21 | |
| *** stanzi has quit IRC | 21:21 | |
| redrobot | dave-mccowan the more unit tests I fix, the more that fail >_> | 21:22 |
| *** stanzi has joined #openstack-barbican | 21:22 | |
| dave-mccowan | redrobot, i can relate. are there still more failing in patch #5? | 21:22 |
| redrobot | dave-mccowan yeah. I moved the keys from functional -> unit tests, so we can use just one source of keys | 21:23 |
| redrobot | dave-mccowan updated all the values for secret normalization | 21:23 |
| redrobot | dave-mccowan going to look at denormailize now | 21:23 |
| redrobot | dave-mccowan most of the failures I'm seeing now are because of the removed get_pem_components helper function | 21:24 |
| dave-mccowan | redrobot, i think that turns out to be helpful, since it points at the code that needs to be fixed. | 21:25 |
| dave-mccowan | redrobot, the code should now be PEM agnostic, so if it's trying to PEMify or de-PEMify something, it's probably wrong. | 21:26 |
| redrobot | dave-mccowan I'll tell you what, if you want to help out on this same CR again, pull down patch #5 and focus on the barbican/tests/tasks/test_certificate_resources.py ? | 21:26 |
| dave-mccowan | redrobot deal. i'll that that one and tests/common/test_validators.py and common/validators.py | 21:27 |
| *** dimtruck is now known as zz_dimtruck | 21:28 | |
| redrobot | dave-mccowan cool, let me know if you need to edit anything in test_translations | 21:28 |
| redrobot | if either of us pushes up a patch, the other one will have to stash/pull down new patch/unstash | 21:28 |
| dave-mccowan | redrobot, OK. i'll ignore translations.py let's try to work only in separate files to help prevent conflicts. | 21:29 |
| *** paul_glass has quit IRC | 21:35 | |
| *** stanzi has quit IRC | 21:36 | |
| *** stanzi has joined #openstack-barbican | 21:37 | |
| *** stanzi has quit IRC | 21:41 | |
| *** xaeth is now known as xaeth_afk | 21:51 | |
| openstackgerrit | Douglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 21:53 |
| redrobot | dave-mccowan done with translations. You'll want to stash / pull down patch # 6 / unstash | 21:53 |
| redrobot | dave-mccowan going to look at StoreCrypto next | 21:54 |
| *** nkinder has quit IRC | 21:57 | |
| *** zz_dimtruck is now known as dimtruck | 22:02 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 22:02 |
| redrobot | dave-mccowan heh... looks like you didn't pull down my changes for test_translations.py | 22:04 |
| dave-mccowan | redrobot. :-( | 22:05 |
| redrobot | dave-mccowan no worries. let me port them over to a new patch | 22:06 |
| redrobot | dave-mccowan maybe we should work on github instead | 22:07 |
| redrobot | dave-mccowan instead of ammending commits we can just start a branch | 22:07 |
| redrobot | and then squash it when we're done | 22:07 |
| dave-mccowan | redrobot, probably should have. but, we're almost done now. i'll remember to squash next time. looks like unit tests are about covered. i'll look at functional. | 22:09 |
| redrobot | dave-mccowan functional is passing in my box | 22:10 |
| dave-mccowan | redrobot. sweet. ship it! | 22:10 |
| redrobot | haha | 22:11 |
| redrobot | dave-mccowan 2 seconds on patch # 8 | 22:11 |
| openstackgerrit | Douglas Mendizábal proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 22:15 |
| redrobot | dave-mccowan ok, pull down # 8 and you can look at the functional | 22:15 |
| redrobot | dave-mccowan I'll finish up the unit tests | 22:15 |
| redrobot | dave-mccowan I think you may have uncommented more functional tests | 22:16 |
| dave-mccowan | redrobot, i think if we merge with my other pending CRs, we'll have all 18 passing now. | 22:17 |
| redrobot | dave-mccowan ok, let's get your CRs merged, and then rebase this big one on top of those. | 22:18 |
| *** alee has joined #openstack-barbican | 22:18 | |
| redrobot | dave-mccowan I like small CRs. All reviewed up. Maybe alee can take a look at them as well | 22:25 |
| *** stanzi has joined #openstack-barbican | 22:25 | |
| dave-mccowan | redrobot, i have some fixes to test_certificate_order.py coming soon to our joint CR | 22:26 |
| *** igueths has quit IRC | 22:26 | |
| redrobot | dave-mccowan just be sure to check for new patches before you --amend | 22:26 |
| dave-mccowan | redrobot, to fetch the latest (after stash): git fetch? git review -d? | 22:29 |
| redrobot | git review -d | 22:29 |
| redrobot | dave-mccowan git review -d 173396 | 22:29 |
| redrobot | so is it all plugins that pass base64(payload) or just secret store? | 22:31 |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 22:32 |
| *** tkelsey has joined #openstack-barbican | 22:34 | |
| alee | redrobot, dave-mccowan -- looking | 22:36 |
| *** tkelsey has quit IRC | 22:38 | |
| elmiko | woodster_: ok, i've got something working for both mysql and postgresql | 22:41 |
| elmiko | woodster_: needs a little more cleanup, but it's a start | 22:42 |
| redrobot | elmiko w00t! | 22:42 |
| elmiko | =D | 22:42 |
| elmiko | redrobot: when i get this set, i was thinking about making a bug and then posting the review against it. sound good? | 22:43 |
| redrobot | elmiko I think this is the bug for it? https://bugs.launchpad.net/barbican/+bug/1336901 | 22:44 |
| openstack | Launchpad bug 1336901 in Barbican "alembic migration[mysql] fails with kek_data migration" [Low,Confirmed] | 22:44 |
| *** stanzi has quit IRC | 22:45 | |
| *** stanzi has joined #openstack-barbican | 22:45 | |
| elmiko | redrobot: haven't hit that one yet, but i'll get to it | 22:48 |
| elmiko | redrobot: there are actually several migrations that need adjustment | 22:48 |
| *** stanzi has quit IRC | 22:50 | |
| *** dave-mccowan has quit IRC | 22:54 | |
| woodster_ | elmiko: redrobot yeah that is an old bug, from before Juno. | 22:59 |
| *** nkinder has joined #openstack-barbican | 23:01 | |
| *** dave-mccowan has joined #openstack-barbican | 23:10 | |
| *** chlong has joined #openstack-barbican | 23:10 | |
| *** kebray has quit IRC | 23:11 | |
| elmiko | woodster_: ack | 23:11 |
| *** dave-mccowan has quit IRC | 23:15 | |
| *** dave-mccowan has joined #openstack-barbican | 23:19 | |
| woodster_ | elmiko: so yeah, we are only working on migrations after at/after Juno then | 23:22 |
| elmiko | woodster_: ok, cool | 23:24 |
| *** dimtruck is now known as zz_dimtruck | 23:24 | |
| elmiko | i'm basically just trying to update a juno db to current master | 23:24 |
| *** zz_dimtruck is now known as dimtruck | 23:26 | |
| *** crc32 has quit IRC | 23:39 | |
| dave-mccowan | redrobot, those four patches still need workflow, so they can merge and then i can do the rebase. | 23:41 |
| *** chlong has quit IRC | 23:43 | |
| *** dimtruck is now known as zz_dimtruck | 23:45 | |
| *** chlong has joined #openstack-barbican | 23:45 | |
| openstackgerrit | Dave McCowan proposed openstack/barbican: Fix base64 decoding of payloads in one-step POST https://review.openstack.org/173396 | 23:49 |
| alee_afk | dave-mccowan, ping | 23:49 |
| dave-mccowan | alee_afk pong | 23:49 |
| alee_afk | dave-mccowan, so looking at your patches .. | 23:50 |
| *** zz_dimtruck is now known as dimtruck | 23:50 | |
| alee_afk | dave-mccowan, https://review.openstack.org/#/c/173163/1/functionaltests/api/v1/smoke/test_rsa.py,cm | 23:50 |
| alee_afk | the substatus "cert_request_pending" | 23:50 |
| alee_afk | is that the status that is set after the request is made to the cert plugin? | 23:50 |
| dave-mccowan | alee_afk, yes. these tests pass when the rest of the code is fixed, so it must be. :-) | 23:51 |
| alee_afk | ok -- I'll be looking at these carefully once these are in .. | 23:52 |
| alee_afk | I just wanted to make sure it is not the substatus set even before that point | 23:52 |
| alee_afk | ie that its actually getting that far. | 23:52 |
| dave-mccowan | alee_afk here is what i get back. http://ur1.ca/k6cpf | 23:53 |
| alee_afk | ok good | 23:54 |
| alee_afk | dave-mccowan, ok -workflowed 3 of them. | 23:56 |
| alee_afk | called to dinner .. | 23:56 |
| alee_afk | dave-mccowan, redrobot - I'll apply and review https://review.openstack.org/173396 later tonight | 23:57 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!