20:00:15 #startmeeting barbican 20:00:16 Meeting started Mon Apr 6 20:00:15 2015 UTC and is due to finish in 60 minutes. The chair is redrobot. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:00:17 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:00:19 The meeting name has been set to 'barbican' 20:00:48 #topic Roll Call 20:01:03 _o/ 20:01:04 o/ 20:01:09 yo/ 20:01:26 (´∇ノ`*)ノ 20:01:52 I guess people are still off for Easter? 20:02:13 as usual the agenda can be found here: 20:02:15 #link https://wiki.openstack.org/wiki/Meetings/Barbican 20:02:19 #topic Action Items 20:02:48 #link http://eavesdrop.openstack.org/meetings/barbican/2015/barbican.2015-03-30-19.59.html 20:03:08 So the only action item was for me 20:03:13 to release Castellan 0.1.0 20:03:18 which I did tag 20:03:19 #link http://git.openstack.org/cgit/openstack/castellan/tag/?id=0.1.0 20:03:30 but for some reason the openstack CI didn't push it out to PyPI 20:03:33 o/ 20:03:38 so I have to figure out what the deal is with that 20:04:03 #action redrobot to figure out why Castellan 0.1.0 tag did not get published to PyPI 20:04:17 awesome news about the release though =) 20:04:30 elmiko indeed! 20:04:43 hopefully I'll get that PyPI release working so people can start pip installing it. 20:05:06 yea, that's gonna be an early Liberty project for me. integrating castellan into sahara. 20:05:23 elmiko nice! 20:05:28 you know you'll have a fan here ;) 20:05:29 elmiko are you going to Vancouver? 20:05:31 yes 20:06:07 elmiko cool! we have a common use cases talk in which I'm going to cover some of Castellan 20:06:19 awesome, i'll keep an eye out for it 20:06:55 ok, moving on to today's agenda 20:07:48 #topic python-barbicanclient behavior with additional keys in the API responses 20:07:53 woodster_ did you want to talk about this? 20:07:59 sure 20:08:21 So if you take a look at this CR, you'll see where sub-status info was added to the order model 20:08:24 #link https://review.openstack.org/#/c/169481/2/barbicanclient/orders.py,cm 20:09:12 My thoughts are that if new optional items are added our APIs, that shouldn't be a contract breaking change to the API 20:09:35 So by extension then, this shouldn't break the deployed client for a given API major version 20:10:18 I think that in a way we are changing the API when we add new attributes to a response 20:10:27 I think usability is not so great if we have to update the version of the barbican client if the minor version of the barbican API introduces new optional response items 20:10:40 if we were doing proper semantic versioning, adding an attribute would prompt for a new minot version 20:10:49 so technically we now have v1.1, not v1 20:11:12 buuuut 20:11:18 I think that's really the heart of the question then 20:11:24 that's not what we're telling people 20:11:50 You do have a good point that it's not a good user experience to have the client break when using a new version of the API 20:11:58 We will just have to do synchronized releases...one for API changes, and one for client changes 20:12:00 especially since the client version in question was the latest available 20:12:25 yeah, it is just trickier to do deployments that way 20:12:37 The easy fix in the client is to allow *args, and **kwargs to deal with unexpected input. 20:12:44 redrobot, speaking of versions, at some point we should revisit your version resource to match other openstack projects 20:13:12 redrobot, a discussion we should probably have for Liberty 20:13:18 redrobot, yeah, but that does have the feel of 'cheating' a bit :) 20:13:48 woodster_ yup. not to mention it does nothing for people who are using the API via other clients (like fog for example) 20:14:35 so, I'm not sure there's a right answer, or an action item out of this. 20:14:50 maybe just be more mindful of the changes of the API and how the affect the client. 20:14:53 I think it would mainly give us time to stage our client vs api releases 20:15:22 redrobot, sorry - I'm not clear on the context of this -- is the client broken? 20:15:24 but we should still be trying to keep our client updated to use for-reals kwargs 20:15:53 alee yeah, the client was blowing up when instantiating new orders since the API response now includes an additional attribute 20:15:55 alee, this regards a broken gate caused during htis CR: https://review.openstack.org/#/c/169481/2/barbicanclient/orders.py,cm 20:16:31 alee the new attribute was being passed as an extra arg to __init__, so it blew up with an unknown arg exception 20:16:43 redrobot, woodster_ so these are new required params? 20:16:50 or optional? 20:17:00 alee, they are the new sub-status related ones, so optional 20:17:17 I guess we could also make the argument that the client should be passing the response to __init__ directly and that it should pull out the things it needs first, and that would also avoid breakages like the one we saw 20:17:32 s/should/shouldn't/ 20:17:48 woodster_, redrobot then yeah -- I'm inclined to agree that adding new optional parameters to the api should not break the client. 20:18:18 I think we're all in agreement... not sure about the right way to prevent it. 20:18:27 so the easy fix as redrobot pointed out is to add **kwargs to the end of the __init__() defs 20:18:33 and if so, then we need to fix the client to allow this. While we will try to keep the client/api in sync, they will never always be in sync 20:18:42 although talking through it now, it seems that parsing the response first would be better than adding *args and **kwargs 20:19:14 woodster_ how about parsing the response first... pull out only what you need instead of trying to pass the entire response to the constructors? 20:19:17 so do you mean parsing out the values from within the __init__ or external to it? 20:19:45 so for an order, are you saying some logic before calling order(...) would parse out just what is needed? 20:20:12 woodster_ yeah... seems to me like it's a better solution than allowing anyting and everything in the constructor 20:21:34 well, I'm thinking that an order model should know what fields are appropriate for it. If you have logic outside of Order do this, then you have coupling to other order-esque logic which could be tricky to manage 20:22:17 i.e. the Order model and the order-selector logic outside Order would have to be kept in sync 20:23:24 so in the case of adding sub-status, I'd have to both add sub-status to the Order init, and also change the parsing to code to allow sub-status to go through 20:23:48 hmmm.... 20:24:09 woodster_, redrobot - yeah -- I'm thinking that kwargs are probably the right way to go here. 20:24:12 I still think the underlying issue is the assumption that the response from the server can be automagically converted into arguments for the constructor 20:24:43 well, you are assuming that the response for an order call can be interpreted as an Order model 20:24:48 redrobot, right - but within the constructor, we should only be selecting the elements we want, right? 20:25:18 alee yea... I think we should still spell out the necessary args... 20:25:42 I'm thinking the Order model is still the best place to determine if the response data can interpreted as an order 20:26:11 I don't like that **kwargs will basically be a catch-all fail-safe .... but I do think I'm outnumbered here :) 20:26:31 redrobot, the problem then is that you need parser code outside of the initializer to examine the parameters being passed in 20:27:02 redrobot, not using kwargs here merely moves the problem 20:27:22 I'm pushing for both the **kwargs addition but also a policy of adding real kwargs for the next release of the client. This CR is a good example of that in action. 20:27:43 personally, I'm not a big fan of the catch-all kwargs idea 20:27:45 If I add a sub-status, I should also have unit tests that verify it is set on the model 20:28:15 It feels like we don't have a well defined interface between components 20:28:44 jvrbanac well we do, the problem is that stuff breaks when we break the interface (e.g. add a new attribute to a previously defined response) 20:29:22 another way to look at this is that we want to provide "forwards compatibility" for _some_ api changes 20:29:53 this really speaks to the question of if adding optional items to the response breaks the API contract (and associated objects) within a major version. 20:31:26 we could certainly draw a strong line here and say that such changes do break the contract, but I think some users will not like that 20:32:02 redrobot, its not as tight as it could be. For example, when I build the dogtag code (which is in java) - I build both client and server code. Both client and server use the same interface files to define the interface so they are never out of sync. 20:32:21 redrobot, its a little more difficult when you have duplicaed classes. 20:32:26 woodster_ that's true... but I don't like the idea of having to make new major api versions for every tiny change 20:32:30 alee that's a good point 20:32:32 I think that could force us into > v1 API changes sooner rather than later as well 20:32:39 redrobot, I agree 20:32:46 woodster_, I think we should be able to add optional params within a major version 20:33:18 alee I think the question is, do we need major/minor versions? ... and I still wouldn't like that either. 20:33:42 hmm... tricky question for sure. Maybe we should get some input from the API Working Group? 20:33:47 I think the reality is that the team managing the api service updates within an organization will be different than the ones building applications that use the client code 20:33:52 redrobot, what do you mean -- dp we need major/minor versions? 20:34:34 +1 on getting input. Also, I have seen a lot of people avoid the kwargy stuff by passing in a dict and explicitly pulling what you need out. 20:34:45 For the API, the main version driver is that pesky '/v1/' in there :) 20:34:59 redrobot, minor version == api changes that do not break compatibility , major version == changes that break compatibility 20:35:10 alee so, for a regular library, adding new methods to a class or whatever would prompt a new minor version but keep the major version the same. On the API we only have a major version, so minor changes can't be differentiated from one major release to the next 20:35:15 jvrbanac, yeah that would work too certainly 20:35:47 redrobot, ah - you are saying we have no minor version .. 20:36:05 jvrbanac, how is that different from kwargs? 20:36:11 alee yep... but I also don't want to support endpoints with /v1 and /v1.1 and /v1.2 20:36:11 yep, just the version in the URI 20:37:07 redrobot, woodster_ , jvrbanac this must have been solved in othr projects .. lets get input. 20:37:11 ...and each one of those endpoint versions need to be bug fixed/patched for a release or two as well I believe 20:37:37 have anyone investigate the ideas about putting minor versioning information into the headers? 20:37:45 alee, the idea is that you method contract is consistent and data is passed consistently. however, you have still have an explicit mapping of what you're pulling out that use-case. 20:37:47 this came up at least once before at the api-wg meetings 20:38:00 again, we should get some input from the api-wg 20:38:02 #action redrobot to email dev list to get input on minor changes in the API 20:38:12 jvrbanac +1 20:38:18 ok, moving on 20:38:21 +1 20:38:32 #topic Juno to Kilo DB migration 20:38:44 so.... with the Kilo final release just around the corner 20:39:07 it would be good to get confirmation that the migration scripts are able to migrate Barbican Juno schema to Barbican Kilo 20:39:35 has anyone exercised the migrations starting with a Juno deployment? 20:39:38 and if not 20:39:54 is anyone available/interested in testing the migrations? 20:40:24 i could run them on my local install 20:40:40 elmiko woot! 20:40:49 #action elmiko to test Juno to Kilo migration 20:40:55 it's probably not terribly interesting though, i've mainly been experimenting with the sahara integration stuff 20:41:09 ack, will do =) 20:41:48 elmiko thanks! Let me know what you find. I don't want to release RC1 unless we know that it will work for people upgrading from the Juno release 20:42:03 which may be a grand total of 0 people upgrading heh... 20:42:06 other projects have been adding an alembic version module that downgrades to the last version for that release 20:42:09 redrobot: ok, i'll try to run em tomorrow 20:42:36 easy to do but something we just need to put a CR up for when we are ready 20:43:06 woodster_ interesting, I thought alembic would just apply the rollback part of each migration in reverse order? 20:43:35 ok, moving on 20:43:39 redrobot, it does. This is just a way to 'mark' a version file as being the official one for a release 20:43:53 woodster_ oh I see 20:43:56 that way you can keep your version file around 20:45:23 #topic Content-Type and payload_content_encoding combinations 20:45:45 (...run away!!!...) 20:45:58 So, after seeing alee and Dave McCowan going back and forth about the content type stuff 20:46:09 I decided to take another look at this stuff 20:46:20 we do have a fine tradition of spending a few hours discussing content-types at each summit 20:46:33 and there's some funky things going on in the implementation that don't seem right to me 20:47:06 I think that the Content-Types work changed the meaning of what "base64" means in the payload_content_type for a one-step secret create 20:47:20 and I don't much care for the new meaning of it 20:47:50 originally, we added payload_content_type = "base64" as a ways to be able to include binary (non-ascii) data inside a json request 20:47:57 the meaning was 20:48:13 o/ 20:48:33 "The payload provided in this request has been base64 encoded so that it can be included in this JSON request" 20:48:48 so Barbican would base64 decode _the entire payload_ 20:49:04 yeah, just a JSON envelope to the payload really 20:49:30 but that's no longer true 20:50:18 I think that the fact that DER includes some base64 parts should not be confused with the purpose of the payload_content_encoding 20:50:48 in other words, the DER can be included in the JSON as is 20:50:52 so there is no need to base64 encode it 20:51:05 the fact that the DER implementation uses base64 should be inconsequential to Barbican 20:51:43 esp. for the opaque type! 20:52:16 so now, when sending a DER, you have to set payload_content_encoding to base64... but instead of decoding _the entire payload_ we're manipulating the payload before decoding it. 20:52:42 I think that for base64 encoding to work correctly with DER, the user should base64 the entire DER 20:52:57 agreed 20:53:05 or just not support payload_content_type = "base64" for asymmetric keys 20:53:10 kfarr, can you speak for rellerreller on this topic? 20:53:16 I'm here now 20:53:54 What was the question? 20:53:56 woodster_ he just joined! 20:53:56 rellerreller, did you get an alert on 'base64' in this channel? 20:54:17 redrobot, we're not going to finish this conversation in 5 minutes. should we do a google hangout right after this? 20:54:38 woodster_ No, I happened to get a few minutes, so I thought I would check in. 20:54:52 rellerreller, ha, just kidding, but good timing 20:54:56 Can we do Google hangout at some later point? 20:54:57 alee I agree... 20:55:05 wfm 20:55:13 rellerreller The question is on how payload_content_type = "base64 20:55:19 My daughter is sleeping and usually when I want to do something she starts waking up. 20:55:20 works with DER payloads 20:56:13 If a payload is DER encoded and base64 encoded then works fine. 20:56:27 she will not be ignored! Well, we coudl continue IRC discussion in the barbican channel too 20:56:28 rellerreller, the gist is that redrobot does not agree with how we handle content-type = base64. On a separate note, I've been working through bugs on the current implementation for cert stuff. 20:56:31 It's the same as with PEM encoding, I believe :) 20:57:02 rellerreller hmmm.... Interesting... I'll have to retest some of the stuff I was doing. 20:57:09 alee Can you send me links to the bugs? 20:57:31 rellerreller, no bugs yet -- I've been working on patches 20:57:41 rellerreller, we can discuss in the barbican channel 20:57:44 alee Have you posted any? 20:57:44 running out of time here... we can continue on the barbican channel 20:57:50 thanks for coming everyone! 20:57:58 #endmeeting