20:00:15 <redrobot> #startmeeting barbican
20:00:16 <openstack> Meeting started Mon Apr  6 20:00:15 2015 UTC and is due to finish in 60 minutes.  The chair is redrobot. Information about MeetBot at http://wiki.debian.org/MeetBot.
20:00:17 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
20:00:19 <openstack> The meeting name has been set to 'barbican'
20:00:48 <redrobot> #topic Roll Call
20:01:03 <jvrbanac> _o/
20:01:04 <alee> o/
20:01:09 <elmiko> yo/
20:01:26 <chellygel> (´∇ﾉ｀*)ノ
20:01:52 <redrobot> I guess people are still off for Easter?
20:02:13 <redrobot> as usual the agenda can be found here:
20:02:15 <redrobot> #link https://wiki.openstack.org/wiki/Meetings/Barbican
20:02:19 <redrobot> #topic Action Items
20:02:48 <redrobot> #link http://eavesdrop.openstack.org/meetings/barbican/2015/barbican.2015-03-30-19.59.html
20:03:08 <redrobot> So the only action item was for me
20:03:13 <redrobot> to release Castellan 0.1.0
20:03:18 <redrobot> which I did tag
20:03:19 <redrobot> #link http://git.openstack.org/cgit/openstack/castellan/tag/?id=0.1.0
20:03:30 <redrobot> but for some reason the openstack CI didn't push it out to PyPI
20:03:33 <woodster_> o/
20:03:38 <redrobot> so I have to figure out what the deal is with that
20:04:03 <redrobot> #action redrobot to figure out why Castellan 0.1.0 tag did not get published to PyPI
20:04:17 <elmiko> awesome news about the release though =)
20:04:30 <redrobot> elmiko indeed!
20:04:43 <redrobot> hopefully I'll get that PyPI release working so people can start pip installing it.
20:05:06 <elmiko> yea, that's gonna be an early Liberty project for me. integrating castellan into sahara.
20:05:23 <redrobot> elmiko nice!
20:05:28 <elmiko> you know you'll have a fan here ;)
20:05:29 <redrobot> elmiko are you going to Vancouver?
20:05:31 <elmiko> yes
20:06:07 <redrobot> elmiko cool!  we have a common use cases talk in which I'm going to cover some of Castellan
20:06:19 <elmiko> awesome, i'll keep an eye out for it
20:06:55 <redrobot> ok, moving on to today's agenda
20:07:48 <redrobot> #topic python-barbicanclient behavior with additional keys in the API responses
20:07:53 <redrobot> woodster_ did you want to talk about this?
20:07:59 <woodster_> sure
20:08:21 <woodster_> So if you take a look at this CR, you'll see where sub-status info was added to the order model
20:08:24 <woodster_> #link https://review.openstack.org/#/c/169481/2/barbicanclient/orders.py,cm
20:09:12 <woodster_> My thoughts are that if new optional items are added our APIs, that shouldn't be a contract breaking change to the API
20:09:35 <woodster_> So by extension then, this shouldn't break the deployed client for a given API major version
20:10:18 <redrobot> I think that in a way we are changing the API when we add new attributes to a response
20:10:27 <woodster_> I think usability is not so great if we have to update the version of the barbican client if the minor version of the barbican API introduces new optional response items
20:10:40 <redrobot> if we were doing proper semantic versioning, adding an attribute would prompt for a new minot version
20:10:49 <redrobot> so technically we now have v1.1, not v1
20:11:12 <redrobot> buuuut
20:11:18 <woodster_> I think that's really the heart of the question then
20:11:24 <redrobot> that's not what we're telling people
20:11:50 <redrobot> You do have a good point that it's not a good user experience to have the client break when using a new version of the API
20:11:58 <woodster_> We will just have to do synchronized releases...one for API changes, and one for client changes
20:12:00 <redrobot> especially since the client version in question was the latest available
20:12:25 <woodster_> yeah, it is just trickier to do deployments that way
20:12:37 <redrobot> The easy fix in the client is to allow *args, and **kwargs to deal with unexpected input.
20:12:44 <jvrbanac> redrobot, speaking of versions, at some point we should revisit your version resource to match other openstack projects
20:13:12 <jvrbanac> redrobot, a discussion we should probably have for Liberty
20:13:18 <woodster_> redrobot, yeah, but that does have the feel of 'cheating' a bit :)
20:13:48 <redrobot> woodster_ yup.  not to mention it does nothing for people who are using the API via other clients (like fog for example)
20:14:35 <redrobot> so, I'm not sure there's a right answer, or an action item out of this.
20:14:50 <redrobot> maybe just be more mindful of the changes of the API and how the affect the client.
20:14:53 <woodster_> I think it would mainly give us time to stage our client vs api releases
20:15:22 <alee> redrobot, sorry - I'm not clear on the context of this -- is the client broken?
20:15:24 <woodster_> but we should still be trying to keep our client updated to use for-reals kwargs
20:15:53 <redrobot> alee yeah, the client was blowing up when instantiating new orders since the API response now includes an additional attribute
20:15:55 <woodster_> alee, this regards a broken gate caused during htis CR: https://review.openstack.org/#/c/169481/2/barbicanclient/orders.py,cm
20:16:31 <redrobot> alee the new attribute was being passed as an extra arg to __init__, so it blew up with an unknown arg exception
20:16:43 <alee> redrobot, woodster_  so these are new required params?
20:16:50 <alee> or optional?
20:17:00 <woodster_> alee, they are the new sub-status related ones, so optional
20:17:17 <redrobot> I guess we could also make the argument that the client should be passing the response to __init__ directly and that it should pull out the things it needs first, and that would also avoid breakages like the one we saw
20:17:32 <redrobot> s/should/shouldn't/
20:17:48 <alee> woodster_, redrobot then yeah -- I'm inclined to agree that adding new optional parameters to the api should not break the client.
20:18:18 <redrobot> I think we're all in agreement...  not sure about the right way to prevent it.
20:18:27 <woodster_> so the easy fix as redrobot pointed out is to add **kwargs to the end of the __init__() defs
20:18:33 <alee> and if so, then we need to fix the client to allow this.  While we will try to keep the client/api in sync, they will never always be in sync
20:18:42 <redrobot> although talking through it now, it seems that parsing the response first would be better than adding *args and **kwargs
20:19:14 <redrobot> woodster_ how about parsing the response first... pull out only what you need instead of trying to pass the entire response to the constructors?
20:19:17 <woodster_> so do you mean parsing out the values from within the __init__ or external to it?
20:19:45 <woodster_> so for an order, are you saying some logic before calling order(...) would parse out just what is needed?
20:20:12 <redrobot> woodster_ yeah... seems to me like it's a better solution than allowing anyting and everything in the constructor
20:21:34 <woodster_> well, I'm thinking that an order model should know what fields are appropriate for it. If you have logic outside of Order do this, then you have coupling to other order-esque logic which could be tricky to manage
20:22:17 <woodster_> i.e. the Order model and the order-selector logic outside Order would have to be kept in sync
20:23:24 <woodster_> so in the case of adding sub-status, I'd have to both add sub-status to the Order init, and also change the parsing to code to allow sub-status to go through
20:23:48 <redrobot> hmmm....
20:24:09 <alee> woodster_, redrobot - yeah -- I'm thinking that kwargs are probably the right way to go here.
20:24:12 <redrobot> I still think the underlying issue is the assumption that the response from the server can be automagically converted into arguments for the constructor
20:24:43 <woodster_> well, you are assuming that the response for an order call can be interpreted as an Order model
20:24:48 <alee> redrobot, right - but within the constructor, we should only be selecting the elements we want, right?
20:25:18 <redrobot> alee yea...  I think we should still spell out the necessary args...
20:25:42 <woodster_> I'm thinking the Order model is still the best place to determine if the response data can interpreted as an order
20:26:11 <redrobot> I don't like that **kwargs will basically be a catch-all fail-safe .... but I do think I'm outnumbered here :)
20:26:31 <alee> redrobot, the problem then is that you need parser code outside of the initializer to examine the parameters being passed in
20:27:02 <alee> redrobot, not using kwargs here merely moves the problem
20:27:22 <woodster_> I'm pushing for both the **kwargs addition but also a policy of adding real kwargs for the next release of the client.  This CR is a good example of that in action.
20:27:43 <jvrbanac> personally, I'm not a big fan of the catch-all kwargs idea
20:27:45 <woodster_> If I add a sub-status, I should also have unit tests that verify it is set on the model
20:28:15 <jvrbanac> It feels like we don't have a well defined interface between components
20:28:44 <redrobot> jvrbanac well we do, the problem is that stuff breaks when we break the interface (e.g. add a new attribute to a previously defined response)
20:29:22 <redrobot> another way to look at this is that we want to provide "forwards compatibility" for _some_ api changes
20:29:53 <woodster_> this really speaks to the question of if adding optional items to the response breaks the API contract (and associated objects) within a major version.
20:31:26 <woodster_> we could certainly draw a strong line here and say that such changes do break the contract, but I think some users will not like that
20:32:02 <alee> redrobot, its not as tight as it could be.  For example, when I build the dogtag code (which is in java) - I build both client and server code.  Both client and server use the same interface files to define the interface so they are never out of sync.
20:32:21 <alee> redrobot, its a little more difficult when you have duplicaed classes.
20:32:26 <redrobot> woodster_ that's true... but I don't like the idea of having to make new major api versions for every tiny change
20:32:30 <redrobot> alee that's a good point
20:32:32 <woodster_> I think that could force us into > v1 API changes sooner rather than later as well
20:32:39 <woodster_> redrobot, I agree
20:32:46 <alee> woodster_, I think we should be able to add optional params within a major version
20:33:18 <redrobot> alee I think the question is, do we need major/minor versions?  ...  and I still wouldn't like that either.
20:33:42 <redrobot> hmm... tricky question for sure.  Maybe we should get some input from the API Working Group?
20:33:47 <woodster_> I think the reality is that the team managing the api service updates within an organization will be different than the ones building applications that use the client code
20:33:52 <alee> redrobot, what do you mean -- dp we need major/minor versions?
20:34:34 <jvrbanac> +1 on getting input. Also, I have seen a lot of people avoid the kwargy stuff by passing in a dict and explicitly pulling what you need out.
20:34:45 <woodster_> For the API, the main version driver is that pesky '/v1/' in there :)
20:34:59 <alee> redrobot, minor version == api changes that do not break compatibility  , major version == changes that break compatibility
20:35:10 <redrobot> alee so, for a regular library, adding new methods to a class or whatever would prompt a new minor version but keep the major version the same.  On the API we only have a major version, so minor changes can't be differentiated from one major release to the next
20:35:15 <woodster_> jvrbanac, yeah that would work too certainly
20:35:47 <alee> redrobot, ah - you are saying we have no minor version ..
20:36:05 <alee> jvrbanac, how is that different from kwargs?
20:36:11 <redrobot> alee yep... but I also don't want to support endpoints with /v1 and /v1.1 and /v1.2
20:36:11 <woodster_> yep, just the version in the URI
20:37:07 <alee> redrobot, woodster_ , jvrbanac this must have been solved in othr projects .. lets get input.
20:37:11 <woodster_> ...and each one of those endpoint versions need to be bug fixed/patched for a release or two as well I believe
20:37:37 <elmiko> have anyone investigate the ideas about putting minor versioning information into the headers?
20:37:45 <jvrbanac> alee, the idea is that you method contract is consistent and data is passed consistently. however, you have still have an explicit mapping of what you're pulling out that use-case.
20:37:47 <elmiko> this came up at least once before at the api-wg meetings
20:38:00 <jvrbanac> again, we should get some input from the api-wg
20:38:02 <redrobot> #action redrobot to email dev list to get input on minor changes in the API
20:38:12 <redrobot> jvrbanac +1
20:38:18 <redrobot> ok, moving on
20:38:21 <woodster_> +1
20:38:32 <redrobot> #topic Juno to Kilo DB migration
20:38:44 <redrobot> so....  with the Kilo final release just around the corner
20:39:07 <redrobot> it would be good to get confirmation that the migration scripts are able to migrate Barbican Juno schema to Barbican Kilo
20:39:35 <redrobot> has anyone exercised the migrations starting with a Juno deployment?
20:39:38 <redrobot> and if not
20:39:54 <redrobot> is anyone available/interested in testing the migrations?
20:40:24 <elmiko> i could run them on my local install
20:40:40 <redrobot> elmiko woot!
20:40:49 <redrobot> #action elmiko to test Juno to Kilo migration
20:40:55 <elmiko> it's probably not terribly interesting though, i've mainly been experimenting with the sahara integration stuff
20:41:09 <elmiko> ack, will do =)
20:41:48 <redrobot> elmiko thanks!  Let me know what you find.  I don't want to release RC1 unless we know that it will work for people upgrading from the Juno release
20:42:03 <redrobot> which may be a grand total of 0 people upgrading heh...
20:42:06 <woodster_> other projects have been adding an alembic version module that downgrades to the last version for that release
20:42:09 <elmiko> redrobot: ok, i'll try to run em tomorrow
20:42:36 <woodster_> easy to do but something we just need to put a CR up for when we are ready
20:43:06 <redrobot> woodster_ interesting, I thought alembic would just apply the rollback part of each migration in reverse order?
20:43:35 <redrobot> ok, moving on
20:43:39 <woodster_> redrobot, it does. This is just a way to 'mark' a version file as being the official one for a release
20:43:53 <redrobot> woodster_ oh I see
20:43:56 <woodster_> that way you can keep your version file around
20:45:23 <redrobot> #topic Content-Type and payload_content_encoding combinations
20:45:45 <woodster_> (...run away!!!...)
20:45:58 <redrobot> So, after seeing alee and Dave McCowan going back and forth about the content type stuff
20:46:09 <redrobot> I decided to take another look at this stuff
20:46:20 <woodster_> we do have a fine tradition of spending a few hours discussing content-types at each summit
20:46:33 <redrobot> and there's some funky things going on in the implementation that don't seem right to me
20:47:06 <redrobot> I think that the Content-Types work changed the meaning of what "base64" means in the payload_content_type for a one-step secret create
20:47:20 <redrobot> and I don't much care for the new meaning of it
20:47:50 <redrobot> originally, we added payload_content_type = "base64" as a ways to be able to include binary (non-ascii) data inside a json request
20:47:57 <redrobot> the meaning was
20:48:13 <dave-mccowan> o/
20:48:33 <redrobot> "The payload provided in this request has been base64 encoded so that it can be included in this JSON request"
20:48:48 <redrobot> so Barbican would base64 decode _the entire payload_
20:49:04 <woodster_> yeah, just a JSON envelope to the payload really
20:49:30 <redrobot> but that's no longer true
20:50:18 <redrobot> I think that the fact that DER includes some base64 parts should not be confused with the purpose of the payload_content_encoding
20:50:48 <redrobot> in other words, the DER can be included in the JSON as is
20:50:52 <redrobot> so there is no need to base64 encode it
20:51:05 <redrobot> the fact that the DER implementation uses base64 should be inconsequential to Barbican
20:51:43 <woodster_> esp. for the opaque type!
20:52:16 <redrobot> so now, when sending a DER, you have to set payload_content_encoding to base64... but instead of decoding _the entire payload_ we're manipulating the payload before decoding it.
20:52:42 <redrobot> I think that for base64 encoding to work correctly with DER, the user should base64 the entire DER
20:52:57 <woodster_> agreed
20:53:05 <redrobot> or just not support payload_content_type = "base64" for asymmetric keys
20:53:10 <woodster_> kfarr, can you speak for rellerreller on this topic?
20:53:16 <rellerreller> I'm here now
20:53:54 <rellerreller> What was the question?
20:53:56 <kfarr> woodster_ he just joined!
20:53:56 <woodster_> rellerreller, did you get an alert on 'base64' in this channel?
20:54:17 <alee> redrobot, we're not going to finish this conversation in 5 minutes.  should we do a google hangout right after this?
20:54:38 <rellerreller> woodster_ No, I happened to get a few minutes, so I thought I would check in.
20:54:52 <woodster_> rellerreller, ha, just kidding, but good timing
20:54:56 <rellerreller> Can we do Google hangout at some later point?
20:54:57 <redrobot> alee I agree...
20:55:05 <woodster_> wfm
20:55:13 <redrobot> rellerreller The question is on how payload_content_type = "base64
20:55:19 <rellerreller> My daughter is sleeping and usually when I want to do something she starts waking up.
20:55:20 <redrobot> works with DER payloads
20:56:13 <rellerreller> If a payload is DER encoded and base64 encoded then works fine.
20:56:27 <woodster_> she will not be ignored! Well, we coudl continue IRC discussion in the barbican channel too
20:56:28 <alee> rellerreller,  the gist is that redrobot does not agree with how we handle content-type = base64.  On a separate note, I've been working through bugs on the current implementation for cert stuff.
20:56:31 <rellerreller> It's the same as with PEM encoding, I believe :)
20:57:02 <redrobot> rellerreller hmmm.... Interesting... I'll have to retest some of the stuff I was doing.
20:57:09 <rellerreller> alee Can you send me links to the bugs?
20:57:31 <alee> rellerreller, no bugs yet -- I've been working on patches
20:57:41 <alee> rellerreller, we can discuss in the barbican channel
20:57:44 <rellerreller> alee Have you posted any?
20:57:44 <redrobot> running out of time here...  we can continue on the barbican channel
20:57:50 <redrobot> thanks for coming everyone!
20:57:58 <redrobot> #endmeeting