20:01:04 <redrobot> #startmeeting barbican 20:01:04 <openstack> Meeting started Mon Aug 25 20:01:04 2014 UTC and is due to finish in 60 minutes. The chair is redrobot. Information about MeetBot at http://wiki.debian.org/MeetBot. 20:01:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 20:01:07 <openstack> The meeting name has been set to 'barbican' 20:01:33 <redrobot> It's that time already 20:01:34 <redrobot> :) 20:01:47 <redrobot> As usual the agenda can be found here: 20:01:50 <redrobot> #link https://wiki.openstack.org/wiki/Meetings/Barbican 20:02:36 <redrobot> looks like we didn't have any action items since last week. 20:02:43 <redrobot> *from last week 20:02:53 <redrobot> so let's move right along to this week's topics 20:03:01 <redrobot> #topic Roll Call 20:03:18 <SheenaG11> o/ 20:03:20 <rellerreller> o/ 20:03:21 <kaitlin-farr> o/ 20:03:23 <chellygel> \o/ 20:03:43 <jvrbanac> _O/ 20:04:06 <redrobot> ^^ jvrbanac has a big head 20:04:22 <jvrbanac> lol 20:04:35 <woodster_> \o 20:05:01 <redrobot> ok, lots of barbicaneers 20:05:03 <redrobot> I like it 20:05:07 <redrobot> #topic CR Sizes 20:05:30 <redrobot> I think we've talked about this before, but it's always nice to have a reminder 20:05:37 <redrobot> jvrbanac did you want to bring up anything specific? 20:06:19 <jvrbanac> Not specific no. I just think we need to try to be mindful of CR sizes. It makes quick turn arounds on CRs quite difficult 20:07:40 <jvrbanac> It would be preferred if we could work better to split of our work, so it reduces the time burden on reviewers. 20:08:12 <redrobot> #agreed please try to keep CRs small. Big CRs take time to review, and slow down things for everyone. 20:08:53 <redrobot> ok, then, moving on 20:09:02 <reaperhulk> o/ (Oops) 20:09:22 <redrobot> #topic String interpoation in logging calls 20:10:08 <redrobot> Ok, so the question was raised by rellerreller about the guidelines for when to use which syntax for string interpolation 20:10:33 <rellerreller> Yes, what is the deal with log.debug printing? 20:10:40 <rellerreller> Should we use the old style %s or the new style with {}? There seems to be conflicting information about this. 20:11:35 <redrobot> My preference is that we use new-style "{curly}".format(curly="syntax") 20:11:51 <redrobot> but 20:11:57 <reaperhulk> that's string interpolation, not log.debug 20:11:58 <woodster_> so log.debug('something: %s', something) vs log.debug('something {0}.format(somthing))? 20:12:14 <reaperhulk> log.debug should use % because the interpolation is not performed if you aren't logging at that level 20:12:24 <redrobot> the concern is that there is a performance penalty when inlining .format in a logging call that is below the configured logging level 20:12:50 <redrobot> ie. log.debug("{0} 20:12:56 <woodster_> the former form doesn't process/format that 'something' input if debug level logging is turned off 20:13:06 <redrobot> woodster_ correct 20:13:48 <redrobot> to add to the confusion, python loggers don't understand new-style formatting 20:13:54 <woodster_> So that's the motivation for the comma-delimited form of logging 20:14:08 <woodster_> yes, that is unfortunate 20:14:10 <redrobot> so log.debug("{something}", "something") blows up 20:14:10 <reaperhulk> I don't think it's unreasonable to use old-style in logger and new-style everywhere else. 20:14:29 <woodster_> Esp. if there is a performance benefit 20:14:52 <woodster_> string manipulation under load is actually surprisingly processor intensive 20:15:02 <reaperhulk> lots of memory copies :) 20:15:04 <woodster_> ...esp. if htere are lots of debug statements 20:15:25 <redrobot> rellerreller does it sound reasonable to you to use new-style formatting everywhere, except for logging? 20:15:46 <rellerreller> So should we say that all new code must use % style for log statement? 20:16:01 <rellerreller> Yes, I'm ok with either style. 20:16:10 <reaperhulk> +1 from me on that. 20:16:19 <rellerreller> I just want to make sure it is consistent whichever format we choose. 20:16:30 <redrobot> rellerreller yes, but that's not enough.... since we also have to ensure that we're not inlining the % operation, but rather passing the values as arguments to the log statement 20:16:56 <redrobot> sounds like we're in agreement then 20:16:58 <rellerreller> redrobot agreed 20:17:04 <reaperhulk> This could potentially be automated with a flake8 extension, but that's an idea for the future... 20:17:13 <reaperhulk> Unless somebody really likes to play around in AST-land 20:17:21 <redrobot> #agreed Prefer new-style formatting everywhere, except for logging. 20:17:45 <redrobot> #agreed Logging should be done using old-style formatting, passing the values as arguments to the log call 20:18:07 <redrobot> ok, moving on 20:18:22 <redrobot> #topic Python 3 support 20:18:31 <redrobot> rellerreller do you want to talk about this? 20:18:43 <rellerreller> Yes, please. 20:18:46 <rellerreller> We tried pushing a global requirements change to add PyKMIP. We met some resistance because PyKMIP did not yet have Python 3 support. 20:18:57 <rellerreller> Does anyone know the OpenStack plan for Python 3 migration? Are there things we should be doing now within Barbican to help with that migration? 20:19:11 <reaperhulk> OpenStack has a general plan for migrating to Python 3 on a nebulous time frame 20:19:21 <rellerreller> i.e. should we be looking for things in code reviews that support Python 3 or are only supported in Python 2? 20:19:25 <reaperhulk> The primary blockers are things like eventlet (which has partial py3 support in github master but not in a released branch) 20:19:54 <reaperhulk> We should look at adding a py3 flake8 job to barbican. This would prevent the introduction of non-py3 compliant syntax 20:20:04 <reaperhulk> but we can't do a real py3 gate until eventlet is ported 20:20:15 <rellerreller> reaperhulk I was thinking the same thing 20:20:44 <redrobot> we could also add a non-voting python3 gate to barbican 20:20:44 <reaperhulk> I actually submitted a CR several months ago (and was merged) that fixed all syntax issues on py3 20:20:56 <reaperhulk> but new ones may have appeared since we don't gate 20:21:15 <reaperhulk> I'm neutral on a non-voting py3 gate since it's guaranteed to fail for the foreseeable future 20:21:47 <rellerreller> Does eventlet have a time frame for when they will support Py3? 20:21:58 <reaperhulk> rellerreller: I'm happy to take a look at pyKMIP for py3 compat if you've got it up somewhere though. I write a lot of py2/py3 code for pyca/cryptography :) 20:22:37 <reaperhulk> rellerreller: no timeframe, but there's a set of issues listed on the eventlet github page https://github.com/eventlet/eventlet/issues 20:22:37 <rellerreller> reaperhulk thanks, but I think we are good. kaitlin-farr is working on this, and I think she should be done in the next day or two. 20:23:08 <woodster_> I think only the oslo-incubator code would hard fail py3. If we can exclude that folder path, we could probably have a python 3 gate 20:23:16 <reaperhulk> oh hey, 0.15 actually does add py3 support but it is listed as "not ready yet" 20:23:19 <reaperhulk> so eventlet is getting pretty close 20:23:47 <reaperhulk> we could add a py3 flake8 gate tomorrow (and probably should). 20:23:58 <woodster_> I thought Guido wanted to kill eventlet in py3? :) 20:25:43 <woodster_> the only place outside of incubator eventlet is imported is the bin/barbican-worker.py script 20:26:12 <reaperhulk> doesn't webob use eventlet. 20:26:28 <reaperhulk> (maybe not, that should be a question mark) 20:27:55 <jvrbanac> reaperhulk, woodster_, oslo.messaging depends on eventlet as well 20:28:04 <reaperhulk> that's one we kinda use :D 20:28:52 <woodster_> I was hoping the gate was just analyzing the barbican source for py3 violation, rather than looking at all dependencies as well 20:29:30 <reaperhulk> tricky to do with tox (which likes to install the package + dependencies) 20:30:17 <reaperhulk> we talked about doing this in a swift session at the atlanta summit actually 20:30:18 <reaperhulk> https://github.com/openstack/swift/blob/master/tox.ini 20:30:19 <redrobot> reaperhulk how does py3 linting in flake8 work? is it a suite of additional rules like hacking? 20:30:39 <reaperhulk> I believe it just invokes the interpreter 20:30:47 <reaperhulk> but check out the py3pep8 job in the swift tox 20:30:58 <reaperhulk> that's pretty much what we'll need to do 20:32:20 <redrobot> I see 20:32:55 <redrobot> reaperhulk would you be able to add that? 20:33:53 <redrobot> also, is this a voting gate for swift? 20:34:43 <reaperhulk> yeah I can add that tox job for sure 20:34:54 <reaperhulk> let me look at whether it's voting 20:35:01 <redrobot> #action reaperhulk to add a py3pep8 tox job 20:35:52 <reaperhulk> looks like it is not yet a gate for them (they have a lot of syntax to clean up still apparently) 20:38:15 <redrobot> ok, so just a tox job for now, it is 20:38:21 <redrobot> any more questions/concerns about py3 ? 20:39:15 <rellerreller> I am good. This helped me a lot. 20:40:09 <redrobot> ok, moving on 20:40:53 <redrobot> actually, that's all I had in the agenda 20:41:10 <woodster_> Just an FYI that I've added more meat to this validator etherpad from last week: https://etherpad.openstack.org/p/barbican-validation-options 20:41:49 <redrobot> #topic Plugin Validation 20:41:58 <woodster_> #link https://etherpad.openstack.org/p/barbican-validation-options 20:41:58 <redrobot> #link https://etherpad.openstack.org/p/barbican-validation-options 20:42:03 <woodster_> ha, sorry! 20:42:05 <redrobot> JINX! 20:42:24 <woodster_> I still had it in my paste queue 20:43:01 <alee> woodster_, will take a look at it tommorow morning 20:43:11 <redrobot> is this something we're trying to land in Juno? 20:43:16 <woodster_> So we've talked about replacing the hard-coded validation (esp. around algorithm, bit-length) with calls to the plugins themselves. This is the subject of the etherpad. 20:43:40 <alee> but it would be nice to land in juno 20:43:46 <woodster_> redrobot: we probably need to land in Juno, so we can leverage the Dogtag and KMIP generation features? 20:44:08 <redrobot> ok... just concerned that j3 is around the corner 20:44:43 <woodster_> well, I think we are talking about maybe adding a new method to the secret_store.py interface...no changes to the API. 20:45:14 <redrobot> woodster_ yeah... just reminded myself we spoke about this earlier 20:46:20 <alee> woodster_, I'm for option 2 20:46:47 <alee> woodster_, and your example there isn't contrived. in fact, I'm coding that right now. 20:47:18 <woodster_> ha, score one for the blind sow! 20:49:09 <woodster_> so it seems we'd be ok with having such plugin-validaiton show up before the final Juno release? 20:50:14 <alee> woodster_, to be honest, I think most of the validation is already there in the supports() method. 20:50:18 <redrobot> It would be nice to get into j3, but I'm ok with it being part of juno final. 20:50:43 <alee> woodster_, certainly kmip and dogtag have both done validation in the supports() method 20:50:44 * redrobot still thinks supporting only one plugin at a time would be way easier :) 20:51:01 <alee> with respect to the key_spec 20:51:21 <alee> the validation that is there already is probably sufficient for now 20:51:42 <alee> we need to think about what further validations are needed --- if any. 20:51:55 <rellerreller> I am confused why validate is needed. I pictured initial validation from Barbican core making sure types are correct (i.e. string, int, etc), but supports does what it does now. 20:52:24 <alee> so - for now at least, this task just means ripping out validations that area lready done based on the key_spec in the plugins 20:52:47 <rellerreller> alee what? 20:53:06 <alee> sorry - I think we agree with each other. 20:53:29 <alee> rellerreller, right now there are validations that take place in barbican-core that need to be ripped out 20:53:46 <rellerreller> alee +1 20:53:46 <alee> like only supporting AES for instance .. 20:54:11 <woodster_> my concern was that the supports() method is used to select and reject plugins. If the rejecting plugins also validate, they would need to issue exceptions saying what is wrong with the data. We'd have to keep looping until we found a plugin that didn't raise a validation exception. If they all throw validation exceptions, what do we send back to the 20:54:11 <woodster_> client as the root problem? 'AllPluginsSayDataStinksException'? :) 20:55:02 <rellerreller> I think you would send back a message stating none of the plugins support the operation with the requested attributes. 20:55:31 <alee> woodster_, we can add that method later if we choose to. Additional data validaion -- other than the key_spec --- ie. the data stinks -- occurs right now on the DRM/CA for dogtag 20:55:36 <woodster_> redrobot might be on to something about just one plugin :) The issue is when one switches from one plugin type to another...after many secrets are stored the old way. You can't get rid of the old plugin in this case 20:55:43 <alee> which returns the relevant error 20:56:18 <woodster_> alee: does it raise the validation sort of error? 20:56:19 <rellerreller> I think it would help to define what validation does and what support does. 20:56:40 <reaperhulk> https://review.openstack.org/#/c/116725/ (py3pep8 tox job) 20:56:50 <woodster_> rellerreller: I tried to give a detailed example in that etherpad. 20:57:01 <alee> woodster_, yes it does 20:57:19 <woodster_> I'm fine with starting with supports() initially and and adding validate() later if it makes sense 20:57:29 <woodster_> alee: thanks 20:57:31 <alee> woodster_,+1 20:57:58 <rellerreller> woodster_ +1 20:58:42 <alee> woodster_, as we consider adding the new common cert api, we'll likely have to add some validation code. but this will likely be validation code in barbican-core 20:58:52 <alee> we can revisit a validate() then 20:59:03 <redrobot> eeek... running out of time here guys 20:59:18 <alee> but we still need to remove the bad validations in barbican-core 20:59:25 <woodster_> Cool, I'll update the etherpad shortly. We'd probably have a CR up for that in the coming weeks then unless folks think it needs to be available sooner? 20:59:55 <alee> woodster_, would like to have the old validations removed by j3 21:00:04 <redrobot> +1 ^^ 21:00:07 <woodster_> will do 21:00:17 <rm_work> Really quickly I'd like to plug https://review.openstack.org/#/c/115080/ for basic reviews -- I still need to refactor the factory methods to use explicit args instead of **kwargs, but I'd love feedback on the overall design. I know people hate reviewing WIP stuff, but this is pretty close and the basic architecture is solid at this point (though since it's in the client, it can wait until after the Juno-3 crunch is over if 21:00:17 <rm_work> people are busy) 21:00:17 <redrobot> ok, guys we're all out of time here 21:00:19 <redrobot> #endmeeting