Wednesday, 2016-09-07

*** _edmund has joined #openstack-barbican00:23
*** _edmund has quit IRC00:23
*** dave-mccowan has quit IRC02:33
*** dimtruck is now known as zz_dimtruck02:50
openstackgerritMerged openstack/barbican: Add Barbican Verification to Install Guide  https://review.openstack.org/36169303:14
openstackgerritMerged openstack/barbican: Fix typo in barbican/tests/keys.py  https://review.openstack.org/36161003:22
*** diazjf has joined #openstack-barbican04:25
*** stevemar has quit IRC04:28
*** diazjf has quit IRC04:44
*** jaosorior has joined #openstack-barbican05:12
*** openstackgerrit has quit IRC05:18
*** openstackgerrit has joined #openstack-barbican05:19
*** stevemar has joined #openstack-barbican06:06
*** stevemar has quit IRC06:11
*** pcaruana has joined #openstack-barbican06:34
*** stevemar has joined #openstack-barbican06:44
*** andreas_s has joined #openstack-barbican06:54
*** woodster_ has quit IRC07:39
*** stevemar_ has joined #openstack-barbican08:07
*** stevemar_ has quit IRC08:12
openstackgerritOpenStack Proposal Bot proposed openstack/barbican: Imported Translations from Zanata  https://review.openstack.org/36473408:12
*** openstackgerrit has quit IRC08:34
*** openstackgerrit has joined #openstack-barbican08:34
jamespagefor reference, the cause of the unit test failures i asked about yesterday is https://bugs.launchpad.net/oslo.context/+bug/162096309:01
openstackLaunchpad bug 1620963 in oslo.context "requires positional >= 1.1.0" [Undecided,New]09:01
jamespagemin version spec of positional is not sufficient for oslo.context09:01
*** stevemar_ has joined #openstack-barbican09:08
*** stevemar_ has quit IRC09:13
jaosoriorjamespage: thanks for digging into it09:37
jamespagejaosorior, np - I figured it was some dependency type issue, just took me a while to dig it out09:37
jaosoriorjamespage: are you planning to submit a fix to the requirements repo?09:38
jamespagejaosorior, doing that now09:39
jaosoriorjamespage: awesome, thanks!09:39
openstackgerritMerged openstack/barbican: Imported Translations from Zanata  https://review.openstack.org/36473409:41
jamespagejaosorior, https://review.openstack.org/366631 for reference09:42
jaosoriorjamespage: ack09:43
*** jsheeren has joined #openstack-barbican09:49
*** jsheeren has left #openstack-barbican09:49
jaosoriorjamespage: seems you'll need to send a mail10:00
jaosoriorjamespage: to get an FFE for that bump10:00
jamespageapparenlty so10:00
jamespageI'll do that later - have toduck out for a meeting10:01
*** jsheeren has joined #openstack-barbican10:06
*** jsheeren has quit IRC10:06
*** jsheeren has joined #openstack-barbican10:06
*** dgonzalez has quit IRC10:45
*** jsheeren has quit IRC10:55
*** dgonzalez has joined #openstack-barbican10:57
*** stevemar_ has joined #openstack-barbican11:09
openstackgerritMerged openstack/castellan: Remove unused requirements  https://review.openstack.org/35794411:09
*** stevemar_ has quit IRC11:13
*** jsheeren has joined #openstack-barbican11:22
*** dave-mccowan has joined #openstack-barbican11:41
*** dave-mccowan has quit IRC11:46
*** chlong has joined #openstack-barbican12:05
*** stevemar_ has joined #openstack-barbican12:10
*** stevemar_ has quit IRC12:14
*** jsheeren has quit IRC12:40
*** woodster_ has joined #openstack-barbican12:42
*** jaosorior has quit IRC12:49
*** jaosorior has joined #openstack-barbican12:50
*** jkf has joined #openstack-barbican12:57
*** stevemar_ has joined #openstack-barbican13:10
*** stevemar_ has quit IRC13:15
*** su_zhang has joined #openstack-barbican13:21
*** kfarr has joined #openstack-barbican13:31
jamespagejaosorior, -dev emailed for FFE13:44
jamespagealthough this is a bug, not a feature - but I understand the need to communicate it well!13:45
jaosoriorjamespage: I did try to explain to one of the infra dudes that this is a bug actually. But he just said it's part of the protocol. So the mail was necessary anyway13:47
jaosoriorjamespage: thanks for looking into this13:47
jamespagejaosorior, its fine - mail is low cost :-)13:48
jamespagejaosorior, I think this is the 3rd or 4th lower bounds type bug we've discovered from CI testing of branchs against Ubuntu this cycle13:48
jamespagehopefully that's adding value rather than noise :-)13:48
*** michauds has joined #openstack-barbican13:52
*** jmckind has joined #openstack-barbican14:03
jamespagefyi - https://launchpad.net/ubuntu/+source/barbican/1:3.0.0~b3-0ubuntu114:03
jamespagenow in ubuntu14:03
jamespage:-)14:03
aleekfarr, ping14:05
kfarralee pong14:05
aleekfarr, any progress on the volume encryption problem?14:05
kfarralee, yeah some14:05
kfarrit has to do with oslo config14:05
aleekfarr, ok ...?14:07
kfarralee, trying to find relevant links14:07
kfarralee, when https://review.openstack.org/#/c/341914/13/cinder/utils.py added the keymgr import in the utils file, it changed how config opts are loaded14:08
kfarralee when it creates the key manager in that utils file, it's almost like it uses a config object that hasn't read the options from disk yet14:09
*** randallburt has joined #openstack-barbican14:10
kfarrand so it always tries to override api_class with ConfKeyManager14:10
kfarralee so I am working on a patch that does the backwards compatibility logic more robustly14:10
aleekfarr, ok14:11
*** randallburt1 has joined #openstack-barbican14:11
aleekfarr, let me know when you have something up14:11
kfarralee, I'll post updates here, though this isn't the final version: https://review.openstack.org/#/c/366750/14:12
aleekfarr, ok thanks - I'll keep an eye on it14:13
kfarralee do you know if there was a bug report opened? otherwise, I'll open one14:13
aleekfarr, I dont think so - so please open one14:14
*** randallburt has quit IRC14:14
*** jgrassler has quit IRC14:20
*** su_zhang has quit IRC14:20
*** su_zhang has joined #openstack-barbican14:21
*** spotz_zzz is now known as spotz14:24
*** su_zhang has quit IRC14:26
*** kfarr has quit IRC14:40
*** zz_dimtruck is now known as dimtruck14:42
*** jgrassler has joined #openstack-barbican14:44
arunkantalee: can you please provide input to multi-backend API docs comments https://review.openstack.org/#/c/341803/ . I will address this accordingly.15:16
aleearunkant, sure - in a meeting right now - but will look afterwards15:17
arunkantalee: thanks15:17
arunkantredrobot, kfarr: KMIP plugin support in newton branch is broken.. check bug https://bugs.launchpad.net/barbican/+bug/1620860 .15:22
openstackLaunchpad bug 1620860 in Barbican "KMIP plugin broken in newton branch" [Undecided,New]15:22
arunkantalee: ^^^ this will be issue with dogtag_plugin as well as it has similar key material storage mechanism similar to kmip.15:23
*** andreas_s has quit IRC15:25
*** stevemar_ has joined #openstack-barbican15:33
*** stevemar_ has quit IRC15:34
*** su_zhang has joined #openstack-barbican15:39
*** chlong has quit IRC15:41
*** pcaruana has quit IRC16:01
*** su_zhang has quit IRC16:02
*** su_zhang has joined #openstack-barbican16:02
*** DandyPandy has joined #openstack-barbican16:02
*** su_zhang_ has joined #openstack-barbican16:06
*** su_zhang has quit IRC16:07
*** su_zhang_ has quit IRC16:07
*** su_zhang has joined #openstack-barbican16:08
*** su_zhang has quit IRC16:08
*** su_zhang has joined #openstack-barbican16:08
*** su_zhang has quit IRC16:09
*** su_zhang has joined #openstack-barbican16:09
openstackgerritPankaj Khandar proposed openstack/barbican: Remove consumer check for project_id to match containers  https://review.openstack.org/25116816:10
*** su_zhang has quit IRC16:11
*** su_zhang has joined #openstack-barbican16:11
*** DandyPandy has left #openstack-barbican16:12
*** DandyPandy has joined #openstack-barbican16:15
*** DandyPandy has left #openstack-barbican16:15
*** su_zhang has quit IRC16:16
*** jaosorior has quit IRC16:33
*** sigmavirus is now known as irvirus16:40
*** irvirus is now known as sigmavirus16:40
*** jmckind_ has joined #openstack-barbican17:24
*** jmckind has quit IRC17:27
*** kfarr has joined #openstack-barbican17:33
kfarralee, fix is up https://review.openstack.org/#/c/366750/217:34
kfarralee, I was testing it manually, seems like it solves the problem17:35
arunkantkfarr : ping17:36
kfarrarunkant pong17:36
arunkantkfarr: Looks like kmip plugin support is broken in newton branch.. check this bug ..https://bugs.launchpad.net/barbican/+bug/162086017:37
openstackLaunchpad bug 1620860 in Barbican "KMIP plugin broken in newton branch" [Undecided,New]17:37
kfarrarunkant, I'm pretty sure I fixed that?  let me look into more. You were able to reproduce the issue recently?17:38
arunkantkfarr: Can you please check if you see the similar behavior.17:38
arunkantkfarr: the change is is master branch..17:38
kfarrarunkant, https://review.openstack.org/#/c/303202/17:39
kfarr^^ I thought that fixed it, but maybe I am wrong17:39
arunkantkfarr: Let me check as I was using mitaka branch which has cherry-pick change from newton..17:40
*** su_zhang has joined #openstack-barbican17:51
*** jkf has left #openstack-barbican17:53
arunkantkfarr: I was able to verify that change successfully for my error case as well. thanks for pointing out the review for correct fix.  I have invalidated that bug.17:56
aleekfarr, looking17:56
aleearunkant, I'm fine with the field name as it is17:57
aleeredrobot, ^^17:58
arunkantalee, thanks.17:58
redrobotarunkant alee are we talking about renaming store_plugin to secret_store_plugin?  I've been asking for that change since early drafts of the BP.  I really would like for the terminology to be consistent...18:00
arunkantredrobot: yes ..this review comment ..https://review.openstack.org/#/c/341803/12/doc/source/api/reference/store_backends.rst18:01
arunkantredrboot: field name is in context of secret-store data so not sure why we need to have prefix with secret here.18:02
redrobotalee arunkant it's a minor change.  Not sure why the push back?  Anyways, I'd like to see that change before I +2.  If other cores want to merge it anyway then go for it.  I'll still submit a patch to rename if that happens.18:03
aleearunkant, redrobot I dont have a strong preference either way, but redrobot seems to.18:05
arunkantredrobot: There is no need to do that. I am not convinced myself around the need. I will make the change.18:05
redrobotarunkant cool deal18:06
arunkantredrobot: Do you want me to rename crypto_plugin field in response to be 'secret_crypto_plugin' as well ?18:06
redrobotarunkant no, we call cryptographic plugins just crypto plugins everywhere else18:07
arunkantredrobot: okay..will do that.18:07
redrobotarunkant my concern is with matching docs to the fields.  for secret stores we always call them "secret stores" see http://docs.openstack.org/developer/barbican/plugin/secret_store.html18:08
redrobotarunkant for crypto plugins we always call them crypto plugins see http://docs.openstack.org/developer/barbican/plugin/crypto.html18:08
arunkantredrobot: Here names  used are  individual fields which are used to refer plugin in context of secret stores. I am not sure what else it can mean here. Anyhow, will make the change.18:12
aleekfarr, interesting -- looks like its failing CI18:12
kfarralee in my experience, the vendor plugin gates are unreliable and most fail18:13
aleekfarr, so they are not gating?18:14
aleeor non-voting?18:14
kfarralee yeah..18:16
openstackgerritArun Kant proposed openstack/barbican: Adding API docs for multiple backend support changes.  https://review.openstack.org/34180318:26
*** su_zhang has quit IRC18:42
*** diazjf has joined #openstack-barbican18:55
*** su_zhang has joined #openstack-barbican19:13
*** su_zhang has quit IRC19:18
openstackgerritArun Kant proposed openstack/barbican: Adding functional tests for multiple backend changes (Part 5)  https://review.openstack.org/36020219:54
openstackgerritArun Kant proposed openstack/barbican: Central logic to sync secret store data with conf data (Part 3)  https://review.openstack.org/35754419:54
openstackgerritArun Kant proposed openstack/barbican: Adding multiple backend db model and repository support (Part 1)  https://review.openstack.org/34809219:54
openstackgerritArun Kant proposed openstack/barbican: Adding rest API for secret-stores resource (Part 4)  https://review.openstack.org/35816219:54
openstackgerritArun Kant proposed openstack/barbican: Changes for multiple backend conf and friendly plugin names (Part 2)  https://review.openstack.org/35428519:54
arunkantredrobot, alee, _woodster: addressed the issue for API doc review . Can that (https://review.openstack.org/#/c/341803/) be reviewed and possibly merged.19:57
*** diazjf has quit IRC19:57
arunkantalee, woodster_ : ^^^ addressed the comments in multiple backend part 1 review ,19:58
woodster_redrobot: can you review/merge https://review.openstack.org/#/c/341803/20:00
arunkantwoodster_ : For part 2 review comment on line #47 (https://review.openstack.org/#/c/354285/12/barbican/plugin/util/multiple_backends.py), do you prefer it to moved to config module .20:03
*** diazjf has joined #openstack-barbican20:03
*** diazjf has quit IRC20:04
*** openstackgerrit has quit IRC20:04
arunkantwoodster_ : I don't have strong preference either way..so will adjust code to whatever is preferred to get +2 ..20:04
*** openstackgerrit has joined #openstack-barbican20:04
woodster_arunkant: I'm ok with that code staying put, but the camel case var I'd like to see changed...that other change you reference slipped in inadvertently20:06
arunkantwoodster_ : Yes..I just noticed that as well..working on addressing it ..will have new patch  for that20:06
woodster_arunkant: alee redrobot kfarr Overall your multiple backends impl seems to be sync db tables with config file settings, which seems a little odd to me, but maybe that's the best way to go for now. Just curious what other folks think though20:07
arunkantwoodster_ : I need to capture available plugin configuration for multiple backend case. So I am just reading the conf and applying it to db tables. This is needed so that project specific preference for a secret store can be defined. What's the concern here?20:12
woodster_arunkant: it's just odd to me to have config in config files, that is then copied into the db, and then the db is used by the the API calls thereafter. Seems like an add way to seed data into the db. I don't think that's enough to halt progress for this release, but might be good to revisit in the future.20:14
woodster_s/add/odd20:14
*** diazjf has joined #openstack-barbican20:17
woodster_arunkant: so in other words it seems cleaner to be 100% run the API from info pulled from config files (but have issue with primary key/ref IDs) or 100% from data seeded into the database (but have issue of needing to have an admin script to populate the database). So your approach blends those two sides. Again not a show stopper for this cycle necessarily,20:18
woodster_but I did want to point that out to other contribs  alee diazjf kfarr redrobot  ^^^^20:18
arunkantwoodster_ : We already have config properties for referring various plugins available in a deploymemt . Now for multiple backend case, there is just a new structure defined for supported plugins.20:18
randallburt1arunkant:  I'd have to agree with woodster_ though. Why have that information in two places? Seems like you'd list the supported plugins and the default one in the config file and then just record which one is used by a project in the project table or am I missing something.20:20
woodster_arunkant: before though those config settings were use used programmatically to auto select plugins...so they were read into memory and then used thereafter that way. There wasn't a database representation, or an API that could dynamically change things.20:20
*** su_zhang has joined #openstack-barbican20:23
woodster_arunkant: not trying to hassle you just to hassle you, rather I think this high level viewpoint will help when other folks are reviewing those code parts.20:24
randallburt1same. just being nosey really ;)20:24
woodster_randallburt1: one complicating factor is that each plugin selection needs to have a unique ID associated with them, so they can be manipulated via API later.20:25
arunkantwoodster_ : I am trying to understand what you have mentioned...let me share my understanding of this comment.20:25
randallburt1woodster_:  aren't they loaded by stevedore? those plugin names already have to be unique in the namespace20:27
arunkantwoodster_ : The preference is either to have API for populating which plugins are available in a deployment ... OR basically seed the secret_store database table via some admin script .. Is that correct?20:27
arunkantrandarllburt1 : yes those plugins are loaded via steverdore20:28
randallburt1woodster_:  but I also understand we're trying to get this shoved in before rc, so yeah, if it works, then we can argue db optimization later I suppose. Bad Thing(TM) is that once its in the database, its hard to get it out again later20:28
randallburt1arunkant:  I'd be loath to have an api to populate which plugins are available. That's a very very very operator-centric configuration that should never be changed without much due consideration and testing20:29
randallburt1arunkant:  you'd also want to prevent taking away a plugin that someone is already using20:30
woodster_randallburt1: well, another complicating factor is that there are two levels of plugins...an outer secret-store layer (KMIP, Dogtab) and an inner layer (the 'crypto' plugins like the HSM and software-only. To use the crypto layer plugins, you need to have a crypto adapter plugin installed at the outer secret-store level. arunkant is allowing us to have these20:30
woodster_run all the same time...e.g. KMIP plugin, Dogtag plugin, crypto-adapter plugin to HSM, crypto-adapter plugin to software-only20:30
arunkantrandallburt1 : Yes, there is a validation added which checks if any of the plugin is already used as preferred secret store when deleted during initialization20:31
woodster_randallburt1: so each of those has to have a unique ID...that's defined by the db in this implementation20:31
randallburt1woodster_:  yeah, I get that part too20:31
randallburt1woodster_:  I posited earlier that their stevedore plugin name suffices for that IMO20:31
woodster_randallburt1: I think earlier concerns were that concatenation these stevedore names would be awkward to clients, esp. if we later allow clients to specific per secret, per project which backend to store their secret in. We wanted to treat these various backend options as secret backend 'flavors' essentially20:33
* woodster_ flavors with cleansed UUID ref things, instead of a bit awkward setup.cfg names20:34
randallburt1woodster_:  sure sure, and that's fair. I still think the client and/or a little more config mangling would also do the trick, but I'm not fussed I guess. Really seems awkward though from an operator's perspective and easy enough to get out of sync20:36
arunkantrandallburt1: So there is logic which keeps configured plugins (specified in barbican.conf) in sync with related db table data whenever barbican starts..20:38
woodster_arunkant: actually, it might be helpful to paste in an example config file that supports the various backend options above. That way folks have an sense of what that looks like before hand. Maybe just add as a comment to the part 1 CR?20:38
randallburt1woodster_:  +120:38
aleewoodster_, randallburt1 , arunkant  -- I understand the issue you're bring up.  It is confusing and protentionaly problematic to have the config in two places.20:38
*** jmckind_ has quit IRC20:39
aleewhat makes this ok-ish for now is that there is no interface to change the plugins other than through the config file20:39
woodster_alee: I think the config would still be (effectively) in one place as far as operators are concerned. The magic is in the db syncing logic that arunkant has in there.20:39
arunkantwoodster_ : Are you looking for line 39-58 in https://review.openstack.org/#/c/341803/13/doc/source/setup/plugin_backends.rst20:40
aleewoodster_, right it would be in the config file which gets synced20:40
woodster_alee: that's true, and the only dynamic elements (project default) are handled via db update and API20:40
aleearunkant, I have not looked at the sync code yet -- but I could see problems possibly arising ..20:41
aleefor instance -- what if someone removes a config store -- but some project is still using it ?20:41
aleewhat happens then?20:41
woodster_arunkant: cool thanks20:41
aleewoodster_, randallburt1 , arunkant -- the advantage of having the secret stores in a db is that you can do checks to ensure consistency --20:43
arunkantalee, there is logic which does the validation if its in use (line # 912 in https://review.openstack.org/#/c/357544/7/barbican/plugin/util/multiple_backends.py).20:43
*** michauds has quit IRC20:43
woodster_alee: agreed20:44
woodster_so has this all gotten an official FFE?20:45
woodster_I'm thinking this series of CRs, and the lbaas integration one (https://review.openstack.org/#/c/251168/) should try to make Newton20:46
arunkantalee, woodster_ : I will think that people will add backend configuration than removing it . But logic still tries to address that situation20:46
woodster_arunkant: I think you have some amount of config file change testing in there IIRC20:46
arunkantwoodster_ : I did talk to redrobot about FFE 2 weeks before and he mentioned he will allow it and probably file for it if needed.20:47
aleearunkant, so in case of misconfig here - the service will not come up?20:48
arunkantalee: yes20:48
aleearunkant, thats ok for now I guess.  to woodster_ point though, we can remove this sync code if the source is in the db only20:49
aleeand we have an admin tool to change things20:50
aleethat admin tool could check for inconsistencies before adding / removing/ modifying secret stores20:50
randallburt1alee:  how would we bootstrap that configuration? install and then make admin api calls?20:51
aleerandallburt1, we could figure something out -- I could write something in puppet or chef or whatever .. we already do things like a dbpsync20:52
aleedb-sync20:52
aleerandallburt1, keep in mind that this feature is not enabled by default20:53
arunkantalee: We still need to provide a mechanism to specify which plugins are in use by service ..sync keeps it automatic as generally plugin configuration addition requires restart. Having another tool means two separate files to keep in sync20:53
randallburt1alee:  a bit cludgy but if everyone thinks the db is the place to store config then so be it.20:53
randallburt1alee:  fair enough20:53
aleerandallburt1, the thing is that you have db tables that use the config.  putting the config in db makes consistency checks possible20:54
aleerandallburt1, as well as eventally allowing for things like HA / multiple nodes ..20:54
aleearunkant, well after the initial population , the data is  in the db20:55
aleearunkant, the tools would be able to list plugins20:56
randallburt1alee:  they're not *impossible* without db config, if it were, you couldn't do that startup check for consistency between the config and db tables. That being said, I'm not going to argue anything on the review given that the folks with the +2 powers are ok with it20:56
aleearunkant, yup -- I'm ok with the approach for now - mostly also because I expect changes in this config to be relatively rare20:57
arunkantalee: I meant if you want to add plugin in multiple backend..there has to be mechanism to add it and same goes for removal case (assuming its not used by any project)20:57
aleearunkant, right - that could be a to-be-developed future tool20:59
aleearunkant, or we could end up just using the config file - and then we use stevedore-uniqueness20:59
aleearunkant, randallburt1 woodster_ if we were to use config file, you could use stevedore uniqueness for the id, and the plugin friendly name for user display21:01
aleewhat you lose is straightforward db consistency checks21:01
aleewhich would be the advantage of db21:02
arunkantalee: I think steverdore uniqueness is not sufficient here as the issue is associated with 2 levels of plugins..so mapping is not always 1 to 1 . There is same secret store plugin used for pkcs11 as well as for software-only. Somewhere that combination needs to be defined and can be linked on a per project level21:02
aleearunkant, well sure -- it would be secret_store_plugin_name + crypto_plugin_name21:03
aleearunkant, I'm not advocating this necessarily -- seems like it would be clunky21:03
arunkantalee: yes..something like that but as discussed earlier around concatenating name, its not client friendly and clunky21:05
aleearunkant, well client friendliness can be solved with the plugin friendly name21:05
aleewhich we had to add in any case21:05
aleeanyways I agree with randallburt1 that changing to solely db will be disruptive to operators21:06
aleeI think we're stuck with config file21:07
arunkantalee , okay .21:07
arunkantwoodster_ : Addressed previously missed comments in part 2 (https://review.openstack.org/#/c/354285/) ..please check21:15
*** alee is now known as alee_run21:18
*** spotz is now known as spotz_zzz21:29
*** kfarr has quit IRC21:34
*** su_zhang has quit IRC21:37
*** su_zhang has joined #openstack-barbican21:38
*** su_zhang has quit IRC21:38
*** su_zhang has joined #openstack-barbican21:38
*** spotz_zzz is now known as spotz21:38
*** su_zhang has quit IRC22:01
*** su_zhang has joined #openstack-barbican22:05
*** chlong has joined #openstack-barbican22:13
*** diazjf has quit IRC22:20
*** su_zhang has quit IRC22:23
*** diazjf has joined #openstack-barbican22:25
*** diazjf has quit IRC22:25
*** dimtruck is now known as zz_dimtruck22:27
*** spotz is now known as spotz_zzz22:37
openstackgerritChangBo Guo(gcb) proposed openstack/python-barbicanclient: Handle container list command correctly  https://review.openstack.org/26465922:47
*** randallburt1 has quit IRC22:48
*** zz_dimtruck is now known as dimtruck22:54

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!