15:00:34 <raildo> #startmeeting oslo-config-plaintext-secrets
15:00:35 <openstack> Meeting started Tue Jun  5 15:00:34 2018 UTC and is due to finish in 60 minutes.  The chair is raildo. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:36 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:39 <openstack> The meeting name has been set to 'oslo_config_plaintext_secrets'
15:00:42 <moguimar> o/
15:00:45 <spilla> o/
15:00:45 <moguimar> #link https://etherpad.openstack.org/p/oslo-config-plaintext-secrets
15:00:46 <raildo> #link https://etherpad.openstack.org/p/oslo-config-plaintext-secrets
15:00:53 <moguimar> first =P
15:01:05 <redrobot> o/
15:01:31 <raildo> ping dhellmann, spilla, bnemec
15:01:41 <moguimar> spilla is here already
15:01:47 <raildo> ops
15:01:52 <spilla> np :D
15:02:09 <bnemec> o/
15:02:34 <raildo> #topic Current status
15:03:01 <raildo> so, based on our work items list in that etherpad, the items 1-6 are "done" just waiting for reviews
15:03:18 <raildo> so, would be great to get any review on those patches
15:04:25 <raildo> so it still remaining to do the last topics from 7 to 9
15:04:38 <moguimar> the remaining working items are
15:04:45 <moguimar> 7 - examples generator
15:04:56 <moguimar> 8 - caching external values
15:05:02 <moguimar> 9 - documentation
15:05:37 <moguimar> 7 - sample generator*
15:06:12 <raildo> also, I forgot, please welcome redrobot :) or for the humans, Doug Mendizabal, he was a Barbican PTL a few cycles ago, and whe joined Red Hat recently and will be helping us on this feature as well :)
15:06:36 <redrobot> thanks for the introduction, raildo! :D
15:06:39 <spilla> o/ redrobot
15:07:10 <redrobot> hi everyone! 👋
15:07:54 <raildo> so, as far as I understood, the item 7 looks to be easy to be done, we just need to add the ini driver in the _list_options_for_discovery()
15:08:11 <moguimar> so, in order to close phase 0, can we finish those 3 items and go through reviews this week?
15:08:23 <moguimar> is it feasible?
15:09:33 <raildo> moguimar, since we're dealing with this stuff as a kind of "feature branch", I believe that we will need to ask for a Feature Freeze exception, for all the patches, but any review is appreciated at any moment :)
15:10:59 <raildo> bnemec, do you know any detail about the item 8 in that list, that can help us? "add a separate cache for values coming from external sources"
15:11:39 <raildo> bnemec, I saw that we have some cache definition in the cfg file, but it's focus on cache the cli options, that can be reuse somehow to point for external options?
15:11:58 <raildo> #link https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L2940-L2952
15:12:27 <raildo> #link https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L2280-L2336
15:12:31 <bnemec> This was the option cache that gets invalidated when the mutable config functionality is activated.
15:12:43 <raildo> ah, got it
15:12:52 <bnemec> We needed a separate one for drivers because we were not going to support mutability in the initial implementation.
15:13:43 <bnemec> I would guess this was referring to https://github.com/openstack/oslo.config/blob/master/oslo_config/cfg.py#L2366
15:14:27 <raildo> bnemec, right
15:15:24 <raildo> #topic Talk about option=value -> option=reference
15:15:29 <raildo> moguimar, ^
15:15:54 <moguimar> ok
15:16:23 <moguimar> right now we only fetch values from external sources that are not declared in the local ini files
15:16:48 <moguimar> at some point we talked about having
15:16:56 <moguimar> [some_group]
15:17:22 <moguimar> password="secrets:{ref-id-to-external-source}"
15:17:53 <moguimar> but then how do we know that an option value is a reference instead?
15:18:17 <moguimar> we came up with: the prefix should be "group_name:"
15:18:30 <moguimar> where group_name was defined in
15:18:33 <moguimar> [DEFAULT]
15:18:40 <moguimar> config_sources=group_name
15:19:14 <moguimar> and then, oslo.config will poke that source directly instead of trying to retrieve the value from the list of sources
15:19:26 <moguimar> in case we have the ref
15:19:53 <moguimar> thoughts on this?
15:20:11 <bnemec> So this would just be a short-circuit for the normal source search process.
15:20:20 <moguimar> yep
15:20:43 <moguimar> as some external sources relly on a key to fetch some value
15:20:53 <moguimar> the we will have
15:21:10 <moguimar> option="group_prefix:external_key"
15:21:22 <moguimar> instead of the option value itself
15:21:44 <moguimar> one question that I had is, will it work with all the option types?
15:21:52 <moguimar> like int, float, boolean?
15:22:13 <moguimar> if we try to parse the value for a reference before doing the type conversion
15:23:19 <moguimar> something like, does value contains ':'? and if so, is prefix in external sources?
15:23:26 * dhellmann slips in the back late
15:24:08 <dhellmann> why do we want to have a special syntax to refer to a secret?
15:25:03 <moguimar> since the castellan driver will require an id to retrieve the value
15:25:26 <dhellmann> we talked about having those IDs in a separate file, mapping option names to IDs
15:25:46 <dhellmann> one of the main points of doing this work using drivers was to avoid having special syntax
15:25:53 <bnemec> I would much prefer that over magic syntax.
15:26:12 <dhellmann> you're all focusing on the secret store aspect of this, but there's nothing special about the fact that the first driver will be for castellan
15:26:29 <dhellmann> in the past we talked about an etcd or hiera driver, for example
15:26:42 <moguimar> I was thinking about this approach yesterday
15:26:44 <moguimar> BEFORE nova.conf
15:26:46 <moguimar> [DEFAULT]
15:26:47 <raildo> ok, so all the secrets config that castellan will be taking care of will needs to be in external file, and then the castellan will just point for this file with all the references
15:26:48 <moguimar> ...
15:26:50 <moguimar> [db]
15:26:52 <raildo> is that right?
15:26:52 <moguimar> username=mysql
15:26:54 <moguimar> password="secret123"
15:26:56 <moguimar> AFTER nova.conf
15:26:58 <moguimar> [DEFAULT]
15:27:00 <moguimar> config_sources=secret
15:27:02 <moguimar> [secret]
15:27:04 <moguimar> driver=castelan
15:27:06 <moguimar> ...
15:27:08 <moguimar> config_refs="db:username={username_castellan_id}"
15:27:10 <moguimar> config_refs="db:password={password_castelan_id}"
15:28:05 <dhellmann> the castellan driver will need to define an option to specify the name of the file that maps config options to config ids. so in that [secret] section there might be a mapping_file option
15:28:23 <dhellmann> then the castellan driver would read that file and save the results
15:28:36 <raildo> ok, that should be fine
15:28:46 <dhellmann> and any time it is asked for an option value, it would do the lookup it needs to figure out what ID that means and then call the backend with that value instead of the option name
15:29:50 <moguimar> so basically the same idea of this config_refs example but in a different file
15:29:55 <dhellmann> yeah
15:30:04 <moguimar> good
15:30:18 <raildo> #topic FFE deadline to request?
15:30:26 <bnemec> Although I wonder if we want config_refs, or just have the mapping file duplicate the structure of the main conf file.
15:30:31 <dhellmann> I think when we talked about it before we said we'd just use configparser for that file, so the format would look the same and group names and option names could map in the same way
15:30:36 <dhellmann> right, what bnemec said :-)
15:30:53 <bnemec> Yeah, the less syntax we have to invent the better.
15:30:53 <dhellmann> so
15:30:54 <dhellmann> [db]
15:31:02 <dhellmann> password=password-id-value
15:31:55 <moguimar> agreed
15:32:06 <dhellmann> that also isolates the "specialness" of the value mapping to the driver, and makes it the driver's responsibility to deal with it, instead of ConfigOpts
15:32:43 <moguimar> that's what I thought codewise
15:32:55 <raildo> dhellmann bnemec do we already know the dates for FFE, so we can be aware of that?
15:33:22 <dhellmann> #link https://releases.openstack.org/rocky/schedule.html
15:33:37 <dhellmann> I don't see a specific oslo FFE date on the schedule
15:34:02 <dhellmann> the final release date is week R-6, which is in 5-6 weeks from now
15:34:06 <bnemec> I vaguely recall that we froze oslo two weeks before general feature freeze.
15:34:13 <dhellmann> that sounds about right
15:34:23 <dhellmann> so that would be R-7
15:34:31 <raildo> ok, so let's working with something like Jul 09 - Jul 13
15:34:32 <bnemec> As anything that landed after that had little chance of being integrated into other projects anyway.
15:34:39 <bnemec> Right.
15:34:40 <raildo> ++
15:35:24 <raildo> I'll prepare something email proposing the FFE for this feature, and explaining the reasons and the remaining work and I'll send to the openstack-dev mailing list
15:35:31 <dhellmann> I will go through all of the existing reviews 1 more time today. Unless there are issues, we should do any additional work in new patches in the series.
15:35:46 <dhellmann> well, you don't need an FFE if you get the work done by Jul 9
15:35:49 <bnemec> raildo: There's no FFE needed.  We haven't reached feature freeze.
15:35:54 <dhellmann> you would need an extension *after* that point
15:35:55 <raildo> ah, ok
15:36:08 <raildo> so, that should be fine :)
15:36:16 <raildo> #topic open discussion
15:36:16 <dhellmann> yep, just keep working away :-)
15:36:26 <raildo> anything else folks?
15:36:52 <dhellmann> nothing from me
15:36:53 <moguimar> I'm cool
15:37:18 <raildo> so, thanks everyone and have a awesome week!
15:37:22 <raildo> #endmeeting