15:00:06 <raildo> #startmeeting oslo-config-plaintext-secrets
15:00:06 <openstack> Meeting started Tue Apr 24 15:00:06 2018 UTC and is due to finish in 60 minutes.  The chair is raildo. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:10 <openstack> The meeting name has been set to 'oslo_config_plaintext_secrets'
15:00:14 <gagehugo> o/
15:00:14 <raildo> #link https://etherpad.openstack.org/p/oslo-config-plaintext-secrets
15:00:19 <moguimar> o/
15:00:22 <spilla> o/
15:00:34 <raildo> hey folks :)
15:00:41 <moguimar> hey
15:01:00 <moguimar> think we're missing doug
15:01:10 <raildo> we can wait a few minutes
15:01:18 <moguimar> sure thing
15:02:26 <moguimar> dhellmann
15:02:31 <dhellmann> o/
15:02:35 <moguimar> great
15:02:36 <dhellmann> sorry I'm late
15:02:38 <raildo> let's start
15:02:40 <raildo> no problem
15:02:41 <moguimar> no worries
15:02:46 <raildo> #topic Status Check
15:02:49 <dhellmann> bnemec was going to try to drop by today, too
15:03:05 <raildo> that would be great
15:03:08 <bnemec> Yes, thanks for the ping
15:03:21 <raildo> hey bnemec, welcome :)
15:03:39 <raildo> so, we have another patch related to this topic
15:03:41 <moguimar> jaosorior is joining as well
15:03:42 <raildo> #link https://review.openstack.org/#/c/562746/
15:03:44 <moguimar> but here is next to me
15:04:01 <raildo> #link https://review.openstack.org/#/q/topic:bp/oslo-config-drivers+(status:open+OR+status:merged)
15:04:09 <raildo> ^ for all the patches related
15:05:08 <raildo> so, I believe that we are starting to get a good progress, but we need to speed up a little bit to have that done on Rocky
15:05:08 <moguimar> about the ini driver
15:05:16 <dhellmann> the file reorg looks pretty good, sorry I didn't think of that earlier in the process
15:05:48 <moguimar> I'm able to fetch the url content and store it to something like self.data
15:06:00 <moguimar> having self.uri and self.data
15:06:24 <dhellmann> we probably want those to be private attributes of the source, so self._uri and self._data
15:06:24 <moguimar> I was talking to jaosorior
15:06:25 <jaosorior> o/
15:06:31 <moguimar> sure
15:06:32 <raildo> moguimar, do you think that you can add some related tests on that patch, about it?
15:06:34 <dhellmann> it's hard to make something private once we publish it as public
15:06:41 <raildo> would be great to have those tests on it
15:06:59 <moguimar> yep, I'm used to do TDD
15:07:06 <dhellmann> but it sounds like you're making good progress if you're fetching data. I haven't looked at the parser in a while, how hard is it to pass data to that without it being in a file?
15:07:06 <raildo> jaosorior, hey :)
15:07:07 <moguimar> and I was digging in the test cases to learn how to do that
15:07:15 <jaosorior> yo
15:07:18 <jaosorior> greetings from Brno
15:07:33 <moguimar> I was talking to ozz on getting the self._data and writing it to a NamedTempFile
15:07:45 <moguimar> and creating a new ConfigOpts on that temporary file
15:08:06 <moguimar> latter in the get() call, we'll register the opt parameter and try to fetch it's value
15:08:10 <dhellmann> we should really only need the parser, right?
15:08:27 <jaosorior> Isn't it possible to make the Ini file parser to read from a string instead of a file?
15:08:30 <moguimar> as multiple calls to register_opt are acceptable
15:08:37 <jaosorior> you could load the string into a StringIO object, and it would read similarly to a file
15:08:40 <jaosorior> so then you don't need to write that to a tempfile
15:08:44 <moguimar> yup
15:08:50 <jaosorior> anyway, implementation details
15:08:59 <dhellmann> but important implementation details
15:09:04 <moguimar> I thought about that as well, but I haven't found some API to acomplish it yet
15:09:16 <dhellmann> I don't think we want a ConfigOpts causing a driver to create another ConfigOpts
15:09:21 <moguimar> if you guys can point me to a easier path, I'll dig into that
15:09:39 <raildo> #link https://docs.python.org/2/library/stringio.html
15:10:10 <dhellmann> we might have to modify oslo_config.cfg.ConfigParser
15:10:19 <jaosorior> dhellmann: lets say that we have a ConfigOpts object already, how do we get it to take another string object after the parsing has been made?
15:10:23 <dhellmann> for now you could write to a temporary file, I guess
15:10:25 <jaosorior> thought it would throw an exception
15:10:30 <dhellmann> jaosorior : we don't want it to
15:10:39 <jaosorior> don't we?
15:10:40 <dhellmann> all of the stuff in the driver should be hidden behind the driver API
15:10:56 <dhellmann> that's the point of the drivesr
15:11:08 <dhellmann> ConfigOpts shouldn't know about all of these other sources of data, many of which won't even be files
15:11:12 <jaosorior> and what we're talking is still behind the driver API
15:11:15 <dhellmann> right
15:11:53 <dhellmann> the spec describes the lookup logic that will be used to invoke the driver, but the driver shouldn't assume anything about ConfigOpts and ConfigOpts shouldn't assume anything about the driver beyond the API we've defined
15:12:08 <moguimar> we need to be carefull about the priority for the lookup logic
15:12:17 <dhellmann> we do, and that's described in the spec
15:12:25 <moguimar> yep
15:12:29 <jaosorior> right
15:12:42 <jaosorior> oh
15:12:45 <jaosorior> I see the issue now
15:12:46 <jaosorior> right
15:12:50 <moguimar> looping through the drivers and trying to fetch the opt
15:12:55 <dhellmann> within a given driver, the driver and source objects just need to do the *minimum* amount of work to fetch a string from their backend data source
15:13:06 <raildo> ok, so moguimar, jaosorior is that something in the spec related to this topic, that's is not clear for you guys?
15:13:40 <jaosorior> dhellmann: I think moguimar might be right in the fact that we might need to create another ConfigOpt object, that would be searched for if that specific driver is used
15:14:05 <dhellmann> how will that work with the castellan driver?
15:14:24 <dhellmann> castellan doesn't have an INI file
15:14:31 <jaosorior> dhellmann: the castellan driver is a bit simpler than this ini file driver, in the sense that in castellan you can search for specific keys in the backend
15:14:32 <moguimar> not really raildo, we came up with that plan to create a ConfigOpt over the tempfile that is doable, but if you guys can point us to a more lightweight solution, we'll dig into it
15:14:42 <dhellmann> the ConfigParser lets you look for specific keys, too
15:15:08 <jaosorior> right, but you would need to build a new one if you want to keep the hierarchy, as far as I've understood
15:15:19 <moguimar> So can I create e ConfigParser from a temp file and try to fetch the options from it?
15:15:24 <dhellmann> yes, you will need a new ConfigParser, but not a new ConfigOpts
15:15:45 <dhellmann> we will introduce lots of bugs if we register an Opt against 2 ConfigOpts at the same time, so we need to not use ConfigOpts
15:15:54 <dhellmann> moguimar : yes, that should work
15:16:03 <moguimar> great
15:16:18 <dhellmann> actually, using the parse() method you might not even need the file
15:16:20 <moguimar> as we haven't spent time on a new ConfigOpts thats even better
15:16:25 <dhellmann> it says it takes a "lineiter" argument
15:16:43 <dhellmann> so if you use splitlines() on the text you get from requests.get() that should work
15:16:52 <jaosorior> dhellmann: that's where I mentioned the StringIO object; but if it takes a lineiter, then that's not even needed and it makes stuff way easier
15:17:06 <dhellmann> yes, I think so
15:17:14 <moguimar> I think that requests is a little heavy for this job
15:17:15 <raildo> jaosorior, ++
15:17:15 <jaosorior> OK, I think that should be fine then
15:17:22 <moguimar> why not urllib.urlopen?
15:17:32 <dhellmann> moguimar : requests is the only acceptable way to handle SSL certs in python apps.
15:17:39 <dhellmann> we use it all over openstack, so it's not adding a new dependency
15:17:42 <moguimar> we only fetch the file, we don't do REST methods over it
15:17:58 <moguimar> got it
15:18:16 <dhellmann> you *might* need a _Namespace() instance to actually look up values
15:18:23 <dhellmann> in a parsed INI file
15:18:45 <dhellmann> that seems to be where the _get_value() method is defined, rather than on the parser itself
15:19:40 <dhellmann> so you would use the parser to parse the data, create a _namespace, call _add_parsed_config_file to set it up, then you can fetch values from it
15:19:48 <dhellmann> using _get_value()
15:20:06 <dhellmann> this code could do with some refactoring :-)
15:20:15 <dhellmann> but let's use it as it is for now
15:21:18 <jaosorior> I honestly got a little bit lost in that code-base
15:21:23 <jaosorior> so I don't entirely follow
15:22:11 <dhellmann> yeah, it's a little messy
15:22:15 * jaosorior still browsing the oslo.cfg codebase
15:22:17 <dhellmann> the parser really just parses the INI file
15:22:29 <dhellmann> the namespace knows about how to do lookups in argparse data and in ini data
15:22:53 <dhellmann> we only need the latter, but because that's where the lookup code is we'll want to use the _Namespace class to do it
15:23:01 <jaosorior> I see
15:23:06 <dhellmann> the alternative is to extend the ini parser, but then we have 2 ways to do the lookup and...
15:23:30 <dhellmann> it's sort of like what you were starting out to do with ConfigOpts, but doesn't go to the extent of registering the options
15:24:55 <raildo> ok, I'm trying to catch up that discussion in the etherpad, so please, take a look on that later, so we can have all the necessary steps on it :)
15:25:22 <raildo> #topic Open Discussion
15:25:27 <raildo> we still have 5 minutes
15:25:47 <raildo> anything else?
15:25:51 <moguimar> ok, what is the plan after the ini driver is done?
15:26:20 <dhellmann> the next step according to the etherpad is to do the work in ConfigOpts so it knows how to use the drivers
15:26:38 <dhellmann> that will mean taking over the patch I started and extending it with the new lookup logic and anything else that is missing
15:26:48 <spilla> theres a WIP sitting out there already
15:26:49 <dhellmann> see item 4 under "work items and owners" at the top
15:26:53 <spilla> what dhellmann said
15:26:56 <raildo> #link https://review.openstack.org/#/c/554316/
15:27:56 <moguimar> fair enough
15:28:47 <moguimar> that is all I have
15:28:49 <moguimar> for today
15:29:44 <raildo> ok, so thanks for your time folks :) have a great week!
15:29:49 <raildo> #endmeeting