14:59:36 <raildo> #startmeeting oslo-config-plaintext-secrets
14:59:37 <openstack> Meeting started Tue Sep  4 14:59:36 2018 UTC and is due to finish in 60 minutes.  The chair is raildo. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:59:39 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:59:41 <openstack> The meeting name has been set to 'oslo_config_plaintext_secrets'
14:59:42 <moguimar> o/
14:59:45 <raildo> #link https://etherpad.openstack.org/p/oslo-config-plaintext-secrets
15:00:08 <spilla> o/
15:01:18 <raildo> courtesy ping dhellmann redrobot
15:01:25 <redrobot> o/
15:03:01 <raildo> I think that we can start it
15:03:06 <dhellmann> o/
15:03:13 <raildo> #topic Denver PTG
15:03:16 * dhellmann is currently debugging a release job failure so may only be paying partial attention
15:03:27 <raildo> dhellmann, no worries :)
15:03:44 <raildo> #link oslo session: https://etherpad.openstack.org/p/oslo-stein-ptg-planning
15:03:53 <raildo> #link Tripleo session: Wednesday 14:00 - 15:00: https://etherpad.openstack.org/p/tripleo-ptg-stein
15:04:44 <raildo> so, we have these two session related to this topic, for the olso side, we're looking for review the next steps related to the oslo.config changes also the castellan driver
15:05:30 <raildo> for the TripleO side, we will be explaining what we are doing, why we are doing, and basically understand what will be necessary to do on TripleO do have that automated over there
15:06:21 <beekneemech> o/
15:06:39 <raildo> hey bnemec :)
15:07:02 <bnemec> Hi
15:07:57 <raildo> so, hope to have a great feedback from those sessions about this feature :)
15:08:06 <raildo> #topic Castellan driver
15:08:14 <raildo> #link https://review.openstack.org/#/c/599589/
15:08:18 <raildo> moguimar, ^
15:09:13 <moguimar> o/
15:09:28 <moguimar> first lines of code on the castellan drive are out
15:09:46 <moguimar> I'm finishing to answer Doug's review
15:09:57 <moguimar> and fix the zuul crashes
15:10:10 <moguimar> unit tests are still to be implemented
15:10:13 <moguimar> and docs as well
15:10:49 <moguimar> o\
15:11:53 <raildo> also, we're preparing a demo/slides for PTG, if needed to explain that for ppl outside this meeting, or if someone wants more details about the current progress, next steps and so on...
15:12:14 <raildo> and we want to include a demo of this castellan driver code running, as well
15:12:23 <raildo> hope to have that done by the end of the week
15:14:46 <raildo> #topic Castellan don't have update operation for secrets
15:15:11 <raildo> so, during the castellan investigation that we made, we noticed that castellan doesn't support update operation
15:15:48 <raildo> dhellmann, bnemec I'm not sure how that it will affect mutable values on oslo.config without that operation
15:16:35 <dhellmann> if a secret is marked mutable, then the mapping value associated with it would have to be updated in order for a new value to be read out of the backend
15:16:55 <raildo> maybe we gonna need to implement secrets updates to enable mutable values?
15:17:07 <raildo> in the castellan side
15:17:33 <dhellmann> the process would be (1) create a new secret (2) update the mapping file with its ID (3) poke the service to have it reload its config
15:17:40 <raildo> dhellmann, the other solution would be generate a new secret and update the mapping file id
15:17:46 <dhellmann> right
15:17:49 <raildo> exactly
15:17:50 <bnemec> Yeah, we'll need to make sure the mapping opts inherit the mutability flag of their parent opt.
15:19:21 <raildo> #topic open discussion
15:19:29 <raildo> that's all that I had for today
15:19:45 <dhellmann> bnemec : I wasn't expecting us to define separate options for the mapping values
15:20:20 <bnemec> dhellmann: Sure, but we have to have them defined somehow, right?
15:20:38 <dhellmann> they're identical to the group and opt names given to the driver for lookup
15:20:52 <bnemec> Yeah, but we can't reuse them because they won't necessarily be the same type.
15:21:04 <dhellmann> they don't need to be
15:21:32 <moguimar> we can poke the option itself
15:21:37 <bnemec> We can't read a string id into an int opt, can we?
15:21:47 <moguimar> the get() recieves group_name, option_name and the opt
15:21:50 <dhellmann> accessing conf.foo.bar causes the driver to load the secret id from the bar value in the foo section and then use that value to talk to the backend and return whatever the backend gives back
15:21:59 <dhellmann> the mapping values are never returned out of the driver
15:22:42 <dhellmann> the driver doesn't need to worry about type conversion, either, the caller does all of the coercion
15:23:13 <bnemec> So we're just going to use a standard ConfigParser to read the mapping file instead of an oslo.config object?
15:23:17 * bnemec should probably go look at the review
15:23:30 <moguimar> I was just looking at that
15:23:33 <dhellmann> that's not how it was written when I looked at it, but that was my suggestion, yes
15:23:42 <moguimar> in the ConfigParser docs
15:24:00 <moguimar> the DEFAULT section behaves in a different way than oslo.conifg
15:24:26 <dhellmann> that supporting mapping file could be YAML or sqlite or anything, we just said we'd use ini for consistency from the deployer's perspective
15:24:41 <dhellmann> oh?
15:25:15 <moguimar> in configparser.ConfirParser, the DEFAULT section provides default values for the other sections when the option is not present
15:25:20 <bnemec> I ran into that once before.  DEFAULT in ConfigParser applies to all groups.
15:25:29 <moguimar> yep
15:25:32 <dhellmann> moguimar : ok, I don't think that's going to apply here, is it?
15:25:43 <bnemec> So if you have an opt DEFAULT/foo and you look up bar/foo, if bar/foo isn't defined you get the value in DEFAULT/foo.
15:25:57 <moguimar> exactly bnemec
15:26:05 <dhellmann> do we want that behavior in this case?
15:26:12 <moguimar> nope
15:26:25 <bnemec> It's kind of obnoxious and I don't think there was a good way to turn it off. :-/
15:26:26 <moguimar> we want to return NoValue
15:26:37 <moguimar> I think there is
15:27:12 <moguimar> default_section=configparser.DEFAULTSECT
15:27:18 <moguimar> in the __init__
15:27:23 <moguimar> maybe setting it to ''
15:27:47 <moguimar> would 'release' the DEFAULT section for what we want it to be
15:27:47 <bnemec> Ah, yeah that seems like it could work.
15:28:58 <moguimar> so I'm also digging in the configparser.ConfigParser alternative
15:30:27 <moguimar> another thing, I got an Oslo Config talk approved for Python Brasil in October =D
15:30:33 <bnemec> Oh, that's right.  I ran into this in https://review.openstack.org/#/c/567950/3/oslo_config/validator.py
15:30:34 <raildo> dhellmann, do you agree with the moguimar's suggestion as well?
15:30:41 <bnemec> I just used the oslo.config ConfigParser class instead.
15:31:09 <bnemec> That might work here too.
15:31:27 <moguimar> that's how I implemented it ben
15:31:47 <bnemec> Ah, okay. I really need to just go look at the review. :-)
15:33:00 <dhellmann> I'm a bit concerned that reusing our custom parsing library for this non-custom file is going to mean that somewhere down the road a change to that parser breaks things. The standard library parser is pretty stable at this point and doesn't have as much fancy baggage on top, so it feels like it's good enough for this case. I can go along with using our custom driver if everyone else feels that it's necessary, though.
15:33:58 <moguimar> dhellmann I think the standard library parser is a viable option
15:34:02 <moguimar> just need to test it
15:34:09 <moguimar> also the erro handling will be easier
15:34:09 <dhellmann> cool
15:34:16 <dhellmann> yeah
15:34:21 <moguimar> just need to disable that DEFAULT magic
15:34:37 <moguimar> as we don't have it in oslo.config
15:35:10 <raildo> cool, looks like we have some next steps for the next weeks :)
15:35:24 <raildo> we're running our of time
15:35:34 <moguimar> 🙄 we ran already
15:35:35 <bnemec> I guess the one argument _for_ using the oslo.config parser is if we do any magic in there with opt names or anything it would automatically apply to the mapping file too.
15:36:09 <bnemec> I don't know if that happens at all though, or if it ever should.
15:36:11 <moguimar> bnemec: but we don't register the options to the mapping file
15:36:28 <moguimar> that falls back to the caller
15:36:51 <moguimar> so I think option deprecation will try to fetch again from the source with the deprecated name
15:37:18 * moguimar have not looked in the deprecation code yet
15:37:26 <moguimar> has*
15:37:42 <dhellmann> yeah, the driver API is designed to make the driver implementations as simple as possible
15:37:50 <bnemec> Yeah, I'm kind of thinking out loud here. I don't know whether that's a relevant issue.
15:37:53 <dhellmann> they are specifically *not* supposed to do any guessing about alternative locations for values
15:38:21 <bnemec> I'm also curious how this would interact with the env var stuff Chris is working on. Seems like a container might want to store secrets in Castellan too.
15:38:25 <dhellmann> they basically implement a key-value lookup for a 2 part key (group and option name) and both parts of the key are always provided
15:38:53 <dhellmann> hmm
15:38:56 <dhellmann> that's interesting, bnemec
15:39:07 <moguimar> I've been reviewing that one
15:39:12 <dhellmann> I thought the idea was that the secret would be passed through the env var?
15:39:15 <moguimar> it is comming good
15:39:47 <bnemec> That's probably not something we can get for free unless we create an entire ConfigOpts structure for the mapping, which it doesn't seem like we want to do.
15:40:05 <bnemec> And I don't think it should block this work, just something we might want to consider in the future.
15:40:18 <raildo> dhellmann, bnemec would be nice to take some time during PTG to see if we can join both efforts for the next release? or see if make sense integrate somehow those efforts?
15:40:38 <dhellmann> it would be good to talk about how they might interact, for sure
15:40:57 <bnemec> raildo: It's already leveraging the driver infrastructure, but it can't be a normal driver because of precedence requirements.
15:41:11 <bnemec> But yeah, we can certainly talk about it at the PTG.
15:41:15 * raildo is really sad because will be not attending PTG this time
15:41:59 <raildo> bnemec, yeah, got it
15:42:08 <bnemec> Maybe we can grab Juan for that session? He's on your team, right?
15:42:18 <moguimar> yep
15:42:29 <bnemec> Although I imagine he's going to be a bit busy PTLing TripleO this cycle. :-)
15:42:50 <raildo> bnemec, yeah, he is, just ping me, when you have some day-time slot for this session, and I'll spoke with him to attend it
15:43:23 <bnemec> Okay, sounds good. I need to sit down and come up with a rough schedule for the Oslo day.
15:43:35 <bnemec> I'll send an email to the list once that's done.
15:43:44 <raildo> that would be great :)
15:44:00 <raildo> so, thank you guys for attending today, have a great PTG for those who will be attending it!
15:44:05 <moguimar> o/
15:44:10 <bnemec> Thanks
15:44:14 <raildo> #endmeeting