00:01:10 <ekcs> #startmeeting congressteammeeting
00:01:11 <openstack> Meeting started Thu Oct  5 00:01:10 2017 UTC and is due to finish in 60 minutes.  The chair is ekcs. Information about MeetBot at http://wiki.debian.org/MeetBot.
00:01:12 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
00:01:14 <openstack> The meeting name has been set to 'congressteammeeting'
00:01:48 <ekcs> hi all! topics here as usual: https://etherpad.openstack.org/p/congress-meeting-topics
00:01:57 <ramineni_> Hi
00:02:05 <ekcs> feel free to comment/edit/add =)
00:02:07 <ekcs> hi ramineni_ !
00:04:34 <ekcs> ok let’s get started.
00:04:41 <ekcs> #topic gate blockage
00:05:03 <ekcs> since the infra transition to zuul3 congress gate has been blocked.
00:05:28 <ekcs> devstack just merged a workaround for one issue: https://review.openstack.org/#/c/508344/
00:06:14 <ekcs> but another issue got surfaced where congress doesn’t explicitly set transport url and credentials. that’s being fixed in this patch.
00:06:18 <ekcs> https://review.openstack.org/#/c/509604/
00:06:49 <ekcs> looks like that should finally unblock the gate.
00:10:28 <thinrichs> Jenkins is failing on that for 35
00:10:33 <thinrichs> Or is that old?
00:10:45 <thinrichs> I see-it's old
00:11:08 <ekcs> yea that’s an old issue we haven’t quite stabilized. needs rechecks once in a while.
00:11:26 <ekcs> should probbaly put that in our priority too.
00:12:16 <ekcs> #topic attach/detach security group
00:12:41 <ekcs> ok since ramineni dropped off and we need her for most of the other topics, i’m moving on to this.
00:13:00 <ekcs> https://review.openstack.org/#/c/508657/
00:14:00 <ekcs> the microseg use case (and probbaly others too) need an action to attach/detach security groups to ports.
00:14:11 <ekcs> the standard neutron API requires updating the entire list of security groups associated with a port (https://developer.openstack.org/api-ref/network/v2/index.html#update-port)
00:14:52 <thinrichs> Makes sense to me.  Do we have someone wanting to use this for microseg?
00:15:39 <ekcs> so here’s a patch for creating custom actions to attach/detach a single security group. ZTE wants it.
00:16:06 <ekcs> the problem is because it’s a read-modify-write process, concurrent changes may be overwritten.
00:16:43 <ekcs> but I don’t think there is really a way to avoid it given the neutron API.
00:16:46 <ekcs> any thoughts?
00:17:56 <thinrichs> See above
00:18:17 <ekcs> the best I could do is include a warning in the action description.
00:18:30 <ekcs> ok then.
00:18:51 <ekcs> ramineni_: hi are you here or still going on/off?
00:19:16 <ramineni_> ekcs: here .. hoping won't get disconnected again
00:19:22 <ekcs> ramineni_: haha =)
00:19:36 <ekcs> ok let’s move along then.
00:19:42 <ekcs> #topic continue drivers conf discussion
00:20:09 <ekcs> ramineni sees user difficulty that drivers they try to instantiate as datasource have not been enabled in conf
00:20:42 <ekcs> there are two ideas being considered for how to avoid that problem.
00:21:38 <ramineni_> thinrichs: hi
00:21:39 <ekcs> 1. remove the drivers config option completely and load all drivers specified by dev.
00:22:04 <ekcs> 2. keep the drivers config option but default it to include all/most drivers.
00:22:31 <thinrichs> ramineni_: hi
00:22:48 <ekcs> 2. is a simpler and backward compatible change, but ramineni_ sees some problem with it that we didn’t finish discussing last time.
00:22:54 <ekcs> ramineni_: you wanna elaborate?
00:24:28 <ramineni_> Ppl often forget to add any newly added driver in Conf .. so will get the error invalidDriver .. I saw that couple of times by user
00:25:20 <ekcs> clarification: drivers added to congress repo, or custom drivers added by deployer?
00:25:34 <ramineni_> So my initial thought was .. asking users to add full class name of driver with path into Conf .. didn't sound to me like good option
00:25:55 <ramineni_> Drivers added to repo
00:26:24 <ramineni_> We have to enable by stevedore instead
00:26:38 <ramineni_> Than keeping as Conf option
00:27:02 <thinrichs> If I remember right, the reason we made the drivers a config option was that we didn't need to discover them by walking through a directory or hierarchy of directories.  So an implementation short-cut.
00:27:16 <thinrichs> It also provided a safety hatch to disable a driver that was causing problems with everything else.
00:27:42 <ramineni_> thinrichs: yes ..but stevedore does that ..if we add in setup.cfg
00:27:51 <ramineni_> It discovers the drivers
00:28:03 <ekcs> ramineni_: ok understood. the scenario is someone has congress running. we add a new driver but when they upgrade they continue to use the same conf file and don’t have access to newly added drivers in new version of congress.
00:28:35 <thinrichs> ramineni_: that sounds nice to have auto-discovery.  Maybe if we had/knew-about that we wouldn't have bothered with the config option.
00:29:09 <thinrichs> It's definitely bad for usability to manually configure every driver.
00:29:33 <ramineni_> thinrichs: yes
00:29:59 <ekcs> at the same time, if we just default the driver config to include all drivers, the new users won’t experience the problem. and existing users won’t have a problem once someone tells them to erase their current driver config.
00:30:54 <ramineni_> With stevedore ..we won't load the driver until datasource is created .. we load only when they need ..that's the idea
00:32:05 <thinrichs> If we have auto-loading by default but want to support manual config as well, we could either think about a whitelist (which drivers to start) or a blacklist (which ones not to start) or both.
00:32:46 <ekcs> thinrichs: yes. proposal 2 would essentially be all by default, whitelist available.
00:32:54 <ramineni_> thinrichs: but driver loading is needed only when datasource is created right
00:33:06 <ekcs> ramineni_: does it matter when we load the driver?
00:33:39 <thinrichs> I could see lazy-loading in any case.
00:34:01 <thinrichs> Though lazy-loading would let people avoid pulling in drivers that are causing problems.
00:34:55 <thinrichs> (kind of)  If an end-user knew a driver was problematic, she could avoid instantiating it.  But without a whitelist/blacklist, there'd be no way for the admin to prevent people from loading certain drivers.
00:35:36 <ramineni_> Yes
00:35:42 <ekcs> sounds kludgy. wouldn’t it be better if the problem is surfaced right away then the deployer/op can do something about it? instead of leaving it up to the individual admin.
00:36:10 <ekcs> deployer starts congress after upgrade. sees problem and fixes it.
00:36:22 <ekcs> instead of something crashing when the everyday admin instantiates something.
00:37:12 <thinrichs> Sounds like we're gravitating toward: (i) stevedore lazy loading and (ii) optional whitelist in config enumerating drivers stevedore is permitted to load.
00:37:30 <thinrichs> Does stevedore support that kind of thing?
00:37:34 <ramineni_> Even if we keep enabled drivers or disable drivers in Conf option .. we should just accept name of driver instead of class name ..
00:37:50 <ekcs> and theese are two fairly orthogonal issues. 1. load on start vs load on use. 2. whitelist vs blacklist
00:38:32 <ekcs> and 3. class path vs driver name.
00:38:45 <ramineni_> ii . We can keep Conf option to take just driver name s
00:40:21 <ramineni_> Stevedore does nt support optional  .. One option to remove from setup.cfg ..that's it .. stevedore won't be able to identify then
00:42:36 <ramineni_> Use stevedore .. And keep disabled_drivers option ..to take driver names to disable
00:43:04 <ramineni_> 2nd option .. Leave the config option as it is
00:43:28 <ramineni_> They can remove the drivers as needed
00:43:35 <ekcs> on 2. whitelist vs blacklist, here are some pros and cons I can see.
00:43:41 <thinrichs> Would the disabled-driver config be used by our code then to stop the code from creating an instance of a driver?
00:43:45 <ekcs> blacklist pro: makes it so that current installations automatically gain access to newly added drivers.
00:43:53 <ekcs> blacklist con: if an installation currently has a custom driver enabled by whitelist, it can get broken.
00:43:55 <ramineni_> thinrichs: yes
00:44:44 <ramineni_> Before loading by stevedore we check if it's disabled ..then proceed
00:44:59 <ekcs> blacklist con: not backward compatible in that if a user currently relies on the config, it stops doing what they expect.
00:48:00 <ekcs> ok seems like a more complex discussion than initially anticipated.
00:48:56 <thinrichs> I'd imagine the following progression...
00:49:11 <thinrichs> (i) stevedore to do lazy-loading, which obviates the need for the config
00:49:28 <thinrichs> that's the default.
00:49:52 <thinrichs> (ii) enforce a whitelist given in config, if it exists, before stevedore loads
00:50:04 <thinrichs> (iii) if we want, add a blacklist config option as well
00:50:29 <thinrichs> If we do (i) and (ii) together, we're backwards compatible and we can tell people to delete their config to get all the new drivers.
00:50:47 <thinrichs> Then if we think it's valuable, add (iii) later
00:51:21 <ekcs> thinrichs: yes but the same effect i+ii can also be accomplished by just setting a config default to load all drivers. one line change.
00:51:39 <ekcs> and we still have the option to add iii later.
00:51:42 <thinrichs> You mean change the line to "*"
00:51:55 <thinrichs> Because "all" changes between releases
00:52:44 <ekcs> I meant enumerate the drivers here: https://github.com/openstack/congress/blob/master/congress/common/config.py#L67
00:52:55 <ekcs> and that can change with the release.
00:53:06 <thinrichs> Right—but it won't solve ramineni_'s problem
00:53:20 <thinrichs> where someone has config from release A and then upgrades to release B without changing their config.
00:54:26 <ekcs> ok so your (ii) is a new option, not the existing drivers option?
00:55:11 <ekcs> To me that’s not a big problem because you just tell them te delete that line in their config then they’re good.
00:55:23 <ekcs> for all future releases.
00:56:21 <ekcs> basically it comes down to whether we want to ignore the existing driver config or not.
00:57:15 <thinrichs> (ii) would say that if the config is there, use it to whitelist stevedore.  If it's not there just load everything like normal.
00:57:29 <thinrichs> But if you have your config there on purpose, we'll still respect it.
00:57:43 <thinrichs> Hence backwards compatible.
00:58:02 <ekcs> thinrichs: ok then we come back to the same problem that someone upgrading from release to release without changing/deleting their drivers config don’t get new drivers.
00:58:43 <ekcs> we can’t tell whether someone has the config there “on purpose” or not. unless we just ignore the existing config and ask users to use a new one.
00:58:49 <ramineni_> Can't we ask to change the way they enable custom drivers if any are there :)
00:59:10 <ramineni_> Mostly they would be testing purpose in most of the cases
00:59:14 <thinrichs> So then they delete it.  There's the capability of always taking all the drivers (which they do once), but if they don't want that, they get the old behavior.
00:59:53 <ekcs> ok we’re out of time. I’ll try to summarize on etherpad or something and contunue.
01:00:29 <thinrichs> Thanks all
01:00:30 <ekcs> thanks all for the detailed discussions!
01:00:45 <ekcs> bye ramineni_ !
01:00:57 <ekcs> #endmeeting