18:01:42 <SumitNaiksatam> #startmeeting networking_policy
18:01:43 <openstack> Meeting started Thu Jul  9 18:01:42 2015 UTC and is due to finish in 60 minutes.  The chair is SumitNaiksatam. Information about MeetBot at http://wiki.debian.org/MeetBot.
18:01:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
18:01:46 <openstack> The meeting name has been set to 'networking_policy'
18:02:08 <SumitNaiksatam> #info agenda https://wiki.openstack.org/wiki/Meetings/GroupBasedPolicy#July_9th_2015
18:02:40 <SumitNaiksatam> update on the k-4 release, we dont have one yet ;-)
18:02:41 <Yi_> hi
18:02:50 <SumitNaiksatam> Yi_: hi there
18:03:15 <SumitNaiksatam> we are working through some of the drivers, and must-fix bugs
18:03:29 <SumitNaiksatam> also, there are some pending items on the API side that we would need to fix
18:03:50 <SumitNaiksatam> any questions on that before we get into the regular agenda?
18:03:52 <songole> Hello
18:04:12 <SumitNaiksatam> songole: hi
18:04:32 <SumitNaiksatam> #topic Bugs
18:05:04 <SumitNaiksatam> we dont have any criticals (there was one during the week, we will discuss in the integration tests topic)
18:05:22 <SumitNaiksatam> other bugs we need to discuss
18:05:30 <SumitNaiksatam> ?
18:05:51 <mageshgv> SumitNaiksatam: I would like to discuss a few bugs that I am working on
18:05:59 <mageshgv> #link https://bugs.launchpad.net/group-based-policy/+bug/1470815
18:05:59 <openstack> Launchpad bug 1470815 in Group Based Policy "NCP Plugin does not handle Service chain Instance update and Spec update correctly" [High,In progress] - Assigned to Magesh GV (magesh-gv)
18:06:00 <SumitNaiksatam> mageshgv: okay sure
18:06:43 <SumitNaiksatam> mageshgv: the patch for that is: #link https://review.openstack.org/197973 ?
18:06:53 <mageshgv> SumitNaiksatam: yes
18:07:36 <mageshgv> This patch handles three issues as mentioned in the commit message
18:07:43 <SumitNaiksatam> mageshgv: okay
18:07:56 <SumitNaiksatam> mageshgv: this is a change in the southboud API to the driver
18:07:59 <ivar-lazzaro> what kind of upgrades were broken?
18:08:03 <SumitNaiksatam> well not change, but addition
18:08:31 <mageshgv> SumitNaiksatam: Not really, the change is in the plugin-drivermanager
18:09:17 <SumitNaiksatam> mageshgv: yes, but i believe we have documented that interface
18:09:46 <mageshgv> SumitNaiksatam: Actually this interface is not available in the spec
18:10:14 <SumitNaiksatam> mageshgv: okay
18:10:54 <mageshgv> So this patch addresses issues with instance and spec updates
18:10:54 <SumitNaiksatam> you are right, its not the node-driver interface
18:11:08 <mageshgv> SumitNaiksatam: yes
18:11:14 <SumitNaiksatam> my bad, sorry
18:12:11 <mageshgv> Did everyone had a chance to go through the patch ?
18:12:19 <ivar-lazzaro> mageshgv: I'm doing right now
18:12:30 <mageshgv> ivar-lazzaro: okay
18:12:48 <rkukura> I just took a quick look
18:13:10 <ivar-lazzaro> mageshgv: so the two main changes are: Calling the updater when a SCI classifier changes
18:13:44 <ivar-lazzaro> mageshgv: and deploy/destroy nodes when they are added/removed from a spec
18:13:48 <ivar-lazzaro> mageshgv: is that right?
18:13:54 <mageshgv> ivar-lazzaro: yes, thats right
18:14:20 <ivar-lazzaro> mageshgv: a couple of questions though
18:14:39 <ivar-lazzaro> mageshgv: classifier_id can't be updated according to our SCI API
18:14:39 <ivar-lazzaro> mageshgv: are we supporting that?
18:15:08 <mageshgv> ivar-lazzaro: right, but the classifier itself can be updated (protocol, port, direction)
18:15:50 <mageshgv> Also, the idea is this patch will also enable the next patch which is enabling classifier update :)
18:16:47 <ivar-lazzaro> ok, I need to see what RMD does today for a classifier update
18:16:52 <mageshgv> ivar-lazzaro: In case of the classifier update, note that the classifier ID will not change in the SC instance
18:17:23 <ivar-lazzaro> yeah, I just recalled that the RMD in this case would destroy and redeploy the chain
18:17:47 <ivar-lazzaro> or even do nothing, I'm not sure why the nodes need to know about a classifier update...
18:18:02 <ivar-lazzaro> I think this should be addressed the same way we address PTs added to PTGs
18:18:34 <mageshgv> ivar-lazzaro: #link https://review.openstack.org/#/c/198639/
18:19:02 <ivar-lazzaro> the node update API is intended to react to config updates mostly
18:19:17 <mageshgv> There is a follow up patch that does an instance update rather than delete and recreate the chain everytime for any updates
18:20:04 <SumitNaiksatam> mageshgv: i believe you are propagating the update to the classifier as an update to the node config?
18:20:07 <ivar-lazzaro> mageshgv: ok but can we discuss how we want to handle specific updates? Maybe we want to address PTs added the same way
18:20:43 <mageshgv> SumitNaiksatam: No, node config is not an updateable parameter today
18:20:54 <ivar-lazzaro> mageshgv: or we want to address classifier attributes update the same way we do with PTGs
18:21:24 <ivar-lazzaro> mageshgv: let's find a consistent way that makes sense for both the framework and the drivers
18:21:35 <mageshgv> ivar-lazzaro: Right, since classifier is an api parameter in SC instance it may have to go through API update atleast in case classifier ID changes
18:21:53 <ivar-lazzaro> mageshgv: in that case yes, however we don't support that today
18:22:06 <mageshgv> ivar-lazzaro: For internal changes, I agree we should have a common mechanism
18:22:09 <ivar-lazzaro> mageshgv: for a classifier attribute update though... We should choose a different path
18:22:40 <ivar-lazzaro> ok let's discuss this then :) maybe we setup a meeting and go through a list of updates we want to support
18:23:30 <ivar-lazzaro> My second concern is around nodes added/removed from an instantiated spec... I'm not sure this can even work with a proper plumber the way it is
18:24:01 <ivar-lazzaro> the shape of the chain is changing, I'm not sure just deleting, creating and reordering some nodes will be enough
18:24:19 <mageshgv> ivar-lazzaro: This will enable any vendor drivers/plumbers to work today since we dont have a reference plumber
18:24:23 <ivar-lazzaro> but I'll drop a comment about it later, I wonder if we want to support this case from day 0
18:24:46 <mageshgv> ivar-lazzaro: We have a requirement on this for a customer
18:25:25 <ivar-lazzaro> yeah but we have to make sure this a correct way of doing it
18:25:37 <ivar-lazzaro> and that it doesn't break once the plumber is pushed
18:25:52 <mageshgv> ivar-lazzaro: Right, A plumber which does not support this update can reject it
18:26:21 <ivar-lazzaro> mageshgv: however destroying and recreating the chain should always work
18:26:30 <ivar-lazzaro> mageshgv: should we iterate from there?
18:27:03 <mageshgv> ivar-lazzaro: Agree, but it would disrupt the existing services even in cases where taking down a service is not required
18:27:03 <ivar-lazzaro> it's suboptimal but always correct, I think it's acceptable for such a complicated case
18:27:39 <ivar-lazzaro> mageshgv: if you are adding or removing a node from the chain the disruption will probably happen anyways. But I'll give more thoughts to it
18:28:14 <SumitNaiksatam> mageshgv: what is the precise requirement from the user?
18:28:18 <ivar-lazzaro> let's make sure we solve the first problem first, around the SCI updates
18:28:34 <mageshgv> ivar-lazzaro: Okay, may be you can think through and get back. I would assume, in case a traffic steering component is present, it would amount to just setting up new flows
18:29:11 <mageshgv> SumitNaiksatam:  We have multiple things required here
18:29:24 <ivar-lazzaro> mageshgv: what about reconfiguring the nodes? some services may need their routes to be recalculated
18:30:25 <mageshgv> ivar-lazzaro:  you mean a scenario where traffic steering is done by pushing routes in service VM ?
18:31:59 <mageshgv> SumitNaiksatam: Just to set the context, the following are the high level requirements: 1) Rendering security groups through Firewall if there is a Firewall in the chain
18:32:19 <ivar-lazzaro> mageshgv: for traffic stitching this may be needed, I guess I should start writing down a plumber spec for a better place of discussion
18:32:46 <mageshgv> 2) Supporting Service configuration update (We would need to allow updating node config in API for this)
18:33:17 <mageshgv> 3) Since the customer is going to go for production, any traffic disruption may not be acceptable
18:33:25 <SumitNaiksatam> mageshgv: okay thanks
18:33:29 <mageshgv> ivar-lazzaro: okay, that would be good
18:33:33 <ivar-lazzaro> mageshgv: about requirement one: you mean pushing all the contract's rules into the FW?
18:33:45 <mageshgv> ivar-lazzaro: yes
18:34:00 <ivar-lazzaro> mageshgv: for all the PRSs, or just the one containing the chain?
18:34:48 <mageshgv> ivar-lazzaro: Any PRSs between the provider-consumer pair which has a Firewall in chain
18:35:25 <mageshgv> ivar-lazzaro: For eg, if there are 5 allow rules and one redirect, the 5 allow rules would translate to firewall rules
18:35:25 <ivar-lazzaro> mageshgv: so let's say I have 2 PTGs, and PRS A, B and C in the middle
18:35:54 <ivar-lazzaro> mageshgv: ok, but that's true only for the PRS containing the redirect, right?
18:36:02 <mageshgv> ivar-lazzaro: yes
18:36:29 <ivar-lazzaro> mageshgv: ok, that looks like a specific node driver problem... But it will need hooks in order to do so
18:37:10 <mageshgv> ivar-lazzaro: Right
18:37:26 <SumitNaiksatam> ivar-lazzaro: it doesnt seem like providing the propoer hooks to support updates is a speciific node driver problem
18:37:36 <ivar-lazzaro> mageshgv, rkukura: being the only ones that worked on a node driver so far, do you feel we should implement a more sophisticated hook mechanism? So that your node driver (or plugin) can react to different GBP changes?
18:37:56 <ivar-lazzaro> SumitNaiksatam: yeah the hooks is a framework problem
18:38:24 <ivar-lazzaro> SumitNaiksatam: the pushing SGs into FW rules is... This may not hold true for all FW node drivers
18:38:27 <rkukura> hadn’t really thought about it, but maybe something like neutron’s listeners would make sense
18:38:40 <ivar-lazzaro> rkukura: that's what I was thinking
18:38:53 <SumitNaiksatam> ivar-lazzaro: i dont thinkg pusing SGs to FW rules has anything to do with the patches in discussion
18:39:24 <mageshgv> ivar-lazzaro: Before we go into this specific customer requirement, we should also fix the missing pieces in our update workflow
18:39:27 <rkukura> But it does seem like something in the framework might need to determine whether any node driver needs to have the chain rebuilt or not
18:39:39 <ivar-lazzaro> SumitNaiksatam: that's one thing mageshgv brought up in the requirements
18:39:55 <mageshgv> ivar-lazzaro: We should not silently update the DB and consider the update as done
18:40:13 <ivar-lazzaro> mageshgv: yeah and adding a listener mechanism could cover that too (for classifier internal changes for instance)
18:40:41 <SumitNaiksatam> ivar-lazzaro: i believe he was enumerating the requirements, the (2) and (3) are relevant to the patches here, (1) is a use case for those
18:41:22 <ivar-lazzaro> it seems to me that many of those requirements/holes we have, and those to come, may be solved if the driver could be invoked for any relevant change it wants to react to
18:41:40 <ivar-lazzaro> rkukura: yeah
18:42:07 <SumitNaiksatam> rkukura: i would rather say that the driver can decide whether it wants to react to the change or not
18:42:27 <SumitNaiksatam> its a simple matter of adding a hook
18:42:30 <ivar-lazzaro> that's all great feedback guys!
18:43:36 <rkukura> If any of these updates may require rebuilding the chain, doesn’t that mean the ncp code needs to ask each driver to validate the update, indicating whether it needs the node rebuilt? Or could the node drivers subscribe to notifications of the changes and somehow trigger rebuilding the chain themselves?
18:43:37 <ivar-lazzaro> SumitNaiksatam: yeah but we need either to define a standard way to add hooks for standard changes... Or a way to simply provide a way to subscribe to any GBP change you want (which scales better)
18:44:24 <mageshgv> rkukura: we have a validate hook already in place
18:44:31 <mageshgv> and a driver can reject an update
18:44:52 <SumitNaiksatam> ivar-lazzaro: i believe the latter would probably amount to more discussion
18:45:00 <rkukura> mageshgv: But that is only for updates to the servicechain resources, not to the GBP resources, right?
18:45:02 <mageshgv> ivar-lazzaro: wont the validate_update and update hook for a node driver not be sufficient
18:45:18 <mageshgv> rkukura: Right, got your point
18:45:55 <rkukura> Maybe node_driver should inherit from policy_driver, so it sees all the gbp updates if it cares?
18:46:08 <ivar-lazzaro> SumitNaiksatam: yeah but much less code to write in the future, especially with new requirements coming up
18:46:39 <SumitNaiksatam> ivar-lazzaro: enhancements and optiimizations are a perpetual process
18:46:55 <ivar-lazzaro> rkukura: then you add the node drivers in the ML2 like call chain?
18:46:55 <rkukura> Would it make sense to register the node_driver class as a policy_driver too, implementing both interfaces?
18:46:58 <SumitNaiksatam> rkukura: that seems like a reasonable idea
18:47:49 <rkukura> That might let the node driver reject GBP resource modifications it can’t handle, but still wouldn’t address triggering a chain rebuild if needed
18:48:43 <mageshgv> One issue though would be that node drivers have to know too much about GBP resources.
18:48:45 <ivar-lazzaro> rkukura: we need to make sure they own at least one node affected by the change though
18:48:53 <SumitNaiksatam> mageshgv: yeah i agree
18:49:01 <ivar-lazzaro> mageshgv: only if they want to
18:49:05 <mageshgv> It would help the case where a single vendor implements both node driver and GBP driver
18:49:42 <SumitNaiksatam> the service chain plugin is the one that should ideally be funneling the GBP resource updates to the drivers
18:50:44 <SumitNaiksatam> our original design goal was to keep the node drivers as much away as possible from having to interpret GBP resource semantics
18:50:46 <rkukura> SumitNaiksatam: I think I agree.
18:50:51 <mageshgv> IMO, It would be good if we have a clear separation and not mix them together
18:51:55 <ivar-lazzaro> SumitNaiksatam: this is right, also the nodes could be called only if the change is somehow affecting them
18:52:39 <ivar-lazzaro> but I'm starting to think that some node drivers may want to make changes based on "unpredictable" events
18:52:57 <SumitNaiksatam> ivar-lazzaro: you are advocating a subscriber mechanism, and i agree thats a good approack
18:53:14 <SumitNaiksatam> i am also thinking that adding hooks might be a simpler first step
18:54:11 <ivar-lazzaro> let's think about it, sometimes it's just the same amount of work given one more hour of thought ;)
18:54:53 <ivar-lazzaro> If that solution happens to be too complicated we will just think of a way for node driver developers to simply add hooks to NCP and their drivers
18:55:13 <mageshgv> SumitNaiksatam: I agree, hooks might be required in the log run for some node drivers, but I think we have to fix the issues we have today urgently
18:55:33 <SumitNaiksatam> ivar-lazzaro: okay
18:55:53 <SumitNaiksatam> mageshgv: okay
18:56:03 <mageshgv> SumitNaiksatam: Especially the update should not be a silent one terminating at the plugin. It should atleast check if the node would handle the update
18:56:18 <ivar-lazzaro> mageshgv: having another driver inheriting from NCP is always an option for velocity
18:56:45 <ivar-lazzaro> that's the whole point of having this stuff pluggable, same goes for the SG manager under review
18:56:48 <SumitNaiksatam> so does it seem acceptable to everyone here to proceed with the approach mageshgv has proposed, as think through this in parallel?
18:56:59 <ivar-lazzaro> mageshgv: s/driver/plugin
18:57:04 <SumitNaiksatam> ivar-lazzaro: you mean plugin
18:57:06 <SumitNaiksatam> ah :-)
18:57:46 <SumitNaiksatam> ivar-lazzaro: but ideally we would hope to not splinter as much and arrive at some good common ground
18:58:03 <ivar-lazzaro> SumitNaiksatam: I think we can add fast fixes for customer requirements in a separate plugin while we think for a clean framework solution
18:58:34 <ivar-lazzaro> SumitNaiksatam: right, but if you need velocity you don't really have too much time to discuss a good common ground
18:58:56 <SumitNaiksatam> ivar-lazzaro: we spent the whole meeting today discussing this one topic ;-)
18:59:10 <SumitNaiksatam> okay we have one minute
18:59:17 <ivar-lazzaro> that doesn't look like a 1 hour matter :D
18:59:18 <SumitNaiksatam> #topic Open Discussion
19:00:08 <SumitNaiksatam> we have changed the way we are running the gate jobs
19:00:29 <SumitNaiksatam> i mean the integration jobs
19:00:44 <SumitNaiksatam> now you can make the devstack cahnges in tree in your patches
19:00:50 <SumitNaiksatam> contact me if you need more info
19:01:02 <SumitNaiksatam> anything else anyone want to quickly bring up?
19:01:26 <SumitNaiksatam> okay thanks everyone
19:01:31 <SumitNaiksatam> bye!
19:01:35 <rkukura> I just realized I hadn’t read the log from last week - I’ll do that
19:01:36 <mageshgv> bye
19:01:37 <rkukura> bye
19:01:50 <SumitNaiksatam> rkukura: there was no meeting last week
19:01:58 <SumitNaiksatam> #endmeeting