18:01:40 #startmeeting networking_policy 18:01:41 Meeting started Thu Jun 12 18:01:40 2014 UTC and is due to finish in 60 minutes. The chair is SumitNaiksatam. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:42 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:44 The meeting name has been set to 'networking_policy' 18:01:49 hi 18:01:50 #info agenda: https://wiki.openstack.org/wiki/Meetings/Neutron_Grou 18:01:56 banix: SumitNaiksatam: Hi 18:02:07 #info agenda: https://wiki.openstack.org/wiki/Meetings/Neutron_Group_Policy 18:02:10 s3wong: rkukura: hi 18:02:12 mandeep: hi 18:02:16 mandeep: hi 18:02:38 #topic Gerrit Reviews 18:02:53 Patches with EP, EPG, L2/3 Policy are in review: 18:03:00 #link https://review.openstack.org/#/c/95900 18:03:07 #link https://review.openstack.org/#/c/96050 18:03:14 #link https://review.openstack.org/#/c/96393 18:03:26 getting good review coverage on the first two patches 18:03:40 all the review comments on the first (API) patch have been addressed 18:03:52 the comments on the second patch have been partially addressed 18:03:54 SumitNaiksatam: Noticed you recently posted updates! 18:04:03 rkukura: yeah, early morning todya 18:04:16 all patches have been rebased to current 18:04:27 will try to wrap the review comments on the second patch shortly 18:05:01 anyone want to discuss anything specific with respect to the above here? 18:05:52 One note is that I’m trying to make sure all the same unit tests from the non-mapping DB and plugin layers run against the mapping versions of these. 18:06:12 rkukura: yes 18:06:17 rkukura: did you see any issues? 18:06:39 So some of my comments were around generaling the support functions a bit so they don’t have to be copied and extended. 18:06:51 rkukura: yeah, got that 18:07:14 rkukura: will have that in the upcoming patch set 18:07:17 Would be nice if I could just pass the mapping attributes to the version of endpoint(), endpoint_group(), … from the base. 18:07:20 cool 18:07:29 rkukura: yep 18:07:59 btw, i noticed that i skipped the first item in the agenda, will circle back to it towards the end 18:08:40 hi everyone 18:08:44 Baba_O_Riley: hi 18:08:58 hello sumit 18:09:00 ok if nothing else, lets move to the next topic 18:09:14 thanks for the reviews btw! 18:09:25 #topic Mapping driver/data path patches update 18:09:42 rkukura: ? 18:10:03 I’ve been spending way more time on the mapping DB layer patch than I thought would be necessary, so its all pushed back about a week 18:10:16 rkukura: ok 18:10:22 rkukura: any blockers? 18:10:31 The API layer could have been pushed, but I’ll need to sync that with SumitNaiksatam’s latest 18:11:04 rkukura: ok 18:11:13 It took me a while to figure out how to reuse the SumitNaiksatam’s DB layer UTs, but I’ve got those working, and am adding the additional UTs fot the mapping attributes 18:11:24 rkukura: ok great 18:11:37 i guess the other folks have to see the patches to have more input 18:11:52 This led to the comments on Sumit’s patches, and I think all should come together nicely 18:12:07 but happy to spend time discussing anything else that anyone wants to bring up on the mapping 18:12:11 I will definitely be posting the API, DB and plugin layer patches this week 18:12:13 rkukura: thanks for the review 18:12:17 rkukura: cool 18:12:22 and the two driver patches will be next week 18:13:10 rkukura: are we all set on the current design decisions, or is there something pending that still needs to be discussed here? 18:13:22 I think its worth the effort to get the UT approaches, patterns, re-use strategies, etc. right on these before we add the next sets of resources 18:13:38 rkukura: okay, we will bring that up in the UT part of the agenda 18:13:50 rkukura: will you be adding mapping policy-rules to security groups in these patches? or is that coming up after? 18:13:51 Last week we decided to drop the “neutron_” prefixes from the mapping attribute names, but that’s about it 18:14:03 rkukura: ok 18:14:22 The first version of the mapping patch will just establish connectivity, like in the PoC 18:14:35 rkukura: OK 18:14:55 A followon patch after the additional resources are ready will deal with enforcing policy 18:15:19 So we will need to make progress in parallel next week on getting the next sets of resources into review 18:15:30 rkukura: yes, that is the plan 18:16:00 rkukura: you mean policies other than “allow”. right? 18:16:10 That’s all I have on the mappong patches 18:16:24 banix: right 18:16:41 rkukura: i thought you meant including ‘allow' 18:16:52 rkukura: not other than ‘allow' 18:17:02 I meant that the first mapping patch will allow all traffic 18:17:09 rkukura: yes 18:17:17 rkukura: i think banix is asking about the allow action 18:17:21 If two EPGs use the same L3P, they’ll be able to talk 18:17:31 and deal with enforcing “other” policies later 18:17:46 or may be not 18:17:47 We won’t have any actions until the PolicyRule, … resources are there 18:17:49 banix: well, we aren't doing any processing of the policy-rules, so "allow" action is given to all traffic between EPGs 18:18:09 rkukura: yeah 18:18:16 SumitNaiksatam: rkukura got it 18:18:23 how will the redirect action to a service chain relate to the service chains defined in the adv-services traffic steering BP? 18:18:25 banix: ok good 18:18:31 s3wong: thanks got it 18:18:43 LouisF: we have an agenda item to discuss that 18:18:51 thx 18:18:59 rkukura: thanks 18:19:02 for the update 18:19:12 #topic Services’ integration update 18:19:20 LouisF: your question :-) 18:19:32 np 18:19:34 s3wong: any progress (i know you are blocked on the dependencies) 18:20:08 SumitNaiksatam: not much (sorry), still working with Kanzhe on service insertion and service base 18:20:16 s3wong: ok 18:20:22 until that is done, the official work can't be done 18:20:29 s3wong: you want to take LouisF’s question? 18:20:33 for LouisF 's question: 18:21:00 the service chain would actually be service chain in mandeep 's bp 18:21:34 i think the traffic steering spec (and design) need a bit more work but that is not perhaps what is being referred to here? 18:22:10 and traffic steering bp (which is not the service chaining bp) should be one of the enabling bp for service chaining 18:22:22 s3wong: right 18:22:49 the traffic steering bp defines a classifer for a chain 18:23:31 LouisF: the traffic steering bp defines classifier for a chain to redirect traffic 18:24:19 on various discussion with cgoncalves, these details may be too deep / complicated to expect tenants to perform 18:24:28 s3wong: i agree 18:24:47 s3wong: right, but can the traffic steering chains be referenced from a GBP redirect action? 18:24:51 LouisF: Service chain defined in the advanced services will be used to "render" the service chain as needed in GBP 18:25:20 LouisF: certainly traffic steering constructs are out of scope for GBP - we don't expect app-owners to start steering traffic 18:25:22 ok, I would agree the traffic steering bp being a bit complicated 18:25:30 It would be really good if the traffic sterring spec gets reviewed more thoroughly but that is for another meeting 18:25:40 LouisF: we can redirect to a service chain 18:26:06 LouisF: but a service chain needs to be defined outside of GBP when you use 'redirect' action 18:26:11 s3wong: Correct 18:26:16 mandeep: ok 18:26:29 LouisF: is your question that we would need the classifier to be able to redirect to a chain? 18:26:45 mandeep: does the service chain plan to use the traffic steering for the ref implementation? 18:26:59 LouisF: Yes, but a service needs to be defined outside GBP as well ... we do not want to recreate parallel funbctionality, just a common place to define the policy that ties up all these pieces. 18:27:20 SumitNaiksatam: LouisF: so there are two classifiers here: (a) classifier for policy-rules in contract for GBP, and (b) classifier used for traffic steering 18:27:27 no, GBP has a classifier, I was concerned about having two different classifiers: one in GBP and one in traffic steering 18:27:36 I assume that "redirect to a service chain" action has been supported in the GBP, right? The real chaining will be done in "service chaining" or "service steering" BP, right? 18:27:56 cathy_: that's right 18:27:59 cathy_: correct 18:28:03 LouisF: good point 18:28:08 banix: From a GBP point of view, i recomnd that we chaining construct in advanced services and leave the implementation to service - it coukld as today or using steering. 18:28:15 LouisF: however, the scope of those classifiers is different 18:28:19 LouisF: think of them different. The classifier in policy-rule is for app-owners to set rules upon which a policy is applied 18:28:43 LouisF: the classifier in traffic steering is for traffic flow between network services 18:28:50 When we specify "redirect", should user specify a chain-ID or should user specify an ordered sequence of service instances or we allow both? 18:28:52 mandeep: agree; since we were discussing here thought i get your take on it 18:29:51 banix: Ok 18:29:53 LouisF: when we 'redirect', it is because we want access to an EPG to be subjected to a network service - most likely defined by admin 18:30:17 s3wong: agree 18:30:19 mandeep: cathy_’s question ^^^ 18:30:24 mandeep: main point being that if the plan is to use traffic steering we need to focus on getting that right in a timely fashion as everything will be built on top of that; which in my opinion is not necessary but lets talk during the advanced services call. 18:30:58 LouisF: when we classify traffic on traffic steering, the intent is for traffic within a chain to flow to the next service to complete the definition of the chain 18:31:28 cathy_: its a reference to the service chain object 18:31:44 banix: I would prefer to keep the two not depend on each other strongly - and the keep that integration easier. But let us discuss that in the adv services issues 18:31:58 LouisF: it is likely that the consumer of these two sets of APIs to be different people (app-owner defining app's behavior on the network; admin defining how the traffic flows through the chain) 18:31:59 cathy_: i dont think we have considered allowing both 18:32:24 What information is defined in the service chain object? 18:32:25 cathy_: For now. we assume that the redirect action is always the last actoion of the chain. 18:32:30 cathy_: a reference to a service or service chain 18:32:54 banix: agreed 18:33:12 cathy_: more info here: https://review.openstack.org/#/c/93524 18:33:12 cathy_: being discussed in the advanced services subgroup 18:33:22 when you say a reference to a service chain, how you specify the service chain? an ID or a sequence of sevice instances? 18:33:23 cathy_: and i know you agreed to review :-P 18:33:35 cathy_: just an ID 18:33:48 Ok, I will give my comments. 18:33:52 an ID as s3wong says 18:34:01 LouisF: good questions, tells me that you are reviewing the BPs as promised, thanks :-) 18:34:06 OK, thanks. 18:34:12 ok lets move on, unless there is something more 18:34:12 cathy_: a service chain needs to be defined by someone (admin or tenant) by putting together the sequence of service instances 18:35:06 Any the sequence information should be passed down to driver to set up the service chain 18:35:22 cathy_: that's the intent 18:35:26 cathy_: That is correct 18:35:30 OK 18:35:47 #topic Client, CLI, Horizon update 18:36:06 hemanth ronak here? 18:36:15 perhaps hemanth is travelling 18:36:23 i wanted to check if there are any blockers 18:36:44 ok they are not here 18:36:47 moving on 18:36:55 where is the horizon work? 18:37:26 LouisF: #link https://review.openstack.org/#/c/93590/ 18:37:31 thx 18:37:50 nati_ueno was looking into helping us make improvements on that. 18:38:06 hi 18:38:27 nati_ueno: hi, do you have the link to your horizon document? 18:38:32 will review 18:38:34 nati_ueno: i cant seem to find it 18:38:42 mandeep: nati_ueno 's proposal seems more like the next step, right? 18:38:44 LouisF: thanks 18:38:52 s3wong: yes 18:38:52 let mee see 18:39:04 nati_ueno: thanks 18:39:09 s3wong: Correct 18:39:41 https://docs.google.com/presentation/d/1SmbhY5GTBKFV0U6XmAlaWn2nm-biV5bFVZDURZslNrg/edit?usp=drive_web 18:39:42 here 18:39:47 nati_ueno: great thanks 18:39:49 LouisF: ^^^ 18:39:54 ok next topic 18:39:59 #topic Unit tests 18:40:22 rkukura: we discussed some of this, you want to weigh in? 18:40:53 We already talked about reusing the base UTs in the mappings, and I’ll work with SumitNaiksatam to make that as simple as possible 18:41:36 The UTs at the DB and plugin layer are currently written using the REST API 18:42:15 There have been some discussions that these kinds of tests should avoid the REST API so they run more efficiently, etc. 18:42:25 rkukura: yeah, agree 18:42:30 Ideally, UTs would test just the code under test, in isolation 18:42:59 rkukura: is there any place in the Neutron code that we do this already? 18:43:25 So I’m wondering if we want to try to switch to having the DB and plugin UTs just call the DB and plugin class methods directly rather than via the REST APIU 18:44:14 I think most exisitng neutron code has been using the REST APIs, but I’m sure we can find some examples where DB or plugin classes are tested directly. 18:44:36 rkukura: i agree with that approach, and i wanted to do that when we started these patches as well, however I did not see any precedence for that 18:44:40 Does anyone see any reason not to switch to calling these classes directly? 18:45:40 I was hoping marun would be here to comment on this, since he has raised this issue before with the existing tests 18:45:47 here 18:46:12 marun: great - didn’t see you before 18:46:33 I agree there is no precedence, and that's a blocker. 18:46:57 marun: yeah, happy to try out any suggestions that you or anyone else has 18:47:18 I'm happy to work with someone to create a useful example, but I don't think it should preclude merging what you already have. 18:47:18 It would be easier to switch now with only 4 resources than later with many more 18:47:25 I would suggest it as an evolution. 18:47:34 Or are the tests no written yet? 18:47:44 marun: tests are written 18:47:58 The tests in https://review.openstack.org/#/c/96050/ would be the place to start 18:48:08 Is there anyone that wants to work with me to use the new approach? 18:48:53 marun: so your suggestion is to do the new approach as a separate enhancement patch? 18:48:54 marun: Is this just a matter of instantiating the DB or plugin class, initializing the DB, and calling methods to test? 18:49:24 SumitNaiksatam: yes 18:49:26 marun: agreed that we should do this as an enhancement/evolutioon 18:49:47 marun: okay, both rkukura and I will be happy to work with you on that 18:50:10 SumitNaiksatam: I can help as well 18:50:16 mandeep: okay thanks 18:50:18 If its not trivial, I agree we can’t take too much time to work this out before proceeding 18:50:43 cool, I’ll continue with the current approach in the mapping UTs for now then 18:50:50 rkukura: okay 18:50:54 rkukura: I agree with marun, that we need to try this as an enhancement 18:51:07 At least for the DB and plugin layers, will probably try to test driver classes directly 18:51:14 marun rkukura mandeep (anyone else): lets synch offline on this 18:51:20 OK 18:51:25 OK 18:51:36 last testing question I had was around functional tests 18:51:54 #action marun rkukura SumitNaiksatam mandeep to discuss how to structure UTs to make direct calls to the DB/Plugin layer (as enhancement patches) 18:52:05 rkukura: yes please go ahead 18:52:25 seems that if we do isolated UTs of the drivers, and maybe other layers, we really need some functional tests that ensure it all works together 18:52:51 rkukura: there functional tests will be different from those used in tempest? 18:53:01 rkukura: i mean scenario tests 18:53:29 I’m thinking of tests that don’t depend on anything outside neutron-server 18:54:18 basically, fire up neutron-server with the GP service and mapping drivers, and test that neutron resources (networks, ports, routers, …) get managed properly. 18:54:52 rkukura: ok 18:55:11 rkukura: i guess tempest tests do the same? 18:55:15 This is basically what the mapping driver UTs in the PoC were doing, but I really don’t think those are UTs 18:55:34 tempest would do the same, but with nova and VMs and all that 18:56:10 These functional tests would run with just a neutron repo, without needing devstack 18:56:21 rkukura: ok 18:56:38 rkukura: so we could start adding those even now, right? 18:56:41 marun: Does this sound at all like your vision of functional tests for neutron? 18:57:33 we do have some functional tests defined here: #link https://github.com/openstack/neutron/tree/master/neutron/tests/functional 18:58:11 rkukura: perhaps move those mapping driver UTs to ^^^ to begin with? 18:58:29 rkukura: yes 18:58:52 SumitNaiksatam: I’m aware of those, but they are very low level tests of interaction with the system, and probably should not run by default 18:59:12 rkukura: they are optional, they'll skip if they don't have the necessary dependencies 18:59:40 rkukura: the functional suite will run in the gate in an environment configured by devstack (for dependencies) but without the services running 18:59:49 marun: Would it make more sense to do these tests via the REST API like the current UTs, or using python-neutronclient? 18:59:53 rkukura: but it's still the right place to put non-unit tests. 19:00:04 rkukura: definitely with the rest api 19:00:15 marun: OK 19:00:23 rkukura: having a dependency on the native client would require too much coordination between the two repos 19:00:49 ok we are out of time 19:00:56 thanks all! 19:00:59 see you next week 19:01:01 bye 19:01:04 I’d have said the same thing, but we already do depend on python-neutronclient, and the mapping driver will be using it 19:01:04 #endmeeting