18:01:38 #startmeeting networking_policy 18:01:39 Meeting started Thu May 12 18:01:38 2016 UTC and is due to finish in 60 minutes. The chair is SumitNaiksatam. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:40 SumitNaiksatam: thanks for tackling that! 18:01:41 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:44 The meeting name has been set to 'networking_policy' 18:02:06 #info agenda https://review.openstack.org/#/q/topic:bp/gbp-network-services-framework 18:02:12 we have a one point agenda today! :-) 18:02:29 just kidding 18:02:35 we can bring up other things 18:02:48 #topic Bugs 18:02:51 ivar-lazzaro: thanks for starting the reviews 18:03:07 anything critical that we need to discuss? 18:03:21 the gate is hopefully restored now on all the branches 18:03:44 okay good 18:04:09 rkukura: i am guessing nothing new on packaging? 18:04:14 nope 18:04:25 rkukura: okay, lets skip to the main topic then 18:04:37 #topic NFP Implementation patches 18:04:47 #link https://review.openstack.org/#/q/topic:bp/gbp-network-services-framework 18:05:35 SumitNaiksatam: the orchestration patches of nfp have been posted and we should have the gate test in a day or so 18:05:54 hemanthravi: okay 18:06:05 hemanthravi: do you want to introduce other authors in your team here? 18:06:14 *patch authors 18:06:57 most of the authors of the patches are here today...ahmed, rpm, ashutosh, jagadish 18:07:10 rpm :-) 18:07:19 rajendra prasad 18:07:35 * tbachman thinks he might be going as rp_ today 18:07:42 :) 18:07:46 jokes apart, welcome ahmed, rp_ ashutosh & jagadish to the team 18:07:48 * tbachman is wrong… again ;) 18:07:52 tbachman: lol! 18:08:32 its good to put patch author names to at least IRC nicks (if not faces) :-) 18:08:45 okay lets proceed with discussion 18:08:50 they can provide any clarifications needed to start with and continue with addressing gerrit comments 18:08:56 hemanthravi: okay 18:09:18 hemanthravi: before that, can you please provide us an overview on the patch series? 18:09:49 i would like to understand (in summary) how the patches are stacked, and what is being attempted in what sequence 18:09:59 and how we can incrementally test this 18:10:18 either hemanthravi or any of the authors can chime in 18:10:24 * igordcard says hi 18:10:29 igordcard: hi there 18:10:38 SumitNaiksatam: I was also going to ask about the stacking - gerrit doesn’t seem to make that very obvious anymore 18:11:03 rkukura: i agree 18:11:10 1) NFP multiprocess library 2) DB 3) Orchestrator 4) Base Mode configurator 18:11:10 rkukura: although i think the order is preserved 18:11:27 rkukura: in the way it shows in gerrit as related patches 18:11:31 ahmed__: sorry, go ahead 18:12:03 thats the stack 18:12:21 for patches 18:12:22 ahmed__: there seem to be more than 4 patches in the stack 18:12:39 the main patches are as 1. the process framework which implements a multiprocessing framework to process the RPCs from the GBP NFP node driver 18:12:46 these are high level categories submitted in multiple patches 18:12:56 SumitNaiksatam: If the related patches is stacked in order, that answers my question 18:13:11 rkukura: i believe that is indeed the case 18:13:16 2. service orchestrator patch that implements the logical network function primitves 18:13:42 hemanthravi: ahmed__ can i request you to please provide links to the specific patches that you are discussing? 18:14:13 1) https://review.openstack.org/#/c/282292/41 18:14:21 3. device orchestarotr patch which implements device related functionality such as instanting VMs, hot-plug of interfaces etc 18:14:23 2) https://review.openstack.org/#/c/298206/20 18:14:47 ahmed__: so (2) is the DB patch, okay 18:15:15 (3) is a client patch #link https://review.openstack.org/#/c/298213/29 18:15:20 3.a) Service Orchestrator - https://review.openstack.org/#/c/298224/27 18:15:26 4. node driver which is the NCP NFP node driver that talks to NFP via RPC 18:15:45 3.b) Device orchestrator - https://review.openstack.org/#/c/298297/52 18:16:19 sorry, missed node driver in my list, its 4) Node driver - https://review.openstack.org/#/c/298380/61 18:16:38 5) base mode configurator - https://review.openstack.org/#/c/298385/60 18:17:04 hmmm 18:17:12 SumitNaiksatam: we should probably have a section on wiki describing this 18:17:20 songole: i agree 18:17:36 because in my gerrit i am not seeing the order that ahmed__ is describing 18:17:42 5) base mode configurator is a reference implementation of a serivce vm that consumes the REST APIs from the orchestrator 18:18:25 it will be helpful to have a single wiki page which lists all the patches, and summarize the context of those patches (and how they relate to the previous and next patch) 18:18:37 the later part should ideally be captured in the commit message 18:18:38 sumit can you go to this https://review.openstack.org/#/c/298385/60 18:18:53 and the patches are in order 1 at the bottom 18:18:56 but i dont see that happening, hence its pretty difficult to get the context 18:19:00 do you see that 18:19:03 Its there in the blueprint page also, https://blueprints.launchpad.net/group-based-policy/+spec/gbp-network-services-framework 18:20:27 sumit do you see the order if you go to https://review.openstack.org/#/c/298385/60 18:21:15 hemanthravi: i see this order: #link https://imagebin.ca/v/2gvR0Yv8dWnE 18:21:40 i am starting with the first patch: #link https://review.openstack.org/#/c/282292 18:22:02 the order your guys are describing here does not match that 18:22:20 it will be helpful if the context you provide is in accordance with that order 18:22:22 that's correct order...i think the numbering is off since we didn' list all of them only the main ones 18:22:50 hemanthravi: okay so lets folllow that 18:23:21 1. NFP framework, 2. DB mode, 3. library utils, 4. service orchestrator 18:23:51 5. Base mode configurator 18:24:04 5. mods to service orchestrator to support reference namespace based service such as haproxy in namespace 18:24:13 6. Device orchestrator 18:24:43 okay at what point in this can i do an end-to-end test? 18:25:04 i would like to do an end-to-end test at the earliest point in this stack of patches 18:25:10 and review that as a whole 18:25:25 end to end test at nfp - base configurattor ref impl 18:25:32 patches on top of that, in my thinking, would be enhancements to that core 18:25:41 hemanthravi: patch link? 18:26:03 https://review.openstack.org/#/c/298385/60 for end to end test 18:26:26 that being “NFP - Base Configurator Reference Implementation”? 18:26:33 yes 18:27:14 you also have the patch #link https://review.openstack.org/#/c/298288/47 (“GBP NFP - Added Base mode support in Service Orchestrator”) 18:27:37 is that not an end-to-end test with haproxy? 18:27:44 (without service VM) 18:29:36 SumitNaiksatam: need patches beyond that upto https://review.openstack.org/#/c/298385/60 18:29:59 hemanthravi: just for haproxy as well? 18:31:16 SumitNaiksatam: some of the patches are beyond that for eg NFP - Node Driver 18:31:56 hemanthravi: i get that, but i thought the “base mode” was to test in the most elemental way, even without having to start a VM 18:31:59 although you don't need all of them, the reason for this order was that the support for namespace haproxy was added after the support for a serivce vm 18:32:36 hemanthravi: but patches seem to be stacked in a way that the namespace haproxy patch is the 5th patch 18:33:08 so it seems to have been inserted early in the chain (even though it was added later) 18:33:14 that's correct but some of the code required comes in a later patch 18:33:19 hemanthravi: hmmm 18:33:20 that's correct 18:33:59 if that was not the case, it would have been much easier to review the first 5 patches as a contained unit versus 10 18:34:17 hence i was trying to get clarification 18:34:27 the first 5 patches and NFP - Node driver can be reviewed first 18:34:57 that will the code required to launch haproxy in namspace 18:35:11 hemanthravi: okay, but you are saying we cant test them in an installation (which is the easiest way to validate and understand :-)) 18:36:28 SumitNaiksatam: looks like some of the code from the patches upto nfp - base configurator is reqd 18:36:35 anyway, like songole suggested if you can put this information on a wiki page, and in there mention what aspect is missing from this base mode patch (and comes later), that will probably give us enough information to review the first fiv 18:36:39 *five 18:36:42 SumitNaiksatam: will do 18:37:00 hemanthravi: thanks 18:37:29 SumitNaiksatam: can we review the first 5...those will be required 18:37:39 hemanthravi: okay 18:37:45 6, 7 are not not required for haproxy namespace 18:37:50 what do the other team members think? 18:38:37 rkukura ivar-lazzaro igordcard tbachman: do we have enough context to review the first 5 patches? 18:38:40 agree :) 18:38:51 * tbachman has a lot of reading to do :) 18:39:08 tbachman: lol, sorry didnt mean to put you in a spot! 18:39:55 that helps - it will definitely require some effort 18:40:01 rkukura: :-) 18:40:18 SumitNaiksatam: kind of 18:40:25 ivar-lazzaro: :-) 18:40:26 I've left some comments in the patch 18:40:32 okay, thanks 18:40:54 what is really bugging me right now is the fact that some classes are copy-pasted in the test directory 18:40:58 and I'm not sure why 18:41:07 like here: https://review.openstack.org/#/c/309056/13 18:41:29 There's exactly the same set of classes as https://review.openstack.org/#/c/298385/60 18:41:53 What is the first patch about? Testing the configurator? 18:42:17 ivar-lazzaro: this was to get around an issue where the unit tests were using methods in base class 18:42:34 as opposed to the method in nfp....couldn't figure out a way to resolve this 18:42:59 which unit tests? 18:44:00 Maybe I should look at the whole chain first, but I'm not really sure what https://review.openstack.org/#/c/298385/60 is about 18:44:21 more context in the commit message would be great :) for all the patches possibly 18:44:38 Sorry I meant: https://review.openstack.org/#/c/309056/13 18:44:42 ivar-lazzaro: the tests i was referring to was not these, don't remember the exact ones 18:44:50 ivar-lazzaro: i have put comments to that effect in the earlier patches 18:45:06 devref document is also required in the relevant patches 18:45:16 and wil help to resolve some of these questions upfront 18:45:31 https://review.openstack.org/#/c/298385/60 is base configurator implementation 18:45:31 to avoid having the reviewers to guess 18:45:35 SumitNaiksatam: what's the tests/contrib package supposed to host? 18:45:52 jagadish: yeah I meant https://review.openstack.org/#/c/309056/13, pasted the wrong link 18:46:14 jagadish: that is basically... The base configurator implementation... But in a different package 18:46:23 ivar-lazzaro: test/contrib is supposed to host “contributions” which are not actively supported by the GBP team but are contributed by the community 18:46:34 this has pecan controller which is same as reference service VM implementation at https://review.openstack.org/#/c/309056/13 18:46:45 ivar-lazzaro: these are meant to be ancillary artifacts used for testing 18:46:58 SumitNaiksatam: urgh 18:47:02 both are pecan controllers with a slight functionality difference. However, the class names can be corrected accordingly 18:47:03 ivar-lazzaro: example being devstack artifacts for gate job 18:47:22 SumitNaiksatam: ok, so this is not stuff that's supposed to run during UTs 18:47:39 ivar-lazzaro: i wouldnt think so 18:47:51 jagadish: there's no difference, the files are identical 18:48:05 ivar-lazzaro: by this, i assume you are referring to “something” that goes in tests/contrib 18:48:23 “this” 18:49:05 SumitNaiksatam: yes. I guess you could put there a backend emulator to run with Devstack for POCs for example 18:49:12 ivar-lazzaro: right 18:49:40 ivar, I don't think they are identical. I will look at them and send email 18:49:42 SumitNaiksatam: but we do have already a configurator in the main tree, so I'm not really sure why we need to duplicate that cose in tests/contrib 18:49:59 ivar-lazzaro: okay, i havent reached that far in the chain :-) 18:50:01 tests/contrib was meant to be a refernce for someone implementing a service cm 18:50:06 service vm 18:50:42 hemanthravi: but i think ivar-lazzaro is indicating this being used in UTs 18:50:55 SumitNaiksatam: I started backwards :D 18:50:58 and if so, we shouldnt be doing that 18:51:19 SumitNaiksatam: don't think there are UTs for impl in tests/contrib 18:51:28 ivar-lazzaro: that would beyond my ken :-) 18:51:34 hemanthravi: okay 18:52:24 ivar-lazzaro: test/contrib configurator runs in a service-vm and is only a reference 18:52:55 configurator in the main tree is the impl to support haproxy in a namespace. 18:53:19 test/contrib should not be treaated as part of the main tree 18:54:02 hemanthravi: sure, but I don't see why maintaining 2 copied of the same stuff instead of just using the controller in the main tree for testing 18:54:48 hemanthravi: perhaps something ^^^ to sync up with ivar-lazzaro offline if you are not able to de-duplicate 18:55:16 hemanthravi: ahmed__ regarding the first patch itself , i undertand that we are putting in a framework 18:55:41 how specific is this to NFP, and/or can parts or all of this be leveraged from somewhere else (like oslo)? 18:55:58 just want to understand taht we have done the due deligence on this 18:56:12 ivar-lazzaro: tests/contrib impl is different, mainly logs the calls 18:56:20 oslo or for that matter any other official or un-official project 18:56:26 I missed the question at HH:38, but I haven't reviewed the NFP patches so I can't say anything about them at this point 18:57:27 igordcard: would appreciate if you can find some time to review :-P 18:57:42 framework is different than oslo, levarages where required, supports distributor-worker model in NFP 18:57:50 ahmed__: okay 18:58:09 certain specifics like communication between processes, sequencing of events 18:58:28 ahmed__ hemanthravi want to make sure that there is no gratituous duplication 18:58:43 event bindings to same worker, abstract polling APIs are implemented 18:59:21 SumitNaiksatam: uses oslo where possible, but the framework provides primtives for an event based multiprocessing impl 18:59:24 ahmed__: hemanthravi wherever applicable we should contribute to other projects if the same aspect is already available or being developed there 19:00:01 hemanthravi: okay, and that you are saying is specific to this context and design and currently not available or being worked on in other projects? 19:00:13 and there is no duplication and is a common design-pattern with python multiprocessing module 19:00:22 this was not planned as generic framework, more like a library to abstract and support NFP modules 19:00:37 hemanthravi: ahmed__ okay thanks for that clarification! 19:00:42 seems like we are out of time 19:00:44 there is no framework that is abstracted from an impl 19:00:50 hemanthravi: okay 19:00:51 in other projects 19:01:06 hemanthravi: it will be good if you can coordinate putting the wiki page for this 19:01:13 while we continue the reviews 19:01:14 SumitNaiksatam: will do 19:01:20 need to yield to the next meeeting 19:01:39 thanks hemanthravi ahmed__ RajendraPrasad ashutosh 19:01:46 we could sechdule a meeting if required before next irc 19:01:50 ok bye till the next meeting 19:01:57 missed jagadish 19:02:00 hemanthravi: yes 19:02:01 bye 19:02:02 bye SumitNaiksatam 19:02:03 thanks! 19:02:03 bye 19:02:07 bye 19:02:11 bye 19:02:12 #endmeeting