18:01:54 #startmeeting networking_policy 18:01:55 Meeting started Thu Oct 9 18:01:54 2014 UTC and is due to finish in 60 minutes. The chair is SumitNaiksatam. Information about MeetBot at http://wiki.debian.org/MeetBot. 18:01:56 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 18:01:58 The meeting name has been set to 'networking_policy' 18:02:17 #info agenda: https://wiki.openstack.org/wiki/Meetings/Neutron_Group_Policy 18:02:47 #chairs s3wong ivar-lazzaro rkukura 18:02:54 just in case i lose connectivity 18:03:07 #topic Patches in review 18:03:20 #link #link https://review.openstack.org/#/q/status:open+project:stackforge/group-based-policy+branch:master,n,z 18:03:34 the above are the server side patches which we are focussing on for now 18:04:09 the good news is that we got some patches merged over the last few days 18:04:14 thanks to all the reviewers! 18:04:24 implicit driver is next in the chain 18:04:35 s3wong: yes, thanks for the reviews 18:05:01 the GBP redirect to service chain patches have also been pushed 18:05:19 i notice that multiple patches are in review 18:05:39 the spec is also in review, so we need to get the spec reviewed, approved, merged first 18:05:49 hemanthravi: good timing ;-( 18:05:55 oopps i meant ;-) 18:06:09 hi 18:06:18 :) 18:06:31 so taking a step back…how is the stackforge process coming along, in terms of pusing patches for reviews, and reveiwing? 18:06:49 anyone face any issues, or things we need to change? 18:07:04 we are following the standard openstack process (so its not new) 18:07:31 hopefully we can get out of the long-chain-of-dependencies mode pretty soon 18:08:00 and not have to spent the rest of lives rebasing! ;-) 18:08:02 will resume reviews from my side 18:08:07 hemanthravi: thanks 18:08:32 lets tackle the service chain spec review as a separate topic, i forgot to add it to the agenda 18:09:17 ivar-lazzaro: rkukura: anything else we need to discuss in terms of the patches that are in review? (we will discuss DB schema and resource mapping driver refactoring separately) 18:09:35 nothing I can think of 18:09:40 rkukura: okay 18:10:07 SumitNaiksatam: nope 18:10:09 btw, i forgot to mention, Ronak mentioned that he cannot make it to the meeting today, and i believe banix is still off work 18:10:12 ivar-lazzaro: okay 18:10:39 #topic GBP DB 18:11:10 Sumit: "meeting today" = this meeting, or some other meetup? 18:11:31 glebo: ah, i meant this meeting 18:11:38 sumit: ack 18:11:41 thx 18:11:43 ivar-lazzaro: so you posted the separate schema patch? 18:12:05 #link https://review.openstack.org/#/c/126383/ 18:12:11 SumitNaiksatam: yes, as it was mentioned in the ML discussion there are roughly two ways of addressing this 18:12:19 ivar-lazzaro: okay 18:12:41 ivar-lazzaro: so we for Juno we have decided to go with a separate DB so as not to break neutron migrations? 18:12:57 SumitNaiksatam: Different Schema Different Chain (DSDC) or Same Schema Different Chain (SSDC) 18:13:08 ivar-lazzaro: :-) 18:13:24 SumitNaiksatam: we definitively know that we need to have a different chain 18:13:38 I thought we prefered a separate chain in the same DB? 18:13:44 SumitNaiksatam: My patch proposes a different schema approach (the other is trivial) 18:13:50 SSDC 18:14:03 any downside to the trivial option? 18:14:25 creating a separate DB means extra work for deployment tools, etc. 18:14:36 rkukura: yes, we still risk that packagers could have some problems with that 18:14:50 rkukura: because we are touching a core project DB 18:15:00 rkukura: even though we are only adding tables 18:15:17 rkukura: the separate schema approach is the safest, no turnarounds whatsoever 18:15:24 We are building a service plugin right now, and we have foriegn keys on the existing core tables, so this just seems like the easier/safer option to me. 18:15:42 ivar-lazzaro: you mean the risk is with SSDC? 18:15:59 SumitNaiksatam: yeah that's the risk with same schema 18:16:18 A separate schema seems to mean more DB connections from the server, possibility of some DB’s not supporting it, and deployment tools needing to create extra DBs and configure their URLs into neutron.conf. 18:16:30 rkukura: DBMSs are made so there's no actual difference from a schema to another... it's just namespacing 18:16:43 rkukura: so there are no performance issues in having a different schema 18:17:15 ivar-lazzaro: According to the guy from Red Hat who maintains sqlalchemy, that is the case for mysql, but not necessarily for other DBs 18:17:17 rkukura: why do we need new deployment tools? our migration will do the trick 18:17:46 puppet scripts need to create the new DBs, set their owners, put their URLs in neutron’s conf file, ... 18:17:59 rkukura: I would say that being safe on the packaging perspective is our top priority... Maybe we can verify that the implementation works with other DBMS 18:18:17 SSDC is much safer from a packaging perspective 18:18:28 rkukura: yeah and that happens for any Stackforge Project 18:19:09 rkukura: In my experience with RH, adding tables to the existing Neutron DB is actually a problem (backport of apic driver) 18:19:12 This stackforge project is a service plugin that may eventually become part of neuton - I don’t see why we want to add complexity that is not needed. 18:19:31 rkukura: if for some reason they add a gbp_endpoint table in Kilo, the migration will fail when you upgrade 18:19:39 rkukura: agreed, but it seems that some of the discussions around packaging have indicated that adding tables to the same neutron schema raises eyebrows with the distro folks 18:19:40 The backport issue is due to the single linked list chain of migtrations, right? 18:20:07 rkukura: right, but I want to make sure that the separate chain is enough 18:20:28 rkukura: I have no preference whatsoever, I just want to make sure our stuff can be packaged and used by people 18:20:40 rkukura: so the approach that maximizes this wins for me 18:20:52 Adding a separate DB requires much more code change and support in the deployment tools. I don’t see why we aren’t just going with the simple solution. 18:21:21 rkukura: I'm aiming for the safest, not the simpliest 18:21:38 ivar-lazzaro: Do you have examples of how adding a new chain of migrations in the same DB might cause difficulty for deployment tools? 18:22:03 rkukura: yes, let's say that in Kilo someone adds a table with the same name as ours 18:22:10 rkukura: (even us if we get merged) 18:22:27 Are you 100% sure that managing multiple DB connections in the same neutron process works correctly with greenthreads and all that? 18:22:28 rkukura: then the migration Juno -> Kilo will fail because of duplication of tables 18:22:47 ivar-lazzaro: If someone adds a table containing the string “gbp”, I will -2 it 18:23:07 rkukura: what if we add it when we merge into neutron? 18:23:08 rkukura: ;-) 18:23:49 rkukura: I understand we need to take some more time validating that the approach scales and works with as many DBMS as possible 18:24:08 rkukura: if we can get an assurance from the distro folks that adding tables to the neutron schema is fine, i think we can go the SSDC route, however i think they have been apprehensive to this point 18:24:15 rkukura: but imho it's worth the effort if we are then 100% sure that packagers won't complain 18:24:21 ivar-lazzaro: If we add it, we’ll need to think about what works best for our existing users. Maybe making the upgrade migration check if the tables already exist would solve it. 18:24:45 rkukura: that requires a core change in Neutron (even alembic maybe) 18:24:53 I think packagers will have more work to do if there are additional DBs to create and configure. 18:25:13 SumitNaiksatam: +1, if we make sure that it's ok with them, I'm all for it 18:25:19 ivar-lazzaro: Are you sure a migration cannot be written to check if the table exists? 18:25:50 ivar-lazzaro: also, i recall that there was something about there being only a single DB revision table that was causing issues? 18:26:05 rkukura: I guess you can do that in the migration file 18:26:39 SumitNaiksatam: that is easy to solve, you just add a new versioning table 18:27:13 rkukura SumitNaiksatam: do we agree that we can go with SSDC if distro guys bless the approach? 18:27:21 Note that I did add a -1 and a comment with my view on this to https://review.openstack.org/#/c/126383/. Thats where we should be having this discussion. 18:28:00 ivar-lazzaro: I’m convinced either will work - just want to figure out which is best from various points of view 18:28:48 rkukura: newbie question here: you say "existing users" how many of these do we think we have for GBP? 18:28:56 rkukura: actually if we were 100% sure that distro guys are ok I would just go with SSDC 18:29:18 rkukura: in fact, it's simpler code and more intuitive approach 18:29:34 rkukura: but packaging has the top priority for me 18:29:45 glebo: If and when we get to the point of merging GBP into neutron, I think it will be largely because we’ve got existing users of our stackforge implementation. 18:30:03 rkukura: ivar-lazzaro: okay lets try to close on this by tomorrow (by checking with the distro/packagin folks) 18:30:38 ivar-lazzaro: I’ve also got concerns about whether DSDC works with postgresql, db2, etc. 18:30:43 SumitNaiksatam: I don't know many of them :) who can follow up? Maybe we should ask guidance to Ihar 18:31:00 rkukura: that has to be validated indeed 18:31:01 rkukura: and how many are there today? 18:31:04 ivar-lazzaro: yes, i thought we were already talking to him 18:31:09 ivar-lazzaro: no data that it doesn’t, just the sqlalchemy maintain’s advice that DBs differ in this 18:31:12 rkukura: I hope it's not that hard 18:32:05 anyone else have thoughts or opinions on this? 18:32:33 glebo: We don’t have enough functionality merged to the stackforge repos yet for their to be any existing users, but hopefully soon ;) 18:33:09 seems like following up with the sqlalchemy maintainer and maybe someone familiar with the puppet scripts would make sense. 18:33:26 rkukura: indeed. So, if current user count is low, then maybe user impact is not germaine at this point? 18:34:00 glebo: i think rkukura’s point is that we expect users to adopt this once we have this in stackforge 18:34:01 glebo: agreed that we aren’t talking about current users as of today. I think we were talking about at the end of kilo or in the L cycle. 18:34:24 glebo: and the expect them keep using this, regardless of where GBP ends up 18:34:45 glebo: so our approach needs to be non-disruptive for them (or at least we have to try) 18:35:01 rkukura: SumitNaiksatam: I have a related question to glebo's quesiton. Currently GBP is developed in Stackforge. Are we still targeting GBP for the Openstack K release? 18:35:10 glebo: but your point well taken as well, i guess we need to weigh the pros and cons 18:35:35 cathy_: GBP is definitely targeted for K release as well 18:35:42 rkukura SumitNaiksatam: so the impact is more from the perspective of the existing OS users who would merge GBP to their existing Neutron deployments, more so than for stackforge GBP users? 18:36:07 cathy_: what will be its eventual home is not clear at this point 18:36:17 cathy_: I’m not very hopeful of GBP getting merged into neutron during kilo, but I do expect our repos to be used with kilo. 18:36:18 cathy_: for Juno its being release via stackforge 18:36:30 *released 18:37:04 glebo: at the moment existing users == stackforge GBP users :-) 18:37:14 cathy_: did that answer your question? 18:37:27 This brings up the issue of branching within our stackforge repos. At some point, I think we’ll need a branch that tracks stable-juno and another branch that tracks master. 18:37:38 Is it widely known to users that they can use GBP in Stackforge ? 18:37:50 rkukura: yes good point 18:37:56 rkukura: +1, sadly so. The priorities for Kilo have been stated VERY clearly by the TC: Neutron stability and closing gap for the eventual deprecation of Nova-networking. Not clear to me that anything outside of those stated goals will get much air time in Kilo 18:37:58 rkukura: +1 18:38:07 #action SumitNaiksatam to explore branching for stackforge projects 18:38:40 SumitNaiksatam: yes. thanks for your answer. 18:38:46 cathy_: our plan is to get the stackforge repos into a useable state before the kilo summit, and do our best their to raise awareness 18:38:56 glebo: I think that was the primary reason for GBP not able to land in Juno as well 18:39:30 SumitNaiksatam: what are plans for gbp meetings at the summit? 18:39:31 s3wong: ack. And, depending on progress toward that goal, may be used again in L 18:39:41 cathy_: this has been discussed in ML, any reason to believe that users are still not aware of this? 18:39:53 SumitNaiksatam: rkukura: in terms of advertising this during the K-Summit... other than our conference presentation, is there anything else? I mean, do we have a booth? :-) 18:39:54 LouisF: yes we will defintiely be meeting 18:40:17 SumitNaiksatam: any sessions on gbp? 18:40:17 we're off on a few tangents. Did we yet close the SSDC vs DSDC topic? 18:40:35 glebo: good point 18:40:44 LouisF: lets circle back to this in the open discussion 18:41:10 glebo: we will close on SSDC versus DSDC by tomorrow based on distro/packagers input 18:41:23 glebo: I believe the conclusion there is if same schema is OK with release and packaging folks, we will go with SSDC --- that is my understanding so far 18:41:29 #action ivar-lazzaro to send update to ML based on decision by tomorrow 18:41:58 s3wong SumitNaiksatam: wfm 18:42:12 #topic Resource Mapping driver refactoring proposal 18:42:16 s3wong SumitNaiksatam: thx for clarifying 18:42:29 ivar-lazzaro: another item for you ;-) 18:42:39 SumitNaiksatam: yeee 18:42:44 ivar-lazzaro: i believe you have posted a LP BP for this 18:42:51 SumitNaiksatam: So I proposed a BP 18:42:56 #link https://blueprints.launchpad.net/group-based-policy/+spec/mapping-extension-refactor 18:42:59 SumitNaiksatam: #link https://blueprints.launchpad.net/group-based-policy/+spec/mapping-extension-refactor 18:43:05 yeah that one 18:43:08 ivar-lazzaro: thanks, but i beat you ;-P 18:43:20 SumitNaiksatam: ;) 18:43:21 ivar-lazzaro: so we will wait for the more detailed spec 18:43:37 ivar-lazzaro: in the meanwhile can you please quickly summarize in which direction you are going? 18:43:46 sure 18:43:55 ivar-lazzaro: This is more radical than I expected - are you suggesting dropping the other mapping attributes, or just moving them to a different extension? 18:43:57 ivar-lazzaro: i think the vendor drivers will be interested in knowing about this 18:44:15 rkukura: the main extension should be PT->Port 18:44:26 rkukura: everything else can be implicit, or different extension 18:44:42 So the rationale behind the proposal is that we want to expose a common mapping API 18:44:48 ivar-lazzaro: I was considering including that part in a base-mapping extension in https://blueprints.launchpad.net/group-based-policy/+spec/gbp-extension-drivers 18:44:57 that user knows is the common denominator between mapping drivers 18:45:38 so ivar-lazzaro are you proposing to remove all other mapping? 18:45:56 ivar-lazzaro: I think this does kind of make sense, especially if policy drivers can have corresponding extension drivers that expose their specific mapping attributes 18:45:56 The extension we have now is complex and set a very big constraint on how drivers should map the GBP model 18:46:36 Also, showing constructs like Network/Subnet/Router may be not very useful, especially in write mode 18:46:38 But I’m a bit concerned that this kind of blocks any possibility of GBP supporting multiple policy drivers simultaneously in the future. 18:47:04 So my proposal is that our main GBP mapping extension (common one) is PT->Port RO 18:47:09 this is needed by Nova 18:47:47 ivar-lazzaro: but to create the port you need all the other mapping as well, right? 18:47:48 and gives the drivers the capability to do whatever mapping they want implicitly, or using extension for explicit mapping as per rkukura proposal 18:47:49 ivar-lazzaro: I think you and I are agreeing pulliing PT->port into a base-mapping extension makes sense. Others? 18:47:56 * s3wong already confused by PT --- forgot that EP is now policy target... 18:48:07 s3wong: :) 18:48:21 SumitNaiksatam: you do, but don't confuse DB with API 18:48:37 SumitNaiksatam: you can do whatever you want in the backend without exposing it to the outside world 18:48:38 ivar-lazzaro: I thought your proposal was going to be more around factoring out code from resource_mapping driver that other drivers can use. 18:48:56 SumitNaiksatam: and I don't see any compelling need to expose those constructs (besides port) 18:48:57 ivar-lazzaro: rkukura: i think you folks have thought about this more than i have, so i need to see the spec or the patch 18:49:22 rkukura: that was my initial understanding as well 18:49:34 rkukura: that is a direct consequence. We converge in a common mapping extension (used by the reference implementation) and then we add extensions 18:49:38 rkukura: seems like we are leapfroggin :-) 18:49:39 rkukura: agreed... that was the item we talked about during the hackathon: which is factoring out implicit driver for EP/EPG/L2P/L3P for a common piece where other policy drivers can use 18:50:08 s3wong: that is a different item, that's about our code architecture per se 18:50:25 s3wong: what I'm bringing up here is about the minimal API we need to set as GBP "standard" 18:50:39 s3wong: The implicit_policy driver is already totally separate from any specific mapping, so it should be useable with any policy driver. 18:50:53 This will help users moving from a driver to another without issues (or even using them together) 18:51:10 ivar-lazzaro: I see --- so you are thinking of making anything other than EP/EPG to be non-mandatory... 18:51:12 rkukura: s3wong: the implicit policy driver is fine 18:51:28 s3wong: only EP -> Port will be standard 18:51:46 s3wong: any other mapping will be decided by the driver itself 18:51:49 ivar-lazzaro: not even EPG, huh? I guess you don't want to always map it to a subnet 18:51:57 Is there a need for an additional BP covering refactoring the resource_mapping driver code, such as the functions to create ports, subnets, networks, etc., into a class that any driver can inherit or compose? 18:51:58 s3wong: abstracting is the whole point of GBP in fact ;) 18:52:24 rkukura: yes, that is also useful, but we can discuss that separately 18:52:44 rkukura: sure, however we need to optimize on the available time as well 18:52:47 So this discussion is on the API. 18:52:53 What I'm saying is... Is there a compelling need for exposing the full mapping? 18:53:14 If not, we should do the simpler 5% effort solution which covers 90% of use cases 18:53:19 ivar-lazzaro: but that does have an impact on the mapping DB, right? 18:53:29 I’d think that, given the port, a user could navigate to the subnet and network, right? 18:53:43 rkukura: correct 18:53:50 rkukura: ivar-lazzaro agree 18:54:26 rkukura: so if we exclude the "write" capabilities, we don't have any need for exposing a more complex model which constraints future drivers 18:54:33 ivar-lazzaro: i guess you are trying to move away from forcing the users of the current mapping extension to do the mapping to routers, subnets, etc. 18:54:50 the PT->port mapping is clearly read-only, right? 18:54:54 ivar-lazzaro: and only limit to ports 18:54:55 My feeling is that nobody will even even need mapping extensions in a first place! unless they want to provide Write capabilities 18:55:13 rkukura: correct 18:55:26 ivar-lazzaro: if i recollect one of the original goals was to provide both neutron and GBP APIs concurrently 18:55:36 ivar-lazzaro: i am not sure if that holds true, we should check on that 18:55:40 I see little risk in splitting the existing mapping extension into a base-mapping extension that just covers PT->port, and a resource-mapping extension that exposes the rest of the attributes. 18:55:43 ivar-lazzaro: okay we need some more offline discussion on this 18:55:50 ivar-lazzaro: it is an interesting take on the idea that some part of mapping driver can be used by other policy-driver; when we set off with mapping driver, it was meant to be a reference implementation 18:55:50 we have 5 minutes remaining 18:55:52 s3wong: the mapping DB right now is what the main plugin inherits from... And that shouldn't be the case! So a refactor there is needed as well 18:55:59 lets circle back to the earlier topic 18:56:05 #topic Open Discussion 18:56:19 LouisF: you asked about GBP session in the Paris summit 18:56:21 s3wong: but that's a problem we have anyway if we want to give drivers the ability to map their own stuff without writing a new Plugin 18:56:21 ivar-lazzaro: My BP will move the build in mapping extension to an extension driver. 18:56:29 ivar-lazzaro: and of course now you are taking it to the extreme where you assume most vendor policy-driver will be using some parts of mapping driver 18:56:43 SumitNaiksatam: yes 18:56:44 rkukura: nice! So our combined effort will solve this problem and give a much nicer API 18:57:02 so we have a conference session/presentation: #link https://openstacksummitnovember2014paris.sched.org/event/3381db355f042c612c11960a588e31de 18:57:16 ivar-lazzaro: I kind of agree, but we do need to get the rest of the team thinking about this, which I think we’ve accomplished 18:57:20 s3wong: No that's the whole point, vendor drivers will most certainly only need PT->Port mapping 18:57:22 LouisF: was your question targeted at a desgin summit session? 18:57:28 s3wong: and that's all we have to give them 18:57:43 ivar-lazzaro: rkukura: I agree that if the intent is to let 3rd party policy driver to leverage mapping driver, the ML2 like framework thing rkukura proposes seems like a good step 18:57:50 both 18:57:53 ivar-lazzaro: rkukura s3wong: can we continue the discussion after the meeting, we have changed the topic 18:57:55 s3wong: with this proposal I'm in fact enhancing the flexibility of future drivers 18:58:13 LouisF: so as far the design summit session is concerned, that is not decided yet 18:58:34 LouisF: the wishlist for Neutron looks like this: #link https://etherpad.openstack.org/p/kilo-neutron-summit-topics 18:58:55 LouisF: we have also requested a session for GBP in the “other projects” track but we are not sure we will get that 18:58:56 SumitNaiksatam: and my understanding is ... even the pods are very limited in time-sharing in this summit 18:59:04 s3wong: okay 18:59:24 SumitNaiksatam: I heard somewhere (from mestery, I believe) that Neutron pods will only be available on Friday? 18:59:26 LouisF: short of anything else, i propose that we will at least have an uncoference session 18:59:39 s3wong: yes the schedule seems to suggest that 18:59:48 *unconference 18:59:58 SumitNaiksatam: thx 19:00:06 LouisF: and of course open to other suggestions/ideas 19:00:16 Most likely, we need to grab a room somewhere that isn't pod/Neutron related, and meet at a mutual time there 19:00:17 anyone else have any other thoughts on these logistics? 19:00:27 s3wong: yes 19:00:47 and this is going to be the case for a lot of other features/new projects 19:00:53 ivar-lazzaro s3wong rkukura: a summary of where you all converged on the mapping topic would be great, either here or in subsequent room 19:00:58 since the agenad is heavily oversubscribed 19:00:59 s3wong: We need to check on that. My understanding was the pods were available all week, but most of friday was dedicated to informal discussions rather than scheduled sessions 19:01:14 glebo: yes, this will be posted to the ML 19:01:28 glebo: I'll propose a spec very soon and we can continue the discussion there 19:01:42 rkukura: OK? because if not, that would be even worse than the good old unconference days 19:01:54 #action ivar-lazzaro rkukura to send update to ML on #link https://blueprints.launchpad.net/group-based-policy/+spec/mapping-extension-refactor and #link https://blueprints.launchpad.net/group-based-policy/+spec/gbp-extension-drivers 19:02:02 glebo: And I’ll be posting the spec for https://blueprints.launchpad.net/group-based-policy/+spec/gbp-extension-drivers 19:02:18 rkukura: but it seems like there are some dedicated "cross-projects" stuff over the first few days for the pod area? 19:02:19 okay we are a couple of minutes over 19:02:33 anything else that anyone wants to urgently bring up? 19:02:57 SumitNaiksatam: we should start working on the presentation via email? 19:03:10 SumitNaiksatam: now that banix will likely not join us... 19:03:19 s3wong: yes, i have an AI to add the policy summit slides to mandeep’s earlier slides 19:03:38 #action SumitNaiksatam to update presentation slides 19:03:40 SumitNaiksatam: OK 19:03:58 s3wong: thanks for the reminder! 19:04:20 alright, lets call it a wrap (we can continue the discussion in #openstack-gbp) 19:04:25 thanks all for joining 19:04:26 bye 19:04:30 #endmeeting