15:30:19 #startmeeting neutron_drivers 15:30:20 Meeting started Wed Nov 19 15:30:19 2014 UTC and is due to finish in 60 minutes. The chair is mestery. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:30:21 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 15:30:24 The meeting name has been set to 'neutron_drivers' 15:30:44 We have a light agenda today, and I'm going to try to wrap this up in 30 minutes or less. Wish me luck. :) 15:30:46 #link https://wiki.openstack.org/wiki/Meetings/NeutronDrivers Agenda 15:30:57 #topic Discuss Vendor Split BP 15:31:01 #link https://review.openstack.org/#/c/134680/ 15:31:06 Hopefully folks have had a chance to review this. 15:31:25 I'd like to thank armax and marun (along with kevinbenton and dougwig) for their work on this one! 15:32:03 I think this captures the reasons for the split quite accurately. 15:32:32 mestery: I am incorporating feedback as it comes…I need to look at yesterday’s feedback 15:32:45 so expect another patchset by the end of today 15:33:13 #info armax to land another version of the split spec by EOD 15:33:16 Thanks armax. 15:33:37 armax: Any huge issues which people have brought up so far which seem insurmountable? 15:34:03 I find the proposal reasonable. We might need to educate contributors a bit when it comes to synchronizing data model changes and library changes, but that’s not going to be impossible. 15:34:29 not that I could think of, to be honest 15:34:31 I think people have no idea where their library code should live 15:34:37 but danger is always lurking behind the corner :) 15:34:40 that’s another thing that can be easily solved 15:34:50 salv-orlando: that’s the beuty of the approach 15:34:50 I have two questions for us however 15:34:56 salv-orlando: it does not matter 15:35:07 I also think we'll need to work hard to evangelize this and make sure people know the deadlines, how to refactor their plugin/driver, etc. 15:35:08 salv-orlando: it’s whatever you want it to lieve 15:35:11 *live 15:35:26 armax: “beauty” is probably a word I would not use in any context where you’re compromising 15:35:31 salv-orlando: would it make sense to post those questions on the spec? it’s easier to track 15:35:37 salv-orlando: meh 15:35:40 I think someboduy did already 15:35:57 so the question for us are... 15:36:20 1) when it comes to building rpms and debs, shall the plugin maintainers work with distros on their own? 15:36:39 that is going to be a big concern for some smaller vendors 15:36:44 I mean not for building the rpms, but for having them available in a repo 15:36:55 they get visibility by being in the tree 15:37:00 salv-orlando, markmcclain: so long that the source code is reachable 15:37:20 the distro will have not difficulty of packaging the code 15:37:34 armax: ++ 15:37:56 armax: it is not that the distro will have challenges packaging 15:37:59 yeah might be… there will be some hiccups and headaches at the beginning. I think it’s a process that will smooth out over time. 15:38:03 it is whether they'll do it in the first place 15:38:05 markmcclain, salv-orlando: I want to stress the fact that packaging is only the first step into getting your hands on any solutioin 15:38:09 if vendor (tagged) git's / release tarballs are available then it should be easy to package them 15:38:12 (hi ;D) 15:38:15 markmcclain, salv-orlando but there’s lot more 15:38:21 agree. To allow distros to package plugins, we (vendors and neturon team) need to clarify version dependencies. 15:38:23 so frankly I am not too worried 15:39:08 but I think it is not a big problem and during clarifying the process it will be addressed. 15:39:11 ok if you have no reason to be worried I have no reason as well. 15:39:13 markmcclain: The visibility thing can be handled by us publicizing the places where the drivers go. That part is an education thing as much as a visibility thing. Maybe I'm simplifying it though. :) 15:39:30 yeah.. just want to make sure that is part of the solution 15:39:30 salv-orlando: I am just saying that even if the source is packaged today, it does not mean it’s usable 15:39:40 tested or validated on any distro 15:39:48 also notice that we don't require the drivers to be apache2 license compatible 15:39:52 distros need a relationship with the vendor 15:40:07 to go beyond just packaging 15:40:11 markmcclain: Can we enforce that if htey are out of tree (licenses)? The parts which live in neutron need to be apache2 though. 15:40:33 But since the code is imported, do we need to enforce apache2? 15:40:50 mestery: I wouldn’t think so 15:40:52 line 255 would require it 15:41:05 I think plugin maintainers (they’re not only “vendors”) will need to learn how to manage the lifecycle of a package 15:41:05 * mestery looks 15:41:05 for the vendor library, I mean 15:41:20 the vendor integration park, being in three is apache2 15:41:29 *part 15:41:45 * salv-orlando will probably look for somebody to this do for me, as it’s not really one of things I want to learn 15:41:50 markmcclain: Yeah, so parts of it are apache2, we should add that in the spec. 15:41:54 * armax thinks about greenery and the beauty of nature 15:42:03 * mestery thinks about coffee 15:42:06 the shim in the tree is obviously apache2, but any deps for the shim must be compatible since that is a dependency 15:42:16 markmcclain: Makes sense 15:42:21 armax: greenery like you’re rolling some weed? 15:42:23 it's the battle we fight w/ global requirements 15:42:26 lol 15:42:30 markmcclain: same considerations apply to any library we use in openstack 15:42:49 correct, but we should restate is here 15:42:49 markmcclain: but I’ll make the point, it seemed obvious but being explicit does not hurt 15:42:54 armax: Agreed, but maybe we should call it out in this spec explicitly. 15:43:00 yeah 15:43:01 fair 15:43:12 Excellent, agreement! :) 15:43:25 * mestery thinks it's the greenery ;) 15:43:32 explict about licencing +1 15:43:36 I do think that anyone that gets a shim must have the source posted somewhere open 15:43:38 +1 15:44:02 markmcclain: I'm somewhat split on that one, I can see arguments both ways, so I'm meh either way. :) 15:44:05 markmcclain: I see no reason why it shouldn't 15:44:15 But it should be documented (if it's not) on what we agree on. 15:44:19 well we're still giving them a hook 15:44:32 they have to trade something and having code locked in corporate repo is bad 15:44:33 * mestery is good with requiring source 15:44:39 markmcclain: but ultimately it’s a choice we have no control of 15:44:48 we do have control on the shim 15:44:56 Yes, we can control it by landing hteir shim 15:44:58 or not landing it 15:45:04 right 15:45:14 markmcclain: I’ll be more explciit that accessibility to the source is an important criterion for assessing how friendly a vendor is 15:45:32 ideally I'd like that code to exist in stackforge and follow our community model 15:45:36 friendly for lack of better word 15:45:45 so don’t quote me on that 15:45:48 please 15:45:49 :) 15:45:56 folks, I think the idea of having the shim in neutron and the plugin library private is just demoniac 15:45:59 diabolic 15:46:06 stackforge is ideal, but if they have it on github, that is ok as well 15:46:09 salv-orlando: someone will try it 15:46:26 markmcclain: and we’ll go medieval on them ;) 15:46:28 someone always tries something 15:46:35 salv-orlando: that is tiring 15:46:41 best to just say no when it happens 15:46:47 markmcclain, salv-orlando: I would cross that bridge when we get to it 15:46:50 honestly 15:47:04 I think explicitely putting it in the spec makes it clear, so lets just put it in there 15:47:08 And face the pushback early rather than later 15:47:09 mestery: sure 15:47:12 Cool 15:47:19 Thanks armax 15:47:21 Is a requirement for shim plugin openness of their library ? 15:47:30 or is it just a suggestion? 15:47:36 amotoki: I think we're saying make it a requirement 15:47:38 I think it’s already expressed int he spec 15:47:43 I think requirement 15:47:44 but I’ll build a stronger case for it 15:47:48 armax: Cool 15:47:49 mestery: agree 15:47:50 I think the requirement made here is that you don’t import a package which is not available in the open 15:48:00 salv-orlando: no 15:48:00 and frankly I think you need that for unit test don’t you? 15:48:02 salv-orlando: +1 15:48:11 armax: you’re the voice of no... 15:48:21 what is the requirment then? 15:48:59 sorry I mean, the package must be accessible somewhere 15:49:01 anyway, this is only my first question… I also have a 2nd one ;) 15:49:06 lol 15:49:17 so, yes :) 15:49:26 accessible like source code available in the open (where in the open == public) 15:49:38 can I go with the 2nd question? 15:49:40 you guys are confusing me…right that’s what the spec states right now 15:49:56 salv-orlando: but it does not make it a strong requirement for acceptance of the shim in the tree 15:49:57 armax: I think the spec suggests but does not mandate 15:50:02 markmcclain: correct 15:50:14 I think a 'shall … ' is in order here 15:50:19 so are we saying that the code must rather than shoud? 15:50:21 We're mandating I think, right? 15:50:23 should? 15:50:36 mestery: ok 15:50:37 will 15:50:42 let’s go with that 15:50:49 must/shall rather may/should 15:51:13 #info Update the spec to indicate any shim plugins/drivers in the tree require their out-of-tree code to be open source on something like stackforge or github 15:51:31 is this sorted then? 15:51:31 markmcclain: ok 15:51:35 salv-orlando: yes 15:51:45 what was Salv's question? 15:51:51 the second question is: how shimmy should the shim be? 15:51:58 salv-orlando: thou shan't make the code private 15:52:03 salv-orlando: does that work? 15:52:09 :) 15:52:13 armax: lol 15:52:18 you went from legalese to biblical 15:52:26 haha 15:52:27 salv-orlando: I know, right? 15:52:51 salv-orlando: now you’re becoming even more pedantic than what you already are 15:53:10 salv-orlando: about the shimminess 15:53:16 * armax thinks it’s not even a word 15:53:22 I think the thinkess of the shim is dependent on the plugin refactor 15:53:26 salv-orlando: I think that the shimmier it is the better 15:53:34 if we do our job right we should be able to make it really thin 15:53:52 The thinner the better IMHO 15:53:57 salv-orlando: hopefully some initial effort will pave the way for ‘coding by example’ 15:54:04 dougwig had an example of a LBaaS V2 driver which was super shimmy 15:54:04 should we evaluate on a case by case basis? What if some plugin already in trunk offers a diet plan in say two or three phases? 15:54:11 in many ways the contrail plugin is closer to want we eventually want 15:54:32 salv-orlando: I think a phased approach should be allowed 15:54:32 The spec makes it clear the diet has to happen during the kilo cycle. 15:54:38 Do we want to change that? 15:54:43 we do have to protect the operators 15:54:49 It needs an end date. 15:54:56 Or else some will drag it out to the bitter end. 15:54:59 ok 15:55:07 the other point about the contrail plugin 15:55:07 it will always drag out to the bitter end :) 15:55:19 this means worst case one could do a proxy to a plugin living somewhere else? 15:55:19 lol 15:55:32 markmcclain: in the spec I set a date, which is suggested to be assessed at every mileston 15:55:33 e 15:55:40 to see whether we are on track with the plan 15:55:46 so there’s some wiggle room 15:55:50 salv-orlando: yes.. a the shim could be proxy 15:55:55 I estimate 4 weeks full time for instance on NSX plugin 15:56:06 which considering how much time I can spend on that means 4 months 15:56:12 but we need to be clear that procrastination is not an option 15:56:15 and I think others will be in the same situation 15:56:22 the thing to remember is I had a few folks say that 6mos was not enough time to enable IPv6 15:56:24 And so we double that for others, meaning 8 weeks for non salv-orlando maintainers ;) 15:56:30 salv-orlando: Does 4 weeks include CI or other setups? 15:56:33 frankly in my case I’d rather accept deprecation that ask for an extension 15:56:33 and that spec has been around 20 yrs :p 15:56:35 markmcclain: Right, thus multi-cycle is needed 15:56:36 if no tangible progress is observed I think we should reserve the right to judge whether the deprecation appleis or not 15:56:38 4 weeks code only 15:56:53 armax: ++ 15:57:11 salv-orlando: I feel the similar too 15:57:17 ok so we will judge on a case by case basis 15:57:18 salv-orlando: you’d do anything to save you from working 15:57:20 :) 15:57:33 Wow, we made it through salv-orlando's questions in < 30 minutes! :) 15:57:51 mestery: yay 15:57:51 it’s not a case if my job title is “distinguished procrastinator" 15:57:59 salv-orlando: oh right, I forgot 15:58:10 salv-orlando: I shan’t forget that, ever 15:58:12 Anything else then? 15:58:19 armax: what is your benchmark for "tangilbe progress"? 15:58:20 Otherwise, armax will update the spec 15:58:23 And we'll review again. 15:58:25 i have two questions: destiny of linux bridge driver, and common driver code in ML2 stuff. 15:58:26 I think I will soon get the Homer Simpson fellowship 15:58:38 anteaya: some progress on the plan outlined on the work item section 15:58:56 amotoki: that was raised on the spec, and it’s been addressed 15:59:08 armax: will check 15:59:09 okay, this will be challenging to assess, since your definition and someone else's will differ 15:59:09 amotoki: let me know if that answer your concerns 15:59:10 anteaya: it’s a situation where it’s really hard to define objective metrics inmho 15:59:15 otherwsie we’ll take it offline 15:59:18 armax: I do think we need a work item for a realistic roadmap for vendors to follow 15:59:27 anteaya: no, if the plan is clear and well detailed 15:59:30 imo 15:59:33 salv-orlando: oh I agree, you can avoid the fight, you can just define the terms 15:59:40 can't 15:59:48 one of the current problems is that we don't provide a happy path for vendors 15:59:58 which leads to the poor impls 16:00:08 markmcclain: It needs to be laid out in detail for them to follow, agreed. 16:00:21 I mean, if no commits have been posted to refactor the code according the proposed structure 16:00:31 commits, there is a metric 16:00:35 merged commits? 16:00:36 it’s pretty obvious to me that no tangible procgress has been made 16:00:38 :) 16:00:48 Well, and if no one has reached out asking questions, etc. 16:00:50 that's another sign 16:00:57 questions, another metric 16:01:02 anteaya: Right. 16:01:09 great, two so far, thanks 16:01:19 ok I’ll cover this area a bit more 16:01:22 honestly, these are metrics we should already have to some extent, we can collect from prior releases to get a feel for what to expect. 16:01:27 Spoiler: It won't be pretty in some cases. 16:01:59 markmcclain: how can we sprinke happiness on vendors? 16:02:04 ha ha ha 16:02:06 I’m a rather grumpy one myself 16:02:16 get them off windows 16:02:17 salv-orlando: donuts 16:02:23 salv-orlando: kittens and puppies 16:03:06 ok, so for every plugin which is moved out into a library I’ll donate $100 to an animal shelter. does that work? 16:03:14 ha ha ha 16:03:17 rofl 16:03:28 haha 16:03:35 and mestery will send a box of 12 donuts - for each developer 16:03:35 salv-orlando: okay let me set up a charity about this 16:03:52 salv-orlando: I’ll send you the bank coordinates 16:03:53 I can add a link to a linux distro download of choice 16:03:53 With sprinkles. Sprinkles make everything better. 16:04:04 :) 16:04:30 seriously. We probably want to have an “ideal story” - both for maintainters of plugin already in trunk and for maintainers of plugins not yet in trunk 16:04:39 right 16:04:46 ideal being? 16:04:55 we need a pioneer and extensively doc what we're doing 16:05:00 like you want to write a novel or something? 16:05:11 Not only that, but I think we need to make ourselves extra available to help people, on IRC, ML, meetings, etc. 16:05:16 armax: more of a checklist that jr dev could follow of steps 16:05:17 markmcclain: oh right, it looks like kevinbenton should help us spearhead the effort 16:05:22 In other words, go out of our way to make this process smooth for people. 16:05:54 I'm not sure how much more extra people have in them 16:05:59 I know I'm all used up 16:06:20 We'll spread the extra around and see how much is left in our collective tanks 16:06:31 markmcclain: hopefully maintainers of vendor code are somewhat more than jr devs 16:06:32 great, I hope someone has some 16:06:34 Since we're moving people out of tree, we need to accomdate them during hte process 16:06:39 armax: I might become a blogger! That would be the apotheosis of the procrastinator 16:06:40 armax: experience says otherwise 16:06:45 otherwise we’d be in trouble with our without this proposal 16:06:46 armax: the ci operators aren't 16:06:57 * mestery shudders at the ci operators 16:07:01 *though of the ci operators 16:07:04 *thought 16:07:07 * mestery drinks more coffee 16:07:08 sorry about that 16:07:10 anteaya: operators should not be affected by this effort at all 16:07:15 didn't meant to invoke that 16:07:18 if they did then we’d failed miserably 16:07:26 armax: shouldn't and are, yet to be seen 16:07:42 and honestly there are a lot of “junior” devs which are way better than me. So it’s a matter of how flexible and adaptable you are, not experience 16:07:56 anteaya: right, but we’re aware that disruption for operator is not an option 16:08:12 for their ci operations? 16:08:30 they will have to change their pointers for git clones at teh very least 16:08:37 if they have devstack support 16:08:37 some having difficulty with that 16:08:39 Folks, I think we're digressing a bit here. 16:08:43 but don't mean to derail 16:08:45 sorry 16:08:49 And we're 8 minutes over my ending time :) 16:08:53 anteaya: if it was just that I’d sign up for that 16:08:55 Or proposed ending time :) 16:09:06 Any other issues with the BP to bring up now? 16:09:09 armax: I'd welcome you 16:09:14 Or should we let armax roll another version and then re-review that one? 16:09:14 I do have a questions on security handling, but I'll post those to the spec 16:09:22 markmcclain: Cool 16:09:30 markmcclain: we covered that part 16:09:45 I want to think through stable updates, specifically Juno/Kilo backports, etc. 16:09:46 markmcclain: let us know if it’s too sparse for you 16:09:49 since it’s been a quiet meeting so far 16:09:54 right, but I have questions about what's covered 16:09:58 let me try and drop a bombshell 16:10:05 * mestery gets his kevlar on 16:10:07 ML2 16:10:11 BOOM! 16:10:12 salv-orlando: drop away 16:10:30 devstack integration is one of questions. in-tree devstack support is acceptable after splitting out? this is beyond the spec, but we need to clarify. 16:10:34 salv-orlando: is there anything that wasn’t covered in the spec that deserve the bomb shelling? 16:10:35 I just did. What do we do with it? I think it’s unfair to give it special treatment and leave it in the tree 16:10:55 salv-orlando: we need a gating reference implementation 16:11:01 salv-orlando: that was documented 16:11:07 not sure what else to add to that 16:11:12 you are speaking legales again. 16:11:18 I know it’s documented. I think it’s unfair. 16:11:19 amotoki: devstack integration has been documented 16:11:31 salv-orlando: unfair on what grounds? 16:11:34 I think the gating requirement is what's keeping it there 16:11:49 on the ground that we grant ML2 and all its drivers the right of staying in the tree 16:11:55 I am not bothered 16:11:56 salv-orlando: you open source nsx and we’d be happy to switch 16:11:58 Wait, all the ML2 drivers are staying in tree? 16:12:03 I missed that part 16:12:09 I thought only the OVS one was staying in-tree (for now)? 16:12:14 * mestery has to re-read that part 16:12:22 armax: the speci mentions just the existing devstack support. my point is future plan (new vendor support). 16:12:24 armax: you just said somethign rather non intelligent 16:12:37 ml2 has plenty of drivers for proprietary 3rd parties 16:12:43 salv-orlando: sometimes I have brainfarts too 16:12:57 salv-orlando: those will get slimmed down too 16:13:07 salv-orlando: as we proposed 16:13:23 what about the comment we received that you can’t split ml2 from its drivers because they work so nicely together? 16:13:30 ok.. linuxbridge has to stay in w/OVS 16:13:41 markmcclain: then we should start gating on it 16:13:43 there is a group of folks who don't want OVS 16:13:49 armax: yes we should 16:13:55 ok 16:14:06 markmcclain: until then we can’t worry bout LB, IMO 16:14:13 salv-orlando: I don’t follow 16:14:32 #info To keep LB in-tree we need to start gating on it. 16:14:33 folks deploy it into production 16:14:40 so we should be gating on it 16:14:42 armax: that was live at the summit - I think it was bob? 16:14:51 and perhaps we’d need to get LB to parity, like DVR perhaps? 16:15:03 armax: ++ 16:15:09 salv-orlando: we’re not advocating to splitting drivers right now 16:15:17 we’re advocating to slim vendor code down 16:15:33 I thought that meant removing drivers 16:15:40 salv-orlando: we’d need to reassess what makes sense when some progress has been done 16:15:46 salv-orlando: no it doesn’t 16:16:01 armax: then until further notice ML2 is not affected by this blueprint at all. 16:16:06 the same way we’re not proposing to remove vendor plugins 16:16:21 This is the first step in that direction, right armax? 16:16:24 OR it coudl be I guess 16:16:51 * markmcclain wonders if this will have unintended effect of vendors going ML2 to stay in tree 16:17:00 Yikes 16:17:06 theoretically *new* ml2 drivers can be completely out of tree. 16:17:09 guys, no vendor gets out of the tree with this proposal 16:17:24 neither ml2 driversnor core plguins 16:17:24 but some drivers already have db models in neutron tree :-( 16:17:31 I am amazed as you are still confused about this 16:18:13 where have we gone wrong? 16:18:19 I think armax’s point is that the plugins and the driver stays but they are going on a diet 16:18:33 now he’s defined a 1,200 kcal/day diet for plugins 16:18:49 but this diet does not apply to ML2, because it’s not a plugin like the others 16:18:50 armax: yes. even after the split, we still have thin drivers/plugin in the tree.. 16:18:57 let’s not forget all the virtues of ML2 16:19:02 salv-orlando: it does apply across the boar 16:19:03 d 16:19:14 bu armax will then define a plan also for putting ML2 drivers on a diet, right? 16:19:33 The spec references ML2 and monolithic plugins 16:19:36 but if some ml2 drivers are already skinny, then there’s nothing that can be done 16:19:37 is there? 16:19:45 ok so ML2 will slim down too? 16:19:54 in theory ML2 drivers should be small :) 16:19:56 that’s what the spec state 16:19:57 salv-orlando: Yes, that was my reading of the spec 16:19:58 I thought it was excluded because of co-gating requiremtns? 16:20:05 salv-orlando: no 16:20:08 so the one who’s confused is me 16:20:18 who said co-gating and reference implementaiton? 16:20:21 was it markmcclain 16:20:27 right 16:20:52 and that’s true but one of the items of the spec is that even the reference implemetnation will go through the same refactoring exercise 16:21:23 even though there’s no point of moving the code elsewhere, at least for the time being 16:21:38 this is black and white on the spec 16:22:20 FYI: 8 minutes left 16:22:22 as a conclusion, does the split out affect ml2 drivers? 16:22:30 I’ll revise the spec one more time to make sure there’s no room for confusion 16:22:53 amotoki: I think the we have to a clear plan for ml2 16:23:11 I have a feeling the “shimming down” for ML2 will not be as obvious as the one for monolithic plugins 16:23:33 but this is just a feeling. You’ll buy me a beer in vancouver if it turns out I was right 16:23:34 there’s a point about this on the spec I am trying to find the line no :) 16:23:38 Note, the APIC ML2 driver went through a "thinning down" at the end of Juno already, that could be an example for folks. 16:23:48 * mestery can't believe he just said use the APIC driver as an example for folks. 16:23:53 yeah that might be harder, but then again maybe designing better interface contracts would slim it down 16:23:54 but anyhow I’ll clear this up, to make it salv-orlando proof 16:24:27 Anyways, at this point, lets let armax spin a new version and continue comments there. 16:24:36 And if there's nothing else right now, shall we close this jolly affair? 16:24:42 :) 16:25:06 line 124 btw 16:25:36 The monolithic plugins and ML2 drivers become integration-only to code that lives outside the tree; 16:25:46 just saying 16:25:48 Thanks armax, and thanks to everyone, I think this was a productive meeting (at least for me). ;) 16:26:03 Looking forward to the new spec rev armax ;) 16:26:21 #endmeeting