21:02:04 #startmeeting networking 21:02:05 o/ 21:02:05 Meeting started Mon Oct 13 21:02:04 2014 UTC and is due to finish in 60 minutes. The chair is mestery. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:02:06 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:02:08 The meeting name has been set to 'networking' 21:02:13 #link https://wiki.openstack.org/wiki/Network/Meetings Agenda 21:02:19 #topic Announcements 21:02:31 We released our second RC for Juno last week 21:02:33 #link https://launchpad.net/neutron/+milestone/juno-rc2 21:02:37 #link https://launchpad.net/neutron/juno/juno-rc2/+download/neutron-2014.2.rc2.tar.gz 21:02:45 Thanks to all who helped with bugs near the end! 21:03:00 At this point, I'm not aware of any RC issues which would cause a third RC to be released. 21:03:08 If anyone knows of anyone, lets discuss during the bugs portion of the meeting. 21:03:26 I would encourage folks to test out the RC2 release as it's likely to be the official 2014.2 release of Neutron. 21:03:49 If anyone finds a release critical bug, please tag it with "juno-rc-potential" and make sure you email me or find me on IRC so I'm aware. 21:04:00 Any questions on the state of Neutron in Juno? 21:04:43 I also wanted to highlight the release notes for Neutron are being worked on actively now as well. 21:04:44 #link https://wiki.openstack.org/wiki/ReleaseNotes/Juno#OpenStack_Network_Service_.28Neutron.29 21:04:55 Feel free to suggest updates there as well. 21:05:22 One final announcement: I've changed the format of the weekly Neutron meeting, per email to the list today: 21:05:23 #link http://lists.openstack.org/pipermail/openstack-dev/2014-October/048328.html 21:05:38 The hope is this allows for more collaboration rather than a simple status report each week. 21:05:51 +100 21:05:55 Thanks to carl_baldwin for updating a lot of wiki pages today on this front! 21:06:01 +100 21:06:38 OK, lets move along then. 21:06:42 #topic Bugs 21:06:46 enikanorov_: Here? 21:07:25 I've been focused on RC candidate bugs for the past week, if anyone has any updates on other bugs which should be on our radar, please jump in here. 21:08:16 hi : lot of dicussions on this one : https://bugs.launchpad.net/neutron/+bug/1361540 21:08:17 Launchpad bug 1361540 in neutron "no mechanism driver calls for gateway port removal" [High,In progress] 21:08:31 mestery: How come all the ones with juno-rc-potential bugs did not make rc2? 21:08:32 matrohon: Yes, we had spoken about this, and I had chatted with markmcclain the past Friday on this one. 21:08:57 Sukhdev: A variety of reasons: Not being a release blocker, not being merged in master are the most likely ones. 21:09:04 #link https://bugs.launchpad.net/neutron/+bug/1361540 21:09:44 matrohon: Are you still of the opinion that 1361540 is an RC candidate bug given our discussion last week? 21:10:20 mestery: still would like to better understand armax's concern on that one 21:10:37 mestery: this is just the tip of the iceberg 21:10:41 seems a simplified fix would be better 21:10:56 the whole external gateway code doesn't handle state on deletes correctly 21:11:14 mestery : this has backend side effects, garyk seems to agree that this is a huge bg 21:11:28 the order of operations is not correctly reversed before new state is applied - we've had to code around this issue internally 21:11:34 regXboi: Is there another bug filed for this? I imagine you discovered this while looking at code recently? 21:11:54 markmcclain :i've tried to do it as simple as I could! 21:12:14 mestery: no on the bug filing - I've been trying to boil it down to the minimal fix needed 21:12:39 regXboi: Ack. Can you file one to track the issue, especially if it affects 1361540 which is under review now? 21:12:48 and if by recently, you mean in this cycle, then yes 21:12:48 Isn’t the fix here to do deletion by calling the plugin? 21:13:06 mestery: give me an action and I'll file something 21:13:30 #action rkukura to file bug for external gw code deletions 21:13:33 #undo 21:13:34 Removing item from minutes: 21:13:41 #action regXboi to file bug for external gw code deletions 21:13:57 mestery: ack 21:14:37 what about https://bugs.launchpad.net/neutron/+bug/1378525 ? 21:14:38 Launchpad bug 1378525 in neutron "Broken L3 HA migration should be blocked" [High,Fix committed] 21:14:47 woops wrong link. 21:14:48 amotoki: Looking 21:14:53 * mestery waits 21:15:05 rkukura : markmcclain's cleanup previous patch introduced more calls to _delete_port, bypassing core_plugin 21:15:30 matrohon: sounds like this needs to be fixed 21:15:34 rkukura: as I remember it, the issue is one of state 21:15:56 because the linear algebra of the operations is wrong 21:15:57 mestery: the link is correct. 21:16:16 mestery: The bug proposes to prohibit L3 HA attribute update 21:17:00 mestery: +1 https://review.openstack.org/#/c/126706/ if RC3 opens 21:17:04 it is an API change and worth considered for RC. 21:17:05 amotoki: This seems like a candidate, yes, but I don't know if it's a release blocker itself. 21:17:10 markmcclain: Looking 21:17:22 mestery: but it's not a release blocker 21:17:35 I think that we can release note that there is an error in the API there 21:17:35 markmcclain: Exactly. If we find something else, I'll cherry-pick that one in. 21:17:41 agree it is not a release blocker. 21:17:41 markmcclain: Ack. 21:17:48 we did that previously 21:18:08 #action mestery to add release note for https://bugs.launchpad.net/neutron/+bug/1378525 indicating the error with the API. 21:18:10 Launchpad bug 1378525 in neutron "Broken L3 HA migration should be blocked" [High,Fix committed] 21:18:15 Any other bugs for discussion? 21:18:19 pleasa have a look to : https://bugs.launchpad.net/neutron/+bug/1372438 21:18:21 Launchpad bug 1372438 in neutron "Race condition in l2pop drops tunnels" [Medium,In progress] 21:18:59 the patch is ready, it re introduce a previous merged patch, deleted by DVR stuff 21:19:19 https://review.openstack.org/#/c/123403/ 21:19:27 matrohon: Lets get that one into master by tomorrow so if we do roll an RC3 I can cherry-pick it back. 21:19:45 Looks like carl_baldwin has some issues with that patch at the moment from looking at the review. 21:20:23 Mostly surprised that no UTs are affected. 21:20:58 These are all good fixes, but I don't think they are release blockers at the moment. 21:21:01 * marun is not so surprised 21:21:02 mestery : thanks, e and vivek are trying to addrees the UT issue, but i'ts really difficult to produce race conditions with UT 21:21:11 matrohon: Ack 21:21:33 marun: Not so much surprised as saddened. :’( 21:22:08 Any additional bugs anyone wants to bringup with the broader team here? 21:23:13 #topic Docs 21:23:15 emagana: Hi there! 21:23:16 Hi! 21:23:20 How are Juno docs looking? 21:23:31 not sure how to answer that... 21:23:33 lol 21:23:37 lol 21:23:46 well, I have a broader question 21:24:07 Does Neutron team consider DVR and L3 HA stable features ready for production? 21:24:19 that is a very good question 21:24:49 emagana: Is this in the context of how much we document or simply how we label them? 21:24:52 given the relative lack of comprehensive testing for these rather complicated features, I would be interested to hear arguments suggesting that they are ready for production. 21:24:59 Do you have any concerns. 21:25:07 Ok, that was the question from the Docs Folks 21:25:33 The plan is to document what is stable and ready for production 21:25:47 in the installation guide which is one of the top ones 21:26:20 So, the point is these features will not be covered in the installation guide but will be covered in the networking guide 21:26:23 emagana: we can help the documentation effort for the dvr. 21:26:46 emagana: I think that sounds like a fair assessment. 21:26:52 Swami: Does that sound ok to you? 21:27:05 Swami: Thanks! I count with that help, but the question here is if NEUTRON team recommends to use these features yet?? 21:27:20 I need to understand what is blocking us from documeting or stating that the dvr is not production ready. 21:27:26 I think you can guess at my vote 21:27:33 Swami: how about some testing that it works? 21:27:59 marun: what kind of test results are you expecting. 21:28:05 Are there any known production deployments? 21:28:14 Swami: Dont take this wrong but our experience with shinny good looking new features is not the best one once a bigger community start testing them 21:28:22 Swami: I'm expecting test plans for all major use cases and implementations of those test plans. 21:28:29 carl_baldwin: none that I know of 21:28:40 carl_baldwin: +1 to markmcclain's comments, especially given how new it is. 21:28:55 carl_baldwin: for Neutron many! 21:29:10 I don’t even know what we are talking about. 21:29:15 emagana: I meant deployments that use either new feature. 21:29:18 carl_baldwin: using those features obviously none but they will consider to use them if we declare them stable enough 21:29:46 we’ve never ever in our craziest days suggested a feature or component was production ready in the 1st release where it appeared 21:29:50 I want the team to vote how to declare those features 21:29:53 salv-orlando++ 21:29:57 mestery: What is your opinion? 21:29:57 perhaps a chicken and egg quesion, how will people start using a feature if its not made available in the docs? 21:29:59 At least I think dvr and l3-ha are new featrues and they are not mature compared to other core plugins. 21:30:11 I think it's hard to call a brand new feature production ready out of the gate. 21:30:20 Now, we expect people to use it and deploy it and report issues. 21:30:21 +1 21:30:27 OK, The proposal is this: 21:30:28 But if no one is using it, it's hard to say it's production quality 21:30:31 My $0.02 21:30:36 folks - are we discussing if they are production ready or if they should be documented? 21:30:36 emagana: unless we've seen considerable usage from customers (and responded to the issues involved), or we have comprehensive testing, of course it isn't production-ready yet. 21:30:39 SumitNaiksatam: yes I agree. 21:30:46 SumitNaiksatam: It will be in the networking guide, just not the install guide. 21:30:48 to me both questions are no brainers honestly 21:30:56 mestery: ah okay 21:31:12 salv-orlando: A voice of sanity :) 21:31:25 DVR is not production ready and needs to be documented. 21:31:26 Declare that these features are new and certain amount of external testing is needed (This is going to be in the Install Guide) 21:31:27 Do people agree? 21:31:35 On the networking guide we will document them properly 21:31:37 ++ 21:31:41 emagana: +1 21:31:42 Is that the gist of the argument here? 21:31:45 emagana: +1 21:31:48 emagana: +1 21:31:53 +1 21:32:01 so we agree to say it’s production ready and not to document it, right? 21:32:10 * mestery throws tomatoes at kevinbenton. 21:32:12 :P 21:32:14 +1 21:32:17 kevinbenton: +1 ;) 21:32:27 emagana: +1 21:32:38 * pc_m incubated feature? proof of concept? 21:32:41 so, install guide will be ready :-) 21:32:55 Too bad we don’t have a preview subtree ;) 21:33:07 Networking guide will take more time and will be completed during kilo release 21:33:22 emagana: I figured as much there. :) 21:33:25 kevinbenton: not quite. I heard: it's not production ready, it will be mentioned in install guide as shiny new, and will be documented in networking guide. Right? 21:33:48 glebo: +1 21:34:00 glebo: +1 that is what I heard too. 21:34:19 my "right" was meant for emagana 21:34:24 emagana: right? 21:34:25 mestery emagana: Can you summarize the conclusion? 21:34:47 Sukhdev: See glebo's response above. 21:34:48 cut n paste my @kevinbenton 21:35:01 emagana: An exciting doc's update this week! 21:35:01 *cross talk* 21:35:05 Sukhdev: Install guide will mention that DVR and L3 HA are new features and more testing is required to be declared stable 21:35:21 emagana: Thanks 21:35:29 Sukhdev: Networking guide will describe both features 21:36:15 install docs have always trailed the release of production features, so I think that emagana's proposal is following what we've done for cycles now 21:36:19 If somebody wants to be part of the Networking-Docs discussions we meet every Frydat 9:00 pst 21:36:26 emagana: So, will I have enough documentation, if I wanted to play with these features and try them out, right? 21:36:26 emagana: and probably ought to mention that they are making their big debut 21:36:54 glebo: Yes! 21:37:11 markmcclain: ++ 21:37:15 Sukhdev: Not right away, ENtworking guide will take more time. All focus are on Install guide and Admin Guide 21:37:22 ENtworking = Networking 21:37:54 emagana: Any other doc updates for us today? 21:38:04 mestery wants to suffer more today! 21:38:10 emagana: :P 21:38:11 mestery: No more! 21:38:13 Thanks sir! 21:38:26 #topic On-Demand Agenda 21:38:37 So, per the new meeting format, we'll walk through today's On-Demand agenda 21:38:49 First up, a new peer review process for core reviewers has been proposed: 21:38:50 #link https://etherpad.openstack.org/p/neutron-peer-review 21:39:06 Please see the etherpad page for the goals behind this process and for the format of the review process. 21:39:37 We're going to try this out in Kilo and see how it goes. 21:40:12 * mestery notes everyone is likely reading now. 21:40:31 so I see two proposals on that page 21:40:42 *which one* will be tried in Kilo, or both :) 21:41:10 regXboi: I think that's up to the team to decide. 21:41:21 regXboi: What marun said. 21:41:42 ok, that wasn't 100% clear from mestery's text 21:41:48 regXboi: It's a proposal, not a fait accompli, and everyone's input will be considered before we go forward. 21:42:31 mestery: Is this process new for all OpenStack? 21:42:38 mestery: Good write up and makes a lot of sense 21:42:39 emagana: Neutron only at this point. 21:42:45 emagana: No, we're the first ones to try it. :) 21:43:07 mestery: My opinion is that brings a lot of overload to the review process but I could be understanding it wrong! 21:43:39 emagana: That was my concern as well, but the idea is to collect relevant data around Neutron cores and provide valid feedback to people. 21:43:48 emagana: We definitely don't want to add process for its own sake, so the goal is balancing the benefits with the cost. 21:43:53 emagana: Hopefuilly we'll try this out and adjust accordingly. 21:44:13 while reading it, the first thing I thought of was a reputation system that people interacting with a reviewer can submit, with # of stars out of 5, and comments section, either with or without name attached. Not unlike rating an app in app store, or a seller in ebay 21:44:18 emagana: If you have suggestions for how to achieve some of the stated goals differently, that would be great to hear about. 21:44:39 glebo: Lets hope contributors don't think of core reviewers as "apps in app store" :P 21:44:40 emagana: Also, the alternate proposal included might be a lower-cost way of achieving a similar result. 21:44:45 mestery: I like better the second proposal but removing the anonymously part. We are all professional who can take feedback properly 21:44:54 emagana: I think you're wrong on that point. 21:45:02 mestery: lmao! 21:45:37 mestery: heavens, no! just trying to think of a very quick, painless way to begin collecting interaction-based data 21:45:53 Besides, if I am not performing to the expected level, there should be a proper enhancement plan and if after that things dont change then bye bye 21:46:00 emagana: It's a common phenomenon that people are nice in public because they don't want to be mean or hurt someone's feelings. Anon feedback is a way of leveling things. 21:46:07 Folks, I see an attempt is being made to make the process work better - but, have one concern/issue regarding the survey 21:46:18 mestery: I want to be candy crush saga 21:46:27 salv-orlando: haha 21:46:36 emagana: if history has shown us anything, it's that humans are absolutely not consistently "professionals who can take feedback properly", though I wish it were so 21:46:37 salv-orlando :_) 21:46:41 salv-orlando: That's a good choice. :P 21:46:42 emagana: that is the intention here, to provide avenues for improvement if someone is not seen to be meeting consensus expectation. 21:47:11 emagana: and if any of us don't want to have to meet the expectations of our peers, it's probably the case that we don't want to be on core anymore. 21:47:27 Sukhdev: do tell! 21:47:46 salv-orlando: ++ *lmao again* 21:48:03 marun: if it’s anonymous and only a subset of the cores, it makes their voice seem over-representative 21:48:37 Folks may have different openions about different core based upon their interactions - I may find one core very useful (in one situation to me) and may not find other core that useful - hence, my survey may be very skewed 21:48:47 and this will be true for most part, no? 21:48:56 I think the first iteration of this survey is more for assessigne the survey itself. 21:49:05 Sukhdev: That is certainly a fair point. 21:49:07 salv-orlando: Yes, we're surveying the survey. 21:49:14 salv-orlando: yeah and I think that is the right approach 21:49:17 once we address feedback and improve it, then the data set will be expanded 21:49:20 mestery: So, core status is only attached to core reviews. Any contribution to community staff such as testing, mailing lists, docs is not considered? 21:49:26 salv-orlando: +1 21:49:29 emagana: erm 21:49:38 emagana: That's in there (at least I recall it being there). 21:49:39 emagana: the survey questions should shed light on your question 21:49:43 * mestery goes to look 21:49:46 emagana: What marun said. 21:50:07 emagana: they encompass more than reviews, because core responsibility is definitely more than that. 21:50:12 So, my suggestion will be that cores should review cores - based upon their involvement in various parts…..not the general community 21:50:16 marun: Go it! 21:50:30 Sukhdev: That's the intent. 21:50:31 Sukhdev: i thought that was already the intention 21:51:04 kevinbenton: The proposal is that the PTL will be responsible for choosing who reviews who, at least initially. 21:51:06 As with most things, once we implement this, we'll iterate a bit and see how it goes, adjusting as necessary. 21:51:18 emagana has one thing to make him sleepless! 21:51:27 Sukhdev: if one of cores main roles is to do reviews for the main community, then the community is a HUGE customer base for the cores. And shouldn't the customer base of that sort have voice to the service being provided? 21:51:31 kevinbenton: It's unlikely that Kyle will assign reviewers where conflict or friction is present. 21:51:32 marun kevinbenton: thanks for clarifying that - I probably missed that :-) 21:51:44 I'll make sure to wave my magic PTL wand gently. 21:52:01 kevinbenton: It's not perfect, but it's a starting point. The goal here is useful feedback more than performance assessment anyway. 21:52:07 marun: a bad pairing could ruin the perspective of the entire community for a core and cause them to burn out, so we need to be really careful here 21:52:21 kevinbenton: I trust the PTL to be careful, as you say. 21:52:50 I'd like to encourage folks to add notes to the etherpad 21:52:51 kevinbenton: and there is always the option of following up with a reviewer who is being really mean, without the person under review being involved or the results being public 21:53:01 marun: if useful feedback is the goal, then the more feedback in total the better; the larger the sample set, the more accurate it is likely to be 21:53:07 glebo: good point - however, you may be pissed off because he was harsh on your code, while the other person may be very happy, as he found the comment very useful - so, how do you come up with true evaluation? 21:53:11 again it seems the goal is to focus first on making sure the tool is right before 21:53:38 hence a small sample should be ok for the first round… I hate to conduct a large survey to find out we asked the wrong questions 21:53:43 glebo: sure, but we're trying to put an upper-bound on the effort involved initially 21:54:03 glebo: once we've learned what works and what doesn't, but all means, involve more people 21:54:10 marun: +100 21:54:12 glebo: this is documented in the proposal, btw 21:54:36 markmcclain++ 21:54:41 Sukhdev: if the purpose is to "get a good grade" then you it's very tough. If the purpose is to provide lots and lots and lots of very simply, quick data points and to thereby easily be able to see overall trends over time, then that's different 21:55:02 reminder: open, transparent 21:55:02 glebo: The latter goal is definitely paramount. 21:55:09 I think that a lot of folk's concerns here will be resolved once the team decides on the criteria which the reviews are based on ... after this happens a lot of the subjectivity will fade away 21:55:17 marun: me thinks so 2 21:55:23 I'm hopeful the community agrees with the direction here. 21:55:35 I like that we're looking at ways to make us better 21:55:38 marun mestery: Cores are providing very valuable service - last thing I would want to do is to discourage them or demotivate them - so, be careful.. 21:55:48 marun: but the current process being proposed seems to contradict that. Closed, hidden, subjective, selective 21:55:49 markmcclain: ++ 21:55:50 the problem with specific checkboxes to fulfull is that it encourages fitting the model 21:56:16 Sukhdev: why should this discourage or demotivate someoby? 21:56:21 a person might be discouraged from spending a whole cycle fixing a really hard problem if it will come back to bite them in other categories 21:56:24 markmcclain: +1. That's the most important part, and shows very well on the community 21:56:30 Sukhdev: It's a balancing act, for sure. 21:56:34 kevinbenton: Fixing something hard should never come back to bite someone. 21:56:42 glebo: in what way? 21:56:48 kevinbenton: +1 21:56:50 Folks, we have 3 minutes left. 21:56:56 Lets continue this duscssion on the ML and on the etherpad. 21:57:01 And next week in the meeting if need be. 21:57:04 I have one more thing to bringup here 21:57:07 At the close of the meeting. 21:57:08 kevinbenton: cores should be graded on intent as much as effort 21:57:11 mestery: All this process will require more commitment time for core in Neutron.. Are we increasing the minimal expected one? 21:57:29 emagana: No. 21:57:33 kevinbenton: I like to think that we still have common sense. It’s not like we would become stupid bureacrauts that just read a survey’s outcome and take action 21:57:37 kevinbenton: if there is a closed door review by a select few, it might well do so. If everyone who saw, touched, intereacted on that project / issue gave a very quick little rating and a comment or two, it would not b so 21:57:39 Per markmcclain, XML support is going away, and we mean it this time: https://bugs.launchpad.net/neutron/+bug/1380787 21:57:40 Launchpad bug 1380787 in neutron "remove XML support" [High,In progress] 21:57:43 salv-orlando: Once you start to label them or start to create whispers….those could be demotivating gestures…hence, extreme care should be used 21:57:49 kevinbenton: and if you read the questions, I think you'll find that constructive involvement is being evaluated. I'm not sure how anyone would game it anymore than they would 'game' being a good developer. 21:57:50 salv-orlando: paperwork does bad things to people 21:58:15 mestery: Can you set up a minimal commitment time in here: https://wiki.openstack.org/wiki/NeutronCore 21:58:17 marun: gaming was a bad term. ‘change their behavior to balance more than focus' 21:58:25 emagana: I can yes. 21:58:34 kevinbenton: If you think the proposal can be improved, your suggestions are definitely welcome. 21:58:34 #action mestery to look at core time commitments in https://wiki.openstack.org/wiki/NeutronCore 21:58:41 mestery: Hopw much will that be? 21:58:43 marun: if the PTL chooses the reviewers and its deal with discretely, then it's the opposite of open, transparent, etc. 21:58:46 Folks, < 1 minute left. 21:59:03 * mestery notes he's yelling into a wind vortex now and will conclude the meeting. 21:59:11 * regXboi happily waves goodbye to XML 21:59:12 I'm happy to see folks engaged in this peer review process. 21:59:13 mestery: Great discussion - thanks for brining it up 21:59:25 And I hope everyone can see the goals it's trying to solve here. 21:59:28 +1 for dropping XML support 21:59:31 Lets continue the discussion on ML and in channel! 21:59:31 glebo: I'd like to hear your useful alternative. 21:59:32 #endmeeting