21:00:56 #startmeeting Networking 21:00:56 hi 21:00:56 Meeting started Mon Dec 2 21:00:56 2013 UTC and is due to finish in 60 minutes. The chair is markmcclain. Information about MeetBot at http://wiki.debian.org/MeetBot. 21:00:57 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 21:01:00 The meeting name has been set to 'networking' 21:01:11 #link https://wiki.openstack.org/wiki/Network/Meetings 21:01:12 hi! 21:01:18 #topic Announcements/Reminders 21:01:33 #link https://launchpad.net/neutron/+milestone/icehouse-1 21:01:53 The Icehouse-1 deadline is tomorrow end of day 21:02:37 For the most part I-1 will be bug fixes and few features that didn't make Havana 21:02:59 #topic Bugs 21:03:19 We have several critical bugs targeted for I-1 that are open 21:03:21 https://bugs.launchpad.net/neutron/+bugs?search=Search&field.importance=Critical&field.status=New&field.status=Confirmed&field.status=Triaged&field.status=In+Progress 21:03:45 anteaya has gone through and cataloged them 21:03:57 anteaya: around to cover them? 21:04:42 he had some flight trouble as far as I heard on #openstack-neutron 21:04:53 right.. she's flying today, so we'll keep going 21:04:53 *she 21:05:00 she* 21:05:08 her takeoff time was right at the start of this meeting 21:05:14 salv-orlando: want to update on https://bugs.launchpad.net/swift/+bug/1224001 21:05:18 Launchpad bug 1224001 in neutron "test_network_basic_ops fails waiting for network to become available" [High,In progress] 21:05:48 occurrences have been reduced since we reverted a patch for ml2. The root cause is the same as bug 125396 21:05:48 Launchpad bug 125396 in vte "Xorg 100% CPU during scrolling gnome-terminal" [Low,Incomplete] https://launchpad.net/bugs/125396 21:05:56 bug 1253896 21:05:59 Launchpad bug 1253896 in neutron "test_minimum_basic_scenario fails with SSHException: Error reading SSH protocol banner" [High,In progress] https://launchpad.net/bugs/1253896 21:06:52 I posted a patch to the tempest test scenario for the network basic ops scenario that holds off on trying to connect until the info shows up in nova 21:07:26 thanks beagles, I will checkout this patch 21:07:37 beagles: great...thanks 21:07:56 I was mucking around and was desperate to see happy tests 21:08:12 happy tests are a good thing 21:08:17 mlavalle: want to update on https://bugs.launchpad.net/neutron/+bug/1251448 21:08:19 Launchpad bug 1251448 in neutron "BadRequest: Multiple possible networks found, use a Network ID to be more specific. " [High,New] 21:08:37 markmcclain: yes 21:08:59 I am still working on bug #1243726 21:09:01 Launchpad bug 1243726 in neutron "tempest failure: No more IP addresses available on network" [Critical,Confirmed] https://launchpad.net/bugs/1243726 21:09:10 mlavalle: what's the update? 21:09:21 armax: thanks for continuing to work on that that bug 21:09:34 the debugging i've done indicates that a test fails to delete a network because there are ports still busy in the subnet 21:09:39 the failure mode underlies multiple root causes 21:10:17 markmcclain, mlavalle: The cases of this I've looked at seem to result from DB lock timeouts. 21:10:24 mlavalle: I've got something that might be good for you to look at 21:10:36 rkukura: good to know 21:10:38 mlavalle, but I'm waiting for the local tests to run 21:11:03 mlavalle, basically it adds some sanity to sync_state operations... might mitigate a bunch of that stuff 21:11:21 beagles: awesome you and mlavalle can sync up after the meeting 21:11:25 * beagles nods 21:11:54 armax: since there are multiple causes 21:12:03 should we create bugs for the different causes? 21:12:24 so it seems to me we have a race condition 21:12:48 markcclain: good point 21:13:01 I'll do that 21:13:15 the tempest test actually waits to verify that the vm has been deleted before attempting to delete the subnet 21:13:18 armax: thanks! 21:13:52 that's where i am in regards to that bug 21:14:09 mlavalle_: thanks for the update.. hopefully beagles work will help to the bottom of it 21:14:32 i'll ping beagles after the meeting to see how can i help 21:14:39 perfect 21:14:48 All- Any other critical/high priority bugs the team should be tracking this week? 21:15:18 markmcclain: how about https://review.openstack.org/#/c/21946/ ? 21:15:28 This patch is needed for fixing security group issue 21:15:47 nati_ueno: +1 to that patch, that is a critical one. 21:15:55 Akihiro set this as critical, but I'm not sure nova side is merged in I1 21:16:03 so I'm ok to target this for I2 also 21:16:28 nati_ueno: I can review 21946 21:16:31 is there consensus that the way of restoring security groups is to add all those binding attributes? 21:16:55 and I was wondering if there is a nova patch too, but these are things best discussed on gerrit 21:16:55 salv-orlando: There was a consensus, but I'm not sure current status. 21:17:11 please link the nova review in teh comments 21:17:16 salv-orlando: nova side patch is https://review.openstack.org/#/c/44596/ 21:17:23 ok 21:17:29 nati_ueno: thanks 21:17:33 markmcclain: We have comment for nova side patch https://review.openstack.org/#/c/44596/ 21:17:46 ok 21:17:55 Any other bugs the team needs to track? 21:17:57 We have also discussion log in the bug report 21:18:05 perfect 21:18:25 #topic docs 21:18:27 emagana: hi 21:18:36 Hello 21:18:47 looks like you've updated the doc bug list 21:18:49 Let's start with Neutron wiki 21:18:52 any you'd like to highlight? 21:18:54 I have updated the wiki with the High Priority tickets in Docs related to Neutron: 21:19:03 https://wiki.openstack.org/wiki/Network/Meetings#Docs_.28emagana.29 21:19:20 markmcclain: Will keep this section updated 21:19:46 ok.. thanks 21:19:49 only one bug is unassigned: https://bugs.launchpad.net/openstack-manuals/+bug/1252131 21:19:51 Launchpad bug 1252131 in openstack-manuals "Neutron settings for Pacemaker out-of-date" [High,Confirmed] 21:20:16 uvirtbot: that one! in deed 21:20:18 emagana: Error: "that" is not a valid command. 21:20:53 I don't recall who wrote the original docs 21:21:02 do we have a volunteer to tackle this one? 21:21:10 There no new features in Icehouse-1, so no more new tickets 21:21:14 if not we can work offline to find one 21:21:15 sorry, bugs! 21:21:23 markmcclain: sure! 21:21:27 emagana: thanks for updating 21:21:31 #topic Nova Parity 21:21:33 beagles: hi 21:21:36 Hi! 21:22:09 looks like the holiday week slowed us down a bit last week 21:22:10 So I was a little dumb last week thinking I was going to get some time on the nova meeting... 21:22:20 it didn't happen because there wasn't a meeting because of the holiday 21:22:32 no worries happens this time of year as folks are in and out 21:22:47 so there is stuff that holds over from the previous things... 21:23:19 it would be good if we could get one or more volunteers to document/spec out how nova deployments can be mimic'd in neutron 21:23:33 yeah agreed 21:23:39 I've done some rough leg work on this but I think we'll need some fine tuned stories 21:23:52 ok.. these are doc items? 21:23:53 The main problem is going to be the multi_host setups - no equivalent in neutron 21:24:00 sc68cal, agreed 21:24:09 sc68cal: right.. for now let's work on all cases but that one 21:24:17 that's going to require some other development 21:24:28 I think aveiga and I can possibly document our delpoyment, from essex & nova to our neutron based setup 21:24:35 sweet 21:24:50 we have some paperwork in progress for that for ipv6 reasons 21:25:10 sc68cal: great 21:25:22 beagles: any other items to raise? 21:25:23 put an action item up for me to follow up 21:25:40 #action sc68cal to follow up on User Stories with beagles 21:25:55 thx 21:27:06 beagles: thanks for working on this.. feel free to post new action items to mailing list 21:27:21 not really, just that tempest is kind of in the way :) so I'll be a little obnoxious with the reaching out to help ;) 21:27:47 great it's good we raise these issues early 21:27:50 #topic Tempest 21:27:52 mlavalle: hi 21:27:53 Sure. If no-one objects I will post a link to the milestones etherpads 21:27:56 I have updated in https://etherpad.openstack.org/p/icehouse-summit-qa-neutron in the API section, the gap analysis I've done for API tests. The idea is that people interested in helping us to develop tests will pick from that list features they want to test 21:28:03 looks like there are lots of new reveiws this week 21:28:36 I am taking the position that we want vVERY comprehensive API test 21:28:47 we do 21:28:47 mlavalle, +1 21:28:52 We could use more neutron devs, not necessarily core, reviewing these patches 21:29:08 so i'm specifying that we need as many attributes and errors as possible 21:29:11 more eyes is a good thing 21:29:13 also negative tests 21:29:38 I will step up my reviews on the patches. 21:29:41 Foks, I can help out with the reviews 21:29:43 i've made it so far through the core API and half L3 21:29:43 * salv-orlando reiterates that on positive tests state transitions should be verified when applicable 21:30:00 ? 21:30:12 I can help as well 21:30:13 salv-orlando: eg? 21:30:29 salv-orlando: absolutely….. the only observation there is that those verifications will be done through the api itself 21:30:32 the router interface goes up when you attach it to a router 21:30:48 salv-orlando: gotcha 21:30:54 that means the l3 agent has actually picked up the API command and implemented it 21:31:05 by tonight I expect to finish floating ip's 21:31:30 yeah.. having some positive steps will help to narrow down the failure causes 21:31:35 mlavalle: great 21:31:51 any other tempest items to discuss this week? 21:32:03 I am also going to create in the neutron and tempest wiki's a how to guide for people interested in helping with API tests 21:32:21 Awesome that will be super helpful for new contributors 21:32:24 that will be done by early next week 21:32:26 I've provided an update on the mailing list for parallel and isolated testing 21:32:34 mlavalle: That will be great! 21:32:36 salv-orlando: thanks for sending out that update 21:32:36 I'm here if anybody wants to know more. 21:32:39 mlavalle: ping me if you need help with any of that, I'd like to help you 21:32:51 roast; perfect 21:32:56 roaet 21:33:17 that's all i have this week 21:33:25 i'd like to ask about tempest scenario tests 21:33:31 we're working on adding some for lbaas 21:33:34 enikanorov: go ahead 21:33:41 the first one is on review 21:34:09 enikanorov: link? 21:34:21 I think it's better to have at least some basic one merged in early i-2 since we try to push vendors to setup their third-patry testing 21:34:36 enikanorov: agreed 21:34:44 looking for the link 21:34:46 ok 21:34:49 Can I ask a question regarding vendor plugin/ML2 driver testing? 21:34:55 Sukhdev: sure 21:35:04 https://review.openstack.org/#/c/58697/ 21:35:10 enikanorov: thanks 21:35:38 I am getting ready to put together testing for ML2 drivers and want to know if there is a precedence alreay in place which I can leverage from? 21:36:38 Sukhdev: currently no one has published anything outside of the infra wiki 21:37:04 what would be a good staring point for me? 21:38:26 Sukhdev: Want to chat offline? Ping me if so. We can share notes here. 21:38:45 Sukhdev: IMHO, a good starting point is getting existing Tempest tests running with your MechanismDriver. 21:38:50 mestery: can you make sure those notes make it online? 21:38:58 mestery: thanks - will ping you offline 21:39:01 markmcclain: Yes, I will make sure to do that. 21:39:08 mestery: thanks 21:39:16 ok we're starting to run short on time 21:39:18 #topic API 21:39:34 question on API 21:39:35 salv-orlando: I'm guessing with the critical bug work nothing new to add here? 21:39:42 enikanorov: what's your question? 21:40:00 the previous meeting there was something about 'API deprecation cycle' or something 21:40:07 enikanorov: correct 21:40:12 can't remember exactly. was said about lbas 21:40:15 could you explain? 21:40:45 enikanorov: when we make changes 21:40:47 in fact i'm asking because I'd like to know our abilities to change the API in incompatible way 21:40:54 (sure i know that we better to avoid that) 21:41:25 we need to provide compatibility for at least one release for both variants 21:41:40 i see. makes sense 21:41:53 this will enable deployers and users to have time to update their tooling 21:42:06 will this be an isse? 21:42:10 *issue? 21:42:28 no, i dont think so 21:42:34 cool 21:42:41 salv-orlando: anything else? 21:42:42 at this point it looks like this requirement can be satisfied 21:43:02 I assume that "at least one release" implies that longer is better... 21:43:05 enikanorov: good to know.. make the commit message includes the DocImpact flag 21:43:15 I think for experimental features, such as fwaas, we can have some big change on API, cann't we? 21:43:17 geoffarnold: where possible yes 21:43:19 markmcclain: ok 21:43:20 nothing else 21:43:37 gongysh: yes.. anything released as preview or experimental is subject to change 21:43:53 #topic ML2 21:44:08 mestery or rkukura: any update? 21:44:25 not much new except our patches getting backed out (temporarily I hope) 21:44:36 no meeting last week 21:44:41 Only update I have is that I've started looking into multi-node devstack testing, which ML2 will require to really be tested thorougly. 21:44:54 I spoke to the qa folks in their meeting last week, will continue that tomorrow. 21:44:56 weekly meetings are now Wednesday's 16:00 UTC on #openstack-meeting-alt 21:45:07 I guess there is no multi-node testing done for the gate at the moment, so we're blazing a new path forward here. 21:45:15 rkukura: do we have an assignee to work on addressing the problem that required the revert? 21:45:20 mestery: great 21:45:53 mestery: yeah it's only single node… the addition of multi node is a good thing 21:46:15 markmcclain: I think tripleo also requires it, so hoping to leverage work between us and them. 21:46:21 markmcclain: the patch I pushed the revert for? 21:46:27 salv-orlando: yes 21:46:36 markmcclain: Do you have the link for that patch? 21:46:39 mestery: awesome win for everyone 21:46:58 I think we can repush again once the patches for suppressing unnecessary notifications are merged, but I am not an ML2 expert. 21:47:04 https://review.openstack.org/#/c/58827/ 21:47:19 salv-orlando: ok.. good to know 21:47:25 thanks rkukura 21:47:31 rkukura: thanks 21:47:37 Any other ML2 items this week? 21:47:45 btw I can confirm that one of the problems that that patch meant to address went away 21:47:47 salv-orlando: Do you feel backing this out actually helped? 21:47:53 as soon as the revert patch landed 21:47:56 I assume you feel not 21:48:11 so at least it wasn't for nothing :) 21:48:11 armax: what issue is that? 21:48:23 https://review.openstack.org/#/c/58827/ 21:48:28 second bug on the commit message 21:48:34 thanks 21:48:42 I'm not doubting notifications are an issue in the agent, but am not clear on why that patch reduces the notifications 21:48:48 mlavalle: sure np 21:49:20 the patch that reduces notification is not 58827 21:49:30 salv-orlando: do you have them at hand? 21:49:56 looks like the DB lock timeout issue 21:50:29 The patch we reverte included a call to update_port for each port in a network being deleted. due to some issues recently observed update_port always notifies the agent. But I was thinking at occurrences of bug #1253896 21:50:30 Launchpad bug 1253896 in neutron "test_minimum_basic_scenario fails with SSHException: Error reading SSH protocol banner" [High,In progress] https://launchpad.net/bugs/1253896 21:51:19 And I do not understand enough of ML2 for making any assertion on what is good and what is bad for it. On the proposed revert we observed 5% failure rate compared with 33% of neutron jobs 21:51:52 I'm thinking the correct code (the patch) is helping expose the underlying issue that needs to be fixed 21:52:10 ok.. we're running short on time so we can continue on the ML 21:52:14 salv-orlando: Lets followup on this offline 21:52:18 thanks for the ML2 update 21:52:20 salv-orlando: I also noticed that the nature of the exceptions in 1251448 changed when the patch was reverted 21:52:23 #topic IPv6 21:52:28 Hello 21:52:30 sc68cal: meeting this week right? 21:53:05 IPv6 work - mostly been focusing on short term needs - provider networking w/ instances configured via SLAAC, with RAs coming from a switch north of OpenStack. Required changes to iptables was merged today, and I'm working on getting the patch to disable hairpinning on the Nova side, in the VIF driver. 21:53:17 for the future, it would be good to know if the core team advises review of a ML2 core team member before approvals. 21:53:35 Meeting this thursday, plan to discuss more medium term blueprints 21:53:49 great.. 2100UTC in #openstack-meeting-alt 21:54:03 #topic Open Discussion 21:54:14 wanted to ask about ssl certificates 21:54:28 nati_ueno has pushed a patch that adds certificate resource 21:54:42 i wanted to know if there is a consensus on storing certs in neutron DB 21:54:55 I want to get a sense as to how confident we feel the neutron tempest sprint will actually take place in January in Montreal. I want to make reservations ASAP to save my employer some money 21:54:55 we're discussing the same among lbaas folks 21:54:58 enikanorov: may be we should have meeting on this 21:55:13 nati_ueno: would be good 21:55:28 yeah we discuss this more on the mailing list 21:55:30 enikanorov: when is your preference? 21:55:48 markmcclain: ok I'll send a mail for the openstack-dev 21:55:58 nati_ueno: i need to think more about this 21:56:20 enikanorov: could you share attributes for ssl certificate for LB? 21:56:25 nati_ueno: thanks.. I'd like to make discussions like this open because there security minded folks who might not attend our meetings 21:56:50 here's the thread: http://lists.openstack.org/pipermail/openstack-dev/2013-December/020785.html 21:56:51 markmcclain: That's make sense, let's dicuss on the mailing list 21:57:04 security-minded olks are most welcomed in such discussions for their cons :) 21:57:05 markmcclain: Thanks 21:57:22 Any other open discussion items? 21:57:26 just wanted to say hi, I'm new here and trying to understand how Neutron>ML2>ML2-agent works, so I'll probably add some PEP257 docstrings along the way 21:57:35 markmcclain: grenade status : we are working on that with jlibosva. Infra patch under review https://review.openstack.org/#/c/59039/ which blocks Grenade patch: https://review.openstack.org/#/c/58695/ 21:57:39 markmcclain: See mlavalle's question too :) 21:57:53 Makdaam: awesome good docstrings helpful 21:57:58 maybe it's more related to infra, isn't it? 21:58:08 the new job I mean 21:58:15 EmilienM: awesome thanks for updating 21:58:17 mestery: thanks buddy ;-) 21:58:21 Makdaam: Probably should do a wiki page - I could help - hard to learn the hard way :) 21:58:34 * mestery is looking at the same thing as mlavelle here. :) 21:59:06 mlavalle: it is a go 21:59:08 sc68cal: I'll ping you later for some help, right now I'm at linuxbridge/ovs level and it's quite clear 21:59:26 cool, i'll make my reservations to be in Montreal January 15 21:59:36 anteaya: knows more info, so you've got questions feel free to reach out to here 21:59:37 *her 22:00:12 * mestery wonders if Montreal will have more snow than Minnesota in January ... 22:00:20 anteaya already offered me some help :) 22:00:42 * markmcclain doesn't deal with snow well 22:00:46 mestery: I'm still pondering Montreal in the winter at all 22:00:47 mestery: I think it will…. 22:01:05 * mestery will bring hats and gloves for folks from more tropical climate zones. :) 22:01:14 markmcclain; those Californians….. 22:01:29 mlavalle: I'm actually Southeast US 22:01:42 mlavalle: colorado rockies 22:01:45 oh!!! 22:01:48 Ok we're at time for this week.. thanks to everyone to stopping in 22:01:53 I just understand my limits in the snow 22:01:56 * mestery has snow falling outside his window right now. 22:01:57 I love colorado! 22:01:59 thanks markmcclain 22:02:02 I think it a good chance to try and see whether the things about alcohol keeping you warm is true. 22:02:03 Have a great week and talk to everyone on teh ML and IRC 22:02:05 BTW, I am scared of Montreal weather as well :-) but, considering going :-) 22:02:10 #endmeeting