21:02:25 <markmcclain> #startmeeting Networking
21:02:26 <openstack> Meeting started Mon Aug 18 21:02:25 2014 UTC and is due to finish in 60 minutes.  The chair is markmcclain. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:02:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:02:31 <openstack> The meeting name has been set to 'networking'
21:02:49 <markmcclain> #info mestery is off today, so I'm filling in
21:02:55 <markmcclain> #link https://wiki.openstack.org/wiki/Network/Meetings Agenda
21:03:04 <markmcclain> #topic Announcements
21:03:19 <markmcclain> #link https://launchpad.net/neutron/+milestone/juno-3
21:03:48 <rudrarug_> hi
21:04:04 <markmcclain> #info Feature Proposal Freeze is this Aug 21st
21:04:12 <markmcclain> #link https://wiki.openstack.org/wiki/FeatureProposalFreeze
21:05:08 <markmcclain> If code is not proposed by the end of the day on the 21st the feature will be deferred
21:05:38 <Sukhdev> Hello - sorry for being late
21:06:00 <markmcclain> Please note deprecation notices for the cisco nexus and mellanox plugins
21:06:17 <markmcclain> Any questions before continue?
21:06:37 <markmcclain> Sukhdev: no worries… we just got going
21:06:42 <markmcclain> #topic Bugs
21:06:51 <enikanorov_> hi
21:06:52 <markmcclain> enikanorov_: hi
21:07:09 <enikanorov_> there are two bugs worth attention this week:
21:07:21 <enikanorov_> https://bugs.launchpad.net/neutron/+bug/1357379
21:07:22 <uvirtbot> Launchpad bug 1357379 in neutron "policy admin_only rules not enforced when changing value to default" [Medium,In progress]
21:07:52 <enikanorov_> that's a kind of security vulnerability where regular user may update admin-only attribute from existing to default
21:07:59 <enikanorov_> https://bugs.launchpad.net/neutron/+bug/1356227
21:08:01 <uvirtbot> Launchpad bug 1356227 in neutron "TestLoadBalancerBasic fails and leads to consequent tests failures" [High,In progress]
21:08:27 <enikanorov_> that is a regression from the fix that got rid of rpc calls inside transaction. the fix is on review
21:08:59 <markmcclain> salv-orlando: you've looked at 1357379 too?
21:09:04 <enikanorov_> I continue to monitor bugs and some types of gate failures
21:09:24 <markmcclain> enikanorov_: thanks for monitoring the bugs
21:09:30 <salv-orlando> markmcclain: I have looked, but it’s now in progress and I can’t find a patch for that
21:09:58 <salv-orlando> enikanorov_: did you speak with the current assignee?
21:10:07 <enikanorov_> salv-orlando: it's WIP for now. we're thinking on the best approach to the fix
21:10:27 <salv-orlando> WIP means we have a patch for it and we’re looking whether we can do something better
21:10:42 <salv-orlando> so where’s the patch? ;) gerrit did not link it to launchpad
21:11:14 <enikanorov_> salv-orlando: https://review.openstack.org/#/c/114531/
21:11:56 <salv-orlando> cool thanks - I’m taking the liberty to link it to the bug report, if you don’t mind
21:12:02 <enikanorov_> sure!
21:12:12 <markmcclain> salv-orlando: thanks
21:13:25 <markmcclain> Any other bugs the team should discuss?
21:13:34 <enikanorov_> not from my side today
21:13:46 <markmcclain> enikanorov_: thanks for tracking these
21:14:01 <markmcclain> #topic Team Discussion - Kumbaya
21:14:11 <enikanorov_> lol
21:14:24 <nati_ueno> Kumbaya?
21:14:24 <sc68cal> yaayt
21:14:35 <enikanorov_> *Kilo?
21:14:35 * dougwig breaks out the smores.
21:14:42 <markmcclain> #link http://en.wikipedia.org/wiki/Kumbaya
21:15:13 <markmcclain> the last few weeks have been a bit bumpy community wise
21:15:26 <sbalukoff> Yay! Understatement!
21:15:28 <sbalukoff> ;)
21:15:39 <salv-orlando> why? we’re making a lot of progress on nova parity features and stability
21:15:53 <salv-orlando> is there anything else that really mattered for juno?
21:15:55 <markmcclain> Kyle and I wanted to remind everyone that all of us are in this together
21:16:22 * salv-orlando sounds like a warning - either we sort this together or we go down together
21:16:30 <emagana> salv-orlando: Yes!
21:16:52 <emagana> salv-orlando: Migration from nova-network to Neutron ... DVR testing
21:17:21 <salv-orlando> emagana: migration won’t happen I’m afraid, but we’re interrupting markmcclain
21:17:23 <markmcclain> so it's important that we remember that it as we go through the busiest part of the cycle the next few weeks
21:17:46 <blogan> markmcclain: any update on the incubator?
21:18:31 <markmcclain> blogan: I was hoping for final update today, but am awaiting word back on an item that were working on over the weekend
21:19:14 <markmcclain> #topic Team Discussion 3rd Party CI systems
21:19:28 <emagana> Hi There!
21:19:31 <markmcclain> There's been a good deal of discussion on the mailing the last few days
21:19:53 <markmcclain> emagana: what items did you want to cover here?
21:20:15 <emagana> markmcclain: CI requirements for Juno
21:20:34 <emagana> it is up to Neutron to define the requirements for third-party CI testing
21:20:48 <emagana> based on the multiple responses that I got to this thread:
21:20:52 <emagana> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/043218.html
21:21:13 <emagana> It clear that we have not defined properly and specifically the CI requirements
21:21:45 <markmcclain> emagana: right we do have some variance
21:21:55 <emagana> I want to quickly define what are the requirements for Juno
21:21:59 <lyxus> emagana, and a definition problem
21:22:11 <dougwig> it's a bit late to be changing too radically from what's here: https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
21:22:14 <emagana> FRom this #link https://wiki.openstack.org/wiki/NeutronThirdPartyTesting
21:23:07 <emagana> It is not really clear what we are asking for and I do agree with dougwig it is too late to enforce strong requirements at this time.
21:23:10 <salv-orlando> I think what dougwig said set the requirements for Juno. They’re a bit flexible, but it would already be a result if all CI systems conformed to that. Then perhaps you can define more stringent requirements for Kilo, and that woukd be fine.
21:23:24 <markmcclain> dougwig: I agree
21:23:27 <dougwig> salv-orlando: +1
21:23:37 <markmcclain> salv-orlando: I agree as well
21:23:40 <emagana> Yes and Yes!
21:23:45 <salv-orlando> as a side note we’re starting to bureaucratize everything. In this case can we just use common sense?
21:23:45 <nati_ueno> +1
21:24:39 <emagana> So, I want to work with each one of the CI owners and understand better their testing approach
21:24:41 <salv-orlando> cool, I think that then emagana would be more than happy to outline requirements for kilo
21:24:45 <dane_leblanc> One thing that we need to consider is that there may be exceptional times when a CI may need to throttle back on the diffs that it's testing against
21:25:08 <salv-orlando> emagana: welll, I hope you don’t want to dictate every CI owner how they should implement their infra!
21:25:13 <dane_leblanc> For example, before diffs are merged upstream which enable hardware to be tested.
21:25:15 <emagana> Present a session during Kilo summit where we all agree on the requirements
21:25:27 <markmcclain> dane_leblanc: right.. there we do have to balance the tests we run vs resources required
21:25:54 <emagana> By Juno I think we should be testing at least: 1) Neutron Tempest tests
21:25:56 <dane_leblanc> If there was a way for us to communicate that status..
21:26:06 <lukego> This might be too fuzzy, but one thing CI operators don’t currently get any guidance on is how much time/resources to commit in order to be successful. the advice seems to range from “it only takes 10 minutes, what is everybody complaining about?” up to suggesting a full time person.
21:26:20 <emagana> 2) Only changes related to the sub-system the driver/plugins belongs to
21:26:33 <emagana> 3) Do not post fake results!
21:26:57 <salv-orlando> lukego: I understand your concerns, but I’m afraid that infra can’t offer anything more than documentation - and they already do
21:27:01 <emagana> dane_leblanc: During today's thirdparty meeting on IRC that topic was discussed
21:27:02 <markmcclain> emagana: I hope nobody is posting fake results
21:27:04 <lyxus> emagana, what about the -1 ? The 3rd party CI recommend to be manual
21:27:04 <dougwig> i think it's important to also not be too heavy-handed in what tests get run.  an lbaas driver does not need to be testing l2 port stuff in detail (except how it might be using it); that's just duplicating openstack jenkins, and increasing the false negative potential.  it's like a duplicate unit test.
21:27:11 <Sukhdev> emagana: I did not know people were posting fake results :-)
21:27:14 <marun1> lukego: The time required is mostly determined by the skill of the implementors, in any case.
21:27:30 <emagana> dane_leblanc: check the meeting logs
21:27:42 <dane_leblanc> emagana: That was about the cinder issue?
21:27:46 <salv-orlando> Sukhdev: yeah I’ve been posting pictures of kittens rather than logs ;)
21:28:01 <Sukhdev> salv-orlando: ha ha funny!!!
21:28:02 <markmcclain> salv-orlando: haha
21:28:07 <salv-orlando> and the nyan cat as well
21:28:11 <emagana> dane_leblanc: In general they are proposing a specific mailing list for CI status
21:28:29 <kevinbenton> emagana: ugh
21:28:35 <dane_leblanc> emagana: Right, about the CI status posting.
21:28:40 <kevinbenton> emagana: i thought we were trying to get away from mailing lists
21:29:00 <emagana> kevinbenton: We are open to suggestions
21:29:08 <kevinbenton> emagana: i liked the wiki with the current status
21:29:21 <dane_leblanc> My point before is that when we're first bringing up a CI for new hardware, only a small subset of change sets will work on the CI.
21:29:22 <rms_13_> +1 kevinbenton
21:29:23 <emagana> Anyway! Is everybody happy with the three requirements for CI in Juno?
21:29:25 <kevinbenton> emagana: we don’t really need a historical record that someone did some maintenance on their CI five weeks ago
21:29:44 <salv-orlando> I think there’s still a lot of confusion. It would be good to have a discussion on the ML on reporting CI status - and it’s worth reminding the stackalatics folks tried alreaydy to help us - maybe it’s worth starting from there.
21:29:48 <emagana> kevinbenton: I agree!.. but please let's finish one discussion first
21:29:57 <lyxus> emagana, we can't a have a proper status until we have 1) list of tests 2)time to execute the tests 3) Definition of a healty CI
21:30:19 <rms_13_> emagana: 2) is unclear to me.
21:30:35 <kevinbenton> rms_13_: +1
21:30:36 <markmcclain> ok… let's focus this discussion a bit
21:30:43 <Sukhdev> emagana: clarify the first requirement - tempest tests - what if some of the tests are not applicable to CI?
21:30:51 <markmcclain> we all agree logs should be real :)
21:30:55 <emagana> markmcclain: I think those are valid questions.
21:31:06 <Sukhdev> markmcclain; +1
21:31:12 <emagana> markmcclain: It could take a lot of time. What if I propose this:
21:31:28 <lukego> emagana: I think myself and several others are finding that we need to upgrade our CI quite a bit to run the scenario tests in a reliable way and are trying not to rush it and cause problems.
21:31:37 <markmcclain> Seems that the general functionality of most systems is that subsystem(s) that a driver/plugin supports be tested
21:31:46 <emagana> markmcclain: Let me improve the wiki today and cover all these questions. Working with all CI owners until everybody is happy
21:32:11 <Sukhdev> lukego: Agree - I see very high failure rates with scenario tests
21:32:14 <markmcclain> if an ML2 implementation wants to skip VPN tests for Juno that should be ok
21:32:23 <emagana> Now, these requirements are minimal. If you CI does more, dont change it   :-)
21:32:54 <dougwig> emagana: regarding full tempest, i chatted with kyle about the the number unnecessary scenario tests with something like an lbaas driver, and we agreed to a subset (which is listed in the wiki.)  i just don't see how running tests that do not affect the third-party code in the slightest will do anything but add to false negative potential.
21:32:55 <markmcclain> so really we ahve to be careful about changing which set of tests run this late in the game
21:33:26 <Sukhdev> dougwig: +1
21:33:29 <markmcclain> dougwig: +1
21:33:35 <rudrarug_> dougwig: +1
21:33:35 <salv-orlando> dougwig: I agree with you. Hence my point on common sense.
21:33:40 <emagana> dougwig: My opinion is that CI should be testing core API.. So, I agreee with ypu
21:33:44 <lyxus> salv-orlando, +1
21:34:09 <emagana> Ok, it seems that everybody feels the same way
21:34:15 <markmcclain> Looks like emagana wlll put together a proposal for kilo changes
21:34:42 <markmcclain> we can either work on it via the ML the next few weeks and if necessary schedule design summit time
21:34:49 <emagana> markmcclain: Yes, I will. For juno I will try to fix the Wiki to make it more clear and being very flexible with all CI systems..
21:34:51 <Sukhdev> emagana: Core API tempest test is the correct way to go
21:35:02 <emagana> Sukhdev: That is for me the minimal requirement
21:35:18 <salv-orlando> folks - point taken. Those are differences we can smooth out.
21:35:21 <Sukhdev> emagana: I am good with that
21:35:26 <marun1> Do you mean the api tests alone?
21:35:38 <dougwig> Sukhdev: plus relevant scenario tests, i'd think.
21:35:51 <emagana> basically, everything under: tempest/api/network/test_networks.py
21:35:51 <marun1> dougwig: +1
21:36:01 <Sukhdev> dougwig: not really - those will be optional
21:36:02 * salv-orlando think we’re having one of those kindergarten moments ;)
21:36:06 <emagana> and tempest/api/network/test_networks_negative.py
21:36:10 <marun1> The api tests don't validate that anything happens below the server
21:36:16 <marun1> Nothing
21:36:17 <marun1> Nada
21:36:26 <Sukhdev> emagana: yes - agree with this classification of minimal tests
21:36:27 <emagana> marun1: I know that marun1
21:36:28 <salv-orlando> marun1: I’m sure scenario tests will be a requirement for Kilo
21:36:40 <emagana> I will propose that for Kilo
21:37:10 <emagana> marun1: It is too late for being hard on the plugins and drivers... I propose to be flexible in Juno but things will change in Kilo
21:37:18 <markmcclain> #action emagana will put together proposal for Kilo testing changes
21:37:19 <dougwig> the wiki linked above specifies api + relevant scenario.  the lbaas subset, e.g. is the entire api suite plus relevant scenario.
21:37:21 <dougwig> emagana: +1
21:37:23 <marun1> emagana: Fair enough
21:37:34 <Sukhdev> marun1: not all CI's deal with anything related to underneath servers
21:37:38 <markmcclain> for how we signal needs such as reviews blocking a CI from functioning what exactly healthy means I don't think we can solve in 23minutes
21:37:53 <markmcclain> those are also issue that the other projects will need to tackle as well
21:37:55 <marun1> There is little need for 3rd party reporting of api testing
21:37:57 <emagana> #action emagana will clarify Juno requirements for CI systems
21:38:04 <markmcclain> might be a good cross project session in Paris
21:38:05 <marun1> At least at the tempest level
21:38:16 <Sukhdev> marun1: hence, API tests should be minimal requirement - those CI's that deal with servers should include scenario or other tests
21:38:22 <dougwig> marun1: strongly disagree, but i'm not sure we need to hash that out right now.
21:38:27 <marun1> +1
21:38:34 <emagana> For all developers, feel free to reach me our directly about any CI related issue. I will put everything together and visible for all
21:39:08 <markmcclain> emagana: thanks for working on this
21:39:09 <emagana> markmcclain: I think we could move to the next topic on the agenda
21:39:13 <marun1> dougwig: We can do that testing better at the functional rather than integration level.
21:40:03 <markmcclain> #topic Parity
21:40:45 <markmcclain> we closed another parity item this week: http://lists.openstack.org/pipermail/openstack-dev/2014-August/043216.html
21:41:06 <markmcclain> thanks to salv-orlando for spearheading that effort
21:41:27 <salv-orlando> there’s a little tweak to this story - please read the last update from the mailing list
21:41:46 <salv-orlando> in a nutshell we’re going to limit tests on the integrated gate to only neutron core services.
21:41:55 <markmcclain> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/043358.html final state
21:41:58 <salv-orlando> This was a request from nova core team backed by qa core team
21:42:19 <markmcclain> I think that is a reasonable approach
21:42:27 <marun1> +1
21:42:46 <markmcclain> #topic Docs
21:43:24 <markmcclain> emagana: any doc items to highlight?
21:43:35 <emagana> markmcclain: Not really!
21:44:10 <emagana> markmcclain: I was focused on the CI part, nothing to cover for docs
21:44:18 <salv-orlando> I wanted to find volunteers for reviewing our current dev docs and writing some more before we close the release cycle
21:44:28 <markmcclain> salv-orlando: awesome
21:44:29 <salv-orlando> I’m talking about the docs in devref in neutron source tree
21:44:49 <markmcclain> I do want to highlight a spec in the doc program
21:44:52 <salv-orlando> I’ll go the mailing list but if you want to cooperate let me know know so I’ll ping you in the next few days.
21:45:01 <markmcclain> #link https://review.openstack.org/#/c/113597/
21:45:02 <sc68cal> Do we have any areas in devref that need attention?
21:45:11 <sc68cal> or desired topics?
21:45:29 <markmcclain> specifically it would go to get several cores to provide feedback to docs team
21:46:33 <markmcclain> #topic L3
21:46:40 <carl_baldwin> markmcclain: hi
21:46:43 <markmcclain> carl_baldwin: hi
21:47:01 <carl_baldwin> Still working through the experimental job that runs DVR.
21:47:12 <carl_baldwin> There are just a couple of tests failing somewhat intermittently.
21:47:22 <carl_baldwin> Otherwise, it has improved quite a lot.
21:47:34 <carl_baldwin> The DVR backlog is trimming down as well.
21:48:08 <markmcclain> nice to see the backlog getting smaller
21:48:27 <carl_baldwin> Starting to shift some attention to L3 HA patches.  Assaf has written a post to help reviewers which I will soon send out.
21:48:38 <markmcclain> cool
21:48:48 <carl_baldwin> Watch the ML for that.
21:49:06 <carl_baldwin> I think that is all I have.  The rest of the status is posted on the wiki.
21:49:39 <carl_baldwin> Thanks markmcclain
21:49:39 <markmcclain> carl_baldwin: thanks for the update
21:49:44 <markmcclain> #topic Advanced Services
21:49:47 <markmcclain> SumitNaiksatam: hi
21:49:51 <SumitNaiksatam> markmcclain: hi
21:50:00 <SumitNaiksatam> i have updated the meeting wiki page
21:50:15 <SumitNaiksatam> for all the bp specs that were approved, we have patches posted
21:50:31 <SumitNaiksatam> happy to discuss that here, or yield time to other higher priority items
21:50:32 <markmcclain> cool
21:51:46 <markmcclain> in the 9mins left let's run through the other teams
21:51:51 <markmcclain> #topic IPv6
21:51:59 <markmcclain> SumitNaiksatam: thanks for update
21:52:03 <markmcclain> sc68cal: hi
21:52:03 <SumitNaiksatam> markmcclain: sure
21:52:08 <sc68cal> ello
21:52:16 <dougwig> (can be back up to adv. services and get a flavors update?)
21:52:19 <dougwig> /be/we
21:52:28 <dougwig> is it in or out?
21:52:37 <markmcclain> #undo
21:52:38 <openstack> Removing item from minutes: <ircmeeting.items.Topic object at 0x2990650>
21:52:48 <markmcclain> dougwig: will be in
21:52:54 <markmcclain> #topic IPv6
21:53:04 <markmcclain> sc68cal: looks like we need reviews
21:53:12 <sc68cal> yeah
21:53:22 <sc68cal> we also are trying to get in contact with the NEC CI owner
21:53:25 <sc68cal> to do a recheck
21:53:37 <sc68cal> https://review.openstack.org/#/c/106299/
21:54:37 <sc68cal> that's all I have
21:55:02 <markmcclain> sc68cal have reached out to amotoki for help contacting NEC CI owner?
21:55:21 <sc68cal> No I started by e-mailing the e-mail that is listed for it on Gerrit
21:55:41 <markmcclain> ok.. ping amotoki too to see if he can assist
21:55:48 <markmcclain> sc68cal: thanks for the update
21:55:58 <markmcclain> #topic ML2
21:56:07 <rkukura> We are tracking reviews at https://wiki.openstack.org/wiki/Tracking_ML2_Subgroup_Reviews
21:56:28 <rkukura> Not much needs discussing here, I think
21:56:44 <rkukura> That’s all I’ve got
21:56:57 <salv-orlando> rkukura: I will re-review the extensionpatch tomorrow - hopefully we’ll sort this out this week
21:57:05 <rkukura> salv-orlando: thanks
21:57:22 <rkukura> also meant to thank banix for organizing the review tracking wiki
21:57:24 <markmcclain> salv-orlando: I left questions on the spec earlier
21:57:49 <markmcclain> still to much ambiguity that needs to be addressed
21:57:54 <salv-orlando> markmcclain: I did not see them. I have way too many browser tabs open.
21:57:55 <nlahouti> markmcclain: rkukura added reply to the comment
21:58:23 <nlahouti> salv-orlando: it is in https://review.openstack.org/#/c/114412
21:58:24 * markmcclain has too many tabs open too
21:58:53 <nlahouti> salv-orlando: that link has patch to the spec that markmcclain requested.
21:59:46 <salv-orlando> nlahouti: bookmarked
21:59:55 * markmcclain notes 1 minute left
22:00:08 <markmcclain> we'll continue discussion on the review
22:00:09 <nlahouti> salv-orlando: thx. I think you have link the the code review.
22:00:13 <salv-orlando> wha’t left on the agenda?
22:00:15 <markmcclain> #topic Open Discussion
22:00:54 <salv-orlando> please stay on lookout for failures in neutron full job. Report that promptly and tag them with neutron-full-job so we can stay on top of them
22:01:05 <markmcclain> good reminder
22:01:27 <markmcclain> as the number of patches pushed to gerrit increases the next few weeks
22:01:31 <salv-orlando> if you really want to be loved by the community provide a logstash trace and an e-r query
22:01:42 <markmcclain> we need to be vigilant to ensure we don't cause gate backups
22:02:16 <dougwig> salv-orlando: do you have a writeup on how to get those two items?
22:02:24 <salv-orlando> otherwise rather than talking about neutron-incubator we’ll be talking about neutron, the openstack incubated project
22:02:44 <salv-orlando> dougwig: I have, it’s on the wiki, but I honestly don’t have a link at hand.
22:03:30 <salv-orlando> dougwig: my google-fu is still strong -> https://wiki.openstack.org/wiki/NeutronGateFailureTriage
22:03:40 <markmcclain> Lastly still meeting with folks in the community on updates to the proposed neutron-incubator changes
22:03:57 <markmcclain> and will post to list soon
22:04:25 <markmcclain> note I'll be using another address since DMARC puts my list emails straight to the spam folder on gmail
22:04:40 <markmcclain> Have a great week all.. thanks for joining today
22:04:43 <markmcclain> #endmeeting