15:00:32 <mlavalle> #startmeeting neutron_l3
15:00:33 <tellesnobrega> thanks guys
15:00:33 <openstack> Meeting started Thu Aug 11 15:00:32 2016 UTC and is due to finish in 60 minutes.  The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:37 <openstack> The meeting name has been set to 'neutron_l3'
15:00:38 <haleyb> hi
15:00:43 <pavel_bondar> hi
15:00:44 <carl_baldwin> o/
15:00:50 <njohnston> o/
15:01:00 <mlavalle> #chair carl_baldwin
15:01:01 <openstack> Current chairs: carl_baldwin mlavalle
15:01:24 <mlavalle> is tidwellr around?
15:01:47 <mlavalle> Agenda for today is here:
15:01:55 <mlavalle> #link https://etherpad.openstack.org/p/neutron-l3-subteam
15:02:08 <mlavalle> #topic Announcements
15:02:28 <jschwarz> \o/
15:02:46 <mlavalle> The obvious reminder is the Neutron mid-cycle meeting next week in Cork Ireland
15:03:27 <carl_baldwin> It is coming right up. Looking forward to seeing people there.
15:04:47 <mlavalle> The other annoucement is we have to keep an eye on Newton-3 milestone
15:05:02 <mlavalle> #link http://releases.openstack.org/newton/schedule.html
15:05:14 <mlavalle> August 29th - September 2nd
15:05:24 <mlavalle> so it is coming our way really quickly
15:05:52 <mlavalle> any other annoucements from the team?
15:06:03 <carl_baldwin> Not from me.
15:06:34 <mlavalle> if not, let's move on
15:06:38 <mlavalle> #topic Bugs
15:07:13 <mlavalle> The first up is one that haleyb and jschwarz have been discussing in the Neutron channel: https://bugs.launchpad.net/neutron/+bug/1612192
15:07:13 <openstack> Launchpad bug 1612192 in neutron "L3 DVR: Unable to complete operation on subnet" [Critical,Confirmed]
15:08:20 <mlavalle> from what I got from the conversation, it might not be a Neutron issue but rather a Tempest one?
15:08:21 <haleyb> mlavalle: yes, there seems to be an issue in a tempest test - add-router-interface is failing, and the unwind is barfing on a port still being in the subnet
15:08:57 <jschwarz> haleyb, mind you, I saw this happen on rally a while back as well
15:08:58 <haleyb> but i see a DBDeadlock on create_port() so wonder if it's an ml2 change
15:09:12 <jschwarz> maybe it's an API change that snuck into Neutron unawares?
15:10:35 * haleyb feels like he slides down the pole info a fire station every morning :)
15:10:58 <mlavalle> so can we say that we need to research this one further?
15:11:35 <haleyb> mlavalle: yes, need to look further, will scream if i need help
15:12:03 <mlavalle> haleyb, jschwarz thank you for keeping an eye on this
15:12:31 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1540983
15:12:31 <openstack> Launchpad bug 1540983 in OpenStack-Gate "Gate failures for neutron in test_dualnet_multi_prefix_slaac" [Undecided,Expired]
15:12:51 <mlavalle> So this morning I went to logstash to try to find ocurrences of this bug
15:13:21 <mlavalle> I am using the query at the top of the bug: message:"in test_dualnet_multi_prefix_slaac" AND voting:1
15:13:35 <mlavalle> and couldn't find a case over the past 7 days
15:13:50 <mlavalle> Am using a wrong query maybe?
15:14:39 <haleyb> mlavalle: logstash wasn't cooperating for me today either, but that one is infrequent
15:15:01 <carl_baldwin> I was thinking there was another related one. Me trying to swap in memories of that.
15:15:19 <haleyb> I did however see a similar failure in the dvr tests, in that case dhcp failed to start, so second VM failed to get IP, and it went downhill from there
15:16:09 <mlavalle> ok, I'll keep an eye on it daily, to make sure we don't get in trouble close to N-3
15:16:27 <carl_baldwin> Could be related to https://bugs.launchpad.net/neutron/+bug/1609540
15:16:27 <openstack> Launchpad bug 1609540 in neutron "Deleting csnat port fails due to no fixed ips" [Critical,In progress] - Assigned to Carl Baldwin (carl-baldwin)
15:17:16 <mlavalle> Yeah, that's the next one in the agenda
15:17:47 <mlavalle> and carl_baldwin and I couldn't find cases of the expected message yesterday
15:17:56 <carl_baldwin> They both involve the same unit test.
15:18:13 <carl_baldwin> mlavalle: Yeah, that is strange. I would expect to see that debug message.
15:18:38 <mlavalle> I will talk to infra today to make sure logstash catches debug level messages
15:19:17 <mlavalle> I guess all we can do for the time being is to be vigilant about these 2 bugs
15:19:40 <mlavalle> I'll check them daily and will talk to infra
15:20:14 <carl_baldwin> mlavalle: Thanks.
15:20:37 <carl_baldwin> It'd be nice to know that we can search for debug messages successfully.
15:21:05 <mlavalle> Next bugs are high importance. First one was reduced to high lately: https://bugs.launchpad.net/neutron/+bug/1562878
15:21:05 <openstack> Launchpad bug 1562878 in neutron "L3 HA: Unable to complete operation on subnet" [High,Confirmed] - Assigned to Ann Taraday (akamyshnikova)
15:21:20 <mlavalle> Thanks to jschwarz for followin up with it. any comments?
15:21:22 <jschwarz> I tried to reproduce this one earlier this week but couldn't
15:21:46 <jschwarz> since it's not occurring in the gate afaik, the importance can be lowered IMO
15:21:58 <mlavalle> even lower than high?
15:22:36 <jschwarz> Medium seems nice since if me and Ann can't reproduce this on rally, this might have been fixed already
15:23:06 <mlavalle> ok, thanks
15:23:38 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1596075
15:23:38 <openstack> Launchpad bug 1596075 in neutron "Neutron confused about overlapping subnet creation" [High,In progress] - Assigned to Kevin Benton (kevinbenton)
15:24:02 <mlavalle> As I said last week, this is a long complicated affair, involving several potential patches
15:24:52 <mlavalle> I pinged kevinbenton yesterday and he is still working on a couple of fixes for this. Once they are ready, he will have some more interaction with the submitter to confir it is fixed
15:25:41 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1603162
15:25:41 <openstack> Launchpad bug 1603162 in neutron "Pluggable IPAM rollback fails with reference driver" [High,In progress] - Assigned to Carl Baldwin (carl-baldwin)
15:25:58 <carl_baldwin> I think I've got this one fixed.
15:26:23 <carl_baldwin> I was just starting to look at the multinode dvr grenade job. I doubt the failure is related.
15:26:24 <carl_baldwin> #link https://review.openstack.org/#/c/348956/
15:26:50 <carl_baldwin> I wanted to be sensitive to rechecks thuogh.
15:26:54 <carl_baldwin> *though
15:27:12 <mlavalle> ++
15:28:21 <mlavalle> if there are no more comments, let's move on. Thanks for the update carl_baldwin
15:28:38 <carl_baldwin> I'll get some reviewers on the fix today.
15:28:49 <mlavalle> Thnaks!
15:29:02 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1610483
15:29:02 <openstack> Launchpad bug 1610483 in neutron "Pluggable IPAM rollback mechanism is not robust" [High,Confirmed]
15:30:09 <mlavalle> any comments on this one carl_baldwin?
15:30:21 <carl_baldwin> This affects external drivers mostly since the reference driver (for now) uses the context DB rollback.
15:31:09 <carl_baldwin> pavel_bondar: Have you guys had a chance to think about this at all?
15:31:43 <pavel_bondar> carl_baldwin: yes, I agree with the issue, current rollback is not actually reliable and has to be reworked
15:32:27 <carl_baldwin> So, in summary, we have no plans yet to fix this but it is an issue that we should plan for soon.
15:33:33 <mlavalle> Thanks!
15:33:43 <pavel_bondar> I would pick this task, but since I am not longer part of Infoblox openstack team (working on Infoblox another project) I don't have enough bandwith to drive it to the end.
15:34:47 <pavel_bondar> Probably I could assist with comments&review, but probably what my current bandwidth allows
15:34:50 <carl_baldwin> pavel_bondar: Anyone else there to pass it on to?
15:35:21 <carl_baldwin> We can take this out of band, mlavalle
15:35:40 <mlavalle> Finally we have https://bugs.launchpad.net/neutron/+bug/1599329
15:35:40 <openstack> Launchpad bug 1599329 in neutron "Potential regression on handing over DHCP addresses to VMs" [High,In progress]
15:35:42 <pavel_bondar> carl_baldwin: it is better to check with John B. about it, I am not sure
15:35:51 <carl_baldwin> pavel_bondar: Will do.
15:36:14 <mlavalle> We were waiting to see if a fix solved this one. Haven't heard anything
15:36:32 <mlavalle> I will check around today about this one
15:36:41 <mlavalle> any other comments?
15:37:00 <haleyb> mlavalle: that looks similar to something i noticed yesterday - VM dhcp fails
15:37:50 <haleyb> http://logs.openstack.org/51/337851/19/check/gate-tempest-dsvm-neutron-dvr-multinode-full/c944b3d/logs/screen-q-dhcp.txt.gz was the info i found so far, but it was a multinode failure, not strict dvr
15:38:23 <mlavalle> ok, will take a look. will ping you if i have questions
15:38:46 <haleyb> tx
15:39:17 <mlavalle> #topic Routed networks
15:39:43 <carl_baldwin> Hi
15:40:04 <carl_baldwin> I think we're doing pretty well here.
15:40:27 <carl_baldwin> We have had some review on the create / delete segment ml2 patch.
15:40:34 <mlavalle> yeah
15:40:44 <carl_baldwin> I'm not sure if xiaohhui is in a position to handle the feedback.
15:41:15 <carl_baldwin> #link https://review.openstack.org/#/c/317358
15:41:23 <carl_baldwin> Looks like xiaohhui is on it.
15:41:49 <mlavalle> yeah he uploaded a revision last night
15:42:23 <carl_baldwin> Good, I had written him an email to see if he needs assistance. I hadn't heard back.
15:42:29 <carl_baldwin> I'll keep watching it.
15:43:09 <carl_baldwin> Do we have anything else pressing for Newton?
15:43:35 <mlavalle> I'll push the segment ids in port patch next revision today
15:43:59 <mlavalle> and we may need to pay some attention to docs
15:44:43 <carl_baldwin> Yes, docs! My only hope now is the plane ride.  :)
15:45:23 <mlavalle> Cool! Let's move on then
15:45:35 <mlavalle> #topic BGP Dynamic Routing
15:45:41 <tidwellr> hi
15:45:45 <carl_baldwin> It'll be nice to go over how we're doing at the mid-cycle and figure out what we need to do for Ocata.
15:45:45 <mlavalle> tidwellr, steve_ruan you are up
15:46:03 <carl_baldwin> That's the last thing about routed networks from me ^
15:46:08 <tidwellr> :)
15:47:01 <steve_ruan> tidwellr, to totally break the dependency
15:47:12 <tidwellr> we've been discussing the eVPN spec this week, we're going to explore a different approach
15:47:27 <steve_ruan> you bgp will not depend on networking-bgpvpn, right?
15:47:29 <tidwellr> however, I don't think we've ever taken the RFE to the drivers team
15:47:41 <tidwellr> steve_ruan: I think we should explore that
15:47:53 <steve_ruan> ok
15:48:26 <tidwellr> anyway, I don't see anything on the eVPN fron getting into Newton, but getting a start on Ocata would be good
15:48:54 <tidwellr> carl_baldwin: has this RFE been discussed at the drivers meeting yet?
15:49:16 <carl_baldwin> tidwellr: no.
15:49:28 <tidwellr> carl_baldwin: we have a spec we've been iterating on, I assume we'd be asked for one anyway
15:50:19 <carl_baldwin> The focus of the drivers meeting has shifted a bit to discussing status of Newton items.
15:50:43 <tidwellr> carl_baldwin: ok, good to know. I don't think there's any rush to explore this with the drivers team at the moment
15:51:03 <carl_baldwin> tidwellr: ok
15:51:17 <tidwellr> I'd like to see it discussed so that maybe we can have something for Ocata
15:51:25 <mlavalle> anything else tidwellr steve_ruan ?
15:51:31 <tidwellr> not from me
15:51:35 <steve_ruan> no,thanks
15:51:44 <mlavalle> Thanks for the update!
15:51:55 <mlavalle> #topic FWaaS
15:52:02 <njohnston> Hi!  So things are looking good for l3 agent extensions - https://review.openstack.org/#/c/339246/ has one +2 (thanks carl_baldwin!), needs another.
15:52:05 <carl_baldwin> tidwellr: Let's get it teed up for discussion. We might be able to touch on it at the mid-cycle.
15:52:29 <carl_baldwin> njohnston: Thanks for the reminder to revisit that one.
15:52:38 <njohnston> carl_baldwin: Sure thing!
15:52:54 <njohnston> The FWaaS side to act as an l3 agent extension is also coming along: https://review.openstack.org/#/c/337699/
15:53:21 <carl_baldwin> njohnston: Excellent.
15:53:23 <njohnston> We're hoping to land significant swaths of FWaaS v2 core functionality on Friday, so it's good to see these things coming together.
15:54:17 <njohnston> I think if I can get https://review.openstack.org/#/c/339246/ merged, all that leaves in the codebase of Neutron proper is a fullstack test, which will get very involved
15:54:30 <njohnston> and I am deferring that work until a little later
15:55:05 <njohnston> I think that's it for me
15:55:21 <mlavalle> njohnston: thanks for the update!
15:55:53 <mlavalle> #topic Conversion to Pluggable IPAM
15:56:51 <carl_baldwin> We got that bug worked out.
15:57:00 <carl_baldwin> I think we're almost in good shape.
15:57:25 <carl_baldwin> I hope to get the bug fix merged and then get a few rechecks on the switch to pluggable before the mid-cycle.
15:57:33 <carl_baldwin> Then, pull the trigger at the mid-cycle.
15:57:51 <mlavalle> Great!
15:58:15 <carl_baldwin> The gate is slow these days.
15:58:46 <njohnston> Queue max delay: 32.70 hours https://twitter.com/openstackstatus/status/763747339177717760
15:59:00 <mlavalle> yikes
15:59:25 <mlavalle> ok team, time is almost up
15:59:28 <carl_baldwin> That's less than 500
15:59:33 <carl_baldwin> Thanks, mlavalle
15:59:37 <njohnston> thanks all!
15:59:52 <mlavalle> Thank you for your attendance and hard work
16:00:01 <mlavalle> #endmeeting