15:00:44 <mlavalle> #startmeeting neutron_l3
15:00:45 <openstack> Meeting started Thu Mar  9 15:00:44 2017 UTC and is due to finish in 60 minutes.  The chair is mlavalle. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:00:48 <openstack> The meeting name has been set to 'neutron_l3'
15:00:59 <haleyb> hi
15:01:02 <john-davidge> Hi o/
15:01:04 <mlavalle> hi
15:01:48 <mlavalle> Agenda for today is here:
15:01:56 <mlavalle> #link https://etherpad.openstack.org/p/neutron-l3-subteam
15:02:31 <mlavalle> #topic Announcements
15:03:30 <mlavalle> We have one month for Pike-1 milestone, Apr10 - Apr14:
15:03:39 <mlavalle> #link https://releases.openstack.org/pike/schedule.html
15:04:44 <mlavalle> Any other annoucements from the team?
15:05:51 <mlavalle> ok, moving on
15:05:57 <mlavalle> #topic Bugs
15:06:30 <mlavalle> First up is https://bugs.launchpad.net/neutron/+bug/1627424
15:06:30 <openstack> Launchpad bug 1627424 in neutron "FlushError on IPAllocation" [High,In progress] - Assigned to Miguel Lavalle (minsel)
15:07:32 <mlavalle> Yesterday, with the help of haleyb we merged the second fix we were expecting for this bug:
15:07:39 <mlavalle> #link https://review.openstack.org/#/c/428774
15:07:56 <mlavalle> The first one was merged just a few days ago:
15:08:22 <mlavalle> #link https://review.openstack.org/#/c/423584
15:09:05 <mlavalle> So I guess the next step is to keep an eye on Kibana and see if we made a dent in the number of hits for this bug
15:09:18 <mlavalle> Hopefully we drove it to 0
15:09:56 <mlavalle> I will be monitoring Kibana over the next few days]
15:10:24 <mlavalle> any questions or comments from the team?
15:10:52 <clarkb> you can write an elastic recheck rule for the bug and have it track for you if you havent already
15:12:04 <mlavalle> clarkb: ok
15:12:26 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1610483
15:12:26 <openstack> Launchpad bug 1610483 in neutron "Pluggable IPAM rollback mechanism is not robust" [High,In progress] - Assigned to Aliaksandr Dziarkach (aliaksandr-dziarkach)
15:12:54 <mlavalle> kevinbenton left a comment a couple of days ago in the proposed solution:
15:13:10 <mlavalle> #link https://review.openstack.org/#/c/390594/
15:13:37 <mlavalle> so it seems we are back to the drawing board with this one
15:14:02 <mlavalle> I will take a closer look later today and maybe ping kevinbenton for some more guidance
15:14:08 <mlavalle> any comments?
15:15:42 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1627480
15:15:42 <openstack> Launchpad bug 1627480 in neutron "create_port can succeed without returning fixed_ips on all requested subnets" [High,Confirmed] - Assigned to James Anziano (janzian)
15:15:54 <mlavalle> I did a bit of analysis on this one
15:17:02 <mlavalle> kevinbenton left a useful log debug sentence in the dhcp agent when this bug happens:
15:17:08 <mlavalle> #link https://github.com/openstack/neutron/blob/master/neutron/agent/linux/dhcp.py#L1303
15:17:21 <mlavalle> this was pointed out to me by janzian
15:18:02 <mlavalle> so digging in kibana for this log message, I found 3 hits over the past 7 days
15:19:15 <mlavalle> here I show, for one of the hits, side by side what happens in the dhcp agent and the neutron server:
15:19:42 <mlavalle> #link http://paste.openstack.org/show/602078/
15:20:23 <mlavalle> You can see there that in the neutron server, the creation of the dhcp port starts normally and in the middle of it, the subnet is deleted
15:20:34 <mlavalle> so it seems to be another race condition
15:21:10 <mlavalle> janzian: let's spend time tomorrow after lunch analyzing other instances of this bug. Is that ok with you?
15:21:14 <john-davidge> great find!
15:21:38 <janzian> mlavalle: Sounds good to me, thanks again for your help with this :)
15:21:45 <mlavalle> :-)
15:22:10 <mlavalle> any other comments?
15:22:40 <janzian> Nope, I've been looking through Kibana as well but you've summed up all I've found myself
15:23:39 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1509004
15:23:39 <openstack> Launchpad bug 1509004 in neutron ""test_dualnet_dhcp6_stateless_from_os" failures seen in the gate" [High,Confirmed]
15:24:54 <mlavalle> This one came up during the Neutron CI meeting a couple of days ago with haleyb and ihrachys
15:25:07 <mlavalle> it seems it is really making noise in the gate
15:25:19 <mlavalle> so I committed to give it some TLC
15:25:41 <mlavalle> and that's all I have to say for the time being
15:25:56 <mlavalle> any other comments?
15:26:38 <mlavalle> Next up is https://bugs.launchpad.net/neutron/+bug/1634330
15:26:38 <openstack> Launchpad bug 1634330 in neutron "[RFE] add verification when narrowing allocation pool" [Medium,In progress] - Assigned to zhichao zhu (rtmdk)
15:26:54 <john-davidge> mlavalle: Hey, so I added this to the agenda
15:27:03 <mlavalle> yeah I know
15:27:12 <john-davidge> mlavalle: I know this was triaged back in october but I only saw it this week
15:27:30 <mlavalle> and because you added it, I took a look yesterday at the proposed fix:
15:27:33 <john-davidge> I wanted to discuss the validity of the bug. This seems like expected behavior to me
15:27:43 <mlavalle> #link https://review.openstack.org/#/c/440648
15:27:57 <mlavalle> the fix was abandoned yesterday
15:28:26 <john-davidge> Yes, but another was pushed to replace it
15:28:27 <john-davidge> https://review.openstack.org/#/c/443400/
15:28:39 <john-davidge> I think they misunderstood the merge conflict
15:28:44 <mlavalle> ahhhh
15:28:46 <mlavalle> ok
15:29:05 <mlavalle> I was surprised earlier today when I saw it
15:29:24 <mlavalle> john-davidge: so any insights?
15:30:33 <john-davidge> So, this change would potentially break workflows if people are using allocation pool resizing as it is today
15:31:02 <mlavalle> by changing the api behavior....
15:31:15 <john-davidge> If the user/admin really wants their vms to stay inside the allocation pool, they can just kill and re-add the port
15:31:33 <john-davidge> or at least update the fixed ip
15:31:48 <john-davidge> which is what they'll have to do anyway after this chnage
15:32:00 <john-davidge> so I don't think it adds anything of value
15:32:37 <john-davidge> Does anyone have an opposing view?
15:33:18 <mlavalle> john-davidge: yeah, I've ran into situation before wehre modifying the behavior of the api, seen in isolation, might make sense
15:33:45 <mlavalle> but given that we have users with workflows, it doesn't make sense anymore
15:33:56 <mlavalle> john-davidge: I agree with you
15:34:06 <haleyb> +1
15:34:23 <john-davidge> Great, in that case I'll mark the bug as invlaid with an explanation
15:34:25 <john-davidge> thanks all
15:34:39 <mlavalle> john-davidge: thanks for bringing this up
15:34:49 <john-davidge> mlavalle: no problem
15:34:57 <mlavalle> any other bugs from the team
15:35:01 <mlavalle> ?
15:35:48 <mlavalle> ok, moving on
15:36:05 <mlavalle> #topic Prefix Delegation
15:36:16 <baoli> Hi All
15:36:21 <baoli> https://review.openstack.org/#/c/413137/
15:36:22 <mlavalle> baoli: you are on
15:36:29 <baoli> Thanks Brian for his +2
15:37:32 <haleyb> thanks for working on it
15:37:34 <baoli> Would appreciate review from others
15:37:54 <mlavalle> I nwill review it over the next few days. I added it to my backlog
15:38:19 <baoli> malvalle: thanks!
15:38:37 <mlavalle> It has some code and a lot uf unite tests, LOL
15:39:17 <baoli> malvalle: I have to address some flaws/bugs in the unit test. Thanks haleyb to find a bug in the tests.
15:39:56 <mlavalle> that's ok. it just caught my attention :-)
15:41:05 <mlavalle> any other comments?
15:41:56 <mlavalle> ok, let's move on
15:42:22 <mlavalle> #topic Creating subnets without an allocation pool
15:42:38 <mlavalle> I think this one was also added by john-davidge
15:43:17 <mlavalle> and I took a look at the proposed WIP patchset
15:43:18 <john-davidge> mlavalle: Ah yes, thank you for your comments on the patch after last week's meeting
15:43:36 <mlavalle> the approach seems sensible to me
15:44:03 <john-davidge> mlavalle: I believe Sindhu will be following up soon with a new patch using one of your suggested approaches
15:44:17 <john-davidge> mlavalle: All the references you gace will be very helpful, thank you
15:44:24 <john-davidge> *gave
15:44:42 <mlavalle> john-davidge: great, I'm glad you and Sindhu found it useful
15:44:58 <mlavalle> any other comments?
15:45:46 <john-davidge> mlavalle: not from me, thanks
15:45:55 <mlavalle> ok, let's move on
15:46:03 <mlavalle> #topic Routed Networks
15:46:25 <mlavalle> I added a pointer to the RFE in our etherpad
15:46:30 <mlavalle> so we have it handy
15:47:09 <mlavalle> it was marked as triaged by kevinbenton a week ago
15:47:52 <mlavalle> so let's hope it is discussed soon during the drivers meeting
15:47:57 <mlavalle> any other comments?
15:48:10 <mlavalle> I am ready to help with a spec and the code
15:49:11 <john-davidge> Yes, last week's drivers meeting was cancelled, but hopefully it will be discussed today
15:49:40 <mlavalle> cool
15:49:51 <mlavalle> let's move on
15:50:01 <mlavalle> #topic Open Agenda
15:50:18 <mlavalle> any other topic folks want to bring up to the team's attention?
15:51:35 <john-davidge> Oh I have one
15:52:15 <john-davidge> I commented on #link https://review.openstack.org/#/c/432506/ this week with a concern, but I've been unable to test it
15:52:30 <john-davidge> Is anybody able to verify if that's a real issue?
15:53:58 <mlavalle> john-davidge: I'll try to play with it
15:54:14 <haleyb> i think the problem is the refresh is expensive, did you need someone to test it?
15:54:30 <john-davidge> mlavalle: Thanks :)
15:54:41 <john-davidge> haleyb: So that patch is kind of doing two things at once
15:55:00 <mlavalle> haleyb: go ahead and test it also
15:55:13 <john-davidge> haleyb: The refresh cost is the main thing it's fixing, but in the process it's getting rid of source-specific filtering for RA traffic
15:56:10 <haleyb> john-davidge: right, but only those ports should be sending
15:56:11 <john-davidge> haleyb: My concern is that could lead to VMs picking up prefixes for other subnets, particularly in networks using service subnets
15:57:26 <john-davidge> I could of course be completely wrong, but my dev environment has been all jacked up this week
15:58:20 <haleyb> no, i hadn't thought of that, was always thinking they saw all RAs for subnets on the network
15:59:37 <mlavalle> ok, I think we ran out of time, we can continue this in channel
15:59:41 <john-davidge> haleyb: Right now they'll only get RAs for their own subnet due to the SG rule.
15:59:53 * cdent makes traditional wave to mlavalle
15:59:55 <haleyb> i will test and look at review again
16:00:01 <john-davidge> haleyb: Thanks :)
16:00:11 <mlavalle> #endmeeting