15:02:28 <haleyb> #startmeeting neutron_l3
15:02:29 <openstack> Meeting started Thu Jan 17 15:02:28 2019 UTC and is due to finish in 60 minutes.  The chair is haleyb. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:02:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:02:32 <openstack> The meeting name has been set to 'neutron_l3'
15:02:37 <mlavalle> o/
15:02:50 <haleyb> mlavalle: hi, didn't expect you :)
15:02:52 <slaweq> hi
15:03:01 <mlavalle> I'm glad to be back
15:03:09 <mlavalle> I'll let you driver, though
15:03:15 <haleyb> #chair mlavalle
15:03:16 <openstack> Current chairs: haleyb mlavalle
15:04:09 <haleyb> #Announcements
15:04:24 <haleyb> #topic Announcements
15:05:13 <xubozhang> hi haleyb and mlavalle
15:05:39 <haleyb> xubozhang: hi
15:05:47 <haleyb> S2 milestone was last week
15:06:51 <haleyb> mlavalle: did we do a release?
15:07:02 <mlavalle> we are going to release this week
15:07:09 <mlavalle> amotoki is working on that
15:07:20 <mlavalle> we are releasing Neutron and the stadium projects
15:07:55 <mlavalle> from now on, when we release Neutron, we will release toegther with the stadium
15:07:55 <haleyb> ack. So we are in S3 which ends first week of March
15:07:58 <amotoki> I am preparing it but neutron-fwaas is broken due to oslo.privsep so I am waiting for the fix a bit
15:08:12 <mlavalle> amotoki: ack. thanks for the update
15:08:15 <haleyb> neutron is broken from oslo.privsep still too
15:08:41 <haleyb> we should probably wait until https://review.openstack.org/#/c/631275/ merges
15:08:42 <slaweq> mlavalle: so maybe we can also wait for this fix/workaround for pyroute2 issue?
15:08:49 <mlavalle> amotoki: didn't mean to rush you ;-)
15:09:03 <mlavalle> slaweq: there is no rush, so yes, it makes sense
15:09:07 <mlavalle> amotoki: ^^^^
15:09:09 <slaweq> thx
15:09:18 <amotoki> thanks
15:09:22 <slaweq> this is patch https://review.openstack.org/#/c/631275/3
15:09:47 <njohnston> o/
15:10:03 <haleyb> it is in the gate right now
15:10:30 <mlavalle> so we this we don't need to go back in privsep vesion, right?
15:11:05 <slaweq> mlavalle: yes, I hope so
15:11:13 <mlavalle> Great!
15:11:23 <amotoki> +1
15:11:44 <haleyb> right, and ralonsoh is continuing the investigation into where the actual issue is
15:11:44 <slaweq> but we will still need some fix in privsep/pyroute2 to make possible work with many threads in privsep
15:11:52 <slaweq> haleyb: exactly :)
15:12:05 <slaweq> but I saw also that bnemec was looking into this today
15:12:12 <slaweq> so maybe he will help us too :)
15:12:40 <mlavalle> so, are we going to unblock the gate after 631275 merges?
15:12:52 <slaweq> mlavalle: yep, I really hope so
15:13:06 <mlavalle> Cool
15:13:08 <slaweq> mlavalle: I tested it 4 times and I didn't saw any error from pyroute2
15:13:13 <liuyulong> Previous privsep seems also has the pyroute2 issue: https://review.openstack.org/#/c/630516/
15:13:18 <mlavalle> Thanks!
15:13:33 <liuyulong> That's slaweq's test patch.
15:14:55 <slaweq> liuyulong: did You saw this "Bad file descriptor" error in jobs on this patch too?
15:17:16 <liuyulong> No, just a quick look, and it has that SSHTimeout and "During handling of the above exception, another exception occurred:
15:17:16 <liuyulong> ".
15:17:48 <liuyulong> Something similar to the Bug Description.
15:18:35 <liuyulong> +1 for that quick fix, a lot of patches will need to rebase on it.
15:18:53 <slaweq> liuyulong: if You are talking about this http://logs.openstack.org/16/630516/6/check/neutron-tempest-plugin-dvr-multinode-scenario/09e8735/compute1/logs/screen-q-l3.txt.gz?level=ERROR#_Jan_16_13_28_16_714173 it comes from rootwrap so it's not the same thing
15:20:54 <liuyulong> They are all pointing to the same bug 1811515? Or maybe I'm missing something.
15:20:59 <openstack> bug 1811515 in neutron "SSH to FIP fails in CI jobs" [Critical,In progress] https://launchpad.net/bugs/1811515 - Assigned to Slawek Kaplonski (slaweq)
15:21:44 <slaweq> in fact this bug's title is maybe wrong as finally it was related to issue with pyroute2
15:21:50 <slaweq> I will update bug title there
15:22:01 <mlavalle> yeah, the title is a little misleading
15:22:11 <slaweq> and we should then check again this dvr multinode job logs and open new bug for it
15:22:13 <mlavalle> it was originally meant as a catch all thing
15:22:24 <slaweq> mlavalle: yes, my bad
15:22:26 <slaweq> sorry
15:22:31 <slaweq> I will update it now
15:22:45 <mlavalle> slaweq: no problem. at the moment we din't know the specific nature of the issue
15:23:05 <mlavalle> so it made sense at that point in time
15:23:11 <slaweq> mlavalle: yes, that is why I did it :)
15:25:10 <mlavalle> slaweq: do I have permission from El Comandante to go back and work on the trunks / dvr bug?
15:25:22 <slaweq> mlavalle: LOL, always :)
15:25:53 <mlavalle> you slapped me pretty hard on the wrists last CI meeting ;-)
15:26:05 <mlavalle> just kidding
15:26:07 <slaweq> really? :)
15:26:15 <slaweq> for sure I can do it harder :P
15:26:32 <mlavalle> LOL
15:26:53 <haleyb> should we move to Bugs then ?
15:27:02 <mlavalle> let mne make a personal announcement
15:27:30 <mlavalle> some of you already know that I am leaving my current employer, Huawei
15:27:54 <mlavalle> my last day is the 25th. I am joining Oath / Verizon on the 28th
15:28:12 <njohnston> COngratulations!
15:28:23 <davidsha> Congratulations!
15:28:25 <mlavalle> This shouldn't have any impact on my community involvement
15:28:30 <slaweq> Congrats mlavalle and good luck in new company :)
15:28:41 <xubozhang> congrats!
15:28:47 <mlavalle> so, no, you can not cheer that you are getting rid of me ;-)
15:29:00 <mlavalle> Thanks everybody
15:29:04 <slaweq> \o/
15:29:18 <njohnston> I've worked for 2 companies that are now part of Verizon, AOL and MCI.  Lots of good people there.
15:29:18 <haleyb> Congrats mlavalle !  and we would never cheer :)
15:29:29 <mlavalle> thanks
15:29:33 <mlavalle> let's move on
15:29:42 <haleyb> #topic Bugs
15:30:35 <haleyb> one new bug this week
15:30:38 <haleyb> https://bugs.launchpad.net/neutron/+bug/1811166
15:30:45 <openstack> Launchpad bug 1811166 in neutron "[RFE] Enforce router admin_state_up=False before distributed update" [Wishlist,New] - Assigned to Matt Welch (mattw4)
15:31:45 <haleyb> mlavalle: looks like we'll get to it at drivers?
15:32:21 <mlavalle> yes, I need to revisit it
15:32:40 <mlavalle> I do my triaging for that meeting Thursday afternoon
15:32:54 <mlavalle> it is my radar screen, though
15:33:07 <haleyb> there was a bug fix i'm remembering
15:34:43 <liuyulong> https://review.openstack.org/#/c/625134/ this one
15:34:56 <haleyb> yes, had just found it
15:35:36 <haleyb> so it was from the StarlingX people it looks
15:36:53 <mlavalle> thanks for digging that
15:37:24 <haleyb> there was another L3 bug filed just yesterday
15:37:31 <haleyb> https://bugs.launchpad.net/neutron/+bug/1812118
15:37:34 <openstack> Launchpad bug 1812118 in neutron "Neutron doesn't allow to update router external subnets" [Undecided,New]
15:38:00 <haleyb> i have not triaged, but the comment in it looks familiar
15:38:10 <haleyb> or maybe not
15:38:50 <haleyb> i also don't know why all those tags were added
15:40:07 <haleyb> so either njohnston (mr bugs this week) or myself will triage/confirm
15:40:31 <njohnston> yes I was just reading that one
15:41:29 <njohnston> going to try to reproduce it
15:41:43 <haleyb> njohnston: ack, thanks
15:42:49 <haleyb> there was one older bug
15:42:51 <haleyb> https://bugs.launchpad.net/neutron/+bug/1806770
15:42:57 <openstack> Launchpad bug 1806770 in neutron "DHCP Agent should not release DHCP lease when client ID is not set on port" [Medium,In progress] - Assigned to Arjun Baindur (abaindur)
15:43:00 <haleyb> https://review.openstack.org/#/c/622634/
15:43:23 <haleyb> oops, https://review.openstack.org/#/c/623066/
15:43:51 <haleyb> i haven't seen an update, so will just work on this myself
15:44:33 <mlavalle> thanks haleyb for picking that one up
15:44:33 <haleyb> there's been a few dhcp fixes recently so the code is somewhat fresh in my head
15:45:30 <haleyb> https://bugs.launchpad.net/neutron/+bug/1811639
15:45:37 <haleyb> https://review.openstack.org/#/c/630297/
15:45:38 <openstack> Launchpad bug 1811639 in neutron "virtual machine cannot get DHCP (DHCPv6) lease for IPV6 due to packet has no checksum." [Medium,In progress] - Assigned to Bence Romsics (bence-romsics)
15:46:09 <haleyb> bence sent a fix already, just stuck due to check queue failures
15:47:12 <haleyb> any other bugs to discuss?
15:47:53 <mlavalle> not from me
15:47:54 <haleyb> #topic Open Discussion
15:48:27 <haleyb> xubozhang: since you joined, did you want to discuss your DVR change?  i can find a link
15:48:38 <xubozhang> yes
15:48:56 <haleyb> https://review.openstack.org/#/c/472289/
15:49:17 <xubozhang> i got failure in grenade-py3
15:49:30 <xubozhang> what is it? and can i run it locally?
15:50:19 <davidsha> xubozhang: It should say it in the logs what tests it's running
15:50:25 <xubozhang> also error in neutron-tempest-dvr
15:51:24 <haleyb> xubozhang: i looks like it can't connect to the floating IP?
15:51:46 <mlavalle> the dvr test is a ssh timeout
15:51:58 <mlavalle> http://logs.openstack.org/89/472289/16/check/neutron-tempest-dvr/32fad73/testr_results.html.gz
15:52:27 <xubozhang> ok, can i run these locally tests?
15:52:37 <davidsha> yup
15:53:00 <davidsha> Command to run is in the log, it might just need a recheck
15:53:10 <davidsha> http://logs.openstack.org/89/472289/16/check/neutron-grenade/fce96e3/job-output.txt.gz#_2019-01-17_01_10_41_459305
15:54:00 <xubozhang> ok thanks david
15:54:28 <mlavalle> xubozhang: how comfortable do you feel digging into check queue logs?
15:55:41 <xubozhang> not much
15:56:34 <mlavalle> mhhh, that's what I thought. Maybe there is previous summits presentation that talks about that
15:56:47 <mlavalle> I'll do some digging
15:57:10 <xubozhang> ok thanks
15:57:23 <davidsha> Actually just on the topic of ovs-dvr, can we get some eyes on the Agent refactor? Just want to make sure we aren't doing anything too insane! https://review.openstack.org/#/c/528336
15:57:56 <mlavalle> can you ping me Monday morning xubozhang? if I don't find anything, I'll give you a brief tutorial
15:58:11 <mlavalle> davidsha: ack
15:58:34 <davidsha> mlavalle: thanks!
15:58:37 <xubozhang> mlavalle, sure. what is your time zone?
15:59:00 <mlavalle> and maybe we need a guide in the docs for new contributors on how to dig in the job results / logs
15:59:18 <mlavalle> I'll try to put something together
15:59:29 <haleyb> anything else, we've got 1 minute
15:59:35 <mlavalle> xubozhang: I'm in Texas, two hours ahead you
15:59:42 <xubozhang> ok
15:59:42 <mlavalle> US central
15:59:57 <mlavalle> I'm done
16:00:11 <davidsha> same, thanks!
16:00:17 <haleyb> #endmeeting