14:00:02 <jlibosva> #startmeeting networking
14:00:03 <openstack> Meeting started Tue Jan 31 14:00:02 2017 UTC and is due to finish in 60 minutes.  The chair is jlibosva. Information about MeetBot at http://wiki.debian.org/MeetBot.
14:00:05 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
14:00:07 <openstack> The meeting name has been set to 'networking'
14:00:08 <john-davidge> \o/
14:00:10 <jlibosva> Hello friends!
14:00:22 <dasanind> o/
14:00:37 <korzen> hi
14:00:43 <jlibosva> #topic Announcements
14:00:49 <bcafarel> howdy
14:00:55 <ajo> o/
14:01:04 <jlibosva> We got some release related announcements
14:01:16 <jlibosva> We're now in the release candidate phase this week
14:01:20 <jlibosva> #link http://lists.openstack.org/pipermail/openstack-dev/2017-January/111074.html
14:01:27 <amotoki> o/
14:01:32 <jlibosva> Which means
14:01:43 <jlibosva> We can't merge more features - unless it's been granted with feature freeze exception (FFE)
14:01:45 <dalvarez> o/
14:01:51 <haleyb_> hi
14:01:51 <jlibosva> #link http://lists.openstack.org/pipermail/openstack-dev/2017-January/111083.html
14:02:04 <hichihara> o/
14:02:05 <ajo> makes sense
14:02:28 <jlibosva> We'll talk more about Ocata in other topic
14:02:42 <jlibosva> next version will be Pike and in order to make it awesome, as you know, there will be a project gathering in middle Feb
14:02:54 <jlibosva> #link http://lists.openstack.org/pipermail/openstack-dev/2017-January/110040.html
14:03:09 <jlibosva> We already have some topics but still feel free to add more items to the etherpad https://etherpad.openstack.org/p/neutron-ptg-pike
14:03:21 <ajo> \m/
14:03:42 <jlibosva> I'm looking forward to see you there :)
14:03:53 <jlibosva> Last week was the week of PTL candidacy nomination
14:03:58 <jlibosva> We have two briliant candidates :)
14:04:02 <jlibosva> The elections are supposed to be this week
14:04:18 <jlibosva> Don't forget to use your ballot, you should get an email once elections are started
14:04:27 <jlibosva> Anybody has anything else to announce?
14:04:57 <jlibosva> let's move on then
14:04:59 <jlibosva> #topic Blueprints
14:05:22 <jlibosva> As already advertised, we're in release candidate phase so we should be focusing on bugfixes scheduled to this milestone
14:05:27 <jlibosva> #link https://launchpad.net/neutron/+milestone/ocata-rc1
14:05:46 <jlibosva> As usual, blueprints that didn't make it to Ocata can be proposed/deferred to Pike.
14:05:59 <jlibosva> ihrachys: korzen do you want OVO topic?
14:06:13 <ihrachys> jlibosva: nah, I think it's better to skip since we are in freeze
14:06:18 <jlibosva> ack
14:06:23 <jlibosva> #topic Bugs and gate failures
14:06:38 <jlibosva> We had two major gate breakages in functional job last week
14:06:42 <jlibosva> Both are solved by now
14:06:56 <jlibosva> but we'd like to get a better visibility of failures in gate
14:07:02 <jlibosva> #link http://lists.openstack.org/pipermail/openstack-dev/2017-January/111214.html
14:07:18 <jlibosva> Please read the above and please keep in mind to look at failures before issuing recheck on your patches
14:07:18 <ihrachys> oh have we solved ovs native one?
14:07:34 <jlibosva> ihrachys: no, that was not considered major by me :)
14:07:39 <ihrachys> haha
14:07:40 <ihrachys> ok
14:07:47 <jlibosva> ihrachys: the functional is now about 10% in check queue
14:07:59 <jlibosva> if it would stay it like this, it would be awesome
14:08:19 <haleyb_> there is a privsep fix working its way through that will help in gate as well
14:08:33 <jlibosva> haleyb_: do you have a link?
14:08:44 <ajo> hmm
14:08:54 <haleyb_> bug 1660305
14:08:54 <openstack> bug 1660305 in neutron "DVR multinode job fails over 20 tests" [High,In progress] https://launchpad.net/bugs/1660305 - Assigned to Oleg Bondarev (obondarev)
14:08:56 <ajo> this reminds me (about ovs native) that I left this half way done: https://review.openstack.org/#/c/425623/
14:09:20 <ajo> I wonder if it's of any use, or if it's actually going to affect anything after latest discussions.
14:09:44 <jlibosva> haleyb_: thanks, I see it's already approved
14:09:51 <haleyb_> although maybe that job isn't voting, but the privsep failures are random
14:09:56 <anilvenkata> jlibosva, ajo ihrachys haleyb As you know, for DVR+HA gate job, this patch is needed https://review.openstack.org/#/c/383827/
14:10:07 <jlibosva> ajo: I saw iwamoto updated the bug with some interesting findings
14:10:14 * ajo reads
14:10:32 <anilvenkata> jlibosva, ajo haleyb but it is not getting devstack-gate core reviewers attentions
14:11:05 <jlibosva> anilvenkata: yeah, we would need to go to openstack-infra and poke people there
14:11:37 <anilvenkata> jlibosva, I am regularly pinging clarkb and sdague
14:11:47 <jlibosva> anilvenkata: thanks
14:11:50 <anilvenkata> for its review
14:12:15 <jlibosva> we also have some failures in other tempest jobs, one of the most critical I have seen are libvirt crashes
14:12:15 <ihrachys> ajo: it seems worth exploring. one thing would be first to check if we see the same failures now that a patch from kevinbenton landed that could influence things
14:12:50 <jlibosva> ihrachys: which patch?
14:12:57 <ajo> which patch?
14:13:09 <ihrachys> https://review.openstack.org/#/c/426032/
14:13:17 <ihrachys> "It's not clear that this was a cause of the bug though, so
14:13:18 <ihrachys> this fix is only related."
14:14:40 <jlibosva> hm, merged very recently .I was wondering whether it can be the cause that functional failure rate is "so low" :)
14:15:19 <jlibosva> we'll see.
14:15:21 <ajo> hmmm :)
14:15:27 <ajo> kevinbenton++
14:15:35 <jlibosva> I also wanted to leave these two bugs related to mentioned libvirt crashes
14:15:39 <jlibosva> https://bugs.launchpad.net/openstack-gate/+bug/1638982 and https://bugs.launchpad.net/nova/+bug/1643911
14:15:39 <openstack> Launchpad bug 1638982 in OpenStack-Gate "libvirtd randomly crashes on xenial nodes with "realloc(): invalid next size"" [Undecided,Confirmed]
14:15:40 <openstack> Launchpad bug 1643911 in OpenStack Compute (nova) "libvirt randomly crashes on xenial nodes with "*** Error in `/usr/sbin/libvirtd': malloc(): memory corruption:"" [Medium,Confirmed]
14:15:47 <jlibosva> just in case you get hit by those in your patches
14:16:00 <ajo> ouch
14:16:28 <jlibosva> ajo: you were a bug deputy last week, right?
14:16:38 <ajo> somebody is smashing the heap :)
14:16:44 <ajo> jlibosva affirmative
14:16:54 <jlibosva> ajo: do you want to highlight some bugs?
14:17:02 <jlibosva> how was the week?
14:17:09 <ihrachys> not sure what we can do with libvirtd. :( crying aloud?
14:17:21 <ajo> let me check if there is something missing beyond what you said.
14:17:26 <ajo> ihrachys may be get the ubuntu guys update it, or move to centos :P
14:17:40 <ajo> jlibosva: The memory  / OOM issues,
14:17:57 <ajo> https://bugs.launchpad.net/neutron/+bug/1656386
14:17:57 <openstack> Launchpad bug 1656386 in neutron "Memory leaks on Neutron jobs" [Critical,Confirmed] - Assigned to Darek Smigiel (smigiel-dariusz)
14:18:02 <ajo> but not sure there's anything we could do about that
14:18:24 <jlibosva> ihrachys: I noticed that the compute one is "Medium" importance so I cried there for bump
14:18:26 <ajo> kevinbenton increased swapiness, but still OOM is invoked while there's only 700k of 7G of swap used
14:18:56 <korzen> I could use more reviews on https://review.openstack.org/#/c/273546 Add support for Keepalived VRRP health check to get this merged in Ocata
14:19:20 <ajo> there was another one related to linuxbridge conntrack clearing, 1 sec
14:19:27 <ajo> https://bugs.launchpad.net/neutron/+bug/1657260
14:19:27 <openstack> Launchpad bug 1657260 in neutron "Established connection don't stops when rule is removed" [Critical,In progress] - Assigned to Slawek Kaplonski (slaweq)
14:19:57 <haleyb_> that was approved but failing gate
14:19:59 <ajo> it's WIP by slawek, in iptables  non-hybrid firewall driver the zones aren't used, but we try to clear the entries -in a zone- so it fails
14:20:03 <jlibosva> korzen: thanks, it's scheduled to rc1 so hopefully it will get proper attention
14:20:08 <haleyb_> i thikn it was approved
14:20:25 <haleyb_> yup
14:21:03 <jlibosva> hmmm :(
14:21:33 <jlibosva> I see that linuxbridge tempest job is about 35% failure rate
14:23:04 <jlibosva> anybody has any other bug that deserves attention?
14:23:14 <jlibosva> or bugfix
14:23:45 <korzen> https://review.openstack.org/#/c/420693
14:23:53 <jlibosva> korzen: thanks
14:24:00 <korzen> haleyb_, can you +W on ^ ?
14:24:23 <jlibosva> haleyb_: yep, I set the bug to rc1, so I think we're good to go
14:24:32 <ihrachys> yeah, and it's test only change
14:24:43 <ajo> jlibosva  where did you see linuxbridge jobs on 35% failure rate?
14:25:03 <jlibosva> ajo: sorry, I was looking at multinode-nv
14:25:22 <ajo> ahh
14:25:23 <ajo> :)
14:25:29 <jlibosva> the voting is at 13%, sorry :)
14:25:46 <jlibosva> ajo: thanks for fixing me :)
14:26:06 <ajo> jlibosva np, I was looking for the specific failures, and I found a much lower rate, so I thought I was looking at the wrong place actually
14:26:42 <jlibosva> ajo: I feel like the linuxbridge job is failing only on the patches that I'm looking at. I see it failing almost the whole time, so I wasn't surprised by 35%
14:27:03 <ajo> :)
14:27:04 <jlibosva> if there are no other bugs or patches, I'd like to get to getting a new deputy for this week
14:27:21 <ajo> you were biased
14:27:29 <jlibosva> I would like to take the next week, if there are no objections :)
14:28:11 <ihrachys> I don't think anyone would object...
14:28:13 <ajo> come'on I offer a beer @ atlanta :P :)
14:28:14 <jlibosva> but we still need a hero for this week
14:29:16 <jlibosva> beer for this week bug deputy in atlanta from ajo. how could anyone say no to this?
14:29:29 <hichihara> The beer is mine!
14:29:35 <jlibosva> hichihara is the winner!
14:29:38 <jlibosva> thanks!
14:29:44 <hichihara> :)
14:29:48 <jlibosva> I'll update the wiki after meeting :)
14:29:48 <ajo> hichihara you got it!! :D thanks :) Spanish or american beer? ;-)
14:30:11 <hichihara> American beer please :)
14:30:27 <ajo> consider it done :D
14:30:53 <jlibosva> #action ajo to buy hichihara an american beer in atlanta
14:31:01 <hichihara> lol
14:31:15 <jlibosva> #topic Docs
14:31:21 <jlibosva> john-davidge: hello!
14:31:22 <ajo> %)
14:31:42 <john-davidge> jlibosva: Hello!
14:31:52 <jlibosva> john-davidge: do you have any updates wrt docs
14:32:36 <john-davidge> jlibosva: Nothing specific from me on docs this week. I've been discussing some possible to changes to how we deal with the DocImpact tag with the new Docs PTL, but we don't have anything to share just yet.
14:32:52 <jlibosva> john-davidge: ok, thanks :)
14:33:06 <jlibosva> so maybe we can move to the next topic
14:33:15 <jlibosva> #topic Transition to OSC
14:33:28 <jlibosva> amotoki: hi :) do you want to update us?
14:33:51 <amotoki> I checked the current status of OSC transition. 95 of 126 are completed.
14:34:22 <john-davidge> amotoki: Great progress!
14:34:26 <amotoki> As far as I checked, the biggest remaining one is FIP associate/disassociate.
14:34:37 <amotoki> this is the progress chart https://docs.google.com/spreadsheets/d/1ueCvIKUfpUaLFm_3YTT4MkuTV-PYIdeErnt7Wleshl0/edit#gid=0
14:34:47 <jlibosva> w00t
14:35:02 <reedip_> o/
14:35:08 <amotoki> I think we can ship neutron CLI with deprecation notice (as we see now)
14:35:40 <jlibosva> amotoki: great news! thanks
14:35:42 <amotoki> other than FIP (dis)associate, most commands are admin-specific or less used like flavor.
14:36:00 <reedip_> for the FIP , I think we have reached a conclusion
14:36:08 <amotoki> thanks all for folks working on OSC transition!
14:36:08 <reedip_> but minor adjustments need to be done
14:36:31 <amotoki> reedip_: yeah, I saw it
14:36:59 <amotoki> but I am not sure OSC team has a plan for a new release in Ocata time span.
14:37:25 <amotoki> reedip_: do you have any?
14:38:24 <reedip_> None as of now that I know amotoki
14:38:41 <reedip_> OSC has almost been finished for Ocata
14:38:54 <jlibosva> amotoki: reedip_ thanks for updates and good news :)
14:39:05 <reedip_> :)
14:39:15 <jlibosva> can we move the next topic?
14:39:22 <amotoki> sure
14:39:27 <jlibosva> oki
14:39:28 <jlibosva> #topic Neutron-lib and planned neutron refactoring
14:39:47 <jlibosva> boden: hi, do you want to say a few things about neutron-lib?
14:40:59 <jlibosva> maybe that means "no" :)
14:41:43 <jlibosva> we have two things added to the on demand agenda but I feel like we've already had the "Disable security group filter refresh on DHCP port changes" one
14:41:56 <jlibosva> and I also don't see mdorman online
14:42:18 <boden> jlibosva sorry I stepped away
14:43:16 <jlibosva> boden: I'm now not sure if you stepped away from keyboard or neutron-lib :)
14:43:30 <boden> things are pretty idle on the neutron-lib front at least in terms of landing patches. I think this is mainly beause we cut stable 1.1.0 for Ocata and folks are focusing on getting neutron out the door
14:44:06 <boden> so not much to say really. There are some “lib impact” patches, but IMO they are obviously not candidates for ocata so we can talk about them later
14:45:04 <jlibosva> boden: ok, thank you for updates :)
14:45:23 <jlibosva> can we move on?
14:45:38 <boden> jlibosva yes thanks
14:45:47 <jlibosva> good
14:45:49 <jlibosva> #topic Logging API for security groups
14:46:14 <jlibosva> rossella_s: you added this topic, right?
14:46:33 <rossella_s> jlibosva, nope
14:46:53 <jlibosva> rossella_s: hmm, there is your name written on the meeting wiki page :)
14:47:40 <rossella_s> jlibosva, that's probably from last week, annp wanted to talk about it
14:48:04 <jlibosva> rossella_s: aha, I checked last meeting logs and didn't see the topic there
14:48:13 <jlibosva> and I don't see annp around
14:48:53 <jlibosva> so maybe we can defer the discussion
14:48:59 <jlibosva> anybody has anything else to bring up?
14:49:01 <rossella_s> he's sleeping I guess, anyway since we are discussing it, it would be good to get more eyes on  https://review.openstack.org/#/c/203509/
14:49:10 <rossella_s> and see how to move that forward
14:50:37 <jlibosva> I saw the spec got some discussions lately on the gerrit
14:51:02 <jlibosva> rossella_s: do you think starting a ML thread would help to proceed?
14:51:14 <jlibosva> or gerrit is sufficient?
14:52:50 <ihrachys> gerrit should be fine, it get enough eyes it seems?
14:54:10 <jlibosva> maybe more people from control plane core team could have a look too. It seems armax and yamamoto are the active ones there
14:54:34 <jlibosva> oh, I also see hichihara :)
14:55:04 <hichihara> yes :) I'm active on the spec.
14:55:09 <jlibosva> we have 5 minutes. anybody has some topic to discuss?
14:55:41 <jlibosva> or we could spend those 5 minutes by telling non-political jokes :)
14:55:46 <jlibosva> or we can end earlier
14:55:54 <hichihara> I have a small topic. I'm working on https://review.openstack.org/#/c/413662/
14:56:13 <hichihara> ihrachys amotoki: What do you think about it?
14:56:52 <hichihara> ihrachys amotoki: Should I skip it on Pike and discuss more?
14:56:55 <jlibosva> the bug is scheduled to mitaka rc1, I think we missed the deadline there :)
14:56:58 <ihrachys> it still doesn't give a discoverable way (api level) to learn if tags for those resources are supported.
14:57:24 <ihrachys> hichihara: the change itself should not be huge, but we need the empty extension
14:57:28 <amotoki> the topic is about API discoverability on how we can add more resources supported by tag API.
14:57:53 <amotoki> according to the current convention, we need to add some shim extension.
14:57:57 <hichihara> I understand
14:58:33 <hichihara> So that's extensions/tag-ext?
14:58:40 <ihrachys> yea
14:58:49 <amotoki> i understand 'tag-xxx', 'tag-xxx2' .... are ugly convention, but tag-ext sounds reasonable now
14:59:20 <hichihara> Iihrachys amotoki: 'm OK. I'll do. Thanks
14:59:35 <jlibosva> hichihara: thanks :)
14:59:49 <jlibosva> thanks everyone for showing up :)
15:00:04 <ihrachys> and we have CI meeting in 1h!
15:00:05 <jlibosva> #endmeeting