21:02:22 <ihrachys> #startmeeting networking
21:02:22 <openstack> Meeting started Mon Mar 20 21:02:22 2017 UTC and is due to finish in 60 minutes.  The chair is ihrachys. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:02:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:02:26 <openstack> The meeting name has been set to 'networking'
21:02:33 <sshank> Hi
21:02:35 <dasm> \o/
21:02:44 <ihrachys> #link https://wiki.openstack.org/wiki/Network/Meetings Agenda
21:02:47 <dasm> ihrachys: should we wrap it up quickly, because it's late for you? :)
21:02:53 <ihrachys> huh
21:03:21 <electrocucaracha> o/
21:04:00 <ihrachys> #chair kevinbenton
21:04:01 <openstack> Current chairs: ihrachys kevinbenton
21:04:08 <ihrachys> #topic Announcements
21:04:48 * kevinbenton back on phone with laptop issues
21:04:54 <ihrachys> as per https://releases.openstack.org/pike/schedule.html we are 4 weeks from Pike-1
21:05:03 <ihrachys> kevinbenton: do you take over?
21:05:17 <kevinbenton> ihrachys: you continue until you have to leave
21:05:35 <kevinbenton> I might lose connection
21:05:49 <ihrachys> I don't have any announcements other than that
21:05:59 <ihrachys> (primarily because I did not expect to chair that meeting)
21:06:39 <ihrachys> in general, Pike is open so post your RFEs and specs
21:06:47 * electrocucaracha this is time for remembering improvisation classes ihrachys
21:06:51 <ihrachys> postmortem for Ocata is also finally merged: https://review.openstack.org/#/c/425990/
21:07:25 <kevinbenton> Oh, is there anyone who is attending this meeting that would have a conflict with having it an hour earlier?
21:07:47 <electrocucaracha> kevinbenton: today?
21:07:55 <ihrachys> yes, same day but -1h
21:08:06 <kevinbenton> electrocucaracha: that would be pretty hard for today ;)
21:08:10 <reedip> +1 on the -1 hour ...
21:08:13 * manjeets is fine 1pm or 2pm pst
21:08:18 <trevormc> +1
21:08:27 <ihrachys> kevinbenton: that may require some leg work to get a slot in a channel
21:08:34 <electrocucaracha> kevinbenton: I was thinking only this meeting, but you're referring in general
21:08:59 <reedip> electrocucaracha : need a time travelling device for today's meeting
21:08:59 <electrocucaracha> +1
21:09:18 <electrocucaracha> hehe that's true reedip
21:09:27 <dasm> I have one announcement, if I can share it.
21:09:45 <dasm> thanks to hard work of ankur-gupta-f4, we have feature matrix implemented (which got merged today)
21:09:47 <dasm> https://review.openstack.org/#/c/318192/
21:10:03 <kevinbenton> #action kevinbenton to move this Monday meeting hour earlier
21:10:05 <dasm> so, if there is something missing, feel free to update it
21:10:13 <ihrachys> oh, also of note, armax started working on stadium inclusion assessment for Pike here: https://review.openstack.org/#/c/445680/ if you plan to add/remove a project from the stadium, it's worth keeping an eye and helping armax capturing the state
21:10:30 <kevinbenton> dasm: awesome
21:10:42 <manjeets> ankur-gupta-f1, dasm \o/
21:11:17 <kevinbenton> dasm: is this rendered onto docs somewhere?
21:11:21 <ihrachys> kudos to persistence of ankur-gupta-f4, and there is a small follow up on it here: https://review.openstack.org/#/c/447656/
21:11:30 <dasm> kevinbenton: it should be
21:11:35 <boden> https://docs.openstack.org/developer/neutron/feature_classification/feature_classification_introduction.html
21:11:39 <boden> I think that’s it?
21:11:45 <dasm> kevinbenton: ^
21:11:47 <dasm> thanks boden
21:11:57 <kevinbenton> Thx
21:12:05 <ihrachys> looks nice
21:12:23 <ihrachys> kevinbenton: gotta pass the chair to you
21:12:30 <kevinbenton> ihrachys: ok
21:12:36 <kevinbenton> Any other announcements?
21:12:38 <ihrachys> I am dropping, the late meeting clashed with life
21:13:21 <kevinbenton> boden: you want to give a neutron-lib update before we go into bugs?
21:13:31 <boden> kevinbenton sure
21:13:41 <boden> so we released neutron-lib 1.3.0 on 2017-03-16... thanks for everyone's efforts!
21:13:43 <kevinbenton> #topic neutron-lib
21:13:50 <boden> hi, so we released neutron-lib 1.3.0 on 2017-03-16... thanks for everyone's efforts!
21:14:08 <boden> we saw some minor "fall-out" in non-stadium projects, but IMHO these are growing pains towards understanding what this x-project effort will require
21:14:32 <kevinbenton> L2 gateway, right?
21:14:56 <boden> TaaS maybe? I can find the email
21:15:01 <reedip> ??
21:15:22 <boden> http://lists.openstack.org/pipermail/openstack-dev/2017-March/114297.html
21:16:01 <reedip> Will look into it boden
21:16:38 <boden> will probably try for another release in about 2 weeks
21:16:40 <reedip> actually gary's patch pretty much fixed it
21:17:03 <boden> reedip: cool.. can discuss on the patch if needed?
21:17:24 <reedip> I had a similar patch :) but it wasnt picked up :P
21:18:09 <boden> reedip friendly reminder that we have that periodic job to early detect these types of issues
21:18:34 <reedip> boden : ok
21:19:15 <kevinbenton> boden: do you have a link to those job results?
21:19:16 <dasm> #link http://grafana.openstack.org/dashboard/db/neutron-lib-failure-rate
21:19:39 <dasm> #link logs.openstack.org/periodic/
21:19:43 <kevinbenton> Thanks
21:19:45 <boden> kevinbenton, but the job needs to be setup for each project..
21:20:13 <kevinbenton> boden: makes sense
21:20:52 <boden> for the next lib release, IMO we should consider getting https://review.openstack.org/#/c/394244 in.. it will cause some ripple and better now than later
21:21:28 <kevinbenton> boden: ack
21:21:35 <mlavalle> boden: yeah that's a good one
21:21:46 <boden> I hope to have an updated patch out tomorrow
21:21:50 <kevinbenton> How are the two project decomposition efforts coming along?
21:21:59 <kevinbenton> OVN and bgp
21:22:07 <boden> kevinbenton: slow
21:22:32 <kevinbenton> Where is the bottleneck, getting into neutron-lib?
21:22:38 <kevinbenton> Or in the projects?
21:23:12 <boden> so a few things
21:23:48 <boden> we don’t have many contributors really :)
21:24:45 <boden> the neutron consumption patches have also been taking awhile; either due to lack of reviews, or to needing something else in lib in order for it to work (i.e. land new lib patch and wait for release)
21:25:17 <boden> but hopefully once we get the neutron-lib plumbing in place, the later problem will shrink
21:25:46 <kevinbenton> Ok
21:25:59 <kevinbenton> So needs reviews in neutron-lib then?
21:26:18 <boden> as well as the “LibImpact” ones in neutron
21:26:24 <mlavalle> boden: do you need coding or reviewin man power? or both?
21:27:00 <boden> mlavalle: either or both :)  there are some topics that still need exploration in terms of how to decouple
21:27:36 <mlavalle> boden: ok, i'll ping you tomorrow. I can help with both
21:27:47 <boden> mlavalle ok cool
21:27:52 <reedip> mlavalle , boden : count me in as well
21:28:04 <boden> reedip: thanks
21:28:22 <kevinbenton> Ok, anything else for neutron-lib
21:28:40 <boden> kevinbenton: nope
21:28:41 <mlavalle> reedip: ++
21:29:50 <kevinbenton> #topic bugs
21:29:56 <kevinbenton> sindhu: around?
21:30:03 <dasm> sindhu is not here, but she send an update
21:30:06 <dasm> http://lists.openstack.org/pipermail/openstack-dev/2017-March/114344.html
21:30:07 <dasm> *sent
21:30:32 <kevinbenton> Ok
21:30:37 <reedip> sindhu was here just now ...
21:30:41 <kevinbenton> Any bugs we need to discuss here?
21:31:03 <dasm> sindhu mentioned about these two: https://bugs.launchpad.net/neutron/+bug/1673124 and https://bugs.launchpad.net/neutron/+bug/1674443
21:31:03 <openstack> Launchpad bug 1673124 in neutron "test_admin_create_network_keystone_v3 failed with KeyError: 'project_id'" [High,New]
21:31:04 <openstack> Launchpad bug 1674443 in neutron "gate-grenade-dsvm-neutron-linuxbridge-multinode fails on smoke test sometimes " [Undecided,New]
21:31:16 <dasm> reedip: yeah, she was in a shuttle, and just arrived
21:31:38 <manjeets> 1674443 was reported by me this morning
21:31:58 <kevinbenton> manjeets: is it still happening?
21:32:02 <manjeets> job failure rate is increasing
21:32:09 <manjeets> kevinbenton yes
21:32:29 <manjeets> my observation is failure rate is high when vms are placed on rax-node cloud
21:33:02 <kevinbenton> manjeets: ack, can you update the bug with a link to a few of the failures?
21:33:05 <manjeets> I've doubt it may be tempest concurrency and sent a patch to force tempest concurrency for smoke tests
21:33:14 <manjeets> kevinbenton sure
21:33:51 <manjeets> https://review.openstack.org/#/c/447628/ I wanted to test tempest_Concurrency = 2 with rax_node cloud if that works
21:35:31 <kevinbenton> ok
21:35:32 <kevinbenton> https://bugs.launchpad.net/neutron/+bug/1673124
21:35:32 <openstack> Launchpad bug 1673124 in neutron "test_admin_create_network_keystone_v3 failed with KeyError: 'project_id'" [High,New]
21:36:22 <kevinbenton> we'll have to keep an eye on that one
21:37:08 <dasm> kevinbenton: based on link provided by ihrachys, just one hit in last 7 days
21:37:14 <dasm> 2017-03-15T00:17:55.727-05:00
21:37:16 <kevinbenton> dasm: yeah
21:37:24 <reedip> that had one hit in 7 days 5 days ago
21:37:26 <kevinbenton> it's strange that it could be something so non-deterministic
21:37:53 <dasm> true
21:38:22 <kevinbenton> dasm: did you work on the project_id population in the API layer?
21:38:45 <dasm> yes, i did. afair, there were no problems with it.
21:39:06 <kevinbenton> dasm: can you link me to that patch. there may be some path in pecan missing it
21:39:19 <kevinbenton> dasm: as ihar pointed out, we switched to pecan last week finally
21:39:54 <kevinbenton> so any strange response behavior like this could be related to that
21:40:15 <dasm> kevinbenton: lemme find it. maybe this is an issue with pecan. and it could be even expected
21:40:41 <kevinbenton> dasm: ack
21:40:48 <dasm> #link https://review.openstack.org/#/c/335786/
21:40:59 <dasm> this one is main implementation for neutron ^
21:41:06 <kevinbenton> dasm: thx, i'll take a quick look at this later today
21:41:16 <kevinbenton> anyone have any other bugs to discuss?
21:41:17 <dasm> ack, thanks
21:41:34 <mlavalle> yeah, https://bugs.launchpad.net/neutron/+bug/1627424
21:41:34 <openstack> Launchpad bug 1627424 in neutron "FlushError on IPAllocation" [High,In progress] - Assigned to Miguel Lavalle (minsel)
21:42:10 <mlavalle> kevinbenton: I went back to this one after the refactoring of delete_subnet and delete_network in ml2 and the db plugin
21:42:20 <mlavalle> it is still happening
21:42:52 <mlavalle> it happens when you create / update a dhcp port and concurrently you delete the subnet the port has ip address on
21:43:31 <mlavalle> here I trace the port update, that spit the FlushError
21:43:42 <mlavalle> #link http://paste.openstack.org/show/603031/
21:43:57 <mlavalle> and here I trace the concurrent subnet delete, that succeeds:
21:44:12 <mlavalle> #link http://paste.openstack.org/show/603030/
21:44:29 <kevinbenton> mlavalle: actually this reminds me
21:44:32 <mlavalle> I'll spend more time anylyzing this at depth later this week, but I may need your help
21:44:36 <kevinbenton> mlavalle: since you are in that code
21:44:52 <kevinbenton> mlavalle: we need to refactor how ipam is attaching fixed_ips a bit
21:44:59 <kevinbenton> mlavalle: because it's problematic for sqlalchemy as well
21:45:12 <kevinbenton> mlavalle: right now we just sort of insert an ipallocation object into the database
21:45:28 <kevinbenton> mlavalle: rather than appending it to the fixed_ips list on the port that is in memory
21:45:54 <mlavalle> kevinbenton: that's right. I'll ping you tomorrow and we can take it from there
21:46:14 <kevinbenton> mlavalle: so we should just pass a port db object into ipam and have it attach ipallocations directly to that
21:46:27 <kevinbenton> mlavalle: here is some more context on where this is causing problems for enginefacade switch https://review.openstack.org/#/c/434454/
21:46:47 <kevinbenton> mlavalle: ok, we can chat about that tomorrow
21:46:51 <dasm> there is one more bug that requires someone who know the details: https://bugs.launchpad.net/neutron/+bug/1672629
21:46:51 <openstack> Launchpad bug 1672629 in neutron "when port admin-state-up=False, the port status should be DOWN" [Undecided,Incomplete] - Assigned to Yi Zhao (zhaoyi44)
21:47:01 <mlavalle> ok, I will take a look and come back with more specific questions
21:47:09 <mlavalle> kevinbenton: ^^^^
21:47:15 <kevinbenton> mlavalle: +1
21:47:27 <kevinbenton> dasm: I read through that one
21:47:49 <kevinbenton> dasm: it is possible there is a race condition
21:48:09 <dasm> hmm..
21:48:45 <kevinbenton> dasm: if l2 agent finishes shutting down a port
21:48:55 <kevinbenton> dasm: and then later dhcp agent fires dhcp_port_ready to server
21:49:56 <kevinbenton> dasm: i think we probably want a check in this function
21:50:04 <dasm> ok, just wanted to gain visibility for that, to have this explained.
21:50:19 <kevinbenton> dasm: https://github.com/openstack/neutron/blob/ff8b9afc6907e5c8d171c06a86b928ae97f7e158/neutron/plugins/ml2/plugin.py#L207-L232
21:51:38 <kevinbenton> ok
21:51:40 <dasm> thanks. nothing more from sindhu (passed by me :P)
21:51:48 <kevinbenton> #topic open discussion
21:51:57 <kevinbenton> anything to bring up in the last 8 mins?
21:52:34 <mlavalle> kevinbenton: Ihar volunteered last meeting to be bug deputy this week. So we are covered for this week
21:53:17 <kevinbenton> mlavalle: thanks
21:53:20 <reedip> JFYI : new VPNaaS patch for OSC has been brought up and for review (JFYI )
21:53:45 <abhiraut> ^ reviewed :)
21:53:51 <kevinbenton> reedip: link?
21:54:04 <abhiraut> link: https://review.openstack.org/#/c/439978/
21:54:06 <reedip> https://review.openstack.org/#/c/441318/4
21:54:25 <reedip> sorry .. wrong link from my end .. abhiraut posted the right one
21:54:37 <kevinbenton> thanks
21:54:52 <kevinbenton> ok. if that's it, let's end the meeting 5 mins early
21:54:55 <kevinbenton> thanks everyone!
21:55:02 <manjeets> thanks
21:55:09 <kevinbenton> #endmeeting