21:00:57 <armax> #startmeeting networking
21:00:58 <openstack> Meeting started Mon Oct 12 21:00:57 2015 UTC and is due to finish in 60 minutes.  The chair is armax. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:58 <regXboi> there we go
21:01:00 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:01:00 <dougwig> o/
21:01:02 <johnsom> o/
21:01:03 <openstack> The meeting name has been set to 'networking'
21:01:09 <armax> hi fellas
21:01:13 <emagana> hello all!
21:01:17 <regXboi> moo
21:01:18 <armax> that was an interesting start
21:01:19 <ZZelle_> Hi neutrinos
21:01:29 <armax> ZZelle_: that’s my line!
21:01:49 <mlavalle> armax: is that official? are we neutrinos now?
21:01:50 <ZZelle_> :)
21:01:52 <carl_baldwin> hi
21:01:55 <Swami> hi
21:01:58 <ihrachys> \o_
21:02:01 * regXboi resists puns concerning neutrinos and flavor changing
21:02:01 <markmcclain> o/
21:02:10 <salv-orlando> mlavalle: only if you can travel faster than light
21:02:21 <salv-orlando> allegedl
21:02:23 <armax> oh well
21:02:26 <dougwig> *cough* off-topic *cough*
21:02:31 <armax> let’s reserve this yak shaving for another time
21:02:36 <armax> cough
21:02:37 <regXboi> salv-orlando: no - they are bounded by c still
21:02:39 <mlavalle> salv-orlando: ah, that's easy
21:02:50 <amuller> https://review.openstack.org/233667 is finally merged so the gate is unborked, yay
21:02:58 <regXboi> amuller: ++
21:03:09 <armax> #announcement The gate is happy
21:03:12 <salv-orlando> regXboi: as you know I am ignorant. I stop myself at the first results from google search but do not bother reading the 2nd
21:03:16 <ihrachys> amuller: for the next 2 hours you mean, right? it can't be unborked for too long
21:03:26 <regXboi> ihrachys: now you've jinxed it!
21:03:28 <amuller> ihrachys: We do have traditions to maintain
21:03:33 <armax> #topic Announcements
21:03:34 <emagana> good job amuller
21:03:38 <armax> #link https://wiki.openstack.org/wiki/Network/Meetings Agenda
21:03:53 <Nakato> On tat topic, after the meeting I'd love to chat to a couple cores about moving some patches to stop the gate from breaking.
21:04:35 <dougwig> Nakato: bring it up in open discussion.
21:04:35 <armax> Nakato: yes, we have Sachi working on this area
21:04:44 <sc68cal> o/
21:04:53 <armax> RC2 is out
21:04:53 <regXboi> sc68cal: yo!
21:04:54 <armax> https://launchpad.net/neutron/+milestone/liberty-rc2
21:04:57 <tchaypo> Nakato is Sachi :)
21:05:19 <armax> tchaypo: I figured something was up
21:05:19 <johnsom> Wahoo
21:05:22 <armax> good to know
21:05:39 <regXboi> armax: as of right now, rc3 is *NOT* planned, correct?
21:05:53 <armax> I should have looked at the author full email  on his patch
21:05:56 <armax> regXboi: not that I am aware
21:06:00 <regXboi> armax: thx
21:06:05 <armax> we’re running out of time
21:06:21 <armax> #link https://wiki.openstack.org/wiki/Liberty_Release_Schedule
21:06:39 <armax> unless something is burning so bad
21:06:47 <armax> I can’t see ourselves spinning RC3 at this point
21:07:24 <armax> but if you do spot something, please alert the authorities
21:07:26 <tchaypo> armax: s/his patch/her patch/ :)
21:07:42 <armax> tchaypo: ok, I should be publicly ashamed
21:07:45 <armax> my bad
21:08:07 <armax> let’s go bad to the other announcements
21:08:14 <armax> haleyb: ping
21:08:46 <armax> #link https://review.openstack.org/#/c/233306/
21:08:58 <haleyb> armax: pong
21:08:59 <armax> this is the first patchset of get-me-a-network spec
21:09:07 <njohnston> o/
21:09:30 <haleyb> it is not working, i'm still wroking on the issues
21:09:31 <armax> haleyb: all eyes are on you
21:09:31 <anteaya> yay
21:09:32 <regXboi> it wasn't a complete jackpot
21:09:37 <anteaya> haleyb: it exists
21:09:39 <anteaya> yay
21:09:46 <anteaya> \o/
21:09:47 <Swami> haleyb: good to see the first oen out
21:09:48 <anteaya> thank you
21:09:52 <sc68cal> yaaaay
21:09:58 <haleyb> having never done an extension i'm still learning....
21:10:09 <armax> haleyb: you’re not alone
21:10:13 <sc68cal> oh it's ... awful
21:10:14 <armax> haleyb: reach out if you have doubts
21:10:25 * sc68cal can't wait for pecan
21:10:38 <armax> but please, pretty please…let’s not let it drag
21:10:44 <armax> ideally we’d have something functional for M1
21:10:52 <kevinbenton> sc68cal: pecan is in, it won't fix that :)
21:10:53 <anteaya> tell me how I can help
21:10:55 <anteaya> and thank you
21:11:01 <kevinbenton> sc68cal: because we had to be backwards compatible
21:11:29 <armax> #link https://launchpad.net/neutron/+milestone/mitaka-1
21:11:50 <markmcclain> kevinbenton: we should probably discuss a shim that we can deprecated and remove in the O-release
21:11:52 <armax> which reminds we I need to start track a few blueprints that are part of liberty backlog
21:12:05 <armax> #action armax to pull his act together and do some PTL work
21:12:17 <kevinbenton> markmcclain: +1
21:12:41 <anteaya> armax: that's not very actionable
21:12:41 <salv-orlando> sc68cal: pecan is not going to remove extensions... not yet at least
21:12:54 <armax> anteaya: I know, that’s why I wrote it like that :)
21:12:59 <anteaya> ha ha ha
21:13:02 <armax> anteaya: so I can’t be held accountable?
21:13:07 <anteaya> nice work
21:13:09 <armax> last reminder: http://lists.openstack.org/pipermail/openstack-dev/2015-October/076817.html
21:13:12 <armax> #link http://lists.openstack.org/pipermail/openstack-dev/2015-October/076817.html
21:13:36 <armax> at that link you can find pointers to the summit agenda
21:14:22 <armax> if you have questions, please feel free to reach out in the neutron channel
21:15:33 <armax> any other announcement that the team wants to share?
21:16:44 <armax> oh well
21:16:46 <armax> moving on
21:17:01 <armax> #topic Bugs
21:17:21 <armax> this week’s deputy was ihrachys
21:18:07 <ihrachys> oh yeah, the failure of the week. not sure I need to present some kind of random data points here?
21:18:28 <armax> ihrachys: I did capture some numbers on
21:18:31 <armax> #link https://wiki.openstack.org/wiki/Network/Meetings#Bugs
21:19:04 <armax> of those you worked on that needed follow up
21:19:12 <armax> I tagged them with ‘needs-attention’
21:19:29 <armax> so if anyone in the team or the next deputy wanna keep an eye on those that would be good
21:19:42 <ihrachys> yeah. so I think the main point here is that we have a lot of bugs reported, we tag them, but we also expect teams to grab their tags and clean/fix
21:20:02 <armax> ihrachys: of those that came in the system during last week
21:20:17 <armax> I could only find 2 bugs that were ‘not processed'
21:20:21 <armax> so good work!
21:20:22 <ihrachys> one thing that is useful for any deputy is subscribing to new bug updates on LP. with that, it's easy peasy.
21:20:40 * regXboi already there
21:20:47 * anteaya heads out to yet another thanksgiving dinner
21:21:02 <ihrachys> regXboi: so you are the next deputy I guess. correct armax?
21:21:08 <armax> ihrachys: correct
21:21:17 <armax> is there anyone willing to step up for next week already?
21:21:25 * regXboi looks for the ceremonial passing of the swatter
21:21:25 <armax> I mean the one after this one
21:21:44 <armax> not all at once?
21:22:06 <armax> …come on…you know you want to do it...
21:22:17 <markmcclain> armax: I can help the week of the 19th
21:22:25 <armax> markmcclain: super
21:22:33 <armax> I’ll put you down for the week of Oct 19th
21:22:48 <armax> in the meantime ZZelle_ flagged a number of stale bugs
21:22:53 <armax> that were a year or more old
21:23:08 <armax> I had a further pass on a number of ‘opinion’ bugs too
21:23:43 <armax> the intention if that in the next couple of months we’ll be dealing with a more manageable number of bugs
21:23:54 <armax> #action markmcclain  to be bug deputy for the week of Oct 19th
21:24:41 <armax> markmcclain: wise move in chosing the week before the summit ;)
21:25:22 <armax> any bug worth raising attention during this meeting?
21:25:50 <markmcclain> haha... I think the week after the summit is more ideal
21:26:21 <ihrachys> one thing is: https://bugs.launchpad.net/neutron/+bug/1504647 which apparently broke upgrade for linuxbridge
21:26:21 <openstack> Launchpad bug 1504647 in neutron "Ensure new interface hashing does not break upgrades" [High,In progress] - Assigned to Sean M. Collins (scollins)
21:26:44 <emagana> markmcclain: yeap.. historically one of the most relaxing weeks for OpenStack in general
21:26:49 <ihrachys> I guess it highlights we need grenade for the driver
21:27:06 <regXboi> armax: I'm seeing four bugs in LP marked High but are unassigned
21:27:26 <armax> regXboi: can you share the links?
21:27:39 <regXboi> bug 1349617
21:27:39 <openstack> bug 1349617 in neutron "SSHException: Error reading SSH protocol banner[Errno 104] Connection reset by peer" [High,Incomplete] https://launchpad.net/bugs/1349617
21:27:47 <regXboi> bug 1484344
21:27:48 <openstack> bug 1484344 in neutron "Decompose Cisco l3 and cfg_agent service plugins" [High,Triaged] https://launchpad.net/bugs/1484344
21:27:54 <regXboi> bug 1496204
21:27:54 <openstack> bug 1496204 in neutron "DVR: no need to reschedule_router if router gateway update" [High,Confirmed] https://launchpad.net/bugs/1496204
21:28:00 <ihrachys> and another bug worth lbaas team attention is: https://bugs.launchpad.net/neutron/+bug/1504465 looks like some gate race in octavia driver
21:28:00 <openstack> Launchpad bug 1504465 in neutron "neutron_lbaas.tests.tempest.v2.api.test_health_monitors_non_admin.TestHealthMonitors failed to clean up loadbalancer" [Medium,Confirmed]
21:28:01 <regXboi> bug 1505108
21:28:01 <openstack> bug 1505108 in neutron "neutron_lbaas.tests.tempest.v2.api.test_listeners_admin failed _check_status_tree" [High,Confirmed] https://launchpad.net/bugs/1505108
21:28:23 <armax> regXboi: ok, I’ll look into those…I think some of these may need revised severity
21:28:36 <regXboi> armax: ack - I'll find you tomorrow AM and we can discuss
21:28:50 <armax> sounds good
21:29:08 <amotoki> how is the bugs sheet updated? by a manual script?
21:29:20 <amuller> armax: We may need to standardize what do the low/med/high/crit priorities actually mean, so that bug deputies are in sync
21:29:27 <ihrachys> and why do we need the sheet at all?
21:29:36 <amuller> a decomp bug should not be high priority as it only affects users of that driver for example
21:29:55 <armax> amotoki: at the moment I pull the data out of LP and put into a sheet
21:30:02 <regXboi> amuller: IIRC, there is a discussion of that... somewhere
21:30:05 <armax> ihrachys: the only reason why I do that is that filtering is better
21:30:05 * regXboi goes and looks
21:30:21 <armax> ihrachys: I have an action to reach out the LP guys to see whether I am missing something
21:30:35 <armax> ihrachys: afaik I can’t filter bugs by date through the LP dashboard
21:30:46 <armax> amuller: agreed
21:30:47 <ihrachys> armax: I do it by number
21:30:53 <armax> ihrachys: true
21:30:56 <ZZelle_> armax, you can order them by date
21:31:03 <armax> numbers are monotic
21:31:30 <armax> but say I want to see the ones that happened since a certain date, is that possible?
21:31:33 <regXboi> amuller: how about this https://wiki.openstack.org/wiki/Bugs#Importance
21:31:53 <armax> amuller: yes, that’s the link that is reported in the bug guidelines
21:31:56 <armax> btw
21:32:15 <armax> the importance as described in LP is totally misleading
21:33:05 <ZZelle_> armax, nope only order them by date and read bugs until you reach the certain date :s ... i had to implement such filter in the script we used last week
21:33:07 <regXboi> armax: is it worth trying to walk through and re-categorize based on the wiki guidelines?
21:33:19 <amuller> regXboi: only unassigned bugs
21:33:52 <regXboi> amuller: I think that can be arranged :)
21:34:02 <armax> ZZelle_: true, but I can’t still filter easily and it’s still pretty prone
21:34:22 <amuller> generally speaking the importance column is often times useless / overrated. It can be useful if someone is looking for a bug to solve, or to tag critical bugs that need faster iteration, but if someone is already working on a bug, then some other guy tagging it as 'high' or 'medium' literally makes no difference
21:34:30 <salv-orlando> armax: LPs just cannot do that kind of query. It is a bit limited, and afaict new features are not going developed (and existing ones are not being improved)
21:34:43 <armax> ZZelle_: I appreciate that a sheet is not ideal, I am trying to see whether I am missing something from the tool
21:35:03 <armax> amuller: the importance field should be uself in telling us what to fix first
21:35:07 <ihrachys> armax: well, if it's targeted for a release, and we actively review stuff for that release, it makes sense.
21:35:11 <armax> amuller: if it is chosen wisely
21:35:30 <ihrachys> sorry, that was for amuller
21:35:42 <armax> ihrachys: understood
21:36:01 <armax> ihrachys: according to the guidelines defined in https://wiki.openstack.org/wiki/Bugs#Importance
21:36:44 <armax> I’d say, one would focus attention on those that have no workaround first
21:37:19 <armax> amuller: but sometimes the one with workaround are the easier to fix..etc…so to some degree I am not fixating on importance as much
21:38:48 <armax> ok, we spent quite a bit on this section…we’ll refine/tweak how we go about bugs management in the forthcoming weeks
21:38:57 <armax> and thanks ihrachys and regXboi for being the first two guinea pigs
21:39:00 <armax> oops
21:39:02 <armax> volunteers
21:39:14 <regXboi> armax: oink oink oink
21:39:20 * ihrachys scratches his tail
21:39:24 <armax> :)
21:39:28 <armax> #topic Docs
21:39:31 <armax> emagana: any update?
21:40:27 <armax> going once, going twice,...
21:40:31 <emagana> armax: here
21:40:48 <emagana> Just to mention that we have some good people joining the networking guide team
21:40:53 <emagana> #link https://wiki.openstack.org/wiki/Documentation/NetworkingGuide
21:41:08 <armax> we should praise these people, care to share their nick?
21:41:09 <emagana> I hope we can get more.
21:41:41 <emagana> I will update the wiki with the nick names/IRC
21:41:48 <armax> emagana: thanks
21:42:02 <emagana> armax: nothing more!
21:42:09 <armax> #topic Open Discussion
21:42:19 <armax> We had a couple of items put on the list during the past week
21:42:22 <armax> first one up
21:42:25 <armax> regXboi: ping
21:42:37 <armax> your ‘Is it time to assemble a performance team'
21:42:39 <regXboi> armax: pong
21:42:58 <regXboi> yes - that was an open question that I've been thinking about over the past few weeks
21:43:09 <armax> I guess this initiave overlaps to some degree with what rally people have been doing
21:43:19 <regXboi> armax: yes and no
21:43:37 <armax> are there any people commiting to the rally subtree listening/participating?
21:43:43 <armax> regXboi: ok, care to elaboratE
21:43:44 <armax> ?
21:43:54 <regXboi> I see rally subtree as pointing the way
21:44:12 <regXboi> but then there is the triage to find the issues the rally subtree exposes
21:44:25 <armax> agreed
21:44:28 <regXboi> and in my mind the "perf team" is about this low level triage
21:44:56 <armax> anotehr aspect would be to make sure we prevent performance regressions from sneaking in in the first place
21:44:58 <salv-orlando> it's like gate triage, but for perf issues rather than failures?
21:45:05 <regXboi> sort of like what I've been doing with stable/kilo network node over the past few weeks
21:45:14 <regXboi> salv-orlando: I couldn't say it better...
21:45:42 <armax> regXboi: so what are you propising?
21:45:44 <armax> proposing
21:46:05 <dougwig> sounds like he's proposing a formal subteam with meetings and such.
21:46:07 <regXboi> armax: I'm not proposing anything yet - I'm asking the gestalt to think about it
21:46:25 <regXboi> dougwig: I'm not that hubris :)
21:46:35 <armax> regXboi: with gate failures we have a framework to identify stuff to work on
21:46:42 <armax> the gate is on fire
21:46:47 <armax> hence we go on and fix it
21:47:09 <kevinbenton> https://review.openstack.org/#/c/226586/
21:47:15 <armax> with performance regressions/bottlenecks until we have such a framework it looks difficult to go on about it
21:47:30 <salv-orlando> fwiw I reckon that if regXboi is willing to set a mission, recruit members, and "manage" them towards success he should not even ask for permission to form the team
21:47:32 <kevinbenton> we need to expand rally to cover internal RPC calls somehow and then we will be able to set SLA's
21:47:45 <kevinbenton> then we can prevent regressions
21:47:53 <regXboi> kevinbenton: I need to read that patch
21:48:01 <kevinbenton> until we can prevent regressions, this is an uphill battle
21:48:13 <amuller> salv-orlando++
21:48:32 <amuller> just do it, the open source way :) people will naturally gravitate towards something worth it
21:48:32 <kevinbenton> i spent a ton of time in SQL optimizations between juno/kilo and new features trampled some out
21:48:35 <armax> well, I think whoever is willing to take a step into tackling this problem should work first into identifying the framework and putting the framework in place so that joining/participating is easier
21:48:36 <dougwig> salv-orlando: +1
21:48:50 <armax> salv-orlando: agreed
21:48:57 <regXboi> salv-orlando: that may be true, but it shouldn't be done "under a rock" either
21:49:19 <dougwig> regXboi: i think you've already satisfied that part.
21:49:21 <ihrachys> regXboi: well, it's not like the team lacks means of communication
21:49:32 <regXboi> ok, I'm good - next topic?
21:50:04 <armax> regXboi: well, I think we’re all good, but I’d like to understand how we’re going about it
21:50:24 <armax> regXboi: but that can be done offline
21:50:29 <regXboi> armax: I'm thinking the next step is to lay that out a bit more formally than here
21:50:44 <armax> sounds good
21:50:46 <regXboi> :)
21:50:48 <ihrachys> regXboi: like think it thru and drop an email to openstack-dev
21:50:51 <amuller> What we really need is historic graphing of Rally results IMO
21:51:09 <regXboi> ihrachys: :)
21:51:40 * regXboi wonders if he should glare at amuller
21:51:44 <amuller> refining SLAs will only get you so far, I'd much rather look at patterns over time
21:51:59 <amuller> this isn't exactly a novel idea it's been suggested many times before
21:52:33 <kevinbenton> refining SLAs would have caught the biggest performance regression we had this cycle
21:52:35 <armax> amuller: perforamnce testing isn’t exactly a novel idea
21:52:38 <kevinbenton> and they are trivial to add
21:52:43 <dougwig> amuller: unsuccessfully, and your statements don't sound exactly supportive. i'd say, let him organize it and try to drive something.
21:52:48 <armax> we need to come up with a way that works for Neutron and the way we operate
21:53:31 <amuller> dougwig: I think my tone was not coming off in this medium, I am 100% supportive, just floating the historical graphs idea out there one more time :)
21:53:45 <dougwig> amuller: gotcha, thanks. :)
21:54:00 <armax> we all talked about performance testing for a while now and performance here and performance there
21:54:30 <armax> but we also had different testing priorities where functional correctness was given more attention to non-functional one
21:54:33 * xgerman thinks about performance-testing-as-a-Service
21:54:43 <regXboi> amuller: if you know how to get the data out of rally jobs into graphite, I can try and do magic
21:54:54 <armax> time is definitely ripe to start making some dent into the hole we have as far as performance goes
21:55:07 <armax> but we’d also have to recognize that we can’t do this in the vacuum
21:55:23 <armax> there’s an entire community, both upstream and downstream who think about this very topic too
21:55:33 <regXboi> armax: ++
21:55:38 <amuller> regXboi: You should talk to Boris, Rally's author
21:55:42 <armax> I think regXboi is offering to plug that hole
21:55:46 <armax> whatever that means
21:55:49 <armax> :)
21:55:55 <amuller> regXboi: I'll forward you an email about this, I already talked to him about this issue a while ago
21:56:05 <regXboi> amuller: ack and thx
21:56:29 <armax> we have 4 mins left
21:56:37 <armax> there was another topic on the agenda
21:56:42 <armax> Testing strategy for of_interface=native OVS agent (iwamoto)
21:56:44 <iwamoto> hi
21:56:46 <armax> iwamoto: udere?
21:57:01 <iwamoto> https://review.openstack.org/#/c/221143/
21:57:32 <iwamoto> I and yamamoto spent some amount of effort to bring in of_interface=native driver
21:57:58 <iwamoto> my long-term goal is to switch the default to that
21:58:01 <armax> #action regXboi to kick off an effort to look into perforamnce testing and validation for Neutron
21:58:17 <regXboi> well now...
21:58:20 <armax> #link https://review.openstack.org/#/c/221143/
21:58:53 <armax> iwamoto: I hope it was clear from my vote that I was not against the change per se…but about how we go about it
21:59:11 <armax> iwamoto: I did flag down a similar change for testing vpnaas
21:59:13 <iwamoto> in interim, I want to make sure the native driver doesn't get bitrot
21:59:21 <regXboi> armax: so I actually filed a performance bug around using the native driver ...
21:59:27 <armax> regXboi: my apologies for the poor choice of words
21:59:43 <armax> regXboi: that’s a different native driver
21:59:53 <armax> regXboi: we’re talking about the of one
21:59:53 <regXboi> armax: ack you are correct
21:59:55 <armax> here
22:00:02 <armax> iwamoto: agreed with the sentiment
22:00:04 <dougwig> time!
22:00:11 * regXboi looks for the ethanol
22:00:15 <iwamoto> yes. it seems I need some other default testing with less load on the gate
22:00:17 <armax> we’ll continue the discussion in the channel
22:00:20 <armax> #endmeeting