21:00:19 <armax> #startmeeting networking
21:00:20 <openstack> Meeting started Mon Jan 18 21:00:19 2016 UTC and is due to finish in 60 minutes.  The chair is armax. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:24 <openstack> The meeting name has been set to 'networking'
21:00:38 <dougwig> lots of folks for a holiday.
21:00:52 <mhickey> Hi
21:00:54 <kevinbenton> Holiday for lots of folks
21:00:56 <armax> for whom made it, thanks for joining!
21:00:57 <emagana> hello all!
21:01:00 <regXboi> ola
21:01:00 <mestery> Holidays are for people who take time off dougwig.
21:01:01 <ihrachys> night for some too!
21:01:14 <rossella_s> night but not holiday
21:01:21 * carl_baldwin likes holidays
21:01:26 <armax> holidays, nights and time off is overrated :)
21:01:29 <emagana> you have a good quorum armax
21:01:38 <mestery> armax: Spoken like a true workaholic!
21:01:43 <armax> I suppose
21:01:47 <armax> ok, let’s dive in
21:02:02 <armax> we’ll see if we can keep this small
21:02:04 <armax> short
21:02:07 <armax> I mean
21:02:20 <armax> the agenda for today:
21:02:22 <armax> #link https://wiki.openstack.org/wiki/Network/Meetings
21:02:23 <mestery> small AND short
21:02:33 <armax> #topic Announcements
21:02:41 <armax> this week is milestone week
21:02:57 <armax> mestery has already a patch up
21:02:59 <armax> #link https://review.openstack.org/#/c/269265/
21:03:08 <mestery> I'm stealthy AND fast
21:03:09 <kencjohnston> o/
21:03:16 <armax> I think we’re gonna aggressively release as soon as we can confirm that all projects are sane
21:03:22 <mestery> How many BPs do we have for M2 armax?
21:03:24 <armax> meaning that their gate CI is green
21:03:28 <mestery> I mean, that finished?
21:03:32 <armax> #link https://launchpad.net/neutron/+milestone/mitaka-2
21:03:38 <armax> 28 bps, and 71 bugs
21:03:47 <mestery> thanks
21:04:09 <armax> 3 blueprints implemented so far
21:04:37 <armax> for whom of you has its neutron work affected by nova
21:04:41 <Sukhdev> hello - sorry a bit late
21:04:48 <armax> next week the nova team meets in Bristol, UK
21:04:57 <armax> the etherpad to the mid-cycle
21:04:58 <armax> https://etherpad.openstack.org/p/mitaka-nova-midcycle
21:05:00 <armax> #link  https://etherpad.openstack.org/p/mitaka-nova-midcycle
21:05:24 <armax> if you have something that you would like to bring to the attention or raise awareness, please reach out to me
21:05:36 <armax> next one
21:05:37 <dougwig> anyone from neutron going to the nova midcycle?  i'm got a conflict.
21:05:44 <Sukhdev> armax: any plans for Neutron mid-cycle?
21:05:46 <armax> dougwig: carl_baldwin and armax  re
21:05:47 <armax> are
21:05:54 <armax> Sukhdev: you beat me to it
21:06:10 <armax> if you guys recall we decided to call the Neutron mid-cycle off during the early part of the cycle
21:06:29 <armax> dougwig and a few others are looking into the logistics for a mid-cycle closer to the end of the cycle
21:06:40 <armax> right now we’re looking at the last week of Feb
21:06:43 <armax> somewhere on the east cose
21:06:43 <mestery> So it's not really a mid-cycle then ...
21:06:44 <armax> cost
21:06:46 <armax> coast
21:06:52 <dougwig> mestery: pedant.
21:06:55 <kevinbenton> FLAHRIDA
21:06:56 <armax> mestery: call it whatever you fancy
21:07:02 <armax> :)
21:07:10 <mestery> OK: "Beers with friends"
21:07:14 <anteaya> can it be first week of march?
21:07:19 <anteaya> last week of feb is infra
21:07:22 <kevinbenton> I'm allowed to invite my friends?
21:07:32 <mestery> kevinbenton: Wait, you're coming?
21:07:37 <armax> I suppose the east coast is relatively easier to get to from both sides of the pond
21:07:38 <mestery> anteaya: It can't
21:07:43 <mestery> LAst week of feb
21:08:01 <armax> we’ll sort things out
21:08:07 <Sukhdev> kevinbenton : can I be your friend?
21:08:27 <kevinbenton> i charge by the hour :)
21:08:33 <armax> since your attendance may be determined on the topics of the meetup
21:08:43 <armax> watch out for a mail that I’ll send later today
21:08:52 <Sukhdev> kevinbenton : I pay in Indian Rupees :-)
21:09:00 <kevinbenton> are there any main topics in mind yet?
21:09:05 <sc68cal> armax: just be aware that the QA midcycle is near end of feb - https://wiki.openstack.org/wiki/QA/CodeSprintMitakaBoston
21:09:21 <armax> sc68cal: thanks for the heads-up
21:09:27 <kevinbenton> or will it just be whatever blueprints are outstanding?
21:09:28 <anteaya> sc68cal: yeah it is the same week as infra
21:09:33 <regXboi> boston @ the end of february - that doesn't sound like good planning
21:09:38 <mestery> rofl
21:09:39 <armax> kevinbenton: I’ll follow up on the ML
21:10:00 <sc68cal> regXboi: winter is coming john snow
21:10:07 <armax> next one
21:10:11 <armax> #topic blueprints
21:10:23 <armax> #link  https://launchpad.net/neutron/+milestone/mitaka-2
21:10:37 <armax> existing assignements for mitaka
21:10:38 <armax> #link https://blueprints.launchpad.net/neutron/mitaka/+assignments
21:11:01 <armax> if you are an assignee or approver for a blueprint please work together to update the BP whiteboard
21:11:18 <armax> if your work doesn’t have substantial code/documentation in place, most likely it will fall off Mitaka
21:11:36 <armax> as for RFE’s, we’ll go through a similiar diet exercise
21:11:54 <armax> after M2 is cut
21:12:21 <armax> the next meeting we’ll go over the list to see what can be spared by the slaugther
21:12:37 * sc68cal has already moved fwaas v2 out of mitaka
21:12:55 <armax> sc68cal: you ruin all the fun
21:13:48 <armax> sc68cal: thanks for letting me know
21:14:01 <armax> I need to document the steps to make deferred BPs show in this dashboard
21:14:04 <armax> https://blueprints.launchpad.net/neutron
21:14:05 <armax> #link https://blueprints.launchpad.net/neutron
21:14:14 <dasm> o/
21:14:14 <armax> the appropriate way
21:15:06 <armax> as for the folks involved in the existing backlog, I’ll try to reach out offline if I have questions/doubts
21:15:51 <armax> I would be ethernally grateful if you drop a note on the whiteboard/report page of the RFE/BP you’re assigned to in whichever capacity
21:17:49 <armax> ok the prolonged silence is my cue for the next section
21:18:08 <armax> #topic Bugs
21:19:01 <armax> so far it looks we fixed nearly 50% of the bugs submitted since Mitaka started
21:19:03 <armax> that’s impressive
21:19:20 <armax> but this means we have another 50% to go
21:19:27 <armax> which roughly counts to 300+ bugs
21:20:04 <armax> am I alone in this meeting?
21:20:13 <regXboi> armax: we're all behind you
21:20:15 <dasm> no :)
21:20:19 * mestery grabs popcorn
21:20:27 <hichihara> I'm here :)
21:20:28 * ihrachysh makes a silly comment for stats
21:20:29 <armax> ah, for a second I thought I was speaking to myself
21:20:32 <dasm> nice job with bugs
21:20:38 * mestery gives ihrachysh a gold star
21:20:53 <armax> I wasn’t asking for cheerleading, but I take it nonetheless :)
21:21:30 <armax> since we’re getting closer to the end of the release
21:21:50 <armax> I would encourage people to be conscious of the state of the gate when pushing stuff into the merge queue
21:22:32 <mestery> I'll amend that to: I'd encourage people to generally always be conscious of the state of the gate
21:22:35 <armax> so give priority to targeted stuff, unless the gate is relatively clear or the patch run on a smaller more reliable set of jobs
21:22:39 <regXboi> mestery: +1000
21:22:40 <armax> mestery: ++
21:22:52 <mestery> But I get your point armax, sorry for the distraction
21:22:52 <hichihara> mestery: ++
21:23:14 <amuller> A great tool to be aware of the state of the gate is: http://docs.openstack.org/developer/neutron/dashboards/check.dashboard.html
21:23:16 <armax> we had a few setbacks during the past weeks
21:23:32 <armax> with the gate being broken, hopefully it’ll be better in the future
21:23:32 <kevinbenton> not setbacks, fire drills
21:23:34 <armax> amuller: ++
21:23:42 <armax> and
21:23:44 <armax> #link http://docs.openstack.org/developer/neutron/dashboards/gate.dashboard.html
21:23:47 <ihrachysh> better? you gotta be kiddin
21:23:57 <armax> ihrachysh: ok
21:24:01 <amuller> For example you can clearly see that the multinode regular and DVR jobs are broken ATM
21:24:08 <armax> we’re doomed and it’s gonan be worse from now on
21:24:10 <armax> ihrachysh: better?
21:24:22 <armax> amuller: we’re way ahead of you :)
21:24:33 <amuller> armax: that's to be expected
21:24:34 <ihrachysh> armax: now I hear an engineer speaking
21:25:02 <ihrachysh> amuller: re multinode seems like failures are gate timeouts, could be just an issue of long bootstrap
21:25:11 <armax> amuller: but yeah I hope someone is on top of the recent failures, I know regXboi has been briefly looking into it?
21:25:35 <armax> luckily the dvr issue is not affecting the gate queue
21:25:40 <dougwig> at some point we should address that our baseline for a "good" gate is ~12% failure rate. that shaky foundation doesn't help any of the jobs built on top of that base.
21:25:51 <regXboi> I went and looked at the recent failures that I could find in jenkins and ihrachysh is correct - most of them are timeouts of the job itself
21:26:08 <mestery> dougwig: You're a prophet sent from the future!
21:26:39 <armax> dougwig: 0%
21:27:14 <armax> if I look at
21:27:16 <armax> #link http://status.openstack.org/openstack-health
21:27:17 <dougwig> what caused the timeout?  a bug we should file against infra?
21:27:20 <armax> we’re 97% pass rate
21:27:21 <regXboi> I am wondering if we've added enough tests that we are blowing the top of the runtime limit
21:27:27 <armax> on the gate queue
21:27:32 <amuller> dougwig: that's for the check queue I think. If you look at the gate queue it's normally 0%, and you can also look at openstack-health which armax just linked, it also shows near 0% on the gate queue
21:27:40 <ihrachysh> speaking of multinode gate, we have some MTU related patches to push for that should fix some issues with ssh connection checks in gate: https://review.openstack.org/#/q/topic:multinode-neutron-mtu
21:27:54 <armax> so I’d say 100% pass rate with +/- 3%? :P
21:27:56 <sc68cal> right - and due to it being MTU, that chews up a ton of retry time
21:27:59 <ihrachysh> regXboi: someone should walk thru logs and check when tests are started and how it's compared to regular runs
21:28:09 <sc68cal> takes 5 minutes before SSH gives up and disconnects
21:28:17 <regXboi> ihrachysh: I might be able to spend some time tomorrow attempting that
21:28:30 <ihrachysh> regXboi: keep me informed
21:28:31 <dougwig> 3% is great.
21:28:39 <armax> btw the linuxbridge job on the gate is a bit flaky recently
21:28:48 <armax> I wonder if sc68cal had a glimpse into that
21:28:55 <regXboi> #action regXboi to look at jenkins logs to compare run times of multinode tests
21:28:59 <sc68cal> one thing to also remember, I think mtreinish mentioned that openstack-health only tracks failures where it gets to running the tests. Doesn't track failures early in the setup
21:29:23 <armax> sc68cal: I think so
21:29:31 <amuller> openstack-health runs code that inserts the test results in to a mysql DB at the end of the run, the run has to complete
21:29:32 <kevinbenton> sc68cal: that's what we want. unless neutron causes setup failures
21:29:50 <anteaya> sc68cal: that is my understanding also
21:30:02 <armax> before we move on from bugs
21:30:13 <armax> anyone willing to be deputy for this week and the weeks coming next?
21:30:16 <armax> we’re exposed right now
21:30:18 <sc68cal> so, the openstack-health % is a datapoint, but not the full picture. I think regXboi 's graphite queries measure the bigger picture
21:30:19 <armax> !!!
21:30:20 <openstack> armax: Error: "!!" is not a valid command.
21:30:27 <armax> oops
21:30:53 <regXboi> sc68cal: actually, the graphite is the overall, openstack-health should eventually get us to a better drill down
21:31:06 <regXboi> at least once it includes the check pipelines
21:31:24 * ihrachysh will need to skip the bug duty for the next three weeks but is happy to get on board after
21:31:29 <armax> I can have a go this week
21:31:42 <armax> mlavalle you interested?
21:31:45 <anteaya> regXboi: I don't know if openstack-health ever will include check
21:32:01 <armax> anyone else?
21:32:11 <armax> sc68cal: ?
21:32:18 <amuller> anteaya: any ideas if it will include stats for non-tempest jobs, like the neutron functional job?
21:32:21 <mlavalle> armax: Next week I can
21:32:27 <armax> ook
21:32:29 <armax> cool
21:32:32 <ZZelle_> i can this week
21:32:42 <armax> #action mlavalle deputy for the week of 25th
21:32:49 <armax> ZZelle_: sweet
21:32:54 <regXboi> anteaya: I'm negotiating on that one :)
21:33:03 <armax> #action ZZelle_ deputy for week of 18th
21:33:03 <anteaya> amuller: I don't believe that is the goal of this tool
21:33:30 <anteaya> I believe it was created to be a gate health tool
21:34:12 <anteaya> sorry to derail
21:34:24 <armax> anteaya: yeah, I was about to tell you off :P
21:34:28 * armax kids
21:34:34 <anteaya> :)
21:34:40 <amuller> I'll talk to mtreinish offline
21:34:46 <armax> ok, anything else on this section? Anything burning that we should be aware of and we aren’t?
21:34:53 <dougwig> sc68cal: so what is our false failure rate, then?
21:34:55 <regXboi> anteaya: again, I'm still negotiating that one
21:35:12 * regXboi now goes and looks for the re-railer
21:35:45 <sc68cal> dougwig: haven't checked lately
21:36:01 <armax> next one
21:36:06 <armax> #topic Docs
21:36:17 <regXboi> mmm docs
21:36:17 <armax> emagana I know that Sam-I-Am is not around
21:36:28 <emagana> I was hoping he was
21:36:31 <armax> he mentioned
21:36:34 <armax> that The networking guide now publishes stable releases rather than continuously so people using an older version of OpenStack can find information relevant to it.
21:36:34 <emagana> He had some points to discuss
21:36:48 <emagana> armax: indeed, that is completed
21:37:00 <armax> he’ll be updating the networking guide scenarios for Liberty. Once those are done, I think he’ll split time between populating the intro content and updates for Mitaka.
21:37:18 <emagana> armax: that is correct
21:37:31 <armax> well, kudos to him :)
21:37:49 <amotoki> now we have http://docs.openstack.org/liberty/networking-guide/ (liberty version) from the top page of docs.o.o :)
21:37:51 <emagana> we dont want to create confusion about scenarios but also we dont want to create more very similar ones
21:37:54 <armax> I personally have this review https://review.openstack.org/#/c/253229/
21:37:59 <armax> sitting for ever in the queue
21:38:08 <emagana> armax: I will take care of it
21:38:17 <armax> this should help us to tag the automatic doc bugs in LP
21:38:24 <armax> emagana that’s not a documentation patch
21:38:28 <emagana> armax: sounds good
21:39:03 <emagana> armax: oh got it.. will review anyway
21:39:05 <armax> amotoki: so I assume that we don’t publish trunk changes at all do we?
21:39:11 <armax> emagana: sure, thanks
21:39:37 <amotoki> armax: I think so too
21:39:38 <emagana> armax: we do..
21:39:41 <armax> ok
21:39:47 <emagana> we pu have both Liberty and current
21:40:03 <emagana> s/pu/do
21:41:09 <emagana> Sam-I-Am wanted to discuss about the ability to attach VMs directly to public/external networks
21:41:47 <emagana> but I do not have all the context what we wanted to review with you all. We have a networking guide meeting this Thursday. I will review it with him and the rest of the folks attending the meeting
21:41:48 <kevinbenton> emagana: what about it? it should work fine if people aren't using that stupid 'external_network_bridge' option in the l3 agent
21:42:14 <emagana> kevinbenton: maybe as simple as a good explanation of how to use it
21:42:20 <sc68cal> I think the issue is dhcp
21:42:28 <kevinbenton> an external network can either be a 'real' network that is wired by the L2 agent
21:42:41 <emagana> kevinbenton: indeed, that is the way we use it..
21:42:46 <kevinbenton> in which case everything works completely fine (even dhcp agents can bind to it if you have dhcp enabled)
21:43:11 <kevinbenton> or you use the 'external_network_bridge' on the l3 agent, and it's completely out of reach by everything but the l3 agent
21:43:14 <sc68cal> The only issue is developer envs - most likely they already have a DHCP service running
21:44:07 <kevinbenton> sc68cal: yes, from a developer workflow they would need to have dhcp disabled
21:44:22 <kevinbenton> or hope dnsmasq is faster :)
21:44:26 <sc68cal> :)
21:44:43 <kevinbenton> emagana: you can loop me into that conversation later
21:44:45 <armax> ok
21:44:49 <armax> shall we wrap it up?
21:44:51 <emagana> kevinbenton: I will do that
21:44:51 <amotoki> correction. we have trunk (Mitaka) version of networking-guide under /draft/ now. http://docs.openstack.org/draft/networking-guide/
21:44:59 * regXboi goes and looks for the bow ?!?!
21:45:11 <emagana> amotoki: thanks for the link. That is that I thought
21:45:20 <armax> amotoki: thanks for the link
21:45:29 <armax> #link  http://docs.openstack.org/draft/networking-guide/
21:45:41 <emagana> armax: about the networking schedule, we ran into the same odd/even bi-weekly meetings
21:45:41 <armax> for tracking master changes to openstack-manuals
21:45:51 <emagana> but we can figure out that by ML
21:46:05 <armax> emagana: ok
21:46:12 <emagana> armax: nothing else but as always extending the invite to help to complete the missing section in the networking guide
21:46:34 <emagana> #link https://etherpad.openstack.org/p/networking-guide
21:46:48 <emagana> feel free to grab a section  :-)
21:47:12 <armax> #topic Open Discussion
21:47:23 <armax> Swami added a topic on the agenda
21:47:30 <armax> obondarev might be around
21:47:35 <armax> but I don’t see Swami
21:48:01 <armax> having said that, this topic is raised at hte nova mid-cycle
21:48:13 <armax> or whichever way mestery wants to call it
21:48:26 <regXboi> friends with beers?
21:48:40 <mestery> armax: After a few of either, the order doesn't matter.
21:48:57 <mhickey> mestery: sure does not! :)
21:49:00 <armax> anyone wants to discuss anything in lieu of that?
21:49:12 <armax> if not, we can get 11 minutes of our lives back
21:49:19 <hichihara> What time is next meeting?
21:49:22 <armax> and keep breaking the gate, ihrachysh am I right?
21:49:33 <regXboi> "repent! cried the tick tock man"
21:49:35 <ihrachysh> as always sir
21:49:35 <armax> hichihara: same time same place same day
21:49:35 <amuller> friendly note: The openstack foundation recommended Austin hotels are running out of rooms
21:49:46 <hichihara> armax: OK. Thanks
21:49:53 <mestery> amuller: Tahnks for sharing
21:50:00 <mlavalle> amuller: thanks for the heads up
21:50:02 * mestery senses the rush to the foundation website from those without a room
21:50:03 <mhickey> amuller: thanks
21:50:05 <armax> amuller: good lad
21:51:14 <sc68cal> austin shouldn't be too hot. Bring your camping gear
21:51:37 <armax> ok, for folks who stayed late, for those which was time off and for those who simply participated thanks for joinin
21:52:00 <armax> have a blast!
21:52:04 <armax> #endmeeting