#openstack-meeting log

21:02:18 <mestery> #startmeeting networking
21:02:18 <openstack> Meeting started Mon Jun 23 21:02:18 2014 UTC and is due to finish in 60 minutes.  The chair is mestery. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:02:19 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:02:21 <openstack> The meeting name has been set to 'networking'
21:02:25 <nati_ueno> o/
21:02:29 <mestery> #link https://wiki.openstack.org/wiki/Network/Meetings Agenda
21:02:49 <mestery> #topic Announcements
21:03:04 <mestery> The mid-cycle sprint focused on nova-network parity is coming up in 2 weeks!
21:03:05 <mestery> #link https://etherpad.openstack.org/p/neutron-juno-mid-cycle-meeting
21:03:17 <mestery> Hope to see all those who signed up there and make some great progress!@
21:04:02 <mestery> #topic Bugs
21:04:17 <enikanorov> hi
21:04:17 <mestery> enikanorov: Hi! Any major bugs the team should be aware of this week?
21:04:27 <enikanorov> i have a quick update
21:04:53 <enikanorov> i've filed a few new bgs on lock wait timeouts/deadlocks
21:05:10 <enikanorov> these were seen in the gate from time to time, quite rarely though
21:05:40 <mestery> Thanks enikanorov! I see them in the agenda as well.
21:05:41 <enikanorov> one of the constant producer of the gate failures related to DB is carl_baldwin's patch that separates rpc_workers into separate processes
21:05:59 <enikanorov> another bug that i've not filed yet is some issue in loadbalancer_db
21:06:20 <enikanorov> it seems to be hit very often in neutron jobs since 23rd of june
21:06:25 <carl_baldwin> enikanorov: that patch shouldn’t be run very often.
21:06:29 <ihrachyshka> enikanorov: vmware one?
21:06:57 <enikanorov> ihrachyshka: not sure it's vmware, but 'ive not looked too closely yet
21:07:03 <salv-orlando> ihrachyshka: the patch causing ut failures has been reverted.
21:07:16 <ihrachyshka> that one? https://bugs.launchpad.net/neutron/+bug/1332724
21:07:17 <uvirtbot> Launchpad bug 1332724 in neutron "neutron.tests.unit.vmware.vshield.test_lbaas_plugin.TestLoadbalancerPlugin.test_create_vip_with_invalid_router often fails  in gate" [High,Confirmed]
21:07:23 <enikanorov> no, not this one
21:07:38 <enikanorov> http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOiBcIidOb25lVHlwZScgb2JqZWN0IGhhcyBubyBhdHRyaWJ1dGUgJ2ZpeGVkX2lwcydcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiODY0MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDAzNTU0NzIxNTYxfQ==
21:07:43 <salv-orlando> ihrachyshka: we’re talking about the load balancing scenario test.
21:07:45 <enikanorov> here's the query for the new issue
21:07:58 <enikanorov> no, that's API test actualy
21:08:18 <salv-orlando> I and enikanorov looked at it today. I have not been able to find the root cause yet (well did not really spent a lot of time investigating)...
21:08:18 <enikanorov> anyway, that's not filed yet, so feel free to do it
21:08:48 <enikanorov> salv-orlando: that one i'm talking about is also different, not related to scenario
21:08:57 <salv-orlando> enikanorov: so that’s yet another issue wrt the one I saw
21:09:06 <enikanorov> salv-orlando: yes
21:09:09 <markmcclain> ok.. please file a classifier for it, so that it shows up in elastic-recheck
21:09:14 * mestery loses track of how many separate issues we're discussing here.
21:09:25 <salv-orlando> folks: it’s 2 issue
21:09:30 <mestery> salv-orlando: Thanks. :)
21:10:03 <mestery> I was also just alerted by infra about an ew version of sqalalchemy (0.9.5) which is causing UT failures for neutron and nova.
21:10:05 <enikanorov> markmcclain: ok
21:10:11 <mestery> I don't have details yet, literally they just pinged me in-channel now.
21:10:12 <salv-orlando> one in API tests - one in scenario tests. I have not yet filed a e-r query for the latter as I’m hoping to find a trace which is a bit more precise than the console log message
21:10:14 <mestery> So, there's that too.
21:10:24 <enikanorov> mestery: that's interesting
21:10:35 <mestery> enikanorov: Indeed. :)
21:10:49 <enikanorov> that's it on bugs from my side
21:10:57 <mestery> Thanks enikanorov!
21:11:03 <mestery> Any other bugs the team should be aware of right now?
21:11:09 <markmcclain> salv-orlando: ah cool
21:11:26 <ihrachyshka> I think we should be aware of https://bugs.launchpad.net/neutron/+bug/1332724
21:11:29 <uvirtbot> Launchpad bug 1332724 in neutron "neutron.tests.unit.vmware.vshield.test_lbaas_plugin.TestLoadbalancerPlugin.test_create_vip_with_invalid_router often fails  in gate" [High,Confirmed]
21:11:31 <mlavalle> mestery: we said Thursday that we want to review the bugs preventing the full Neutron job to execute
21:11:42 <mestery> mlavalle: Should we do that during the tempest section or now?
21:11:42 <ihrachyshka> this is a unit test failing in gate, sporadically
21:11:57 <mestery> ihrachyshka: enikanorov is assigned that issue at the moment it looks like.
21:12:04 <mlavalle> mestery: let's do it in the tempest section
21:12:08 <mlavalle> thnanks
21:12:10 <mestery> mlavalle: OK, thanks. :)
21:12:13 <ihrachyshka> I think unit tests were quite stable recently, but now they fail too, not nice
21:12:38 <enikanorov> ihrachyshka: i have some observations about this bug, but were not able to find the root cause
21:12:43 <ihrachyshka> mestery: I'm not sure whether anyone is *actively* working on it (if not, I may be of help tomorrow)
21:12:55 <ihrachyshka> enikanorov: ok, let's sync on this tomorrow
21:12:55 <mestery> ihrachyshka: Please sync with enikanorov on that one tomorrow if you could.
21:13:00 <mestery> ihrachyshka: Ha! :)
21:13:00 <enikanorov> ihrachyshka: sure
21:13:09 <salv-orlando> ihrachishka, mestery: we have reverted the patch causing that bug
21:13:10 <mestery> OK, lets move on.
21:13:27 <mestery> salv-orlando: Link by chance?
21:13:57 <salv-orlando> https://review.openstack.org/#/c/101789/
21:14:08 <ihrachyshka> thanks
21:14:20 <mestery> salv-orlando: thanks!
21:14:27 <mestery> OK, now moving on.
21:14:30 <mestery> #topic Nova Parity
21:14:31 <mestery> markmcclain: Hi!
21:14:39 <markmcclain> hi
21:14:50 <markmcclain> I've updated with links to the specs
21:15:09 <markmcclain> salv-orlando and I are leading the charge on reviewing the db healing ones
21:15:28 <mestery> OK, cool!
21:15:34 <markmcclain> healing is a must for us to work on adding grenade testing back
21:16:38 <mestery> Thanks markmcclain. I also encourage folks to review obondarev's nova spec for hte migration API.
21:16:53 <salv-orlando> sure will do.
21:16:57 <armax> mestery: it’s on my todo list
21:16:59 <markmcclain> mestery: ++
21:17:07 <mestery> armax salv-orlando markmcclain: Great!
21:17:11 * salv-orlando said the man who has a 5-minute memory span
21:17:24 <enikanorov> :)
21:17:58 <mestery> #topic Tempest
21:18:03 <mestery> mlavalle: Hi there!
21:18:06 <mlavalle> hi
21:18:37 <mlavalle> I want to see if we want to review the bugs in the full neutron job
21:18:54 <mlavalle> from those who have been working on that
21:19:08 <mestery> mlavalle: Yes, but lets give some background: QA is no longer reviewing neutron tests until we get the full job running.
21:19:16 <mestery> mlavalle: Thus the urgency on that front I think.
21:19:24 <salv-orlando> so why don’t I get some attention here: https://review.openstack.org/#/c/88289/
21:19:31 * mestery looks
21:19:38 <mlavalle> mestery: yeah, that's the urgency
21:19:39 <salv-orlando> the job has been deemed as rather stable - sdague’s words ;)
21:19:59 <mestery> salv-orlando: I believe the failure rate was deemed too high during last weeks QA meeting, right mlavalle?
21:20:00 <salv-orlando> and I think that we better enable it and deal with the inevitable raise in failure rate
21:20:07 <salv-orlando> than wait forever
21:20:09 <mestery> salv-orlando: +1 to that though
21:20:22 <sdague> also, the trusty switch over this week should help, right?
21:20:32 <mestery> sdague: +1
21:20:33 <sdague> iirc new openvswitch was more resilient
21:20:47 <mestery> sdague: It's light years ahead to be honest
21:20:49 <salv-orlando> mestery: that high failure rate might have been “tainted” by the failures we have been plagued with for that timing issue which caused images to not boot right.
21:21:05 <mestery> I think we should push to get this merged then.
21:21:07 <mestery> sdague: ^^^
21:21:21 <mestery> sdague: What else do you need from us here? We're ready to triage and fix issues once it merges.
21:21:30 <salv-orlando> mestery, sdague: if we get to the end of juno-2 without doing that, we’ll just end up not doing it
21:21:43 <salv-orlando> as people will be afraid of enabling a potentially gate breaking job in juno-3
21:21:44 <mestery> I'm with you salv-orlando on this front.
21:21:50 <sdague> mestery: I think that's about it. Why don't we wait for trusty flip, which should be in the next couple of days
21:21:53 <sdague> then do it
21:22:05 <mestery> sdague: Agreed. I'll touch base with you later this week.
21:22:08 <mestery> salv-orlando: Sound good?
21:22:18 <salv-orlando> sdague, cool il ping you back on thurs morning
21:22:24 <sdague> salv-orlando: sounds great
21:22:28 <mlavalle> sdague: and I will raaisee the point also during the tempest meeting on Thursday
21:22:34 <mestery> Excellent!
21:22:41 * mestery loves it when a plan comes together.
21:23:03 <mestery> mlavalle: Anything else on the agenda for Tempest today?
21:23:34 <mlavalle> mestery: just to report to the team we made good progress with api tests for new LBaaS last week
21:23:44 <mestery> mlavalle: Awesome!
21:23:44 <mlavalle> I should push somehting for review later this week
21:23:52 <mlavalle> as wip
21:24:10 <mlavalle> other than that, I will be following up with the full neutron job
21:24:17 <mlavalle> and that's it from me this week
21:24:34 <afazekas> Is anyone has any idea why the full neutron  job fails more than the smoke ?
21:25:42 <salv-orlando> afazekas: There are a few independent issues. some are because a few tests are not well designed, some because there are a few bugs in neutron which need to be polished.
21:26:00 <salv-orlando> nothing serious imho though. But bear in mind that the last time I did this analysis it was about 6 weeks ago.
21:26:16 <afazekas> bug numbers ?
21:27:02 <salv-orlando> afazekas: gimme a few minutes, in the meanwhile you guys can go on with the meeting
21:27:11 <mestery> salv-orlando: Thanks!
21:27:32 <mestery> #topic L3
21:27:36 <mestery> carl_baldwin: Hi and welcome back!
21:27:38 <carl_baldwin> mestery: hi
21:27:39 <carl_baldwin> Thanks.
21:27:51 <carl_baldwin> The DVR patches have seen a lot of updates.
21:28:19 <carl_baldwin> One of them is nearly ready and we’ve discussed plans to get the others in to shape splitting efforts between me and Armando as the core reviewers.
21:28:48 <mestery> carl_baldwin: Perfect, thanks!
21:29:02 <carl_baldwin> I’m also working on having all the patches available together for testing by others in devstack.
21:29:27 <salv-orlando> afazekas: here’s the list (needs update though - some bugs might have been already fixed): http://pastebin.com/DFySpmu1
21:29:28 <markmcclain> carl_baldwin: awesome
21:29:30 <carl_baldwin> I think we could also pull together a demo soon.
21:29:31 <mestery> carl_baldwin: That would be most excellent! Are the patches already linked in gerrit?
21:29:43 <carl_baldwin> Yes, they are all on the same topic.
21:29:46 <armax> carl_baldwin: I’ll send a note when the demo is ready
21:29:53 <mestery> carl_baldwin: Cool!
21:29:55 <carl_baldwin> #link https://review.openstack.org/#/q/topic:bp/neutron-ovs-dvr,n,z
21:30:08 <armax> mestery: I am going to work on one of the patches so that it can be more digestibile
21:30:17 <mestery> armax: Awesome!
21:30:24 <sc68cal> Yeah that was one of my concerns, that one patch is like 2K lines
21:30:26 <armax> mestery: so expect a lot more traffic this week :)
21:30:27 <carl_baldwin> mestery: I’ll be working on another in a similar fashion.
21:30:34 <armax> sc68cal: indeed
21:30:35 <sc68cal> but I hate to be that guy that -1s and grumbles about big patches
21:30:46 <armax> sc68cal: I have a strategy
21:30:48 <armax> stay tuned
21:30:52 <carl_baldwin> Large patches are a valid concern.
21:30:59 <mestery> Sounds like this is moving forward nicely carl_baldwin!
21:31:01 <markmcclain> sc68cal: still not the biggest patch we have in the queue
21:31:03 <carl_baldwin> Other subteam projects are making progress and status is on the wiki.
21:31:28 <mestery> carl_baldwin: Thanks for the updates here. Anything else to add?
21:31:32 <carl_baldwin> mestery: that is all I have.  See you all at the meeting on Thursday.
21:31:43 <mestery> carl_baldwin: Thanks!
21:32:01 <mestery> #topic Advanced Services
21:32:03 <mestery> SumitNaiksatam: Hi!
21:32:13 <SumitNaiksatam> mestery: hi
21:32:32 <SumitNaiksatam> so, as last week we have a few specs waiting for review/approval
21:32:49 <SumitNaiksatam> the most notable among them is the flavors spce put by enikanorov
21:33:08 <markmcclain> SumitNaiksatam: I'm going to post an alt proposal for flavors
21:33:10 <SumitNaiksatam> that it hit a bit of a road block last week, just as we were beginning to thin that we are getting consensus
21:33:31 <markmcclain> SumitNaiksatam: much simpler version that is implementable during J and still cross service
21:33:53 <SumitNaiksatam> markmcclain: any reason why the suggestions cannot be factored into the current proposal?
21:34:04 <afazekas> salv-orlando: which causes the most failure on the full jobs ?
21:34:28 <SumitNaiksatam> markmcclain: my concern is that we repeatedly reset the discussion
21:34:28 <mestery> afazekas salv-orlando: Can you guys pop over to #openstack-neutron since we're in the meeting here at the moment? Thanks!
21:34:30 <markmcclain> SumitNaiksatam: I don't think the current prop is the proper direction upon further thinking
21:34:56 <salv-orlando> afazekas - they’re pasted in order of failure rate. But I don’t want to give you obsolete numbers. Let me run again checks on logstash tonight and I’ll update you tomorrow morning in openstack-qa
21:35:09 <markmcclain> SumitNaiksatam: additionally current prop does not solve original problem of operator control
21:35:21 <enikanorov> markmcclain: i think it does
21:35:32 <afazekas> salv-orlando: ok
21:35:38 <enikanorov> let's get togather and analyse how we can converge
21:36:02 <SumitNaiksatam> markmcclain enikanorov mestery: can i propose that we use this week’s adv services meeting to arrive at a consensus on what we plan to implement in Juno?
21:36:03 <markmcclain> enikanorov: sure we can chat offline
21:36:17 <enikanorov> markmcclain: right
21:36:19 <SumitNaiksatam> if we push this any further, i dont think it will make it into Juno
21:36:31 <mestery> markmcclain enikanorov: IF there is a way to retrofit this into the existing BP, it may be easiest.
21:36:36 <markmcclain> SumitNaiksatam: -1 have conflict with that timeslot this week
21:37:29 <nati_ueno> markmcclain: when we can get your proposal?
21:37:30 <SumitNaiksatam> markmcclain: what are your alternate suggestions?
21:38:15 <markmcclain> hopefully this evening
21:38:24 <enikanorov> sounds good
21:38:41 <mestery> OK, lets try to resolve this sometime this week so we can move forward, flavors are important to a lot of sub-teams I believe.
21:38:52 <SumitNaiksatam> mestery: yes
21:38:58 <markmcclain> mestery: very true
21:39:23 <SumitNaiksatam> mestery: other than that we have the services’ insertion and traffic steering specs which have received some decent attention
21:39:44 <markmcclain> SumitNaiksatam: concerned about those as we have _lots_ of other work to land
21:39:56 <SumitNaiksatam> mestery: so we would like to close on them, hopefully this week, so that these can be pursued in implementation for J2
21:40:22 <SumitNaiksatam> markmcclain: i believe these have been scheduled for J2
21:40:49 <mestery> markmcclain SumitNaiksatam: We should understand if there are issues here as well and sort that out this week with the flavors stuff if possible.
21:41:07 * mestery notes there is a seemingly endless supply of stuff to land this cycle. :)
21:41:23 <SumitNaiksatam> mestery: sure, these have been in discussion for a long time
21:41:35 <mestery> SumitNaiksatam: Yes, agreed.
21:41:49 <SumitNaiksatam> mestery: so hopefully whoever feels strongly about these, can make it to the IRC meeting, and we can make progress
21:41:54 <markmcclain> SumitNaiksatam: understand the length of time they've been discussed.. I just question if the team have bandwidth to land with quality
21:42:16 <salv-orlando> mestery: meh we have enough history to know our limits on what we can land in a milestone. We should use them to decide what we can reasonably target.
21:42:21 <SumitNaiksatam> markmcclain: there are several people reviewing this
21:42:28 <salv-orlando> Otherwise we’ll end up generating frustation and disappointment
21:42:53 <markmcclain> salv-orlando: I agree… the potential load is still really high
21:43:18 <SumitNaiksatam> mestery: thats pretty much it from me on this, thanks for the time
21:43:19 <mestery> #link https://wiki.openstack.org/wiki/NeutronJunoProjectPlan
21:43:31 <mestery> There is a lot on there, agreed, we're already behind according to that plan.
21:43:37 <mestery> But, lets move on. Thanks SumitNaiksatam!
21:43:46 <mestery> #topic IPv6
21:43:48 <mestery> sc68cal: Hi there!
21:43:51 <sc68cal> Hello
21:44:20 <sc68cal> So the two big items in my opinion are the spec for radvd and this small patch to complete ipv6 provider net slaac
21:44:43 <mestery> sc68cal: Is there a spec for radvd yet?
21:44:45 <sc68cal> #link https://review.openstack.org/64578 provider net slaac bug
21:44:51 <sc68cal> mestery: yep - linking now
21:44:55 <mestery> sc68cal: cool
21:44:56 <sc68cal> #link https://review.openstack.org/101306
21:45:06 <markmcclain> sc68cal: awesome will add to review queue
21:45:33 <sc68cal> I believe xuhanp and shshang will be pivoting to do dhcpv6 stateful support in dnsmasq
21:45:48 <sc68cal> will provide a link to the spec when it is published
21:46:47 <sc68cal> If we can get some core reviewers on the provider net slaac patch we can close that bp from the j2 milestone
21:46:57 <mestery> sc68cal: I'll review that one.
21:47:00 <sc68cal> put one in the proverbial win column :)
21:47:03 <sc68cal> that's it for me today
21:47:07 <mestery> Thanks sc68cal!
21:47:14 <mestery> #topic ML2
21:47:17 <mestery> Sukhdev: Hi!
21:47:25 <Sukhdev> mestery: Hi
21:47:28 <mestery> Anything on ML2 this week?
21:47:34 <Sukhdev> Couple of items -
21:47:56 <Sukhdev> the team has been reviewing specs - please see here https://wiki.openstack.org/wiki/Tracking_ML2_Subgroup_Reviews#Under_Review
21:48:15 <Sukhdev> Few of the patches are ready for blessing by Cores - so, will be pinging you guys
21:48:30 <mestery> Sukhdev: OK, sounds good.
21:48:41 <Sukhdev> Additionally, work has been kicked off on the Modular L2 agent
21:49:08 <Sukhdev> discussion is on the ML - if one wants to participate
21:49:23 <Sukhdev> mestery: that is it for now - unless some questions
21:49:31 <mestery> Thanks Sukhdev.
21:49:44 <mestery> #topic Group Based Policy
21:49:48 <mestery> SumitNaiksatam: Hi again!
21:49:56 <SumitNaiksatam> mestery: hi, and thanks again
21:50:12 <SumitNaiksatam> rkukura was able to land the mapping patches this past week
21:50:35 <SumitNaiksatam> he will be landing some more driver patches later this week
21:50:48 <mestery> Yes, and I see some good iterations on the first couple of patches as well, addressing the UT concerns marun had pointed out.
21:50:55 <SumitNaiksatam> we already have had a good review churn on the initial model patches
21:51:07 <SumitNaiksatam> mestery: those have all been addressed
21:51:30 <SumitNaiksatam> thanks to the reviewers, and hopefully this is ripe now for more reviewer interest
21:51:52 <SumitNaiksatam> the series starts here #link https://review.openstack.org/#/c/95900
21:52:05 <mestery> #link https://review.openstack.org/#/c/95900
21:52:05 <nati_ueno> I'll join reviewing
21:52:26 <SumitNaiksatam> nati_ueno: thanks
21:53:03 <SumitNaiksatam> mestery: thats it from us on GP
21:53:09 <mestery> OK, thanks SumitNaiksatam!
21:53:28 <mestery> I encourage people to review the fresh GBP patches as well.
21:53:36 <mestery> #topic Open Discussion
21:53:48 <mestery> markmcclain: We have a request for a python-neutronclient release here:
21:53:49 <mestery> #link http://lists.openstack.org/pipermail/openstack-dev/2014-June/038389.html
21:53:57 <mestery> markmcclain: Can we make htis happen anytime soon? Thoughts?
21:54:03 <markmcclain> yes
21:54:16 <mestery> markmcclain: cool!
21:54:20 <mestery> I'll reply on that thread on the ML then.
21:54:20 <oda-g> I'd like cores to take a look at the blueprint https://review.openstack.org/#/c/89969/ (Enable to set DHCP port   attributes) I want core's suggestion to move forward.
21:54:24 <markmcclain> I'll work offline to show you how
21:54:37 <mestery> markmcclain: Perfect, thanks!
21:55:13 <mestery> oda-g: I'll have a look at that BP.
21:55:28 <oda-g> mestery: thanks
21:55:36 <carl_baldwin> oda-g: I also have some comments to make on that BP.
21:56:18 <mestery> OK, if that's it, we'll see you all on the ML and IRC!
21:56:23 <mestery> And of course, back here next week.
21:56:25 <mestery> #endmeeting