16:03:32 <Sukhdev> #startmeeting networking_ml2
16:03:33 <openstack> Meeting started Wed Feb 11 16:03:32 2015 UTC and is due to finish in 60 minutes.  The chair is Sukhdev. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:03:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:03:36 <openstack> The meeting name has been set to 'networking_ml2'
16:04:06 <Sukhdev> #topic: Agenda
16:04:12 <Sukhdev> #link: https://wiki.openstack.org/wiki/Meetings/ML2
16:04:41 <Sukhdev> Hopefully more people will join as we go along
16:04:54 <Sukhdev> #topic: Announcements:
16:05:14 <Sukhdev> Kilo-2 is out
16:05:43 <Sukhdev> FPF for Kilo is approaching  - March 5th
16:06:01 <Sukhdev> keep this in mind if you plan on proposing anything new in Kilo release
16:06:36 <Sukhdev> I do not have any other announcements - does anybody have any?
16:07:24 <Sukhdev> #topic: ML2 decomposition discussion
16:07:39 <Sukhdev> We had a very healthy discussion last week
16:08:19 <Sukhdev> I wanted to report that I completed the switchover to stackforge for the Arista Driver
16:08:23 <rkukura> have we made any progress documenting how to decompose?
16:08:48 <rkukura> Sukhdev: +1
16:08:53 <shivharis> Sukhdev: including CI?
16:09:13 <Sukhdev> rkukura: I plan on adding something to document
16:09:23 <Sukhdev> shivharis: Yes - it is all done
16:09:47 <shivharis> cool +1, i am focused on CI at this time
16:10:10 <Sukhdev> Since I implemented the CI by myself (did not use infra provided zuul framework), it is very easy for me to enhance
16:10:36 <Sukhdev> shivharis: CI on the neutron side or stackforge side?
16:10:54 <shivharis> neutron+stackforge
16:11:11 <shivharis> i.e.:
16:11:28 <shivharis> neutron with stackforge of your driver
16:11:43 <Sukhdev> I chatted with Armando about the documentation - he suggested to add to https://github.com/openstack/neutron/blob/master/doc/source/devref/contribute.rst
16:12:42 <Sukhdev> If anybody needs any help or clarification, feel free to ping me - I will be happy to help
16:13:13 <shivharis> i will also add to this if i see holes to be covered
16:13:26 <Sukhdev> you can see Arista code by simply invoking "sudo pip install networking_arista"
16:13:52 <Sukhdev> shivharis: cool
16:14:35 <Sukhdev> Anything else on the decomposition?
16:15:24 <Sukhdev> #topic: ML2 sync - Task Flow
16:15:43 <Sukhdev> manishg: are you here?
16:16:13 <Sukhdev> manishg: posted update to https://review.openstack.org/#/c/154333/
16:16:15 <manishg> hi
16:16:27 <manishg> I started with this patch
16:16:29 <Sukhdev> I do not if folks have had a chance to look at it
16:16:44 <manishg> there is still bunch of stuff to add.
16:17:00 <manishg> I hope to have the async part done today or tomorrow.
16:17:09 <Sukhdev> manishg: I stated to look at it did not understand the mock1/2 stuff - other was reasonably clear
16:17:17 <rkukura> manishg: Since most of us aren’t very familiar with TaskFlow, can you summarize how TaskFlow is used in this so far?
16:17:33 <manishg> mock1/2 is just a sample set of drivers that shows how you can split
16:17:37 <manishg> tasks in a driver.
16:17:48 <manishg> and it shows some of the reverting mechanism.
16:18:05 <manishg> so the driver can split tasks internally any which way.
16:18:08 <Sukhdev> manishg: So, those will become tests ?
16:18:11 <manishg> will add more documentation
16:18:19 <manishg> no those are just samples in PoC
16:18:45 <manishg> those won't really go in patch - but it's just to give an idea on how one could break a driver
16:18:46 <Sukhdev> I see - please add some documentation - that will help
16:18:48 <manishg> if they wanted to
16:18:51 <manishg> sure
16:18:57 <manishg> need to add a lot more
16:19:08 <HenryG> Maybe name them example1/2 instead of mock.
16:19:23 <Sukhdev> HenryG: +1
16:19:26 <manishg> HenryG: sounds good.  will do.
16:19:34 <manishg> here are the steps I'm thinking of :
16:19:49 <manishg> 1) same as current, except revert etc. is easier (with taskflow)
16:20:04 <manishg> 2) add db changes (for intermediate states)
16:20:11 <manishg> 3) add async part
16:20:25 <rkukura> manishg: In the normal case, does TaskFlow call the execute() and then post_execute() methods on each MD? Does it do this after the REST call may have returned?
16:20:26 <manishg> 4) persistence - so if process dies tasks can be resumed
16:21:06 <manishg> rkukura: post-execute is always called (it's mainly for cleanup etc. if needed). re REST call
16:21:21 <manishg> I need to make it async first. (#3 above).
16:21:44 <manishg> the issue is that there needs to be an intermediate state for meaningful failure handling
16:21:47 <manishg> in async case.
16:21:55 <rkukura> manishg: So right now, the calls to .run() run them synchronously?
16:22:16 <manishg> yeah.  so it just gives an idea of some change.
16:22:30 <manishg> I could replace .run() with executor.submit(eng.run)
16:22:34 <manishg> and it will make it async
16:22:56 <manishg> but it won't be perfect as exception handling won't be that clear (revert etc)
16:23:19 <HenryG> manishg: This is post-Kilo work, right?
16:23:22 <rkukura> manishg: Is see. Thanks.
16:23:28 <manishg> Do we all agree on the intermediate states part?
16:23:45 <manishg> HenryG: probably.  we want to explore some ways to solve this problem.  and I suspect
16:23:56 <manishg> that API/ RPC split will also have an impact on this
16:24:05 <manishg> which is kilo-3 now, right?
16:24:29 <manishg> I think kevinbenton hasn't put out a patch yet for the pecan work.
16:24:34 <rkukura> manishg: By intermediate states do you mean that there would be creating/updating/deleting states until all tasks for these complete?
16:25:07 <manishg> yeah, if we make it async then the idea is we want to do the db update (after validation) and then mark it as 'in-progress' (creating, etc.)
16:25:16 <manishg> and then dispatch tasks to drivers.
16:25:25 <Sukhdev> manishg: new neutron API will be introducing intermediate states - will this not align with those?
16:25:40 <manishg> we could have an intermediate table (or let taskflow handle that) where each driver state is maintained until successful.
16:26:14 <manishg> Sukhdev: yeah.  that's why I had stopped work on this earlier (due to API/ RPC split)
16:26:48 <manishg> which would basically split all API work into - db related update and the actual work (which can be async and could be done on another node)
16:27:00 <rkukura> Sukhdev: Is “new neutron API” part of the pecan BP?
16:27:10 <manishg> rkukura: yes.
16:27:22 <manishg> + there is another spec (by salvatore) about object model.
16:27:30 <Sukhdev> rkukura: I am not absolutely sure about it - but, my guess is yes
16:27:30 <HenryG> Sorry, what is the "API/RPC split"?
16:27:52 <manishg> HenryG: I'm refering to the pecan and object-model work. this was discussed in
16:28:03 <manishg> paris and mid-cycle.
16:28:22 <HenryG> manishg: thanks
16:28:47 <manishg> The idea that mark had mentioned that REST call shouldn't be waiting for the backend
16:28:51 <manishg> which can take a while.
16:29:03 <manishg> similar requirements for ML2 as well as LBaaS
16:29:23 <manishg> LBaaS is using taskflow (based on last chat with one of the devs)
16:29:45 <manishg> they have very similar requirements but don't have postcommit, etc.
16:29:56 <Sukhdev> manishg: So, then what is the concern about intermediate states - that you asked earlier? This should align well, right?
16:30:45 <manishg> rkukura: with the async part, we'll need to track the state of each mech driver.  I was thinking that could be another table (which will hold interim states of each driver given a resource-id + operation)
16:30:59 <manishg> Sukhdev: no concern.  just wanted to make sure we are in sync
16:31:12 <shivharis> manishg: are these tasks thread safe? will they potentially run in parallel
16:31:15 <rkukura> manishg: Something like that may be needed
16:31:44 <manishg> shivharis: yes.  they can run in parallel (it is up to us).  for example,
16:32:04 <manishg> within a driver you could generate a set of tasks to complete the job.  and they maybe serial or parallel.
16:32:15 <rkukura> manishg: I think your current patch is a great start, and will enable more concrete discussion of the next steps.
16:32:33 <manishg> ok, cool.  will add more and add lot more documentation :)
16:32:41 <manishg> including about thoughts on future work
16:32:43 <manishg> sounds good?
16:32:54 <shivharis> manishg: +1
16:32:54 <rkukura> manishg: +1
16:33:02 <Sukhdev> manishg: +1
16:33:17 <Sukhdev> Look forward to clear documentation, manishg
16:33:37 <Sukhdev> Folks, anything else on this topic?
16:33:39 <manishg> The link to patch is on agenda.  please add yourself as reviewer.
16:34:11 <rkukura> I’ll review, and also need to review the related kilo specs.
16:34:35 <Sukhdev> I already started to review it an got lost with mock1/2 :-):-)
16:35:00 <Sukhdev> shall we move on?
16:35:00 <manishg> I'll replace mock with example (my bad, thanks for suggestion HenryG).
16:35:25 * Sukhdev waiting for any more comments before moving on
16:35:26 <manishg> let's move on.  if you have comments please comment on the patch.
16:35:40 <Sukhdev> #topic: ML2 Extension Driver API enhancements
16:35:57 <Sukhdev> Is Shweta here ?
16:36:00 <shwetaap> yea
16:36:23 <shwetaap> so my patch for the extension driver is here
16:36:24 <shwetaap> https://review.openstack.org/#/c/152759/4
16:36:51 <shwetaap> i have a question regarding the extend_xxx_dict  method in driver_api
16:37:25 <rkukura> Note that this is needed for the PortSecuity BP implementation, which is trying to re-use the portsecurity_db mixin code via an ML2 extension driver.
16:37:41 <shwetaap> the existing implementation passes the db  session to the method
16:39:07 <Sukhdev> I have not had a chance to review this yet
16:39:44 <shwetaap> Although when it is called in https://github.com/openstack/neutron/blob/master/neutron/plugins/ml2/plugin.py
16:39:45 <rkukura> I’ve been reviewing this one, and see that the existing ED API doesn’t work well with the existing portsecurity_db code.
16:40:30 <shivharis> if you pass the session, then the extension driver must be very quick in whatever it needs to do... right?
16:40:50 <rkukura> And I am convinced that some non-backwards-compatible changes to the ED API are needed to enable this. The other option would be to not re-use the portsecurity_db code.
16:41:20 <rkukura> shivharis: These ED calls are made within transactions - they are intended to move data between dicts and DB models.
16:41:42 <shivharis> rkukura: ok
16:41:46 <shwetaap> i could pass the session i guess, but the function _ml2_md_extend_network_dict(self, result, netdb): in plugin.py also has the netdb variable that is never made use of
16:41:52 <shwetaap> in the existing implementation
16:41:57 <shwetaap> do we leave that as is?
16:42:47 <rkukura> Basic question to me seems to be whether breaking existing EDs is a concern.
16:43:39 <yamahata> what's the existing EDs?
16:43:40 <rkukura> If we agree ED API enhancements are needed and acceptilble, then we can worry about the details.
16:43:48 <yamahata> so far I don't know any.
16:44:31 <rkukura> I don’t see any EDs in-tree.
16:45:11 <Sukhdev> If what yamahata says is true - then this seems to be slam dunk - but, seems too good to be true :-):-)
16:45:21 <rkukura> I know Cisco’s DFA driver plans to use one, but isn’t merged.
16:46:15 <Sukhdev> rkukura: So, if one or two cases are impacted, it may be easier to coordinate with them  - it may not be too bad
16:46:29 <rkukura> The GBP code also supports a similar ED API, but we can update that if it makes sense.
16:46:59 <rkukura> So one enhancement is basically to pass plugin_context in place of, or in addition to, the DB session.
16:47:29 <rkukura> And the other is to pass the base resource’s model instance to the extend_dict functions.
16:48:17 <shwetaap> both these changes are in my current patch
16:48:29 <rkukura> One thoughts was that some or all the ED methods should take some sort of ML2 Context objects so they can be extended in the future without breaking drivers.
16:48:58 <rkukura> Similar to the PortContext, NetworkContext, and SubnetContext we pass to MDs.
16:50:13 <rkukura> I’d suggest we discuss via the gerrit review, and move on.
16:50:43 <shwetaap> ok
16:50:59 <Sukhdev> rkukura: Can you add this as review comments to the patch and let others review the patch along with you comments and hopefully get to some resolution
16:51:13 <rkukura> Sukhdev: will do
16:51:26 <Sukhdev> Anything else on this patch?
16:51:39 <shwetaap> no
16:51:40 <Sukhdev> #topic: Bugs
16:51:46 <shivharis> hi
16:52:02 <shivharis> we need help in reviews
16:52:06 <Sukhdev> shivharis: what say you, sir
16:52:11 <shivharis> for:https://bugs.launchpad.net/neutron/+bug/1179223
16:52:11 <openstack> Launchpad bug 1179223 in neutron "Retired GRE and VXLAN tunnels persists in neutron db" [High,In progress] - Assigned to Romil Gupta (romilg)
16:52:32 <shivharis> can i seek some volunteers here?
16:52:58 <manishg> shivharis: I'll try to review this week.
16:53:07 <rkukura> I’ve been reviewing this one, and will review the latest patch set.
16:53:08 <nlahouti> shivharis:I can help
16:53:13 <Sukhdev> I will look at it as well
16:53:29 <shivharis> manishg, rkukura, Sikhdev, nlahouti: thanks
16:53:43 <shivharis> other high bugs seem to be all in good hands
16:53:53 <manishg> rkukura, HenryG, romilg thanks for reviewing this one - https://review.openstack.org/#/c/124917/ - can we get another core to review?
16:54:16 <shivharis> moving forward, i hope all the vendor specific bugs will move to stackforge as well?
16:54:55 <Sukhdev> shivharis: Not until L… release
16:55:21 <shivharis> i usually do not worry too much about those now
16:55:45 <Sukhdev> shivharis: Happy to report that one of Arista driver bug has already been diverted to stackforge
16:56:16 <shivharis> please pick up some medium non-vendor specific bugs if you can
16:56:17 <rkukura> shivharis: I hope to get the next step of https://review.openstack.org/#/c/152759/ out for review before next weeks meeting
16:56:35 <shivharis> rkukura: thanks that is k-3
16:56:54 <rkukura> shivharis: yes - the first part made k-2
16:57:13 <shivharis> yes, thanks rkukura
16:57:15 * Sukhdev time check 3 min.
16:57:37 <Sukhdev> Anything else on bugs>
16:57:38 <shivharis> overall we look fairly good for ml2 bugs at this time. thanks folks for the efforts
16:57:54 <shivharis> moving on...
16:57:57 <Sukhdev> #topic: Open Discussion
16:58:08 <Sukhdev> Anybody has anything?
16:58:16 * Sukhdev waiting
16:58:32 <rkukura> shivharis: Do you feel the current bug importance setting are reasonable?
16:59:01 <shivharis> rkukura: seems to be looking fine, no show stoppers per se
16:59:06 <rcurran> anyone using linuxbridge MD + another MD that requires context.bound_segment info?
16:59:37 <Sukhdev> rcurran: not me
16:59:42 <rcurran> finding that once port['status'] leaves DOWN state that bound_segment is None
16:59:52 <rcurran> this isn't happening w/ openvswitch
17:00:06 <rkukura> rcurran: Sounds like a bug
17:00:27 <rcurran> (actually I'm using rkukura's HPB code so I'm accessing via top/bottom_bound_segment's
17:00:27 <HenryG> rcurran: Do you suspect a bug in the LB MD?
17:00:28 <Sukhdev> rcurran: Open a bug and attach your finidings
17:01:11 <Sukhdev> Folks we are up against the clock
17:01:13 <rkukura> rcurran: The MDs are trivial and share the same base code. Could be difference in agent RPC behaviour.
17:01:21 <Sukhdev> Thank you all for joining
17:01:33 <rcurran> rkukura, looks that way
17:01:45 <rkukura> Thanks Sukhdev!
17:01:51 <Sukhdev> Thanks
17:01:55 <Sukhdev> #endmeeting