16:03:53 <rkukura> #startmeeting networking_ml2
16:03:54 <openstack> Meeting started Wed Mar 25 16:03:53 2015 UTC and is due to finish in 60 minutes.  The chair is rkukura. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:03:55 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:03:58 <openstack> The meeting name has been set to 'networking_ml2'
16:04:14 <rkukura> #topic Agenda
16:04:38 <rkukura> #link https://wiki.openstack.org/wiki/Meetings/ML2
16:05:25 <rkukura> just the usual agenda - anyone want to add anything?
16:06:13 <rkukura> OK, …
16:06:19 <rkukura> #topic Announcements
16:06:30 <rkukura> RC's are April 9-23 and Kilo release on April 30
16:07:01 <rkukura> #link RC-1 content: https://launchpad.net/neutron/+milestone/kilo-rc1
16:07:43 <rkukura> Any other announcements?
16:08:44 <rkukura> #topic ML2 Drivers decomposition discussion
16:09:00 <rkukura> Anything new on this, or any issues to discuss?
16:10:11 <amotoki> I have nothing, but i think we need to discuss linuxbridge CI for liberty.
16:10:20 <rkukura> sadasu: You had some issues last time - have they been resolved?
16:10:34 <shivharis> is it still possible for some shim to still get in by rc1
16:11:01 <shivharis> some seems to be marked as such
16:11:02 <Sukhdev> sadasu: I see your bug is included in RC1 content list - So, I assumed everything worked out
16:11:23 <Sukhdev> shivharis: yes
16:11:33 <rkukura> amotoki: agreed CI is needed - rcurran found one issue with LB introduced by my HPB work
16:11:40 <amotoki> shivharis: no strict criteria so far. If it is stable, I think it is okay.
16:11:41 <Sukhdev> shivharis: Does it require a spec?
16:11:41 <shivharis> Sukhdev: ok, thanks
16:11:58 <shivharis> Sukhdev: no, but the bug got marked as RC1
16:12:00 <Sukhdev> shivharis: If spec is required, then no - otherwise, yes
16:12:11 <shivharis> Sukhdev: ok
16:12:25 <sadasu> rkukura, Sukhdev: I am all set with the vendor spilt..thansk for checking!
16:12:34 <rkukura> sadasu: great!
16:12:42 <rkukura> Anything else on decomposition today?
16:13:05 <Sukhdev> sadasu: I noticed your patch is not linked to your bug - found it a bit odd :-)
16:13:21 <amotoki> Sukhdev: shivharis: a spec for decomposition is not required.
16:13:33 <amotoki> we no longer need a spec for new drivers too.
16:13:44 <Sukhdev> amotoki: good point - thanks
16:13:48 <rkukura> amotoki: Do we need these linked to bugs then?
16:14:06 <shivharis> amotoki: got it, a bug is sufficient
16:14:37 <amotoki> yes. all vendor decomposition stuffs are tracked thru bugs.
16:14:56 <rkukura> OK, anything else on decomposition before we move on?
16:14:59 <amotoki> it is recommended in the vendor decomposition documents
16:15:02 <shivharis> would like to commend armax for all the help he provided at all odd hours, three cheers
16:15:10 <Sukhdev> rkukura: If you notice, sadasu bug is included in the RC1, but, her patch is not linked to the bug - hence, I thought it was bit odd
16:15:31 <rkukura> sadasu: Can you update your patch to reference the bug in the commit message?
16:15:49 <armax> shivharis: thanks! I was impressed all the good work that happened this cycle
16:16:23 <shivharis> armax: thanks a LOT (all caps)
16:16:40 <rkukura> I have a question for the team on decomposition...
16:16:52 <sadasu> rkukura: I have, but not sure why it isn't showing up in the bug..will look again
16:17:15 <amotoki> no spec is required for new drivers. it is documented here http://docs.openstack.org/developer/neutron/devref/contribute.html#blueprint-spec-submission-strategy thanks armax
16:18:22 <rkukura> When this was all debated and discussed and planned, we seemed to have some concensus around keeping the code that implements and uses the ML2 driver API as part of the in-tree shim layer. I see that some drivers have done this, while others have all executable code outside the neutron tree.
16:18:32 <Sukhdev> sadasu: check commit message on the patch
16:19:00 <sadasu> Sukhdev: will do
16:19:32 <rkukura> Do we still feel there is potential value in keeping the code that is exposed to driver API evolution in-tree?
16:19:45 <Sukhdev> rkukura: during that discussion, it was left to the vendors to make the decision as to how much they want to keep in the tree and what they want to take out
16:20:40 <armax> rkukura: I think we need to sit down at the summit and discuss next steps
16:20:49 <rkukura> Sukhdev: Agreed there is flexibility, and I’m not arguing for complete uniformity.
16:20:50 <Sukhdev> rkukura: I think over time folks may move more and more code out of the tree depending upon the changes
16:20:52 <armax> rkukura: I don’t believe we’re completed yet
16:20:53 <shivharis> rkukura, Sukhdev: that was my understanding as well, the vendor decides
16:21:14 <rkukura> Defintitely seems like a good design sumit topic
16:21:38 <shivharis> summit topic: +1
16:21:38 <rkukura> OK, anything else on decomposition today?
16:21:59 <rkukura> #topic ML2 Sync and error handling (Task Flow)
16:22:07 <rkukura> manishg_: Any update on this?
16:22:26 <rkukura> I have not had a chance to revisit the current patch and the comments on it myself
16:22:33 <shivharis> #action: Add decomposition extent as Summit topic in some etherpad (to be decided)
16:22:52 <manishg_> rkukura: the only comment I got in patch was from Sukhdev.  So want to discuss that.
16:23:06 <rkukura> #action Add next steps for decomposition to list of design summit topics
16:23:36 <manishg_> To do what Sukhdev is suggesting, we need to have some RPC or API change.  But I suggest we do that in round two.
16:24:01 <banix> i am not sure if that is needed though
16:24:20 <Sukhdev> banix: would love to hear your point of view
16:24:22 <manishg_> banix: can you elaborate
16:24:26 <rkukura> #link https://review.openstack.org/#/c/154333/
16:24:28 <banix> is it possible to eventually reach a consistent state without freezing updates sukhdev?
16:25:14 <banix> I am thinking (and I may be totally off!) that if the resync happens on an accumulative basis, we may not need the freeze
16:25:15 <Sukhdev> banix: eventually, maybe, but, will create a lots of churn in the system -
16:25:17 <manishg_> Sukhdev's point is that the backend switch may at any point (after reaching consistent state) may declare to neutron plugin that the state has changed
16:26:09 <banix> Manish, well the authorative source is the Neutron DB. right?
16:26:14 <Sukhdev> banix: keep in mind a scenario where you have multiple Neutron controllers -
16:26:23 <rkukura> I think we need to decide between a model where the the backends are eventually consistent with the DB state vs. a model where DB state can’t change until backend state is synced.
16:26:35 <manishg_> banix: and Sukhdev is proposing that we change that so that we let the backend update the db state.
16:27:13 <Sukhdev> rkukura: good way to put it
16:27:27 <manishg_> I think there may be confusion here about eventually consistent.  From what I understand, Sukhdev's point is that after DB state is ACTIVE (everyone agreed it is ACTIVE ;
16:27:34 <manishg_> everyone = all registered drivers)
16:27:41 <banix> hmmmm manishg_ Sukhdev that sounds problamatic but sukhdev has thought much more about this
16:28:11 <manishg_> and then the backend can come and say 'hey , wait a minute.... I'm all messed up again... pls change state to CREATINGT'
16:28:15 <Sukhdev> banix: I want to make sure that you understood the issue correctly
16:28:28 <banix> so i guess the question is if that is necessary
16:28:33 <manishg_> Sukhdev, can you comment on if I expressed your point correctly?
16:28:42 <banix> Sukhdev: yes i may be missing the point
16:28:50 <Sukhdev> manishg_: you are right on -
16:29:07 <manishg_> thanks Sukhdev.
16:29:31 <Sukhdev> banix: Say you are in ACTIVE state - all is OK evreybody is in sync
16:29:40 <manishg_> banix: so my point is , that if we want to do this then you need another API to come back and update sate (you can't do it under the covers)
16:29:55 <rkukura> manishg_, Sukhdev: Are you to proposing the further state changes via the API be blocked when one or more backends are out of sync?
16:30:19 <manishg_> rkukura: Sukhdev's point is independent of wether we block or not.
16:30:20 <Sukhdev> and, you have two or three controllers running (of course one common DB), with 10K+ ports in question on 4K networks
16:30:55 <manishg_> I pointed out two clear issues in the patch.  one of them is about blocking further operations.
16:30:56 <Sukhdev> at this point, the back-end resets (or switches over in HA case) - you need to bring it back into sync state
16:31:01 <manishg_> another is about what Sukhdev raised.
16:31:36 <banix> Sukhdev: come on now. i thought you were using a single node install for everything :) just kidding… please continue
16:31:38 <manishg_> Sukhdev: what you are asking for is basically monitoring feedback and based on that updating the active state.
16:32:19 <rkukura> I’d like some read-only attribute in the API (status or something else) to show whether or not the backends are all in sync, but am opposed to failing or blocking new API requests while backends are out of sync.
16:32:34 <manishg_> From neutron point of view, all we really need to decide is if state-change is permissible by driver.  If yes, need an api.
16:32:45 <Sukhdev> while you are brining the back-end into sync the other cotrollers are spraying addiional transactions, some of them are updating resources that you have aready synced
16:32:50 <manishg_> rkukura: complications about blocking or non-blocking mentioned in patch comments.
16:33:06 <Sukhdev> and some are being updated while they are in the process of being synced...
16:33:54 <manishg_> Sukhdev: your point is similar to this.... nova should change the state of VM based on wether it's pingable or not (just an example)
16:34:14 <Sukhdev> You potentially could end up in a state when DB and back-end go out of sync (unless you create a protocol between the controllers to signal that they are done)
16:34:15 <manishg_> and there could be an agent that changes this state - the one that is pinging (e.g.).
16:34:30 <banix> Sukhdev: I think the sync shouldn’t be a start from scratch kind of operation…. blocking the incoming requests would be a big issue
16:35:08 <manishg_> How about we discuss the blocking issues first then.
16:35:14 <rkukura> I’d really recommend we focus first on the right behavior from the point of view of normal and admin client APIs, then figure out how to evolve ML2 and its drivers to provide that.
16:35:15 <manishg_> Has anyone read my comments in the patch?
16:35:26 <amotoki> btw (I am a bit confused. let me ask as a newcomer)  Is there any docs about problem description, what we try to address, and possible approaches?
16:35:26 <manishg_> rkukura: +1
16:35:27 <Sukhdev> we all said things in parallel - let me go back and read everybody's comments :-)
16:35:46 <amotoki> it seems we are discussing multiple things derived from async discussion: state model, neutron-backedn sycn, asycn operation and so on.
16:35:54 <banix> my thinking was and I may be wrong let me admit, that the controller is teh agent that could detect if anything is out of sync
16:35:57 <manishg_> Sukhdev: that's why let's discuss one by one.  banix: Sukhdev's main point is independent of blocking issue.
16:36:16 <banix> manishg_: go ahead
16:36:24 <manishg_> let's discuss the blocking issue.  the issue is this
16:36:43 <manishg_> while we are in intermediate state, should operations be permitted on that resourse
16:36:59 <manishg_> i.e. based on a transition table.
16:37:15 <banix> (by controller i meant the plugin in my last comment)
16:37:20 <manishg_> or should we not block and just let users do whatever they would have done , as-if sync
16:38:00 <manishg_> I have pointed out the issue in the comment in the patch.  I think the discussion can happen there too - since these are fundamental issues.
16:38:18 <rkukura> Do we have any hard requirements that drivers see all intermediate states? Or do they just need to move from whatever their previous state was to whatever the current state is?
16:38:19 <banix> manishg_: i think this is a bit different from what Sukhdev had written with respect to blocking
16:39:07 <manishg_> banix: there are two issues. Sukhdev , as far as I could see, wsa talking about specific implementation.
16:39:37 <manishg_> can folks please read the patch?  I have pointed out the issues (hopefully in a lucid manner but let me know if there is confusion)
16:40:09 <Sukhdev> sorry - I was reading everybody's comments - back now :-)
16:40:28 <banix> Sukhdev: but now you are out of sync with the latest
16:40:33 <GLaupre> (hard to follow :P)
16:40:43 <Sukhdev> So, I am not really proposing blocking for everything - just on a specific resource which is being sync'ed
16:40:59 <Sukhdev> banix: :-)
16:41:42 <Sukhdev> Or alternatively, a separate flag which states that the system is not in fully sync state
16:41:50 <rkukura> Sukhdev: Doesn’t that sort of blocking push a lot of complexity into the clients? Clients need to know to retry, deal with API timeouts, etc.
16:41:54 <banix> manishg_: Sukhdev got it wrt blocking
16:42:03 <banix> manishg_: could you state the second issue
16:42:05 <amotoki> In my understanding, out of sync and being in a pending state are different things.
16:42:19 <manishg_> let me copy/ paste from the patch comment.
16:42:19 <Sukhdev> rkukura: correct - it does make it complex -
16:42:31 <manishg_> This was discussed in ML2 meeting again today. Two main points from there:
16:42:31 <manishg_> 1) Allow changing of state from ACTIVE -> Creating based on some event. 2) Allow operations while in Intermediate state
16:42:48 <rkukura> manishg_: I was confused on that.
16:42:52 <Sukhdev> rkukura: hence, I am not saying solve it only in one way - we should collectively think and come up with a clean soluton
16:43:34 <manishg_> based on above #1 is what Sukhdev was asking for.  #2 is what rkukura was suggesting.
16:43:59 <manishg_> #2 here "Allow operations while in Intermediate state" is more of a fundamental decision that need to be made.
16:44:08 <rkukura> manishg_: Right
16:44:13 <manishg_> I suggest we put the other issue on the backburner till we're done with this.
16:44:24 <manishg_> otherwise we'll keep doing some crosstalk and confuse the issues.
16:44:35 <rkukura> Let’s try to wrap up this topic in a coupe minutes so we have some time to discuss bugs.
16:44:58 <Sukhdev> manishg_: keep going...
16:45:03 <manishg_> rkukura: many folks said they are interested in this however I do not see anyone commenting in the patch.
16:45:18 <amotoki> manishg_: the code does not explain the thing enough. can we document the approaches we discussed so far?
16:45:28 <Sukhdev> manishg_: one word - busy :-):-)
16:45:40 <manishg_> amotoki : the comments have details.
16:45:41 <banix> manishg_: we are introvert kind of people. talk to ourselves
16:45:55 <manishg_> haha :)
16:46:00 <rkukura> manishg_: I expect myself (and maybe others) to be able to give this more attention once kilo work is wrapped up
16:46:03 <banix> will go to the review page
16:46:22 <manishg_> rkukura: okay thanks.  Should we put this on hold till then?
16:46:30 <manishg_> otherwise, we won't be able to make much progress.
16:46:49 <manishg_> as the time is limited here.
16:46:54 <Sukhdev> manishg_: I did review the code
16:47:12 <manishg_> Sukhdev : just the comments (where I point out the issues).  Should I create a doc?
16:47:20 <manishg_> we can do that too.
16:47:27 <manishg_> let me know what is a convenient forum.
16:47:46 <Sukhdev> manishg_: Actually writing a document is a great idea
16:47:47 <manishg_> in the comment (Mar/11) I pointed out issues and problems solving some of them.
16:47:48 <rkukura> Maybe we should schedule a separate meeting on this topic rather than spending a few minutes on it each week. I think we could all better prepare for that
16:48:00 <manishg_> rkukura: +1
16:48:16 <amotoki> +1
16:48:27 <manishg_> Sukhdev: ok, will write doc.  If I can get some folks to review it!
16:48:30 <Sukhdev> manishg_: I would also consider a design session or pod session in Vancouver - document will help with it
16:48:34 <banix> rkukura: +1
16:48:56 <manishg_> Sukhdev: considering not going to Vancouver this time... so will do a doc.
16:49:03 <amotoki> we can start etherpad or google doc to gather all things.
16:49:20 <banix> etherpad should do
16:49:22 <rkukura> #action manishg_ to gather discussion points into doc, schedule meeting soon to discuss, all leading to a plan for a design summit session
16:49:27 <Sukhdev> manishg_: if you go to vancouver, I'll buy you a beer
16:49:51 <manishg_> rkukura: sounds good.
16:49:58 <manishg_> Sukhdev: :)
16:49:58 <rkukura> thanks
16:50:11 <rkukura> #topic Bugs
16:50:16 <rkukura> shivharis: ...
16:50:22 <shivharis> hi
16:50:45 <shivharis> we have a bug (if it is really a bug) will require attention ASAP
16:50:50 <shivharis> https://bugs.launchpad.net/neutron/+bug/1435216
16:50:51 <openstack> Launchpad bug 1435216 in neutron "unit tests: KeyError: 'port_security' when building Debian package" [High,Confirmed] - Assigned to yalei wang (yalei-wang)
16:51:39 <shivharis> eugene: marked it as confirmed, but not sure on what basis
16:51:41 <yamahata> I'll look into it with Yalei.
16:52:00 <shivharis> thanks yamahata
16:52:15 <shivharis> if this is really an issue it is a must fix
16:52:28 <shivharis> for kilo
16:53:11 <shivharis> other than this one we look good - some are in progress for rc1 but again those are not as critical as the one above
16:53:42 <shivharis> any other question regarding bugs
16:53:52 <rkukura> I’m still hoping to get a fix for https://bugs.launchpad.net/neutron/+bug/1367391 into review soon - good progress the past few days
16:53:54 <openstack> Launchpad bug 1367391 in neutron "ML2 DVR port binding implementation unnecessarily duplicates schema and logic" [High,In progress] - Assigned to Robert Kukura (rkukura)
16:54:15 <shivharis> rkukura: thanks for that
16:54:42 <shivharis> rkukura: this one does not bother me much for 2 reasons
16:54:53 <shivharis> it is in good hands and it is not a show stopper
16:55:27 <shivharis> yamahata: the one you will work out with yalie is a show stopper
16:55:28 <amotoki> it is not a show stopper but i think it help us maintain the released code much
16:55:36 <rkukura> shivharis: definitely not a show stopper, but substantial amount of change
16:55:40 <yamahata> shivharis: sure.
16:55:58 <shivharis> thanks rkukura and yamahata
16:55:58 <yamahata> anyway I'll look into it.
16:56:09 <shivharis> thats all for bugs
16:56:36 <rkukura> shivharis: Thanks!
16:56:47 <rkukura> #topic Open Discussion
16:57:12 <rkukura> Anything to discuss, or should we finish a couple minutes early?
16:58:05 <GLaupre> Bye? ^^
16:58:45 <rkukura> Thanks everyone!
16:58:46 <manishg_> thanks rkukura. bye!
16:58:47 <banix> thanks and bye
16:58:50 <rkukura> #endmeeting