09:00:20 <gsagie> #startmeeting dragonflow
09:00:21 <openstack> Meeting started Mon Jan 25 09:00:20 2016 UTC and is due to finish in 60 minutes.  The chair is gsagie. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:00:24 <openstack> The meeting name has been set to 'dragonflow'
09:00:31 <yuli_s> Hello !
09:00:53 <gsagie> #info gampel, Shlomo_N, yuli_s, oanson, gsagie in meeting
09:01:09 <gsagie> Even that oanson is not talking ;)
09:01:36 <gsagie> #topic security groups
09:01:45 <dingboopt> o/
09:01:52 <gsagie> hi dingboopt
09:01:57 <dingboopt> hi
09:01:59 <gsagie> #info dingboopt in meeting as well :)
09:02:07 <dingboopt> I have some discussion with beijing guy
09:02:18 <gsagie> dingboopt: I havent seen Raofei works on security groups ?
09:02:19 <dingboopt> and we improve the flow design
09:02:25 <gampel> dingboopt: just reviewed your patch take a look
09:02:32 <gsagie> he is listed to work on this and DNAT..
09:02:48 <gsagie> dingboopt: you mean the design with the conjunction?
09:02:53 <dingboopt> yes
09:03:10 <dingboopt> some improvements
09:03:12 <gsagie> dingboopt: can you please post a document describing the new design?
09:03:19 <dingboopt> and reach agreents
09:03:22 <dingboopt> not a new desing
09:03:25 <dingboopt> design
09:03:48 <gsagie> dingboopt: yes, just we need something that cover and has all the details about it
09:03:51 <dingboopt> ok, we will send you all a letter describing the final desing
09:04:01 <dingboopt> desgin
09:04:10 <dingboopt> I will
09:04:10 <gsagie> dingboopt: its better to just upload it to the specs in Dragonflow and we can iterate it in the review board
09:04:16 <gsagie> its much easier to comment
09:04:21 <gsagie> Is that ok?
09:04:21 <dingboopt> ok
09:04:26 <dingboopt> ye
09:04:39 <gsagie> #action dingboopt update design for security groups with improvments and conjunction
09:05:04 <gsagie> dingboopt: you in contact with the guy with beijing, is he going to work on it as well? (contributing code)
09:05:17 <gampel> duankebo will join in a minute
09:05:18 <gsagie> Hello DuanKebo :)
09:05:22 <dingboopt> he is now get familiar with the code
09:05:31 <gsagie> #info DuanKebo in meeting as well :)
09:05:47 <raofei> yes, we discuss the security group solution together with Beijing team.
09:06:01 <gsagie> raofei: are you going to help them implement this?
09:06:08 <DuanKebo> Hi Gal!
09:06:51 <raofei> this week, maybe tomorrow, we will commit the spec to review. sure, I will join the implemention.
09:07:15 <gsagie> okie great, so first step lets see the spec and agree, we will review it quickly and dingboopt already working on the plugin side
09:07:28 <gsagie> What is the guy name from beijing that works on it?
09:07:37 <dingboopt> yuanwei
09:07:39 <gsagie> yuanwei right?
09:07:41 <gsagie> okie
09:08:05 <gsagie> #action dingboopt, raofei, yuanwei to start splitting work on security groups after spec is merged
09:08:36 <gsagie> Okie, you will need to split the work between you three, dingboopt, raofei keep in mind you guys also listed as working on port security and DNAT
09:08:55 <gsagie> okie, anything else on security groups?
09:08:59 <gsagie> hi kexiaodong :)
09:09:07 <gsagie> #info kexiaodong in meeting
09:09:11 <gampel> we need to update the DNAT spec to refract the changes
09:09:34 <gsagie> gampel: yes i will do it but its very similar, lets talk about it when we reach that topic soon
09:10:07 <gsagie> DuanKebo: anything from your team regarding security groups? or things seems to be ok?
09:10:10 <gampel> ok
09:10:47 <raofei> no more,  what we need to do is commit the spec patch ASAP for review
09:10:47 <DuanKebo> Yuanwei is working together with Raofei and Dingbo
09:10:55 <DuanKebo> Everything is OK
09:11:06 <gsagie> okie, we will review it fast for you so you can start working on it
09:11:10 <gsagie> #topic port security
09:11:29 <gsagie> dingboopt: i need to update the patch with your comments which were good, anything else we need here?
09:11:44 <dingboopt> no
09:11:48 <gsagie> okie
09:12:02 <gsagie> #action gsagie update port security spec with latest comments
09:12:24 <gsagie> dinboopt: will need to work on this as well in addition to security groups, i will put everything we talked about in the spec ASAP
09:12:38 <dingboopt> ok
09:12:46 <gsagie> #action dingboopt implement port security spec once merged
09:12:59 <gsagie> #topic DB consistency
09:13:06 <gsagie> ok, nick-ma isnt here
09:13:38 <gsagie> Hujie from DuanKebo team is working on this as well
09:13:48 <DuanKebo> Hujie is working on this feature
09:14:48 <gsagie> From the feedback i received,  its possible to serialize write transactions to get consistency and most performance issues with SQL clusters (Galera) are for the reads (as Horizon/Nova) perform alot of reading from neutron
09:15:11 <DuanKebo> He will submit a spec for review this week
09:15:16 <gsagie> so implementing the distributed lock with a special table record
09:15:58 <gampel> gsagie: so does Galera support looked write ?
09:16:18 <gsagie> gampel: yes, there is a different behaivor as the transaction is terminated
09:16:48 <DuanKebo> Hujie and me are not familiar with you solution about the distribution lock.
09:17:12 <gsagie> but we can still have serialized transactions, i talked with kevinbenton about it, they used a similar solution for Big Switch, he said they didnt notice problems on the writes at all in large setups
09:17:15 <DuanKebo> Will you please give us some more detailed info.
09:18:16 <gsagie> DuanKebo: the idea in general was to keep a "transaction id" in a special table in Neutron, have the same id in Dragonflow DB and lock our transactions on it, then if these id's differs at some point we do full sync
09:18:47 <gampel> gsagie: so we serialized write transactions into a global lock and do write to the Cluster only from there ?
09:18:49 <gsagie> DuanKebo: this means we basically serialize all our write transactions, but from the feedback i recieved thats not a big problem
09:19:20 <gsagie> gampel: basically yes, most of the problems from the feedback i recieved are for reads, and usualy dont for HA as well (the Galera cluster)
09:19:42 <gsagie> I think that maybe we can keep this as first solution, as its not that hard to implement and improve if we find problems
09:19:58 <DuanKebo> The writer action from df local controllers will also be serialized?
09:20:01 <gsagie> But we can also view Hujie spec once he upload it
09:20:19 <gampel> gsagie: Sound good
09:20:35 <gsagie> DuanKebo: no, but the local controllers only register the compute nodes and the port status
09:20:39 <DuanKebo> Will this introduce performance decrease?
09:20:53 <gsagie> DuanKebo: the publish/subscribe part, gampel will talk in a second
09:21:05 <gsagie> i am only talking about the Neutron Db - Dragonflow DB consistency
09:21:05 <DuanKebo> Ok
09:21:28 <gsagie> I will email both you and Hujie detail information about it
09:21:40 <gsagie> But he can submit his spec and we can review it anyway and decide the best solution
09:21:56 <gsagie> #action gsagie write email with the detailed transaction id solution for DB consistency
09:22:06 <gsagie> #action Hujie upload spec for DB consistency
09:22:25 <gsagie> Lets talk about the publish-subscribe part soon, but thats another different problem we need to sovle
09:22:26 <gampel> gsagie: lets try to get nick-ma feedback as well on this
09:22:35 <DuanKebo> His spec is abut consistency about dragonflow db and local cache
09:22:36 <gsagie> gampel: of course.. :)
09:22:48 <gsagie> DuanKebo: ok good, so different topics :)
09:22:57 <dingboopt> Add me please :)
09:23:08 <dingboopt> email
09:23:12 <gsagie> #action gsagie add dinboopt/nick-ma to the conversation of course
09:23:18 <gsagie> dinboopt: sure, will send it to everyone
09:23:41 <gsagie> anything else on this topic ?
09:24:11 <gsagie> #topic publish-subscribe abstraction
09:24:19 <gsagie> gampel: would you like to update on this?
09:24:40 <gampel> OK I will update the spec today or tomorrow
09:25:00 <gsagie> i mean update here the status :)
09:25:02 <gampel> There is all ready a working version with ZMQ
09:25:03 <gampel> https://review.openstack.org/#/c/263322/
09:25:21 <gsagie> #info working pub-sub with zeromq https://review.openstack.org/#/c/263322/
09:25:29 <gampel> I will add the Pluggability today
09:25:52 <gsagie> gampel: any update on the topics registration? is it happening on server side or client side?
09:26:05 <gampel> the last topic that is not  in the spec yet is the reliability
09:26:20 <gampel> No yet please add it to my action items
09:26:51 <gsagie> #action gampel verify if zeromq filters topics on server side or client side
09:27:15 <gampel> The reliability we want to cover
09:27:16 <gsagie> gampel: you want to update the spec or i should do it?
09:27:29 <gampel> I will do that add it to my action items
09:27:42 <gsagie> #info https://review.openstack.org/#/c/263733/  pub-sub spec
09:27:56 <gsagie> #action gampel update pub-sub spec and add reliability ideas
09:28:15 <gsagie> I think we can iterate on your patch and on Hujie spec for that and find the best solution, if both you and DuanKebo agree
09:28:25 <gsagie> lets review it there and discuss
09:29:13 <gsagie> #topic Redis DB driver
09:29:22 <gsagie> DuanKebo: any update on this?
09:29:39 <gsagie> feipeng is going to work on it right?
09:29:49 <DuanKebo> Feipeng has a draft spec
09:30:24 <DuanKebo> He is update it according our advice.
09:30:26 <gsagie> DuanKebo: for the pub-sub reliability is there anything you would like to ask/discuss now? or we should do it over the review?
09:30:46 <DuanKebo> He will submit it for you review this week too.
09:31:04 <gsagie> #action feipeng upload spec and start work on redis driver integration with dragonflow
09:31:22 <gsagie> ok great, we heard some good things about Redis, so maybe its going to be a good solution for us
09:31:32 <gsagie> not sure about the publish/subscribe part there
09:31:43 <gampel> DuanKebo: I hope today to have an example how to plug pub/sub submitted
09:31:53 <gampel> today
09:32:05 <gsagie> #topic testing
09:32:40 <gsagie> okie, dingboopt any update on the HW in Hangzhou?
09:32:44 <gsagie> about the lab
09:33:15 <dingboopt> i havn't contact huangkun yet
09:33:26 <dingboopt> he is in charge of the servers
09:33:42 <gsagie> dingboopt: ok, please update us for next meeting because i think we all need to know
09:33:43 <dingboopt> I will contact him later
09:33:52 <dingboopt> ok
09:33:56 <gsagie> #action dingboopt update for next meeting regarding lab in Hangzhou
09:34:07 <gsagie> Shlomo_N: has anything to update on your scale testing work?
09:34:54 <Shlomo_N> I have finished the bandwidth performance tests with DF+Etcd on 4 different scenarios
09:35:16 <dingboopt> I just ask hungkun about  the server. They will arrive after chinese spring festival
09:35:33 <gsagie> dingboopt: ok thanks for the update
09:35:44 <gampel> Shlomo_N: can you please share the scenarios
09:35:50 <gsagie> Shlomo_N: can you work on some document describing these
09:35:58 <gsagie> and maybe send the plan for review
09:36:30 <gampel> Shlomo_N: i agree we need a performance  testing spec
09:36:36 <gsagie> I think we will want to perform dataplane performance tests for 1 hosts on a regular basis, maybe even as a CI
09:37:05 <Shlomo_N> Sure. I am still working on the testing spec.
09:37:24 <gsagie> #action Shlomo_N write spec about performance test scenarios and plan
09:37:43 <gsagie> DuanKebo: do you guys are also planning to do some testing in Beijing?
09:37:58 <DuanKebo> Yes
09:38:09 <DuanKebo> He is now evaluating gmdb.
09:38:37 <DuanKebo> Can work on performance test later.
09:38:48 <gsagie> DuanKebo: even regardless of the DB, i am talking about L3 tests with iperf or something like that, can you maybe share your scenarios/plan if you have any document
09:38:50 <gsagie> with Shlomo_N
09:38:59 <gsagie> anyone he can contact for more details?
09:39:55 <gampel> Yuli: did you see my comment on the fullstack flow testing ?
09:39:58 <DuanKebo> No, he only do some control plane test
09:40:20 <DuanKebo> mostly for db r/w and sub/pub performance.
09:40:24 <yuli_s> gampel: yes
09:40:36 <gsagie> yuli_s: anything to share regarding fullstack tests?
09:41:10 <yuli_s> i need to review the comments
09:41:11 <gsagie> DuanKebo: okie, we would also like to have some setup to do these tests here if possible so we would love any document you have on how you conduct these tests
09:41:18 <yuli_s> and send a new version
09:41:22 <gampel> i think that the flow test should follow the application logic so when we update the app it will be very easy to update the test
09:41:25 <yuli_s> of the test patch
09:41:26 <gsagie> Because we also need to be able to work and fix things while testing :)
09:41:39 <yuli_s> https://review.openstack.org/265642/
09:42:09 <gsagie> yes i agree with gampel, doing the diff is a bit problematic
09:42:11 <DuanKebo> OK, Zhu weiqiong will supply the doc about how these test are done.
09:42:16 <gampel> the flow test should follow the application pipeline so the test will be clear
09:42:24 <gsagie> DuanKebo: thanks! that will be great for us
09:42:52 <gampel> yuli_s:  do you understand what i mean ?
09:42:55 <gsagie> #action Zhu weiqiong send documents regarding control plane testing plan and scenarios for DB and for pub/sub
09:43:23 <yuli_s> gampel: yes, the test code is a bit messy
09:43:42 <gampel> yuli_s: let take the use case of l2_app  we will have VLAN app and  Tunnel app
09:44:25 <gampel> we want to test the pipeline for the application L2 L3 DHCP
09:45:26 <yuli_s> the dhcp part is preaty covered. we have test for dhcp enable and disbaled
09:45:34 <gampel> the pipeline will change and we want it to be easy for a developer to update the pipeline in the test
09:45:40 <yuli_s> and we have tests to change dhcp status to enable / disable
09:45:48 <gsagie> gampel, yuli_s: lets discuss it in the #openstack-dragonflow, we are running out of time :)
09:45:57 <gampel> Ok
09:46:04 <yuli_s> and the system works ok
09:46:05 <gsagie> #action gampel, yuli_s work on improving the current OVS flows tests
09:46:06 <yuli_s> great !
09:46:14 <gsagie> #topic selective proactive
09:47:00 <gsagie> DuanKebo: you are listed as working on it with me  and Omer Anson (new employee that joined us)
09:47:13 <gsagie> Would you like to write the spec? or i can do it as well
09:47:16 <yuli_s> gsagie: u talk about L2 or L3 ?
09:47:23 <DuanKebo> I can do it
09:47:40 <gsagie> #action DuanKebo write spec about selective proactive
09:47:51 <gsagie> okie, we can iterate on it
09:48:07 <gsagie> Omer is right now doing some work at simplifying our applications so we have more control
09:48:10 <DuanKebo> based on tenant at the first stage?
09:48:21 <gsagie> DuanKebo: i think so, what the rest think?
09:48:25 <yuli_s> gsagie: selective proactive L2 or L3 ?
09:48:33 <gampel> yes i agree
09:48:49 <gsagie> yuli_s: its not related to L2 or L3, its about how we sync the DB with the local controller
09:49:04 <DuanKebo> I agree. But i also think it's not enough
09:49:06 <yuli_s> ok
09:49:06 <gsagie> You will see in the spec that DuanKebo will upload
09:49:26 <DuanKebo> let's us just start from tenant grained selective proactive.
09:49:49 <gsagie> DuanKebo: ok, if you want you can also upload to the repository other ideas you had and we can start iterating on them in parallel
09:50:04 <gampel> DuanKebo:  very reasonable why to  start
09:50:06 <gsagie> keep in mind we also need to cover cases of deletion
09:50:30 <DuanKebo> Yes.
09:50:43 <gsagie> okie, so we can continue iterating on the spec
09:50:46 <gampel> we have resources that are cross tenant
09:51:01 <gsagie> btw DuanKebo, your team is familiar with how to upload and work with OpenStack Gerrit and the review board?
09:51:15 <gampel> like shared networks define by admin we need to make sure we cover them as well
09:51:17 <DuanKebo> gampel, I can discuss with you after the meeting.
09:51:28 <gampel> Ok
09:51:45 <gsagie> #topic distributed DNAT
09:52:12 <gsagie> sorry for moving fast but we have 2 more topics and no time, lets continue to iterate on the spec that DuanKebo will upload
09:52:24 <gsagie> DuanKebo: i think its big task so i will also work on this with you
09:52:41 <gsagie> we need to also work on the DB parts for this and on the pub/sub parts for this
09:52:43 <DuanKebo> Hi Gal, I will deal with the uploading and review processing
09:53:05 <gsagie> Raofei: you are listed as working on the distributed DNAT, is the spec clear?
09:53:12 <gsagie> DuanKebo: great :)
09:53:27 <DuanKebo> Great Gal, definitely I need your help!
09:53:50 <gsagie> Raofei: i will update it with the latests changes, but just need to make sure everything is understandable
09:53:59 <gsagie> there is one thing there that needs special care
09:54:09 <raofei> Yes, I have read your spec of DNAT.
09:54:09 <gampel> Just Please try to review all the dragonflow status open
09:54:19 <gsagie> and thats updating the GW MAC address in the flows
09:54:46 <gsagie> raofei: ok, please send any questions you might have, you have time to work on it?
09:55:02 <gsagie> its an important task for the overall picture
09:55:55 <gsagie> #action raofei working on distributed dnat implementation
09:55:57 <raofei> yes, I think so. we need change the both dst ip and mac address to simulate gw sending.
09:56:28 <gsagie> raofei: yes, the problem is that the GW MAC of the external network can change and we might need to update flows
09:56:34 <gsagie> #topic controller reliability
09:56:48 <gsagie> heshan is working on this right?
09:56:52 <DuanKebo> Yes
09:57:04 <gsagie> #action heshan upload spec regarding controller reliability
09:57:04 <DuanKebo> Most part of the spec has been completed.
09:57:18 <DuanKebo> Soon you can review it
09:57:26 <gsagie> okie great work! thanks
09:57:34 <gsagie> #topic open discussion / bugs
09:57:37 <gampel> we need to get more reviews from all the group Please try to review all the dragonflow status open even if you are not listed
09:57:54 <gsagie> #action everyone to review Dragonflow patches
09:58:03 <DuanKebo> But our guys are not good at english, we net to refine the expression
09:58:21 <gsagie> DuanKebo: its important everyone review the repository as we need everyone to be updated with all the code changes
09:58:32 <gsagie> so they keep track of things happening
09:58:54 <gsagie> But its understandable, dont worry about the english, we all understand
09:59:12 <gampel> It is very impotent  to get the feedback don't worry about the english
09:59:23 <DuanKebo> OK
09:59:27 <gsagie> thanks for joining everyone!
09:59:37 <gsagie> if anyone has anything to add, please do
09:59:41 <gsagie> and Welcome to Omer :)
09:59:43 <DuanKebo> bye!
09:59:46 <yuli_s> we havent covered bugs
09:59:50 <gampel> welcome
09:59:58 <Shlomo_N> welcome Omer
09:59:59 <oanson> Thanks!
09:59:59 <gsagie> yuli_s: this is a joint topic, any update?
10:00:10 <gsagie> just dont have time :(
10:00:17 <yuli_s> i closed some duplicate bugs today
10:00:34 <gsagie> yuli_s: cool, good work, anything important is open ?
10:00:40 <yuli_s> yes
10:00:43 <gsagie> we can put it as an action item
10:00:45 <yuli_s> cloud-init
10:00:54 <yuli_s> i want to work on it
10:00:56 <gsagie> yuli_s: good point
10:01:09 <gsagie> #action yuli_s working on cloud-init support for dragonflow
10:01:14 <gsagie> yuli_s: no problem
10:01:19 <yuli_s> ok, great
10:01:23 <gsagie> There is a bug opened right?
10:01:33 <yuli_s> yes, I opened it
10:01:54 <gsagie> okie, we are out of time, lets continue this in #openstack-dragonflow, thanks and good night for all our friends from China and talk to you later
10:01:57 <gsagie> #endmeeting