09:00:42 <gsagie> #startmeeting dragonflow 09:00:43 <openstack> Meeting started Mon Feb 1 09:00:42 2016 UTC and is due to finish in 60 minutes. The chair is gsagie. Information about MeetBot at http://wiki.debian.org/MeetBot. 09:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 09:00:46 <openstack> The meeting name has been set to 'dragonflow' 09:00:55 <gsagie> Hello everyone, who is here for the dragonflow meeting? 09:01:02 <dingboopt> o/ 09:01:02 <gampel> Hi gal 09:01:04 <diga_> o/ 09:01:05 <steve_ruan> hi gal 09:01:05 <oanson> Hi. I am. 09:01:08 <nick-ma_> Hi gal 09:01:20 <raofei> i'm here 09:01:34 <Shlomo_N> Hi 09:01:57 <gsagie> #info dingboopt, gampel, diga, steve_ruan_oanson, nick-ma, raofei, Shlomo_N in meeting 09:02:31 <gsagie> #info dingboopt, gampel, diga, steve_ruan, oanson, nick-ma, raofei, Shlomo_N in meeting 09:03:03 <gsagie> gampel: DuanKebo is coming? maybe contact him 09:03:43 <gampel> Ok i will 09:03:47 <raofei> His espace is offline 09:04:08 <gsagie> okie, hopefully he will join soon 09:04:10 <gsagie> hi yuli_s 09:04:15 <gsagie> #topic security groups 09:04:20 <yuli_s> Hello ! 09:04:30 <gsagie> Raofei: when do you think we will be ready to start implementing this? 09:04:34 <yuli_s> da jia hao 09:04:41 <gsagie> after all the constraints are removed 09:04:49 <gsagie> from the controller side 09:05:01 <gsagie> dingboopt: here? 09:05:05 <dingboopt> yes 09:05:23 <gsagie> would you like to update what you have so far, i saw you made some very good progress 09:05:27 <raofei> We are startting implenttng controller app in local setup. 09:05:30 <gsagie> and even fixed some bugs on the way for router :) 09:05:52 <dingboopt> I think the next step is to implement the SG app 09:06:30 <gsagie> okie, so we make sure to merge and test any patches that are open and hope we can see the sg app in the upstream code soon 09:06:33 <gsagie> so we can all review it 09:06:46 <dingboopt> yes 09:06:52 <gsagie> #topic port security 09:06:59 <raofei> yes, we think so. 09:07:04 <gsagie> ok great 09:07:26 <gsagie> dingboopt, raofei: any update on this? i think we solved most problems in the spec 09:07:42 <gsagie> so everything should be clear to start implementing this, any open issues? 09:08:51 <gsagie> #action gsagie get port security review merged 09:08:53 <gsagie> #info https://review.openstack.org/#/c/263019/ 09:08:55 <dingboopt> yes 09:09:07 <gsagie> ok, so no open issues 09:09:13 <raofei> yes. port security will be started after security group. 09:09:17 <gsagie> okie good 09:09:33 <gsagie> #topic DB consistency 09:09:49 <gsagie> nick-ma: want to update on this? 09:10:13 <gsagie> I think gampel would like us to have some discussion on this, mostly the spec you sent vs the journal implementation 09:10:14 <nick-ma_> i proposed the spec and also researched ODL journal. I think we do the same thing. the only difference is the implementation of distributed lock. 09:10:39 <gsagie> #info https://review.openstack.org/#/c/268005/ DB consistency spec 09:11:03 <gampel> In ODL they use the SQL DB as the lock ? 09:11:05 <nick-ma_> in ODL, distributed lock is implemented by a local update lock (select for update). in our spec, we try it in different way. 09:11:09 <nick-ma_> yes 09:11:44 <gampel> nick-ma_: Ok I will review it today 09:11:46 <gsagie> so if i understand right, the only difference is that in your spec we do full sync in case of a problem and in the journal its more ordered 09:12:21 <nick-ma_> yes. we have a full-sync mechanism. 09:12:28 <gsagie> of course we need to align some code for the journal to work, but its doable 09:13:06 <nick-ma_> we can reuse the journal db implementation as the distributed lock and incorporate it with our sync mechanism. 09:13:08 <gsagie> nick-ma: so you believe the approach you wrote in the spec is better then the journal? 09:13:40 <diga_> hey 09:13:46 <gsagie> hi diga! 09:14:01 <nick-ma_> we can stick to our spec. 09:14:12 <diga_> I have started on writting test for API, will send 1st patch today EOD 09:14:34 <gsagie> diga: ok, we arent in testing yet, but we will get there soon :) 09:14:41 <diga_> okay 09:15:00 <diga_> do you want me to start on other stuff ? 09:15:03 <gsagie> nick-ma: to be honest i am unclear what is the better approach 09:15:38 <nick-ma_> the journal is working there. 09:15:42 <gampel> I think that in the phase we should keep it simple 09:17:14 <nick-ma_> the problem is that we need to fix all the inconsistency problems at the mean time. it is not that simple. 09:17:27 <gampel> so maybe in the first step do the journal ? I am not sure i fully understand the differences lets take it to our channel latter 09:17:52 <gsagie> nick-ma: how much time you think each solution should take 09:18:13 <gsagie> if you have time constraints maybe i can take some or help 09:18:51 <nick-ma_> you are working on the db sync, right? 09:18:53 <nick-ma_> gal 09:19:04 <gsagie> nick-ma: either me or oanson will finish it soon 09:19:31 <nick-ma_> i'll update you guys with my thoughts in email or review. we can start other topics then. :-) 09:19:48 <gsagie> okie 09:20:12 <gsagie> so lets put an action item just to pick one of the two options and start working on it 09:20:48 <gsagie> #action gsagie, nick-ma, gampel decide which is the better db consistency solution and start working on it 09:20:55 <nick-ma_> ok 09:21:01 <gsagie> #topic publish-subscribe 09:21:07 <gsagie> gampel: would like to update? 09:21:23 <gampel> yes 09:21:36 <gampel> I just updated the spec 09:21:41 <gampel> #info https://review.openstack.org/#/c/263322/ 09:21:53 <gsagie> #info https://review.openstack.org/#/c/263322/ pub-sub spec 09:22:14 <gampel> it include reliability and plugability 09:22:17 <gsagie> So this spec talks about our pub-sub abstraction and i believe also the reliability mechanism between the pub-sub module and the local controllers 09:22:50 <gampel> yes it is missing the TOPIC support for the selective distribution 09:23:06 <gsagie> okie, so everyone please review it, we are going to merge it soon probably so if anyone notice any problem 09:23:18 <gampel> code is submitted without the reliability 09:23:19 <gsagie> i think we going to merge it with default True, so its working by default 09:23:21 <raofei> OK 09:23:33 <nick-ma_> ok 09:23:35 <gampel> #info https://review.openstack.org/#/c/263733/ 09:23:37 <gsagie> so if you notice anything strange that might be relate to it, open a bug please on gampel ;) 09:23:48 <dingboopt> ok 09:23:58 <gsagie> j/k, you can assign it to me too :) 09:24:03 <gsagie> ok 09:24:24 <gsagie> gampel: are there any open issues there? 09:24:24 <gampel> please review the patches 09:24:47 <gampel> as we discussed today the pubsub use client side filtering 09:24:48 <gsagie> We need to update Duan also to point the right people into this patch 09:25:05 <gampel> currently in zmq 09:25:39 <gampel> so when used with the selective proactive messages will still reach the clients 09:25:40 <gsagie> we need to make sure to merge this with https://review.openstack.org/#/c/274336/ 09:26:37 <gsagie> btw nick-ma, saw the migration patch, looks good to me, i added some reviewers from Neutron which knows these areas good and after their review we can merge it, but thanks for working on this 09:27:02 <gsagie> gampel: need to make sure to document this in the spec 09:27:08 <nick-ma_> ok. 09:27:12 <gampel> yes it is there 09:27:17 <gsagie> ok great 09:27:24 <gampel> in the pub-sub spec 09:27:33 <gsagie> anything else on this topic? 09:28:04 <gampel> No i hope to add the TOPIC registration support to the spec this week and finish the reliability part 09:28:04 <gsagie> okie, lets move on 09:28:10 <raofei> about the DNAT, although it's merged. 09:28:26 <gsagie> okie sounds good, lets just put action to manage it 09:28:29 <raofei> but for external network gateway MAC, your solution need statistic configuration 09:28:41 <gsagie> #action gampel finish pub-sub spec in regards to TOPIC registeration and reliability 09:28:53 <gsagie> raofei: lets talk about this now 09:28:55 <gsagie> #topic DNAT 09:29:06 <raofei> I have a suggestion/solution to get/update external gw mac dynamically 09:29:26 <gsagie> raofei: okie great, we had some ourselfs as well, can you send us in email and we can iterate on this? 09:29:28 <raofei> I added some commit to the merged patch last week 09:29:33 <raofei> OK 09:29:40 <gsagie> ohhh, didnt see it, i will take a look 09:29:52 <gsagie> #action gsagie look at distributed dnat spec comments 09:30:05 <raofei> So, for DNAT, do you think to raise a more detailed spec or just coding? 09:30:07 <gsagie> btw raofei, you can propose new patch with your ideas on top of the merged spec 09:30:32 <gampel> i didn't see it either sorry , please ping us next time 09:30:37 <raofei> OK 09:30:40 <gsagie> raofei: i think we can start coding, there is alot of things that are unrelated to this problem and we can start working on them, like 09:30:51 <gsagie> starting to bring all the floating ip data from the plugin to DF DB 09:30:59 <gsagie> and reading it in the controller 09:31:06 <gsagie> and then writing the application 09:31:31 <raofei> yes, I see 09:31:57 <gsagie> But its good that we also solve the case you mention, we do need to support dynamic change of MACs in the egress flows in case the MAC for external gateway changes, which might happen in active/standby cases 09:32:18 <gsagie> But we can continue discussing in parallel i know gampel had some idea to use learning flows 09:32:31 <gsagie> so we can discuss it 09:32:39 <gsagie> raofei: so you also start working on dnat? 09:32:41 <raofei> OK, agree 09:33:10 <raofei> yes. security group will be done mainly by dingbo 09:33:22 <gsagie> #action raofei start implementing distributed dnat 09:33:42 <gsagie> #action gsagie, gampel review raofei ideas and together solve the external MAC changes problem in the design 09:33:46 <gsagie> okie great 09:33:48 <gampel> Ok 09:33:56 <raofei> I hope we can complete some feature indenpendent in coding 09:34:36 <gsagie> raofei: yeah, hoping to see the DNAT patches soon, if you need help at any stage please let us know 09:34:36 <gampel> raofei: I am not sure I understand ? 09:35:06 <raofei> gampel, what's your doubt? 09:35:12 <raofei> sure. 09:35:28 <gsagie> #topic redis db 09:35:40 <gsagie> DuanKebo is not here, but there is a spec patch please help review 09:35:50 <gsagie> #info https://review.openstack.org/#/c/274340/ 09:36:03 <gsagie> #topic selective proactive 09:36:34 <gsagie> For this we also need DuanKebo, but lets put an action item to speak about it next meeting, i will start doing some work for this 09:36:41 <gsagie> and we can all iterate on the spec 09:36:57 <gsagie> #info selective proactive spec https://review.openstack.org/#/c/273877/ 09:37:22 <gsagie> We need to make sure to add "topics" registeration both for the DB and for the pub-sub APIs 09:37:39 <gampel> yes i will add it in another patch 09:37:43 <gsagie> but basically, its going to be the DB drivers responsbility to implement this 09:37:57 <gsagie> #action gsagie start working on OVSDB monitor thread 09:38:10 <gsagie> #action gsagie start working on adding tenant information from plugin 09:38:29 <gsagie> anyone has any question regarding this? i prefer we talk about it next week when Duan is here unless anything important 09:39:23 <gsagie> okie, if anyone has any question feel free to raise it in the channel 09:39:25 <gsagie> #topic testing 09:39:34 <gsagie> oanson: might sharing what are you working on ? 09:39:55 <steve_ruan> is "service injection " started? 09:39:59 <oanson> I'm looking into testing the ovs flows without setting up VMs 09:40:20 <oanson> The idea is to create a fast lightweight testing I/S for DF applications. 09:40:31 <gampel> steve_ruan: this is not targeted to mitaka 09:40:35 <gsagie> steve_ruan: hi, this is currently not in our priorities for this release 09:40:53 <oanson> Using e.g. ovs-appctl ofproto/trace, we can trace virtual (and generate real) packets through the flow. 09:41:01 <steve_ruan> @gampel @gsagie, thanks 09:41:06 <gsagie> steve_ruan: but certainly its the next topic we want to tackle, and i think provide alot of extra value, so we might start this in parallel 09:41:27 <gsagie> steve_ruan: if you guys are actually planning to use it, it might be an important feedback for us and we can give it more priority 09:41:42 <gsagie> oanson: cool 09:42:00 <steve_ruan> thanks 09:42:03 <gsagie> so everyone can build a simulated tests based on oanson framework to test applications 09:42:15 <oanson> That's the plan. 09:42:22 <gampel> oanson: I think this will improve our CI testing and allow application level testing Ping/DHCP etc 09:42:23 <gsagie> which can be good for the security groups and the dnat apps 09:42:31 <oanson> I will create a (hopefully) simple API for others to rely upon. 09:42:41 <gsagie> yuli_s: want to share your update regarding OVS flows tests and fullstack tests? 09:42:53 <Shlomo_N> Sound great. Good job oanson. 09:43:15 <yuli_s> gsagie: sure, 09:43:16 <gsagie> I think in general we want everyone that adds a new feature/code from now on to also add tests unit/fullstack/functional 09:43:37 <yuli_s> I commited the DHCP tests 09:44:03 <gsagie> #info DHCP tests done by yuli_s https://review.openstack.org/#/c/273548/ 09:44:03 <yuli_s> fixing a number of issues found during review 09:44:23 <yuli_s> so, hope it will go in master code today 09:44:42 <gsagie> yuli_s: ok good, and next you addressing the destructor problem right? 09:44:52 <yuli_s> I also commited class I used to create VMs 09:45:18 <yuli_s> so, I will be able to commit more tests soon 09:45:21 <gsagie> we also have these APIs tests that we want to do scale/performance testing for many API calls on the Neutron server 09:45:26 <gampel> #info https://review.openstack.org/#/c/273595/ 09:45:59 <gsagie> #action yuli_s fixing the current fullstack tests destructor problem 09:46:18 <gsagie> #action yuli_s perform API performance testing plan 09:46:38 <gsagie> ok good job yuli_s, we do need you to start in parallel on the API/DB scale thinking as well 09:46:51 <yuli_s> ok 09:47:05 <gsagie> because we are testing different solutions and its important we have something that can verify we dont hurt performance at each patch 09:47:07 <gsagie> okie 09:47:20 <gsagie> Shlomo_N: would like to update regarding scale tests for data plane? 09:47:32 <Shlomo_N> sure 09:47:45 <gsagie> diga: you going to submit some unit tests soon? 09:47:56 <gsagie> think you mentioned that in the start of the meeting :) 09:48:26 <Shlomo_N> I am currently trying to setup a testing env. with DVR with Liberty. Have few problems in setup. 09:49:24 <Shlomo_N> I have uploaded a performance testing spec https://review.openstack.org/#/c/272559/. 09:49:26 <gampel> Shlomo_N: you have initial result for DF 09:49:53 <Shlomo_N> Yes, already shared with you the results. 09:49:53 <gsagie> Shlomo_N: okie, lets take this offline then, i saw DuanKebo sent you the details of the person in Beijing that does 09:50:02 <gsagie> scale tests 09:50:09 <Shlomo_N> ok 09:50:46 <gsagie> okie, anyone else would like to discuss about the tests? 09:50:48 <gampel> All please try not to avoid trailing white spaces in the spec and avoid more then 80 char line length 09:51:01 <gsagie> #topic open discussion 09:51:13 <gsagie> steve_ruan: i would like to talk with you about the use case 09:51:57 <gsagie> steve_ruan: the question is how seriously you guys need it :) because we also have some internal use cases that needs this and 09:53:48 <gampel> We need to stabilize our CI it fails some times randomly (the none voting) 09:54:19 <gsagie> okie 09:54:26 <gsagie> We need to think when its time to make it voting 09:54:30 <gsagie> and which one :) 09:54:52 <gsagie> anyway steve_ruan: we can discuss this after this meeting and see what we can do to help with this use case 09:55:05 <gsagie> #topic bugs 09:55:11 <gsagie> anything special here? 09:55:19 <oanson> About the non-voting CIs, I think we should wait for yuli_s's destructor fix. I see some strange race issues there. 09:55:35 <gampel> Ok 09:56:06 <gsagie> oanson: yes i agree, but not all tests are related to these, we can think about making tempest voting 09:56:40 <gsagie> but lets postpone this call to the next meeting, as you said its not stable enough 09:57:02 <gsagie> #action oanson, yuli_s help make the fullstack tests stable 09:57:20 <gsagie> Thanks everyone for joining :) 09:57:26 <Shlomo_N> Thank you 09:57:31 <nick-ma_> thank you. 09:57:33 <gampel> Thank you 09:57:37 <oanson> Thank you. 09:58:07 <gsagie> #endmeeting