15:04:06 <pc_m> #startmeeting vpnaas
15:04:07 <openstack> Meeting started Tue Feb 17 15:04:06 2015 UTC and is due to finish in 60 minutes.  The chair is pc_m. Information about MeetBot at http://wiki.debian.org/MeetBot.
15:04:08 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
15:04:11 <openstack> The meeting name has been set to 'vpnaas'
15:04:31 <pc_m> #topic Announcements
15:05:24 <pc_m> Functional gate is non-voting. Will defer making it voting, as some changes may be needed (will disuss)
15:06:04 <pc_m> StrongSwan functional/scenario tests will need a separate job due to setup needs are different.
15:06:42 <pc_m> If you have functional tests, please put them out for review and check functional test results from Jenkins.
15:07:07 <pc_m> Please help out on reviews
15:07:17 <pc_m> Anyone have any other new announcements?
15:07:52 <pc_m> #topic Functional Testing
15:08:38 <pc_m> As mentioned, check dsvm-neutron-vpnaas-functional test results on your reviews
15:09:22 <pc_m> For StrongSwan, we're seeing that we need to have it installed (for Ubuntu 14.04) and AppArmor disabled.
15:09:37 <pc_m> As a result a new job will be needed to test this.
15:10:03 <pc_m> Must figure out the best way to organize the directory structure to support OpenSwan vs StrongSwan testing.
15:10:36 <pc_m> I posted an email on the ML this morning asking for feedback from QA team on this an many other items.
15:11:18 <pc_m> My thoughts are that maybe it may be best to have a sub-dir called strongswan and use filters to only run tests in that area, when the new job runs.
15:11:51 <pc_m> We do need to develop functional tests for both OpenSwan and StrongSwan.
15:12:25 <pc_m> If you have time to help out, please chime in.
15:13:08 <pc_m> You can provide review comments on Zhang's functional tests for StrongSwan under https://review.openstack.org/#/c/144391/
15:13:27 <pc_m> Any questions/comments on Functional testing?
15:13:27 <nfedotov1> Sorry for interupting you. Are there any difference between OpenSwan and StrongSwan from API point of view?
15:13:48 <pc_m> nfedotov1: Sure, np at all... no the API is the same.
15:14:10 <nfedotov1> In my understanding OpenSwan and StrongSwan is a two diferrent implementations of a singel API.
15:14:33 <pc_m> StrongSwan has more capabilities, and I think, for some things, like encryption protocols, the client API accepts strings, so can handle both.
15:14:49 <pc_m> nfedotov1: From a OpenStack standpoint yes.
15:15:45 <pc_m> Any other Qs on the functional tests?
15:16:09 <nfedotov1> And I expect functional tests have (or maybe should have) same  Design Principles as tempest. I mean "Tests in Tempest should only touch public interfaces, API calls"
15:18:51 <pc_m> There's a lot of confusion (on my part at least) between the tempest tests (which I understand tempest is a tool and repo), and functional tests (which we're placing in our repo, and which use devstack, but not the tempest libraries).
15:19:37 <pc_m> Please see the email I sent, on openstack-dev mailing list and chime in on that topic. There I expressed what I think are our goals, namely...
15:20:10 <nfedotov1> Ahh. OK. Thanks for reply. I will take a look
15:20:22 <pc_m> To have functional test coverage of the device driver implementations (we have limited coverage), and to have a end to end scenario test to check operation.
15:20:42 <nfedotov1> Yes there are a lot confusions around functional tests
15:21:20 <pc_m> nfedotov1: There is a bunch of issues around the scenario tests (also discussed in that email). Let's talk about that for a bit...
15:21:32 <pc_m> #topic Scenario Testing
15:21:45 <pc_m> So here are the issues here...
15:22:09 <pc_m> 1) We'll need different jobs, as StrongSwan needs a different setup that OpenSwan.
15:23:06 <pc_m> 2) Right now we have two (slightly different) scenario tests created and out for review, under Tempest repo. We need to decide which to go forward with.
15:23:25 <nfedotov1> Do you mean jobs in check gate?
15:24:06 <pc_m> 3) They are not allowing *aaS tests to be added to Tempest, because that is migrating to Neutron in tree and a tempest lib is being setup.
15:25:03 <pc_m> nfedotov1: We'll need to have two separate jobs that run in check/gate pipelines, one that sets up for OpenSwan, and a separate job for StrongSwan.
15:25:47 <pc_m> Can't run them both under same job, because they both cannot be installed in a system at the same time, and because StrongSwan needs to have apparmor disabled.
15:27:11 <pc_m> So #1 is sort of mechanical. I guess a pre_test_hook will be needed to setup the environment right.
15:27:49 <pc_m> For #2, we, as a team, need to decide on which of the two implementations should be used. We have...
15:28:00 <pc_m> #link https://review.openstack.org/#/c/140072/8
15:28:14 <pc_m> #link https://review.openstack.org/#/c/153292/5
15:28:54 <pc_m> Both authors are fine with either proceeding. Zhang is on vacation for 10 days.
15:28:55 <nfedotov1> Yes there should be some script that configures test environment. Is it possible to use Devstack for it?
15:29:30 <pc_m> nfedotov1: My undertstanding (from talking to people) is that the pre_test_hook should set that up.
15:29:53 <pc_m> However, issue #3 is the big one.
15:30:15 <matrohon> I don't understand what is the fundamental difference between those two scenario tests?
15:30:32 <matrohon> they cant be merged?
15:30:55 <pc_m> matrohon: Not much, two independent implementations of the same test. One does pings and one does ssh to check connection.
15:31:05 <pc_m> The one with pings checks before and after.
15:31:42 <pc_m> Nikolay is willing to proceed forward with his version, and Zhang was cool with that as well.
15:31:54 <pc_m> Is that OK with folks?
15:32:11 <matrohon> zhang's patch also seems to ping
15:32:29 <matrohon> ok, the most important here is to have a first test available
15:32:41 <pc_m> matrohon: I think Zhang's pings, and Nikolay's ssh'es
15:33:00 <matrohon> pc_m : ha ok!
15:33:06 <pc_m> If I scanned them right. Not sure that much about the tests.
15:33:18 <nfedotov1> There is a basic use case for vpnaas. Here: https://wiki.openstack.org/wiki/Neutron/VPNaaS/HowToInstall , section "IPSec Site-to-site Connection Creation"
15:33:52 <nfedotov1> I think both patchsets do this scenario. At least my patchset follows steps described there.
15:33:55 <pc_m> With Zhang's it is obvious what the subnet's IPs are in the test.
15:34:42 <pc_m> nfedotov1: Does yours use a single node and two routers/nets/subnets (hard to tell what the tempest calls all do)
15:34:59 <pc_m> ?
15:35:47 <nfedotov1> It uses two instaces. Each one is connected to dedicated private network + dedicated router
15:36:26 <pc_m> Do we know if Zhang's also does that? I thought it used one devstack instance...
15:36:33 <nfedotov1> so there are two sides: instance -> private network (subnet) -> router (router IP)
15:37:43 <pc_m> nfedotov1: Just to clarify, by instance, do you mean that there are two devstack nodes started up, or is this one devstack node, with two routers and two private nets?
15:38:13 <pc_m> (and two VM instances on that single devstack)
15:38:34 <nfedotov1> Yes. There is one devstack node (all-in-one), one tenant, one user
15:39:24 <pc_m> nfedotov1: So two routers, subnets, on one node. So, I think the two implementations are pretty close.
15:39:39 <nfedotov1> and two instances
15:39:47 <nfedotov1> It looks like Zhang's patchset does same thing.
15:40:45 <pc_m> Does anyone have any strong preference for one over the other? If not, then we should focus on reviewing nfedotov1, because he's available to proceed on this now.
15:40:51 <pc_m> Thoughts?
15:41:57 <pc_m> #action team to review https://review.openstack.org/#/c/140072/8 for Scenario test
15:41:58 <nfedotov1> well I can pick some best things from Zhang's patchset. And later if Zhang wants to change something I will not mind
15:42:26 <pc_m> nfedotov1: Great, will be good for you two to collaborate, when Zhang is back.
15:42:31 <nfedotov1> Ok
15:42:46 <pc_m> The big issue is #3.
15:43:18 <pc_m> I've asked in the email if we can get an exception and add this scenario test into tempest, before the migration.
15:44:06 <nfedotov1> But eventually they will be removed. Yes?
15:44:17 <pc_m> Otherwise, we have to either wait for migration and availability of tempest lib, or try to adapt this to run without tempest, which doesn't really sound good to me (not leveraging over the avail infra).
15:44:54 <pc_m> nfedotov1: Well, they will be migrated to run under Neutron (in-tree), and then the VPN ones will be moved to the VPN repo.
15:45:12 <pc_m> Concern here is over whether we can do all that for K-3.
15:45:49 <pc_m> Does anyone have any other ideas on this? I don't know if there is any better way to do the scenario test.
15:46:41 <nfedotov1> Maybe it would be better to place tests to neutron-vpnaas. In my opinion
15:46:42 <pc_m> #action nfedotov1 will collaborate with zhang and merge in useful bits from other commit.
15:47:07 <pc_m> nfedotov1: To place them directly in the VPN repo now?
15:48:00 <nfedotov1> Maybe I do not know some obstacles that prevents us from doing this.
15:48:05 <pc_m> To do that, we need the tempest lib or modify the test to run w/o tempest libs. Not sure when that will be available (I asked in email).
15:48:56 <pc_m> #action Get decision from QA team on whether scenario test can be added to tempest or when tempest lib will be available
15:49:34 <pc_m> Any more discussion on the scenario/functional tests? We've got 10 mins left for other items.
15:50:30 <pc_m> #topic Bugs
15:51:09 <pc_m> Please take a look at the bugs out and provide review comments. There are about 10 out.
15:52:11 <pc_m> Oh, I forgot to mention. After setting up coverage testing, I've put out another commit for review to enable coverage testing of any functional tests.
15:53:10 <pc_m> Here's the bug list: https://review.openstack.org/#/c/140072/8 If you have any of interest t discuss here, please speak up.
15:53:52 <matrohon> pc_m : ^^wrong link?
15:54:12 <pc_m> Whoops...  https://review.openstack.org/#/q/status:open+project:openstack/neutron-vpnaas,n,z
15:54:59 <pc_m> #topic Open Discussion
15:55:22 <pc_m> Anyone have any other questions/concerns to discuss?
15:56:31 <pc_m> OK. that's a wrap then. Thanks for joining in, and please take a look at the open reviews so people can progress on their commits.
15:56:56 <pc_m> #endmeeting