17:25:00 <dwalleck> #startmeeting
17:25:01 <openstack> Meeting started Thu Jun 14 17:25:00 2012 UTC.  The chair is dwalleck. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:25:02 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:25:17 <dwalleck> #topic Status of Swift tests
17:25:34 <dwalleck> JoseSwiftQE: ?
17:25:40 <JoseSwiftQE> Just uploaded the pep8 fixes.
17:25:49 <JoseSwiftQE> that was silly of me to miss that, but it's all fixed now.
17:25:50 <dwalleck> I haven't had a change re-review anything you've put in this week
17:25:54 <jaypipes> #link http://wiki.openstack.org/Meetings/QATeamMeeting
17:26:39 <JoseSwiftQE> Once they're in, I plan on making smaller more frequent commits to get more features and tests in.
17:26:39 <dwalleck> Okay, good deal. I'll try to take a look today. I think we're getting close...
17:26:40 <jaypipes> JoseSwiftQE: k, I will test it locally. thx!
17:26:47 <jaypipes> ++
17:27:00 <dwalleck> sweet!
17:27:23 <jaypipes> looks like ravi forgot to update the agenda link... the one I posted above is from a while ago.
17:27:28 <dwalleck> #topic Status of parallelization
17:27:39 <jaypipes> oooh, me plese
17:27:40 <dwalleck> I've got his email that I'm reading from
17:27:41 <jaypipes> o/
17:27:42 <dwalleck> hehe
17:28:02 <dwalleck> go, because I know I've been swamped and haven't been able to jump in =P
17:28:03 <jaypipes> can I describe the strategy I discussed with you and davidkranz yesterday?
17:28:10 <dwalleck> yes!
17:28:10 <jaypipes> ok, cool.
17:28:12 <JoseSwiftQE> jaypipes:  side question for you.  Somehow the author got set to David Kranz on my commit, and I can't fix it.  I'm a bit of a git newb, but even git commit --amend --autho=me isn't working :\
17:28:33 <jaypipes> JoseSwiftQE: that's likely a bug in gerrit.. just iignore.
17:28:38 <jaypipes> for now. being worked on.
17:28:45 <JoseSwiftQE> coolbeans
17:29:24 <jaypipes> OK, so the strategy I described for parallelizing the tests (easily) was to make some adaptations to the base test case classes that would create a user/tenant for each test case object
17:29:43 <jaypipes> this would get us past the big issue with currently rtunning with --processes=N, which is running out of quotas...
17:30:10 <dwalleck> yup yup
17:30:11 <jaypipes> I am going to be working on this today and tomorrow. Should have a patch up that shows the code so we can discuss
17:30:24 <dwalleck> and isolates testing well also
17:30:29 <jaypipes> yep.
17:30:48 <jaypipes> do we have any HPers on the channel? I'd like to discuss the negative/fuzz testing.
17:31:10 <jaypipes> nijaba: around?
17:31:35 <davidkranz> Oh. I see we reconvened.
17:31:37 <jaypipes> I don't see any other HPers...
17:31:38 <pcrews> jaypipes: I'm lurking, but haven't dug too deep into randgen applications for that stuff yet (next week)
17:31:39 <nijaba> jaypipes: o/
17:32:03 <jaypipes> nijaba: any other HPers? we'd like to discuss strategy with the negative tests and also how bugs are being used for tracking these tests
17:32:14 <jaypipes> pcrews: k, no worries.
17:32:46 <jaypipes> nijaba: if you want, you can say "I'd rather see an email about this to the ML first, discuss on ML, and I will have HPers at the next meeting to disucss" :)
17:32:56 <nijaba> jaypipes: HPers?  not sure I follow you?
17:33:12 <jaypipes> nijaba: ah, sorry, I thought you worked at HP in Ravi's group!@
17:33:16 <jaypipes> sorry about that!
17:33:32 <nijaba> jaypipes: nope, still working fro canonical ;)
17:33:32 <jaypipes> doh, you're nic barcet.
17:33:51 <nijaba> no worries ;)
17:33:57 <jaypipes> OK, so it looks like there aren't any other HPers...
17:34:26 <jaypipes> davidkranz, dwalleck: I will write up an email with discussion points about changing the tracking of these things in our bug list and in slowing down pace of these negative test
17:34:37 <dwalleck> sounds good
17:34:38 <davidkranz> jaypipes: Thanks.
17:34:41 <jaypipes> np
17:34:49 <jaypipes> what;'s the next topic?
17:35:21 <dwalleck> ...I think this is all copied from last week
17:35:32 <davidkranz> How about : Expectations of Tempest execution time
17:35:50 <dwalleck> We can do that
17:36:05 <dwalleck> #topic Tempest execution time
17:36:09 <dwalleck> boom
17:36:45 <jaypipes> I believe *smoke* tests should take no longer than 10 minutes to complete.
17:36:47 <davidkranz> I think there are three cases: 1) gating job 2) Good tempest run 3) complete regression test
17:37:02 <jaypipes> that would be 600 seconds, or approximately 40% of the current runtime.
17:37:13 <davidkranz> jaypipes: Does smoketest == gating job?
17:37:38 <dwalleck> I could see even 15, but 10 would very nice
17:37:41 <jaypipes> davidkranz: it should be the *first* gating job, yes. if the core projects decide they would like the complete regresion test to gate after some time, cool with me.
17:38:24 <jaypipes> davidkranz: the idea being we should have a hierarchical, waterfall-type gate job, with the smoke tests being the first gate and finer-grained tests run only after successful completion of smoke tests
17:38:32 <sam_rackspace> +1 for David's opinion
17:38:47 <jaypipes> davidkranz: which opinoin is that? :)
17:39:00 <sam_rackspace> As in I agree that a smoke is gate number 1 and the most important/required :-)
17:39:18 <davidkranz> jaypipes: I wasnt' sure if there was a jenkins-determined time that we need to stay under to be a good citizen of gating jobs.
17:40:08 <jaypipes> davidkranz: lol, I think we've blown up our "being a good citizen Karma" with our Ci overlords ;)
17:40:11 <sam_rackspace> and then a waterfall-type gating process/job to suit needs, but the smoke tests have to run before any given build is even considered testable/useable.
17:40:15 <jaypipes> right, jeblair? :)
17:40:24 <jaypipes> sam_rackspace: yup, 100%
17:40:27 <dwalleck> If there is, that's something we probably need to keep in mind
17:40:56 <jaypipes> dwalleck: my idea is to break the current job (which executes everything) into a series of dependent jobs.
17:41:01 <davidkranz> Our tools to reduce the time are parallelize, only create servers when necessary, reduce smoke cases.
17:41:15 <jaypipes> dwalleck: that way we will have finer-grained control of test runs and what the core projects want to prevent a merge with a failure
17:41:26 <jaypipes> davidkranz: ++
17:41:29 <jaypipes> davidkranz: no, +100
17:41:47 <dwalleck> jaypipes: ++
17:42:24 <davidkranz> I can take a look at the server creation and try to trim it down if no one else is doing that.
17:42:25 <dwalleck> I see what you're saying, that makes sense.
17:42:34 <jaypipes> davidkranz: OK, if I work on the parallel execution today, can either you or dwalleck work on identifying precisely which tests are decorated as smoke, but arenty'?
17:42:42 <jaypipes> davidkranz: ++
17:42:52 <jaypipes> davidkranz: smaller commits, the better...
17:42:59 <jaypipes> davidkranz: it makes merge hell easir.
17:43:01 <davidkranz> jaypipes: Sure.
17:43:10 <dwalleck> I can take a pass at it
17:43:53 <davidkranz> OK, so Jay:parallelize, Daryl: smoke accuracy, David: server creation overhead?
17:44:16 <jaypipes> ++
17:44:41 <jaypipes> kk, I say we get to work then...
17:44:43 <davidkranz> As we have discussed, when a new server is needed is somewhat of a judgement call so we can iron that out in reviews of code submissions.
17:45:25 <dwalleck> jaypipes: ++ O
17:45:36 <jaypipes> davidkranz: My rule of thumb: if a server needs to be restarted, rebooted, updated metadata, snapshotted, or modified in any way, the server needs to be created and destroyed in the test method. If not, it can be added as a class-level shared instance.
17:45:45 <dwalleck> I'll make a first pass through and put it up for a review
17:45:53 <jaypipes> awesome
17:46:03 <davidkranz> Sounds good.
17:46:09 <jaypipes> alrighty. sounds like a plan.
17:46:16 * jaypipes eager to get to work..
17:46:23 <dwalleck> Boom!
17:46:24 <davidkranz> So long then...
17:46:27 <dwalleck> #endmeeting