17:02:52 <sdague> #startmeeting QA
17:02:52 <openstack> Meeting started Thu Dec 20 17:02:52 2012 UTC.  The chair is sdague. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:53 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:55 <openstack> The meeting name has been set to 'qa'
17:02:59 <Ravikumar_hp> hi
17:03:16 <sdague> ok, so I know we're missing jay and davidkranz, so it's going to be a pretty open floor today
17:03:25 <sdague> does anyone have topics for discussion to bring up?
17:03:39 <davidkranz> sdague: I am here for the moment but may drop out at any time.
17:03:44 <sdague> ok
17:04:00 <Ravikumar_hp> one question: should we remove nova volume tests now
17:04:07 <davidkranz> sdague: I think the bigggest issue is the full gate.
17:04:20 <sdague> ok, lets talk about gate first
17:04:28 <sdague> #topic Tempest Full Gate
17:04:40 <davidkranz> sdague: There is no evidence that the build ERROR problem remains.
17:04:45 <sdague> so, I think that mtreinish and I got to the bottom of the flakey issue
17:05:04 <davidkranz> sdague: That was good work. Thanks.
17:05:11 <sdague> which turned out to be hitting the memory limits on the ci guests because we didn't delete synchonously
17:05:38 <sdague> davidkranz: yeh, it was fun to get to the bottom of that one, couldn't have done it without mtreinish
17:05:53 <sdague> davidkranz: so our remaining issue is time to run, correct?
17:06:07 <davidkranz> sdague: Right. Parallel is the only solution.
17:06:32 <sdague> davidkranz: right, cyeoh is looking into doing a testr conversion like nova just did
17:06:36 <davidkranz> sdague: Especially because new tests keep coming online, which is good but increases time.
17:06:38 <chunwang> May I know how is parallel execution solution going on?
17:06:42 <afazekas> we should trace  the expensive resources like server and volume, when we try to schedule tests in parallel
17:07:06 <afazekas> We should  try to reuse servicers on more cases
17:07:12 <sdague> afazekas: testr does some of that automatically
17:07:20 <davidkranz> afazekas: It is possible we could turn an almost full gate on by dropping most of the server tests until we have paralle.
17:07:47 <sdague> chunwang: because of the holidays, it probably won't see much progress until Jan
17:07:51 <Ravikumar_hp> davidkranz: yes. but include one of two servwer tests
17:07:57 <chunwang> will a existing test framework or tools used for Tempest parallel exeution?
17:07:59 <afazekas> We should add some annotation in order to mark the what resource a test case needs, and  case should push back resources to the pool
17:08:12 <davidkranz> Ravikumar_hp: At the summit we talked about marking some tests as "slow".
17:08:17 <chunwang> Or there is a brand new framework from every begining?
17:08:29 <sdague> chunwang: the intent is to use testr, which is just a different test runner
17:08:41 <davidkranz> I've got to run now and wil be back in 15 min or so if you all are still around.
17:08:41 <sdague> if you look at nova, it just converted to that last week
17:08:58 <chunwang> sdague: ok, go it
17:09:14 <sdague> some cleanups will have to be made, based on the nova experience
17:09:25 <sdague> it will make for overall cleaner tests
17:09:55 <afazekas> sdague:OK.  I will check the 'testr' capabilites before I try to reinvent the wheel
17:10:12 <chunwang> sdague: will the new scripts expected to be created in new format for testr from now on?
17:10:19 <donaldngo> what happened to the exploration of testtools?
17:10:28 <sdague> #link https://testrepository.readthedocs.org/en/latest/MANUAL.html
17:11:00 <sdague> chunwang: the test format doesn't really change
17:11:50 <chunwang> sdague:ok
17:12:12 <sdague> donaldngo: there could still be alternative approaches, testr has some nice regression bisection and speedups that I think are good
17:12:30 <Ravikumar_hp> sdague: who is doing POC for testr?
17:12:42 <Ravikumar_hp> the material looks good and promising
17:12:47 <sdague> cyeoh, he's on my team in australia
17:12:53 <Ravikumar_hp> great
17:13:15 <donaldngo> ill take a look a testr as well
17:13:41 <sdague> yeh, lifeless, who is in #openstack-dev is the maintainer as well, so has been helping on conversions
17:13:52 <sdague> and weird edge conditions
17:14:52 <sdague> mostly in nova we had that because tests caused side effects, and running in parallel exposed that
17:15:20 <chunwang> sdague: what kind of side effects?
17:15:23 <sdague> the last full gate comment I had, was it might be good to turn it on for devstack
17:15:43 <mtreinish> sdague: is there anything blocking that right now?
17:15:47 <sdague> as it's not flakey any more, and it will run a little more often then on tempest, as the devstack code stream is moving faster
17:15:58 <sdague> not except asking dtroyer if he minds
17:16:06 <mtreinish> sdague: heh, ok
17:16:24 <sdague> chunwang: like one test would change a global variable, which would effect how a test ran later
17:16:30 <sdague> basically state leakage
17:16:47 <sdague> ok, I think we're probably done on that topic
17:17:07 <chunwang> sdague: then suppose it's will take some time to fix...
17:17:09 <sdague> #topic Removing Nova Volume Tests
17:17:21 <sdague> Ravikumar_hp: go
17:18:01 <Ravikumar_hp> sdague: looks like nova blueprint to remove nova volume and migrate to cinder is almost complete
17:18:11 <Ravikumar_hp> shoud we remove nova volume tests
17:18:22 <sdague> Ravikumar_hp: sounds good to me
17:18:24 <mtreinish> Ravikumar_hp: do you mean the volume extension tests?
17:18:43 <Ravikumar_hp> other wise tempest will break. yes volume extension tests
17:19:22 <sdague> Ravikumar_hp: you willing to do the patch to do that?
17:19:28 <sdague> we can just handle it in review
17:19:29 <mtreinish> Ravikumar_hp: I'm pretty sure that won't cause any issues being in there (despite being slower)
17:19:32 <Ravikumar_hp> yes . i will
17:19:51 <sdague> ok, so we can make the decision on the review then, as we're missing a bunch of key folks here
17:19:52 <mtreinish> it all gets piped to cinder eventually
17:20:15 <sdague> #action Ravikumar_hp to propose review to remove nova volume tests
17:20:35 <sdague> next topic?
17:20:41 <Ravikumar_hp> mtreinish: Thanks . will check with Nova dev team and do accordingly
17:20:49 <mtreinish> sdague: I can talk a bit about the coverage stuff
17:21:00 <sdague> #topic Tempest Coverage
17:21:01 <chunwang> I want to know the progress of quantum related test scripts.
17:21:18 <sdague> chunwang: we'll do that next
17:21:18 <mtreinish> so the nova coverage extension got merged last week
17:21:30 <chunwang> sdague: sure.
17:21:34 <mtreinish> I've got a few cleanup patches in the queue before it's 100%
17:22:12 <mtreinish> but after those go through I need to add support to devstack-gate and jenkins to do the runs with coverage enabled
17:22:27 <mtreinish> the issue is going to be syncronizing those changes with tempest
17:22:49 <mtreinish> because the way I wrote the tempest support breaks devstack gate without an exclude arg to nose
17:23:22 <mtreinish> but it's almost ready, it's the final stretch
17:23:30 <sdague> mtreinish: nice
17:23:53 <rohitk> mtreinish: great work on the coverage extension!
17:24:06 <mtreinish> rohitk: thanks
17:24:48 <sdague> ok, anything else on coverage?
17:25:02 <mtreinish> sdague: not from me
17:25:11 <sdague> #topic Quantum in Tempest
17:25:21 <sdague> ok, does anyone have an update there?
17:26:05 <sdague> going once... going twice ?
17:26:12 <chunwang> I saw there are 2 blue print for quantum tests...but seems no update for both of them.
17:26:45 <sdague> yeh, I don't know, and I don't know who's spear heading those
17:26:47 <chunwang> and not sure whether both of the blue print will be adopted.
17:27:02 <sdague> ok, so I'm going to suggest we move on
17:27:09 <sdague> #topic Open Discussion
17:27:20 <sdague> ok, any other topics people want to bring up?
17:27:26 <chunwang> I have another question is...
17:27:46 <chunwang> whether there is any one creating some scripts to verify every compute node is working
17:28:42 <sdague> chunwang: can you explain more?
17:28:50 <chunwang> https://blueprints.launchpad.net/tempest/+spec/multinode-testing <-- I saw a blue print for multiple node, but not sure whether it's the same meaning as mine.
17:29:00 <chunwang> for example
17:29:10 <sdague> chunwang: no, that's having the tempest runs run on multiple servers
17:29:18 <sdague> instead of just a single machine devstack
17:29:25 <sdague> so it would be more realistic
17:29:26 <chunwang> there are 20 compute nodes in the deployed env, but some of them are not working
17:29:48 <chunwang> when instances assigned to the unavailable nodes, it will be fail
17:30:12 <sdague> chunwang: so that I think is really outside the role of tempest. It's testing interface functionality, it's not a health monitor
17:30:30 <chunwang> yes, but I suppose tempest is for integration test
17:30:54 <chunwang> who will care about the real deployment env test?
17:31:25 <chunwang> not tempest?
17:31:41 <sdague> chunwang: that I think is beyond the scope of this project. I could see that as another project that someone might want to create.
17:31:47 <sdague> but that's just my opinion
17:32:37 <afazekas> sdague: +1
17:33:10 <Ravikumar_hp> sdague: yes. I agree. +1
17:33:23 <sdague> ok, so any other topics for discussion
17:34:20 <sdague> going once...
17:34:38 <sdague> going twice ...
17:34:46 <sdague> ok, I'm going to call it
17:34:49 <sdague> #endmeeting