17:03:08 #startmeeting qa 17:03:09 Meeting started Thu Nov 15 17:03:08 2012 UTC. The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:03:10 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:03:11 The meeting name has been set to 'qa' 17:03:14 davidkranz: Error: Can't start another meeting, one is in progress. 17:03:17 hehe 17:03:30 :) 17:03:52 Should we go with the agenda I posted, or is there anything else to add? 17:04:01 davidkranz: nothing from me. 17:04:12 davidkranz: I think the agenda looks good 17:04:20 the posted agenda looks good 17:04:27 the rest we can do in open discussion if things come up 17:04:36 Let's start with the small ones. 17:05:02 I got zero response to the message about getting dev help to review significant new api tests. 17:05:25 :( 17:05:35 davidkranz: are these existing reviews? 17:05:53 The one in the agenda has been sitting in the queue. 17:06:13 Is any one in our group swifty enough to review it? 17:06:25 oh, the swift reviews 17:06:33 THe last patch grew it significantly. 17:06:34 https://review.openstack.org/#/c/15554/ ? 17:06:38 I honestly don't know anything about swift 17:06:58 #topic swift tempest tests need review 17:07:01 davidkrnz: we will test it out and review https://review.openstack.org/#/c/15554/ 17:07:24 ravkumar_hp: Thanks! That is great. 17:07:42 we need to have theste tests in as those are critical for gating 17:07:48 #topic Full tempest gate 17:07:56 so the way we've been handling this is devstack is that we ask devs from the team to provide reviews 17:07:59 I'm swiftly enough also, can take a look. I saw some comments from jaypipes, hadn't seen if those had already been addressed 17:08:09 and if they +1 it enough then we take the word at it 17:08:45 sdague: we've already asked notmyname and he gave a review on it, but no followup review IIRC. 17:09:54 ok, I'll take a look at the review as well. 17:09:56 So that will move forward now. We can ask a specific question if we have it. 17:10:40 Where do we stand on getting an "acceptance test" for swift up? 17:10:47 notmyname was very interested in that. 17:11:09 https://review.openstack.org/#/c/15554/ once checked in can be acceptance tests . right? 17:11:37 ravkumar_hp: Yes, part of one. 17:11:58 He was referring to a complete test that would be run to say that swift upstream was ready to be released. 17:12:21 That would not be a gate test. 17:12:43 davidkranz: honestly, I think that's probably something that needs to go back on the swift team 17:13:04 Until recently it was basically RAX qa that performed that function. 17:13:29 sdague: Yes, you are right. 17:13:59 About the full tempest gate. 17:13:59 davidkranz: from hp , we will add some tests once the current one review is finished 17:14:06 so, tempest full gate.... 17:14:06 ravkumar_hp: Cool. 17:14:18 I added an hourly run with email to me. 17:14:24 we still run those tests, as well as new ones for features as they come down the pipe. cors headers for instance. 17:14:46 jose-rax: which tests do you run? 17:15:02 jose-rax: I know. We wanted tests out in the open that are acceptance tests. 17:15:07 ++ 17:15:40 also +1 17:16:23 pretty much everything 17:16:50 jose-rax: I'm still not clear... everything that is in tempest, or something else? 17:16:55 jose-rax: Gigi said at the summit that you guys were going to put the tests up so we could cooperate on maintaining/improving them. 17:16:59 Rick_RS: heyo 17:17:03 so full tempest gate... :) has there been any progress there? a couple of folks on my team are going to get some cycles in the next couple weeks, so I was going to point them in that direction. 17:17:10 but I don't want to be duping anyone else's work 17:17:31 sdague: davidkranz mentioned above that there is now an hourly full run that emails him on failures 17:17:32 jaypipes: hello 17:17:35 sdague: All we need are no flakies and acceptable run time. 17:17:47 There are no more volume failures as of yesterday. 17:17:52 sdague: depends on what you mean by full tempest gate. I'm opening blueprints for everything myself and my guys are working on 17:18:03 dwalleck: cool 17:18:05 davidkranz: good. those were the ones that were consistently hitting me locally. 17:18:14 davidkranz: great. 17:18:20 davidkranz: what's the run time clocking in at now? 17:18:30 jaypipes: There is still a server build flaky tha sdague is looking at. 17:18:36 which is admin tests, benchmarking/combining tests, and adding a benchmarking suite 17:19:18 dwalleck: ok, cool. If you carve those off in interesting ways we can probably get some more hands on it. 17:19:34 sdague: About 25 minutes for full tempest without smoke. 17:19:41 davidkranz: oh, that's not too bad 17:20:08 smoke is another 6 minutes 17:20:17 It's actually mostly done, just waiting for approvals (for admin tests at least). These are things we already have, just trying to get them submitted 17:20:25 If we could parallelize we would run much faster with room to spare for more tests 17:20:31 dwalleck: did you follow my response about the policy.json determining the "adminness" of API calls? :) 17:20:34 dwalleck: Great! 17:20:57 Let's finish Daryl's thread before moving on. 17:21:03 k 17:21:07 davidkranz: so I was kicking around an idea with mtreinish earlier about injecting log messages into nova when tests start / stop. One of the problems in looking through those logs with stack traces is having no idea what triggered them 17:21:10 jaypipes: I do, and for now we can go with your suggestion. I'd like to revist it later to make things more flexible, but it'll work for now 17:21:17 kk 17:21:55 dwalleck: cool, so best guess on when we'd see these things in the review queue? 17:22:20 sdague: Yeah. More instrumentation will likely be needed. 17:22:21 davidkranz: we rely on the tempest swift tests for gating into our pipeline, however we have an exhaustive body of tests that are directed specifically at a deployed version of swift, not a devstack instance. 17:22:37 next week is thanksgiving so that makes things a bit weird, but certainly by the meeting the week afterwards 17:22:48 dwalleck: cool, sounds great 17:22:58 sam-rax: We discussed this at the configuration testing session. 17:23:25 davidkranz: in regards to Gigi's commitment to putting the tests out in the community, we are working on it. We are in the process of doing some cleanup before we put them out there. 17:23:36 sam-rax: We just need a way to report the results of those tests back 17:24:05 Rick_RS: Great. We just wanted to get an idea of where things were because notmyname was asking about it. 17:24:37 #topic parallel tempest 17:24:55 Is any one working on this yet? 17:25:24 I haven't had much luck with testtools at the moment 17:25:32 :( 17:25:46 Tinkering with just simply using eventlet, have a bit more success there 17:26:19 dwalleck: what were the testtools issues? 17:26:21 dwalleck: At this point we just need something that works. 17:26:24 But I thought others were working on a demo of what testtools would look like? 17:26:24 dwalleck: that is nosetests using evenlet? 17:26:40 No, just using raw unittest with eventlet 17:28:09 dwalleck: ok, so that's something I can have a couple folks look at soon. 17:28:43 dwalleck: does existing temptest tests need lot ot rework to use with eventlet? 17:29:22 ravkumar_hp: Not really. Regardless of what we do, we're going to have to remove our dependencies on Nose at some point 17:29:43 which is mostly skip decorators, tags, etc 17:30:27 so mordred has been looking at converting nova over to non nose, I wonder what insights he has there. 17:31:02 (sorry, just got back online) sam-rax: jose-rax: what would it take to get to get your swift-specific tests more open so that others can run them? 17:31:03 what testtools, testr (testrepository) gives us is more than parallel testing. It gives us ways to record previous test runs and timings as well as do regression testing locally... 17:32:39 jaypipes: What type of timing does it provide? Per test timings? per request? 17:33:18 per test 17:33:21 jaypipes: ok, so probably makes sense to do a port to testtools / testr regardless to see how that goes, even if there might be other approaches 17:33:22 notmyname: The idea is that they'll be part of what we put out there after we finish cleaning this stuff up. 17:33:28 notmyname: We are in the process of making things cleaner, removing some rax specific things (like billing, etc...) and ready to provide to the community 17:33:41 sdague: yes, I gave a shot at it about 2 weeks ago but ran into issues and never got the time to email lifeless about them. 17:34:19 jose-rax: sam-rax: cool. I'd love to see them. been waiting for a while ;-) 17:34:30 jaypipes: well if you have work in progress, let me know. 17:34:45 sdague: will do. 17:35:34 Anything else on the previous topics? 17:35:53 sdague: hey 17:36:08 When th hourly run goes two days without failure we can decide whether to put in the full gate at 25 minutes. 17:36:23 sdague: yes - testtools/testr/fixtures/subunit toolchain is pretty often 17:36:41 jose-rax: sam-rax: (we can take this to another channel if you want) what can I do to help facilitate getting the tests out? 17:36:43 often? 17:36:43 There are also easily 5 minutes of slow tests that could be turned off for most projects. 17:36:48 dwalleck: let me know if I can be helpful 17:38:02 notmyname: we are executing on a plan to expedite this. we can talk some more off-line if you like. 17:38:10 davidkranz: "turn off" is a bit of a worrysome term. If it came to that, I think creating a secondary, non-gating job might be helpful 17:38:40 Rick_RS: awesome. me@not.com or in IRC. whatever is more convenient for you 17:38:40 dwalleck: There is already the hourly, and nightly job. 17:39:11 I just meant that some of the expensive server action scenarios could just be run on nova, for example. 17:39:29 And the periodic runs of course. 17:40:23 The last topic was the status of the blueprints. 17:40:57 notmyname: we can do a f2f if you want. I sit in the compute area. 17:41:10 Rick_RS: I sit in San Francisco ;-) 17:41:14 LOL 17:41:22 lmao 17:42:01 * jaypipes buys Rick_RS a plane ticket out to SFO ;) 17:42:08 notmyname: I have some open QE SDT reqs on my team if you want. :-D 17:42:26 notmyname; duh... I'll send you some information later. 17:42:33 :) 17:42:45 notmyname: But seriously, what Rick_RS said, we can chat later offline if you want, but we honestly are working on it, just need to extricate some things and clean it up properly before sharing..... 17:43:15 ok, cool. I'm really happy to hear that. please let me know how I can help. It's a huge need for others in the swift community 17:43:57 sdague: I think the blueprints cover what we discussed at the summit. 17:44:32 davidkranz: you want to take a pass at prioritizing them, and putting targets on them? 17:45:05 We should take up with the ci group the issue of coordinating status of real-cluster testing out side of jenkins. 17:45:41 sdague: OK, I will do that. Do you mean targets like Grizzly-1, etc? 17:45:44 davidkranz: agreed, but that's just one of them, and I honestly think that it's lower priority than a lot of the others 17:46:02 davidkranz: yeh, it might be nice to give ourselves goals that line up with the milestones 17:46:24 davidkranz: what group can modify those fields? 17:46:28 sdague: OK, but it doesn't make sense to give a target unless there is also an assignee :) 17:46:33 davidkranz: agreed 17:46:42 +1 that. :-) 17:46:52 sdague: I presume tempest administrators? 17:46:53 priority can be set without an assignee though 17:47:02 davidkranz: tempest-drivers 17:47:07 sdague: Sure. 17:47:12 and maybe incent folks to take a look at some of them :) 17:47:20 davidkranz: you are an admin so feel free to give others admin access if you want 17:47:30 sdague: none of us are innocent. 17:47:35 * jaypipes snickers 17:47:40 jaypipes: :P 17:48:02 sdague: I'll add you. 17:48:07 davidkranz: cool 17:48:44 * jaypipes would like to help more if he wasn't waist-deep in writing Chef Spec tests :( 17:49:50 Anything else? 17:50:01 not from me 17:50:17 Counting 30 seconds. 17:50:32 I'm good 17:51:02 40 seconds turkish 17:51:21 50 seconds turkish 17:51:34 jaypipes: I guess you can end the meeting. 17:51:52 #endmeeting