17:00:23 #startmeeting qa 17:00:24 Meeting started Thu Sep 13 17:00:23 2012 UTC. The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:25 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:26 The meeting name has been set to 'qa' 17:00:52 dansmith, donaldngo, dwalleck: heyo 17:01:01 howdy! 17:01:13 present 17:01:19 dwalleck: so, I've been fighting with nosetest this week. 17:01:25 dwalleck: and losing the fight :( 17:01:43 uh oh :( what's wrong? 17:02:11 dwalleck: well, there's the multiprocess plugin's inability to deal with module and package-level setups: https://github.com/nose-devs/nose/issues/550 17:02:16 gah 17:02:18 stupid wifi 17:02:23 dwalleck: https://github.com/nose-devs/nose/issues/550 17:02:25 dwalleck: well, there's the multiprocess plugin's inability to deal with module and package-level setups: https://github.com/nose-devs/nose/issues/550 17:02:52 dwalleck: as well as (apparently), nosetests buggy implementation of mod and package-level fixtures: https://github.com/nose-devs/nose/issues/551 17:03:23 Yeah, never tried package level fixtures 17:03:38 dwalleck: I think that Antonin from HP may have been right originally when he remarked that trying to use nose/unittest for functional integration testing is like trying to fit a round peg in a square hole :( 17:03:45 torandu: welcome. :) 17:04:08 dwalleck: we've been using module and package level fixtures in Tempest for a while now :) 17:04:17 dwalleck: /tempest/tests/compute/__init__.py 17:04:20 jaypipes: I'm starting to agree with that thought 17:04:39 dwalleck: yeah... it's like pulling teeth to get it to behave the way you think it should 17:04:52 I've been looking around at alternatives lately 17:05:22 dansmith, dwalleck: so my question is this: is it worth spending the time to rewrite ourselves out of nose/unittest, or spending the time to try fixing nose upstream? 17:05:50 Nothing fits the bill quite right, but I've been liking this a bit. Made a few fun prototypes http://heynemann.github.com/pyvows/ 17:05:59 dwalleck: or go with something like Vek's integration testing infrastrcture? can't remember the name of it off top of head 17:06:20 jaypipes: I don't really have any visibility into what nose really gives us, nor the issues in the stack 17:06:26 sdague: did you get a feeling yet? 17:06:30 I think Vek's is also built on top of nose 17:07:17 I'm using py.test right now, which I've had some success with, but it's handling of fixtures when running tests multithreaded...well, destroys the purpose of fixtures 17:07:27 dwalleck: no, I don't think so. 17:08:00 dwalleck: backfire? soemthing like that? 17:08:06 dwalleck: name of Vek's lib... 17:08:14 jaypipes: Sounds right 17:08:49 https://github.com/klmitch/dtest 17:08:54 DTest 17:09:01 is the framework 17:10:00 * jaypipes reading a bit more about pyvows. 17:10:07 It has it's own challenges though from what I recall. I believe test suites have to been created programatically 17:10:23 pyvows reminds me of Ruby's rspec 17:11:12 I kind of like it because it gives you just enough order in testing and then runs in parallel when it can 17:11:25 dansmith: no, I haven't looked at the parallization yet 17:12:31 dwalleck: it's whether or not it can deal with out-of-process execution and shared fixtures is the issue 17:12:53 pyvows looks interesting. Wasn't there also a team at the last summit that had a cucumber based testing framework? 17:13:34 There's lettuce, which is a port of cucumber 17:14:11 But I've never felt very good about having tests as text. I think the reason I can stand pvows is because the tests are still in code :-) 17:14:21 sdague: I'd really prefer to stick with Python TBH 17:14:29 dwalleck: ++ 17:15:13 And if not that, maybe a solution similar to that provides similar functionality 17:15:19 jaypipes: yeh, sure python is good. Though I like the idea of behavior oriented, because it reduces the chance that you've got a copy and paste bug from existing python code 17:15:48 sdague: agreed. 17:16:02 The plus for me is that it's very descriptive about what exactly you're testing 17:16:13 sdague: pyvows looks interesting in that regard... good mix of BDD and code-based Python framework 17:16:37 yep, agreed 17:17:11 dwalleck: good spot on PyVows. How about we agree to look through PyVows and maybe put together a quick prototype that does some OpenStack compute API tests> 17:17:27 dwalleck: trying to reuse as much as possible from tempest's base clients, etc 17:18:24 jaypipes: Can do. I something prototypish together already. I could have something to look at by tomorrow 17:18:37 dwalleck: that would be awesome. 17:18:46 dwalleck: post to the ML for us all to take a look at. 17:18:50 dwalleck: cool, would love to look. 17:18:53 can do 17:19:18 and even if it's not right, maybe it will keep pointing us in the right direction 17:19:30 * jaypipes would prefer to move quickly on this if we can 17:19:33 though I doubt I'll jump in until after we've got all the sample api tests over in nova that vishy would like for the release 17:19:54 yeah... 17:20:05 jaypipes: how quickly? Seems like it might be good to get some prototypes playing and then hack a bit at summit 17:20:12 sdague: ++ 17:20:23 jaypipes: ++. Very fast. Once we have a solution we like, I'd like to see what we could accomplish by the summit 17:20:32 dwalleck: rock on. 17:20:33 sdague: ++, great idea 17:20:43 OK, y'all OK with ending the meeting early? 17:20:51 sure 17:21:04 #endmeeting