17:00:23 <jaypipes> #startmeeting qa
17:00:24 <openstack> Meeting started Thu Sep 13 17:00:23 2012 UTC.  The chair is jaypipes. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:25 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:26 <openstack> The meeting name has been set to 'qa'
17:00:52 <jaypipes> dansmith, donaldngo, dwalleck: heyo
17:01:01 <dwalleck> howdy!
17:01:13 <dansmith> present
17:01:19 <jaypipes> dwalleck: so, I've been fighting with nosetest this week.
17:01:25 <jaypipes> dwalleck: and losing the fight :(
17:01:43 <dwalleck> uh oh :( what's wrong?
17:02:11 <jaypipes> dwalleck: well, there's the multiprocess plugin's inability to deal with module and package-level setups: https://github.com/nose-devs/nose/issues/550
17:02:16 <jaypipes> gah
17:02:18 <dwalleck> stupid wifi
17:02:23 <jaypipes> dwalleck: https://github.com/nose-devs/nose/issues/550
17:02:25 <jaypipes> dwalleck: well, there's the multiprocess plugin's inability to deal with module and package-level setups: https://github.com/nose-devs/nose/issues/550
17:02:52 <jaypipes> dwalleck: as well as (apparently), nosetests buggy implementation of mod and package-level fixtures: https://github.com/nose-devs/nose/issues/551
17:03:23 <dwalleck> Yeah, never tried package level fixtures
17:03:38 <jaypipes> dwalleck: I think that Antonin from HP may have been right originally when he remarked that trying to use nose/unittest for functional integration testing is like trying to fit a round peg in a square hole :(
17:03:45 <jaypipes> torandu: welcome. :)
17:04:08 <jaypipes> dwalleck: we've been using module and package level fixtures in Tempest for a while now :)
17:04:17 <jaypipes> dwalleck: /tempest/tests/compute/__init__.py
17:04:20 <dwalleck> jaypipes: I'm starting to agree with that thought
17:04:39 <jaypipes> dwalleck: yeah... it's like pulling teeth to get it to behave the way you think it should
17:04:52 <dwalleck> I've been looking around at alternatives lately
17:05:22 <jaypipes> dansmith, dwalleck: so my question is this: is it worth spending the time to rewrite ourselves out of nose/unittest, or spending the time to try fixing nose upstream?
17:05:50 <dwalleck> Nothing fits the bill quite right, but I've been liking this a bit. Made a few fun prototypes http://heynemann.github.com/pyvows/
17:05:59 <jaypipes> dwalleck: or go with something like Vek's integration testing infrastrcture? can't remember the name of it off top of head
17:06:20 <dansmith> jaypipes: I don't really have any visibility into what nose really gives us, nor the issues in the stack
17:06:26 <dansmith> sdague: did you get a feeling yet?
17:06:30 <dwalleck> I think Vek's is also built on top of nose
17:07:17 <dwalleck> I'm using py.test right now, which I've had some success with, but it's handling of fixtures when running tests multithreaded...well, destroys the purpose of fixtures
17:07:27 <jaypipes> dwalleck: no, I don't think so.
17:08:00 <jaypipes> dwalleck: backfire? soemthing like that?
17:08:06 <jaypipes> dwalleck: name of Vek's lib...
17:08:14 <dwalleck> jaypipes: Sounds right
17:08:49 <jaypipes> https://github.com/klmitch/dtest
17:08:54 <jaypipes> DTest
17:09:01 <jaypipes> is the framework
17:10:00 * jaypipes reading a bit more about pyvows.
17:10:07 <dwalleck> It has it's own challenges though from what I recall.  I believe test suites have to been created programatically
17:10:23 <jaypipes> pyvows reminds me of Ruby's rspec
17:11:12 <dwalleck> I kind of like it because it gives you just enough order in testing and then runs in parallel when it can
17:11:25 <sdague> dansmith: no, I haven't looked at the parallization yet
17:12:31 <jaypipes> dwalleck: it's whether or not it can deal with out-of-process execution and shared fixtures is the issue
17:12:53 <sdague> pyvows looks interesting. Wasn't there also a team at the last summit that had a cucumber based testing framework?
17:13:34 <dwalleck> There's lettuce, which is a port of cucumber
17:14:11 <dwalleck> But I've never felt very good about having tests as text. I think the reason I can stand pvows is because the tests are still in code :-)
17:14:21 <jaypipes> sdague: I'd really prefer to stick with Python TBH
17:14:29 <jaypipes> dwalleck: ++
17:15:13 <dwalleck> And if not that, maybe a solution similar to that provides similar functionality
17:15:19 <sdague> jaypipes: yeh, sure python is good. Though I like the idea of behavior oriented, because it reduces the chance that you've got a copy and paste bug from existing python code
17:15:48 <jaypipes> sdague: agreed.
17:16:02 <dwalleck> The plus for me is that it's very descriptive about what exactly you're testing
17:16:13 <jaypipes> sdague: pyvows looks interesting in that regard... good mix of BDD and code-based Python framework
17:16:37 <sdague> yep, agreed
17:17:11 <jaypipes> dwalleck: good spot on PyVows. How about we agree to look through PyVows and maybe put together a quick prototype that does some OpenStack compute API tests>
17:17:27 <jaypipes> dwalleck: trying to reuse as much as possible from tempest's base clients, etc
17:18:24 <dwalleck> jaypipes: Can do. I something prototypish together already. I could have something to look at by tomorrow
17:18:37 <jaypipes> dwalleck: that would be awesome.
17:18:46 <jaypipes> dwalleck: post to the ML for us all to take a look at.
17:18:50 <sdague> dwalleck: cool, would love to look.
17:18:53 <dwalleck> can do
17:19:18 <dwalleck> and even if it's not right, maybe it will keep pointing us in the right direction
17:19:30 * jaypipes would prefer to move quickly on this if we can
17:19:33 <sdague> though I doubt I'll jump in until after we've got all the sample api tests over in nova that vishy would like for the release
17:19:54 <jaypipes> yeah...
17:20:05 <sdague> jaypipes: how quickly? Seems like it might be good to get some prototypes playing and then hack a bit at summit
17:20:12 <jaypipes> sdague: ++
17:20:23 <dwalleck> jaypipes: ++. Very fast. Once we have a solution we like, I'd like to see what we could accomplish by the summit
17:20:32 <jaypipes> dwalleck: rock on.
17:20:33 <dwalleck> sdague: ++, great idea
17:20:43 <jaypipes> OK, y'all OK with ending the meeting early?
17:20:51 <dwalleck> sure
17:21:04 <jaypipes> #endmeeting