16:02:21 <flaper87> #startmeeting marconi
16:02:22 <openstack> Meeting started Fri Aug  2 16:02:21 2013 UTC and is due to finish in 60 minutes.  The chair is flaper87. Information about MeetBot at http://wiki.debian.org/MeetBot.
16:02:23 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
16:02:25 <openstack> The meeting name has been set to 'marconi'
16:02:32 <cppcabrera> Awesome.
16:02:41 <Sriram> :)
16:02:43 <flaper87> Yo yo guys!!!!!!
16:02:56 <flaper87> #topic agenda
16:02:56 <Sriram> yo! :P
16:02:59 <flaper87> #link https://wiki.openstack.org/wiki/Meetings/Marconi#Agenda
16:03:07 <flaper87> so, that's the agenda for today's meeting
16:03:25 <flaper87> lets get it started
16:03:29 <cppcabrera> Time to review some action items. :D
16:03:29 <malini> Can we also add a new meeting time to the agenda, plz?
16:03:32 <flaper87> #topic review action items
16:03:36 <flaper87> #link http://eavesdrop.openstack.org/meetings/marconi/2013/marconi.2013-07-18-19.05.html
16:03:59 <flaper87> cppcabrera: to turn apiclient-marconi etherpad into bps/bugs
16:04:18 <flaper87> I should've been more involved in that, sorry
16:04:21 <cppcabrera> I don't believe I got around to that.
16:04:23 <flaper87> any news about that ?
16:04:33 <flaper87> ok, lets schedule it for the next meeting
16:04:35 <cppcabrera> I may have done some Trello'ing, but that's a blur to me at the moment.
16:04:40 <flaper87> #action cppcabrera flaper87 to turn apiclient-marconi etherpad into bps/bugs
16:04:44 <cppcabrera> +1
16:04:56 <flaper87> cppcabrera: to tie up loose ends re python-marconicient v1 design
16:04:57 <flaper87> ?
16:05:13 <cppcabrera> Yes, there's been progress there. :)
16:05:15 <cppcabrera> #link https://wiki.openstack.org/wiki/Python_Marconi_Client
16:05:24 <cppcabrera> I've since added some ideas on pubsub design.
16:05:35 <cppcabrera> I still have some concerns.
16:05:46 <flaper87> cppcabrera: awesome, thanks!
16:05:55 <cppcabrera> I want to talk about openstack sdk/cli design with someone who's written such clients.
16:06:04 <cppcabrera> For example, I'm a little dubious about the naming of methods.
16:06:06 <cppcabrera> list
16:06:08 <cppcabrera> get
16:06:10 <cppcabrera> post
16:06:28 <flaper87> Is this something that worths a thread in the m-l ?
16:06:28 <cppcabrera> It's... very HTTP-targeted, when I, as a user of the SDK, think it terms of resources.
16:06:35 <cppcabrera> Like client.queues, etc.
16:06:41 <cppcabrera> I believe so.
16:06:47 <cppcabrera> s/it/in
16:06:50 <flaper87> We can get some of the best minds around this design process
16:07:01 <tedross> m-l?
16:07:04 <cppcabrera> I'll start that discussion on the mailing list.
16:07:05 <flaper87> tedross: mailing list
16:07:16 <flaper87> awesome! thanks!
16:07:17 <cppcabrera> openstack-dev, woot. :)
16:07:33 <flaper87> flaper87: to investigate mongo insert performance
16:07:45 <flaper87> so, I did, I've some work done on that area
16:07:45 <cppcabrera> #action cppcabrera to ask about client design on mailing list
16:07:57 <whenry> cppcabrera, pls feel free to reachout to tedross or me re client apis
16:07:58 <flaper87> there's still some work left there
16:08:07 <cppcabrera> whenry: Thanks!
16:08:09 <tedross> we'll look at the link
16:08:12 <whenry> let us know how we can help
16:08:24 <Sriram> woot! awesome.
16:08:30 <flaper87> tedross: whenry awesome, thanks guys!
16:08:39 <cppcabrera> Yes, much appreciated. :)
16:08:59 <flaper87> last action item was: to finalize placement service strategy
16:09:15 * cppcabrera looks for the wiki page...
16:09:20 <flaper87> I guess cppcabrera worked more on that one and we'll discuss it in the second part of this meeting
16:09:32 <flaper87> lets get to the next topic and then go back there
16:09:36 <Sriram> There was quite a discussion on that. Lots of progress there :)
16:09:40 <whenry> pls let us know the process: e.g. is it, opesntac-dev [marconi] for banter around ideas and then someone will update the docs?
16:09:40 <cppcabrera> #link https://wiki.openstack.org/wiki/Marconi/bp/placement-service
16:09:55 <whenry> oops sorry off topic
16:10:03 <flaper87> whenry: we discuss most of the things on #openstack-marconi
16:10:17 <flaper87> but, if there are some discussions that worth bringing up to the m-l, then we do it there
16:10:27 <cppcabrera> I actually added our IRC channel to the official list a few minutes ago, heh. :P
16:10:27 <flaper87> #topic Test re-factor and improvements
16:10:29 <whenry> flaper87, ah, good to know.
16:10:35 * whenry adds to favs
16:10:44 <flaper87> cppcabrera: malini ^
16:10:48 <malini> https://etherpad.openstack.org/marconi-test-refactoring
16:10:52 <flaper87> Floor is all yours
16:11:01 <cppcabrera> #link https://blueprints.launchpad.net/marconi/+spec/refactor-system-tests
16:11:17 <cppcabrera> Awesome, malini.
16:11:24 * whenry closes #opensack-marconi and fixes typo
16:11:40 <flaper87> whenry: lol
16:11:40 <malini> Can we first take a stab at the long term vision, how we want the tests organized?
16:11:49 <flaper87> malini: sure
16:12:09 <cppcabrera> Currently, I have some concerns about our functional tests. I think that as we're growing them now, they're getting a little fragile and need some DRYing up.
16:12:10 <malini> we have had quite some discussions around this & have it documented in the etherpad
16:12:43 <malini> cppcabrera: can you elaborate on the fragile part?
16:12:51 <malini> more specifics
16:12:54 <cppcabrera> malini and I spoke about it a lot offline, and I'll elaborate on that for sure.
16:13:08 <flaper87> malini: cppcabrera lets keep going on the structure and then get back there
16:13:12 <cppcabrera> There are two problems that I see wrt fragility.
16:13:14 <malini> +1
16:13:22 <malini> flaper87: +1
16:13:42 <malini> w.r.t structure, lets get all the tests out of the core marconi code
16:13:59 <flaper87> malini: AFAIS, the structure in the etherpad is much like we discussed, right?
16:14:01 <cppcabrera> 1. There are hidden ordering dependencies in the test at the moment. They've been crafted up to this point very carefully, and I see that. However, when others start contributing  tests, they might not be aware of the conventions and introduce subtle breaks.
16:14:04 <flaper87> am I missing something ?
16:14:43 <flaper87> malini: ^
16:14:56 <malini> flaper87: yes, + we did some pokng around other OS repos & came up with what we have in the etherpad
16:15:00 <cppcabrera> 2. The tests allocate resources and don't guarantee clean up of those resources in the presence of test assertion failures.
16:15:08 <cppcabrera> That's it for fragility.
16:15:18 <zyuan> those are bad for testing
16:15:31 <zyuan> that's why i +1 for using context manager
16:15:36 <malini> cppcabrera, zyuan: lets first address the structure part first
16:15:45 <flaper87> malini: I think the structure in the etherpad makes sense
16:15:50 <malini> does everybody agree on the etherpad?
16:15:51 <cppcabrera> Alright, structure. :)
16:15:57 <clarkb> zyuan: the fixtures lib gives you context manager for free
16:16:08 <clarkb> zyuan: with a bunch of common fixtures baked in
16:16:13 <Sriram> Yes, the structure looks good.
16:16:27 <zyuan> clarkb: iknow. just... they are not common
16:16:30 <cppcabrera> I think system should be on the same level as unit
16:16:49 <malini> cppcabrera: as in not under v1??
16:16:51 <cppcabrera> So 'tests/{system, unit, smoke, ...}
16:17:03 <zyuan> where is the etherpad?
16:17:10 <flaper87> mmh, actually, I agree with cppcabrera
16:17:12 <cppcabrera> zyuan: https://etherpad.openstack.org/marconi-test-refactoring
16:17:25 <flaper87> then we should have {v1, v2} where needed
16:17:35 <flaper87> i.e: transport
16:17:42 <malini> cppcabrera: the argument somebody had for tht was unit tests won't change between versions
16:18:10 <cppcabrera> malini: +1. The unit tests should evolve with the code.
16:18:15 <cppcabrera> They are version agnostic.
16:18:24 <flaper87> cppcabrera: mmh, not sure about that
16:18:34 <zyuan> does system test use utils under tests/
16:18:35 <zyuan> ?
16:18:54 <flaper87> If we make changes in the API, the API unittests for v1 will remain the same and then we'll have some for v2
16:18:54 <cppcabrera> zyuan: Not yet.
16:19:04 <flaper87> so, we need v1 and v2 under transport
16:19:28 <cppcabrera> flaper87: I see what you're saying. It makes sense to me. I typically envision unit test structure mirroring implementation structure.
16:19:56 <cppcabrera> I'm less certain about the structure of functional/system tests.
16:20:23 <malini> the functional tests are independent of the code structure
16:20:29 <cppcabrera> +1 malini
16:20:35 <malini> application code structure*
16:20:42 <cppcabrera> I feel like they should mirror the endpoints, instead.
16:20:51 <flaper87> so, what about unit and functional in the same level and then adding {v1,v2} under functional instead ?
16:20:51 <malini> it should be more in terms of functionality
16:20:59 <flaper87> to keep consistency in the structure
16:21:08 <flaper87> type/functionality/version
16:21:15 <flaper87> unit/transport/v1
16:21:16 <malini> flaper87: +1
16:21:18 <cppcabrera> I like the idea of that.
16:21:21 <flaper87> functional/transport/v1
16:21:43 <cppcabrera> functional/http/v1, functional/zmq/v1?
16:22:01 <flaper87> we can either do that or just have test_impl_zmq.py
16:22:07 <flaper87> and test_impl_wsgi.py
16:22:11 <flaper87> I'm good with both
16:22:35 <cppcabrera> Alright, I'm +1 with the proposed structuring.
16:22:36 <Sriram> yes.. functional/transport/http/v1 also seems good.
16:22:37 <zyuan> functional/v1/test_impl_wsgi.py
16:22:50 <malini> hmmm...
16:22:53 <zyuan> functional tests should be at a very high level
16:23:08 <malini> yes..tht's why  we will not have functional/v1/test_impl_wsgi.py
16:23:11 <zyuan> it does not know about "transport", IMO
16:23:28 <zyuan> it only know "the procedures to use"
16:23:41 <malini> we'll have functional/v1/test_queue.py,  functional/v1/test_messages.py,  functional/v1/test_claim.py etc.
16:24:02 <cppcabrera> +1 malini. That matches up more clearly for me with the idea of testing endpoints.
16:24:10 <malini> or  functional/v1/http/*  ??
16:24:22 <malini> functional/v1/zmq/*
16:24:22 <cppcabrera> Yeah, we might break it down by transport at some point.
16:24:22 <zyuan> ^^ yea
16:24:26 <malini> etc. ??
16:24:29 <cppcabrera> When we get there. :)
16:24:34 <zyuan> if test_impl_wsgi is too large
16:24:35 <cppcabrera> s/might/should
16:24:38 <zyuan> break it into http/
16:24:43 <flaper87> +1
16:24:51 <whenry> why have version down inside the topology ?
16:24:59 <whenry> instead of higher up?
16:25:10 <whenry> sorry I missed that
16:25:13 <flaper87> whenry: consistency with the structure
16:25:19 <flaper87> type/../../
16:25:21 <flaper87> unit/../
16:25:24 <flaper87> functional/...
16:25:26 <flaper87> and so on
16:25:28 <whenry> openstack test structure ?
16:25:33 <flaper87> not exactly
16:25:43 <flaper87> most projects do that, though
16:25:47 <malini> whenry: were you referring to v1 ?
16:25:53 <whenry> so type might have a diff version than functional?
16:25:57 <whenry> malini, yes
16:26:02 <flaper87> plus, functional tests can be used for other things than just transport tests
16:26:13 <whenry> or does v1 mean the same v across these ?
16:26:16 <flaper87> what if we've a cache library we want to test ?
16:26:16 <malini> v1 is the application code version tht it'll test
16:26:22 <flaper87> we might need some functional tests for that
16:26:28 <zyuan> api version
16:26:29 <flaper87> that have to live under functional/
16:27:13 <flaper87> 16:26 UTC, Can we say we agreed on the structure ?
16:27:19 <cppcabrera> +1
16:27:42 <cppcabrera> I'd like to discuss the test strategy re: fragility. ^
16:27:52 <flaper87> #action malini to update the etherpad and start working on the re-factor
16:27:54 <flaper87> cppcabrera: go ahead
16:28:34 <cppcabrera> I mentioned previously that the tests are fragile atm. 1) hidden ordering dependencies and 2) resources allocated not guaranteed to be cleaned up in the presence of test assertion failures.
16:28:47 * kgriffs is here - had to get a new chair and make it through the welcome gauntlet
16:28:49 <cppcabrera> I have a patch pending that addresses these issues, and I'd like to gather thoughts on that.
16:29:10 <malini> can I first address the two issues, before we discuss the patch ?
16:29:26 <cppcabrera> Certainly, malini. :)
16:30:05 <malini> The functional tests are designed to simulate user behavior
16:30:39 <malini> i.e follow sequence of events like create queue, post message, claim them, delete messages etc. , sort of what we expect in a real world
16:31:23 <malini> so for message tests, the system tests have a pre-requisite to create a queue
16:31:29 <zyuan> malini: iirc, there is a testing tool to use a | separated table to discribe it?
16:31:59 <malini> zyuan: you are refrring to robot framework ?
16:32:07 <zyuan> yea
16:32:15 <malini> we moved away from robot to nose, to make it easier for everybody to understand
16:32:27 <cppcabrera> +1 to that, malini
16:32:29 <zyuan> we all understand the need of action dependency
16:32:33 <Sriram> +1 for nose! :)
16:32:43 <flaper87> +1 for nose
16:32:44 <malini> ok..so back to what I was referring
16:32:47 <zyuan> just, such a dependency should not across tests
16:33:10 <cppcabrera> I agree that there is a place for tests that simulate random user behavior, and can get messy.
16:33:11 <malini> tht where we disgree
16:33:21 <zyuan> you can use context manager, fixture, or even very long tests if you want
16:33:22 <cppcabrera> Thanks to those tests, we were able to find the truncated JSON reply.
16:33:37 <cppcabrera> My argument is that the functional tests, or the system tests, are not the place for that.
16:33:46 <malini> yes, & we found it because we had data from the prev test
16:33:47 <cppcabrera> we need some form of test that is highly reproducible and consistent.
16:34:09 <zyuan> i have a suggestion
16:34:13 <cppcabrera> To guarantee that reproducibility, we need to eliminate dependencies across test case boundaries.
16:34:14 <malini> we found the truncated JSON because thesystem tests cud repro it consistently
16:34:16 <kgriffs> IMHO random behavior is kind of like fuzzing in securty - it can find particular types of errors, but is probably less helpful than other types of tests.
16:34:25 <zyuan> put test in non-test member-functions
16:34:30 <flaper87> kgriffs: +1
16:34:30 <malini> we are not talking random behavior
16:34:32 <zyuan> and call them randomly, explicitly
16:34:55 <zyuan> namely, controlly messy
16:35:00 <malini> we are talking abt using the API realistically
16:35:16 <kgriffs> malini: correct me if I'm wrong, but that is what the functional tests do today, right?
16:35:41 <malini> kgriffs: can you clarify the what in ur question?
16:35:48 <flaper87> I'd like to see functional tests being triggered by tox
16:35:51 <flaper87> with other tests
16:35:55 <malini> flaper87: +1
16:35:58 <kgriffs> sorry, just confirming
16:36:05 <cppcabrera> tox -e nightly, tox -e smoke, tox -e system
16:36:10 <cppcabrera> :)
16:36:10 <flaper87> and IMHO, the functional tests suit should run marconi itself
16:36:19 <malini> flaper87: +1
16:36:34 <malini> * seems to agree with everything flaper87 has'
16:36:35 <kgriffs> I think if we want to do random behavior (with 2-3 seeds for repeatability) that is a new bp
16:36:36 <amitgandhi> to me random behaviour testing (or using api realistically) falls under load testing (where diff scenarios are fed as inputs)
16:36:39 <kgriffs> and we should do it later
16:36:47 <flaper87> Can we say that's the next step for our functional tests?
16:36:56 <cppcabrera> amitgandhi, malini: Makes me think of tsung.
16:36:59 <malini> clarification: we are NOT testing random behavior in system tests now
16:37:06 <cppcabrera> malini is right.
16:37:08 <flaper87> 1) re-structure 2) run marconi 3) run with tox ?
16:37:12 <cppcabrera> The tests atm are NOT random.
16:37:23 <cppcabrera> They just have ordering dependencies.
16:37:28 <kgriffs> right, I am just saying that we should not do a lot of work now to support that
16:37:30 <cppcabrera> They must execute as test_001, test_002, ...
16:37:33 <cppcabrera> Or an error occurs.
16:37:43 <kgriffs> (to support arbitrary test ordering)
16:38:04 <cppcabrera> The work is done, kgriffs. :P
16:38:15 <kgriffs> we already have tsung doing some of that, albeit not repeatable randomness
16:38:27 <cppcabrera> The work is done to support arbitrary ordering, I should say,
16:38:31 <cppcabrera> #link https://review.openstack.org/#/c/39369/
16:38:40 <kgriffs> cppcabrera: does it break linear testing?
16:38:44 <malini> will come to the patch soon
16:38:53 <malini> can we finish the discussion first?
16:39:05 <cppcabrera> malini: Yes. :)
16:39:10 <cppcabrera> kgriffs: Linear testing?
16:39:30 <kgriffs> sorry, ordered testing, whatever you want to call it
16:39:42 <kgriffs> functional testing, I guess
16:39:52 <cppcabrera> Is the "it" the patch?
16:39:53 <malini> So if we want to keep the tests independent, we should first have make sure that the pre-requisites are covered
16:40:05 <malini> 'it' = patch
16:40:10 <cppcabrera> Cool.
16:40:22 <cppcabrera> The patch does not break ordered testing.
16:40:57 <malini> it doesnt make sense to have an independent list queues test that can run in parallel with create queue, if all list queue does is return a list with one ASCII queue
16:40:57 <cppcabrera> Could you elaborate on the pre-reqs, malini?
16:41:20 <flaper87> I don't think functional tests should be independent
16:41:21 <malini> hence the current ordering in today's tests
16:41:33 <flaper87> I'm missing something, I guess
16:41:38 <malini> flaper87: +1
16:41:46 <cppcabrera> I feel like I'm missing something, too...
16:41:49 <kgriffs> ok
16:41:58 <kgriffs> let me see If can summarize
16:41:59 <cppcabrera> I might be thinking of a different testing strategy that checks whether the API behaves as expected.
16:42:09 <cppcabrera> Versus what a user might do..
16:42:11 <kgriffs> it seems like we have two different kinds of tests in mind
16:42:15 <cppcabrera> Yes!
16:42:18 <kgriffs> and trying to do it with a single set of tests
16:42:29 <malini> we need to guarantee the same behavior on each test run, which we do today
16:42:48 <flaper87> 16:42 UTC (there won't be time for placement)
16:42:57 <malini> yeap, we are from different schools of thought
16:43:17 <malini> I am thinking user, cppcabrera is thinking code
16:43:18 <kgriffs> I think it would be better to make a random-ish/load test as a separate thing
16:43:24 <kgriffs> and factor out the common bits
16:43:32 <kgriffs> that can be shared with functional tests
16:43:33 <cppcabrera> kgriffs: +1
16:43:39 <flaper87> kgriffs: +1
16:43:44 <malini> we dont have random-ish tests today
16:43:46 <flaper87> but that's a third test suit for me
16:43:52 <kgriffs> malini: understood
16:44:09 <flaper87> unit, functional. random/load
16:44:21 <kgriffs> load is tricky
16:44:35 <flaper87> unit, functional, random :P
16:44:36 <flaper87> :D
16:44:44 <cppcabrera> :P
16:44:45 <kgriffs> I guess you would simulate kosher client behavior at the same time as naughty behavior
16:44:52 <kgriffs> (to kind of simulate the real world)
16:44:54 <malini> the point of disagreement is, do the functional tests  (within a test suite) need to be independent of each other ?
16:45:13 <zyuan> i want them to be independent
16:45:16 <cppcabrera> Yeah, I think it comes down to that.
16:45:18 <kgriffs> I don't think so, since by definition, they are dependent?
16:45:24 <kgriffs> or
16:45:37 <zyuan> functional tests are test
16:45:40 <zyuan> not simulation
16:45:41 <kgriffs> is it a matter of just refactoring so each test does just-in-time setup
16:45:44 <flaper87> zyuan: +!
16:45:46 <flaper87> zyuan: +1
16:45:59 <cppcabrera> The dependencies are established at the suite level, and the test cases perform operations.
16:46:05 <flaper87> kgriffs: yes, that's what needs to be done!
16:46:12 <malini> I think test suite shud be independent, no the tests themeselves
16:46:20 <flaper87> malini: +1
16:46:30 <malini> it makes sense for unit tests to be independent, not the functional tests
16:46:33 <cppcabrera> Using contextmanagers, JIT dep setup is easy. :D
16:46:52 <malini> cppcabrera: what are we gaining by doing that ?
16:47:14 <ametts> Seems like it would slow things down.
16:47:17 <zyuan> easy to add new tests
16:47:29 <malini> ametts: what wud slow things down ?
16:47:44 <ametts> Setting up and tearing down for each and every test in a suite.
16:47:48 <zyuan> more important: less time spend on find the reason of a very waired result
16:47:52 <cppcabrera> ametts: That setup/teardown is actually not very expensive. At the moment, the suite takes 10s to execute.
16:47:56 <zyuan> ametts: not every, some.
16:48:02 <cppcabrera> All 43 tests.
16:48:04 <zyuan> ametts: it's donw with cppcabrera's patch
16:48:21 <flaper87> cppcabrera: but when it comes to functional tests it might be a bit slower
16:48:24 <amitgandhi> cppcabrera: 10 sec curently with or without teardown?
16:48:37 <cppcabrera> 10s w/ teardown + setup
16:48:40 <flaper87> I mean, setUp marconi-server (db, whatsoever) run everything and then restart
16:48:41 <cppcabrera> Against mongodb
16:48:43 <cppcabrera> Live marconi
16:48:44 <amitgandhi> nice
16:48:51 <malini> & the current test suite takes 13 sec ,let me add
16:49:03 <zyuan> malini: if you worry about it becomes hard to write complex & long test, you can call test in another test
16:49:07 <kgriffs> malini: so, currently, each method within the class is numbered to ensure they run in a certain order, correct?
16:49:08 <cppcabrera> Performance is not an issue, IMO.
16:49:24 <malini> kgriffs: yes
16:49:24 <kgriffs> cppcabrera: +1
16:49:25 <zyuan> currently yes
16:49:25 <cppcabrera> My argument is for helping long-term maintainability.
16:49:27 <Sriram> yes they are ordered.
16:49:31 <kgriffs> let's stay on topic
16:49:39 <kgriffs> ok
16:49:48 <kgriffs> and in Robot they are ordered, right?
16:50:03 <malini> yes (& Robot is dead now :(  )
16:50:13 <malini> it's just nosetests
16:50:26 <kgriffs> ok, so this is the way you found to simulate what robot was doing
16:50:41 <malini> yes..
16:50:53 <kgriffs> I'd like to just make an a-priori assumption here that order matters in tests, and malini needs to be able to enforce it somehow
16:50:58 <malini> & really the numbering is just for the entire test suite setup
16:51:15 <malini> if 000 is the set up & 999 is the teardown for the suite
16:51:15 <kgriffs> also...
16:51:18 <flaper87> git lgo
16:51:20 <flaper87> ops
16:51:37 <malini> as long as those happen,  the tests are mostly covered
16:51:42 <flaper87> is the order really necessary ?
16:51:45 <kgriffs> the downside is that adding a new test can kind of be a pain
16:51:59 <kgriffs> since you have to shift the numbers
16:52:04 <zyuan> and it's hard find the reason beased on the result
16:52:07 <cppcabrera> 000 can be replaced by setUpClass, and 999 can be replaced by tearDownClass - those are respected bu python test runners.
16:52:07 <malini> kgriffs: no
16:52:15 <zyuan> because the result depends on unrelated running of tests
16:52:23 <malini> but setuoClass & teardown is just at test level
16:52:25 <flaper87> Why do we need that order ?
16:52:27 <malini> not for the test suite
16:52:34 <kgriffs> malini: good point
16:52:37 <cppcabrera> No. setUp is at test case level. setUpClass is at suite level. :)
16:52:38 <malini> tht''s the only reason we have 000 & 999
16:52:46 <kgriffs> setUpClass()
16:52:48 <malini> cppcabrera: no, I tries it
16:52:49 <cppcabrera> I verified that a few times working on my last project.
16:52:53 * kgriffs learns something everyday
16:52:59 <cppcabrera> @classmethod setUpClass
16:53:01 <malini> cppcabrea: tried*
16:53:07 <cppcabrera> @classmethod tearDownClass()
16:53:46 <kgriffs> hmmm
16:53:48 <malini> cppcabrera: IF you are right, we can get rid of the numbers in the suite
16:53:55 <malini> 'IF' ;)
16:53:57 <cppcabrera> :D
16:54:08 <zyuan> actually, i thing waht Robot does it to simulate a state machine
16:54:19 <malini> lets not talk Robot..its goe
16:54:23 <malini> gone*
16:54:25 <zyuan> but a sequencial numbering is just a linear action
16:54:31 <malini> gone loooong back!!
16:54:43 <flaper87> ok, 6 mins to go
16:54:47 <flaper87> lets fire some actions
16:54:49 <cppcabrera> Yup, let's wrap up.
16:54:50 <flaper87> :D
16:54:51 <zyuan> i suggest we to find some better appraoch to simulate a state machine
16:54:59 <zyuan> (without using the numbers)
16:55:06 <malini> I am adding it to the bp
16:55:06 <flaper87> So, what are the next steps w.r.t functional tests?
16:55:13 <cppcabrera> Suggested actions: discuss patch further, develop user testing strategy, restructure tests
16:55:17 <malini> right now..FREEZE!
16:55:25 <flaper87> Can we make them launch marconi-server and then make them run with tox ?
16:55:44 * flaper87 freezes
16:55:47 <cppcabrera> flaper87: Definitely.
16:55:50 * flaper87 is frozen
16:55:57 <cppcabrera> flaper87: Even launch a DB
16:55:58 <malini> we have too many changes in the api at the moment & the next two weeks are critical for us to wrap things up.
16:56:03 * cppcabrera defrosts flaper87
16:56:10 <ametts> malini: +1
16:56:13 <flaper87> cppcabrera: thaanks :D
16:56:15 <flaper87> malini: good point
16:56:21 <flaper87> didn't think about that
16:56:23 <kgriffs> +1
16:56:26 <malini> I wont suggest making any radical changes right now, but need to add it to the bp & address it in two weeks
16:56:33 <malini> after two weeks*
16:56:38 <kgriffs> + 1 zillion
16:56:39 <Sriram> +1
16:56:45 <flaper87> functional tests are important for incubation, though!
16:56:56 <flaper87> we have to have a good test suit
16:57:03 <malini> flaper87: is refactoring the tests important?
16:57:34 <flaper87> malini: it is important running them with tox so they can run in the gate
16:57:57 <malini> flaper87: ok.let's add tht to the things needed in 2 weeks
16:58:01 <flaper87> and having them, obviously (which most probably means updating them with the latest API)
16:58:07 <flaper87> cool
16:58:33 <malini> I can get it running in tox with the current tests (which is being updated for API changes)
16:58:36 <flaper87> #action malini to make functional tests run with tox
16:58:40 <cppcabrera> +1
16:58:55 <kgriffs> I think we can polish later. We will need to do some non-trivial tooling to get really elegant, robot-like behavior out of nose/unittest
16:59:26 <malini> I am going to add our discussion items to bp..we certainly have some great ideas to start hacking on in two weeks
16:59:38 <flaper87> #action malini cppcabrera to update the etherpad with new strategies and changes for functional tests
16:59:44 <flaper87> malini: and to the etherpad ^
16:59:46 <flaper87> pls
16:59:54 <flaper87> :D
16:59:56 <kgriffs> be sure to include a note about state machine for future discussion
16:59:57 <malini> now lets talk placement :D
16:59:59 <cppcabrera> Any lasSure thing. :)
17:00:04 <Sriram> +200
17:00:10 <flaper87> #action kgriffs to buy pop-tarts for the next summit
17:00:16 <flaper87> erm, 1 min left
17:00:19 <malini> kgriffs: sure
17:00:22 <cppcabrera> **the typos^^
17:00:23 <kgriffs> :D
17:00:33 <cppcabrera> I think we're out of time. :P
17:00:40 <Sriram> oh noes.
17:00:42 <flaper87> AWESOME MEETING!!!!
17:00:50 <flaper87> #endmeeting