17:03:29 <sdague> #startmeeting qa
17:03:30 <openstack> Meeting started Thu May  9 17:03:29 2013 UTC.  The chair is sdague. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:03:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:03:33 <openstack> The meeting name has been set to 'qa'
17:04:07 <sdague> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_May_9_2013_meeting
17:04:14 <sdague> ok, proposed agenda for today, lets get rolling
17:04:31 <sdague> #topic Blueprint check in for everyone with H1 blueprints
17:04:42 <sdague> #link https://launchpad.net/tempest/+milestone/havana-1
17:05:31 <sdague> so we have a few of those blueprints with a status of Unknown, we're half way through H1, anyone have updates to share?
17:06:11 <sdague> guess not :)
17:06:39 <ravikumar_hp> out of 15 , 2 blueprints are already implemented
17:06:39 <sdague> the directory restructure bits are underway
17:07:05 <sdague> #link https://review.openstack.org/#/q/topic:bp/tempest-repo-restructure,n,z
17:07:12 <sdague> would appreciate some reviews so we can keep those moving
17:07:23 <ravikumar_hp> sdague: I proposed some changes. can you address that?
17:07:33 <sdague> ravikumar_hp: proposed where?
17:07:45 <ravikumar_hp> as review for patch 3/4
17:08:03 <ravikumar_hp> https://review.openstack.org/#/c/28453/
17:08:55 <sdague> ok, so scenario tests may be part of the gate
17:09:15 <sdague> honestly, right now, except for the stress directory, we're not kicking anything out of the gate by default
17:09:28 <sdague> this is about making what we have in tree already clearer
17:09:39 <sdague> so it's easier for people to extend and contribute to
17:09:59 <sdague> it's also subject to evolution over time, but we needed a starting place
17:10:26 <afazekas> I wonder can we extend the stress tests to collect performance data ?
17:10:32 <sdague> afazekas: probably
17:10:40 <sdague> that would be a good exercise
17:11:27 <sdague> ok, but the series needs at least another +2 to move it forward, so that would be appreciated if people are good with it
17:11:38 <afazekas> Probably I can add some system tap scripts to the tools/ folder for tracing same kernel event , for example for cpu usage accounting
17:11:41 <sdague> any other blueprint updates from the list?
17:11:51 <sdague> otherwise I'll jump to next topic
17:11:57 <ravikumar_hp> ok
17:12:11 <sdague> #topic Obsoleting blueprints in new/unknown with no series nomination
17:12:36 <sdague> ok, so we said that we'd clean up the blueprints, and drop everything that people didn't series nominate (with a milestone)
17:12:44 <sdague> I'm going to hold folks to that tomorrow
17:13:13 <sdague> https://blueprints.launchpad.net/tempest is the whole list, I expect it to shrink a bunch
17:13:34 <sdague> so if there is a blueprint you are actually going to champion, please propose for a havana milestone
17:14:14 <sdague> there are also way too many Quantum blueprints in there, do we have a volunteer to consolodate them a little?
17:14:48 <mlavalle> sdague: I can do that
17:14:54 <sdague> mlavalle: thank you!
17:15:12 <sdague> #action mlavalle to consolidate tempest blueprints
17:15:16 <sdague> #action mlavalle to consolidate tempest quantum blueprints
17:15:32 <sdague> mlavalle: ping me in -qa if there is anything you need to get that done
17:15:43 <mlavalle> sdague: ok
17:16:04 <sdague> last scheduled topic
17:16:10 <sdague> #topic Critical reviews tied to blueprints
17:16:29 <sdague> now is fair game for any reviews you want to pimp that are tied to a blueprint :)
17:16:39 <sdague> or a bug
17:16:41 <davidkranz> stress test add/remove
17:16:50 <sdague> davidkranz: cool, url?
17:17:13 <davidkranz> https://review.openstack.org/#/c/28554/
17:17:30 <davidkranz> https://review.openstack.org/#/c/28553/
17:17:32 <sdague> also can we get a blueprint for it? just to help us keep track of things. As one of our goals out of summit was to be more organized :)
17:17:45 <davidkranz> https://blueprints.launchpad.net/tempest/+spec/stress-tests
17:17:56 <afazekas> 'There may be cases where multiple actions interact with each other'
17:17:59 <sdague> sweet
17:18:19 <afazekas> davidkranz: how the action will be able to interact with each other ?
17:18:29 <sdague> davidkranz: if you put bp:stress-tests in the commit messages, they'll auto link back to that as well. Useful for keeping track
17:18:45 <sdague> I'll take a look at the stress tests again this afternoon
17:18:53 <davidkranz> sdague: Thx
17:18:59 <sdague> any others from folks?
17:19:37 <sdague> ok, we'll assume not
17:19:42 <davidkranz> afazekas: I don't have a detailed scenario.
17:19:47 <sdague> #topic open discussion
17:19:58 <davidkranz> afazekas: The point was just that when the killing starts, the test is already complete.
17:20:15 <psedlak> are there any rules for contributing or backporting to tempest/stable-*?
17:20:35 <sdague> psedlak: my feeling is that we should be the same as other projects
17:20:39 <sdague> so
17:20:44 <sdague> 1) it needs a bug
17:20:52 <sdague> 2) it needs to be fixed in master first
17:20:52 <davidkranz> sdague: Gotta go again
17:21:09 <sdague> 3) the stable patch should be a cherry pick of master
17:21:40 <sdague> we are definitely only fixing bugs in stable, not adding new test cases
17:22:18 <psedlak> sdague: so it's not possible to get new test cases covering some stable feature added to master and backported to stable?
17:22:48 <sdague> I don't think so, as it would have implications on the gate
17:22:59 <sdague> I think we can take it as a case by case basis
17:23:09 <sdague> but my instinct would be not to do that
17:23:11 <psedlak> sdague: what kind of 'implications'?
17:23:35 <sdague> we are changing the criteria for the stable gate
17:23:47 <afazekas> is jaypipes here ?
17:24:04 <psedlak> sdague: like not covering thinks that are actually already in use/production ... that should not break things much/at all ...
17:24:22 <sdague> psedlak: it's always fair to propose a change like that, and see what folks think
17:24:39 <sdague> honestly, until recently there wasn't any backport activity to tempest
17:24:51 <sdague> so it's sort of a new problem for us to figure out as we go
17:25:03 <sdague> definitely needs to be artifact tracked though
17:25:25 <maurosr> I have one point/question to you guys
17:25:28 <maurosr> sdague and guys: cyeoh and I were talking how we're going to test nova-api-v3 in tempest during our port process to v3 tree... there are other projects with 2 apis right? how is the process to test those cases?
17:25:37 <psedlak> sdague: ok, thanks
17:25:45 <afazekas> sdague: I am planing to backport tests to the folsom and grizzly branches
17:25:48 <sdague> maurosr: we just have both versions in tree
17:26:12 <sdague> afazekas: folsom? really?
17:26:23 <afazekas> yes :)
17:26:27 <sdague> ok :)
17:26:40 <sdague> well lets start with some detailed bugs
17:26:53 <maurosr> sdague: so no problem duplicating the number of tests then?
17:27:18 <sdague> maurosr: for now, no. but we're going to need to get a little stricter on what we are accepting in the gate
17:27:27 <sdague> because we're at 45m right now
17:27:44 <sdague> so it may be that not all of them get run in the gate, and we go to periodic runs
17:27:52 <sdague> especially for experimental APIs
17:28:01 <maurosr> right, make sense
17:28:17 <maurosr> sdague: do I need to write a bp about it?
17:28:35 <sdague> for nova v3 tests? yes please
17:28:42 <sdague> it helps keep us organized
17:28:46 <maurosr> ok, thanks
17:29:03 <sdague> other discussion topics?
17:29:52 <sdague> going once...
17:29:53 <afazekas> We should document somehow the existing test cases
17:30:08 <sdague> afazekas: sure, can you explain more what you mean?
17:30:42 <afazekas> For new camers it is difficult to see what is not tested by tempest
17:31:01 <sdague> do you have a suggestion about how we could do that well?
17:31:01 <afazekas> The RTFS is not a good answer for everyone
17:31:16 <sdague> I definitely agree with you, I've just not come up with a good idea there
17:31:38 <afazekas> sdague: I think we should use the wiki or an rst files in the repository
17:32:02 <sdague> rst file in the repo sounds like a good idea, then when people make commits they could take stuff off the list
17:32:05 <afazekas> I do not have exect plans, may be we should discuss it on the next meeting
17:32:14 <sdague> afazekas: sure
17:32:28 <sdague> you want to take an action to come up with a proposal to discuss for the next meeting?
17:32:30 <afazekas> sdague: cool
17:32:40 <sdague> or we can do it on the mailing list
17:32:46 <sdague> I think it's a good goal
17:33:11 <afazekas> We should start in the ML
17:33:19 <ravikumar_hp> yes
17:33:31 <sdague> ok, lets do that. I'll give you the todo to start the thread
17:33:50 <dwalleck> +1
17:33:52 <sdague> #action afazekas to start mailing list thread on documenting test cases we need for new contributors
17:33:57 <ravikumar_hp> as routine - Email on Thursday after this meeting ,and due by Wednesday
17:34:23 <sdague> any other topics?
17:34:56 <sdague> going once...
17:35:07 <sdague> going twice...
17:35:14 <sdague> ok, lets call it
17:35:17 <sdague> #endmeeting