17:02:36 <davidkranz> #startmeeting qa
17:02:37 <openstack> Meeting started Thu Nov  1 17:02:36 2012 UTC.  The chair is davidkranz. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:02:38 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:02:39 <openstack> The meeting name has been set to 'qa'
17:03:22 <davidkranz> I have uploaded a patch to turn on a daily full tempest job.
17:03:38 <davidkranz> The failures will be sent to the qa list.
17:03:50 <sdague> davidkranz: very cool
17:04:03 <ravkumar_hp> davidkranz: cool. is it automated mail ?
17:04:13 <davidkranz> ravkumar_hp: Yes.
17:04:45 <ravkumar_hp> davidkranz: It runs all tests including negative tests . right?
17:04:49 <davidkranz> Not sure what is going on with https://bugs.launchpad.net/tempest/+bug/1072841
17:04:51 <uvirtbot> Launchpad bug 1072841 in tempest "test_list_servers_negative failing one test" [High,Confirmed]
17:05:10 <davidkranz> ravkumar_hp: It runs the same tests as checkins to tempest at the moment.
17:05:32 <sdague> davidkranz: that seems like the right place to start, at least we can catch those breaks early.
17:05:35 <davidkranz> In the future it will probably run additional tests that are too slow for gating as well, such as fuzz testing, etc.
17:05:56 <sdague> jaypipes was around earlier, I'd be curious how that bug is going, and if he needs help on it
17:06:09 <sdague> #link https://bugs.launchpad.net/tempest/+bug/1072841
17:06:10 <uvirtbot> Launchpad bug 1072841 in tempest "test_list_servers_negative failing one test" [High,Confirmed]
17:07:13 <sdague> davidkranz: is there anything else you need on the full tempest runs in periodic, or do you think it's covered with the patch you sent into ci?
17:07:15 <davidkranz> sdague: That bug is blocking everything so I'm not sure why it hasn't gotten fixed.
17:07:30 <davidkranz> sdague: The patch should be all that is needed I think.
17:07:47 <davidkranz> https://review.openstack.org/#/c/15210/
17:07:57 <sdague> #link https://review.openstack.org/#/c/15210/
17:08:20 <jaypipes> crap, sorry guys... got lost in wiki  documentation...
17:08:33 <davidkranz> Obviously not much attention is being paid to puppet-lint :)
17:09:02 <sdague> davidkranz: I would up the timeout in that
17:09:15 <davidkranz> This was also just reported https://bugs.launchpad.net/tempest/+bug/1074039
17:09:17 <uvirtbot> Launchpad bug 1074039 in tempest "KeyError: "'novanetwork' in XML test" [Undecided,New]
17:09:25 <davidkranz> sdague: Timeoutin what?
17:09:34 <sdague> the machines these run on are slower than you might imagine, so 90 minutes is probably not enough
17:09:39 <sdague> I'll comment on the review
17:10:11 <davidkranz> sdague: Oh, that timeout. You're right.
17:10:56 <davidkranz> mnewby: Here?
17:11:00 <sdague> jaypipes: so on the test_list_servers bug, how's it going?
17:11:03 <sdague> need help?
17:11:06 <mnewby> ayo
17:12:03 <davidkranz> mnewby: No one responded to Dan W's email about resources quantum tests. Is the stuff you are working on related to that?
17:13:36 <jaypipes> sdague: well, unfortunately, I haven't been able to get a clean tempest run today and bcwaldon and I are disagreeing about the results of the requests in question. :) I'm running a new devstack/tempest run at the moment and will update the bug when I have more info
17:13:38 <mnewby> davidkranz: I think Dan was hoping for more than just my part-time work doing tempest.  I am still working on it, but I've been sidelined ensuring that quantum+ovs works on xen.
17:14:15 <mnewby> davidkranz: I think the hope was that Rackspace had resources to bring to bear on the issue.
17:14:39 <sdague> jaypipes: ok, well shout if you need help. That seems to be something we should get fixed up one way or another.
17:15:36 <jaypipes> sdague: will do...
17:15:39 <davidkranz> mnewby: OK. Not sure where to go with that. There is evidence that the model of developers doing stuff and then looking for "qa resources" elsewhere may not work in openstack.
17:15:45 <sdague> davidkranz: as additional topics for conversation, dansmith sent out the following to the openstack-dev list about the live upgrade session we did at summit, which has a number of potential QA blueprints. Would be curious on people's feedback - http://lists.openstack.org/pipermail/openstack-dev/2012-October/001969.html
17:16:01 <mnewby> davidkranz: I do intend to get the tempest change that adds a basic quantum test ready for real review this week, though.
17:16:28 <davidkranz> mnewby: Great. That is a good start.
17:16:51 <davidkranz> mnewby: Perhaps once that framework is in place it will be easier for folks in the know about quantum to add tests.
17:17:17 <sdague> davidkranz, mnewby: there is also a bunch of quantum support still landing in devstack, so I expect there will end up being a lot of iteration on that front for a bit
17:17:29 <mnewby> davidkranz: Hopefully.  And I'm planning on using tempest to validate any quantum changes I make going forward, so it should be maintable.
17:18:19 <davidkranz> Fundamentally, writing a tempest test takes a lot of the same knowledge as writing a unit test.
17:18:28 <sdague> oh, right a bunch of that is from mnewby, I need to connect nicks more often :)
17:18:39 <davidkranz> So I think the eligible pool for quantum is a lot smaller than other projects.
17:18:45 <mnewby> sdague: Definitely.  The devstack changes are immediately useful, but won't test a non-devstack config, so I'm going to make sure tempest can do whatever the devstack exercise scripts do.
17:19:19 <davidkranz> I am absorbing ceilometer and will contribute tempest tests as soon as Doug says the API is stable.
17:19:43 <davidkranz> I will reply to Dan Wendlandt.
17:20:50 <davidkranz> I also exchanged email with the mirantis folks about getting there tempest stuff into the main branch but have not seen anything yet.
17:21:08 <davidkranz> Anything else?
17:21:30 <ravkumar_hp> davidkranz: parallelization solution
17:21:38 <ravkumar_hp> any news from daryl ?
17:22:31 <sdague> davidkranz: I'm curious about using blueprints to track some of the big items in the grizzly qa cycle
17:22:47 <davidkranz> ravkumar_hp: Haven't heard anything from him except the message he sent 1/2 hour ago.
17:23:08 <sdague> there were definitely a few captured out of the nova sessions, and the full tempest gate session
17:23:47 <davidkranz> sdague: We just need to create them.
17:24:00 <sdague> should I just start writing up things and propose them in the tool, or get them on a mailing list, or what? we'll be signing up to do a number of them, but want to make sure the work is visible and coordinated (so not duping anyone)
17:24:41 <davidkranz> sdague: How about sending a list of titles to the mailing list, or point to an etherpad first?
17:24:53 <sdague> davidkranz: sure, will do
17:25:12 <davidkranz> sdague: Thanks. If you are planning on working on one you can just put it up immediately as well.
17:25:50 <sdague> have we considered getting rid of the qa list in favor of just driving this traffic on the main dev list? It might expose it to more folks, and maybe draw in a few more volunteers
17:26:22 <sdague> opinions from anyone on that?
17:26:38 <davidkranz> sdague: I don't think we should get rid of the qa list but it would be fine to put this on the dev list as far as I can see.
17:26:51 <sdague> davidkranz: ok, I'll cross post
17:27:26 <sdague> I'll try to get a consolidated view of what I saw in various meetings, and try to get that discussion running
17:27:33 <davidkranz> sdague: Great!
17:29:16 <davidkranz> sdague: Just sumitted patch for puppet change.
17:29:22 <davidkranz> ANything else from any one?
17:29:40 <sdague> not i
17:29:54 <davidkranz> 30 seconds and counting...
17:30:24 <davidkranz> OK, see you all soon.
17:30:29 <davidkranz> #endmeeting