17:00:47 <mtreinish> #startmeeting qa
17:00:52 <openstack> Meeting started Thu Oct 23 17:00:47 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:53 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:55 <openstack> The meeting name has been set to 'qa'
17:01:01 <mtreinish> hi, who's here today?
17:01:11 <andreaf> o/
17:01:15 <afazekas> o/
17:01:27 <dkranz> o/
17:01:30 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_October_23rd_2014_.281700_UTC.29
17:01:35 <mtreinish> ^^^ Today's agenda
17:02:34 <mtreinish> ok let's get started, maybe more people will trickle in later
17:02:41 <mtreinish> #topic Blueprints
17:03:11 <mtreinish> So I still need to go through the list of open juno bps and move them to kilo
17:03:22 <mtreinish> but that's just administrative nothing will change with those
17:03:43 <dkranz> mtreinish: I closed the clients-check-success because it is superseded by clients-return-one-value
17:04:09 <mtreinish> dkranz: ok, I was going to ask you for the relationship between the specs as a review comment
17:04:27 <mtreinish> dkranz: do you have a change up for dropping the success spec?
17:04:42 <dkranz> mtreinish: I just closed it as Implemented
17:05:15 <mtreinish> dkranz: so we're going to call it implemented, then I guess we should move it the implemented dir
17:05:27 <dkranz> mtreinish: It was not completely finished but there is no point in doing that for the remaining few clients separately from the return-one-value
17:05:51 <dkranz> mtreinish: Yes, we should move it or discard it.
17:06:20 <mtreinish> dkranz: well that's what I was asking about :) If the status in lp is implemented I say let's just move it
17:06:30 <dkranz> mtreinish: works for me
17:06:45 <dkranz> mtreinish: It was a weird case because it was superseded after being mostly implemented :)
17:06:57 <mtreinish> dkranz: ok I'll add it to my pending change doing that
17:07:21 <mtreinish> dkranz: and I'll review the new spec this afternoon (sorry I haven't gotten to it yet)
17:07:37 <dkranz> mtreinish: ok, great
17:08:00 <mtreinish> ok does anyone have any open blueprints that they would like to update status on or discuss?
17:08:28 <andreaf> mtreinish: resource-cleanup
17:08:45 <mtreinish> andreaf: oh, yeah I need to review the framework patch
17:08:48 <mtreinish> andreaf: go ahead
17:08:49 <andreaf> mtreinish: I'm waiting for feedback on the proposed framework before I can continue to part 2
17:09:04 <andreaf> mtreinish: yes that's it
17:09:16 <mtreinish> andreaf: do you have a link for the minutes
17:09:26 <andreaf> mtreinish: coming...
17:09:39 <andreaf> #link https://review.openstack.org/#/c/115353/
17:09:49 <mtreinish> cool, thanks
17:10:21 <andreaf> Also the ssh-auth one, which is actually a spec, not yet a bp, but I'd love to get input on that - perhaps it's something we could discuss as well on Friday at the summit
17:10:53 <mtreinish> andreaf: oh, yeah we punted on that until kilo (which I guess it is now :) )
17:11:08 <mtreinish> andreaf: if you've got another link...
17:11:25 <andreaf> #link https://review.openstack.org/94741
17:11:28 <dkranz> andreaf: About the test-accounts, now that it is finished, is the plan to make using them the default in the gate?
17:11:37 <mtreinish> dkranz: it's not ready for that yet
17:11:47 <mtreinish> there still some missing functionality before we could gate on it
17:12:03 <andreaf> dkranz, mtreinish: indeed - but I plan to prepare a spec for what's missing
17:12:11 <mtreinish> after that's done it's definitely worth discussing
17:12:14 <dkranz> mtreinish: ok, is it ready to use in real deployments?
17:12:30 <andreaf> so we can keep the code exercised
17:12:30 <dkranz> mtreinish: I would like to try but not if it is known to have major issues
17:13:03 <mtreinish> dkranz: it has the same networking limitations that non-isolated did. Which can get confusing for multitenant users
17:13:28 <mtreinish> but for the non-admin auth side it should work fine as long as you're not relying on networking
17:13:39 <andreaf> dkranz: and also if you run in parallel with pre-provisioned accounts you'll skip admin tests
17:13:40 <mtreinish> (admin being the other reason we can't use it in the gate yet)
17:14:01 <dkranz> mtreinish: ok, I thought that if each user had their own network it would be ok
17:14:22 <dkranz> mtreinish: anyway, if there are issues I will raise them outside this meeting
17:14:29 <mtreinish> dkranz: yeah if the default is configured sanely it should work, it's just for tests that use fip it might not
17:14:38 <andreaf> there's a assumption that all user will belong to the same network
17:15:02 <dkranz> andreaf: ok. That is the same issue as with nova-network vlan which also does not work
17:15:20 <dkranz> We should really fix that
17:15:27 <mtreinish> that'll be part of the second half
17:15:35 <dkranz> mtreinish: ok, fair enough
17:15:51 <mtreinish> allowing the network to be specified in the yaml for each cred set
17:15:52 <andreaf> dkranz: yes it shouldn't be to difficult in fact, only we wanted to start with something not too complex at once
17:16:51 <mtreinish> ok, moving on I have an update on the tempest-lib bp
17:17:19 <mtreinish> finally after about 2 weeks of fighting pypi, requirements, and other issues tempest-lib usage is ready to land in tempest
17:17:29 <mtreinish> #link https://review.openstack.org/117649
17:17:38 <mtreinish> #link https://review.openstack.org/122469
17:18:21 <mtreinish> which switch the cli tests to use tempest-lib and switches subunit-trace to be the tempest-lib copy
17:19:46 <mtreinish> on that note I also pushed out the tempest-2 tag to signal the opening of kilo dev
17:20:26 <mtreinish> although I guess that's really unrelated :)
17:20:49 <mtreinish> ok if there aren't any other bp updates I guess we can move on
17:21:43 <mtreinish> #topic Devstack
17:21:49 <mtreinish> dtroyer: are you around?
17:22:07 <mtreinish> the only recent devstack thin I'm aware of was that stable/juno was cut
17:23:32 <mtreinish> ok well if dtroyer shows up later we can come back to devstack
17:23:43 <mtreinish> #topic Grenade
17:24:18 <mtreinish> so the big update here is thanks to clarkb we've migrated grenade to do juno -> master and icehouse->juno
17:24:53 <mtreinish> although during the period we were gating on icehouse->master nova managed to land an upgrade breaking change
17:25:19 * dtroyer runs in at the sound of his name
17:25:42 <mtreinish> dtroyer: heh, are there any update on devstack or grenade?
17:27:13 <dtroyer> devstack also has a stable./juno cut.  otherwise its mostly shaping stuff up for summit.  I have a couple of topic we can talk about if nothing more immediate comes up in other sessions
17:28:05 <dtroyer> I'm also going to get the bug/bp launchpad stuff under control Real Soon Now…
17:28:29 <mtreinish> dtroyer: oh yeah I still need to draft the ML post for devstack and grenade qa-specs usage
17:29:38 <mtreinish> ok if there's nothing else on devstack and grenade I guess we'll move on
17:29:49 <dtroyer> thats it, thanks
17:30:11 <mtreinish> #topic Bugs
17:30:21 <mtreinish> #link https://etherpad.openstack.org/p/Tempest-bug-report
17:30:43 <mtreinish> I think those numbers are a bit out of date
17:30:57 <mtreinish> but it looks like we're finally close to 0 new bugs
17:31:39 <mtreinish> we still have ~190 open bugs but getting the incoming triage under control is a good first step
17:32:14 <mtreinish> next week's triage volunteer will be gmann: https://etherpad.openstack.org/p/qa-bug-triage-rotation
17:32:35 <mtreinish> does anyone have anything else to discuss about bugs?
17:33:59 <mtreinish> ok then I guess we'll move on
17:34:08 <mtreinish> #topic Critical Reviews
17:34:30 <mtreinish> does anyone have any reviews they'd like to bring up?
17:36:16 <mtreinish> wow, must be a first no reviews that need extra attention :)
17:37:06 <mtreinish> ok, then I guess I'll open the floor
17:37:14 <mtreinish> #topic Open Discussion
17:37:29 <mtreinish> does anyone have a topic they'd like to discuss that wasn't on the agenda?
17:38:18 <dkranz> mtreinish: My cross-project proposal was discussed positively in the tc meeting.
17:38:20 <andreaf> mtreinish: do you still have open slots at the summit?
17:38:28 <mtreinish> andreaf: 1
17:38:30 <dkranz> mtreinish: The one about "impact of moving func tests to projects"
17:38:46 <mtreinish> assuming dkranz's topic gets picked up in the cross-project track
17:39:01 <dkranz> mtreinish: Actually it was mentioned that that was the only test-related proposal.
17:39:35 <mtreinish> if it does and there isn't another proposal for the slot, I'll give the slot to infra or someone else
17:39:41 <mtreinish> dkranz: hmm, that's surprising
17:40:09 <andreaf> mtreinish: I was wondering if discussing what level of validation we want in API tests is worth a session - i.e. do we want all tests to check the created server have proper networking, that volumes attached can actually be seen attached to a VM etc
17:40:43 <mtreinish> andreaf: basically a session on your ssh auth spec? :)
17:40:44 <dkranz> andreaf: I think that is part of the "what is tempest after moving fun tests to projects discussion"
17:41:02 <dkranz> mtreinish: I think it is more general than that.
17:41:43 <andreaf> mtreinish, dkranz: well yes the spec is about how to do it, but I wanted to discuss if / where we want to do it
17:41:47 <dkranz> mtreinish: I think we should double up the "future of tempest" slot rather than giving it away
17:42:13 <dkranz> andreaf: right, that was my point I think
17:42:14 <mtreinish> 80min is really a lot of time for one topic
17:42:28 <mtreinish> I'd rather just use part of friday if we can't fit it all into 40
17:42:34 <andreaf> there is this constant problem that tests doing ssh to a VM is less reliable - they are more complex granted but maybe it's also because we do not test it enough in the gate
17:43:01 <dkranz> mtreinish: I think "future of tempest" with folks from outside our group could easily take more than 40 min :)
17:44:08 <dkranz> andreaf: The problem is that it is flaky
17:44:31 <dkranz> andreaf: If ssh was expected to be reliable I would be all for doing it always
17:45:11 <dkranz> I had one more topic
17:45:31 <andreaf> dkranz:  there may be other ways to do deeper validation which do not rely on ssh, perhaps custom image logging things on console would be another option
17:45:39 <andreaf> dkranz, go ahead
17:45:43 <dkranz> We should really have many more periodic jobs running various "stress" tests
17:46:18 <mtreinish> dkranz: as a summit topic or for discussion now?
17:46:35 <dkranz> I meant discussion now
17:46:47 <dkranz> It is just a matter of who will watch them.
17:46:51 <mtreinish> ok, because andreaf has a summit session that too :)
17:46:59 <dkranz> oh, I didn't see that
17:47:08 <dkranz> mtreinish: Is it on the etherpad?
17:47:32 <mtreinish> dkranz: QA and CI after merge
17:47:42 <andreaf> dkranz, I have a session about post merge QA
17:47:56 <mtreinish> it's not exactly about stress tests, but the same basic issues will be discussed
17:48:01 <dkranz> mtreinish: ok, we can discuss it then.
17:48:16 <mtreinish> dkranz: what we really need to have is a dashboard view for the results
17:48:34 <andreaf> mtreinish: yes a dashboard would help
17:48:36 <mtreinish> which shows the job history and trends over time
17:48:44 <dkranz> mtreinish: Yes, and a rotation of people to watch
17:48:57 <mtreinish> the problem is someone needs to step up to make it
17:49:13 <andreaf> mtreinish: but it could even  something like if a job displays signs of definite bit-rot, start voting -1 or patches or so
17:49:58 <mtreinish> andreaf: I think that would be much harder to do
17:50:58 <andreaf> mtreinish: it would be harder to implement? Yes but it's something that we need I think, else the gate won't scale in future
17:51:48 <andreaf> mtreinish: ok we discuss this at the summit
17:52:07 <mtreinish> andreaf: both to implement and it also very different from what we say gating is now
17:52:10 <mtreinish> andreaf: sure
17:52:46 <mtreinish> dkranz: anyway I think it would be fine to add more varied stress jobs now we can figure out the reporting/watching side at summit
17:54:19 <mtreinish> ok if there's nothing else I guess we'll end here
17:54:22 <mtreinish> thanks everyone
17:54:35 <mtreinish> #endmeeting