22:01:30 <mtreinish> #startmeeting qa
22:01:30 <openstack> Meeting started Thu Aug 21 22:01:30 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:01:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:01:33 <openstack> The meeting name has been set to 'qa'
22:01:39 <mtreinish> hi who's here today?
22:01:48 <dkranz> hi
22:01:49 <masayukig> hi
22:02:03 <mtreinish> heh, it'll be a nice quick meeting today :)
22:02:05 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_August_21_2014_.282200_UTC.29
22:02:07 <asselin> hi
22:02:09 <mtreinish> ^^^ Today's agenda
22:02:25 <andreaf> o/
22:02:41 <mtreinish> ok let's get started, maybe more people will trickle in later
22:02:54 <mtreinish> #topic Devstack now part of the QA program
22:03:06 <mtreinish> I just wanted to make an announcement on this
22:03:14 <mtreinish> dtroyer's governance change landed earlier this week
22:03:26 <mtreinish> so the devstack repo is now officially part of the qa program
22:03:38 <mtreinish> this was announced on the ML a couple of weeks ago
22:03:40 <dkranz> mtreinish: does that really change anything?
22:03:43 <mtreinish> no
22:03:48 <andreaf> yay welcome dtroyer and devstack team to the QA program
22:03:51 <mtreinish> but it felt news worthy :)
22:03:57 <dkranz> mtreinish: sure :)
22:04:15 <mtreinish> ok anything else on this?
22:04:22 <masayukig> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/041731.html
22:04:32 <masayukig> ^^^?
22:04:43 <mtreinish> masayukig: yep that was the ML post
22:05:03 <mtreinish> ok let's move on
22:05:03 <masayukig> ok, thanks :)
22:05:20 <mtreinish> #topic Starting a QA liaison system (like oslo) (mtreinish)
22:05:39 <mtreinish> so this was another topic that came up briefly on the ML
22:05:51 <mtreinish> as a branch on the giant future of the integrated release thread
22:06:14 <mtreinish> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/042938.html
22:06:33 <mtreinish> I'm thinking setting up a similar liaison program to oslo might make sense for qa
22:06:53 <mtreinish> I was hoping to get some opinions on it before I made an announcement on the ML
22:07:24 <mtreinish> the concept is there is a dedicated person who will attend the qa meeting and champion project specific things
22:07:28 <dkranz> mtreinish: you mean to liason with domain experts in projects around tempest?
22:08:09 <mtreinish> dkranz: yeah so a nova-dev volunteers and will be a goto point for anything overlapping between tempest and nova
22:08:16 <mtreinish> or grenade and tempest
22:08:18 <mtreinish> etc
22:08:23 <dkranz> mtreinish: doesn't moving functional tests to projects reduce the need for that. Not saying it is a bad idea.
22:08:25 <mtreinish> wow can't type
22:08:44 <mtreinish> dkranz: well only to a certain extent
22:09:06 <mtreinish> and not necessarily, since functional testing haven't really been spun up in force
22:09:09 <mtreinish> yet
22:09:11 <dkranz> mtreinish: right, but would we have 15 new people at our meeting doingnot so much?
22:09:37 <mtreinish> well hopefully they'd be involved
22:10:07 <dkranz> mtreinish: I have advocated for this in the past.
22:10:18 <andreaf> is it restricted to meetings?
22:10:18 <mtreinish> my thoughts were this would be especially useful with the tempest lib migration
22:10:25 <mtreinish> andreaf: no it's more than just meetings
22:10:46 <mtreinish> #link https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons
22:10:51 <dkranz> mtreinish: ok, then I have no real concern
22:10:51 <mtreinish> that's the oslo page for them
22:11:12 <andreaf> ok because it would be useful if I'm reviewing a cinder test patch to have a contact on cinder side to perhaps jump him and clarify a few things
22:11:27 <dkranz> andreaf: or ask for a review
22:12:00 <dkranz> Until now the message we got was "Add Core reviewers" which doesn't really work
22:12:19 <mtreinish> andreaf, dkranz: well normally I just bug jgriffith on irc for those :)
22:12:48 <mtreinish> but, yeah that's the idea having a dedicated person who will help us for other project use cases
22:12:57 <gmann_> mtreinish: so in case of review, will it be compulsory to have +1/2 from them before approval? or just their opinion would work may be through IRC etc
22:13:14 <mtreinish> gmann_: no, it wouldn't be compulsory
22:13:32 <mtreinish> gmann_: it's just about having a known person to contact
22:13:58 <mtreinish> gmann_: mkoderer is signed up right now as the oslo liaison for tempest so it might make sense to ask his opinion on this too
22:14:27 <andreaf> it seems like a good idea to me
22:14:40 <mtreinish> andreaf: yeah that was what I thought
22:14:44 <dkranz> I agree
22:14:50 <masayukig> agree
22:14:56 <mtreinish> anyway I'll take it to the ML
22:15:01 <mtreinish> and we'll go from there
22:15:21 <mtreinish> but I think starting at the end of Juno/start of Kilo we'll probably have this in place
22:15:45 <mtreinish> ok if there isn't anything else on this let's move on
22:15:47 <dkranz> mtreinish: yes, we can corner people at the summit :)
22:16:03 <mtreinish> dkranz: heh, yeah
22:16:09 <mtreinish> #topic Can we start using smoke tag for its intended purpose? (dkranz)
22:16:28 <mtreinish> dkranz: the floor is yours
22:16:29 <dkranz> mtreinish: So I just wanted to know if there is any blocker on this ow
22:16:31 <dkranz> now
22:16:36 <mtreinish> well your sub bullet
22:16:46 <mtreinish> because master is still used for icehouse
22:16:54 <dkranz> The only issue I see is the icehouse neutron runs that still use smoke
22:17:29 <mtreinish> but if we switch that job to just run network
22:17:33 <mtreinish> I think we can get around it
22:17:43 <mtreinish> my only concern is knowing what is valid for smoke and not
22:17:51 <dkranz> mtreinish: that would be good
22:18:04 <dkranz> mtreinish: you mean how we decide to label something as smoke?
22:18:08 <mtreinish> dkranz: yeah
22:18:16 <mtreinish> I think having a db of run history will be useful for figuring that out
22:18:40 <mtreinish> which means I need to get https://review.openstack.org/108003 working soon
22:18:45 <dkranz> mtreinish: Yeah. One idea was to have just one test for each specific api be smoke, and no negative.
22:19:31 <mtreinish> dkranz: yeah I was imagining a more metric based approach tests that fail above a certain freq would be good smoke tests (because they're testing common fail points)
22:19:44 <mtreinish> dkranz: because picking one test still has the issue of which test
22:20:16 <dkranz> mtreinish: Sure but many will not have failed much, except for unrelated reasons
22:20:33 <dkranz> mtreinish: So I'm not sure how much metric info will be available
22:21:20 <mtreinish> dkranz: it's those unrelated reasons that make it a good test, because they're tickling a common issue
22:21:35 <mtreinish> dkranz: well that was the long running db thing I'm working on is to collect and aggregate that data over a relase or 2
22:21:42 <mtreinish> but yeah it's still a long term thing
22:21:48 <dkranz> mtreinish: Well, for many fails it is just trying to create a server
22:21:55 <mtreinish> so we'll probably need to figure out criteria in the short term
22:21:58 <dkranz> mtreinish: sure
22:22:17 <dkranz> mtreinish: I'm throwing out the "one per api" as a starting point
22:22:18 <mtreinish> dkranz: yeah but over a long time the incidental failures can be filtered out
22:22:25 <dkranz> mtreinish: Because we will want at least that
22:23:03 <dkranz> mtreinish: so what exactly has to be done to deal with the icehouse issue?
22:23:33 <mtreinish> dkranz: we need to add a new job definition, verify that it runs the same tests, and switch config to use it
22:23:38 <dkranz> mtreinish: You want to change the icehouse neutron job to just run network tests?
22:24:00 <mtreinish> dkranz: yeah just use the regex 'network' and exclude slow should do the trick
22:24:39 <dkranz> mtreinish: but that won't run the same tests as the current job whic hruns smoke from all services
22:25:24 <mtreinish> dkranz: then we need to add the attr network-legacy-test or something like that which'll be picked up by the regex
22:25:40 <dkranz> mtreinish: anyway, we can discuss this outside the meeting
22:25:41 <mtreinish> or we could do network and new attr tag
22:25:44 <mtreinish> dkranz: yeah
22:25:51 <mtreinish> ok is there anything else on this topic?
22:25:55 <dkranz> no
22:26:00 <mtreinish> ok then let's move on
22:26:14 <mtreinish> #topic Specs Review
22:26:22 <mtreinish> does anyone have a spec review to bring up
22:26:38 <mtreinish> andreaf: sorry I haven't had a chance to look at the ssh auth one in detail yet
22:26:41 <andreaf> #link https://review.openstack.org/#/c/94741/
22:26:50 <mtreinish> andreaf: :)
22:26:59 <andreaf> mtreinish, ok that's the only one I had in mind
22:27:49 <mtreinish> andreaf: ok, I'll try to take a look at it soon, its a good one to work on
22:27:59 <mtreinish> but at this point I don't see it getting finished until kilo
22:28:22 <mtreinish> I feel we've got enough open dev work for juno at this point
22:28:28 <andreaf> mtreinish, probably not - other bp are taking most of the time
22:28:50 <mtreinish> ok does anyone have any other specs to bring up?
22:29:02 <andreaf> mtreinish, but it's a tricky one so it would be good to get some ideas before the summit
22:29:20 <mtreinish> andreaf: heh, yeah I'll definitely look at it before summit...
22:29:46 <mtreinish> ok let's move on then
22:29:50 <mtreinish> #topic Blueprints
22:29:56 <mtreinish> #link https://blueprints.launchpad.net/tempest/
22:29:56 <dkranz> https://blueprints.launchpad.net/tempest/+spec/client-checks-success
22:30:06 <mtreinish> dkranz: ok how's that going?
22:30:23 <dkranz> There are some patches up we should look at quickly because they suffer from rebases
22:30:38 <mtreinish> dkranz: link?
22:30:49 <dkranz> Also, we chould increase the priority because this is needed before clients can be moved to tempest-lib
22:31:35 <dkranz> mtreinish: https://review.openstack.org/#/q/topic:bp/client-checks-success,n,z
22:31:47 <mtreinish> #link https://review.openstack.org/#/q/topic:bp/client-checks-success,n,z
22:32:22 <dkranz> I gave -1 to the top one
22:32:32 <mtreinish> dkranz: as for the prio I viewed this as more of a best effort thing which is why I made it low
22:32:59 <dkranz> Sure. Just noting that the tempest-lib thing needs it
22:33:00 <mtreinish> mostly because we can update things without really changing funcitonality
22:33:17 <mtreinish> dkranz: does it? because we still need to return the headers from the lib
22:33:23 <mtreinish> it would just add an extra assert
22:33:41 <mtreinish> and if we do it in pieces we can do it right before I guess too
22:33:47 <dkranz> sure
22:34:04 <mtreinish> dkranz: but I'm fine bumping it if you think it's warranted
22:34:12 <dkranz> These patches don't stop the headers from being returned but they are ignored
22:34:12 <mtreinish> set it at whatever you feel it should be
22:34:16 <dkranz> mtreinish: ok
22:34:52 <mtreinish> so we've still got a bunch of open bps for the cycle
22:34:55 <dkranz> mtreinish: That's it. Just a request for reviews on these
22:35:00 <mtreinish> and not so much time
22:35:08 <asselin> will bp's be accepted after J3?
22:35:18 <dkranz> mtreinish: I have not had a chance to look at the cleanup patch
22:35:27 <asselin> bp implementation I mean.
22:35:29 <mtreinish> asselin: yes
22:35:29 <jgriffith> andreaf: BTW, always feel free to just grab me on IRC as mtreinish suggested :)
22:35:48 <andreaf> jgriffith, will do thanks :)
22:35:50 <mtreinish> jgriffith: heh, I was just giving you a hard time :)
22:35:50 <dkranz> jgriffith: I do :)
22:36:01 <jgriffith> mtreinish: adahms :)  I think it's perfectly valid :)
22:36:05 <jgriffith> sorry to interrupt
22:36:16 <mtreinish> asselin: we don't have a feature freeze like the other projects
22:36:30 <asselin> mtreinish, I plan to continue work on https://blueprints.launchpad.net/tempest/+spec/stress-api-tracking after J3. I need to focus on cinder work due before J3. Unless there are people waiting for it
22:36:38 <asselin> mtreinish, good to know
22:36:44 <mtreinish> mostly because contributions for tempest tend to come mostly at the end of the cycle
22:36:56 <mtreinish> after everyone else has a feature freeze and they need to implement tests
22:37:08 <mtreinish> asselin: case and point :)
22:37:12 <asselin> yup :)
22:37:26 <mtreinish> on the high prio bps
22:37:48 <mtreinish> masayukig, andreaf : any update on https://blueprints.launchpad.net/tempest/+spec/tempest-client-scenarios
22:37:56 <mtreinish> I've seen a lot of patches moving on that
22:38:01 <mtreinish> how many left do we have?
22:38:22 <masayukig> yeah, The (migrating) patches statuses are MERGED:2, APPROVED:2, SUBMITTED:9, NOT YET:5 now.
22:38:33 <masayukig> #link https://etherpad.openstack.org/p/tempest-client-scenarios
22:38:41 <masayukig> #link https://review.openstack.org/#/q/topic:bp/tempest-client-scenarios,n,z
22:38:46 <mtreinish> masayukig: ok awesome, we're getting there then
22:39:02 <mtreinish> if people could prioritize reviews on that bp, because it blocks a couple of other things
22:39:15 <andreaf> mtreinish, masayukig there were some more merged for related stuff like the os-networks client or the custom matcher
22:40:04 <mtreinish> andreaf: yeah fair point that's just a list of test patches
22:40:11 <mtreinish> ok are there any other bp updates?
22:40:21 <andreaf> I have the orchestration ones assigned to me but I will be off for a week and I've been busy on other stuff  so I'll remove my name for now
22:40:27 <andreaf> mtreinish, test-accounts
22:40:40 <andreaf> #link http://specs.openstack.org/openstack/qa-specs/specs/test-accounts.html
22:40:52 <mtreinish> andreaf: heh, well that still needs some work, we're probably going to have to drop setup class to get it working...
22:41:09 <dkranz> mtreinish: ick
22:41:21 <mtreinish> dkranz: yeah the problem is by reusing creds if we fail we leak
22:41:27 <mtreinish> and fail includes a skip
22:41:31 <mtreinish> in setupclass
22:41:49 <andreaf> mtreinish, yes I just wanted to say that it uncovered somehow expected issues - in both setup and teardown at class level
22:42:30 <andreaf> even when using safe_setup and teardown is invoked, teardown often fails because of un-initialized attributes
22:42:47 <andreaf> and I've seen at least one case where credentials are deleted before other cleanups
22:42:50 <mtreinish> sigh, yeah it's a real mess
22:43:11 <andreaf> so a lot of issues uncovered - or made more relevant - by this bp
22:43:50 <mtreinish> andreaf: but when it's all said and done we should have much cleaner tempest runs at least :)
22:44:01 <andreaf> mtreinish, yep
22:44:15 <mtreinish> ok so if there aren't any other bps let's move on because we're down to ~15min
22:44:20 <andreaf> mtreinish, I was wondering if someone is working on the branchless extension?
22:44:32 <mtreinish> andreaf: salv-orlando volunteered on monday
22:44:41 <mtreinish> and I actually need to ping him about it after the meeting
22:44:51 <andreaf> mtreinish, great
22:44:52 <mtreinish> because I had some suggestions
22:45:04 <mtreinish> yeah it's awesome, because that's holding up some patches
22:45:08 <salv-orlando> mtreinish: yeah I’m actually pushing the patches now
22:45:14 <mtreinish> salv-orlando: ok cool
22:45:19 <andreaf> mtreinish, I'm happy to do reviews there if needed
22:45:42 <salv-orlando> but your suggestions are still welcome
22:45:52 <mtreinish> andreaf: cool that'll help, but most of it will probably be devstack and devstack-gate
22:46:01 <mtreinish> ok let's move on
22:46:08 <mtreinish> #topic Neutron testing
22:46:31 <mtreinish> so the big news here is salv-orlando finally got neutron full parallel gating everywhere
22:46:34 <mtreinish> \o/
22:46:57 <mtreinish> I'm not sure what other updates there are here though
22:46:59 <andreaf> :) great job
22:47:01 <masayukig> great!
22:47:03 <salv-orlando> bad news is that today an old bug came back. It started about 24 hours ago, I will track it down tomorrow
22:47:17 <salv-orlando> https://bugs.launchpad.net/neutron/+bug/1265495 <- 5 hits in gate today
22:47:18 <uvirtbot> Launchpad bug 1265495 in neutron "Error reading SSH protocol banner" [Critical,Fix released]
22:47:29 <mtreinish> oh more ssh fun...
22:47:44 <salv-orlando> mtreinish: yeah it’s a tricky part where the connection works but auth fails
22:48:06 <mtreinish> that always feels server metadata related to me
22:48:44 <salv-orlando> Indeed that was the first thing we looked at last time. It would be fun if the metadata server is returning some other server’s ssh key
22:49:10 <mtreinish> heh, yeah I think we've seen that before
22:49:26 <mtreinish> ok are there any other neutron updates?
22:49:34 <mtreinish> otherwise let's move on
22:50:03 <mtreinish> #topic Bugs
22:50:26 <mtreinish> dkranz: is there any update on finding someone to organize a bug day?
22:50:48 <dkranz> mtreinish: I will do it. Is Tuesday Sept. 2 OK for folks?
22:51:03 <mtreinish> dkranz: I think that should work for me
22:51:26 <dkranz> mtreinish: ok, I was going to send out an email but wanted to get some feedback here
22:51:38 <andreaf> dkranz, sounds ok
22:51:59 <dkranz> We have been getting a lot less of the random stacktraces going to tempest bugs I think
22:52:10 <mtreinish> dkranz: ok yeah I think that'll work for most people put it on the ML
22:52:14 <dkranz> But I'm sure there are stil some
22:52:16 <mtreinish> dkranz: oh that would be a nice change
22:52:18 <dkranz> mtreinish: will do
22:52:36 <mtreinish> I get emails for each new bug and I've seen a few go by
22:52:48 <dkranz> yeah
22:52:50 <mtreinish> but I think most people are starting to catch on
22:52:56 <dkranz> right
22:52:59 <mtreinish> or just filing them against openstack-ci instead :)
22:53:16 <mtreinish> ok let's move on
22:53:17 <dkranz> mtreinish: at least that is more likely to be correct :)
22:53:23 <mtreinish> #topic Critical Reviews
22:53:35 <mtreinish> so does anyone have any reviews that need some extra attention?
22:53:37 <adam_g> yea!
22:53:50 <mtreinish> adam_g: ok, links?
22:53:52 <adam_g> ironic has some qa requirements that need to be fulfilled before/right around feature freeze
22:54:08 <adam_g> we need migration testing using greande from nova-bm -> ironic
22:54:14 <adam_g> #link https://review.openstack.org/#/q/topic:ironic_grenade+status:open,n,z
22:54:15 <dkranz> https://review.openstack.org/#/c/112581/ (cleanup) and the client checks
22:54:43 <adam_g> in addition to some grenade work, this is dependent on general tempest work we've been chipping away at, which is now shrunk to a small set of patches to devstack+tempest
22:54:51 <adam_g> #link https://review.openstack.org/#/q/topic:ironic_tempest+status:open,n,z
22:55:17 <mtreinish> adam_g: oh lucky you, grenade patches move slowly, and 2/4 people with +2 on it are at linuxcon this week...
22:55:18 <adam_g> the grenade sideways migration testing is based off discussion we had at the nova midcycle, and will probably progress faster when sdague is back
22:55:28 <adam_g> right
22:55:35 <mtreinish> #link https://review.openstack.org/#/c/112581/
22:56:02 <adam_g> it would be *great* if the remaining tempest related work could get some review cycles, in the meantime
22:56:13 <clarkb> adam_g: the devstack-gate side of the sideways upgrade/migration tests is done
22:56:22 <mtreinish> adam_g: the 2 patches
22:56:24 <mtreinish> ?
22:56:25 <adam_g> clarkb, yeah, i saw that it merged. thanks
22:56:33 <clarkb> adam_g: now we need https://review.openstack.org/#/c/111853/ and its child
22:56:39 * clarkb will bug fungi for that now
22:56:58 <adam_g> mtreinish, yeah, 2 tempest patches @ https://review.openstack.org/#/q/topic:ironic_tempest+status:open,n,z and a few more devstack patches
22:57:10 <mtreinish> I have one commit this week too:
22:57:12 <mtreinish> #link https://review.openstack.org/99451
22:57:20 <mtreinish> oh 2 actually:
22:57:22 <mtreinish> #link https://review.openstack.org/111635
22:57:38 <mtreinish> adam_g: ok cool
22:57:48 <mtreinish> ok does anyone else have any reviews to bring up?
22:58:04 <mtreinish> dkranz: on that one, yeah that'll be good to have
22:58:11 <mtreinish> dkranz: I do have some initial comments already
22:58:23 <mtreinish> but I'll do a deep dive on it tomorrow
22:58:27 <dkranz> mtreinish: I was going to get to that tomorrow
22:58:43 <dkranz> mtreinish: I think it has some issues
22:58:50 <mtreinish> it's also kind of a big script, which is why I was hoping for a more iterative approach earlier
22:59:01 <mtreinish> dkranz: yeah, that's what I thought too
22:59:10 <dkranz> mtreinish: Well, it does implement the spec :)
22:59:32 <mtreinish> heh, fair enough
22:59:41 <mtreinish> ok if there aren't any other reviews
22:59:50 <mtreinish> I guess we'll call it a meeting with 20sec left
22:59:55 <dkranz> bye all
23:00:09 <mtreinish> #endmeeting