22:00:44 <mtreinish> #startmeeting qa
22:00:45 <openstack> Meeting started Thu Feb 19 22:00:44 2015 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:46 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:48 <openstack> The meeting name has been set to 'qa'
22:00:54 <mtreinish> hi, who's here today?
22:01:03 <oomichi> hi
22:01:03 <masayukig> hi
22:01:16 <dkranz> hi
22:01:17 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_February_19th_2015_.282200_UTC.29
22:01:22 <mtreinish> ^^^ Today's agenda
22:01:48 <oomichi> but I need to leave here in 30 mins
22:01:58 <mtreinish> oomichi: ok
22:02:15 <mtreinish> dtroyer, sdague, jogo: ping
22:02:28 <mtreinish> well let's get started
22:02:35 <mtreinish> #topic QA End of Cycle Code Sprint
22:02:54 <mtreinish> so I just wanted to make a quick mention of this in case someone missed the ml post:
22:03:00 <mtreinish> #link http://lists.openstack.org/pipermail/openstack-dev/2015-February/057168.html
22:03:15 <jogo> mtreinish: o/ a bit distracted because blocking ceilometer, cinder, glance, requirments, devstack, grenade are wedged
22:03:36 <mtreinish> but we'll be doing a qa code sprint in nyc at the end of March
22:03:59 <mtreinish> if anyone is interested in attending the slots are filling up, I think there are only 3 left
22:04:08 <mtreinish> the email has the details
22:04:13 <gmann> hi
22:04:27 <pennerc> o/
22:04:28 <mtreinish> that's all I had for this topic, if there's nothign else we can move on
22:04:42 <dtroyer> o/
22:05:04 <mtreinish> #topic Spec Reviews
22:05:19 <mtreinish> does anyone have any open spec reviews to discuss this week?
22:05:31 <mtreinish> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
22:06:14 <mtreinish> heh, I'll take that as a no then :)
22:06:30 <mtreinish> let's move on then
22:06:51 <mtreinish> #topic Blueprints
22:07:04 <mtreinish> does anyone have an in progress bp to discuss
22:07:12 <mtreinish> I see dkranz put one on the agenda
22:07:17 <dkranz> yes
22:07:26 <dkranz> Final reviews https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/clients-return-one-value,n,z
22:07:49 <dkranz> four of them, two have a +2 already. They are not big.
22:08:05 <mtreinish> ok, I'll take a look after the meeting
22:08:09 <mtreinish> should be pretty quick
22:08:10 <dkranz> I am proposing to leave swift as is, at least for now.
22:08:25 <dkranz> I didn't get much response or proposal.
22:08:29 <mtreinish> dkranz: yeah I'm fine with that. I had a feeling that would be the case because it's all headers
22:08:41 <dkranz> The swift api is an outlier, using headers instead of dicts for many things
22:09:03 <dkranz> So I will close the bp as soon as the above patches merge.
22:09:09 <oomichi> dkranz: +1
22:09:15 <mtreinish> dkranz: awesome
22:09:41 <dkranz> that's it
22:09:45 <mtreinish> ok, does anoyone else have an in progress bp to discuss?
22:10:47 <mtreinish> I know andreaf_ has been working on the auth tempest-lib migration
22:11:03 <mtreinish> there was a question for oomichi at one point about the tokenclient migration
22:11:17 <mtreinish> oomichi: did andreaf_ get a chance to ask you about that?
22:11:38 <oomichi> mtreinish: nothing from him
22:11:58 <oomichi> mtreinish: are there any concerns about auth migration?
22:12:01 * dstanek was running late, but is here now
22:12:12 <gmann> mtreinish: oomichi : CONF removal patch for token clients https://review.openstack.org/#/c/156906/
22:12:24 <mtreinish> oomichi: ok, IIRC the issue is the auth layer depends on the tokenclient and there were some questions about how/when to migrate that to the lib
22:12:56 <mtreinish> gmann: oh, that patch might have just been all that was needed :)
22:14:01 <mtreinish> well without andreaf_ here to outline the specific questions, it's not really something we can discuss I guess
22:14:20 <mtreinish> so I'll circle back with him tomorrow and make sure everything is still on track for that
22:14:33 <mtreinish> ok, are there any other bps to discuss? Otherwise we can move on
22:14:45 <oomichi> mtreinish: thanks in advance:)
22:15:40 <mtreinish> #topic Devstack
22:16:00 <mtreinish> dtroyer: now I know there's been some cool stuff this past week in devstack
22:16:08 <dtroyer> just a bit ;)
22:16:29 <dtroyer> We're deep into the venv changes, the wheel cache builder merged today
22:16:55 <dtroyer> I'm working ahead on getting the layer 1 services into individual venvs, nova is a @#$%^&*%#@@!#
22:17:28 <jogo> dtroyer: per service venvs?
22:17:36 <dtroyer> I'm planning to put the higher layer services into a single default venv so we dont' have to go around and change everyone's plugins
22:17:54 <dtroyer> jogo: think that's overkill?  it turned out to be nearly the same work either way
22:17:59 <mtreinish> heh, I can imagine. Is it just the libvirt stuff?
22:18:13 <dtroyer> and rootwrap, so cinder and ironic are looming
22:18:18 <jogo> dtroyer: overkill probably not. just wondering
22:18:38 <jogo> dtroyer: just thinking distros won't like it
22:18:47 <mtreinish> jogo: and? :)
22:18:52 <jogo> mtreinish: and nothing
22:18:52 <dtroyer> jogo: it's eposing some assumptions that I think are good to flesh out, like all the stuff nobody knows is required because another project installed it
22:19:19 <jogo> dtroyer: yeah, I am excited to see this coming along nicely
22:19:58 <dtroyer> anyway, another related thing that sdague proposed is removing all distro-packaged pure python packages.  that was going to effectively happen with venvs, probably good to make it explicit
22:20:56 <dtroyer> anyway, that's pretty much my quiet week in Weatherby Lake
22:21:29 <mtreinish> heh, ok does anyone else have anything to bring up on devstack?
22:22:01 <dkranz> jogo: is there a simple explanation of what "distros won't like" and why?
22:23:15 <jogo> dkranz: yes, because distros don't use venvs to install things. they want one set of common deps
22:23:31 <jogo> so it just means our test env is not the way a distro would do it
22:23:54 <jogo> doesn't mean we automatically break things for them though. g-r should take care of that
22:24:17 <dkranz> jogo: ok
22:24:36 <dkranz> jogo: the future is probably containers anyway
22:24:46 <dkranz> jogo: which solve the same problem as vms
22:24:53 <dkranz> jogo: venvs
22:25:01 <jogo> dkranz: :)
22:25:20 <mtreinish> ok if there isn't anything else let's move on
22:25:32 <mtreinish> #topic Grenade
22:25:58 <mtreinish> jogo, dtroyer: anything new with grenade this week?
22:26:41 <dtroyer> I haven't even looked at it, except in how it will handle the venvs
22:27:02 <jogo> nothing on my end
22:27:18 <mtreinish> ok
22:27:27 <mtreinish> does anyone else have anything to discuss on grenade?
22:28:15 <mtreinish> ok, then let's move on
22:28:29 <mtreinish> #topic test_server_basic_ops testscenarios usage (mtreinish)
22:28:46 <mtreinish> so this is something that came up in a bug/review
22:29:09 <mtreinish> we've got a test class in tempest.scenario that uses testscenarios over a matrix of images and flavors
22:29:27 <mtreinish> and I was wondering if that is still useful for people
22:29:50 <andreaf_> o/
22:30:10 <dkranz> mtreinish: as indicated in the review, I agree that we are abusing testscenarios and like andreaf suggestion
22:30:11 <andreaf_> mtreinish: sry I'm late :)
22:30:31 <mtreinish> dkranz: sure, that's a path out of the mess in it's current implementation
22:30:55 <mtreinish> but I was just taking a step back and wondering if that functionality is really used by anyone
22:31:13 <dkranz> andreaf: Didn't you put it in?
22:31:15 <mtreinish> I still think andreaf_'s suggestion long term is something we want (images.yaml or whatever)
22:31:25 <andreaf_> mtreinish: well we do use it in out testing
22:31:45 <mtreinish> andreaf_: ok that's all I was looking for :)
22:31:53 <mtreinish> I just didn't know if there was a real use case for it
22:32:17 <andreaf_> mtreinish: well we use it for routine validation of multiple guest images
22:32:18 <mtreinish> because the test that's looping over the matrix is pretty simple
22:33:13 <mtreinish> anyway it's nothing critical because we have a workaround to stop the failure
22:33:17 * cyeoh wanders in very late
22:33:18 <mtreinish> and a long term path
22:33:25 <mtreinish> I just was curious
22:33:47 <mtreinish> because that utils module is a source of constant bugs with discovery and test listing
22:34:11 <mtreinish> ok that's all I had for this topic. Does anyone else have anything to add?
22:34:45 <mtreinish> #topic Bugs
22:35:14 <mtreinish> #link https://etherpad.openstack.org/p/Tempest-bug-report
22:35:32 <mtreinish> from last week I still need to schedule the bug backlog day
22:36:01 <mtreinish> I had the bug rotation this week and only a few new bugs came in
22:36:11 <dkranz> ALso, gmann signed up for next week but no one after that
22:36:22 <dkranz> https://etherpad.openstack.org/p/qa-bug-triage-rotation
22:36:57 <masayukig> dkranz: I'll do :)
22:37:02 <dkranz> mtreinish: I wanted to discuss https://bugs.launchpad.net/tempest/+bug/1296955
22:37:03 <openstack> Launchpad bug 1296955 in tempest "Cannot run tempest without admin creds" [Medium,Incomplete]
22:37:08 <mtreinish> dkranz: sure
22:37:14 <gmann> mtreinish: dkranz: we might need to circulate mail again
22:37:29 <mtreinish> gmann: sure, can you take care of doing that?
22:37:31 <gmann> masayukig: thanks :)
22:37:33 <dkranz> mtreinish: I tried several variants of the conf file and all failed quite miserably
22:37:44 <gmann> mtreinish: sure, i will send today
22:38:02 <mtreinish> gmann: great, thanks
22:38:10 <dkranz> mtreinish: as per my last comment in the ticket, I need a statement of exactly what is expected to work by the implementors of test-accounts because this is not documented
22:38:24 <mtreinish> #action gmann to send a call for volunteers on bug triage
22:38:38 <mtreinish> dkranz: sure I'll write up a doc and push it out
22:39:09 <dkranz> mtreinish: really all that is needed is a diff from a standard devstack gate created tempest.conf and one that is claiming to work with no admin
22:39:34 <dkranz> mtreinish: my guess is that what I tried is actually in the "supposed to work" category but does not
22:40:07 <mtreinish> dkranz: sure that might be, we don't test this case frequently so it might not work
22:40:12 <dkranz> mtreinish: I can debug this more once we establish what is supposed to work
22:40:12 <andreaf_> dkranz, mtreinish: some patches I'm also working on will help make things more consistent with admin credentials - now many tests / base classes allocate credentials their own way
22:40:26 <mtreinish> but I'll put together a doc on how to run without admin and test it too
22:40:33 <dkranz> mtreinish: thanks!
22:40:52 <dkranz> mtreinish: there was also the comment in Sean's testing recap about getting rid of admin tests in tempest
22:41:07 <mtreinish> andreaf_: sure, that was one of the advantages of the resource split bp
22:41:17 <andreaf_> indeed
22:41:18 <dkranz> Not sure I agree with that
22:41:42 <mtreinish> dkranz: that's really something else. He's calling for the removal of the api admin tests not the removal of tests requiring admin
22:42:19 <mtreinish> dkranz: I'm not in full agreement with it either, especially because what's an "admin api" is configurable
22:43:00 <mtreinish> but I think the intent there was to not test things like agregates/azs in tempest because they're really error prone because of the global state issue
22:43:20 <dkranz> mtreinish: Yes, but there will not be many tests requiring admin that are not testing admin apis
22:43:51 <dkranz> mtreinish: sure. And tests that shut down hosts :)
22:44:32 <dkranz> mtreinish: But it is certainly true that policy files make it not clear what is an admin api and what not.
22:44:39 <dkranz> mtreinish: that's part of what I didn't like
22:44:46 <dkranz> mtreinish: But we can move on
22:44:56 <mtreinish> dkranz: sure, maybe leave some comments on the review
22:45:07 <mtreinish> I already did, but it looks like sdague just ignored most of them
22:45:15 <mtreinish> I also don't feel super strongly about it
22:45:32 <mtreinish> ok does anyone else have any other bugs to discuss?
22:46:24 <mtreinish> #topic Critical Reviews
22:46:39 <mtreinish> ok does anyone have a review they'd like to get some extra eyes on?
22:47:24 <mtreinish> well if no one else does I'll put some of mine :)
22:47:30 <mtreinish> #link https://review.openstack.org/154210
22:47:40 <mtreinish> that should be pretty simple just an oslo lib migration
22:48:02 <mtreinish> and
22:48:06 <mtreinish> #link https://review.openstack.org/157182
22:48:19 <mtreinish> that isn't critical but getting some extra opinions on it would be cool
22:49:11 <dkranz> mtreinish: oh, yeah. I wasn't sure about showing all the jitter by default
22:49:18 <dkranz> mtreinish: on the last one
22:49:20 <dstanek> i have a keystone review i'd love to get feedback on
22:49:24 <dstanek> #link https://review.openstack.org/#/c/153300/
22:49:37 <dstanek> it's how we are thinking about functional testing
22:49:55 <mtreinish> dstanek: sure I'll take a look
22:50:15 <dstanek> that's the spec and i do have some code for devstack plugins and other stuff
22:50:16 <mtreinish> dkranz: well it only shows up if there is data to do it with
22:50:34 <mtreinish> dkranz: but I'm fine with making it opt-in
22:50:35 <dstanek> mtreinish: thx
22:50:54 <dkranz> mtreinish: I think that would be better, at least at first
22:51:29 <mtreinish> dkranz: heh, the jitter is kinda of fun on quick unit tests though
22:51:49 <mtreinish> dkranz: like I got "[0.439191s +5966.17%]" on one test :)
22:52:08 <dkranz> mtreinish: :)
22:52:15 <mtreinish> ok does anyone else have any reviews to discuss?
22:52:17 <andreaf_> mtreinish: when you do the preseeding, do you use data from the same job only?
22:52:31 <andreaf_> or is it the overall avg of a test across all jobs?
22:52:33 <dkranz> mtreinish: but seriously, that means the percent allowed needs to be scaled to the actual time somehow
22:52:53 <mtreinish> andreaf_: no it was originally an average across all jobs
22:53:09 <mtreinish> but that didn't work because it would strip the attrs from the subunit
22:53:27 <mtreinish> so now it's just the 10 most recent samples during nodepool image creation
22:54:03 <mtreinish> we can't filter it by job because we do it with a nodepool script so it'll be used with all the jobs
22:54:22 <andreaf_> mtreinish: ok cool
22:54:44 <mtreinish> andreaf_: fwiw, it seems pretty hit or miss whether it does anything
22:54:53 <mtreinish> I still need to do more debug to figure out why
22:55:11 <mtreinish> well I think we've already moved into the open discussion
22:55:15 <mtreinish> #topic Open Discussion
22:55:28 <mtreinish> so in the last 5 min. does anyone have anything to bring up that wasn't on the agenda?
22:56:03 <mtreinish> I see dims put up a reminder about the GSoC stuff
22:56:11 <mtreinish> #link https://wiki.openstack.org/wiki/GSoC2015
22:57:10 <mtreinish> does anoyone have anything else to discuss?
22:57:16 <mtreinish> otherwise we can just end here for today
22:57:56 <masayukig> thanks everyone :)
22:58:04 <mtreinish> heh
22:58:07 <mtreinish> #endmeeting