17:04:56 <oomichi_> #startmeeting qa
17:04:57 <openstack> Meeting started Thu Aug  4 17:04:56 2016 UTC and is due to finish in 60 minutes.  The chair is oomichi_. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:04:58 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:05:00 <openstack> The meeting name has been set to 'qa'
17:05:09 <oomichi_> hi sorry for late starting
17:05:16 <oomichi_> who here today?
17:05:18 <mtreinish> o/
17:05:18 <andreaf> o/
17:05:55 <oomichi_> #link https://wiki.openstack.org/w/index.php?title=Meetings/QATeamMeeting#Agenda_for_July_28th_2016_.280900_UTC.29
17:06:06 <mtreinish> oomichi_: that's last week's agenda...
17:06:12 <oomichi_> ^^^ is the previous agenda
17:06:26 <oomichi_> but I'd like to use it on the same topic
17:06:36 <oomichi_> sorry for miss-updating it
17:06:59 <oomichi_> ok, let's start the meeting
17:07:05 <andreaf> well it probably mostly applies for this week as well :)
17:07:17 <mtreinish> andreaf: heh, well hopefully :)
17:07:20 <oomichi_> #topic Spec review
17:07:37 <auggy> o/
17:07:44 <oomichi_> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
17:07:53 <oomichi_> we have some active specs now on qa-specs
17:08:12 <oomichi_> I will check the first spec of resouce later
17:08:13 <mtreinish> oomichi_: heh, it only took over half the cycle :)
17:08:20 <oomichi_> andreaf: thanks for your review
17:08:38 <oomichi_> mtreinish: yeah, I'd like to see more tbh
17:09:08 <andreaf> oomichi_: what is not clear to me from the review is if we want to resolve resources from the YAML before we pass them to tests or not
17:09:09 <oomichi_> andreaf: can we switch the resource spec to my owner?
17:09:21 <andreaf> oomichi_: sure
17:09:36 <mtreinish> oomichi_: gerrit doesn't let you do that. The review owner is the original submitter
17:09:38 <oomichi_> andreaf: thanks, different owner is easy to forget updating
17:10:04 <oomichi_> mtreinish: yeah, I will abondan current one and create a new one based on current content
17:10:16 <andreaf> oomichi_, mtreinish: well we can make the change a different review, but that's a pity to lose the history
17:10:36 <mtreinish> oomichi_: yeah, don't do that you'll lose the history
17:10:59 <oomichi_> andreaf: yeah, then I will add the original commit-id on the message
17:11:04 <mtreinish> oomichi_: just reset the author on the commit, gerrit will say andreaf but the git log will show you
17:11:27 <mtreinish> it's not a burden to have the review owned by andreaf, it doesn't block you from doing anythng
17:12:28 <oomichi_> mtreinish: different owner make me forget to check it on my gerrit dashboard
17:12:40 <mtreinish> oomichi_: star it
17:13:02 <oomichi_> mtreinish: yeah, I already did it but forgot..
17:13:08 <andreaf> oomichi_, mtreinish: ok can we discuss this bit later and talk about the content of the spec?
17:13:18 <oomichi_> andreaf: yeah, that is nice
17:14:21 <mtreinish> andreaf: by resolve the resources you mean query the api to get the details? Or just build out the matrix of what's available
17:14:33 <oomichi_> andreaf: do you want to keep the label on the resource?
17:14:35 <andreaf> I mean query the API
17:14:55 <mtreinish> andreaf: oh, yeah we probably don't want to do that outside of a test class context
17:15:23 <mtreinish> if it happens too early we end up doing it during discovery and that's not something we really want to do
17:15:23 <andreaf> the main dilemma I think is when to query the API, because otherwise we need to give clients around and have a tenant for that etc
17:15:26 <andreaf> mtreinish: exactly
17:15:56 <andreaf> mtreinish: however the downside of that is that we can only filter on whatever we set in the YAML and not in things which are available via API
17:16:34 <mtreinish> andreaf: I think that's ok. It's how we do most of the config based logic today anyway (especially now that we removed the testscenarios scenariot test :P)
17:16:50 <andreaf> but I guess if we provide a tool to generate the YAML it should be fine
17:17:50 <andreaf> oomichi_ what do you think? are you ok with leaving the API part to the tests?
17:18:11 <oomichi_> andreaf: If tempest doesn't query before the test, I guess the YAML resource instances in the tests are different from original ones
17:18:38 <oomichi_> andreaf: that would make the usage of YAML resource difficult in the tests I guess
17:19:36 <andreaf> oomichi_: well it depends
17:20:13 <andreaf> with uuids they won't be different
17:20:22 <andreaf> but if we use names they may be different
17:20:23 <oomichi_> andreaf: humm, current resource ids of tempest.conf are just uuids
17:20:28 <andreaf> but that may even be ok
17:20:44 <oomichi_> andreaf: yeah, I think now we can remove the content as the first step
17:20:50 <andreaf> with pre-provisioned credentials, it may be that resources are tenant scoped and have the same name
17:21:03 <oomichi_> andreaf: I think I did overthink
17:21:11 <andreaf> that would help in running tests in public clouds, where you don't want to have a cirros image available to all of your customers
17:21:56 <mtreinish> andreaf: why not, it's such a functional os :p
17:22:09 <oomichi_> andreaf: yeah, ok. I will update it based on that
17:22:18 <oomichi_> mtreinish: heh
17:22:43 <oomichi_> ok, the next one is https://review.openstack.org/#/c/349730/
17:22:50 <oomichi_> that is a new one from andreaf
17:23:04 <andreaf> yes so this is part of the code cleanup we talked about
17:23:22 <oomichi_> I am writting some comments on that, but not yet finish
17:23:25 <andreaf> there are already a couple of patches around for validation resources and credentials
17:23:45 <andreaf> but I thought it would be good to go one step back and have a common approach
17:24:03 <mtreinish> andreaf: yeah, I think we should level set and then move forward
17:24:29 <andreaf> oomichi_,mtreinish one thing missing is the ability to schedule a cleanup after tearDownClass
17:24:43 <andreaf> oomichi_, mtreinish: because that would allow use to use fixtures with class level things
17:25:11 <mtreinish> andreaf: right, I was thinking we could just add a useFixtureClass function to the base, and have it create a lifo of registered cleanup functions
17:25:12 <andreaf> something like addClassCleanup and self.useClassFixture( "give-me-creds" )
17:25:35 <mtreinish> we do something like that in scenario (or we used to)
17:26:21 <andreaf> mtreinish, oomichi_ : I have a WIP prototype http://paste.openstack.org/show/549276/
17:26:48 <oomichi_> andreaf: cool, that is easy understanding for me
17:27:11 <andreaf> mtreinish, oomichi_: there are things like collecting all exceptions and setup a multipleexception and logging it properly etc
17:27:17 <andreaf> mtreinish, oomichi_ but it should be doable
17:27:42 <andreaf> that would be a nice way to remove all the 100 ways we have at the moment to do that in different places
17:27:51 <andreaf> including tempest.test.py
17:28:12 <oomichi_> andreaf: yeah, that seems a good direction
17:28:47 <andreaf> oomichi_, mtreinish: this spec was not directly in the list of priorities for newton but I think it's an enabler for the top prio items, so it should be ok to focus on it
17:29:43 <oomichi_> andreaf: high prio items are on good progress
17:29:57 <oomichi_> andreaf: I think we can focus on that in newton
17:30:10 <mtreinish> andreaf: I think we just treat it as the spec for the cruft busters stuff
17:30:40 <andreaf> oomichi_, mtreinish: having this in the test framework would require doing things in testsuite.py, and use a different TestSuite class in Tempest I think
17:30:47 <oomichi_> mtreinish: then cruft busters becomes bigger scope
17:30:57 <mtreinish> andreaf: I definitely like to have a spec approved by germany. I think this will be good work to iterate on f2f there (although I know you'll miss it)
17:31:39 <oomichi_> mtreinish: yeah, and we have enough time before that
17:31:51 <andreaf> ok good
17:31:58 <andreaf> yes I won't be in Germany
17:32:25 <andreaf> I will be in Sicily celebrating my son's birthday (and mine), eating granita and enjoying the sunshine
17:32:30 <oomichi_> andreaf: yeah that is sorry for us
17:32:43 <oomichi_> andreaf: oh, that is good
17:32:55 <oomichi_> andreaf: enjoy your time :-)
17:33:03 <andreaf> thanks ;)
17:33:37 <oomichi_> ok, can we move on the next topic?
17:33:54 <andreaf> oomichi_, mtreinish: I started looking at doing the addClassCleanup in testtools directly as well, but I'd prefer to live that as a potential follow-up
17:34:26 <andreaf> fine for me
17:34:30 <mtreinish> andreaf: that's fine. Although it might mean we have to refactor everything again to make it suitable for testtools and then to use that
17:34:58 <mtreinish> but I think rolling it ourselves first will probably be a faster path
17:35:21 <oomichi_> mtreinish: yeah, that way is easier in most cases
17:35:30 <oomichi_> #topic Tempest
17:35:56 <oomichi_> #link https://review.openstack.org/#/q/project:openstack/tempest+status:open
17:36:11 <oomichi_> we have a lot of patches on the queue
17:36:42 <mtreinish> oomichi_: not really more than usual. Tempest normally has 200-300 open at any point
17:37:07 <oomichi_> mtreinish: yeah, we have always many
17:37:35 <oomichi_> mtreinish: and service clients' patches on good progress
17:38:24 <oomichi_> and new tests still come
17:38:49 <oomichi_> one interesting one is https://review.openstack.org/#/c/315786/
17:39:09 <oomichi_> that comes from some nova feature
17:39:35 <oomichi_> but the test itself doesn't use REST API, just kick the target feature in ssh login
17:39:49 <oomichi_> that is different from current Tempest way
17:40:16 <oomichi_> but they want to test on devstack or some env
17:40:50 <oomichi_> I guess that would be able to run on nova repo
17:41:23 <andreaf> oomichi_: that could be in a nova tempest plugin...
17:41:35 <oomichi_> andreaf: yeah, that can be
17:41:51 <oomichi_> ok, I will reply it more later
17:41:54 <mtreinish> andreaf: I relaly hate seeing core projects having tempest plugins
17:42:03 <mtreinish> and this is not a good justification for that
17:42:12 <oomichi_> do we have more topics on Tempest?
17:42:13 <mtreinish> this should totally be testable from a nova functional test
17:42:47 <andreaf> mtreinish: sure but why a functional test cannot use the tempest framework?
17:42:50 <oomichi_> mtreinish: nova fuctional test is not using devstack, right?
17:43:30 <mtreinish> andreaf: no, it's lower level. BUt that's what they're actually testing that libvirt configured the guest correctly
17:44:09 <mtreinish> oomichi_: no, but you don't actually need devstack to verify this. All they need to do is test that libvirt configured the guest correctly
17:44:20 <mtreinish> why do you need to spin up an entire stack for that
17:44:21 <oomichi_> mtreinish: ah, that seems good for this case
17:45:00 <oomichi_> mtreinish: yeah, they just want to use libvirt feature on the test
17:45:28 <oomichi_> thanks for feedback, I will do response
17:45:36 <oomichi_> are there any topic on Tempest more?
17:45:56 <mtreinish> oomichi_: also there is an implicit assumption in that test that the kernel is built with CONFIG_WATCHDOG=y (but that's probably true for all distros, its a default iirc)
17:46:45 <mtreinish> oomichi_: nothing from me (that I can remember at least)
17:46:53 <oomichi_> mtreinish: yeah, I'd like to check another kernels which the feature is enable on the default
17:47:19 <oomichi_> mtreinish: I guess RHEL enables it at least
17:47:31 <mtreinish> oomichi_: it's a kernel config default to be on IIRC
17:47:43 <mtreinish> so unless someone goes out of their way to disable it should be there
17:47:54 <mtreinish> but that's a side note, not worth really looking at
17:48:04 <oomichi_> mtreinish: ok, thanks. I will check it
17:48:11 <oomichi_> let's move  on
17:48:22 <oomichi_> #topic devstack + grenade
17:48:48 <oomichi_> do we have items on them at this time?
17:49:23 <oomichi_> maybe no ;) ok, let move on
17:49:26 <mtreinish> oomichi_: sdague, sc68cal, and dansmith have been working on switching over to lib/neutron this past week
17:49:26 <andreaf> not from me
17:49:43 <mtreinish> and making that devstack default IIRC
17:50:06 <mtreinish> that's kind of a big one
17:50:06 <oomichi_> mtreinish: oh, then we have specific job with n-network?
17:50:25 <oomichi_> mtreinish: yeah, that is big one
17:50:54 <mtreinish> well switching to neutron as default was always the next step after we finished the mgiration to the new lib/neutron
17:51:33 <oomichi_> mtreinish: IIUC, that happened before. and the gate became unstable
17:51:58 <oomichi_> mtreinish: current one is good status now?
17:52:15 <mtreinish> that aws with the old lib/neutron (which is now lib/neutron-legacy) and it wasn't a gate issue (we've always used it in the gate)
17:52:31 <mtreinish> it didn't work for people outside the gate running with a single interface
17:53:04 <mtreinish> which was part of the motivation of a rewrite to clean it up so you could actually debug it
17:53:35 <oomichi_> mtreinish: cool, thanks for heading up
17:53:46 <oomichi_> ok, lets move on
17:53:48 <oomichi_> #topic openstack-health
17:54:11 <oomichi_> how about o-h this week?
17:54:59 <mtreinish> nothing from me, I haven't had any time to work on it the past couple weeks
17:55:26 <oomichi_> ok, let's move on
17:55:40 <oomichi_> #topic critical reviews
17:55:55 <oomichi_> do we have any reviews we need to review soon?
17:56:28 <andreaf> oomichi_: only the last two in the client refactor
17:56:40 <andreaf> #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/client-manager-refactor
17:57:15 <oomichi_> andreaf: oh, just need last +2s
17:57:30 <oomichi_> andreaf: ok, will review after this
17:57:40 <andreaf> thx
17:57:41 <oomichi_> how about another ones?
17:57:52 <oomichi_> ok, lets move on
17:57:58 <oomichi_> #topic open discussion
17:58:22 <oomichi_> just notification
17:58:25 <oomichi_> #link https://wiki.openstack.org/wiki/Sprints/QAInfraNewtonSprint
17:58:48 <oomichi_> the remaining sheets are 10 on the code sprint
17:59:05 <oomichi_> please register soon if interested in
17:59:35 <oomichi_> that is all from me
17:59:56 <oomichi_> the time comes, thanks all
18:00:00 <oomichi_> #endmeeting