09:00:32 <gmann> #startmeeting qa
09:00:33 <openstack> Meeting started Thu Mar 24 09:00:32 2016 UTC and is due to finish in 60 minutes.  The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:00:34 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:00:37 <openstack> The meeting name has been set to 'qa'
09:00:44 <gmann> hi who's here for today?
09:00:54 <jlanoux> hi
09:01:04 <jordanP> hi
09:01:15 <gmann> jlanoux: jordanP : hi
09:01:44 <gmann> Let's wait for few min to have more people
09:01:53 <andreaf> o/
09:02:24 <gmann> andreaf: hi
09:02:50 <andreaf> gmann: hi
09:02:54 <gmann> masayukig will not be able to join today
09:02:59 <gmann> So let's start
09:03:05 <gmann> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_March_24th_2016_.280900_UTC.29
09:03:08 <gmann> ^^ today agenda
09:03:23 <gmann> #topic Specs Reviews
09:03:35 <gmann> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
09:04:05 <gmann> andreaf: most of spec from you :) anything you want to bring up here
09:04:26 <andreaf> gmann: just that I've been swamped with internal stuff in the past weeks
09:04:43 <andreaf> gmann: but I will pick up specs and reviews again now, sorry about the delay on those
09:04:51 <gmann> ohk
09:05:09 <gmann> np, you already doing lot of thing :)
09:05:43 <gmann> andreaf: for client refactor m almost good, just a suggestion if we can simplify the access of registered clients
09:05:47 <gmann> #link https://review.openstack.org/#/c/92804/
09:06:06 <gmann> may be you can take look later if that makes sense
09:06:15 <andreaf> gmann: I will check your comment
09:06:24 <gmann> andreaf: Thanks :)
09:06:41 <andreaf> gmann: initially my approach was to have support for aliases and simplified names to access clients
09:07:00 <jordanP> the less "meta programming" the better, imo
09:07:01 <andreaf> gmann: but after discussing with mtreinish we agreed that was too much complexity
09:07:12 <jordanP> I am looking closely at this
09:07:15 <gmann> yea.
09:07:34 <gmann> jordanP: yea agree on that. Thanks
09:07:45 <andreaf> gmann, jordanP: well exactly so for now names will be what they are now
09:07:54 <jordanP> great
09:08:10 <andreaf> gmann, jordanP: and for registered client it will be plugin_client_name
09:08:23 <andreaf> gmann, jordanP: so we can avoid name conflicts across plugins
09:08:47 <jordanP> I am not sure this was a big concern, but that's good to know
09:09:00 <andreaf> gmann. jordanP: a follow up will be to cleanup and standardise names across internal clients and least and get rid of all the aliasing in test base classes
09:09:23 <mkoderer__> hello
09:09:23 <gmann> andreaf: yea, i was thinking if  "client_name" part can be taken from users itself instead of building from class name etc
09:09:30 <jordanP> I am not sure people will move massively to the plugin models
09:09:36 <andreaf> jordanP: well, flavor_client can be in many places :D
09:09:44 <gmann> mkoderer__: hi
09:10:05 <andreaf> gmann: yes it will be defined statically and not taken from the class name
09:10:17 <andreaf> at least until we have a standard defined and enforced
09:10:28 <gmann> andreaf: great. that solve my main concern
09:10:52 <gmann> andreaf: yea, we should do standard definition there but may be later
09:10:53 <andreaf> gmann, jordanP: there is another problem with different instances of the same client in the manager
09:11:07 <andreaf> like identity does for instance with public and admin for v2
09:11:27 <andreaf> but I guess those can still defined old stile by tempest client manager
09:11:29 <jordanP> what's the problem (apart from being ugly)
09:12:14 <andreaf> jordanP: it's just a deviation from one-client -> one-attribute
09:12:14 <gmann> andreaf: yea for those we need some exception in naming if we do standard one for all
09:12:39 <andreaf> jordanP: but I don't want to extend the model to being 1 client -> n attributes because it makes things more complex
09:13:03 <andreaf> jordanP: so extra attributes can be added "by hand" by the client manager definition
09:13:16 <gmann> and we might need that as those 2 instances can be needed in single tests
09:13:45 <jordanP> ok
09:14:42 <andreaf> jordanP, gmann: thanks for your comments on this - I will update the spec and try to keep things as simple and non-meta-programming as possible :)
09:15:07 <gmann> ok, I think that is close to merge now and let's review after update .
09:15:16 <gmann> Thanks andreaf  for all hard work on this :)
09:15:30 <gmann> anything on any other spec ?
09:16:24 <gmann> ok, let's move
09:16:28 <gmann> #topic Priority Items
09:16:42 <gmann> #link https://etherpad.openstack.org/p/mitaka-qa-priorities
09:16:52 <gmann> so i want to update on microversion one
09:17:04 <gmann> all framework is now merged in tempest/lib
09:17:12 <gmann> and doc also added
09:17:28 <gmann> there is small updates in doc left
09:17:31 <gmann> #link https://review.openstack.org/#/c/296939/
09:17:45 <gmann> it is simple one if you guys can review
09:17:54 <jordanP> about microversion: https://review.openstack.org/#/c/258391/32/tempest/lib/api_schema/response/compute/v2_19/servers.py
09:17:54 <gmann> I will close the BP after this
09:18:08 <jordanP> how come this looks suboptimal ?
09:18:20 <jordanP> there's no otherway around this ?
09:18:50 <gmann> jordanP: yea actually tempest own the compute response schema and they change across versions
09:19:06 <jordanP> yeah I get the why. But this looks ugly no ?
09:19:13 <gmann> jordanP: in 2.19 only those API schema has been changed so need to version those
09:20:01 <gmann> jordanP: humm, yea right. it was more if we do in same file. so i tried to separate those in separate file
09:20:15 <gmann> jordanP: I am open to have more better way if any.
09:20:23 <gmann> at least more readable way if we can find
09:20:58 <gmann> also i do not want to duplicate the schema among version so copying those from previous version schema
09:21:16 <jordanP> yeah, I see.
09:21:21 <jordanP> anyway, it's minor
09:21:43 <gmann> jordanP: and another hard thing is if we implement the version tests randomly like we did for 2.20
09:22:10 <gmann> we have to version previous microversion schema also, there is no way to aboid that
09:22:13 <gmann> avoid
09:22:33 <gmann> I have documented those extra step in doc to make it understand
09:22:39 <jordanP> ok
09:23:50 <gmann> ok
09:24:09 <jordanP> guys, not in priority items, but while I worked on replacing httplib2 by urllib3 I noticed that we use a lot "oslotest.mockpatch" in our unit tests
09:24:10 <gmann> so any other items anyone want to discuss on priority items?
09:24:18 <andreaf> on (10) the ssh-auth-bp is basically done, jlanoux has prepared docs updates
09:24:23 <jordanP> mockpatch is deprecated and is going to be removed
09:24:31 <andreaf> once those merge I propose we close the bp
09:24:36 <gmann> jordanP: I see
09:24:43 <jlanoux> andreaf: actually it merged this night
09:24:47 <gmann> andreaf:  great, do we have link for that patch ?
09:24:48 <jlanoux> we can close it :)
09:24:55 <gmann> jlanoux: oh great
09:25:03 <andreaf> jlanoux: oh, even better
09:25:05 <andreaf> :)
09:25:13 <jlanoux> (yay)
09:25:18 <gmann> jlanoux: andreaf Thanks :)
09:25:30 <gmann> jordanP: so on that do you have patch up?
09:25:49 <jordanP> to replace httplib2 yes: here: https://review.openstack.org/#/c/295900/
09:26:07 <jordanP> it's not a WIP anymore, I am waiting for jenkins to be green
09:26:10 <gmann> jordanP: Thanks
09:26:11 <jordanP> it should be
09:26:13 <gmann> #link https://review.openstack.org/#/c/295900/
09:26:20 <gmann> we will check
09:26:29 <jordanP> but someone could work on getting ride of "oslotest.mockpatch"
09:26:39 <gmann> and about "oslotest.mockpatch" what is other option ?
09:26:40 <jordanP> and use only "mock"
09:26:51 <gmann> jordanP: ok
09:27:10 <jordanP> mock is good, it's in Python 3 stdlib
09:27:12 <gmann> jordanP: I will check may be in next week if no one pickup
09:27:22 <gmann> jordanP: +1
09:27:25 <jordanP> it's the "the facto" library for this kind of thing
09:27:48 <jordanP> and mockpatch was just a small wrapper around it, not super useful imp
09:27:50 <jordanP> imo
09:28:11 <gmann> humm, yea we can cleanup those
09:28:19 <jordanP> also, I am not a fan of how we use "useFixture" from the testtools library
09:28:45 <gmann> jordanP: the way we use or "useFixture" not good?
09:29:06 <gmann> i feel "useFixture" very useful for auto cleanup etc
09:29:22 <jordanP> in Tempest we use useFixture() method from the testtools library, most of the time we don"t need it
09:29:43 <jordanP> yeah but mock.patch "undo" the mock automatically if you use the decorator form
09:30:19 <jordanP> I don't like autocleanup, too magical.
09:30:28 <mkoderer__> jordanP: +1
09:30:38 <gmann> jordanP: but mostly we use it for config options setting
09:30:49 <jordanP> too much "auto"everything and then you forget what the code actually does
09:31:24 <jordanP> gmann, what I found very common in Tempest is: self.useFixture(oslotest.mockpatch('..'))
09:31:42 <jordanP> why a simple @mock.patch around the test itself would work very well
09:31:46 <jordanP> *while
09:32:05 <jordanP> anyway, I may be able to work on it too
09:32:53 <jordanP> anyway, guys I spent 2 days on https://review.openstack.org/#/c/295900 (Get rid of httplib2, use urllib3 instead) so I'd appreciate any review
09:33:00 <gmann> jordanP: i see
09:33:08 <gmann> like this https://github.com/openstack/tempest/blob/master/tempest/tests/common/test_alt_available.py#L46
09:33:24 <jordanP> it's not sexy to review, and I fought with unit tests, but it's an improvement I am sure of it
09:33:54 <jordanP> yeah
09:34:00 <gmann> jordanP: ok
09:34:14 <gmann> we will review those. Thanks jordanP
09:34:27 <jordanP> thanks
09:34:54 <gmann> moving next..
09:35:00 <gmann> #topic Tempest
09:35:30 <gmann> i think on negative tests we have good discussion in ML
09:35:46 <gmann> anyone want to talk on this or we move on?
09:36:13 <jordanP> I am not sure we came to a clear conclusion
09:36:35 <jordanP> I still stand by my position: they don"t hurt, and some of them are useful
09:36:44 <gmann> jordanP: yea, so i added it in Austin design session candidate which we will talk later
09:36:50 <jordanP> ok
09:37:11 <gmann> I have nother to discuss, v2.20 microversion tests
09:37:14 <gmann> #link https://review.openstack.org/#/c/258391/
09:37:20 <jlanoux> I have a general comment linked to what was said regarding the negative tests
09:37:24 <gmann> jordanP: jlanoux Thanks for review and working
09:37:36 <gmann> jlanoux: sure
09:38:15 <jlanoux> More and more features are added to OpenStack. So as I agree we need to keep an eye on time testing, we need to face the fact that the global time will increase in the future as we add tests for coverage.
09:39:03 <jlanoux> that's all
09:39:05 <jordanP> quick feedback is of prime importance in software development
09:39:05 <gmann> jlanoux: so thats performance testing right
09:39:21 <gmann> we do via gate job timing. no?
09:39:23 <jordanP> if tests run in hours, it's a pain in the ass
09:39:44 <jlanoux> time in gate yes
09:39:46 <jordanP> people already complained on the ML about this (the neutron folk, the nova folk)
09:39:55 <jlanoux> there is no other solution
09:40:17 <jordanP> we must be super carefully, each additionnal minutes is super super costly, because it's multiplied by 1000 each day
09:40:23 <jordanP> jlanoux, there is
09:40:32 <gmann> jordanP: if scenario is like that then it might take time right
09:41:09 <gmann> ok so back to v2.20 patch.
09:41:14 <gmann> So in that v2.20 tests, we need to tests volume attach/detach when server is in shelved or shelved offload state
09:41:30 <jlanoux> the volume attach shelve is a very good example. it's a long test but its place belong in Tempest, not in the Nova functional test
09:41:33 <gmann> jordanP: jlanoux i replied on that. please check
09:41:35 <jlanoux> yes gmann
09:42:05 <gmann> so actually if we do write separate tests then shelved tests will always be skipped for offload time >=0
09:42:26 <gmann> so i want a single tests which can tests both state as per configuration
09:42:35 <jlanoux> actually, I'm fine with having only the shelve iffload
09:42:41 <gmann> i do not want to have tests on gate which always in skip state
09:42:41 <jlanoux> *offload
09:43:03 <gmann> jlanoux: ok
09:43:17 <gmann> so when offload time is -1 it will tests with shelved state
09:43:30 <jordanP> what I also dislike in that patch is that we now have a function called "_create_and_attach" that : creates a volume, can shelve a server, and attach a volume
09:43:43 <jordanP> that a lot for a single server
09:43:43 <jlanoux> gmann: yes
09:43:45 <gmann> so i want opinion here in that case should we make server in offload state and tests
09:43:49 <jordanP> *single function
09:43:51 <gmann> or in shelved state only
09:43:56 <jlanoux> jordanP: it's a real use case
09:44:07 <gmann> scenario tests for shelve are doing the same tests with shelved offload state
09:44:07 <jlanoux> jordanP: the most worthwile tests for me
09:44:33 <gmann> jordanP: yea thats what scenario here is we need to tests
09:44:52 <jlanoux> gmann: yes, however for the volume attach/detach in shelve, the code path is slighty different
09:44:58 <jlanoux> but I think it's fine
09:45:05 <gmann> jordanP: i agree that it will take time but we have to perform all operation to tests 2.20
09:46:16 <jordanP> I'd like to have mtreinish or sdague opinion on this
09:46:51 <gmann> because we do not have separate gate job for those config so we have to go with single tests and let is tests what you cloud is configured for
09:47:29 <gmann> ok, so i will update that after re thinking on those two states (either always shelved offload or one by one)
09:47:40 <gmann> and we can get more feedback about scenario there
09:48:19 <gmann> thats all on this
09:48:25 <gmann> anything on Tempest ?
09:49:32 <gmann> ok, let's move
09:49:37 <gmann> #topic Austin Summit
09:49:50 <gmann> so we have etherpad for austin design sessions
09:50:07 <gmann> #link https://etherpad.openstack.org/p/newton-qa-summit-topics
09:50:22 <gmann> please have a look and add your ideas there
09:50:38 <andreaf> gmann
09:50:49 <andreaf> gmann: sorry I lost my network, back in now
09:50:59 <gmann> andreaf: ohk
09:51:15 <gmann> andreaf:  so we talked about this https://review.openstack.org/#/c/258391/
09:51:25 <gmann> i will update and you can have look
09:51:36 <gmann> andreaf:  and now starting Austin Summit
09:51:36 <jordanP> guys, I am sorry but I may not be in Austin. Future is uncertain here for me :)
09:51:41 <gmann> #link https://etherpad.openstack.org/p/newton-qa-summit-topics
09:51:48 <andreaf> gmann: I added one proposal about removing cruft from the code, making more debuggable and easier to contribute to
09:51:51 <gmann> jordanP: oh why?
09:52:11 <andreaf> jordanP: :(
09:52:12 <gmann> andreaf: great, that is very useful and must for tempest :)
09:52:31 <gmann> jordanP: thats not good news :(
09:52:35 <jordanP> business w.r.t openstack is not going great here
09:52:42 <gmann> hummm
09:52:50 <mkoderer__> jordanP: :(
09:53:22 <andreaf> sorry to hear that - I hope you'll be able to come nonetheless
09:53:44 <gmann> yea
09:53:46 <jordanP> thanks guys
09:53:50 <gmann> mkoderer__: you will be there right?
09:54:07 <mkoderer__> gmann: yes
09:54:15 <gmann> jordanP: we thought of having drink party with you as you were missed always :(
09:54:20 <gmann> mkoderer__: great :)
09:54:36 <gmann> ok 5 min left
09:54:38 <gmann> #topic Critical Reviews
09:54:44 <gmann> any critical review?
09:55:20 <gmann> ok, i have one not critical, simple one :) - https://review.openstack.org/#/c/296939/
09:55:22 <andreaf> just the name of the cycle - newton is too similar to neutron - I will mix them up continuously
09:55:32 <jordanP> haha, +1 !
09:55:32 <gmann> heh
09:55:39 <gmann> me too
09:56:09 <andreaf> why Microversion with capital M?
09:56:45 <gmann> andreaf: actually for consistency i made capital, or we can make all with small m
09:57:30 <andreaf> well I'm not native english speaker - so probably I should just shut-up :P
09:57:52 <gmann> not like that :)
09:58:05 <gmann> ok 3 min, let's jump to open
09:58:08 <gmann> #topic Open Discussion
09:58:16 <gmann> anything else to discuss from anyone
09:59:24 <gmann> ok let's close meeting if nothing more
09:59:28 <gmann> 3...
09:59:32 <gmann> 2..
09:59:40 <gmann> 1.
09:59:49 <gmann> Thanks all for joining
09:59:51 <jordanP> good timing :)
09:59:56 <jlanoux> bye!
09:59:59 <andreaf> bye
10:00:03 <gmann> #endmeeting