09:00:42 <gmann> #startmeeting qa
09:00:43 <openstack> Meeting started Thu Dec 10 09:00:42 2015 UTC and is due to finish in 60 minutes.  The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot.
09:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
09:00:46 <openstack> The meeting name has been set to 'qa'
09:00:57 <gmann> hi, who's here today?
09:01:01 <jlanoux> o/
09:01:02 <dwalleck> o/
09:01:02 <dmellado> Hi gmann o/
09:01:08 <gmann> dmellado: hi
09:01:14 <gmann> jlanoux: hi
09:01:34 <jlanoux> hi gmann
09:01:43 <gmann> let's for min to join more people
09:01:47 <dmellado> sure
09:02:07 <ylobankov> hi
09:02:28 <gmann> ylobankov: hi
09:02:44 <gmann> ok let's start
09:02:51 <gmann> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_December_10th_2015_.280900_UTC.29
09:02:59 <gmann> && Today meeting agenda
09:03:18 <gmann> #topic Specs Reviews
09:03:38 <gmann> we have most of spec under review
09:03:41 <gmann> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
09:04:15 <gmann> andrea updated tempest-resource spec, good to review that
09:04:18 <gmann> #link https://review.openstack.org/#/c/173334/
09:04:19 <jordanP> gi
09:04:20 <jordanP> hi
09:04:27 <gmann> jordanP: hi
09:04:27 <dmellado> morning jordanP ;)
09:04:50 <gmann> There are other 2 in list of agenda
09:04:53 <gmann> # https://review.openstack.org/#/c/230183/
09:05:17 <gmann> I have not gone through it yet, but its under review
09:05:25 <gmann> anyone has anything on this to discuss
09:05:58 <gmann> anyways next one is list plugin one
09:06:04 <gmann> #link https://review.openstack.org/#/c/247575/
09:06:13 <dmellado> I haven't gone over there yet, but I'd like to somehow couple the config script inners with the bp-resources
09:06:20 <gmann> this one was straight forward
09:06:22 <dmellado> I'll update it when I've some time
09:06:29 <gmann> dmellado: cool, Thanks
09:06:38 <gmann> plugin list one is merged.
09:06:54 <jordanP> yep
09:07:11 <gmann> I also found difficulty while debugging ironic plugin which was in progress
09:07:23 <gmann> jordanP: yea it will be nice addition
09:07:34 <gmann> anything on spec
09:07:58 <gmann> or any other spec anyone want to bring up for discussion
09:08:06 <dwalleck> I have one - https://review.openstack.org/#/c/255023/
09:08:27 <gmann> dwalleck: sure, please go ahead
09:08:54 <dwalleck> There's been a lot of discussion about how the remote client is tied to very generic Linux implementation. This spec allows for an extendable implementation
09:09:30 <dwalleck> I know there's some work around this in the resources spec, but I don't think it will go far enough to solve the problem. I put this up as another possibility to consider
09:09:54 <gmann> dwalleck: i see, so it will be having drives for each connectivity protocol ?
09:10:09 <jordanP> dwalleck, I like this spec. The current linux client and implementation is a bit all over the place, a refactor and architecture change would be good
09:10:23 <jordanP> with more consistency
09:10:25 <dmellado> +also adding support for the dreaded windows
09:10:27 <dmellado> xD
09:10:43 <gmann> jordanP: yea looks like nice one
09:10:45 <jordanP> but, are we *that* interested in testing something else than Linux ?
09:10:49 <dwalleck> gmann: Right. A driver per protocol, and the remote client implementation picks the driver it needs
09:10:57 <jordanP> I mean it's gonna be some work, you sure it's worth it ?
09:11:15 <gmann> dwalleck: 1 question does this one contain how our tests goind to switch between those
09:11:18 <dwalleck> The dreaded windows work is done :D If you check one of the bottom links, I link to an external implementation
09:11:28 <gmann> dwalleck: ok, so remote client will pick up dynamically
09:11:29 <dmellado> dwalleck: jordanP: also, will be the gates running anything else? otherwise this might end up being outdated in case it gets implemented
09:11:50 <gmann> dmellado: nice point
09:11:57 <dwalleck> gmann: My thought was to have it be a property in the resource file for the image
09:12:16 <gmann> dwalleck: i see.
09:12:21 <gmann> dwalleck: Thanks
09:12:34 <dwalleck> dmellado: We run a lot of Windows gates at Rackspace. But yes, testing of multiple drivers is one of the possible pain points
09:12:54 <jordanP> well at least if it's unit tested, it's a good start
09:13:16 <dwalleck> But I'd be more than glad to help with that. I also suggest making the remote client extensions plugins, which would put the maintenance back to the developer of the plugin
09:13:23 <dmellado> +1 from me, if we can couple it with unit tests
09:13:50 <gmann> and if missing one can be added as we planed for resource things
09:13:59 <gmann> anyways its good one for review and lets start reviewing it and have comment/suggestion there
09:14:05 <gmann> dwalleck: Thanks .
09:14:11 <gmann> let's move on
09:14:15 <dwalleck> thanks everyone :-)
09:14:18 <dmellado> dwalleck: I'll take a look later and add comments, thanks ;)
09:14:33 <gmann> #topic Priority Items
09:14:41 <gmann> dmellado: Thanks, that will be nice
09:15:06 <gmann> so we have priority items decided in Summit with their targeted deadline
09:15:14 <gmann> #link https://etherpad.openstack.org/p/mitaka-qa-priorities
09:15:28 <gmann> first one is Tempest Microversion support
09:15:34 <gmann> this missed the M1 :(
09:15:50 <gmann> but patches are in review and good to merge soon
09:16:05 <gmann> lets discuss this later
09:16:48 <gmann> Other service client  migration we will talk later in agenda
09:17:04 <gmann> jordanP: one is your
09:17:10 <gmann> #link Tempest Lib Migrations
09:17:16 <jordanP> yep
09:17:23 <gmann> jordanP: do you have anything on this
09:17:53 <jordanP> hummm now that andreaf and jlanoux have completed their work, I think I can start to thing about this migration
09:18:03 <jordanP> *think
09:18:09 <gmann> jordanP: cool
09:18:22 <jordanP> other than that I don"t know, it shouldn't be too hard
09:18:35 <jlanoux> jordanP: It is not completed yet. For sure, one change needs to get in, perhaps another one. Then, I'll take care of compute and remote.
09:18:39 <gmann> jlanoux: jordanP anything pending for ssh thing, need review etc
09:18:52 * andreaf sneaks in
09:18:54 <gmann> jlanoux: you have link >
09:19:04 <jlanoux> #link https://review.openstack.org/#/c/253444/
09:19:04 <gmann> andreaf: hi :)
09:19:18 <gmann> jlanoux: Thanks
09:19:25 <gmann> i will have look tomorrow
09:19:39 <jlanoux> And I don't know if I'll make the wrapper around compute and remote before the migration of after.
09:19:46 <jlanoux> *or
09:20:01 <jlanoux> gmann: thanks
09:20:11 <gmann> jlanoux: its better to make it before migration
09:20:27 <jordanP> jlanoux, count me in to review your work
09:20:30 <jlanoux> gmann: ok - I'll do that than
09:20:34 <jlanoux> jordanP: thanks
09:20:38 <jlanoux> *then
09:20:41 <gmann> jlanoux: after migration we have to maintain all published interfaces with backward compatibility mode
09:20:48 <gmann> jordanP: great  Thanks
09:20:54 <jlanoux> gmann: ok
09:21:00 <gmann> Next one is Tempest CLI Improvements
09:21:35 <gmann> #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/tempest-cli-improvements,n,z
09:21:41 <dwalleck> I'm about 2 hours from done with the "tempest run" spec
09:21:42 <gmann> There are lot work going on
09:21:54 <gmann> dwalleck: awesome.
09:22:04 <gmann> masayukig: you there?
09:22:23 <gmann> i think not :)
09:22:41 <gmann> Let's move on
09:22:48 <gmann> #topic Eslint-config-openstack review approval policy
09:23:03 <gmann> krotscheck : your turn
09:23:38 <gmann> not sure about this one, anyone has any idea
09:23:50 <dmellado> not really :\
09:23:57 <gmann> ok, let's move
09:24:09 <gmann> #topic Tempest
09:24:21 <gmann> first one is Service client from oomichi
09:24:29 <gmann> he updated the progress there
09:24:41 <gmann> #link  https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/consistent-service-method-names,n,z
09:24:52 <gmann> review progress and work is going good
09:25:05 <gmann> but still keep doing more and more reviews on those
09:25:34 <andreaf> I'm trying to reviews as many as possible of those
09:25:55 <gmann> another thing he mentioned is to update patches link on etherpad to track those correctly
09:25:56 <dmellado> I'd appretiate some reviews, once we deal with what to do with javelin ;)
09:25:57 <gmann> #link  https://etherpad.openstack.org/p/mitaka-tempest-service-clients
09:26:12 <gmann> andreaf: Thanks, it will make thing fast
09:26:27 <gmann> dmellado: sure, we will discuss on that next
09:26:51 <gmann> any query on service client work
09:27:13 <gmann> one i have about volume client doc string update
09:27:26 <gmann> as we have base client and v1 and v2 on separate classes/file
09:27:49 <gmann> I think updating v2 doc path is nice as jordanP done in QOS client
09:28:10 <jordanP> yeah v1 is already deprecated
09:28:15 <jordanP> soon to be removed
09:28:21 <gmann> because there might be question that those base client are for both
09:28:35 <gmann> jordanP: yea that is my consideration
09:28:41 <gmann> ok
09:29:05 <gmann> next is microversion testing support
09:29:09 <gmann> #link  https://review.openstack.org/#/q/status:open++branch:master+topic:bp/api-microversions-testing-support,n,z
09:29:30 <gmann> as i mentioned it has missed the M1 :( but i think that will be ok
09:29:46 <gmann> andreaf: i updated devstack and Config patches as per discussion
09:30:00 <gmann> andreaf: please review those
09:30:13 <andreaf> gmann: cool thanks, I will do after the meeting
09:30:19 <gmann> andreaf: Thanks
09:30:41 <gmann> also there are couple of patches there for schema conversioning etc
09:31:10 <gmann> and last one is to test Nova 2.2 version testing which can be checked to make sure all framework works fine
09:31:13 <gmann> #link #link  https://review.openstack.org/#/q/status:open++branch:master+topic:bp/api-microversions-testing-support,n,z
09:31:25 <gmann> reviews from everyone are welcome and helpful
09:32:30 <gmann> let's move on
09:32:35 <gmann> next one is Javelin script testing
09:32:53 <jordanP> what is javelin exactly ? :)
09:33:11 <dmellado> jordanP: I didn't know that too
09:33:14 <gmann> as we observed in keystone client separation work, this script was untested on gate
09:33:19 <dmellado> and was asking sdague and checking
09:33:28 <dmellado> and it seems that it was a script used in the beginning of grenade
09:33:31 <dmellado> to create resources
09:33:41 <gmann> jordanP: dmellado that was actually for creating/deleting resources which was used for upgrade testing
09:33:47 <dmellado> and it isn't used anymore at all, in sdague's words -in grenade-
09:34:16 <gmann> dmellado: yes, it was removed from grenade
09:34:24 <gmann> #link https://review.openstack.org/#/c/186559/
09:34:26 <jordanP> EmilienM, ping
09:34:32 <dmellado> so my point is: shouldn't we remove it as well from tempest if it's not going to be used?
09:34:34 <jordanP> hum it's night time for him
09:34:37 <gmann> I think this patch ^^
09:34:55 <gmann> dmellado: yea nice point
09:35:18 <gmann> dmellado: i had same discussion with oomichi and he also feels to remove it
09:35:30 <gmann> so if there is no more usage of this script then we can remove
09:35:31 <dmellado> +1 from me
09:35:51 <gmann> otherwise we have to modify its unit tests to cover more at deep level
09:35:51 <dmellado> I'll update the patch removing grenade+tests
09:36:09 <dmellado> if that's ok for everyone
09:36:13 <gmann> dmellado: you mean javelin script?
09:36:24 <jordanP> don't remove grenade ! :)
09:36:34 <gmann> jordanP: heh :)
09:36:35 <dmellado> gmann: yep, javelin script + test_javelin
09:36:54 <gmann> dmellado: awesome, please add mtreinish and sdague as reviewer
09:37:00 <gmann> if anything we are missing about that
09:37:01 <dmellado> gmann: will do
09:37:02 <andreaf> dmellado, gmann, jordanP: I think we may want to bring this to the ML first?
09:37:05 <gmann> dmellado: cool
09:37:07 <ylobankov> As far as I understood this script was used only for grenade. Now it is not used. So I think it is OK to remove the script.
09:37:15 <jordanP> that is wise indeed
09:37:18 <dmellado> andreaf: will do.
09:37:27 <jordanP> we might have hidden users
09:37:29 <gmann> #action dmellado to remove Javelin script and get feedback on review
09:37:39 <dmellado> I was speaking yesterday with sdague and he was ok, but I'll send a mail just in case
09:37:48 <gmann> andreaf: ok that is also nice way before going ahead on gerrit
09:37:54 <jordanP> yes
09:37:55 <dmellado> could you review the current patch? we can always modify it later on
09:38:05 <gmann> dmellado: thanks
09:38:26 <gmann> #action dmellado to first send ML about removal of Javelin from tempest
09:38:44 <gmann> jordanP: yea
09:39:04 <gmann> jordanP: may be someone using it explicitly
09:39:29 <gmann> ok next one
09:39:30 <gmann> Neutron LBAAS Scenario Tests
09:39:36 <gmann> dmellado: your turn please
09:39:56 <dmellado> Basically I'm not sure about the current scope on this and wanted to have feedback.
09:40:09 <dmellado> IIUC, the scenario tests were *not* going to be pulled from main tempest
09:40:16 <gmann> dmellado: actually they were removed from tempest
09:40:24 <dmellado> what's the idea for the LBAAS, FWAAS and so on?
09:40:27 <gmann> #link https://review.openstack.org/#/c/186559/
09:40:31 <dmellado> what's the status of the plugin interface for neutron?
09:40:43 <dmellado> are they not being covered in tempest jobs now, but in neutron?
09:40:54 <gmann> dmellado: i think your main concern is to run those along with tempest tests right?
09:41:01 <dmellado> gmann: exactly
09:41:20 <gmann> dmellado: yea so i will say they should have plugin there as you mentioned
09:41:31 <gmann> but not sure status of that
09:41:34 <dmellado> gmann: but it's the plugin interface for neutron done yet?
09:41:38 <dmellado> who was tackling that?
09:41:50 <gmann> dmellado: I do not know exactly
09:42:01 <dmellado> otherwise, it feels strange for me to remove then while the interface's not ready
09:42:10 <dmellado> s/then/them
09:42:45 <gmann> it was question from yfried  also many times
09:43:09 <gmann> dmellado: actually they removed to keep the ownership for those tests cases
09:43:15 <gmann> which can always be run from their repo
09:43:47 <gmann> but yea plugin support is always nice to make them run from single entity (Tempest)
09:43:56 <dmellado> but wouldn't that 'split' tempest ownership? I'm well aware of 'tests to its plrojects'
09:44:12 <dmellado> but always thought that as scenario tests were using multiple ones they'd stay within Tempest
09:44:39 <dmellado> it just feels strange to me, tbh
09:44:54 <dmellado> but I can check with dougw
09:45:28 <dmellado> I've also a question, so from now any new LBAAS test should go there? and how about FWAAS or designate?
09:45:32 <andreaf> dmellado: scenario tests for the core 6 services can be in tempest, as they are good integration tests and they run all the time
09:46:04 <dmellado> andreaf: wouldn't lbaas be a part of neutron? or am I just missing something?
09:46:10 <andreaf> dmellado: but lbaas and fwaas are kind of independent from neutron
09:46:17 <gmann> dmellado: yea scenario tests can be added in tempest as per Big tent scope
09:47:00 <gmann> dmellado: andreaf yea, they have their own repo and tests ownership etc
09:47:02 <andreaf> dmellado: it's kind of an extra feature, it's not something the other services would heavily rely on for the basic working
09:47:21 <dmellado> gmann: andreaf thanks
09:47:23 <gmann> dmellado: i remember removal things was for neutron advance services
09:47:32 <gmann> only
09:47:35 <dmellado> I assume that'll be the same for every neutron 'advance services'
09:47:40 <andreaf> so I think it makes sense for those to run on fwaas / lbaas / neutron changes only
09:47:45 <gmann> dmellado: we have all neutron tests
09:48:06 <gmann> dmellado: exactly for all advance services
09:48:13 <dmellado> I see,
09:48:27 <dmellado> so andreaf how are the fwaas/lbaas triggered then on the neutron changes?
09:48:36 <dmellado> do we have a triggered job for it?
09:48:58 <dmellado> or will they be relying on the so called pluggable interface, as Monasca and so on?
09:49:04 <andreaf> dmellado: at the moment they have use pre/post run hooks to setup things and run those tests
09:49:22 <dmellado> andreaf: ack, thanks for all the clarifications
09:49:41 <andreaf> dmellado: using a plugin would be nicer, I agree, but it's not there yet
09:49:58 <gmann> dmellado: but its not like they should not write plugin :)
09:50:06 <andreaf> as the plugin is a stable interface to add things into tempest, other things may break over time
09:50:08 <dmellado> andreaf: gmann thanks for the summary!
09:50:14 <gmann> dmellado:  how about writing for them
09:50:26 <gmann> andreaf: nice point
09:50:42 <dmellado> gmann: is there any patch that I could use as an example? If so I could tackle that
09:51:11 <gmann> dmellado: yea, i have couple of from Congress(my teammate worked on those)
09:51:25 <gmann> dmellado: and help on Ironic one which is in progress
09:51:30 <dmellado> then I'll ask you after the meeting ;)
09:51:38 <gmann> dmellado: i will provide the link on qa channel after meeting
09:51:44 <gmann> dmellado: sure, Thanks
09:51:46 <andreaf> dmellado, gmann: well the idea of plugins is to scale better so project teams can maintain their tests - so if we start doing it it's kind of the opposite direction :D but sure you can do that talk to the fwaas team
09:51:46 <dmellado> thanks gmann
09:52:13 <gmann> andreaf: yea i just pointed on dmellado :)
09:52:25 <dmellado> andreaf: I see :D
09:52:33 <gmann> anyways lets move on
09:53:00 <gmann> we will skip Devstack and grenade if nothing from anyone
09:53:39 <gmann> let's move on
09:53:43 <gmann> #topic OpenStack Health
09:53:50 <gmann> andreaf: do you have anything on this
09:54:24 <andreaf> gmann: well the forth page is now up and running :) the DB migration from uuid to int keys complete
09:54:58 <gmann> #link http://status.openstack.org/openstack-health/#/
09:55:01 <gmann> andreaf: cool
09:55:26 <andreaf> see for instance http://status.openstack.org/openstack-health/#/test/tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_update_with_few_fields
09:55:37 <gmann> nova is around 97% pass :)
09:56:07 <andreaf> we had for M1 to deliver group-by and filter-by capabilities, but that's still WIP
09:56:18 <gmann> andreaf: any failure example, can we see log from there?
09:56:23 <andreaf> group-by is likely to be the next main feature on the dashboard
09:56:45 <andreaf> gmann: no, attachments are not available yet
09:56:46 <gmann> andreaf: and what all category will be in group-by
09:56:55 <gmann> andreaf: ok
09:57:44 <andreaf> gmann: run metadata keys, even though not all of them will be shown, it's going to be configurable which to display and which not
09:57:59 <andreaf> gmann: metadata keys are basically ZULL_* parameters passed to jobs
09:58:17 <gmann> andreaf: ok
09:59:00 <gmann> andreaf: 1 basic question, for how many days data it load by default
09:59:11 <andreaf> if you scroll to the bottom of http://status.openstack.org/openstack-health/#/job/periodic-tempest-dsvm-full-test-accounts-master you can see for instance that object store tests are flacky with pre-prov creds
10:00:37 <andreaf> on the 3rd page (test details) it should be 1 day (because it's a lot of data), but if you select a periodic job it's more, 15days I think
10:00:47 <gmann> andreaf: where at bottom?
10:00:50 <andreaf> because periodic tests are less frequent
10:00:56 <andreaf> http://status.openstack.org/openstack-health/#/job/periodic-tempest-dsvm-full-test-accounts-master
10:01:12 <gmann> andreaf: yea because it took long time to load for me
10:01:30 <masayukig> sorry for late, it's almost end, though.
10:01:31 <gmann> anyways we will check it
10:01:35 <gmann> andreaf: Thanks a lot
10:01:41 <gmann> masayukig: hi :)
10:01:54 <gmann> time up now
10:01:58 <andreaf> gmann: you can also search the test name by ObjectACL
10:02:03 <gmann> #topic critical reviews
10:02:09 <gmann> andreaf: i see
10:02:21 <gmann> and i will check and ping you if any question
10:02:28 <gmann> any reviews from anyone?
10:02:45 <gmann> ok Let's end the meeting
10:02:51 <gmann> Thanks All
10:02:53 <masayukig> thanks :)
10:02:55 <dmellado> thanks! ;)
10:03:02 <gmann> #endmeeting