09:00:42 <gmann> #startmeeting qa 09:00:43 <openstack> Meeting started Thu Dec 10 09:00:42 2015 UTC and is due to finish in 60 minutes. The chair is gmann. Information about MeetBot at http://wiki.debian.org/MeetBot. 09:00:44 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 09:00:46 <openstack> The meeting name has been set to 'qa' 09:00:57 <gmann> hi, who's here today? 09:01:01 <jlanoux> o/ 09:01:02 <dwalleck> o/ 09:01:02 <dmellado> Hi gmann o/ 09:01:08 <gmann> dmellado: hi 09:01:14 <gmann> jlanoux: hi 09:01:34 <jlanoux> hi gmann 09:01:43 <gmann> let's for min to join more people 09:01:47 <dmellado> sure 09:02:07 <ylobankov> hi 09:02:28 <gmann> ylobankov: hi 09:02:44 <gmann> ok let's start 09:02:51 <gmann> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_December_10th_2015_.280900_UTC.29 09:02:59 <gmann> && Today meeting agenda 09:03:18 <gmann> #topic Specs Reviews 09:03:38 <gmann> we have most of spec under review 09:03:41 <gmann> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z 09:04:15 <gmann> andrea updated tempest-resource spec, good to review that 09:04:18 <gmann> #link https://review.openstack.org/#/c/173334/ 09:04:19 <jordanP> gi 09:04:20 <jordanP> hi 09:04:27 <gmann> jordanP: hi 09:04:27 <dmellado> morning jordanP ;) 09:04:50 <gmann> There are other 2 in list of agenda 09:04:53 <gmann> # https://review.openstack.org/#/c/230183/ 09:05:17 <gmann> I have not gone through it yet, but its under review 09:05:25 <gmann> anyone has anything on this to discuss 09:05:58 <gmann> anyways next one is list plugin one 09:06:04 <gmann> #link https://review.openstack.org/#/c/247575/ 09:06:13 <dmellado> I haven't gone over there yet, but I'd like to somehow couple the config script inners with the bp-resources 09:06:20 <gmann> this one was straight forward 09:06:22 <dmellado> I'll update it when I've some time 09:06:29 <gmann> dmellado: cool, Thanks 09:06:38 <gmann> plugin list one is merged. 09:06:54 <jordanP> yep 09:07:11 <gmann> I also found difficulty while debugging ironic plugin which was in progress 09:07:23 <gmann> jordanP: yea it will be nice addition 09:07:34 <gmann> anything on spec 09:07:58 <gmann> or any other spec anyone want to bring up for discussion 09:08:06 <dwalleck> I have one - https://review.openstack.org/#/c/255023/ 09:08:27 <gmann> dwalleck: sure, please go ahead 09:08:54 <dwalleck> There's been a lot of discussion about how the remote client is tied to very generic Linux implementation. This spec allows for an extendable implementation 09:09:30 <dwalleck> I know there's some work around this in the resources spec, but I don't think it will go far enough to solve the problem. I put this up as another possibility to consider 09:09:54 <gmann> dwalleck: i see, so it will be having drives for each connectivity protocol ? 09:10:09 <jordanP> dwalleck, I like this spec. The current linux client and implementation is a bit all over the place, a refactor and architecture change would be good 09:10:23 <jordanP> with more consistency 09:10:25 <dmellado> +also adding support for the dreaded windows 09:10:27 <dmellado> xD 09:10:43 <gmann> jordanP: yea looks like nice one 09:10:45 <jordanP> but, are we *that* interested in testing something else than Linux ? 09:10:49 <dwalleck> gmann: Right. A driver per protocol, and the remote client implementation picks the driver it needs 09:10:57 <jordanP> I mean it's gonna be some work, you sure it's worth it ? 09:11:15 <gmann> dwalleck: 1 question does this one contain how our tests goind to switch between those 09:11:18 <dwalleck> The dreaded windows work is done :D If you check one of the bottom links, I link to an external implementation 09:11:28 <gmann> dwalleck: ok, so remote client will pick up dynamically 09:11:29 <dmellado> dwalleck: jordanP: also, will be the gates running anything else? otherwise this might end up being outdated in case it gets implemented 09:11:50 <gmann> dmellado: nice point 09:11:57 <dwalleck> gmann: My thought was to have it be a property in the resource file for the image 09:12:16 <gmann> dwalleck: i see. 09:12:21 <gmann> dwalleck: Thanks 09:12:34 <dwalleck> dmellado: We run a lot of Windows gates at Rackspace. But yes, testing of multiple drivers is one of the possible pain points 09:12:54 <jordanP> well at least if it's unit tested, it's a good start 09:13:16 <dwalleck> But I'd be more than glad to help with that. I also suggest making the remote client extensions plugins, which would put the maintenance back to the developer of the plugin 09:13:23 <dmellado> +1 from me, if we can couple it with unit tests 09:13:50 <gmann> and if missing one can be added as we planed for resource things 09:13:59 <gmann> anyways its good one for review and lets start reviewing it and have comment/suggestion there 09:14:05 <gmann> dwalleck: Thanks . 09:14:11 <gmann> let's move on 09:14:15 <dwalleck> thanks everyone :-) 09:14:18 <dmellado> dwalleck: I'll take a look later and add comments, thanks ;) 09:14:33 <gmann> #topic Priority Items 09:14:41 <gmann> dmellado: Thanks, that will be nice 09:15:06 <gmann> so we have priority items decided in Summit with their targeted deadline 09:15:14 <gmann> #link https://etherpad.openstack.org/p/mitaka-qa-priorities 09:15:28 <gmann> first one is Tempest Microversion support 09:15:34 <gmann> this missed the M1 :( 09:15:50 <gmann> but patches are in review and good to merge soon 09:16:05 <gmann> lets discuss this later 09:16:48 <gmann> Other service client migration we will talk later in agenda 09:17:04 <gmann> jordanP: one is your 09:17:10 <gmann> #link Tempest Lib Migrations 09:17:16 <jordanP> yep 09:17:23 <gmann> jordanP: do you have anything on this 09:17:53 <jordanP> hummm now that andreaf and jlanoux have completed their work, I think I can start to thing about this migration 09:18:03 <jordanP> *think 09:18:09 <gmann> jordanP: cool 09:18:22 <jordanP> other than that I don"t know, it shouldn't be too hard 09:18:35 <jlanoux> jordanP: It is not completed yet. For sure, one change needs to get in, perhaps another one. Then, I'll take care of compute and remote. 09:18:39 <gmann> jlanoux: jordanP anything pending for ssh thing, need review etc 09:18:52 * andreaf sneaks in 09:18:54 <gmann> jlanoux: you have link > 09:19:04 <jlanoux> #link https://review.openstack.org/#/c/253444/ 09:19:04 <gmann> andreaf: hi :) 09:19:18 <gmann> jlanoux: Thanks 09:19:25 <gmann> i will have look tomorrow 09:19:39 <jlanoux> And I don't know if I'll make the wrapper around compute and remote before the migration of after. 09:19:46 <jlanoux> *or 09:20:01 <jlanoux> gmann: thanks 09:20:11 <gmann> jlanoux: its better to make it before migration 09:20:27 <jordanP> jlanoux, count me in to review your work 09:20:30 <jlanoux> gmann: ok - I'll do that than 09:20:34 <jlanoux> jordanP: thanks 09:20:38 <jlanoux> *then 09:20:41 <gmann> jlanoux: after migration we have to maintain all published interfaces with backward compatibility mode 09:20:48 <gmann> jordanP: great Thanks 09:20:54 <jlanoux> gmann: ok 09:21:00 <gmann> Next one is Tempest CLI Improvements 09:21:35 <gmann> #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/tempest-cli-improvements,n,z 09:21:41 <dwalleck> I'm about 2 hours from done with the "tempest run" spec 09:21:42 <gmann> There are lot work going on 09:21:54 <gmann> dwalleck: awesome. 09:22:04 <gmann> masayukig: you there? 09:22:23 <gmann> i think not :) 09:22:41 <gmann> Let's move on 09:22:48 <gmann> #topic Eslint-config-openstack review approval policy 09:23:03 <gmann> krotscheck : your turn 09:23:38 <gmann> not sure about this one, anyone has any idea 09:23:50 <dmellado> not really :\ 09:23:57 <gmann> ok, let's move 09:24:09 <gmann> #topic Tempest 09:24:21 <gmann> first one is Service client from oomichi 09:24:29 <gmann> he updated the progress there 09:24:41 <gmann> #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/consistent-service-method-names,n,z 09:24:52 <gmann> review progress and work is going good 09:25:05 <gmann> but still keep doing more and more reviews on those 09:25:34 <andreaf> I'm trying to reviews as many as possible of those 09:25:55 <gmann> another thing he mentioned is to update patches link on etherpad to track those correctly 09:25:56 <dmellado> I'd appretiate some reviews, once we deal with what to do with javelin ;) 09:25:57 <gmann> #link https://etherpad.openstack.org/p/mitaka-tempest-service-clients 09:26:12 <gmann> andreaf: Thanks, it will make thing fast 09:26:27 <gmann> dmellado: sure, we will discuss on that next 09:26:51 <gmann> any query on service client work 09:27:13 <gmann> one i have about volume client doc string update 09:27:26 <gmann> as we have base client and v1 and v2 on separate classes/file 09:27:49 <gmann> I think updating v2 doc path is nice as jordanP done in QOS client 09:28:10 <jordanP> yeah v1 is already deprecated 09:28:15 <jordanP> soon to be removed 09:28:21 <gmann> because there might be question that those base client are for both 09:28:35 <gmann> jordanP: yea that is my consideration 09:28:41 <gmann> ok 09:29:05 <gmann> next is microversion testing support 09:29:09 <gmann> #link https://review.openstack.org/#/q/status:open++branch:master+topic:bp/api-microversions-testing-support,n,z 09:29:30 <gmann> as i mentioned it has missed the M1 :( but i think that will be ok 09:29:46 <gmann> andreaf: i updated devstack and Config patches as per discussion 09:30:00 <gmann> andreaf: please review those 09:30:13 <andreaf> gmann: cool thanks, I will do after the meeting 09:30:19 <gmann> andreaf: Thanks 09:30:41 <gmann> also there are couple of patches there for schema conversioning etc 09:31:10 <gmann> and last one is to test Nova 2.2 version testing which can be checked to make sure all framework works fine 09:31:13 <gmann> #link #link https://review.openstack.org/#/q/status:open++branch:master+topic:bp/api-microversions-testing-support,n,z 09:31:25 <gmann> reviews from everyone are welcome and helpful 09:32:30 <gmann> let's move on 09:32:35 <gmann> next one is Javelin script testing 09:32:53 <jordanP> what is javelin exactly ? :) 09:33:11 <dmellado> jordanP: I didn't know that too 09:33:14 <gmann> as we observed in keystone client separation work, this script was untested on gate 09:33:19 <dmellado> and was asking sdague and checking 09:33:28 <dmellado> and it seems that it was a script used in the beginning of grenade 09:33:31 <dmellado> to create resources 09:33:41 <gmann> jordanP: dmellado that was actually for creating/deleting resources which was used for upgrade testing 09:33:47 <dmellado> and it isn't used anymore at all, in sdague's words -in grenade- 09:34:16 <gmann> dmellado: yes, it was removed from grenade 09:34:24 <gmann> #link https://review.openstack.org/#/c/186559/ 09:34:26 <jordanP> EmilienM, ping 09:34:32 <dmellado> so my point is: shouldn't we remove it as well from tempest if it's not going to be used? 09:34:34 <jordanP> hum it's night time for him 09:34:37 <gmann> I think this patch ^^ 09:34:55 <gmann> dmellado: yea nice point 09:35:18 <gmann> dmellado: i had same discussion with oomichi and he also feels to remove it 09:35:30 <gmann> so if there is no more usage of this script then we can remove 09:35:31 <dmellado> +1 from me 09:35:51 <gmann> otherwise we have to modify its unit tests to cover more at deep level 09:35:51 <dmellado> I'll update the patch removing grenade+tests 09:36:09 <dmellado> if that's ok for everyone 09:36:13 <gmann> dmellado: you mean javelin script? 09:36:24 <jordanP> don't remove grenade ! :) 09:36:34 <gmann> jordanP: heh :) 09:36:35 <dmellado> gmann: yep, javelin script + test_javelin 09:36:54 <gmann> dmellado: awesome, please add mtreinish and sdague as reviewer 09:37:00 <gmann> if anything we are missing about that 09:37:01 <dmellado> gmann: will do 09:37:02 <andreaf> dmellado, gmann, jordanP: I think we may want to bring this to the ML first? 09:37:05 <gmann> dmellado: cool 09:37:07 <ylobankov> As far as I understood this script was used only for grenade. Now it is not used. So I think it is OK to remove the script. 09:37:15 <jordanP> that is wise indeed 09:37:18 <dmellado> andreaf: will do. 09:37:27 <jordanP> we might have hidden users 09:37:29 <gmann> #action dmellado to remove Javelin script and get feedback on review 09:37:39 <dmellado> I was speaking yesterday with sdague and he was ok, but I'll send a mail just in case 09:37:48 <gmann> andreaf: ok that is also nice way before going ahead on gerrit 09:37:54 <jordanP> yes 09:37:55 <dmellado> could you review the current patch? we can always modify it later on 09:38:05 <gmann> dmellado: thanks 09:38:26 <gmann> #action dmellado to first send ML about removal of Javelin from tempest 09:38:44 <gmann> jordanP: yea 09:39:04 <gmann> jordanP: may be someone using it explicitly 09:39:29 <gmann> ok next one 09:39:30 <gmann> Neutron LBAAS Scenario Tests 09:39:36 <gmann> dmellado: your turn please 09:39:56 <dmellado> Basically I'm not sure about the current scope on this and wanted to have feedback. 09:40:09 <dmellado> IIUC, the scenario tests were *not* going to be pulled from main tempest 09:40:16 <gmann> dmellado: actually they were removed from tempest 09:40:24 <dmellado> what's the idea for the LBAAS, FWAAS and so on? 09:40:27 <gmann> #link https://review.openstack.org/#/c/186559/ 09:40:31 <dmellado> what's the status of the plugin interface for neutron? 09:40:43 <dmellado> are they not being covered in tempest jobs now, but in neutron? 09:40:54 <gmann> dmellado: i think your main concern is to run those along with tempest tests right? 09:41:01 <dmellado> gmann: exactly 09:41:20 <gmann> dmellado: yea so i will say they should have plugin there as you mentioned 09:41:31 <gmann> but not sure status of that 09:41:34 <dmellado> gmann: but it's the plugin interface for neutron done yet? 09:41:38 <dmellado> who was tackling that? 09:41:50 <gmann> dmellado: I do not know exactly 09:42:01 <dmellado> otherwise, it feels strange for me to remove then while the interface's not ready 09:42:10 <dmellado> s/then/them 09:42:45 <gmann> it was question from yfried also many times 09:43:09 <gmann> dmellado: actually they removed to keep the ownership for those tests cases 09:43:15 <gmann> which can always be run from their repo 09:43:47 <gmann> but yea plugin support is always nice to make them run from single entity (Tempest) 09:43:56 <dmellado> but wouldn't that 'split' tempest ownership? I'm well aware of 'tests to its plrojects' 09:44:12 <dmellado> but always thought that as scenario tests were using multiple ones they'd stay within Tempest 09:44:39 <dmellado> it just feels strange to me, tbh 09:44:54 <dmellado> but I can check with dougw 09:45:28 <dmellado> I've also a question, so from now any new LBAAS test should go there? and how about FWAAS or designate? 09:45:32 <andreaf> dmellado: scenario tests for the core 6 services can be in tempest, as they are good integration tests and they run all the time 09:46:04 <dmellado> andreaf: wouldn't lbaas be a part of neutron? or am I just missing something? 09:46:10 <andreaf> dmellado: but lbaas and fwaas are kind of independent from neutron 09:46:17 <gmann> dmellado: yea scenario tests can be added in tempest as per Big tent scope 09:47:00 <gmann> dmellado: andreaf yea, they have their own repo and tests ownership etc 09:47:02 <andreaf> dmellado: it's kind of an extra feature, it's not something the other services would heavily rely on for the basic working 09:47:21 <dmellado> gmann: andreaf thanks 09:47:23 <gmann> dmellado: i remember removal things was for neutron advance services 09:47:32 <gmann> only 09:47:35 <dmellado> I assume that'll be the same for every neutron 'advance services' 09:47:40 <andreaf> so I think it makes sense for those to run on fwaas / lbaas / neutron changes only 09:47:45 <gmann> dmellado: we have all neutron tests 09:48:06 <gmann> dmellado: exactly for all advance services 09:48:13 <dmellado> I see, 09:48:27 <dmellado> so andreaf how are the fwaas/lbaas triggered then on the neutron changes? 09:48:36 <dmellado> do we have a triggered job for it? 09:48:58 <dmellado> or will they be relying on the so called pluggable interface, as Monasca and so on? 09:49:04 <andreaf> dmellado: at the moment they have use pre/post run hooks to setup things and run those tests 09:49:22 <dmellado> andreaf: ack, thanks for all the clarifications 09:49:41 <andreaf> dmellado: using a plugin would be nicer, I agree, but it's not there yet 09:49:58 <gmann> dmellado: but its not like they should not write plugin :) 09:50:06 <andreaf> as the plugin is a stable interface to add things into tempest, other things may break over time 09:50:08 <dmellado> andreaf: gmann thanks for the summary! 09:50:14 <gmann> dmellado: how about writing for them 09:50:26 <gmann> andreaf: nice point 09:50:42 <dmellado> gmann: is there any patch that I could use as an example? If so I could tackle that 09:51:11 <gmann> dmellado: yea, i have couple of from Congress(my teammate worked on those) 09:51:25 <gmann> dmellado: and help on Ironic one which is in progress 09:51:30 <dmellado> then I'll ask you after the meeting ;) 09:51:38 <gmann> dmellado: i will provide the link on qa channel after meeting 09:51:44 <gmann> dmellado: sure, Thanks 09:51:46 <andreaf> dmellado, gmann: well the idea of plugins is to scale better so project teams can maintain their tests - so if we start doing it it's kind of the opposite direction :D but sure you can do that talk to the fwaas team 09:51:46 <dmellado> thanks gmann 09:52:13 <gmann> andreaf: yea i just pointed on dmellado :) 09:52:25 <dmellado> andreaf: I see :D 09:52:33 <gmann> anyways lets move on 09:53:00 <gmann> we will skip Devstack and grenade if nothing from anyone 09:53:39 <gmann> let's move on 09:53:43 <gmann> #topic OpenStack Health 09:53:50 <gmann> andreaf: do you have anything on this 09:54:24 <andreaf> gmann: well the forth page is now up and running :) the DB migration from uuid to int keys complete 09:54:58 <gmann> #link http://status.openstack.org/openstack-health/#/ 09:55:01 <gmann> andreaf: cool 09:55:26 <andreaf> see for instance http://status.openstack.org/openstack-health/#/test/tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_group_update_with_few_fields 09:55:37 <gmann> nova is around 97% pass :) 09:56:07 <andreaf> we had for M1 to deliver group-by and filter-by capabilities, but that's still WIP 09:56:18 <gmann> andreaf: any failure example, can we see log from there? 09:56:23 <andreaf> group-by is likely to be the next main feature on the dashboard 09:56:45 <andreaf> gmann: no, attachments are not available yet 09:56:46 <gmann> andreaf: and what all category will be in group-by 09:56:55 <gmann> andreaf: ok 09:57:44 <andreaf> gmann: run metadata keys, even though not all of them will be shown, it's going to be configurable which to display and which not 09:57:59 <andreaf> gmann: metadata keys are basically ZULL_* parameters passed to jobs 09:58:17 <gmann> andreaf: ok 09:59:00 <gmann> andreaf: 1 basic question, for how many days data it load by default 09:59:11 <andreaf> if you scroll to the bottom of http://status.openstack.org/openstack-health/#/job/periodic-tempest-dsvm-full-test-accounts-master you can see for instance that object store tests are flacky with pre-prov creds 10:00:37 <andreaf> on the 3rd page (test details) it should be 1 day (because it's a lot of data), but if you select a periodic job it's more, 15days I think 10:00:47 <gmann> andreaf: where at bottom? 10:00:50 <andreaf> because periodic tests are less frequent 10:00:56 <andreaf> http://status.openstack.org/openstack-health/#/job/periodic-tempest-dsvm-full-test-accounts-master 10:01:12 <gmann> andreaf: yea because it took long time to load for me 10:01:30 <masayukig> sorry for late, it's almost end, though. 10:01:31 <gmann> anyways we will check it 10:01:35 <gmann> andreaf: Thanks a lot 10:01:41 <gmann> masayukig: hi :) 10:01:54 <gmann> time up now 10:01:58 <andreaf> gmann: you can also search the test name by ObjectACL 10:02:03 <gmann> #topic critical reviews 10:02:09 <gmann> andreaf: i see 10:02:21 <gmann> and i will check and ping you if any question 10:02:28 <gmann> any reviews from anyone? 10:02:45 <gmann> ok Let's end the meeting 10:02:51 <gmann> Thanks All 10:02:53 <masayukig> thanks :) 10:02:55 <dmellado> thanks! ;) 10:03:02 <gmann> #endmeeting