17:00:20 #startmeeting qa 17:00:21 Meeting started Thu Feb 4 17:00:20 2016 UTC and is due to finish in 60 minutes. The chair is andreaf. Information about MeetBot at http://wiki.debian.org/MeetBot. 17:00:22 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 17:00:24 The meeting name has been set to 'qa' 17:00:38 Agenda for today: #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_February_4th_2016_.281700_UTC.29 17:00:51 0/ 17:00:54 o/ 17:00:57 rather 17:01:02 Hello, who's here today? 17:01:13 o/ 17:01:14 hi dpaterson 17:01:15 hi 17:01:18 hi 17:02:03 sdague, dtroyer, afazekas, mkoderer: around? 17:02:21 ok, let's start 17:02:42 #topic QA Code spring 17:03:14 so I just wanted to remind folks about the upcoming sprint #link https://wiki.openstack.org/wiki/QA/CodeSprintMitakaBoston 17:03:57 and dpaterson added a comment on the agenda about the t-shirt size, if you are planning to attend let him know your preferred size 17:03:59 o/ 17:04:40 anything else on the sprint? 17:05:09 apart from the fact that I spelled it wrong in the topic :D 17:05:20 ok let's move on 17:05:32 #topic Specs Reviews 17:05:45 anyone needs info about Boston please feel free to ping me 17:05:51 david_paterson@dell.com 17:06:22 thanks dpaterson 17:06:33 #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z 17:06:48 does anyone have a spec to discuss? 17:06:54 * afazekas o/ 17:07:13 I put up the tempest run spec, I'd appreciate any feedback: https://review.openstack.org/#/c/269934/ 17:07:28 dwalleck: will take a look 17:07:45 It might be something easier to discuss in Boston too 17:07:52 dwalleck: I have it on my todo list 17:08:00 Thanks! 17:08:10 dwalleck: yes that's a good topic for Boston 17:08:55 dwalleck: you may want to fix the docs gate failure 17:09:46 #info "tempest run" spec to be discussed during the sprint in boston 17:09:53 #link https://review.openstack.org/#/c/269934/ 17:10:14 Anything else on specs? 17:11:17 I need to update https://review.openstack.org/#/c/173334/ based on mtreinish input, hopefully it will be ready to go afterwards 17:11:28 #link https://review.openstack.org/#/c/173334/ tempest test resources 17:11:49 ok let's move on 17:12:01 #topic Priority Items 17:12:04 #link https://etherpad.openstack.org/p/mitaka-qa-priorities 17:12:35 is there any update on priority items for Mitaka? 17:12:45 hi, I actually do have one spec I'd like to discuss if possible: https://review.openstack.org/#/c/274205/ 17:12:54 #link https://review.openstack.org/#/c/274205/ 17:13:02 ccneill sure 17:13:14 #topic Specs Reviews 17:13:29 ccneill: the floor is yours 17:13:45 so I've discussed this back and forth with mtreinish in late '15, but I'd like to figure out what I can do to 1) better explain, 2) better document, better test, etc. 17:14:50 the basic premise is providing utilities that can be used by projects to write their own functional security "fuzz" tests 17:14:55 ccneill: a highlevel interface description would be useful. What the calls are and what they would return 17:14:57 i.e. sending XSS payloads, SQL injection payloads, etc. 17:15:31 ccneill: it sounds reasonable to me 17:15:32 dpaterson: sounds reasonable. would that be best to add to the spec, or in docstrings in the code? 17:15:57 ccneill: spec first them migrate to docstrings in code 17:15:58 I also probably need to add a few more unit tests to more fully test the functionality 17:16:01 ccneill: that wouldn't be for tempest testing right? 17:16:35 ccneill: yes I was about to day how are you going to keep that tested - you'll need unit test coverage for that 17:16:35 andreaf: the backstory is I already wrote tests for designate and barbican using these utilities, and they didn't want to house them in their repo 17:16:58 andreaf: so they may have a place in tempest, but my primary motivation is just to provide the tools if people want to use them, and not to have to ship them with every project individually 17:18:50 ccneill: I've been meaning to have a look at that spec. I'll put it for on my to do list for today 17:18:51 there are 2 relatively straightforward parts, for anyone who doesn't want to dig through the code right now: datasets, and methods for "repackaging" them based on the testing framework used by the project, and very simplistic verification that a request didn't return a response with an obvious indicator of vulnerability 17:18:57 dwalleck: awesome, thanks! 17:19:26 ccneill: ok, if you could details about the interface and testing it would be great 17:19:37 ccneill: I'll add it to my review list as well 17:19:54 andreaf: great, thanks 17:20:02 I'll try to add some more details today 17:21:03 ccneill: cool thanks 17:21:46 ok, anything else on specs? 17:22:41 #topic Mitaka priorities 17:22:44 #link https://etherpad.openstack.org/p/mitaka-qa-priorities 17:23:01 any update on priorities? 17:23:40 I believe all the cmd modules have been migrated over to use cliff lib for cli. 17:24:00 I started adding unit tests for account_generator which are here: 17:24:11 #link https://review.openstack.org/#/c/270576 17:24:36 I've not had a chance to complete, if anyone feels like jumping in that would be great. 17:25:17 dpaterson: nice work thanks - is there an etherpad you're using to track this work? 17:25:31 dpaterson: so someone could take ownership of items if they wanted to 17:25:34 It's linked to from mitaka priorites 17:26:03 dpaterson: right, thanks #link https://etherpad.openstack.org/p/tempest-cli-improvements 17:26:51 regarding (9) resource configuration, it won't make M3 I guess as it's still at the spec 17:27:36 regarding (10) ssh auth bp, it's very close to be finished - we may open a follow-up one for items left to be cleaned-up 17:28:50 ok, next topic 17:28:51 #topic Tempest 17:29:34 the only item I had on the agenda for this topic is the "ideas for refactor" which was discussed earlier 17:29:55 I just wanted to share the link to the etherpad so people can contribute to it 17:30:13 #link https://etherpad.openstack.org/p/tempest-refactor-ideas 17:30:15 andreaf: Is that list open for other folks to add items? 17:30:59 dwalleck: yes if you have good ideas related to the topic for sure 17:31:25 andreaf: Cool, thanks 17:31:55 and if anyone has any concern on the topics there, please bring them up at the QA meetings as well 17:32:23 I have one item to refactor probably. What about scenario tests? 17:32:42 I think we should refactor them 17:33:12 ylobankov: yes I believe some of the topics in that list are related to scenario tests as well 17:33:50 Anyway I would like to add this idea to the etherpad :) 17:34:04 ylobankov: some of the scenario tests are rather convoluted, the code has many many layers of abstractions which makes debugging and changing the code difficult 17:35:31 andreaf: So that is why we should refactor that code to make things easier 17:36:07 ylobankov: please go ahead, and please be specific on the pain points that require refactor 17:36:35 ylobankov: the main objectives are: make it easier to contribute tests, make it easier to debug them (during test development and in the gate) and reduce technical debt 17:37:15 ylobankov: well we don't want to scare off new contributors and mostly the code must be debuggable - a failure in the tests may have a big impact on the gate so we need the code to be debuggable 17:38:01 andreaf: I absolutely agree with you 17:38:23 ylobankov: also we do have unit test coverage for the common parts of the framework, but if test code becomes too complex it may become difficult to separate failures in the tests from actual failures 17:39:05 ylobankov: in my personal opinion it's a fine balance we need to find, but now we're are off in the direction of complexity 17:39:45 anyways, I think we need to move on 17:39:45 #topic DevStack + Grenade 17:39:53 sdague, dtroyer: around? 17:40:11 anything on devstack or grenade? 17:41:17 ok, let's move on 17:41:24 #topic Critical Reviews 17:41:41 Any critical review to highlight? 17:43:26 ok looks like not 17:43:45 let's open the floor for discussion then 17:43:57 #topic Open Discussion 17:45:20 ok, it there's nothing else I guess we'll end here 17:45:32 thanks everyone 17:45:35 thanks andrea 17:45:39 thanks all 17:45:42 andreaf: Thanks! 17:45:43 o/ 17:45:53 thanks all 17:45:54 o/ 17:45:56 #endmeeting