17:00:20 <andreaf> #startmeeting qa
17:00:21 <openstack> Meeting started Thu Feb  4 17:00:20 2016 UTC and is due to finish in 60 minutes.  The chair is andreaf. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:24 <openstack> The meeting name has been set to 'qa'
17:00:38 <andreaf> Agenda for today: #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_February_4th_2016_.281700_UTC.29
17:00:51 <dpaterson> 0/
17:00:54 <dpaterson> o/
17:00:57 <dpaterson> rather
17:01:02 <andreaf> Hello, who's here today?
17:01:13 <ccneill> o/
17:01:14 <andreaf> hi dpaterson
17:01:15 <ylobankov> hi
17:01:18 <dpaterson> hi
17:02:03 <andreaf> sdague, dtroyer, afazekas, mkoderer: around?
17:02:21 <andreaf> ok, let's start
17:02:42 <andreaf> #topic QA Code spring
17:03:14 <andreaf> so I just wanted to remind folks about the upcoming sprint #link https://wiki.openstack.org/wiki/QA/CodeSprintMitakaBoston
17:03:57 <andreaf> and dpaterson added a comment on the agenda about the t-shirt size, if you are planning to attend let him know your preferred size
17:03:59 <dwalleck> o/
17:04:40 <andreaf> anything else on the sprint?
17:05:09 <andreaf> apart from the fact that I spelled it wrong in the topic :D
17:05:20 <andreaf> ok let's move on
17:05:32 <andreaf> #topic Specs Reviews
17:05:45 <dpaterson> anyone needs info about Boston please feel free to ping me
17:05:51 <dpaterson> david_paterson@dell.com
17:06:22 <andreaf> thanks dpaterson
17:06:33 <andreaf> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
17:06:48 <andreaf> does anyone have a spec to discuss?
17:06:54 * afazekas o/
17:07:13 <dwalleck> I put up the tempest run spec, I'd appreciate any feedback: https://review.openstack.org/#/c/269934/
17:07:28 <dpaterson> dwalleck: will take a look
17:07:45 <dwalleck> It might be something easier to discuss in Boston too
17:07:52 <andreaf> dwalleck: I have it on my todo list
17:08:00 <dwalleck> Thanks!
17:08:10 <andreaf> dwalleck: yes that's a good topic for Boston
17:08:55 <andreaf> dwalleck: you may want to fix the docs gate failure
17:09:46 <andreaf> #info "tempest run" spec to be discussed during the sprint in boston
17:09:53 <andreaf> #link https://review.openstack.org/#/c/269934/
17:10:14 <andreaf> Anything else on specs?
17:11:17 <andreaf> I need to update https://review.openstack.org/#/c/173334/ based on mtreinish input, hopefully it will be ready to go afterwards
17:11:28 <andreaf> #link https://review.openstack.org/#/c/173334/ tempest test resources
17:11:49 <andreaf> ok let's move on
17:12:01 <andreaf> #topic Priority Items
17:12:04 <andreaf> #link https://etherpad.openstack.org/p/mitaka-qa-priorities
17:12:35 <andreaf> is there any update on priority items for Mitaka?
17:12:45 <ccneill> hi, I actually do have one spec I'd like to discuss if possible: https://review.openstack.org/#/c/274205/
17:12:54 <ccneill> #link https://review.openstack.org/#/c/274205/
17:13:02 <andreaf> ccneill sure
17:13:14 <andreaf> #topic Specs Reviews
17:13:29 <andreaf> ccneill: the floor is yours
17:13:45 <ccneill> so I've discussed this back and forth with mtreinish in late '15, but I'd like to figure out what I can do to 1) better explain, 2) better document, better test, etc.
17:14:50 <ccneill> the basic premise is providing utilities that can be used by projects to write their own functional security "fuzz" tests
17:14:55 <dpaterson> ccneill: a highlevel interface description would be useful.  What the calls are and what they would return
17:14:57 <ccneill> i.e. sending XSS payloads, SQL injection payloads, etc.
17:15:31 <andreaf> ccneill: it sounds reasonable to me
17:15:32 <ccneill> dpaterson: sounds reasonable. would that be best to add to the spec, or in docstrings in the code?
17:15:57 <dpaterson> ccneill: spec first them migrate to docstrings in code
17:15:58 <ccneill> I also probably need to add a few more unit tests to more fully test the functionality
17:16:01 <andreaf> ccneill: that wouldn't be for tempest testing right?
17:16:35 <andreaf> ccneill: yes I was about to day how are you going to keep that tested - you'll need unit test coverage for that
17:16:35 <ccneill> andreaf: the backstory is I already wrote tests for designate and barbican using these utilities, and they didn't want to house them in their repo
17:16:58 <ccneill> andreaf: so they may have a place in tempest, but my primary motivation is just to provide the tools if people want to use them, and not to have to ship them with every project individually
17:18:50 <dwalleck> ccneill: I've been meaning to have a look at that spec. I'll put it for on my to do list for today
17:18:51 <ccneill> there are 2 relatively straightforward parts, for anyone who doesn't want to dig through the code right now: datasets, and methods for "repackaging" them based on the testing framework used by the project, and very simplistic verification that a request didn't return a response with an obvious indicator of vulnerability
17:18:57 <ccneill> dwalleck: awesome, thanks!
17:19:26 <andreaf> ccneill: ok, if you could details about the interface and testing it would be great
17:19:37 <andreaf> ccneill: I'll add it to my review list as well
17:19:54 <ccneill> andreaf: great, thanks
17:20:02 <ccneill> I'll try to add some more details today
17:21:03 <andreaf> ccneill: cool thanks
17:21:46 <andreaf> ok, anything else on specs?
17:22:41 <andreaf> #topic Mitaka priorities
17:22:44 <andreaf> #link https://etherpad.openstack.org/p/mitaka-qa-priorities
17:23:01 <andreaf> any update on priorities?
17:23:40 <dpaterson> I believe all the cmd modules have been migrated over to use cliff lib for cli.
17:24:00 <dpaterson> I started adding unit tests for account_generator which are here:
17:24:11 <dpaterson> #link https://review.openstack.org/#/c/270576
17:24:36 <dpaterson> I've not had a chance to complete, if anyone feels like jumping in that would be great.
17:25:17 <andreaf> dpaterson: nice work thanks - is there an etherpad you're using to track this work?
17:25:31 <andreaf> dpaterson: so someone could take ownership of items if they wanted to
17:25:34 <dpaterson> It's linked to from mitaka priorites
17:26:03 <andreaf> dpaterson: right, thanks #link https://etherpad.openstack.org/p/tempest-cli-improvements
17:26:51 <andreaf> regarding (9) resource configuration, it won't make M3 I guess as it's still at the spec
17:27:36 <andreaf> regarding (10) ssh auth bp, it's very close to be finished - we may open a follow-up one for items left to be cleaned-up
17:28:50 <andreaf> ok, next topic
17:28:51 <andreaf> #topic Tempest
17:29:34 <andreaf> the only item I had on the agenda for this topic is the "ideas for refactor" which was discussed earlier
17:29:55 <andreaf> I just wanted to share the link to the etherpad so people can contribute to it
17:30:13 <andreaf> #link https://etherpad.openstack.org/p/tempest-refactor-ideas
17:30:15 <dwalleck> andreaf: Is that list open for other folks to add items?
17:30:59 <andreaf> dwalleck: yes if you have good ideas related to the topic for sure
17:31:25 <dwalleck> andreaf: Cool, thanks
17:31:55 <andreaf> and if anyone has any concern on the topics there, please bring them up at the QA meetings as well
17:32:23 <ylobankov> I have one item to refactor probably. What about scenario tests?
17:32:42 <ylobankov> I think we should refactor them
17:33:12 <andreaf> ylobankov: yes I believe some of the topics in that list are related to scenario tests as well
17:33:50 <ylobankov> Anyway I would like to add this idea to the etherpad :)
17:34:04 <andreaf> ylobankov: some of the scenario tests are rather convoluted, the code has many many layers of abstractions which makes debugging and changing the code difficult
17:35:31 <ylobankov> andreaf: So that is why we should refactor that code to make things easier
17:36:07 <andreaf> ylobankov: please go ahead, and please be specific on the pain points that require refactor
17:36:35 <andreaf> ylobankov: the main objectives are: make it easier to contribute tests, make it easier to debug them (during test development and in the gate) and reduce technical debt
17:37:15 <andreaf> ylobankov: well we don't want to scare off new contributors and mostly the code must be debuggable - a failure in the tests may have a big impact on the gate so we need the code to be debuggable
17:38:01 <ylobankov> andreaf: I absolutely agree with you
17:38:23 <andreaf> ylobankov: also we do have unit test coverage for the common parts of the framework, but if test code becomes too complex it may become difficult to separate failures in the tests from actual failures
17:39:05 <andreaf> ylobankov: in my personal opinion it's a fine balance we need to find, but now we're are off in the direction of complexity
17:39:45 <andreaf> anyways, I think we need to move on
17:39:45 <andreaf> #topic DevStack + Grenade
17:39:53 <andreaf> sdague, dtroyer: around?
17:40:11 <andreaf> anything on devstack or grenade?
17:41:17 <andreaf> ok, let's move on
17:41:24 <andreaf> #topic Critical Reviews
17:41:41 <andreaf> Any critical review to highlight?
17:43:26 <andreaf> ok looks like not
17:43:45 <andreaf> let's open the floor for discussion then
17:43:57 <andreaf> #topic Open Discussion
17:45:20 <andreaf> ok, it there's nothing else I guess we'll end here
17:45:32 <andreaf> thanks everyone
17:45:35 <dpaterson> thanks andrea
17:45:39 <ccneill> thanks all
17:45:42 <dwalleck> andreaf: Thanks!
17:45:43 <ccneill> o/
17:45:53 <ylobankov> thanks all
17:45:54 <andreaf> o/
17:45:56 <andreaf> #endmeeting