22:00:09 <mtreinish> #startmeeting qa
22:00:09 <openstack> Meeting started Thu Jan  9 22:00:09 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:10 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:12 <openstack> The meeting name has been set to 'qa'
22:00:27 <mkoderer> hi
22:00:30 <mtreinish> hi who do we have today?
22:00:33 <masayukig> hi
22:00:38 <rahmu> hi
22:00:40 <dkranz> hi
22:00:44 <dmorita> hii
22:00:48 <gfidente> hi
22:00:50 <vponomaryov> hi
22:01:10 <mtreinish> today's agenda:
22:01:12 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting
22:01:57 <mtreinish> let's get started
22:01:58 <sdague> o/
22:02:07 <mtreinish> I guess first up we have
22:02:09 <mtreinish> #topic Negative test status/discussion (dkranz)
22:02:46 <mkoderer> #link https://github.com/mkoderer/tempest/tree/feature/negative_tests
22:02:49 <dkranz> ok, so I got only positive feedback for the negative test gen approach I pushed, and a request for testscenarios usage
22:03:07 <dkranz> Marc made a fork for us to share and pushed the testscenario stuff
22:03:12 <mtreinish> dkranz: that's not fair I had a few negative comments :)
22:03:20 <rahmu> dkranz: are you talking about this? https://review.openstack.org/#/c/64733/
22:03:32 <dkranz> rahmu: yes
22:03:49 <mkoderer> I will push tomorrow the newest state
22:03:56 <dkranz> The current state is in https://github.com/mkoderer/tempest/tree/feature/negative_tests
22:04:12 <dkranz> I am going to add validation for the schemas
22:04:40 <mkoderer> and I added testscenarios today
22:04:53 <mkoderer> so feedback is welcome ;)
22:04:58 <dkranz> We can discuss and then Marc and I would like to know what we think should go in before we remove the WIP
22:05:31 <dkranz> So the current plan is testscenarios (already there) and validation (soon)
22:05:42 <dkranz> Comments?
22:05:52 <mkoderer> and unit tests ;) (not pushed yet)
22:06:01 <dkranz> mkoderer: right
22:06:10 <mtreinish> mkoderer: that's the right attitude
22:06:27 <mtreinish> dkranz: I think that's probably a good starting point for now
22:06:35 <dkranz> I am not talking about what is needed to be "complete" but what is needed to actually submit so others can contgribute
22:06:56 <mtreinish> dkranz: yeah that's probably a good base so we everyone can iterate on it
22:07:02 <dkranz> mtreinish: ok, so we will put in validation and some unit tests and then push for real
22:07:09 <rahmu> Are we done with separating the existing negative tests?
22:07:33 <mkoderer> rahmu: shouldn't be much left
22:07:47 <mtreinish> rahmu: yeah I don't think it's quite a 100% yet
22:08:03 <rahmu> my patch for the Swift tests needs some love :) https://review.openstack.org/#/c/61874/
22:08:04 <mtreinish> but that really isn't the highest priority the separation just helps when we decide to pull the negative tests
22:08:23 <mtreinish> rahmu: we've got a section at the end of the meeting for reviews that need attention
22:08:30 <dkranz> mtreinish: Right. We will have to decide how many generated test to run in the gate
22:08:43 <rahmu> okay
22:08:55 <mkoderer> rahmu: I will have a look to the patch tomorrow
22:09:31 <mtreinish> dkranz: I imagine these tests are fairly quick, the current negative tests are for the most part
22:09:57 <dkranz> mtreinish: Yes, but these will generate a lot more tests
22:10:24 <dkranz> mtreinish: one, for each api, for each resource or json value that can be supplied
22:10:46 <dkranz> mtreinish: I guess we can worry about that later
22:11:04 <dkranz> mtreinish: The biggest part of this job is adding new schemas for each api
22:11:10 <mkoderer> yes .. let's care about that later
22:11:12 <dkranz> mtreinish: So it will take a while
22:12:02 * gfidente can't join today, sorry :(
22:12:05 <dkranz> That's it from me
22:12:12 <mtreinish> dkranz: ok
22:12:16 <mtreinish> then let's move on
22:12:30 <mtreinish> #topic Reconsideration of https://blueprints.launchpad.net/tempest/+spec/refactor-rest-client, according to changes (vponomaryov)
22:12:39 <mtreinish> vponomaryov: are you around?
22:12:43 <vponomaryov> yes
22:12:49 <mtreinish> ok go ahead
22:12:58 <mkoderer> ok I am off.. cu tomorrow
22:13:10 <dkranz> mkoderer: Sleep well!
22:13:15 <vponomaryov> So, after last BP review it have had changed
22:13:52 <vponomaryov> Little changes to rest client and two examples of yousing it: https://review.openstack.org/#/c/62923/ and https://review.openstack.org/#/c/64948/
22:14:24 <mtreinish> vponomaryov: so my concern with this blueprint is actually shown in this file: https://review.openstack.org/#/c/64948/6/tempest/services/compute/xml/servers_client.py
22:14:37 <mtreinish> things don't actually change that significantly
22:14:47 <mtreinish> you drop the one line methods
22:14:54 <mtreinish> but the bulk of the file stays the same
22:15:08 <mtreinish> and there is a lot of review effort and rebase churn for something like this restructure
22:15:26 <vponomaryov> we have 25% of code clearing on адфе здфсу
22:15:33 <vponomaryov> flat place
22:15:50 <vponomaryov> without inconsistencies
22:16:07 <dkranz> I have mixed feelings about this
22:16:27 <dkranz> The newer projects have all made sure their json and xml are mostly isomorphic an so avoid this issue
22:16:44 <dkranz> We are really talking about the legacy code
22:16:54 <dkranz> And whether it is worth refactoring it
22:17:36 <dkranz> I don't really mean legacy but rather already complete
22:17:40 <sdague> yeh, with current review load it seems really low priority
22:18:01 <mtreinish> dkranz: yeah that's part of my concern here. IMO the effort for doing this really doesn't seem worth it
22:18:15 <dkranz> vponomaryov: What was bugging you that made this your highest priority?
22:18:35 <vponomaryov> Copy -paste issue
22:19:21 <vponomaryov> I am working on service clients on new project, and found it as really easy to fix and nice look
22:20:24 <mtreinish> vponomaryov: well would just the first part be enough for you then. That way we can avoid the review churn of converting everything?
22:20:35 <dkranz> vponomaryov: So why not just do it there, and only use the first part?
22:21:01 <vponomaryov> second part was made after sdague comment
22:21:10 <vponomaryov> about selecting hardent client pair
22:21:17 <dkranz> We can make it nicer for new projects
22:21:21 <vponomaryov> s/hardent/hardest
22:21:45 <dkranz> Well sdague was assuming I think this would be done for all or none but that may not be right
22:21:54 <vponomaryov> I agree, at least, with approving changes for rest clietn itself
22:22:47 <vponomaryov> when, who want make his client pair without copy-paste he just do it
22:22:51 <mtreinish> vponomaryov: I think doing it that way will be fine, but if you're refactoring the rest client make sure you add unit tests to cover the changes
22:23:02 <mtreinish> that way we can verify functionality moving forward
22:23:12 <mtreinish> especially if you're not converting any of the exisiting service clients
22:23:45 <vponomaryov> mtreinish: unittests should be in the same patch?
22:23:50 <mtreinish> vponomaryov: yes
22:23:59 <vponomaryov> ok
22:24:44 <mtreinish> ok I think this topic is done for now
22:24:50 <mtreinish> let's move on
22:25:05 <mtreinish> #topic Blueprints
22:25:18 <mtreinish> are there any other blueprints that people think need attention
22:25:25 <mtreinish> or status updates from any open blueprints
22:25:53 <mtreinish> I actually have a quick update on bp config-cleanup today
22:26:27 <mtreinish> I think what will be the last 2 reorg patches are up for review right now: https://review.openstack.org/#/c/65751/ and https://review.openstack.org/#/c/62063/
22:26:53 <mtreinish> but I wanted to make sure that everyone thinks the reorg is finished or good enough before I think it's read to close
22:26:57 <sdague> cool, honestly I've been not really doing many reviews this week with the gate backed up the way it is. Trying to not make it worse
22:27:06 <mtreinish> and that I didn't miss any configurable options
22:27:26 <sdague> mtreinish: seems good enough for my tastes, we can pick up other issues after if they come up
22:27:30 <mtreinish> sdague: that's fine the second one actually bounced of the gate after about 48 hours because of a merge conflict
22:27:52 <mtreinish> sdague: ok then when these last 2 merge I close the bp
22:28:40 <sdague> so had you intended that bp to also cover the config isolation from the class hierarchy?
22:28:46 <sdague> or was that outside of it
22:29:00 <mtreinish> sdague: not originally, but that ended up being part of it too
22:29:14 <mtreinish> because I hit some issues with the decorators that I had to add for the extension stuff
22:29:28 <mtreinish> I could make it a separate bp or just tack it on to this one
22:29:36 <sdague> so we might want to make a task on it then to run through and clean up assigning config pieces to the objects
22:30:40 <ken1ohmichi> related to config cleanup, is the CONF of each base.py necessary based on http://openstack.10931.n7.nabble.com/qa-changes-to-interacting-with-Tempest-config-td27732.html ?
22:30:47 <mtreinish> sdague: ok will do
22:31:11 <sdague> ken1ohmichi: so honestly, it should be imported when you need it
22:31:34 <mtreinish> ken1ohmichi: that's what we're talking about, getting rid of it from the base test classes
22:31:36 <ken1ohmichi> sdague: ok, I see
22:31:39 <mtreinish> and switching it to the imports
22:31:39 <sdague> it's in the basy.py just because we used to build it in the base.py classes and assign it to cls.config
22:31:49 <sdague> but we should stop doing that entirely
22:31:54 <sdague> and treat it like logging object
22:32:04 <sdague> completely outside of the class hierarchy
22:32:10 <sdague> just import it when yuo need it
22:32:29 <sdague> it magically lazy loads the first time anyone gets an attr
22:33:17 <mtreinish> ok does anyone else have bp updates? otherwise lets move on
22:33:31 <vponomaryov> sdague, but, for now, it has static proxy class attrs, and in cli tests oslo.config can not be removed
22:33:54 <vponomaryov> because its grop/options defined in cli module
22:34:23 <vponomaryov> and never assigned to proxy
22:34:25 <sdague> vponomaryov: so that's probably true, honestly the cli tests are a bit of a different beast
22:34:33 <sdague> so I'm not that concerned about them
22:35:13 <mtreinish> vponomaryov: I'm not actually convinced of that either but we can save that discussion for later
22:35:27 <mtreinish> ok lets move on to the next topic
22:35:41 <mtreinish> #topic Bugs
22:36:12 <mtreinish> so are there any critical/high bugs that we need to bring up
22:37:03 <mtreinish> sdague: you always have something to say about bugs :)
22:37:19 <sdague> heh
22:37:33 <sdague> honestly, not this week. I've been too buried trying to get my er data analyzer working
22:37:53 <mtreinish> ok, fair enough
22:38:01 <mtreinish> then let's jump to the next topic
22:38:11 <mtreinish> #topic Neutron testing (mlavalle)
22:38:18 <mtreinish> mlavalle: are you around?
22:38:36 <mlavalle> mlavalle: the current status is in the agenda, which I updated a couple of hours ago
22:39:07 <mlavalle> I just want to highlight that he api gap analysis is complete
22:39:12 <mlavalle> in the etherpad
22:39:24 <mlavalle> folks are picking up gaps and developing tests
22:39:50 <mlavalle> not as much as many as I wanted but it seems to be picking up
22:40:07 <masayukig> mlavalle: where is the etherpad?
22:40:07 <mlavalle> and I am making a point of following up daily with them and review their code
22:40:35 <mlavalle> https://etherpad.openstack.org/p/icehouse-summit-qa-neutron
22:40:41 <mlavalle> in the api section
22:40:47 <mtreinish> #link https://etherpad.openstack.org/p/icehouse-summit-qa-neutron
22:41:06 <masayukig> mlavalle: thanks :)
22:41:09 <mlavalle> all he negative tests were removed for the ether pad as of last week
22:41:24 <mtreinish> mlavalle: I think it might be worth separating the gap analysis and tracking to separate place
22:41:50 <mtreinish> it just seems a little crowded on that etherpad :)
22:41:51 <mlavalle> mtreinish: you mentioned that a few weeks ago…….
22:42:06 <mlavalle> but folks are already going to the etherpad
22:42:14 <mlavalle> I don't want to confuse / loose them
22:42:46 <mlavalle> that's why I've been hesitant to take the analysis somewhere else
22:42:56 <mtreinish> ok, that's fine.
22:43:36 <mlavalle> salv-orlando has been making progress with the bugs related to the gate and parallel testing
22:43:53 <mlavalle> if he is around I would like him to give some detail
22:44:11 <salv-orlando> I am around if there is any question
22:44:26 <salv-orlando> sorry mlavalle did not see your last post
22:44:40 <mlavalle> that's all I have
22:45:01 <mlavalle> and see you next week in balmy Montreal ;-)
22:45:06 <salv-orlando> in a nutshell: the most pressing issue, with the connection not being up because of various agent issue will be sorted once the patches under review are merged.
22:45:26 <salv-orlando> I have filed a list of bugs for "minor" issues. They're all tagged neutron-parallel.
22:45:54 <salv-orlando> We only have one big problem with a scenario for port quotas, but it's more about how the scenario itself is designed. I'm taking care of that.
22:46:33 <salv-orlando> furthermore, nati-ueno is looking at a metadata issue that was uncovered once the rest of agent flakiness has been sorted.
22:47:00 <salv-orlando> this metadata issue causes ssh to not work - even if the vm is reachable (the 'ssh protocol banner' error).
22:47:07 <salv-orlando> that's all from me.
22:47:11 <mtreinish> salv-orlando: ok cool it sounds like things are moving
22:47:30 <mtreinish> salv-orlando: oh, so it was related the ssh issue was related to cloud-init
22:48:14 <salv-orlando> mtreinish: the original ssh issue was related to the agent taking ages to wire ports
22:48:38 <salv-orlando> and applying iptables over and over, thus delaying further port wiring
22:48:55 <salv-orlando> once that has been sorted (even if the patches are technically still under review)
22:49:17 <salv-orlando> I have seen appearing a lot (about 20% of runs) this ssh protocol banner error
22:49:41 <salv-orlando> nati-ueno has found triaged the bug and assigned it to himself.
22:50:20 <mtreinish> ok cool
22:50:37 <mtreinish> let's go to the last topic on today's agenda
22:50:46 <mtreinish> #topic Critical Reviews
22:51:04 <mtreinish> so does anyone have any reviews that need some attention?
22:51:11 <mtreinish> rahmu: you had one earlier right?
22:51:34 <rahmu> mtreinish: yes I was mentioning separating negative tests for Swift https://review.openstack.org/#/c/61874/
22:51:50 <mtreinish> #link https://review.openstack.org/#/c/61874/
22:52:13 <dkranz> There are a lot of reviews sitting with just a -1 from jenkins
22:52:17 <mtreinish> rahmu: ok, I'll take a look at it after the meeting
22:52:29 <mtreinish> dkranz: yeah that's probably because of the state of the gate now
22:52:46 <mtreinish> but I feel the burden on the recheck should be the patch owner
22:52:48 <dkranz> mtreinish: Should people be doing rechecks, or not?
22:53:04 <dkranz> mtreinish: I agree. I'll send a message to the list.
22:53:12 <mtreinish> dkranz: I have been, but I guess it just contributes to the load on things
22:53:28 <dkranz> mtreinish: That's why I asked
22:53:41 <mtreinish> sdague: do you have an opinion?
22:54:06 <sdague> so in general I think authors should be responsible for their patches getting past jenkins
22:54:14 <sdague> but honestly, right now, I wouldn't create more load
22:54:25 <sdague> which is why I've been trying to just let things clear
22:54:36 <sdague> unless it's a fix for something which will make the gate better
22:54:46 <sdague> dkranz: so I wouldn't send an email at this point
22:54:59 <dkranz> sdague: ok
22:55:02 <sdague> until the gate actually dips below 10 items in it
22:55:27 <dkranz> sdague: Perhaps we should start reviewing things that are in this state
22:55:27 <sdague> because it hasn't been < 40 since sunday
22:55:43 <dkranz> sdague: Cause otherwise we are jkust shutting down
22:55:49 <sdague> dkranz: agreed
22:56:28 <dkranz> sdague: I'll send a message about that then :)
22:56:46 <dkranz> sdague: Unless you think I shouldn't
22:56:49 <sdague> yeh, well that's why I'm organzing Gate Blocking Bug day
22:57:09 <dkranz> sdague: I know but its 2.5 weeks away, right?
22:57:16 <sdague> yes
22:57:30 <sdague> but with montreal next week, that was about as good as I was going to get
22:57:42 <ken1ohmichi> mtreinish: I also have https://review.openstack.org/#/c/63365/ for fixing a gate failure
22:57:52 <mtreinish> #link https://review.openstack.org/#/c/63365/
22:58:20 <mtreinish> ken1ohmichi: ok, I'll take a look in a bit
22:58:29 <sdague> ken1ohmichi: ok, great
22:58:37 <ken1ohmichi> mtreinish: Ithanks in advance
22:58:51 <dkranz> sdague: Speaking of Montreal, should those of us not there try to participate some? What's the best way to do that?
22:58:52 <sdague> ken1ohmichi: +A, I'll get fungi to put it up to the top of the gate
22:59:07 <sdague> dkranz: be on IRC?
22:59:20 <sdague> other than that, I don't think there will be much to do
22:59:22 <dkranz> sdague: I'm always on IRC :)
22:59:24 <sdague> yeh
22:59:29 <dkranz> sdague: ok
22:59:32 <sdague> it's not really optimized for people far away
22:59:41 <dkranz> sdague: Sure
22:59:42 <ken1ohmichi> sdague: thanks!!
22:59:43 <sdague> but maybe we'll send people to irc if we get swamped
23:00:12 <mtreinish> ok well we're out of time for today
23:00:16 <mtreinish> thanks everyone
23:00:18 <dkranz> bye all
23:00:19 <mtreinish> #endmeeting