22:00:17 <mtreinish> #startmeeting qa
22:00:19 <openstack> Meeting started Thu Mar 20 22:00:17 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:21 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:23 <openstack> The meeting name has been set to 'qa'
22:00:34 <mtreinish> hi who's here today?
22:00:37 <masayukig> hi
22:00:38 <ken1ohmichi> hi
22:00:45 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting
22:00:49 <mtreinish> ^^^ Today's agenda
22:00:58 <mlavalle> hi
22:00:59 <cyeoh> hi
22:01:32 <maurosr> hi \o
22:01:59 <mtreinish> sdague, dkranz: you guys around?
22:02:05 <mtreinish> well let's get started
22:02:06 <dkranz> Yes, just here
22:02:17 <mtreinish> #topic Blueprints
22:02:18 <sdague> o/
22:02:44 <mtreinish> so sdague I'm assuming you left the note on the agenda about the specs repo
22:02:52 <sdague> yes
22:03:12 <sdague> I wanted to make sure that we started talking about the qa-specs repository
22:03:18 <sdague> #link https://github.com/openstack/qa-specs
22:03:55 <sdague> so based on the conversation last week we're going to try to do something very similar to what nova is doing with a -specs repository for reviewing blueprints
22:04:21 <sdague> I think the right thing to do is unapprove all non-high items, and have specs proposed back in through this new process
22:04:40 <sdague> that will hopefully help us actually clarify blueprints in advance
22:04:54 <mtreinish> yeah I agree that's probably the right approach for the exisiting bps
22:04:56 <sdague> a couple of questions for folks
22:05:18 <sdague> 1) do you think we need a more formal template? or is a free form better to get started?
22:05:36 <sdague> I am leaning towards free form, as I'd like to build template by experimentation
22:05:42 <dkranz> sdague: I'd say free form
22:05:48 <sdague> however, if people feel strong the other way we can do that
22:05:54 <cyeoh> maybe a very rough template for some guidance? But I'm happy either way
22:05:56 <dkranz> Until we get a feel for what people will put there
22:06:10 <mtreinish> sdague: yeah I think free form is better too, but maybe some guidelines on what content is expected in the proposal
22:06:35 <mtreinish> sdague: for example our debate on where to put target release
22:06:42 <sdague> cyeoh: you have some ideas about what rough guidance?
22:06:49 <cyeoh> can probably steal the basic stuff from the nova template - things like pointing back to the blueprint etc
22:06:55 <cyeoh> ids of who is going to work on it etc
22:07:09 <sdague> cyeoh: ok, sure, that's probably fair
22:07:30 <sdague> cyeoh: you want to write a few things here - https://etherpad.openstack.org/p/qa-spec-template
22:07:34 <cyeoh> I can do that this afternoon if you'd like (just can't do it this morning)
22:07:46 <sdague> cyeoh: sure
22:08:03 <sdague> so if you are going to do it your afternoon, yuo want to just propose it as a review to the repo?
22:08:06 <sdague> and we'll do it that way
22:08:24 <cyeoh> sure, will do
22:08:58 <sdague> I think basic problem statement, and approach, target time frame, people working on it, and where it would go in the tempest tree are the things I can think of at the moment
22:09:06 <sdague> my brain is a bit gate fried though :)
22:09:19 <mtreinish> sdague: yeah that seems like a reasonable list
22:09:21 <sdague> so the assistance on proposing template is appreciated
22:09:23 <cyeoh> so are we planning on having people resubmit every cycle?
22:09:34 <sdague> cyeoh: I don't think so
22:09:41 <cyeoh> ok
22:09:55 <sdague> I think if an idea goes obsolete then we mark it somehow in the template
22:10:01 <sdague> sorry, in the spec
22:10:15 <sdague> but I think we can figure out that workflow later
22:10:20 <ken1ohmichi> should we stop unapproved bp tasks until approval with this process?
22:10:29 <cyeoh> sdague: sounds good to me
22:10:52 <sdague> ken1ohmichi: honestly, I don't want you to stop doing anything :)
22:10:55 <sdague> you're too productive
22:11:03 <cyeoh> +1
22:11:09 <masayukig> +1
22:11:28 <ken1ohmichi> thanks:-)
22:11:32 <sdague> mostly I want us to go back through this process to make sure things are clear
22:11:44 <sdague> for juno
22:11:59 <sdague> because we definitely got into a lot of confusion on things like the advanced network debugging
22:12:12 <sdague> that I feel like needs an overview document to make sure we're on top of it
22:12:39 <sdague> so consider this an "in parallel" for anything you are actively working on
22:12:49 <sdague> and a pre req for new things
22:13:07 <dkranz> sdague: So should be be more accepting of proposals that are project-specific?
22:13:24 <dkranz> sdague: Both neutron and heat wanted to have more blueprints in tempest but we said no
22:13:36 <sdague> yes, I feel like doing this I'd be ok with that
22:13:50 <dkranz> sdague: Yeah, we should at least be open to it
22:14:29 <sdague> because we don't end up with them populating 50 lp artifacts and no one noticing :)
22:14:52 <sdague> and important part of this process is "Bring forward the proposed item to the openstack-qa meeting for summary"
22:15:16 <sdague> and I don't want us +Aing anything that misses that step
22:15:18 <dkranz> sdague: Right. And there could be project subdirs in the specs area
22:15:29 <sdague> dkranz: maybe, lets try flat first
22:15:43 <dkranz> sdague: k
22:15:47 <sdague> we can always easily reorg if it's gone crazy
22:15:55 <mtreinish> dkranz: I'm still not convinced a bp or this repo are the best way to track doing test tracking either
22:16:00 <cyeoh> so there may be some people who find it really hard to get to the irc meeting
22:16:01 <mtreinish> but we can iterate and explore
22:16:08 <cyeoh> are we ok with people just putting it on the agenda if that's the case?
22:16:14 <cyeoh> (and not turning up)
22:16:25 <dkranz> mtreinish: Yes, since real project management tools are out-of-bounds :)
22:16:36 <sdague> cyeoh: even with the rotating times?
22:16:44 <mtreinish> yeah that's fine. I guess we don't have coverage for certain tz even with the rotating sched
22:16:48 <mtreinish> sdague: yeah like china
22:17:04 <sdague> right that's 6am now?
22:17:17 <mtreinish> yep
22:17:45 <sdague> well, one of the issues I really want to address is people off working on bp that aren't communicating via irc or email about them
22:18:19 <sdague> because we definitely have seen people not be successful if they only communicate in gerrit
22:18:33 <sdague> and I'd like to have people be more successful
22:18:34 <mtreinish> sdague: yeah I agree but if the meeting can't be attended by everyone we need some way to enforce communicate besides just the meeting
22:18:51 <sdague> sure, so alt to meeting is ML discussion
22:19:03 <dkranz> mtreinish: We could insist on initial ml post
22:19:16 <sdague> well, lets see how the first round goes
22:19:17 <mtreinish> ok that works
22:19:31 <mtreinish> sdague: do you think we should update the readme
22:19:34 <sdague> lets start with all of us that have bp that are < high proposing up stuff
22:19:36 <mtreinish> because it doesn't mention the ml
22:19:39 <sdague> in the next week
22:19:55 <sdague> then out of that set we can start to figure out what's workign and what's not
22:20:31 <sdague> so we'll do a review next week of everything that's been proposed and comment on what seems to be working and not with the repo
22:20:36 <sdague> sound good to folks?
22:20:41 <dkranz> +1
22:20:42 <mtreinish> works for me
22:20:57 <ken1ohmichi> +1
22:21:02 <mlavalle> +1
22:21:03 <maurosr> +1
22:21:04 <masayukig> +1
22:21:13 <sdague> after we have that, we can communicate more the the broader community to get more folks doing it as well
22:21:24 <sdague> hopefully that will provide plenty of good examples for them to copy from
22:21:55 <cyeoh> +1
22:21:56 <sdague> ok, I think that topic is done unless there are last questions
22:22:35 * afazekas is reading more frequently IRC than ML
22:22:42 <mtreinish> ok then let's move on to the bp review
22:23:06 <mtreinish> I think for time we should skip the standard high prio bp walkthrough
22:23:17 <sdague> sure
22:23:27 <mtreinish> and just ask if anyone has any updates for open bps
22:23:37 <mtreinish> or needs feedback or attention on any bps
22:24:00 <mtreinish> #link https://blueprints.launchpad.net/tempest/+spec/unit-tests
22:24:23 <mtreinish> I think I'll just mention that we've made a bunch of progress with unit tests this past week
22:24:36 <mtreinish> we've had some new contributions from masayukig and others
22:24:38 <sdague> question on the unit tests, in fixing something for the logging
22:24:54 <sdague> we seem to be using a lot of fake objects
22:25:02 <sdague> instead of just mock calls directly
22:25:08 <sdague> is there a reason for that?
22:25:31 <mtreinish> sdague: there are a few fake objects for some common things that will be used everywhere
22:25:39 <mtreinish> but for the most part it should be more mocks and fixtures
22:26:02 <mtreinish> for example look at the isolated_creds tests
22:26:25 <mtreinish> anyway I think we'll be at >200 unit tests by the end of icehouse
22:26:30 <sdague> yeh, I would lean on trying to get rid of the fake objects if possible, mostly because it adds a level of coupling when you need to change them
22:26:37 <sdague> mtreinish: nice
22:26:38 <mtreinish> considering we had <10 at the start
22:26:54 <mtreinish> sdague: yeah some of it was me just being lazy at first
22:26:59 <mtreinish> and it being reused
22:27:26 <mtreinish> we can start to clean that up over time (like the config refactor to use the fixture)
22:27:35 <sdague> yeh, cool
22:28:16 <sdague> honestly, I've only really started wrapping my head around mock recently, but it's become the openstack standard, so we should use it more to make it easier for people to jump between projects
22:28:21 <sdague> all goodness
22:28:36 <mtreinish> yeah I agree
22:28:46 <mtreinish> ok I didn't have anything else to mention about unit tests
22:28:52 <mtreinish> does anyone else have a bp to bring up
22:28:56 <mtreinish> otherwise we can move on
22:29:30 <mtreinish> #topic Neutron testing
22:29:51 <mtreinish> mlavalle: are you around?
22:29:55 <mlavalle> hi
22:30:22 <mtreinish> mlavalle: any update on neutron testing?
22:30:23 <mlavalle> We have continued merging api tests. Since our last meeting, another 2 patch sets have merged
22:30:41 <mlavalle> The grand total over the past 4 weeks is 14
22:31:32 <mlavalle> There are 3 abandondes patch sets. I contacted their authors by mail. One of them responded (he owns 2) and has given me permission to assign the patch sets to someone else
22:32:13 <mlavalle> I also have 3 patch sets that only rehire one additional +2 to merge. Can you guys help?
22:32:21 <mtreinish> mlavalle: sure do you have links?
22:32:33 <mlavalle> yes, give me aminute
22:33:09 <mlavalle> https://review.openstack.org/#/c/71251
22:33:09 <mlavalle> https://review.openstack.org/#/c/68597
22:33:10 <mlavalle> https://review.openstack.org/#/c/63999
22:33:28 <mtreinish> #link https://review.openstack.org/#/c/71251
22:33:32 <mtreinish> #link https://review.openstack.org/#/c/68597
22:33:37 <mtreinish> #link https://review.openstack.org/#/c/63999
22:33:43 <mlavalle> In summary, continuing making good progress
22:33:47 <mtreinish> mlavalle: ok I'll take a look tomorrow morning
22:33:53 <mtreinish> but someone will probably beat me to it
22:33:55 <mlavalle> thanks :-)
22:34:00 <mlavalle> That's all I have
22:34:13 <mtreinish> ok cool thanks
22:34:29 <mtreinish> I guess we can move on if no one has any questions about neutron testing
22:35:08 <mtreinish> #topic Heat testing
22:35:30 <mtreinish> sdague: I added this to the template
22:35:35 <mtreinish> but who should I ping about it
22:35:59 <sdague> stevebaker was the primary one code was coming from
22:36:18 <stevebaker> \o
22:36:24 <sdague> and there he is!
22:36:32 <sdague> lefty and all
22:36:37 <mtreinish> stevebaker: hi, do you have any updates on heat testing with tempest
22:37:24 <stevebaker> I've refreshed the autoscaling test, its working locally https://review.openstack.org/#/c/44967/
22:37:34 <sdague> stevebaker: cool
22:37:41 <stevebaker> but it is waiting on firewall changes to land before it can run https://review.openstack.org/#/c/81375/
22:38:14 <stevebaker> efforts to shame other heat developers to write tests may yet yield some results ;)
22:38:26 <sdague> stevebaker: I was trying to pull apart what would be needed for an "ha" test using heat
22:38:43 <sdague> where should I dig in to learn enough to do that?
22:38:44 <stevebaker> I hope to start writing some software-config tests once a few fixes land
22:39:05 <sdague> basically "restart vm if it goes away", which I gather we can do from heat
22:39:53 <stevebaker> sdague: the autoscaling scenario test can continue to evolve so that it scales arbitrary stacks, and is fronted by a neutron load balancer
22:40:06 <stevebaker> sdague: most likely as new tests
22:40:12 <stevebaker> i mean new scenarios
22:40:18 <sdague> stevebaker: sounds good
22:40:37 <sdague> maybe I'll ping you in -dev tomorrow to ask some more questions about how to help here
22:40:41 <stevebaker> I'll be focusing on software config though
22:41:13 <stevebaker> sdague: ok, thanks
22:41:34 <mtreinish> stevebaker: ok cool, is there anything else?
22:42:14 <stevebaker> we need some stack update scenarios, but it remains to be seen who will write those
22:42:52 <mtreinish> stevebaker: maybe a call for help on the ML?
22:43:51 <stevebaker> mtreinish: maybe. Existing developers know the tests are needed, its just a matter of resourcing it
22:43:58 <sdague> cool
22:44:20 <sdague> look forward to more here. It's definitely good to have the heat job voting on everyone now
22:44:37 <stevebaker> yes, it seems solid
22:44:49 <sdague> yep
22:45:25 <sdague> ok, next up?
22:45:28 <mtreinish> ok if there's nothing else to discuss let's move onto the next topic
22:45:34 <mtreinish> #topic Bugs
22:45:47 <mtreinish> so we had our bug day yesterday
22:45:55 <mtreinish> maurosr: do you have any results from it
22:45:57 <mtreinish> ?
22:46:02 <maurosr> yup
22:46:14 <maurosr> so just the numbers to be clear
22:46:26 <maurosr> bugs that needed triage from 62 to 18 http://bit.ly/1cXhPcF
22:46:35 <sdague> yay!
22:46:36 <maurosr> ok 19 now (just have a new one)
22:46:40 <maurosr> Open bugs from 166 => 146
22:46:48 <maurosr> To prioritize we kept on 29  (no evolution)
22:46:54 <maurosr> In progress 51 to 57 http://bit.ly/1hzsjfH (which is kind of result of taking triage bug into action)
22:48:07 <maurosr> I tended to move some of the bugs to Incomplete instead on invalid giving a chance to the reporter make it clearer... that is why we reduced only in 20 the open bugs
22:48:28 <sdague> sure, that sounds fair
22:48:34 <sdague> thanks for driving on this one
22:48:41 <maurosr> yw
22:48:41 <mtreinish> +1
22:48:45 <dkranz> maurosr: Yes, thanks
22:49:13 <maurosr> btw
22:49:37 <maurosr> in one of our meeting we were discussing thwe kind of stuff that we would accept as bugs
22:49:44 <maurosr> not only traces
22:50:03 <sdague> yeh, if it's just a trace I was pretty aggressively marking as invalid
22:50:10 <sdague> as it's almost never a tempest bug
22:50:29 <maurosr> so it happens people are using tempest bugs to do rechecks, so there are lots of bugs that should at least affect other projects too that were there
22:50:41 <dkranz> sdague: But don't we want to get these things into elastic recheck?
22:50:44 <maurosr> there was even a cinder unit test bug
22:51:09 <sdague> dkranz: it needs to be more than a stack trace for ER
22:51:10 <mtreinish> maurosr: heh, yeah I saw that one that cracked me up
22:51:24 <sdague> and it really shuold be assigned to the project where the problem is
22:51:35 <sdague> and delete tempest from the bug if it's not actually a tempest bug
22:51:42 <dkranz> sdague: I'm fine with that rather than just marking it invalid
22:51:51 <sdague> because if the ER bugs aren't actionable
22:51:55 <sdague> then they aren't useful
22:52:00 <dkranz> sdague: Let the projects mark them as invalid :)
22:52:07 <maurosr> invalid != remove tempest of the affected list?
22:52:25 <sdague> maurosr: so honestly either is fine
22:52:35 <mtreinish> maurosr: if there is more than one project listed on the bug you can remove one from the list
22:52:55 <maurosr> right, understood what you meant
22:53:09 <sdague> yeh, I tend to remove the wrong projects
22:53:27 <sdague> because I find that otherwise people get confused about the bug, especially if they reopen it later
22:53:43 <sdague> also, feel free to rewrite the summary and title to be more descriptive
22:53:49 <sdague> if the base bug is bad
22:53:58 <sdague> that helps create clarify over time
22:54:32 <sdague> this is really policy for anyone, not just maurosr  :)
22:54:38 <mtreinish> heh
22:54:50 <mtreinish> maurosr: ok is there anything else on bugs?
22:54:51 <sdague> but especially bugs that we want to put into ER, I like to have good summary line
22:55:01 <maurosr> nop, I will send a summary to the list later, but I guess related to the bug day it's pretty much it
22:55:12 <mtreinish> ok
22:55:21 <mtreinish> let's move on then
22:55:28 <sdague> 5 minutes
22:55:28 <mtreinish> #topic Critical Reviews
22:55:41 <mtreinish> does anyone have any reviews they'd like to get eyes on?
22:56:18 <sdague> if nothing else - https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:rest_client_logging,n,z
22:56:29 <mtreinish> #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:rest_client_logging,n,z
22:56:31 <sdague> I'm trying to tighten up the tempest log so it's useful
22:56:48 <sdague> which means dumping a lot of the noise from it
22:56:57 <afazekas> For the leak related change I need to xml clients to behave more closely to the json clients https://review.openstack.org/#/c/81847/ https://review.openstack.org/#/c/78345/
22:57:04 <sdague> not complete, but that definitely cleans it up a bunch
22:57:11 <mtreinish> #link https://review.openstack.org/#/c/81847/
22:57:17 <mtreinish> #link https://review.openstack.org/#/c/78345/
22:57:33 <mtreinish> sdague: yeah it's definitely a step in the right direction
22:57:51 <sdague> I even killed the cli output on success in the last patch
22:57:54 <mtreinish> although with os-log-analyzer there is too much blue right clumped together with the patch
22:58:13 <sdague> heh
22:58:25 <sdague> well we can debate colors at the next summit
22:58:31 <sdague> I could unbold info as well
22:58:40 <mtreinish> I think that would probably be enough'
22:58:51 <sdague> yeh, I can propose that tomorrow
22:59:10 <afazekas> Looks like the instance validation with n-net with HARD reboot is flaky https://review.openstack.org/#/c/81834/ :(  , at my home desktop the test seams stable.
22:59:39 <afazekas> I am looking for advises how to move forward
22:59:59 <mtreinish> ken1ohmichi: ^^^ that seems like it's in your court
23:00:05 <mtreinish> anyway we're out of time
23:00:07 <mtreinish> thanks everyone
23:00:22 <mtreinish> I'll add the agenda item we couldn't get to onto next week's
23:00:25 <malini_afk> mtreinish: Can you move my agenda item to next week?
23:00:25 <mtreinish> #endmeeting