17:01:28 <mtreinish> #startmeeting qa
17:01:28 <openstack> Meeting started Thu Mar 27 17:01:28 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:29 <ttx> I'll fix it
17:01:30 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:01:32 <openstack> The meeting name has been set to 'qa'
17:01:39 <mtreinish> hi who do we have here today?
17:01:54 <mlavalle> hi
17:01:56 <sdague> o/
17:02:11 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_March_27_2014_.281700_UTC
17:02:13 <yfried> hi
17:02:17 <mtreinish> ^^^ Today's agenda
17:02:36 <andreaf> o/
17:03:05 <mtreinish> ok well let's get started
17:03:14 <mtreinish> #topic QA Specs Repo Proposals (sdague)
17:03:25 <sdague> ok, so we have the qa-specs repo
17:03:41 <sdague> https://github.com/openstack/qa-specs
17:03:49 <mtreinish> #link https://github.com/openstack/qa-specs
17:03:51 <sdague> and we even have an agreed to template now
17:04:06 <sdague> https://github.com/openstack/qa-specs/blob/master/template.rst
17:04:21 <mtreinish> #link https://github.com/openstack/qa-specs/blob/master/template.rst
17:04:37 <dkranz> o/
17:04:44 <sdague> so I think the point for this agenda item was to go through a spec or two and see what our feedback should be on it
17:04:56 <sdague> mtreinish: you have a couple up, right?
17:05:06 <sdague> do you want to volunteer one of those?
17:05:13 <mtreinish> sdague: yeah I pushed up the non high prio bps as specs reviews:
17:05:14 <mtreinish> https://review.openstack.org/#/c/82933/
17:05:19 <mtreinish> and https://review.openstack.org/#/c/83264/
17:06:30 <sdague> ok, lets take 82933
17:06:47 <mtreinish> #link https://review.openstack.org/#/c/82933/
17:07:19 <sdague> feedback that I'd like to see there is the that it's a new tool, so it should include where that tool will go in the source tree
17:07:59 <sdague> and we should probably have the -h output for it mocked out, so what's optional or not can be seen up front
17:08:04 <sdague> in the spec
17:08:17 <sdague> then I'd consider it good
17:08:24 <sdague> anyone else have feedback?
17:08:34 <sdague> or feel that feedback is too specific?
17:08:35 <mtreinish> heh, well it's already in the tools dir :)
17:08:44 <mtreinish> sdague: no those are good points
17:08:56 <dkranz> The alternative is to generate the config file and supply the admin creds as an argument
17:09:14 <dkranz> That is basically what I did in the sample generator I submitted as WIP
17:09:40 <dkranz> But seeing the -h output would make it clearer
17:09:48 <mtreinish> dkranz: I had an alternative listed in the spec for something like that
17:10:11 <dkranz> mtreinish: Yes, I saw that. My comment was about that part.
17:10:12 <sdague> so the alternatives section is one I'm sort of meh about
17:10:21 <mtreinish> yeah I'll add the -h option, although honestly it'll have only one bool option
17:10:34 <mtreinish> sdague: it was in the template so I used it
17:10:44 <dkranz> sdague: meh because you don't like it or because there should not be such a section?
17:10:48 <sdague> it's important in nova because they get a lot of competing duplicative approaches
17:11:05 <sdague> dkranz: right, I wonder if people feel it's important for qa-specs in general
17:11:09 <sdague> I could go either way
17:11:27 <dkranz> sdague: I don't see anything wrong with it but perhaps not required
17:11:36 <mtreinish> I think it's listed as optional in the template
17:11:46 <sdague> mtreinish: ok, that's fine
17:12:06 <mtreinish> sdague: that was one of my first review comments when cyeoh posted the first template draft
17:12:17 <sdague> ok, great
17:12:32 <sdague> ok, any other feedback on this one?
17:13:00 <yfried> sdague: that might have already been said,
17:13:54 <sdague> https://review.openstack.org/#/c/83264/2/specs/add-service-tags.rst is the 2nd one
17:13:56 <yfried> sdague: but I think once the config-verification is done, it should point to this spec somehow for futur docs
17:14:15 <sdague> yfried: what exactly do you mean?
17:14:16 <yfried> *future documentation
17:15:10 <yfried> sdague: a new guy looking at that module will find all this discussion very helpful
17:15:22 <sdague> yfried: so the specs repo is largely to define the blueprint
17:15:32 <sdague> then the implementation will happen there
17:15:46 <sdague> in tree based on the approved blueprint
17:16:06 <sdague> I'm not sure it's going to be better than the actual implementation for documentationm
17:16:24 <sdague> https://review.openstack.org/#/c/83264/2/specs/add-service-tags.rst - mtreinish I couple of comments there
17:16:46 <sdague> first, like with the last one, it's probably worth making it explicit where the code is getting added in tempest
17:17:03 <sdague> which is sort of there in prose, but could probably be broken out
17:17:17 <vrovachev> sdague: I want remove all unused variables in tempest. What you think about it.
17:17:33 <sdague> also, I think a code sample of what the code would look like afterwards would be useful
17:17:54 <mtreinish> sdague: ok, so basically we should include more examples in specs then
17:18:04 <mtreinish> we probably should put that in the template or readme
17:18:05 <sdague> yeh, I think examples will make it more reviable
17:18:19 <sdague> at least for me, that's how I wrap my head around things
17:18:38 <mtreinish> no that's a good point
17:19:02 <sdague> vrovachev: ok, is that part of the specs discussion? or is that for open discussion?
17:19:19 <sdague> i.e. should it wait until the end of the meeting
17:19:33 <sdague> anyone else have comments on - https://review.openstack.org/#/c/83264/2/specs/add-service-tags.rst  ?
17:19:45 <mtreinish> vrovachev: yeah that's off topic, if we have an open discussion section you can bring it up then
17:20:38 <sdague> other than that, I think this is looking pretty solid
17:20:48 <mtreinish> sdague: ok cool
17:20:58 <mtreinish> so would you say we're open for real on this then?
17:21:01 <sdague> mtreinish: you want to make another iteration on those? We can handle any remaining feedback in review
17:21:11 <mtreinish> sdague: yeah I'll do it after the meeting
17:21:16 <sdague> mtreinish: lets plan for end of next week to make that open beyond this group
17:21:25 <sdague> I'd like landed content that are good examples
17:21:42 <vrovachev> mtreinish: Good, sorry.
17:22:03 <mtreinish> sdague: ok yeah that'll probably be helpful
17:22:35 <sdague> andreaf: can you convert yours to this template - https://review.openstack.org/#/c/81294/2 ?
17:22:45 <sdague> I think that would be another good one to review during the week
17:23:07 <andreaf> sdague: sure I was hoping to do that before the meeting but didn't manage to
17:23:14 <sdague> andreaf: yep, no problem
17:23:41 <mtreinish> ok then I guess we can move on if there isn't anything else
17:23:54 <andreaf> sdague: one question re the template, do we include copyright notice in those like the code?
17:24:15 <mtreinish> andreaf: yeah the cc license like in the template
17:24:33 <sdague> andreaf: if your employer really wants a called out copyright, you can
17:24:48 <sdague> as long as it has the CC license in it
17:25:08 <andreaf> sdague: ok thanks
17:25:29 <sdague> ok, I think we're making good progress here
17:25:36 <sdague> anything else on specs?
17:26:01 <sdague> back to you mtreinish
17:26:08 <mtreinish> #topic Blueprints
17:26:22 <mtreinish> sdague: did you want to do a high prio bp review
17:26:37 <mtreinish> or skip it this week because of the specs discussion
17:27:12 <sdague> I think we can skip
17:27:32 <sdague> especially as we seem to have a small audience
17:27:37 <mtreinish> ok then did anyone have any bps they'd like to bring up
17:27:41 <mtreinish> otherwise we'll move on
17:28:01 <andreaf> mtreinish: ok one
17:28:07 <mtreinish> andreaf: sure go ahead
17:28:25 <andreaf> #link https://blueprints.launchpad.net/tempest/+spec/keystone-v3-jobs
17:28:36 <andreaf> it's related to the multi-auth bp
17:28:51 <andreaf> but on the infra side of things
17:29:29 <andreaf> I have to align the spec to the template for this one too, but it would be good to get some review on it
17:29:55 <mtreinish> andreaf: yeah we can do the review in gerrit now :)
17:30:00 <andreaf> at the moment is not prioritized, but it shall probably be same prio as the multi auth one
17:30:16 <mtreinish> andreaf: yeah that'll come after the spec review (see the readme)
17:30:20 <andreaf> #link https://review.openstack.org/#/c/81307/
17:30:25 <mtreinish> but it seems pretty straightforward
17:31:11 <andreaf> mtreinish: thanks that's all
17:31:22 <mtreinish> andreaf: ok thanks
17:31:30 <mtreinish> ok are there any other bps to discuss?
17:32:06 <mtreinish> ok then let's move on
17:32:10 <mtreinish> #topic Neutron testing
17:32:21 <mtreinish> mlavalle: are there any updates on neutron testing?
17:32:26 <mlavalle> Yes
17:32:35 <mlavalle> We have continued merging api tests
17:32:54 <mlavalle> I am tracking 28 patchsets. As of today, we have merged 16
17:33:41 <mlavalle> There are 5 abandoned. I have sent email to the authors. 3 responded and will be restarting their patchsets over the next couple of days
17:33:49 <mlavalle> The other 2 I assigned to myself
17:34:17 <mlavalle> I have 2 patchsets that only need one more +2 to merge:
17:34:39 <mlavalle> https://review.openstack.org/#/c/63723/
17:34:39 <mlavalle> https://review.openstack.org/#/c/68597
17:34:46 <mtreinish> #link https://review.openstack.org/#/c/68597
17:34:54 <mtreinish> #link https://review.openstack.org/#/c/63723/
17:35:01 <mtreinish> mlavalle: ok cool
17:35:06 <yfried> mlavalle: this might be off-topic, but - is there any chance to apply this productive approach to network-scenarios (as well as api)
17:35:10 <mlavalle> Please take a look at them and see if we can merge them
17:35:12 <mtreinish> hopefully we'll get this all merged before the release
17:35:48 <mtreinish> yfried: this is just for getting api coverage on neutron
17:35:55 <mlavalle> yfried: of course. I just don't want to spread myself to thin in this cycle, but scenario testing is my next goal
17:36:10 <mtreinish> scenarios are a bit more involved so I'm not sure a similar structure would be as effective
17:36:21 <mtreinish> unless there was a predefined list of scenarios to implement
17:36:23 <mlavalle> we can give it a try :-)
17:36:42 <mlavalle> and start with a predefined list
17:36:56 <yfried> mlavalle: I understand. I think the first step would be to draft a list of scenarios to implement
17:37:03 <mlavalle> yes
17:37:19 <mlavalle> let's partner on that
17:37:36 <mlavalle> that's all I have
17:37:44 <mtreinish> ok is there anything else on neutron testing from anyone else?
17:38:11 <sdague> do we have an idea on how close the full job is?
17:38:17 <afazekas> what is required to make the full neutron job voting ?
17:38:24 <mtreinish> sdague: there has been a ml thread on that
17:38:38 <mtreinish> salv-orlando and rossella_s have been looking at it
17:39:05 <mtreinish> I think there are still a few bugs but it's looking like we'll greenlight the week after release
17:39:20 <salv-orlando> We've been looking at failure rate, and rossella_s has a script which keeps spinning a patch in the check queue
17:39:38 <salv-orlando> So far it does not seem much worse than the smoke job
17:39:51 <salv-orlando> so I would say after release, we switch it to voting
17:39:58 <sdague> salv-orlando: I wonder if we could get rossella_s talking with the infra team about background queue jobs
17:40:04 <sdague> instead of doing that in check like that
17:40:17 <salv-orlando> she
17:40:18 <rossella_s> sdague: I'd be glad to talk to them
17:40:19 <sdague> because that's something we actually very much wanted to get baselines on all the test suites
17:40:21 <salv-orlando> she's online...
17:40:28 <sdague> rossella_s: awesome
17:40:29 <mtreinish> sdague: do those still get picked up by e-s?
17:40:37 <salv-orlando> e-s?
17:40:41 <sdague> mtreinish: the point would be to make them
17:40:44 <sdague> elastic search
17:40:48 <sdague> and elastic recheck
17:40:57 <sdague> so basically we could have control data from master
17:40:59 <salv-orlando> sorry got my acronyms mixed up
17:41:04 <sdague> yep, no worries
17:41:09 <mtreinish> salv-orlando: it's ok, I'm just lazy
17:41:51 <afazekas> BTW: is the neutron more stable with pgsql based on statistics ?
17:41:55 <sdague> rossella_s: ok, so how about post meeting we jump to -infra and talk about this
17:41:57 <salv-orlando> if you were really lazy you would have avoided using the dash as well; you probably had to stretch your right middle finger for that!
17:42:04 <sdague> heh
17:42:09 <rossella_s> sdague: ok, awesome!
17:42:19 <salv-orlando> afazekas: I would say mysql failure rate is actually higher than postgres
17:42:34 <sdague> salv-orlando: there is another difference in those jobs
17:42:42 <sdague> I believe mysql is using the metadata server
17:42:45 <sdague> and pg is not
17:43:01 <salv-orlando> does pg uses config drive?
17:43:02 <afazekas> salv-orlando: so the eventlet vs mysql native driver issue is visible..
17:43:10 <sdague> salv-orlando: I think so
17:43:24 <sdague> the layout.yaml definition should say
17:43:32 <salv-orlando> right. so keep going with the meeting I'll be back in 5 with numbers from the past week
17:43:46 <mtreinish> salv-orlando: ok cool
17:43:57 <mtreinish> then I guess we should move onto the next topic
17:44:06 <mtreinish> #topic Heat testing
17:44:22 <mtreinish> sdague, stevebaker: I've seen stuff from both of you on this
17:44:27 <mtreinish> are there any updates?
17:45:12 <sdague> mtreinish: I did the refactor to get the templates out of the code
17:45:17 <sdague> that's landed now
17:45:27 <sdague> which I think makes things a lot easier to understand
17:45:33 <sdague> and review
17:45:40 <sdague> there are some additional patches inbound
17:45:57 <sdague> I went through a stack by shardy this morning and gave some feedback
17:46:18 <sdague> also SergeyLukjanov was talking about adding sahara scenario tests that would use the heat provisioning
17:46:24 <sdague> which would help all around
17:46:27 <mtreinish> ok cool, I'll take a look at them when they come back through
17:46:31 <SergeyLukjanov> I'm here
17:46:36 <SergeyLukjanov> reading scrollback
17:46:37 <mtreinish> yeah that would be cool to see come through
17:46:45 <mtreinish> we'd run it on the heat slow job I'm assuming
17:46:50 <sdague> the sahara part will have to wait until we cut stable/icehouse
17:47:16 <mtreinish> yeah that's fine, it's only a couple weeks away anyway
17:47:17 <SergeyLukjanov> sdague, yup, I hope to have some PoC heat-based tests for sahara in summit time
17:47:30 <sdague> which we should probably decide what else has to land before we do that, as we are going to start getting open masters on projects again soon
17:47:40 <sdague> keystone has a juno master now IIRC
17:48:00 <mtreinish> sdague: I think I saw an email from ttx abount cinder too
17:48:10 <SergeyLukjanov> I think all projects will cut their first i-rc till the end of next week
17:48:15 <sdague> SergeyLukjanov: that's the hope
17:48:39 <SergeyLukjanov> sdague, I've prepared change for d-g to cut conditions
17:48:50 <SergeyLukjanov> https://review.openstack.org/#/c/83075/
17:48:55 <SergeyLukjanov> and enable sahara in gate
17:48:59 <SergeyLukjanov> https://review.openstack.org/#/c/83076/
17:49:35 <SergeyLukjanov> of course, the first one depends one rc1 cuts and the second one is on your wish :)
17:50:06 <mtreinish> SergeyLukjanov: well I wasn't planning on cutting tempest until release day, which is what've done for the past few cycles
17:50:21 <sdague> SergeyLukjanov: so I think the issue is we don't get stable/icehouse branches till much later, right?
17:50:37 <sdague> now we're sitting on milestone proposed?
17:50:40 <mtreinish> sdague: yeah I think that's right it's milestone proposed until release day
17:50:54 <sdague> we should maybe revisit that
17:51:11 <sdague> anyway, we should move on 10 minutes
17:51:17 <mtreinish> ok yeah
17:51:28 <SergeyLukjanov> oh, yup, sure, so, April 18 will be ready :)
17:51:34 <mtreinish> #topic Bugs
17:51:34 <SergeyLukjanov> we'll
17:51:53 <mtreinish> so does anyone have any bugs that they would like to bring up?
17:51:55 * SergeyLukjanov disappearing
17:52:02 <mtreinish> or are there any critical bugs that need attention?
17:53:01 <mtreinish> ok I guess not
17:53:04 <mtreinish> so let's move on
17:53:12 <mtreinish> #topic Critical Reviews
17:53:24 <mtreinish> does anyone have any reviews they'd like to get eyes on?
17:53:38 <andreaf> I do
17:53:41 <andreaf> as usual: https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/multi-keystone-api-version-tests,n,z
17:53:51 <mtreinish> #link https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/multi-keystone-api-version-tests,n,z
17:54:09 <mtreinish> andreaf: yeah I'll try to give it a thorough review this week
17:54:15 <mtreinish> it does take some time though
17:54:19 <andreaf> It's a chain of 7 patch-sets for the multi-auth bp
17:54:31 <andreaf> mtreinish: thanks - I tried to reduce the size as much as possible
17:54:59 <mtreinish> ok are there any other reviews?
17:55:05 <andreaf> mtreinish: in the meantime I started working on auth provided via keystone official client for the scenario tests - bit still WIP
17:55:14 <afazekas> The ssh instance validation things has enough problem without discussing the uec or not uec default image https://review.openstack.org/#/c/81486/ , I would recommend to stay with uec anyway
17:55:47 <mtreinish> #link https://review.openstack.org/#/c/81486/
17:56:12 <mtreinish> andreaf: I thought we used the keystoneclient for auth in scenario already
17:56:23 <mtreinish> or at least with tenant isolation we do
17:56:28 <mtreinish> or did you mean tokens?
17:56:47 <andreaf> mtreinish: yes but it's a fixed version of keystone client
17:56:58 <andreaf> I'm moving it to auth so it uses the right version
17:57:01 <mtreinish> ahh ok
17:57:03 * salv-orlando has the failure rate numbers and is waiting for open discussion
17:57:16 <sdague> salv-orlando: go for it, we only have 3 minutes
17:57:23 <sdague> or we can take it to -qa after
17:57:30 <mtreinish> #topic Open Discussion
17:57:33 <mtreinish> salv-orlando: go ahead
17:57:38 <salv-orlando> past 7 days: mysql failure rate 2.79%, pg: 2.04%
17:57:41 <mtreinish> we can move to -qa if it takes longer
17:58:03 <salv-orlando> there is a tail in mysql of the effects of bug 1283522
17:58:04 <uvirtbot> Launchpad bug 1283522 in neutron "DB lock timeout errors with parallel operations" [Critical,Fix committed] https://launchpad.net/bugs/1283522
17:58:11 <salv-orlando> considering past 5 days only
17:58:21 <salv-orlando> mysql failure rate: 2.26%, pg:3.34%
17:58:41 <salv-orlando> but the amount of pg jobs is rather small so I would not regard the last info as statistically significant
17:58:45 <salv-orlando> that's all
17:58:52 <mtreinish> salv-orlando: is that the full parallel jobs or the smoke jobs?
17:58:59 <mtreinish> oh I guess it's smoke if it's pg nm
17:59:21 <mtreinish> ok well we're out of time for today
17:59:23 <salv-orlando> smoke. Full parallel job we have only 21 baseline runs so far. too little to say anything. But we found 3 failures in full job (1/7)
17:59:34 <mtreinish> we can pick anything else up on -qa
17:59:40 <mtreinish> thanks everyone
17:59:48 <mtreinish> #endmeeting