17:00:24 <oomichi> #startmeeting qa
17:00:25 <openstack> Meeting started Thu Sep 15 17:00:24 2016 UTC and is due to finish in 60 minutes.  The chair is oomichi. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:26 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:29 <openstack> The meeting name has been set to 'qa'
17:00:35 <oomichi> hi, who's here today?
17:00:37 <dpaterson> o/
17:00:42 <scottda> hi
17:00:49 <frickler> o/
17:01:50 <mtreinish> o/
17:01:59 <clu_> o/
17:02:10 <oomichi> OK, let's start a meeting
17:02:18 <oomichi> #topic QA Code Sprint
17:02:28 <oomichi> #link https://wiki.openstack.org/wiki/Sprints/QAInfraNewtonSprint
17:02:46 <oomichi> we have mid-cycle meetup of QA/Infra next week
17:03:39 <oomichi> I have sent a remider for attendees, please let me know if some attendees dont recieve it
17:04:01 <oomichi> The topics of the meetup is written on  https://etherpad.openstack.org/p/qa-infra-newton-midcycle
17:04:12 <topol> o/
17:04:12 <mtreinish> oomichi: we should also cancel next week's irc meeting because most people will be travelling
17:04:14 <oomichi> Good enough topics on now
17:04:23 <oomichi> mtreinish: +1
17:04:46 <oomichi> yeah, the next meeting is not our timezone, but I will send a mail for the notification
17:04:57 <oomichi> thanks for picking it up
17:05:20 <oomichi> #action oomichi sends a mail of cancellaration of next meeting
17:05:45 <oomichi> please write more topics on the etherpad if having
17:05:57 <oomichi> #link  https://etherpad.openstack.org/p/qa-infra-newton-midcycle
17:06:19 <oomichi> ok, I am guessing we have prepared for the next meetup :)
17:06:36 <oomichi> let's move on if not having more topic about the meetup
17:07:02 <oomichi> #topic Next OpenStack Summit
17:07:30 <oomichi> this is just a notification, QA team has 7 design session slots for the next summit.
17:07:59 <mtreinish> do we have a planning etherpad up yet?
17:08:13 <oomichi> that is as our expectation, and we will need to consider what discussions should be on the summit after the meetup
17:08:27 <oomichi> mtreinish: we will have the meetup nextweek before the summit
17:08:42 <oomichi> mtreinish: so it is good to prepare the etherpad after :)
17:09:00 <oomichi> based on the meetup discussion
17:09:15 <mtreinish> oomichi: sure, but not everyone will be at the meetup and there are some topics we won't necessarily have time to discuss in walldorf next week
17:09:37 <mtreinish> it doesnt hurt to have the etherpad setup and available so people can add to it now
17:10:18 <oomichi> mtreinish: That makes sense. ok, I will prepare the etherpad for next summit
17:10:38 <oomichi> #action oomichi creates/prepare an etherpad for the next summit
17:11:15 <oomichi> OK, lets move on if we dont have more topics for next summit
17:11:43 <oomichi> #Spec Reviews
17:12:07 <oomichi> #topic Spec Reviews
17:12:12 <oomichi> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
17:12:29 <oomichi> I pushed some patches for the cleanup
17:13:20 <oomichi> and some specs still need to be reviewed. tempest-stable-fixtures and resource
17:13:33 <oomichi> happy if getting more eyes
17:14:15 <oomichi> anyone have thinking about qa-specs?
17:14:51 <mtreinish> oomichi: well I have some general thoughts about it, because we've been not so good about specs in newton (or mitaka too)
17:15:02 <mtreinish> so maybe we need to discuss changes to make the process
17:15:10 <mtreinish> but that's something we can save for barcelona
17:15:46 <oomichi> mtreinish: yeah. on nova-specs, it has backlog dir. it is nice to have such dir on qa-spec
17:16:18 <mtreinish> oomichi: we have the opposite problem, not enough specs and not enough reviews
17:17:18 <oomichi> humm, how about having some spec review day?
17:17:47 <oomichi> in general, we don't requrie specs so strong in many cases like implementing test cases
17:18:20 <oomichi> then we don't have so many specs now, I prefer current way.
17:19:03 <oomichi> and the remaining specs need more reviews to discuss, and the spec review day will help for moving forward
17:19:33 <mtreinish> oomichi: heh, a specs review day with < 5 open specs :)
17:19:54 <oomichi> mtreinish: yeah, right :)
17:19:55 <mtreinish> but it's just something I think we want to start thinking about for ocata
17:20:35 <oomichi> ok, that will be a nice step for ocata because of the short dev time
17:21:36 <oomichi> OK, Let's move on the next topic
17:21:40 <oomichi> #topic Tempest
17:22:02 <oomichi> we still are triaging bugs on tempest
17:22:27 <oomichi> and the report number continues decreating like https://github.com/oomichi/bug-counter#current-graph
17:23:07 <mtreinish> oomichi: it's kinda hard to tell there, the legend is over the graphs
17:23:07 <oomichi> but still many open reports, so it is great if we have more helps for triaging
17:23:34 <mtreinish> oomichi: I'll take a look after lunch and push a patch to the repo to fix that
17:23:41 <mtreinish> (and the color map)
17:23:54 <oomichi> mtreinish: yeah, I am not good at doing that. The help of graph is also great for us/me :)
17:24:05 <oomichi> mtreinish: thanks ! :)
17:24:48 <oomichi> yeah, the latest data is not readable on the graph, sorry :-(
17:25:24 <oomichi> ok, the next topic of tempest is negative test on the agenda
17:25:33 <oomichi> clu_: that is your turn?
17:25:41 <clu_> hi oomichi, yes
17:25:48 <clu_> What is the consensus on negative test cases?
17:25:49 <oomichi> clu_: please go ahead
17:26:04 <oomichi> #link https://github.com/openstack/tempest/blob/master/HACKING.rst#negative-tests
17:26:22 <oomichi> we have written the consensus on the above
17:26:33 <clu_> yes, i've seen that blurb - but i still see negative tests in tree
17:26:39 <mtreinish> oomichi: I thought you and hogepodge were going to provide some more detailed guidance in that paragraph
17:26:49 <mtreinish> because it's pretty high level right now
17:26:51 <clu_> those that are similar to the ones i'd like the write
17:27:24 <oomichi> clu_: basically, new negative tests should be implemented in each project repo
17:27:28 <clu_> are there plans to move all negative tests into separate plugins per project?
17:27:46 <oomichi> clu_: which project do you work for?
17:28:16 <clu_> swift
17:28:17 <oomichi> clu_: no, the existing negative tests are kept on tempest repo
17:29:07 <clu_> what about for interoperability? these are valid end user cases
17:29:14 <oomichi> clu_: if swift repo has functional tests, it is nice to implement negative tests on that
17:30:06 <oomichi> clu_: yeah, that is case by case. that is reason why current tests are kept on tempest
17:30:19 <clu_> yes, but tests for interoperability would be beneficial for tempest
17:31:06 <mtreinish> clu_: right I agree. But, we have to be targetted here. The negative space is infinite
17:31:11 <oomichi> clu_: yeah, but tempest is not good place for all coverage of negative tests as the hacking
17:31:15 <clu_> if we put new negative tests in a separate plugin, how do people benefit from it?
17:31:18 <mtreinish> and we don't want to be in the business of fuzz testing every possible thing
17:31:54 <topol> So one issue on this is having a perceived level playing field for interop tests as opposed to being in  a project.  And so when I had conversations with TC folks in Austin Tempest was the agreed to levele playing field
17:32:37 <mtreinish> topol: right I agree, that's why I was saying we include some negative test
17:32:44 <mtreinish> I don't know which patch this is in reference too
17:33:19 <mtreinish> but we just need to be cautious about adding negative tests, ebcause it's a slippery slope where everyone wants to add a test that has '$" or a 'DROP TABLE' in it
17:33:35 <clu_> https://review.openstack.org/#/c/308508/
17:33:39 <mtreinish> we had to say no to all negative tests for a while because of that in the past
17:34:12 <mtreinish> but that was a couple years ago
17:34:26 <oomichi> yeah, but it was hard to say at the time
17:35:09 <oomichi> clu_: these cases are implemented in swift side?
17:36:16 <mtreinish> yeah looking through those test cases, I'm not opposed to most of them. They seem like some standard interop sanity checks
17:36:42 <mtreinish> oomichi: this is why I was expecing you and hogepodge to expand on that section in the docs to provide more targetted guidance
17:36:47 <mtreinish> because right now it's very confusing
17:37:03 <clu_> yes they are, but we've had customers say we work with said they wanted to see 100% compatibility, and for these tests to be tempest
17:37:22 <clu_> because they wanted these to be picked up by refstack
17:37:57 * notmyname sees that people are talking about swift
17:37:59 <oomichi> clu_: refstack is picking tests from current tempest
17:38:31 <clu_> they want these new negative tests to be picked up by refstack
17:38:43 <topol> interop is gonna be a huge theme in OpenStack Barcelona and little steps like beefing up the interop tests also help
17:38:56 <oomichi> clu_: sometimes, error response codes are wrong for each project team and they changed error status codes
17:39:27 <oomichi> then it is nice to handle all coverage on each project. The policy of error status code is different from projects
17:39:34 <mtreinish> topol. clu_: I understand the desire here, and I think most of the tests in the patch are fine for inclusion personally. Rebase the patch and get it in a mergable state and I'll give it a real review
17:39:55 <topol> awesome mtreinish, THANKS
17:40:05 <clu_> mtreinish: i will clean it up
17:40:30 <mtreinish> oomichi: and I really strongly think we need to reword that paragraph to include more details. Because it doesn't give us any real guidance now
17:40:38 <mtreinish> oomichi: this conflict isn't going to go away
17:41:06 <oomichi> mtreinish: OK, I see the point to avoid such confution.
17:41:10 <mtreinish> and not having real guidelines on where we draw the line between an ok negative test for interop and a useless fuzz test is going to just cause problems
17:42:09 <oomichi> mtreinish: OK, let's push a new paragraph after the meeting
17:42:44 <oomichi> do we have more topics for Tempet?
17:43:07 <mtreinish> oomichi: I had one quick fix for a pretty obvious ace condition armax found yesterday
17:43:25 <mtreinish> #link https://review.openstack.org/#/c/370479
17:43:26 <oomichi> mtreinish: oh, for gate problem?
17:43:49 <armax> mtreinish: thanks!
17:45:30 <oomichi> mtreinish: ok, I will review it after the meeting
17:45:37 <mtreinish> oomichi: thanks
17:46:12 <oomichi> is it fine to move on the next topic?
17:46:39 <oomichi> #topic DevStack + Grenade
17:47:15 <oomichi> do we have items for devstack and grenade this week?
17:47:27 <frickler> there is a patch for devstack to unfail gate-tempest-dsvm-neutron-identity-v3-only-full-nv
17:47:28 <mtreinish> I don't have anything this week on devstack or grenade
17:47:36 <frickler> see https://bugs.launchpad.net/glance-store/+bug/1620999 and https://review.openstack.org/369675
17:47:37 <openstack> Launchpad bug 1620999 in glance_store "swift driver ignores user_domain_name and project_domain_name settings" [High,Triaged]
17:47:37 <mtreinish> frickler: do you have a link?
17:47:43 <mtreinish> ah, you're too quick :)
17:47:51 <mtreinish> #link https://review.openstack.org/#/c/369675/
17:48:21 <frickler> I'll update the commit message with nikhils comments after this
17:48:38 <mtreinish> frickler: ok cool, I've added it to my review list for after the meeting
17:49:55 <oomichi> cool. OK, let's move on
17:50:03 <oomichi> #topic openstack-health
17:50:20 <oomichi> do we have items for o-h?
17:50:35 <mtreinish> there are 2 things worth mentioning
17:50:46 <oomichi> mtreinish: ok, go ahead
17:51:03 <mtreinish> timothyb89 got a mostly complete version of the nvd3 removal up:
17:51:05 <mtreinish> #link https://review.openstack.org/363934
17:51:17 <mtreinish> the patch still needs cleanup (and to be split, its too big)
17:51:31 <mtreinish> but he's looking for feedback on missing things in the ui
17:51:41 <timothyb89> there's a demo with the changes as well: http://logs.openstack.org/34/363934/7/check/gate-openstack-health-nodejs4-npm-run-test/a48991c/reports/build/#/
17:51:43 <mtreinish> since it does change a lot of things
17:51:55 <mtreinish> #link http://logs.openstack.org/34/363934/7/check/gate-openstack-health-nodejs4-npm-run-test/a48991c/reports/build/#/
17:52:27 <mtreinish> timothyb89: heh, the mouse tracing is still is all messed up for me :)
17:52:32 <timothyb89> the main difference is that it should perform a lot better
17:52:41 <timothyb89> mouse input is still a bit broken for high-dpi displays, though
17:53:13 <mtreinish> the other thing was I started a WIP on adding a run_time graph to other pages:
17:53:16 <mtreinish> #link https://review.openstack.org/370913
17:53:28 <mtreinish> it still needs a ton of work though
17:53:42 <mtreinish> if someone wanted to help out on that, since I'm not sure when I'll get another chance to revisit it
17:54:45 <oomichi> cool, I'd like to see it deeper after this.
17:54:51 <oomichi> mtreinish: thanks for introducing them
17:55:18 <oomichi> ok, lets move on next topic
17:55:26 <oomichi> #topic building custome cirros images
17:55:36 <oomichi> frickler: is that your turn?
17:55:41 <frickler> right
17:55:43 <frickler> so I've written up my argumentation and rough plan at this etherpad
17:55:50 <frickler> #link https://etherpad.openstack.org/p/cirros-respin
17:55:56 <frickler> I've been wondering how to do this for quite some time now, and when another round of cirros-bashing started earlier today in #openstack-neutron,
17:55:59 <frickler> I came up with this etherpad
17:56:08 <frickler> main question I want to raise right now is whether this project would fit under the QA team umbrella
17:56:13 <frickler> or if you rather say "go to infra" or "don't do this, we will stick to cirros"
17:56:21 <frickler> I also must admin the smoser does not seem to like this plan, but I still hope that he might be convinced to join us in the end
17:56:45 <frickler> (I expected time to run short so I prepared this for copy&paste ;)
17:56:55 <mtreinish> heh
17:56:57 <oomichi> frickler: I feel it is the best to work together with cirros project itself, is it hard?
17:57:09 <mtreinish> frickler: so I think if we can work with smoser and cirros that would be the best
17:57:28 <mtreinish> but if it's an intractable problem then it fits fine under qa
17:57:29 <frickler> I've tried this since vancouver and it is very very slow moving
17:58:20 <frickler> it is also hard to get other people to contribute the way cirros is currently built
17:58:47 <mtreinish> frickler: yeah it's all lp right? (and does it use bzr)
17:58:56 <mtreinish> that workflow is very hard to get people to use
17:58:59 <frickler> bzr is pita, yes
17:59:29 <frickler> and there is not CI and in the end smoser is the single head
17:59:50 <oomichi> it is nice to get opinions widely I feel, how about sending a mail about that?
17:59:58 <mtreinish> yeah, that's not a good situation to have something we depend on be in
18:00:03 <frickler> yeah, that would be the next step probably
18:00:23 <oomichi> ok, the time is coming
18:00:25 * regXboi looks at the clock
18:00:35 <oomichi> let's discuss on qa-channel
18:00:39 * mtreinish winds back the clock for extra time
18:00:39 <oomichi> #endmeeing
18:00:40 <frickler> k, thx
18:00:44 <oomichi> #endmeeting