17:00:20 <andreaf> #startmeeting qa
17:00:21 <openstack> Meeting started Thu Mar 30 17:00:20 2017 UTC and is due to finish in 60 minutes.  The chair is andreaf. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:22 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:24 <openstack> The meeting name has been set to 'qa'
17:00:38 <andreaf> Today's agenda #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_March_30rd_2017_.281700_UTC.29
17:00:50 <andreaf> hi - who's here today?
17:00:55 <prateek> Hi andreaf
17:00:57 <chandankumar> andreaf: hello
17:01:02 <oomichi> hi :)
17:01:08 <jordanP_> hi
17:01:15 <blancos> o/
17:01:18 <dustins> \o
17:01:53 <jlwhite> o/
17:02:00 <dankolbrs> o/
17:02:15 <jordanP_> (gerrit is going to be rebooted, so links to review.o.o may not work)
17:02:32 <andreaf> mtreinish, oomichi, gmann, luzC, scarab_: around?
17:02:34 <dpaterson_> 0/
17:03:01 <andreaf> jordanP_: doh! thanks for the info
17:03:02 <oomichi> andreaf: I said hi already :)
17:03:04 <luzC> o/
17:03:05 <scarab_> here
17:03:21 <andreaf> oomichi: hi :) sorry I cannot read
17:03:28 <oomichi> hehe
17:03:41 <andreaf> ok let's get started
17:03:42 <mtreinish> o/
17:03:53 <andreaf> #topic Previous Meeting Action review
17:04:31 <andreaf> There was an action for me to submit topics for the forum - I haven't done it yet but working on it - I'll do it until tomorrow
17:04:31 <clarkb> jordanP_: they should work fine at this point
17:04:45 <jordanP_> yeah
17:05:08 <andreaf> #action andreaf Submit forum session until April 2nd
17:05:13 <andreaf> clarkb: thanks
17:05:58 <andreaf> #topic The Forum
17:06:22 <andreaf> So about the forum - at some point we need to put together a presentation for the onboarding session
17:06:42 <andreaf> I was thinking it could be something not too distant from the project update talk anyways
17:07:28 <andreaf> but there's still time for that, and if you have something that you think it's really good material to attract new contributors please share in the etherpad #link https://etherpad.openstack.org/p/BOS-QA-onboarding
17:07:39 <andreaf> moving on as we have a long agenda :)
17:07:46 <andreaf> #topic Gate stability
17:08:06 <andreaf> Latest update was in the ML from clarkb and mtreinish: http://lists.openstack.org/pipermail/openstack-dev/2017-March/114648.html
17:08:25 <jordanP_> thanks for the update, nice email
17:08:26 <andreaf> since the mail the patch to fix a race was merged which is good
17:08:35 <mtreinish> things are actually pretty good right now
17:08:42 <clarkb> except for libvirt
17:08:50 <jordanP_> I am especially looking forward to see the test with the new libvirt
17:09:27 <jordanP_> I'd rather canonical release a fixed libvirt 1.3.1 then to ask everybody to move to 2.5 in production...
17:09:46 <clarkb> yes, there are problems with being split like that too (wheel mirror for example)
17:09:47 <jordanP_> idk what's the plan here
17:10:10 <clarkb> mostly just trying to see if it is viable before worrying about details, though I think many of the deployment tools out there today use UCA
17:10:20 <jordanP_> is someone from Canonical aware that the current 1.3.1 have issues w.r.t mem management ?
17:10:37 <clarkb> yes jamespaige was notified in nova channel yesterday
17:10:40 <jordanP_> yeah, UCA is a viable option
17:10:44 <jordanP_> good !
17:11:57 <andreaf> thanks everyone for the good work on getting things back in a good state
17:12:10 <jordanP_> I guess if we sort the libvirt thing out, we can call close the "gate stability fire"
17:12:34 <andreaf> There's a new job I've been working on in parallel, since we stopped running a lot of the scenarios
17:12:38 <jordanP_> I still think we good use less memory for our jobs
17:12:58 <jordanP_> I have this patch that I would like to have eyes on:
17:13:01 <jordanP_> #link https://review.openstack.org/#/c/450207/
17:13:06 <clarkb> jordanP_: yes on the memory usage front I definitely think we should keep pushing on that particularly to the various projects themselves
17:13:11 <andreaf> jordanP_: yeah there is more ongoing work to be done I think - which is something I'd like to talk about at the forum
17:14:34 <andreaf> Scenario job: #link https://review.openstack.org/#/c/451769/
17:14:59 <andreaf> We have a new non voting job which runs all scenario against tempest changes (in check only)
17:15:46 <andreaf> I tried adding compute migration tests in there and setting concurrency to 2, and it seems to work well (I only have three test runs though)
17:15:49 <jordanP_> will this patch increase the concurrency for all jobs or just that non voting job ?
17:16:24 <andreaf> jordanP_: for all the jobs that use scenario env - which is only the non voting job
17:16:32 <jordanP_> I think this is a bit early, running migration tests will slow our jobs, and pushing the concurrency is risky at  the moment
17:16:44 <jordanP_> ah ok, in that case, lgtm
17:17:28 <andreaf> jordanP_: in any case I'd like to keep that job running on tempest for a while and see how it behaves
17:17:44 <jordanP_> yep
17:17:45 <andreaf> jordanP_: if it proves itself stable we could then take scenario tests out of the main job and add this one to the gate
17:17:49 <felipemonteiro> o/
17:17:57 <jordanP_> yep
17:18:10 <andreaf> in any case... time to move on to the next topic
17:18:32 <andreaf> #topic spec reviews
17:18:50 <andreaf> Upgrade testing toolset: #link https://review.openstack.org/#/c/449295/
17:19:27 <andreaf> castulo: do you want to say something about it?
17:19:35 <castulo> yes, we created this spec as a result of our conversation during the PTG about testing upgrades
17:20:05 <castulo> so please review and provide feedback, we would like to hear your concerns and opinions before starting to work on this
17:20:11 <jlwhite> Has anyone had a chance to review it?
17:20:22 <jlwhite> If not we could speak on the functionality
17:20:33 <andreaf> I read it before the meeting
17:20:45 <andreaf> masayukig had some comments on it as well
17:21:27 <andreaf> I guess my main concern with the spec as it looks today is that it looks a bit too wide in scope
17:21:45 <andreaf> I think it would be good to focus on a minimalistic set of requirements / use cases
17:22:18 <jlwhite> Well initially for Pike it will cover system health checks and persistence
17:22:23 <jlwhite> Similar to what grenade runs
17:22:44 <jlwhite> When you say minimalistic set in what sense are you referring andreaf?
17:22:51 <jlwhite> Just getting more background
17:23:01 <castulo> yes, we want to focus in the test needed for a rolling upgrade during pike
17:23:24 <andreaf> jlwhite: yes I mean to identify what are the things people are working to focus on first
17:23:32 <castulo> during future release(s) we would add more tests to cover zero downtime upgrades and zero impact upgrades
17:23:35 <andreaf> jlwhite: the MVP if you will
17:24:32 <jlwhite> okay got it
17:24:56 <jlwhite> The goal of the tool is to be utilized by the deployment teams when constructing there upgrade methodologies
17:25:04 <luzC> would it work to have the spec as the general broader picture, have use cases to define the architecture of each suite  and define tasks to divide the work?
17:25:24 <jlwhite> We spoke with Michal from Kolla and and our goals for Pike align with his vision of a MVP
17:26:18 <andreaf> jlwhite: well things like rollback and a gui sounded to me a bit far reaching requirements
17:27:02 <andreaf> jlwhite: yeah I agree a good value to get would be information for deployer on how to build a good upgrade strategy
17:27:26 <andreaf> jlwhite: similar to what grenade does in terms of identifying special steps to be executed during upgrade
17:28:11 <jlwhite> andreaf: cool
17:28:14 <andreaf> jlwhite: the new upgrade testing tool should be able to identify orchestration needs to achieve seamless upgrade
17:29:17 <jlwhite> andreaf: yep I agree
17:29:27 <andreaf> e.g. capture dependencies like you need to upgrade keystone across the board before you can start rolling nova (just an example, not true)
17:30:24 <andreaf> jlwhite: and also I believe there would be great value in running those in the gate so we can see when a change brings in a new dependency and think twice if there is any way we can change things to keep the upgrade experience better
17:31:16 <andreaf> I left these comments on the spec more or less as well, it would be great if folks could review it as well
17:31:29 <jlwhite> andreaf: Down the road we see this as a possibility. But for now we are just aiming for those Pike goals
17:31:46 <jlwhite> andreaf: I figured that :), I'm making you repeat.  I will review it
17:32:34 <andreaf> jlwhite: I think we should start getting something very small in place and putting it in the gate as soon as it does something even minimal
17:33:38 <jlwhite> andreaf: We will definitely start small, as everything needs to work together
17:33:44 <andreaf> jlwhite, castulo, luzC : do you think there is space for further discussing upgrade testing at the forum? are you going to be in Boston?
17:33:56 <castulo> andreaf: in a future stage we envision that the tool will handle the orchestration of the upgrade too, so it can be used at the gate, but as a first stage it will provide tools that can be consumed by users or downstream projects to test upgrades
17:34:04 <castulo> andreaf: no, we are not going :/
17:34:34 <jlwhite> same here, 82% not going
17:34:35 <jlwhite> haha
17:35:19 <andreaf> castulo: ok, that was not clear for me from the spec
17:35:19 <jlwhite> But I do feel it can be discussed in some way if we are able to call in
17:35:25 <luzC> andreaf right now the idea is for deployer to consume the tool and add it to their upgrade gate jobs,
17:35:32 <jlwhite> Or if someone can bring it up on our behalf
17:35:43 <tosky> castulo: if I read correctly, the tool should still be able to work independently even when the orchestration is implemented, is it correct?
17:36:03 <castulo> tosky: yes
17:36:09 <tosky> thanks; it would be interesting to integrate it for example in TripleO upgrade jobs
17:36:11 <luzC> andreaf as castulo and jlwhite mentioned initially we are planning on providing the test suites needed to test across the upgrade from a single place
17:36:33 <luzC> tosky yes
17:37:50 <luzC> andreaf do you think the topic should be added as in Boston?
17:38:46 <andreaf> luzC: well it may be a bit early to make this a cross project discussion but I wanted to ask you what you though about it?
17:39:39 <andreaf> ok, I'll assume not for now
17:39:40 <luzC> it would be good to get early feedback, we are talking to people from deployment working group,
17:39:53 <andreaf> luzC: heh sorry I was too quick
17:39:58 <luzC> hahaha
17:40:09 <luzC> and keystone
17:40:27 <luzC> but just collecting requirements since we still have nothing to show
17:40:53 <andreaf> luzC: ok do you want to propose a session?
17:41:11 <luzC> let me give think about it and get back to you
17:41:27 <andreaf> ok cool
17:41:32 <andreaf> next topic?
17:41:35 <andreaf> 3...
17:41:37 <andreaf> 2..
17:41:38 <jlwhite> sure
17:41:38 <andreaf> 1.
17:41:50 <andreaf> #topic Tempest
17:42:21 <andreaf> 4/4 is Mitaka EOL
17:42:23 <jordanP_> we have one flaky test here; https://review.openstack.org/#/c/450032/
17:42:31 <jordanP_> I am not sure what's best with it
17:42:34 <andreaf> so we'll tag Tempest for end of support on Mitaka
17:42:53 <mtreinish> andreaf: did you ever get things setup in the releases repo?
17:43:26 <andreaf> mtreinish: no, I need to go and read the docs until then
17:43:38 <felipemonteiro> Question: is there any current goal to make the orchestration client stable and add it to lib? I don't even think it has unit tests?
17:43:40 <andreaf> and talk to the release folks :)
17:44:24 <jordanP_> felipemonteiro, no, technically we should remove all heat tests and clients out of tzmpest
17:44:28 <andreaf> felipemonteiro: no the goal is the other way around, to move it out of tempest, at least until interop folks decide to include heat in their program
17:45:28 <andreaf> jordanP_: heh my preference would be to fix the race in the tests rather than skip it
17:45:29 <andreaf> Racy test: #link https://review.openstack.org/#/c/450032/
17:45:42 <felipemonteiro> jordanP_ andreaf: I see, thanks.
17:45:46 <andreaf> jordanP_: as I fear the test will then stay skipped forever, and the failure rate is 0.1%
17:46:00 <jordanP_> yeah, that would be my preference too but in the mean time...
17:46:17 <jordanP_> it's not the only test that is skipped
17:46:34 <andreaf> if I can give you an action to propose a fix :D
17:46:45 <jordanP_> no, that won't work :)
17:46:46 <andreaf> I'm fine with skipping meanwhile
17:47:25 <jordanP_> from time to time, people with some free time, go over the list of skipped tests and fix them
17:47:30 <andreaf> well the bug is there so if you want to skip I'm fine with it - I did +1
17:48:08 <jordanP_> you should +3 it, you're the ptl :p
17:48:17 <andreaf> :D
17:48:42 <andreaf> #topic Tempest Bug Triage
17:48:52 <andreaf> prateek: do you have any update for us?
17:49:13 <prateek> andreaf, yup, i have listed my findings in the etherpad
17:49:34 <prateek> also there are some bugs which have been marked for Feb27 week
17:49:43 <prateek> which are of this week actually
17:49:53 <prateek> https://etherpad.openstack.org/p/tempest-weekly-bug-report
17:50:08 <prateek> i have written IuzC to recheck that
17:50:32 <prateek> I am doing triage for first time so I am not sure how I am to report that in the meeeting
17:51:07 <prateek> there are 5 bugs which are confirmed
17:51:15 <prateek> 1 in progress
17:51:20 <prateek> and 3 incomplete
17:51:25 <andreaf> prateek: yeah that's fine, if there is any bug which requires attention or clarification this is a good forum to mention them
17:51:35 <prateek> i couldnt test 1 beacuse it required ssl setup
17:52:05 <jordanP_> ssl can be setup with enable_service tls-proxy in local.conf
17:52:20 <jordanP_> (for information)
17:52:52 <jordanP_> but I mean, you don"t have to repro every bug yourself, that's a lot of work !
17:52:59 <andreaf> prateek: ok thanks for your work on triaging
17:53:05 <jordanP_> yes, thank you
17:53:06 <prateek> andreaf, nope, there is no bug that needs clarification
17:53:16 <prateek> welcome guys
17:53:35 <prateek> jordanP_, ok, so i would find some video on how to triage
17:53:38 <andreaf> jordanP_: well I won't complain about a job well done :)
17:53:44 <prateek> :)
17:54:50 <andreaf> prateek: in many cases a log from the gate is enough to prove a bug, but if there is no log and no log link in the bug and it's complex to reproduce you can push back and mark as incomplete
17:55:04 <andreaf> asking for the relevant data
17:55:24 <prateek> andreaf, ok, would do, thanks
17:55:27 <jordanP_> yeah, that's usually the way to go
17:55:56 <andreaf> so, time is running out - is there any topic someone in the meeting would like to cover still?
17:56:06 <andreaf> else I'll switch to critical reviews
17:56:52 <andreaf> #topic critical reviews
17:57:02 <mtreinish> #link https://review.openstack.org/#/c/447263/
17:57:09 <mtreinish> #link https://review.openstack.org/#/c/449422/
17:57:16 <mtreinish> just 2 fixes from masayukig on o-h
17:58:41 <andreaf> and this change for the scenario job from my side: #link https://review.openstack.org/#/c/451769/
17:59:13 <oomichi> Id like to put https://review.openstack.org/#/c/450411/ here
17:59:15 <oomichi> #link https://review.openstack.org/#/c/450411/
17:59:58 <andreaf> yeah well Catherine Diep already +1 so I think it should be fine
18:00:10 <andreaf> but perhaps try to ping her
18:00:18 <andreaf> thanks everyone
18:00:21 <andreaf> #endmeeting