17:00:05 <andreaf> #startmeeting qa
17:00:06 <openstack> Meeting started Thu May 25 17:00:05 2017 UTC and is due to finish in 60 minutes.  The chair is andreaf. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:07 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:09 <openstack> The meeting name has been set to 'qa'
17:00:32 <andreaf> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Agenda_for_May_25th_2017_.281700_UTC.29 Today's agenda
17:00:50 <andreaf> Hello, who's here for the meeting today?
17:00:54 <blancos> o/
17:01:10 <mtreinish> o/
17:01:36 <prateek> o/
17:01:38 <felipemonteiro> o/
17:02:37 <andreaf> oomichi and chandakumar cannot make the meeting  today
17:02:48 <andreaf> ok let's start
17:03:47 <andreaf> So first on the agenda, I wanted to welcome zhufl to tempest-core!
17:06:17 <mtreinish> heh, it's a bit late in china...
17:09:26 <mtreinish> andreaf: did you drop?
17:10:07 <andreaf> Also I sent an email proposing to move the 9:00 UTC meeting to 8:00 UTC, which is probably not very relevant for folks in this meeting today
17:10:09 <andreaf> but please let me know if you have any opinion about the proposed changed
17:10:10 <andreaf> change
17:10:12 <andreaf> #topic Previous Meeting Action review
17:10:16 <andreaf> Actions from last week where for me to send an email to the ML about the proposed change in the meeting schedule, which I did
17:10:21 <andreaf> and one for blancos / felipemonteiro to update the spec for patrole to a patrole folder, which was also done
17:10:22 <andreaf> #link https://review.openstack.org/#/c/461140/
17:10:23 <patchbot> patch 461140 - qa-specs - RBAC testing multiple policies
17:10:24 <andreaf> #topic PTG
17:10:27 <andreaf> About PTG, it may be a bit early but the format for next PTG is being defined, so if you have ideas / opinions about how it should be structured now it's the time to speak :)
17:10:29 <andreaf> #link http://lists.openstack.org/pipermail/openstack-dev/2017-May/117369.html ML thread by thierry
17:10:36 <andreaf> The idea for QA is that day1 and 2 we will have a help room where we will take turns to be there and help people with questions / issues on QA topics
17:10:37 <andreaf> who's not in the help room can roam around in the WG rooms in parallel
17:10:39 <andreaf> day 3,4,5 we can have QA sessions and time to hack on something
17:10:41 <andreaf> mtreinish: indeed probably 3am or so
17:10:43 <andreaf> ping
17:10:59 * andreaf gets out of the hammock and looks for better wifi signal :P
17:11:26 <andreaf> Any comment on the PTG?
17:12:08 <mtreinish> andreaf: it's an interesting idea. We'll see how it turns out
17:13:30 <andreaf> yes it's hard to say in advance - but I think it's interesting for a lot of us to be in WG sessions - so if we can rotate in the help room hopefully it will work
17:13:51 <andreaf> ok moving on
17:13:57 <andreaf> #topic Gate stability
17:14:40 <andreaf> the gate was broken shortly because of an issue in oslo.i18n, but it's now back online and it looks stable enough
17:15:11 <andreaf> #link https://goo.gl/ptPgEw
17:15:16 <clarkb> our classification rate is 2% for gate failures
17:16:09 <andreaf> clarkb: heh I'll have to check the exact numbers :)
17:16:51 <mtreinish> clarkb: yeah not many people are pushing e-r queries
17:17:14 <andreaf> clarkb: oh I got it know e-r classification rate
17:17:30 <mtreinish> granted are failure rate is much lower than it has been for a while so keeping the classification percentage up is harder
17:17:36 <clarkb> that said I think things are relatively stable, its just that the issues we do have we aren't catching/tracking well
17:17:41 <mtreinish> but 2% is not good, we're basically catching nothing
17:17:48 <mtreinish> whcih probably means well get bit before too long
17:18:57 <andreaf> clarkb, mtreinish: I've not hit failures recently - I wonder if we should setup a kind of rota, like for bug triage?
17:19:16 <andreaf> clarkb, mtreinish: a weekly task to keep an eye on unclassified failures?
17:20:30 <andreaf> I can setup a calendar for that and send a message to the ML - if we get volunteers I'm more than happy to help them getting started
17:21:52 <andreaf> you're silence tells me that either by bouncer is failing again or you don't believe this will help :P
17:21:59 <andreaf> but I'm going to give it a try ;)
17:22:49 <mtreinish> andreaf: it's worth a try I guess. I just know in the past people seem confused by the process of getting a bug out of a gate failure and then opening an e-r query
17:23:47 <andreaf> mtreinish: yes but perhaps if we offer mentoring it will help
17:24:20 <andreaf> mtreinish, clarkb: do you have a recorded session about troubleshooting the gate?
17:24:21 <andreaf> I could add the link
17:24:37 <clarkb> mtreinish: and mriedem have one somewhere iirc
17:24:50 <mtreinish> andreaf: https://www.youtube.com/watch?v=sa67J6yMYZ0
17:24:57 <andreaf> thanks
17:25:22 <andreaf> #action andreaf send an email about classification rota and mentoring
17:25:52 <andreaf> Also about gate stability, I proposed enabling ssh validation by default on master in devstack
17:25:59 <andreaf> #link https://review.openstack.org/#/c/458678/ devstack patch
17:26:05 <patchbot> patch 458678 - openstack-dev/devstack - Enable ssh validation by default
17:26:10 <mriedem> look at those young hunks
17:26:13 <mriedem> so full of hope
17:26:25 <andreaf> I'd like more opinions on that
17:26:46 <mtreinish> heh, the first thing that stood out to me was how much more hair I had back then
17:26:51 <andreaf> heh that was Vancouver
17:28:20 <andreaf> anything else on gate stability?
17:28:21 <andreaf> #topic Spec reviews
17:29:54 <andreaf> There are a few outstanding specs - I did not have time to read them yet
17:30:05 <andreaf> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
17:30:24 <mtreinish> andreaf: #link https://review.openstack.org/#/c/173334/
17:30:25 <patchbot> patch 173334 - qa-specs - Add bp tempest-resources
17:30:32 <mtreinish> that is one back from the dead :)
17:31:13 <andreaf> heh - oomichi took it over some time ago but then we had other priorities
17:32:20 <andreaf> mtreinish: there was no comment / complaint in that area in the user feedback session at the summit, but maybe the right audience for that was missing...
17:32:59 <andreaf> mtreinish: at the moment that's a a bit low in my list of priorities since we have limited bw and a few urgent things we need to close around stable interfaces
17:33:00 <mtreinish> andreaf: yeah I'm not sure that was the right audience, it was kind of a quiet session
17:34:45 <andreaf> #link https://review.openstack.org/349730 is a bit dead in the water, since we decided to make the current test.py stable
17:34:46 <patchbot> patch 349730 - qa-specs - Add bp tempest-stable-fixtures
17:35:01 <andreaf> so I should probably abandon that
17:36:21 <andreaf> #link https://review.openstack.org/#/c/443504/ HA testing could use some reviews
17:36:22 <patchbot> patch 443504 - qa-specs - Added spec for high availability testing
17:36:26 <andreaf> if we can create common tools for non functional type of testing it's a win for everyone
17:37:49 <andreaf> #link https://review.openstack.org/#/c/461140/ also needs reviews, I already mentioned this one
17:37:50 <patchbot> patch 461140 - qa-specs - RBAC testing multiple policies
17:38:05 <andreaf> anything else on specs?
17:38:56 <andreaf> #topic Tempest
17:39:09 <andreaf> Anything to discuss on Tempest?
17:39:28 <mtreinish> andreaf: are we still spewing deprecation warnings on normal tempest runs?
17:39:33 <mtreinish> or did all the fixes merge
17:39:48 <andreaf> mtreinish: I think they all merged (3 of them)
17:39:55 <mtreinish> #link https://review.openstack.org/#/c/466344/
17:39:56 <patchbot> patch 466344 - tempest - Add option for whether the cloud supports floating...
17:40:05 <mtreinish> that fixes my complaint from the feedback session
17:40:16 <mtreinish> andreaf: ok, we should be more careful in the future
17:40:22 <andreaf> mtreinish: but i haven't check if there is any deprecation warning left
17:40:39 <mtreinish> deprecation warnings are to signal users, and we shouldn't emit them as part of normal operation
17:40:50 <mtreinish> unless its because of a config or cli flag or some other user choice
17:41:20 <andreaf> mtreinish: heh good point - otherwise people will get into the habit of ignoring them
17:41:46 <andreaf> mtreinish: I'm halfway through the review of your patch - it looks good to me so far
17:42:27 <andreaf> chandakumar has been working on a sanity check for tempest plugins
17:42:55 <andreaf> the idea is to have a job that takes any known plugin and one by one it tries to install it, run tempest init and tempest run -l
17:43:12 <andreaf> we will maintain a blacklist of known broken plugins
17:43:42 <andreaf> it's not guarantee that those will work 100% but at least they will not break other plugins at install/ list time
17:44:31 <andreaf> and this helps us knowing which plugins are kind of dead and which are actually in use
17:44:54 <mtreinish> andreaf: ok, but why is that a tempest job?
17:45:25 <andreaf> mtreinish: because it helps us when doing changes to tempest to see if we break something
17:45:37 <andreaf> mtreinish: since plugins use non stable interfaces
17:46:00 <mtreinish> andreaf: except the vast majority of those failures will be caused by setup issues in the plugins
17:46:46 <andreaf> mtreinish: the discovery bit should detect broken imports
17:47:12 <mtreinish> ok, so we're adding a job to tempest to catch broken imports in other projects?
17:47:33 <mtreinish> still seems out of place to me
17:47:39 <andreaf> mtreinish: yes because we may break those import by changing things in tempest
17:48:20 <andreaf> mtreinish: if we had all stable interface ready earlier we would not need this
17:48:33 <andreaf> mtreinish: but I think it's fair to try and keep an eye out for plugins that depend on tempest unstable interfaces
17:48:37 <felipemonteiro> i can't get patrole to work in py35-backed environment due to import error, although I can't re-create the issue locally with devstack USE_PYTHON3=True, still investigating
17:48:59 <mtreinish> felipemonteiro: that sounds like more of a py3 incompatibility
17:49:02 <andreaf> mtreinish: at least as long as we finish the work on stable interfaces
17:49:19 <mtreinish> andreaf: I still think we're just inviting a bunch of random failures into the gate
17:49:33 <andreaf> mtreinish: it's not going to be a voting job
17:49:37 <mtreinish> but as long as it's not voting (I'm a hard -2 on making it voting) all we're doing is throwing away a node
17:49:57 <mtreinish> felipemonteiro: do you have a link to logs? I can take a look
17:50:30 <felipemonteiro> #link http://logs.openstack.org/17/465717/22/check/gate-tempest-dsvm-patrole-py35-member-ubuntu-xenial-nv/cb7cc13/console.html
17:50:33 <felipemonteiro> and thanks :)
17:50:51 <andreaf> mtreinish: I don't think it's thrown away - we're finding issues in the plugins which also helps us in turn to write better docs on how to write them
17:51:29 <andreaf> mtreinish: my guess is that after the job becomes stable and we have enough stable interfaces we can just remove it
17:51:46 <andreaf> ok there are 9min left
17:53:13 <andreaf> #topic Patrole
17:53:33 <andreaf> felipemonteiro, blancos: anything on patrole other than the issue above ^^^
17:53:47 <andreaf> ?
17:53:55 <felipemonteiro> yeah release notes for 0.1.0 release
17:54:03 <felipemonteiro> #link https://review.openstack.org/#/c/464072/
17:54:04 <patchbot> patch 464072 - patrole - Prepare release notes for release 0.1.0
17:54:59 <felipemonteiro> so we can then move forward with https://review.openstack.org/#/c/463578/
17:54:59 <patchbot> patch 463578 - releases - Release Patrole 0.1.0
17:55:44 <andreaf> felipemonteiro: heh sorry I didn't manage to look at that yet I will do later today
17:56:22 <felipemonteiro> that's it from me...then once we get that in we can start removing some deprecated tests that tempest has already
17:56:59 <andreaf> felipemonteiro: +A
17:57:33 <andreaf> #topic Critical reviews
17:57:44 <andreaf> any other review that needs attention?
17:57:53 <mtreinish> #link https://review.openstack.org/467366
17:57:54 <patchbot> patch 467366 - openstack-dev/devstack - Add a function to get an available random port
17:58:01 <mtreinish> #link https://review.openstack.org/459451
17:58:01 <patchbot> patch 459451 - openstack-dev/devstack - Use uwsgi for glance-api
17:58:51 <andreaf> #topic Open Discussion
17:59:41 <felipemonteiro> yes, i have a question regarding patrole gating
17:59:48 <andreaf> go ahead
18:00:02 <felipemonteiro> in other services, i'd like to know what your/qa cores thoughts are?
18:00:15 <felipemonteiro> time's up though
18:00:21 <andreaf> perhaps in qa room
18:00:22 <andreaf> it's time
18:00:23 <andreaf> #endmeeting-qa