17:00:46 <mtreinish> #startmeeting qa
17:00:47 <openstack> Meeting started Thu Feb 18 17:00:46 2016 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:00:48 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:00:50 <openstack> The meeting name has been set to 'qa'
17:00:59 <mtreinish> hi, who's here today?
17:01:01 <jordanP> hi
17:01:02 <slowrie> <
17:01:05 <jlanoux> hi
17:01:05 <ylobankov> hi
17:01:18 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_February_18th_2016_.281700_UTC.29
17:01:18 <silos> o/ *first timer*
17:01:24 <mtreinish> ^^^ today's agenda
17:01:31 <mtreinish> silos: welcome
17:02:00 <silos> mtreinish: thanks
17:02:50 <mtreinish> lets get started
17:02:57 <mtreinish> #topic QA Code Sprint
17:03:03 <mtreinish> #link https://wiki.openstack.org/wiki/QA/CodeSprintMitakaBoston
17:03:13 <mtreinish> I probably should have removed this from the agenda for this week
17:03:24 <mtreinish> since it's next week
17:03:47 <mtreinish> it's kinda too last min for anyone to decide to come :)
17:04:12 <mtreinish> but, anyway the code sprint is next week. I'm looking forward to it
17:04:21 <jordanP> I won't come unfortunately :(
17:04:34 <jordanP> but I'll be in Austin
17:04:37 <mtreinish> jordanP: :( that's too bad
17:05:01 <jordanP> yeah, I didn't even ask
17:05:24 <mtreinish> jordanP: cool, austin will be good too
17:05:30 <mtreinish> jordanP: the next time we do a code sprint I'll try to make it in a more EU friendly location
17:05:50 <jordanP> yeah, the location was not so bad for me, we have an office in Boston too
17:05:54 <jordanP> anyway
17:06:26 <mtreinish> one other thing is for those attending the sprint, if you have any topics you'd like to try and tackle during the sprint feel free to add them to the etherpad
17:06:49 <mtreinish> #link https://etherpad.openstack.org/p/mitaka-qa-code-sprint
17:07:05 <mtreinish> I initially populated it with some ideas off the top of my head
17:07:17 <jordanP> I have some ideas but I'll talk about it in the "free discusion" part of that meeting
17:07:24 <mtreinish> ok
17:07:44 <mtreinish> does anyone have anything else to discuss on the code sprint?
17:08:48 <mtreinish> #topic Specs Reviews
17:08:57 <mtreinish> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
17:09:07 <mtreinish> does anyone have any open spec reviews they'd like to discuss?
17:10:08 <mtreinish> I want to point people to:
17:10:10 <mtreinish> #link https://review.openstack.org/275966
17:10:38 <mtreinish> which is my spec about moving tempest-lib code back into tempest
17:10:55 <mtreinish> it needs another respin, but I'd appreciate any thoughts on it
17:11:32 <jordanP> I agree with that. The tempest-lib effort was not in vain. We did some code code cleanup, added some unit tests, etc..
17:12:09 <mtreinish> jordanP: right, and that will still continue. The plan is to keep the lib interface alive, just under the tempest repo
17:12:34 <mtreinish> "migration" work won't change, except for instead of copying the code to a separate repo it'll just mv to tempest/lib
17:13:39 <jordanP> yep. I especially appreciate the "make methods use **kwarg" and "make method name consistent" and the client slit also. Big thanks to all of the people who worked on that
17:13:49 <jordanP> we should definitively continue on that track
17:14:34 <mtreinish> ++
17:14:57 <mtreinish> ok, does anyone have anything else to discuss on specs?
17:15:11 <ccneill> https://review.openstack.org/#/c/274205/
17:15:12 <ccneill> :D
17:15:16 * ccneill sneaks in
17:15:37 <mtreinish> #link https://review.openstack.org/#/c/274205/
17:15:58 <ccneill> more than a review, I just kind of want to ask.. is there any hope of this merging with the tempest migration on the table?
17:16:06 <mtreinish> ccneill: I think sdague's -1 there raises a good point
17:16:19 <mtreinish> you could spin that out as a seperate tool pretty easily
17:16:46 <ccneill> mtreinish: so I do agree that tempest-lib shouldn't do *all the things*, but I would consider the capabilities of what I have to be pretty in line with other data generators
17:16:59 <jordanP> yeah sdague has a point. The current code here https://review.openstack.org/#/c/237263/3/tempest_lib/common/utils/security_utils.py doesn"t rely on tempest-lib at all
17:17:33 <ccneill> jordanP: true, it is definitely an odd duck of sorts, and doesn't HAVE to live in tempest-lib
17:17:36 <jordanP> ccneill, do you know bandit ?
17:17:45 <ccneill> jordanP: yep, I'm iin the OSSP meeting right now too
17:17:48 <jordanP> ok
17:18:21 <jordanP> this looks more like something that could live in the bandit repo. Or a repo by itself
17:18:34 <ccneill> jordanP: if you haven't heard my pitch before, basically I have written tests for barbican and designate, they suggested I try to put it in tempest-lib because it made sense to them and they didn't both want to have just one file floating in their project
17:18:48 <ccneill> and I can't promise I really have time to maintain this as a separate tool
17:19:03 <mtreinish> right, I think if the spec was to add another seperate tool (which could consume the rest_client from tempest-lib if it wanted) that would go over a bit better
17:19:21 <ccneill> mtreinish: so right now it should work with either tempest-lib or requests-based clients
17:20:15 <mtreinish> ccneill: right, the more I hear about this doing it as a seperate thing makes more sense (or as jordanP suggested if bandit likes it that works too)
17:20:39 <mtreinish> we can add another qa project for this pretty easily if necessary
17:20:53 <ccneill> mtreinish: gotcha. I'll stop bugging you guys about it for now and try to figure out a better path forward
17:21:41 <ccneill> because I've discussed combining this work with Syntribos before (OSSP project), but it just seemed more accessible to include it in something everyone already uses, rather than pitching a new tool
17:21:45 <ccneill> but that's my problem, not yours :)
17:22:06 <mtreinish> ccneill: heh, well don't go off into the dark, feel free to keep talking about it :)
17:22:36 <jordanP> ccneill, creating a dedicated repo for it is kinda of easy. And you will be PTL :)
17:22:55 <mtreinish> jordanP: well not if it's a qa project :p
17:23:06 <jordanP> erf, yeah, !
17:23:17 <ccneill> jordanP, mtreinish: it seems like this might be a better fit for the OSSP anyway
17:23:23 <ccneill> I understand the concerns about scope creep
17:24:04 <ccneill> for the record, if it were ONLY a data generator, would it be more or less appealing from a tempest-lib perspective?
17:24:13 <ccneill> just out of curiosity
17:24:46 <mtreinish> ccneill: I think it would be closer aligned, but then I'm not sure what the benefit would be :)
17:25:10 <mtreinish> but, I think we need to move on to other topics now
17:25:13 <ccneill> np
17:25:29 <mtreinish> #topic Priority Items
17:25:37 <mtreinish> #link https://etherpad.openstack.org/p/mitaka-qa-priorities
17:26:12 <mtreinish> so m-3 is approaching kinda quickly
17:26:16 <mtreinish> it's a couple weeks away
17:26:50 <mtreinish> we've still got a fair number of open priority items
17:27:12 <jordanP> 6 and 7 are less relevant I think
17:27:26 <mtreinish> that's probably fair
17:27:57 <jordanP> jlanoux, what about 10 'Finalize ssh-auth bp' ?
17:28:09 <mtreinish> well, they'll still happen but the "migration" will be a git mv :)
17:28:41 <jlanoux> jordanP: yep - I'd like this one to get in: https://review.openstack.org/#/c/259515/ and then we can close it.
17:29:09 <mtreinish> #link https://review.openstack.org/#/c/259515/
17:29:55 <jordanP> jlanoux, and what then ? I the -ssh job is non voting and failing a lot, do we have a plan ?
17:31:18 <mtreinish> yeah, it'd be good if we could be in a place where we were comfortable enough with turning that flag on by default
17:31:18 <jlanoux> jordanP: I'll keep working on this. I have some ideas. But they are beyond the scope of the original blueprint. I'll put a new spec together and see what you guys think.
17:31:43 <mtreinish> it's more than just pass rate too, but also the debug path
17:31:53 <jlanoux> yes
17:32:11 <jordanP> we don"t need a spec for to make something work. I am sorry to rant here but we have too many things in Tempest/QA that just doesn't work
17:32:13 <mtreinish> if we can't figure out why the ssh failed, that doesn't help much
17:32:56 <jordanP> agreed but I don"t see how that patch helps
17:33:20 <jordanP> I am fine with having it, but I don't know it will make our life better
17:33:22 <jlanoux> We talked about that and ssh validation in Tempest is messy - I'd like to refactor that and add some logging. This will be a step
17:33:50 <jlanoux> then I think we have to deal with race conditions from other services - not much we can do there apart giving them some data
17:34:17 <jordanP> but a refactoring is about making the code nicer. Currently the code doesn"t work. We need to fix bugs
17:34:57 <jordanP> but if we had that many race conditions, how come the tempest scenarios are quite reliable ?
17:35:30 <afazekas> FYI:  ssh login can fail if the root disk is not available as well.
17:35:50 <jlanoux> jordanP: we can't fix all at once - I'm willing to continue looking on ssh
17:35:51 <mtreinish> afazekas: haha, yeah there are a lot of possible causes
17:35:54 <jlanoux> it's not that bad
17:36:16 <jordanP> ok, jlanoux I'll love to work with you to turn that ssh_validation flag to True
17:36:17 <jlanoux> afazekas: yeah and hardware can be against us as well
17:36:25 <jlanoux> jordanP: +1
17:36:40 <jordanP> ok, I'll review and +2 that patch
17:36:46 <jlanoux> thanks
17:37:03 <jordanP> (I was "angry" with PS24 btw)
17:37:30 <jordanP> PS25 looks better :)
17:37:47 <jlanoux> ;)
17:38:43 <mtreinish> ok, is there anything else to discuss on priority items?
17:39:57 <mtreinish> #topic Tempest
17:40:11 <mtreinish> does anyone have anything to disucss on tempest today?
17:40:31 <mtreinish> although I have a feeling the ssh topic kinda covers this for today's meeting :)
17:40:45 <jordanP> nope, also I have:
17:40:54 <jordanP> https://review.openstack.org/#/c/249100/
17:41:02 <jordanP> I got -1 by sdague and it seems unfair
17:41:05 <mtreinish> #link https://review.openstack.org/#/c/249100/
17:41:11 <jordanP> if we follow that line we should -1 a lot of patch too
17:41:30 <sdague> it's a new 2.5 minute test
17:41:34 <jordanP> I got two +1 from Cinder core reviewers.
17:41:40 <sdague> in the base run for all services
17:42:02 <jordanP> it's not the longest test and it bring a lot of value
17:42:09 <sdague> we can't just keep throwing in new tests > 60s on the integrated gate, that is the path to madness
17:42:22 <jordanP> sdague, ok. Then I'll -1 similar patch
17:42:53 <jordanP> I am fine with that and we can work around it, by having cinder "in tree functionnal tests"
17:43:06 <jordanP> but then we must be consistent
17:43:46 <sdague> I'm fine with that. This seemed a particularly large growth path
17:44:00 <sdague> especially when the volumes tests remain the most problematic from reliability
17:44:18 <jordanP> noope. Neutron is worst :p
17:44:33 <sdague> well, at least they are run on less jobs :)
17:44:51 <jordanP> yes
17:45:01 <jordanP> so other topic is: the multinode jobs
17:45:12 <jordanP> they fail a lot like 50%
17:45:13 <sdague> but in all seriousness, we need the volumes behavior nailed down in cinder specific tests
17:45:31 <sdague> that run just on cinder changes to figure out why things are problematic
17:45:33 <jordanP> especially tempest.api.compute.admin.test_live_migration.LiveBlockMigrationTestJSON.test_volume_backed_live_migration
17:45:44 <jordanP> my libvirt skill are a bit short there
17:46:18 <afazekas> jordanP: Neutron showed lot of improvement in the past year
17:46:24 <jordanP> I will work on it tomorrow too. But my general feeling at the moment is, there's no need to pile up tests on top of something that doesn't work
17:46:43 <mtreinish> jordanP: multinode or cinder?
17:46:47 <jordanP> multinode
17:46:58 <jordanP> cinder works in my opinion :)
17:47:40 <mtreinish> heh
17:48:03 * smcginnis feels relieved
17:48:06 <jordanP> lol
17:48:09 <sdague> well, on the live migration front, we're building a dedicated job that *only* does live migration testing to try to sort out these things. That's the dedicated hammer approach.
17:48:38 <jordanP> sdague, that won't help imo. We need human to look at it
17:48:54 <sdague> jordanP: right, and we will have humans looking at it
17:49:00 <jordanP> that's great
17:49:12 <jordanP> but the jobs have been there for months, why now ?
17:49:36 <sdague> jordanP: ok, what is your specific proposal, I guess I'm lost on that one
17:50:00 <jordanP> first raise awareness
17:50:12 <jordanP> we need to look at the problem before adding more multinode tests
17:50:23 <jordanP> I am thinkg about https://review.openstack.org/#/c/259746/
17:51:00 <mtreinish> jordanP: well I don't think there is much value in that test in general
17:51:05 <mtreinish> I'll leave a comment on that one
17:51:34 <jordanP> ok, maybe we can move on. I am done :)
17:52:03 <mtreinish> #topic DevStack + Grenade
17:52:33 <mtreinish> silos: you put a something on the agenda for this topic?
17:52:38 <silos> mtreinish: yes
17:52:51 <silos> I mainly contribute to barbican and we are interested in writing a grenade job for it.
17:53:21 <mtreinish> silos: are you just looking for help on how to do that?
17:53:24 <silos> I was wondering if there is a formal process like creating a spec? If the PTL in barbican needs to be aware?
17:53:34 * redrobot pokes head in
17:54:14 <mtreinish> silos: it's strictly a project decision on adding the job or not. Everything will be done self service as a plugin in the barbican repo
17:54:34 <mtreinish> http://docs.openstack.org/developer/grenade/plugins.html
17:54:47 <mtreinish> silos: we can pick this up in -qa after the meeting
17:54:54 <mtreinish> since we're a bit pressed for time
17:54:56 <silos> mtreinish: Ok. sounds good.
17:55:13 <mtreinish> does anyone have anything else on devstack or grenade?
17:55:41 <jordanP> SSL support in devstack is a pain :)
17:55:52 <jordanP> many patched lying around trying to fix it
17:55:56 <jordanP> *patches
17:56:14 <jordanP> and if we want a suggestion, I propose to remove all of it :)
17:56:41 <mtreinish> jordanP: heh, isn't it more a general openstack issue. I think those patches were blocked because the ssl configuration for services is kinda weird
17:56:41 <jordanP> (sorry, that's a debate for another time)
17:56:53 <mtreinish> but, it's been a while so my memory is a little hazy on the details
17:57:14 <mtreinish> but, yeah we can pick that up at a different time
17:57:19 <jordanP> yep
17:57:46 <mtreinish> #topic Critical Reviews
17:58:02 <mtreinish> does anyone have any reviews they'd like to get extra eyes on?
17:59:24 <sdague> jordanP: I thought there was some middleware now that did the headers right?
17:59:29 <sdague> or did that never happen
17:59:59 <jordanP> I remember we talked about it, but I forgot what we said about it !
18:00:23 <mtreinish> well, we're at time. We can pick things up in -qa
18:00:26 <mtreinish> thanks everyone
18:00:30 <jordanP> thanks
18:00:31 <mtreinish> #endmeeting