17:01:49 <mtreinish> #startmeeting qa
17:01:50 <openstack> Meeting started Thu May 22 17:01:49 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
17:01:51 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
17:01:53 <openstack> The meeting name has been set to 'qa'
17:02:04 <mtreinish> hi, who's here today?
17:02:09 <mkoderer> Hi
17:02:12 <giulivo> hi
17:02:14 <andreaf> hi
17:02:18 <ylobankov> hi
17:02:21 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_May_22_2014_.281700_UTC.29
17:02:25 <mtreinish> ^^^ Today's agenda
17:02:47 <mtreinish> it's a pretty light one this week
17:03:57 <mtreinish> ok, lets get started
17:04:08 <mtreinish> #topic Blueprint Purge (mtreinish)
17:04:37 <mtreinish> So I put this on the agenda just to let everyone know that after next weeks meeting I'm going to purge the bps without a spec at least proposed
17:04:45 <mtreinish> I'm going to send a note to the list too
17:05:04 <mkoderer> all right.. I need to write some spec's then
17:05:05 <mtreinish> but we've got a lot of bps listed on lp but very few specs proposed right now
17:05:07 <mkoderer> ;)
17:05:29 <mtreinish> mkoderer: do you have a bp without a spec?
17:05:49 <mkoderer> mtreinish: dkranz has one about porting negative tests
17:05:51 <rockyg> specs are not code, so they're hard for developers ;-)
17:06:26 <mkoderer> I will add some specs next week
17:06:26 <mtreinish> rockyg: it's more about cleaning up dead bps from the list
17:06:46 <andreaf> mtreinish: for the purge do you care about the status of the bp in lp as well, or the spec only?
17:06:47 <mtreinish> it's just a filter, there is nothing stopping someone from coming back and restoring it with a spec later
17:07:09 <rockyg> Agreed.  I've got a BP that I'll abandon, even thought it's really needed.  The docs.  I can write the spec if you ever think someone will pick it up.
17:07:19 <mtreinish> andreaf: it's everything that currently with an undefined priority
17:07:35 <mtreinish> without a spec in review
17:07:59 <mtreinish> andreaf: I think all of yours have specs already :)
17:08:48 <mtreinish> #action mtreinish to send a email to list about bp cleanup
17:09:05 <mtreinish> ok does anyone have anything else they'd like to talk about on this topic?
17:09:54 <mtreinish> ok, then let's move on to the next topic
17:10:03 <mtreinish> #topic Specs Review
17:10:20 <mtreinish> does anyone have an open specs reviews they'd like to bring up
17:10:24 <mtreinish> or to discuss in more depth
17:10:53 <andreaf> mtreinish: #link https://review.openstack.org/#/c/94741/
17:11:23 <andreaf> mtreinish: not to discuss I think, it just needs review
17:11:52 <andreaf> mtreinish: it's the ssh-auth-strategy afazekas has been working on
17:11:53 <mkoderer> andreaf: I promise to do it tomorrow :)
17:12:12 <mtreinish> andreaf: ok, cool thanks. I'll try to take a look at it soon.
17:12:13 <andreaf> mkoderer: thanks
17:12:33 <andreaf> mtreinish: also #link https://review.openstack.org/#/c/92804/
17:12:51 <andreaf> client manager refactor
17:12:53 <mtreinish> but I've been reluctant to review ones without a +2 already, because I have to look at every spec to +A it...
17:13:20 <andreaf> I'd love to get any kind of feedback on this one to see if there is interest in it
17:13:34 <andreaf> mtreinish: fair enough
17:14:05 <mtreinish> andreaf: that one looks like it will have a lot of review overhead (when/if you implement it)
17:14:12 <mtreinish> another giant patch series :)
17:14:30 <mtreinish> the question I think is are the benefits worth the overhead
17:15:07 <andreaf> the overhead of coding time you mean?
17:15:13 <mtreinish> and review time
17:16:04 <andreaf> I think this may help a lot with micro version in nova
17:16:22 <mtreinish> the coding should be pretty simple once you do the initial refactor. It's more taking the time for everyone to look at it.
17:16:43 <mtreinish> andreaf: you should explicitly outline the benefits of having the refactor done then
17:16:49 <andreaf> the current approach is not very scalable you get one new attribute for every new combination of api / version
17:17:17 <andreaf> quote: "Another issue with the current structure is that new API versions lead to
17:17:18 <andreaf> proliferation of client attributes in the client manager classes."
17:17:24 <mtreinish> andreaf: fair enough, I think on something like this the why is more important than the what
17:18:00 <andreaf> mtreinish: ok thanks I'll try to get more detail on the why
17:18:25 <mtreinish> ok cool
17:18:34 <mtreinish> does anyone else have any specs reviews to bring up?
17:19:30 <mtreinish> ok then let's move on
17:19:34 <mtreinish> #topic Blueprints
17:19:53 <mtreinish> does anyone have any in progress bps that they'd like to discuss
17:20:27 <andreaf> mtreinish: on the ssh-auth-strategy, NithyaG has been working on it (she could not attend today)
17:20:45 <sdague> we did find the first actual instance of needing the next level of feature grid earlier this week (re: branchless tempest)
17:21:08 <mtreinish> sdague: cool
17:21:26 <sdague> turns out grenade config was wrong and let a break slip through. I'm going to circle around on that once I get grenade enforcing resources spanning the gap from old to new
17:21:33 <sdague> which apparently we lost at some point
17:21:47 <mtreinish> oh that was the keystone cert test thing right?
17:21:50 <sdague> yep
17:21:53 <sdague> we just reverted it
17:22:11 <sdague> and the grenade config fix looks right now, as the revert^2 now fails
17:22:20 <mtreinish> I thought that there was something else that someone brought up at summit too
17:22:39 <sdague> there is also some ipv6 neutron bits that need it
17:22:42 <mtreinish> this will be a good first test of the process
17:22:48 <sdague> but I think that's slightly more complex
17:23:06 <sdague> I'm going to get together with sc68cal next friday and figure out some of that
17:23:40 <mtreinish> ok, yeah I imagine ipv6 makes things more complex :)
17:23:49 <sdague> so I'll tenatively say 3 - 4 weeks hopefully to get that all landed
17:24:25 * sdague done on branchless tempest updates
17:24:36 <andreaf> mtreinish: re ssh-auth-strategy, the server rebuild test is failing when ssh check is enabled, but only when it's run in combination with other tests
17:24:39 <mtreinish> sdague: ok cool
17:24:55 <andreaf> mtreinish: NithyaG is working on it
17:25:02 <mtreinish> andreaf: is there something outside a tenant scope being used there?
17:25:41 <andreaf> there is a single server reused by all tests in the class
17:26:04 <mtreinish> yeah that always causes problems...
17:26:28 <afazekas> mtreinish: but we want to catch those, and test
17:26:32 <andreaf> e.g. reboot test waits for VM to go to ACTIVE, and then rebuild kicks in
17:27:12 <sdague> andreaf: is it only those classes that reuse servers a ton where this breaks?
17:27:30 <sdague> because if so, we could annotate them to not do it and get the other 150 servers
17:27:39 <andreaf> sdague: atm it's only that class with ssh enabled
17:27:46 <sdague> ok
17:28:15 <mtreinish> afazekas: honestly I feel like that workflow is more for scenario tests
17:28:19 <andreaf> sdague: creating a new server may be an option but I'd like to understand my it breajs
17:28:26 <sdague> so, honestly, I'd like to include sshing into every server as part of the validation of compute creation
17:28:38 <mtreinish> these reuse classes always have problems because of leftover state in the api tests
17:28:56 <sdague> mtreinish: agreed
17:29:04 <andreaf> mtreinish, afazekas, sdague: some API tests cannot test much without ssh
17:29:08 <sdague> how much time hit will we take on getting read of them
17:29:14 * mtreinish watches run time skyrocket with ssh everywhere
17:29:18 <andreaf> for instance attaching a config drive
17:29:28 <sdague> andreaf: agreed
17:29:39 <afazekas> sdague: It would increase the test time, unless we increase the number of subunit process
17:29:52 <sdague> afazekas: by how much?
17:30:15 <andreaf> I think if we start by getting the existing ssh working it's a first good step
17:30:23 <afazekas> sdague: good question, butwe have two worker at the moment it is too few
17:30:46 <sdague> andreaf: I think we're on the same page. I honestly want create_server(wait='ACTIVE') to actually not only wait for active, but also make sure the guest is sshable
17:30:47 <andreaf> than we can see what's the impact of getting more ssh tests in
17:31:20 <sdague> andreaf: sure, I just wonder if we'll find it easier to debug when things go wrong if we make it a base validation case for every server create
17:31:48 <andreaf> sdague: sure that would help
17:31:48 <sdague> afazekas: we're mostly 4 workers now, all our nodes are going to 8 core
17:32:22 <afazekas> sdague: :)
17:32:25 <andreaf> sdague: also getting things like console log printed in case of error and other debugging may help
17:32:46 <mtreinish> andreaf: we can do that now though through the api we don't need ssh to do that
17:32:52 <sdague> andreaf: yep. I think this would help even in getting ssh to work in the tricky cases though. So could be an immediate patch
17:32:59 <mtreinish> the heat tests do that already
17:33:10 <andreaf> sdague: on the same like of wait for active you could say for attaching a volume that you need to ssh and fdisk to confirm
17:33:15 <afazekas> sdague, andreaf: What is your onion about making the current debug strategy more intelligent ?
17:34:06 <sdague> afazekas: I'm for it, especially a plugable way to go collect that, but we need to talk it through. Can you propose a qa-spec on it?
17:34:13 <afazekas> Now it just prints everything, for a human takes a long time  to understand  the log
17:34:29 <afazekas> sdague: yes
17:34:43 <andreaf> afazekas, sdague: are these the debug options from the conf?
17:34:56 <afazekas> sdague: Do we want to support multnode , other things than neutron ml2 ovs ?
17:35:14 <afazekas> andreaf: yes
17:35:16 <sdague> andreaf: yeh.
17:35:26 <sdague> I think we've gotten pretty far from the blueprints at hand though
17:35:49 <sdague> afazekas: can you please write down some of the ideas about enhancing debug in a spec?
17:36:03 <afazekas> sdague: ok
17:36:10 <andreaf> I would keep the scope of the ssh-auth-strategy bp fixed now, and handle other improvements in separate specs
17:36:24 <sdague> andreaf: ok, that's fair
17:36:36 <sdague> andreaf: I'll try to review it today
17:36:58 <andreaf> sdague: thanks
17:37:17 <andreaf> we have a lot of ideas again for the todo.rst
17:37:23 <andreaf> ^_^
17:37:36 <mtreinish> oh, thanks for the reminder Ive got to add that soon...
17:38:02 <mtreinish> ok does anyone have any other bps to discuss
17:38:44 <sdague> not i
17:38:52 <mtreinish> ok then let's move on
17:39:03 <mtreinish> #topic Neutron testing
17:39:09 <mtreinish> mlavalle: are you around?
17:39:18 <mlavalle> mtreinish: Yes
17:39:34 <mtreinish> any update on neutron testing?
17:39:39 <mlavalle> mtreinish: since our last meting, we have merged another 5 api tests
17:39:56 <mlavalle> so of the original 28 we were tracking, we have merged 25
17:40:01 <mtreinish> cool
17:40:27 <mlavalle> I have added a couple of tests for ipv6, so we have another 5 to go
17:40:38 <mlavalle> but the progress keeps steady
17:40:53 <mlavalle> I have a couple of questions
17:41:00 <afazekas> I have question  related to https://review.openstack.org/#/c/83627/ 'I still have a question, does your work cooperate with Mh Raies as you have the same changes with him https://review.openstack.org/#/c/47816/25/tempest/api/network/base.py and https://review.openstack.org/#/c/47816/25/tempest/services/network/network_client_base.py ?'
17:41:50 <mlavalle> for the scenario test tutorial / dcoumentation I am assuming you want me to add to http://docs.openstack.org/developer/tempest/field_guide/scenario.html, corrrect?
17:42:11 <mtreinish> mlavalle: maybe, it might warrant another section
17:42:17 <mtreinish> but we can always move things around later
17:42:21 <mlavalle> afazekas: yeah, i need to follow up with him and coordinate the patchsets
17:42:48 <mlavalle> mtreinish: how do I get to edit that? does anyone has to give access?
17:43:11 <mtreinish> mlavalle: it gets auto published by the readme.rst in tempest/scenario
17:43:19 <mlavalle> ah, ok
17:43:31 <mlavalle> will work on that
17:43:59 <mlavalle> mtreainish: second question: where do you want me to document the new Neutron scenario tests blueprints?
17:44:18 <mtreinish> mlavalle: for tracking progress
17:44:26 <mtreinish> like what you did with an etherpad for the api tests
17:44:41 <mtreinish> or just in general?
17:44:54 <mlavalle> previous cycle we were using https://blueprints.launchpad.net/tempest/+spec/neutron-advanced-scenarios
17:45:10 <mtreinish> oh, you just mean having a bp
17:45:10 <mlavalle> I can use an etherpad, though. I know how to do it :-)
17:45:33 <mtreinish> yeah I'd throw up a quick spec to qa-specs outlining the total scope of goals for the work
17:45:39 <mtreinish> and link to an etherpad or something else
17:45:48 <mtreinish> to track the progress more granuarly
17:45:55 <mlavalle> cool, i'll start it this coming long weekend
17:46:15 <mlavalle> that's all I have
17:46:19 <mtreinish> the spec doesn't have to be too involved this one is pretty self explanatory
17:46:27 <mlavalle> ok
17:46:30 <mtreinish> ok does anyone else have anything to discuss on neutron testing?
17:47:10 <mlavalle> can I get a core review for this https://review.openstack.org/#/c/92436
17:47:13 <mlavalle> ?
17:47:24 <mtreinish> mlavalle: sure
17:47:32 <mtreinish> ok, then let's move on to the next topic
17:48:02 <mtreinish> #topic Heat testing
17:48:33 <sdague> I'm not sure we have a rep for this today
17:48:39 <mtreinish> sdague: you were the one who made this a sticky agenda item so I'm looking to you...
17:48:49 <sdague> I've not been poking it, as I need to dive down the grenade hole
17:48:59 <sdague> so I'd say we pull it off standing agenda for now
17:49:08 <andreaf> just one not
17:49:11 <andreaf> note
17:49:39 <andreaf> the non-voting slow heat tests were failing because of a change in python-novaclient, which is now fixed in heat
17:49:57 <sdague> andreaf: ok, that's not the only fail reason
17:50:29 <sdague> we were having boot stability issues with the fedora image as well, which I don't think are resolved yet
17:50:35 <andreaf> sdague: probably at least that one cause consistent failure :)
17:50:49 <sdague> there is a patch series up that brings in disk image builder
17:50:54 <sdague> to maybe get around that
17:51:09 <sdague> but it's going to be a bit, because we need to start also running tempest on dib then
17:52:11 <mtreinish> ok well let's move on to critical reviews because we're <10 min left
17:52:23 <mtreinish> #topic Critical Reviews
17:52:24 <sdague> one last thing, tempest release?
17:52:32 <mtreinish> sdague: oh
17:52:36 <mtreinish> yeah I'll do that today
17:52:36 <sdague> I think we committed to making one at summit
17:52:41 <sdague> ok, cool.
17:52:48 <sdague> naming conventions?
17:52:49 <mtreinish> I'll pick a sha1 before andreaf's refactor I think
17:52:56 <mtreinish> 2014.1 ?
17:53:18 <sdague> so we'll have a 2014.2 which in no way relates to OS 2014.2?
17:53:44 <sdague> that's part of my concern on that naming convention, because we said 4 times a year
17:54:25 <mtreinish> how do the clients do it?
17:54:38 <mtreinish> we could just do the same
17:54:46 <mtreinish> oh maybe 1.0
17:54:49 <mtreinish> for 2014
17:55:00 <mtreinish> and increment the minor for each release
17:55:40 <mtreinish> nah that won't work the major version should mean something other than chronology
17:55:51 <sdague> we can figure that offline
17:55:54 <mtreinish> yeah
17:56:04 <mtreinish> ok does anyone have any reviews to bring up?
17:56:54 <afazekas> https://review.openstack.org/#/c/94203/
17:57:04 <mtreinish> #link  https://review.openstack.org/#/c/94203/
17:57:30 <mtreinish> afazekas: ok I'll take a look soon
17:57:35 <mtreinish> are there any other reviews?
17:58:05 <ylobankov> https://review.openstack.org/#/c/93301/
17:58:28 <mtreinish> #link  https://review.openstack.org/#/c/93301/
17:58:48 <mtreinish> ok well with 2 mins let's go to the last topic :)
17:58:57 <mtreinish> #topic Summit follow-up (andreaf)
17:59:07 <mtreinish> andreaf: we can start off next weeks with this if you'd prefer
17:59:26 <andreaf> I just wanted to ask if have a place to track any action / follow-up from the summit
17:59:52 <andreaf> other than the etherpads which is not very consolidated
18:00:17 <mtreinish> no the etherpads were about it
18:00:22 <andreaf> things like releases, the todo.rst, the specs we need to create (tempest as a service) and so
18:00:24 <sdague> andreaf: honestly, I'd suggest building a summary etherpad that we can work through
18:00:54 <sdague> I was going to do that personally for all the things I committed to (have been doing it locally, but only about 25% of the way through collecting it)
18:01:07 <hyakuhei> I'll give you guys a second or two to wrap up before we start the OSSG meeting :)
18:01:16 <mtreinish> ok well we're at time
18:01:21 <mtreinish> andreaf: we can follow up on -qa
18:01:25 <andreaf> ok
18:01:26 <mtreinish> #endmeeting