22:00:25 <mtreinish> #startmeeting qa
22:00:26 <openstack> Meeting started Thu Apr 17 22:00:25 2014 UTC and is due to finish in 60 minutes.  The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot.
22:00:27 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
22:00:29 <openstack> The meeting name has been set to 'qa'
22:00:31 <sdague> o/
22:00:35 <masayukig> hi
22:00:36 <mtreinish> hi who do we have here today?
22:00:40 <oomichi> hi
22:00:47 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_April_17_2014_.282200_UTC.29
22:00:53 <mtreinish> ^^^ Today's agenda
22:01:41 <dkranz> o/
22:01:46 <sdague> all hail the juno PLT! :)
22:01:56 <mtreinish> PLT?
22:01:56 <dkranz> hail
22:02:00 <sdague> PTL
22:02:03 <mtreinish> heh
22:02:20 <mtreinish> thanks, I guess the torch was passed today wasn't it
22:02:27 <sdague> yep
22:02:33 <mtreinish> well let's get started
22:02:43 <mtreinish> #topic  Summit sessions (mtreinish)
22:02:53 <mtreinish> #link
22:02:53 <mtreinish> https://etherpad.openstack.org/p/Juno-QA-design-summit-topics
22:03:21 <mtreinish> so I just wanted to remind people that if they have session proposals the deadline for summit.openstack.org is the 20th
22:03:37 <mtreinish> so make sure you get your session proposals in before then
22:03:41 <sdague> I'll do write ups in the morning for my 2
22:03:48 <mtreinish> sdague: ok thanks
22:03:54 <sdague> my english skills are mostly faded for the day
22:04:04 <mtreinish> that's fine
22:04:11 <mtreinish> masayukig: also you had an entry in the etherpad
22:04:18 <masayukig> Yes, I've added a proposal of Tempest GUI.
22:04:18 <mtreinish> can you add it to the tool?
22:04:27 <mtreinish> masayukig: yeah the diagram looked cool
22:04:40 <masayukig> mtreinish: Thanks :0
22:05:24 <mtreinish> the only other session proposal in the etherpad that didn't have an entry in the tool was mlavalle yfried__
22:05:32 <mtreinish> I'll ping them about it tomorrow
22:05:49 <mtreinish> I plan to have a discussion about the session at the next meeting
22:05:57 <mtreinish> after the list of proposal is finalized
22:06:20 <mtreinish> does anyone have anything else about summit sessions?
22:06:29 <sdague> where is discussion on qa-specs going to slot in?
22:07:15 <mtreinish> I was planning to discuss it in the juno policy session I was going to have
22:07:32 <mtreinish> I don't think we'll have enough slots for a dedicated session
22:07:38 <sdague> sure
22:07:43 <sdague> that's not in the list yet right?
22:07:50 <mtreinish> no although it's in the tool
22:07:54 <mtreinish> I'll add it to the etherpad
22:07:55 <sdague> ok, cool
22:08:03 <sdague> it doesnt' really need to be in the etherpad
22:08:11 <mtreinish> that's what I thought too :)
22:08:29 <mtreinish> I was probably going to make that the last session for the qa track
22:08:34 <sdague> yep
22:08:47 <mtreinish> ok lets move on then
22:08:55 <oomichi> how many slots for qa in the summit?
22:09:04 <mtreinish> oomichi: it's looking like we'll have 10 slots for qa
22:09:28 <mtreinish> which is down 3 from HK
22:09:41 <mtreinish> #topic Oslo Liaison (mtreinish)
22:09:43 <oomichi> thanks, I got it.
22:09:56 <mtreinish> #link https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons
22:10:14 <mtreinish> so the oslo guys are looking for people from the other projects to be focal points as the convert things to libs
22:10:19 <mtreinish> and to keep up with oslo stuff
22:10:24 <mtreinish> for better cross team communication
22:10:39 <mtreinish> I figured we should have someone from the tempest-core group also on that list
22:10:47 <mtreinish> does anyone want to volunteer?
22:11:45 <mtreinish> well I'll take that as a TODO then
22:12:01 <mtreinish> #action mtreinish to find a volunteer to be oslo liason
22:12:34 <mtreinish> ok if no one is going to volunteer today
22:12:37 <mtreinish> let's move on
22:12:48 <sdague> get people to sign up at summit with promise of beer :)
22:13:01 <mtreinish> sdague: that's not a bad idea
22:13:12 <mtreinish> #topic Blueprints
22:13:41 <mtreinish> I guess let's start with the specs review first
22:14:02 <mtreinish> #link https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z
22:14:24 <mtreinish> it looks like we have 4 specs without a -1
22:14:29 <sdague> with the release this week, I was lazy on qa-specs
22:14:31 <mtreinish> and one of those has a +2 from dkranz
22:14:34 <dkranz> mtreinish: So I started the run-as-non-admin one even though it has not yet been approved
22:14:50 <mtreinish> dkranz: yeah we need to get more eyes on these
22:15:01 <mtreinish> I'll make another pass through them tonight or tomorrow
22:15:05 <sdague> I'll take a pass on monday, I expect tomorrow is mostly going to be recovery
22:15:25 <oomichi> I also will see it
22:15:31 <mtreinish> dkranz: I'd still like to see a +2 from someone else before I +A something
22:15:35 <sdague> mtreinish: you should probably approve my blueprint now that you land the spec as well
22:15:37 <sdague> :)
22:15:54 <mtreinish> sdague: oh yeah that's probably a good idea :)
22:16:02 <mtreinish> oomichi: ok thanks
22:16:27 <sdague> mtreinish: you going to send out the email announcing qa-specs and process to people?
22:16:34 <mtreinish> yeah I plan to do that tomorrow
22:16:38 <sdague> I think we've got enough examples now
22:16:40 <dkranz> mtreinish: I think we are dealing a little with the fact that approval used to be yeah, yeah, click the button
22:16:41 <sdague> awesome
22:16:54 <mtreinish> sdague: yeah 3 is probably enough
22:16:56 <dkranz> mtreinish: And now requires much more work from reviewers
22:17:04 <mtreinish> dkranz: yeah I think that's part of it
22:17:15 <mtreinish> people aren't used to looking at this on a regular basis yet
22:17:28 <dkranz> mtreinish: I made a bookmark :)
22:17:38 <sdague> right, so probably specs review should be part of every meeting. And everyone needs to try to keep on top of those.
22:18:00 <mtreinish> sdague: yeah I'm going to make it a separate agenda item moving forward
22:18:14 <rockyg> +1
22:18:39 <sdague> also should encourage anyone to help review these
22:18:42 <sdague> even if not core
22:18:46 <sdague> all comments are good
22:18:47 <mtreinish> yeah definitely
22:18:52 <mtreinish> I'll make that part of the email
22:19:02 <mtreinish> #action mtreinish to send an email about qa-specs to the ML
22:19:03 <sdague> great
22:19:32 <mtreinish> ok I think we can go onto bp status now, unless someone has something more to discuss about the specs review
22:19:52 <mtreinish> so on the agenda there was a bp linked
22:19:55 <mtreinish> #link https://blueprints.launchpad.net/tempest/+spec/nova-api-attribute-test
22:20:03 <mtreinish> oomichi: ^^^ I think that's yours right
22:20:08 <mtreinish> did you put it on the agenda?
22:20:10 <dkranz> mtreinish: I put that there
22:20:13 <mtreinish> dkranz: oh ok
22:20:23 <mtreinish> dkranz: go ahead then
22:20:29 <dkranz> mtreinish: There was an issue in reviews about the status of extra keys added to a return dict
22:20:43 <dkranz> mtreinish: and whether the schema should allow additoinalProperties or not
22:21:05 <dkranz> THere was a comment that it would be easier to extend if we allowed properties to be added
22:21:15 <dkranz> But the stability guidelines say you need an extension for that
22:21:32 <dkranz> oomichi: Do you have a comment about that?
22:21:43 <mtreinish> dkranz: to add something to the body you need an extension
22:21:48 <mtreinish> doesn't jsonschema allow for inheritance or something similar though
22:21:55 <mtreinish> jaypipes: was talking about that before on the ML
22:21:56 <oomichi> dkranz: do you mean schema should block non dict data?
22:22:29 <dkranz> oomichi: The issue I think was whether our validation should accept extra values or insist on a specific set
22:22:54 <dkranz> If we don't set additionalProperties to False then new keys can be added without failing the test.
22:23:21 <sdague> do we have a feeling on how the nova team feels about it?
22:23:24 <oomichi> dkranz: yes, right. and the default value of additionalProperties is True.
22:23:32 <dkranz> oomichi: exactly
22:23:49 <oomichi> so now validation accepts extra attributes.
22:23:52 <dkranz> sdague: I'm not sure but we should find out
22:24:01 <dkranz> sdague: Because some schemas have already gone in.
22:24:05 <sdague> sure
22:24:10 <dkranz> sdague: I only noticed this recently
22:24:23 <sdague> ok, maybe something to queue up for the nova meeting next week?
22:24:39 <dkranz> Do we have one of us that attends there as well?
22:24:42 <mtreinish> sdague: yeah that's probably a good idea
22:24:58 <sdague> well it's an hour before this one
22:25:05 <dkranz> :)
22:25:06 <sdague> we should get a volunteer for that
22:25:16 <mtreinish> dkranz: do you want to drive that on the nova meeting?
22:25:25 <dkranz> mtreinish: ok
22:25:28 <mtreinish> it's a bit early for oomichi right?
22:25:33 <mtreinish> dkranz: cool thanks
22:25:36 <dkranz> 6am
22:25:42 <sdague> the nova meeting does do a morning EST one too
22:25:45 <oomichi> yes,
22:25:50 <sdague> oomichi: are you on that one?
22:25:53 <oomichi> a little for me:-(
22:26:01 <mtreinish> #action dkranz to discuss jsonschema additionalProperties on nova meeting
22:26:07 <sdague> oomichi: what about when they flip to late meeting for you?
22:26:15 <dkranz> mtreinish: I have some sympathy for ssaying that extra values could be added.
22:26:31 <oomichi> but I will join it.
22:26:34 <dkranz> I'm not sure what the difference is to say you check for an extension and then look for the value
22:26:39 <oomichi> 6:00am meeting.
22:26:42 <dkranz> vs just seeing if the value is there
22:26:55 <mtreinish> sdague: it's 1400 or 2100 UTC
22:27:09 <sdague> dkranz: the theory is that it's discoverable in advance
22:27:39 <dkranz> sdague: sure, but is the real value of that compensating for extension hell over time?
22:27:43 <jaypipes> mtreinish: yes, jsonschema allows to specify another schema doc.
22:27:53 <dkranz> sdague: I'm just saying it is debatable
22:27:59 <sdague> dkranz: absolutely
22:28:11 <sdague> our current model just sucks around all this, no argument here :)
22:28:11 <dkranz> sdague: BUt we really have to nail it down soon.
22:28:38 <dkranz> sdague: ok, we'll see what the nova folks say
22:28:46 <sdague> jaypipes will fix it all, I have faith ;)
22:29:02 <mtreinish> heh
22:29:08 <mtreinish> dkranz: ok is there anything else on this bp?
22:29:13 <oomichi> I guess so:)
22:29:16 <dkranz> No
22:29:32 <mtreinish> ok then does anyone else have a bp to bring up or discuss?
22:29:59 <sdague> branchless-tempest
22:30:08 <sdague> this is very close
22:30:14 <mtreinish> sdague: ok cool
22:30:19 <mtreinish> what's left?
22:30:23 <sdague> https://review.openstack.org/#/c/88302/ - will make us gate tempest master on stable/icehouse
22:30:44 <sdague> all the stable/icehouse projects are gating on tempest master because of the way our branch fall through works
22:31:18 <mtreinish> ok, yeah that should do it
22:31:20 <sdague> and the service selection code for devstack-gate was redone so it would be sane to extend it to extensions
22:31:27 <sdague> though the extensions work isn't done yet
22:31:36 <sdague> https://review.openstack.org/#/q/status:open+project:openstack-infra/devstack-gate+branch:master+topic:feature_grid,n,z
22:31:42 <mtreinish> ok, well we shouldn't have any trouble until a new extension or feature is added
22:31:43 <sdague> that all went into the gate
22:31:48 <sdague> correct
22:31:50 <dkranz> sdague: What's the reason for including non-voting icehouse jobs?
22:32:02 <sdague> they aren't non voting
22:32:12 <dkranz> sdague: the cells job?
22:32:56 <sdague> good point
22:33:00 <sdague> I'll dump that later
22:33:14 <sdague> I thought that had been moved to experimental, apparently missed it
22:33:39 <mtreinish> dkranz: -1 it
22:33:48 <dkranz> mtreinish: k
22:34:03 <sdague> yeh, clarkb wants another change as well
22:34:05 <sdague> so -1 away
22:34:21 <mtreinish> sdague: shouldn't we gate on some of those icehouse jobs too?
22:34:37 <sdague> mtreinish: with clean check, I'm less concerned about that
22:34:46 <mtreinish> ok yeah that's fair
22:34:54 <mtreinish> it won't get to gate without it passing
22:34:59 <sdague> exactly
22:35:05 <sdague> I figure lets try with check only for now
22:35:19 <sdague> and if we break ourselves, move more into gate
22:35:20 <mtreinish> ok, I was thinking maybe just the mysql full for a sanity check
22:35:23 <mtreinish> but that's fine
22:35:51 <oomichi> we are going to apply tempest/master to stable/havana also?
22:35:56 <dkranz> mtreinish: I think relying on check is going to be more common going forward
22:36:00 <sdague> oomichi: if we can make it work
22:36:10 <mtreinish> oomichi: it doesn't work right now
22:36:18 <dkranz> mtreinish: because we can spread different configs across check
22:36:21 <sdague> it's still a little debatable if it's fully worth the time, given it eol in 5 months
22:36:25 <mtreinish> oomichi: dkranz has a spec proposal in progress for that effort
22:36:44 <mtreinish> I'm honestly fine with just taking this moving forward
22:36:49 <mtreinish> and leaving havana a branch
22:36:54 <oomichi> thanks, I will see it later.
22:37:06 <dkranz> sdague: Yes, one of things we have to discuss is that havana is not going eol soon for a lot of people, presumably including refstack
22:37:15 <dkranz> sdague: But not now
22:37:24 <sdague> dkranz: sure
22:37:43 <oomichi> now new API test, which is added since Icehouse, is in review. https://review.openstack.org/#/c/84977
22:38:15 <sdague> oomichi: the API is new in juno?
22:38:23 <oomichi> so I guess we need some option which controlls enable/disable the API test for havana.
22:38:26 <mtreinish> oomichi: hmm make sure you put an extension decorator around it then
22:38:52 <oomichi> sdague: no since Icehouse. server-group API of Nova.
22:39:00 <mtreinish> oomichi: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/test.py#n182
22:39:03 <dkranz> mtreinish: I am going to look at the havana thing when I am done with the non-admin which will hopefully be soon.
22:39:23 <sdague> oomichi: if it's in icehouse, we're good still
22:39:27 <mtreinish> oomichi: or is it larger scope than just an api extension?
22:39:38 <mtreinish> sdague: yeah it won't fail, but we still want to make sure we properly flag things
22:39:42 <mtreinish> dkranz: ok
22:39:45 <sdague> oomichi: https://github.com/openstack-infra/devstack-gate/blob/master/features.yaml - the theory is add the supported list of compute extensions into this mix as well
22:40:27 <sdague> then devstack-gate will set the right extension lists per feature and branch
22:40:40 <sdague> we'll talk about it at summit as well
22:40:55 <oomichi> OK, a lot of info to me. I will check them later carefully.
22:41:28 <mtreinish> sdague: ok is there anything else on branchless tempest?
22:41:33 <sdague> nope
22:41:43 <mtreinish> ok then does anyone else have a bp to discuss?
22:41:49 <mtreinish> otherwise we'll move onto the next topic
22:42:21 <mtreinish> #topic Neutron Testing
22:42:31 <mtreinish> so mlavalle said he couldn't make it today
22:42:37 <mtreinish> but he put an update on the agenda
22:43:00 <mtreinish> the take away is that there still 9 outstanding neutron api patches
22:43:15 <mtreinish> he pointed out 3 that have 1 +2:
22:43:22 <mtreinish> #link https://review.openstack.org/#/c/67547
22:43:28 <mtreinish> #link https://review.openstack.org/#/c/63723
22:43:34 <mtreinish> #link https://review.openstack.org/#/c/66541
22:44:00 <mtreinish> the only other thing there was he proposed another joint qa neutron session
22:44:06 <mtreinish> I guess to plan out things for juno
22:44:27 <sdague> sounds like a good idea
22:44:32 <mtreinish> yeah it does
22:44:44 <mtreinish> although I think we're in really good shape on that front compared to at the start of I
22:45:17 <mtreinish> I'll talk to mestery about that session and coordinate the scheduling for it
22:45:30 <mtreinish> ok does anyone else have anything to say on this topic?
22:45:58 <HenryG> Slightly related ...
22:46:04 <mtreinish> HenryG: sure
22:46:40 <HenryG> So marun_afk couldn't attend today but he is planning to submit a session too
22:46:52 <rockyg> also very slightly related (but more to the previous topic)  A joint Refstack/QA meeting possible?
22:47:00 <mtreinish> HenryG: yeah I talked to him about that today
22:47:05 <mtreinish> it's already on the etherpad
22:47:10 <dkranz> HenryG: I spoke to him. He put it on the etherpad
22:47:13 <mtreinish> he said he'd put it in the tool
22:47:24 <HenryG> ok, cool
22:47:37 <sdague> rockyg: are you thinking of that as a design summit session?
22:47:37 <mtreinish> rockyg: probably not unfortunately unless it's in other projects
22:47:49 <mtreinish> rockyg: the qa track is going to be full
22:48:11 <mtreinish> oh I just assumed you meant a summit session
22:48:14 <rockyg> No.  Just an irc or phone conference to get some more info on how our stuff dovetails with the new, improved tempest
22:48:17 <dkranz> Does refstack have any slots
22:48:32 <sdague> it looks like it has one - http://summit.openstack.org/cfp/details/197
22:48:33 <mtreinish> rockyg: yeah that's something we can do
22:48:38 <rockyg> We've got one to introduce everyone to RefStack.
22:48:43 <dkranz> rockyg: That would be good since I think refstack is part of what instigated this
22:48:45 <rockyg> Sorry, Refstack.
22:48:50 <mtreinish> rockyg: send an email to the ML about it
22:48:57 <rockyg> Will do.
22:49:07 <sdague> we should make sure to try to get that to not overlap the QA track
22:49:16 <sdague> so folks here can go to the refstack one
22:49:33 <mtreinish> sdague: yeah I'll talk to ttx about that (it looks like he's scheduling that track)
22:49:55 <mtreinish> ok well let's move on to the next topic
22:49:57 <sdague> yeh, I just left a comment
22:50:06 <sdague> we could also do refstack / tempest as a lunch thing one day
22:50:20 <rockyg> Thanks for the scheduling help
22:50:32 <rockyg> Oooh.  I kinda like that.
22:50:32 <mtreinish> sdague: yeah that would be good too
22:50:56 <mtreinish> #topic Heat Testing
22:51:06 <sdague> oh, update here
22:51:07 <rockyg> Refstack was supposed to be on Icehouse for the summit, but looks unlikely.
22:51:08 <mtreinish> sdague: so I know we have some updates here right
22:51:17 <sdague> the heat job is currently non voting
22:51:30 <sdague> http://jogo.github.io/gate/ - explains why
22:51:42 <sdague> the failure rate was going way out of control
22:51:56 <mtreinish> sdague: ooh nice spike
22:51:58 <sdague> it didn't hit 100%, that's an artifact of rolling averages
22:52:04 <sdague> but it was above 50%
22:52:12 <mtreinish> heh ok
22:52:22 <sdague> there are a few possible issues in it, heat team is looking at them
22:52:40 <sdague> but we need to not be bouncing things like the icehouse branch opennings
22:53:04 <mtreinish> yeah it's unfortunate though, we just turned it on not too long ago
22:53:04 <sdague> hopefully the failure rate can be addressed and it can come back to voting some time next week
22:53:19 <sdague> yeh, but honestly every new major effort has fits and starts like this
22:53:29 <mtreinish> yeah that's true
22:53:30 <sdague> we flipped the neutron job non voting a couple times last year
22:53:54 <mtreinish> was the switch to parallel execution part of the increase in instability?
22:53:55 <sdague> the heat jobs are starting to get enough content that they are flushing out some nice bugs as well
22:54:09 <sdague> that bug we managed to fix
22:54:15 <mtreinish> sdague: ok
22:54:18 <sdague> it was actually a keystoneclient bug
22:54:25 <sdague> on cert corruption
22:54:44 <sdague> this looks more like there are some async behavior where not expected
22:54:52 <sdague> and possibly the fedora guest is crashing
22:55:11 <mtreinish> is there a reason it's a fedora guest?
22:55:18 <mtreinish> that seems pretty heavyweight for the gate
22:55:30 <sdague> a number of the heat tests need real cloud-init or cfn-tools
22:55:36 <dkranz> mtreinish: the heat slow tests need a "real" image
22:55:37 <sdague> neither of which are provided by cirros
22:55:44 <sdague> dkranz: well some of them do
22:55:50 <mtreinish> hmm ok
22:55:58 <sdague> I think we could actually do some with just curl in cirrors
22:56:07 <dkranz> sdague: right. I suppose those that don't could be moved to the regular run
22:56:20 <sdague> dkranz: they will still be slow :)
22:56:35 <dkranz> Ah, yes.
22:56:40 <sdague> honestly, I think the heat dedicated job makes sense
22:56:45 <dkranz> yup
22:56:47 <sdague> it provides plenty of time budget
22:56:55 <sdague> just got to work through some issues here
22:57:05 <sdague> I expect to be working with them through juno to help
22:57:33 <mtreinish> sdague: ok is there anything else on heat?
22:57:37 <sdague> nope
22:57:37 <dkranz> I have to go but I can finish run-as-non-admin in short order I think once https://review.openstack.org/#/c/88347/ goes in
22:57:46 <mtreinish> dkranz: ok
22:57:58 <mtreinish> ok with <3 min let's do reviews
22:58:03 <mtreinish> #topic Critical Reviews
22:58:15 <mtreinish> does anyone have any reviews that they'd like to get eyes on?
22:59:02 <sdague> not from me, just nap time after the release
22:59:07 <mtreinish> heh ok
22:59:17 <mtreinish> well I guess if there aren't any reviews we can just end it here today
22:59:30 <mtreinish> thanks everyone
22:59:35 <rockyg> Thanks, guys!
22:59:44 <oomichi> thanks:)
22:59:47 <mtreinish> #endmeeting