22:01:30 <mtreinish> #startmeeting qa 22:01:30 <openstack> Meeting started Thu Aug 21 22:01:30 2014 UTC and is due to finish in 60 minutes. The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:01:31 <openstack> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 22:01:33 <openstack> The meeting name has been set to 'qa' 22:01:39 <mtreinish> hi who's here today? 22:01:48 <dkranz> hi 22:01:49 <masayukig> hi 22:02:03 <mtreinish> heh, it'll be a nice quick meeting today :) 22:02:05 <mtreinish> #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_August_21_2014_.282200_UTC.29 22:02:07 <asselin> hi 22:02:09 <mtreinish> ^^^ Today's agenda 22:02:25 <andreaf> o/ 22:02:41 <mtreinish> ok let's get started, maybe more people will trickle in later 22:02:54 <mtreinish> #topic Devstack now part of the QA program 22:03:06 <mtreinish> I just wanted to make an announcement on this 22:03:14 <mtreinish> dtroyer's governance change landed earlier this week 22:03:26 <mtreinish> so the devstack repo is now officially part of the qa program 22:03:38 <mtreinish> this was announced on the ML a couple of weeks ago 22:03:40 <dkranz> mtreinish: does that really change anything? 22:03:43 <mtreinish> no 22:03:48 <andreaf> yay welcome dtroyer and devstack team to the QA program 22:03:51 <mtreinish> but it felt news worthy :) 22:03:57 <dkranz> mtreinish: sure :) 22:04:15 <mtreinish> ok anything else on this? 22:04:22 <masayukig> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/041731.html 22:04:32 <masayukig> ^^^? 22:04:43 <mtreinish> masayukig: yep that was the ML post 22:05:03 <mtreinish> ok let's move on 22:05:03 <masayukig> ok, thanks :) 22:05:20 <mtreinish> #topic Starting a QA liaison system (like oslo) (mtreinish) 22:05:39 <mtreinish> so this was another topic that came up briefly on the ML 22:05:51 <mtreinish> as a branch on the giant future of the integrated release thread 22:06:14 <mtreinish> #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/042938.html 22:06:33 <mtreinish> I'm thinking setting up a similar liaison program to oslo might make sense for qa 22:06:53 <mtreinish> I was hoping to get some opinions on it before I made an announcement on the ML 22:07:24 <mtreinish> the concept is there is a dedicated person who will attend the qa meeting and champion project specific things 22:07:28 <dkranz> mtreinish: you mean to liason with domain experts in projects around tempest? 22:08:09 <mtreinish> dkranz: yeah so a nova-dev volunteers and will be a goto point for anything overlapping between tempest and nova 22:08:16 <mtreinish> or grenade and tempest 22:08:18 <mtreinish> etc 22:08:23 <dkranz> mtreinish: doesn't moving functional tests to projects reduce the need for that. Not saying it is a bad idea. 22:08:25 <mtreinish> wow can't type 22:08:44 <mtreinish> dkranz: well only to a certain extent 22:09:06 <mtreinish> and not necessarily, since functional testing haven't really been spun up in force 22:09:09 <mtreinish> yet 22:09:11 <dkranz> mtreinish: right, but would we have 15 new people at our meeting doingnot so much? 22:09:37 <mtreinish> well hopefully they'd be involved 22:10:07 <dkranz> mtreinish: I have advocated for this in the past. 22:10:18 <andreaf> is it restricted to meetings? 22:10:18 <mtreinish> my thoughts were this would be especially useful with the tempest lib migration 22:10:25 <mtreinish> andreaf: no it's more than just meetings 22:10:46 <mtreinish> #link https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons 22:10:51 <dkranz> mtreinish: ok, then I have no real concern 22:10:51 <mtreinish> that's the oslo page for them 22:11:12 <andreaf> ok because it would be useful if I'm reviewing a cinder test patch to have a contact on cinder side to perhaps jump him and clarify a few things 22:11:27 <dkranz> andreaf: or ask for a review 22:12:00 <dkranz> Until now the message we got was "Add Core reviewers" which doesn't really work 22:12:19 <mtreinish> andreaf, dkranz: well normally I just bug jgriffith on irc for those :) 22:12:48 <mtreinish> but, yeah that's the idea having a dedicated person who will help us for other project use cases 22:12:57 <gmann_> mtreinish: so in case of review, will it be compulsory to have +1/2 from them before approval? or just their opinion would work may be through IRC etc 22:13:14 <mtreinish> gmann_: no, it wouldn't be compulsory 22:13:32 <mtreinish> gmann_: it's just about having a known person to contact 22:13:58 <mtreinish> gmann_: mkoderer is signed up right now as the oslo liaison for tempest so it might make sense to ask his opinion on this too 22:14:27 <andreaf> it seems like a good idea to me 22:14:40 <mtreinish> andreaf: yeah that was what I thought 22:14:44 <dkranz> I agree 22:14:50 <masayukig> agree 22:14:56 <mtreinish> anyway I'll take it to the ML 22:15:01 <mtreinish> and we'll go from there 22:15:21 <mtreinish> but I think starting at the end of Juno/start of Kilo we'll probably have this in place 22:15:45 <mtreinish> ok if there isn't anything else on this let's move on 22:15:47 <dkranz> mtreinish: yes, we can corner people at the summit :) 22:16:03 <mtreinish> dkranz: heh, yeah 22:16:09 <mtreinish> #topic Can we start using smoke tag for its intended purpose? (dkranz) 22:16:28 <mtreinish> dkranz: the floor is yours 22:16:29 <dkranz> mtreinish: So I just wanted to know if there is any blocker on this ow 22:16:31 <dkranz> now 22:16:36 <mtreinish> well your sub bullet 22:16:46 <mtreinish> because master is still used for icehouse 22:16:54 <dkranz> The only issue I see is the icehouse neutron runs that still use smoke 22:17:29 <mtreinish> but if we switch that job to just run network 22:17:33 <mtreinish> I think we can get around it 22:17:43 <mtreinish> my only concern is knowing what is valid for smoke and not 22:17:51 <dkranz> mtreinish: that would be good 22:18:04 <dkranz> mtreinish: you mean how we decide to label something as smoke? 22:18:08 <mtreinish> dkranz: yeah 22:18:16 <mtreinish> I think having a db of run history will be useful for figuring that out 22:18:40 <mtreinish> which means I need to get https://review.openstack.org/108003 working soon 22:18:45 <dkranz> mtreinish: Yeah. One idea was to have just one test for each specific api be smoke, and no negative. 22:19:31 <mtreinish> dkranz: yeah I was imagining a more metric based approach tests that fail above a certain freq would be good smoke tests (because they're testing common fail points) 22:19:44 <mtreinish> dkranz: because picking one test still has the issue of which test 22:20:16 <dkranz> mtreinish: Sure but many will not have failed much, except for unrelated reasons 22:20:33 <dkranz> mtreinish: So I'm not sure how much metric info will be available 22:21:20 <mtreinish> dkranz: it's those unrelated reasons that make it a good test, because they're tickling a common issue 22:21:35 <mtreinish> dkranz: well that was the long running db thing I'm working on is to collect and aggregate that data over a relase or 2 22:21:42 <mtreinish> but yeah it's still a long term thing 22:21:48 <dkranz> mtreinish: Well, for many fails it is just trying to create a server 22:21:55 <mtreinish> so we'll probably need to figure out criteria in the short term 22:21:58 <dkranz> mtreinish: sure 22:22:17 <dkranz> mtreinish: I'm throwing out the "one per api" as a starting point 22:22:18 <mtreinish> dkranz: yeah but over a long time the incidental failures can be filtered out 22:22:25 <dkranz> mtreinish: Because we will want at least that 22:23:03 <dkranz> mtreinish: so what exactly has to be done to deal with the icehouse issue? 22:23:33 <mtreinish> dkranz: we need to add a new job definition, verify that it runs the same tests, and switch config to use it 22:23:38 <dkranz> mtreinish: You want to change the icehouse neutron job to just run network tests? 22:24:00 <mtreinish> dkranz: yeah just use the regex 'network' and exclude slow should do the trick 22:24:39 <dkranz> mtreinish: but that won't run the same tests as the current job whic hruns smoke from all services 22:25:24 <mtreinish> dkranz: then we need to add the attr network-legacy-test or something like that which'll be picked up by the regex 22:25:40 <dkranz> mtreinish: anyway, we can discuss this outside the meeting 22:25:41 <mtreinish> or we could do network and new attr tag 22:25:44 <mtreinish> dkranz: yeah 22:25:51 <mtreinish> ok is there anything else on this topic? 22:25:55 <dkranz> no 22:26:00 <mtreinish> ok then let's move on 22:26:14 <mtreinish> #topic Specs Review 22:26:22 <mtreinish> does anyone have a spec review to bring up 22:26:38 <mtreinish> andreaf: sorry I haven't had a chance to look at the ssh auth one in detail yet 22:26:41 <andreaf> #link https://review.openstack.org/#/c/94741/ 22:26:50 <mtreinish> andreaf: :) 22:26:59 <andreaf> mtreinish, ok that's the only one I had in mind 22:27:49 <mtreinish> andreaf: ok, I'll try to take a look at it soon, its a good one to work on 22:27:59 <mtreinish> but at this point I don't see it getting finished until kilo 22:28:22 <mtreinish> I feel we've got enough open dev work for juno at this point 22:28:28 <andreaf> mtreinish, probably not - other bp are taking most of the time 22:28:50 <mtreinish> ok does anyone have any other specs to bring up? 22:29:02 <andreaf> mtreinish, but it's a tricky one so it would be good to get some ideas before the summit 22:29:20 <mtreinish> andreaf: heh, yeah I'll definitely look at it before summit... 22:29:46 <mtreinish> ok let's move on then 22:29:50 <mtreinish> #topic Blueprints 22:29:56 <mtreinish> #link https://blueprints.launchpad.net/tempest/ 22:29:56 <dkranz> https://blueprints.launchpad.net/tempest/+spec/client-checks-success 22:30:06 <mtreinish> dkranz: ok how's that going? 22:30:23 <dkranz> There are some patches up we should look at quickly because they suffer from rebases 22:30:38 <mtreinish> dkranz: link? 22:30:49 <dkranz> Also, we chould increase the priority because this is needed before clients can be moved to tempest-lib 22:31:35 <dkranz> mtreinish: https://review.openstack.org/#/q/topic:bp/client-checks-success,n,z 22:31:47 <mtreinish> #link https://review.openstack.org/#/q/topic:bp/client-checks-success,n,z 22:32:22 <dkranz> I gave -1 to the top one 22:32:32 <mtreinish> dkranz: as for the prio I viewed this as more of a best effort thing which is why I made it low 22:32:59 <dkranz> Sure. Just noting that the tempest-lib thing needs it 22:33:00 <mtreinish> mostly because we can update things without really changing funcitonality 22:33:17 <mtreinish> dkranz: does it? because we still need to return the headers from the lib 22:33:23 <mtreinish> it would just add an extra assert 22:33:41 <mtreinish> and if we do it in pieces we can do it right before I guess too 22:33:47 <dkranz> sure 22:34:04 <mtreinish> dkranz: but I'm fine bumping it if you think it's warranted 22:34:12 <dkranz> These patches don't stop the headers from being returned but they are ignored 22:34:12 <mtreinish> set it at whatever you feel it should be 22:34:16 <dkranz> mtreinish: ok 22:34:52 <mtreinish> so we've still got a bunch of open bps for the cycle 22:34:55 <dkranz> mtreinish: That's it. Just a request for reviews on these 22:35:00 <mtreinish> and not so much time 22:35:08 <asselin> will bp's be accepted after J3? 22:35:18 <dkranz> mtreinish: I have not had a chance to look at the cleanup patch 22:35:27 <asselin> bp implementation I mean. 22:35:29 <mtreinish> asselin: yes 22:35:29 <jgriffith> andreaf: BTW, always feel free to just grab me on IRC as mtreinish suggested :) 22:35:48 <andreaf> jgriffith, will do thanks :) 22:35:50 <mtreinish> jgriffith: heh, I was just giving you a hard time :) 22:35:50 <dkranz> jgriffith: I do :) 22:36:01 <jgriffith> mtreinish: adahms :) I think it's perfectly valid :) 22:36:05 <jgriffith> sorry to interrupt 22:36:16 <mtreinish> asselin: we don't have a feature freeze like the other projects 22:36:30 <asselin> mtreinish, I plan to continue work on https://blueprints.launchpad.net/tempest/+spec/stress-api-tracking after J3. I need to focus on cinder work due before J3. Unless there are people waiting for it 22:36:38 <asselin> mtreinish, good to know 22:36:44 <mtreinish> mostly because contributions for tempest tend to come mostly at the end of the cycle 22:36:56 <mtreinish> after everyone else has a feature freeze and they need to implement tests 22:37:08 <mtreinish> asselin: case and point :) 22:37:12 <asselin> yup :) 22:37:26 <mtreinish> on the high prio bps 22:37:48 <mtreinish> masayukig, andreaf : any update on https://blueprints.launchpad.net/tempest/+spec/tempest-client-scenarios 22:37:56 <mtreinish> I've seen a lot of patches moving on that 22:38:01 <mtreinish> how many left do we have? 22:38:22 <masayukig> yeah, The (migrating) patches statuses are MERGED:2, APPROVED:2, SUBMITTED:9, NOT YET:5 now. 22:38:33 <masayukig> #link https://etherpad.openstack.org/p/tempest-client-scenarios 22:38:41 <masayukig> #link https://review.openstack.org/#/q/topic:bp/tempest-client-scenarios,n,z 22:38:46 <mtreinish> masayukig: ok awesome, we're getting there then 22:39:02 <mtreinish> if people could prioritize reviews on that bp, because it blocks a couple of other things 22:39:15 <andreaf> mtreinish, masayukig there were some more merged for related stuff like the os-networks client or the custom matcher 22:40:04 <mtreinish> andreaf: yeah fair point that's just a list of test patches 22:40:11 <mtreinish> ok are there any other bp updates? 22:40:21 <andreaf> I have the orchestration ones assigned to me but I will be off for a week and I've been busy on other stuff so I'll remove my name for now 22:40:27 <andreaf> mtreinish, test-accounts 22:40:40 <andreaf> #link http://specs.openstack.org/openstack/qa-specs/specs/test-accounts.html 22:40:52 <mtreinish> andreaf: heh, well that still needs some work, we're probably going to have to drop setup class to get it working... 22:41:09 <dkranz> mtreinish: ick 22:41:21 <mtreinish> dkranz: yeah the problem is by reusing creds if we fail we leak 22:41:27 <mtreinish> and fail includes a skip 22:41:31 <mtreinish> in setupclass 22:41:49 <andreaf> mtreinish, yes I just wanted to say that it uncovered somehow expected issues - in both setup and teardown at class level 22:42:30 <andreaf> even when using safe_setup and teardown is invoked, teardown often fails because of un-initialized attributes 22:42:47 <andreaf> and I've seen at least one case where credentials are deleted before other cleanups 22:42:50 <mtreinish> sigh, yeah it's a real mess 22:43:11 <andreaf> so a lot of issues uncovered - or made more relevant - by this bp 22:43:50 <mtreinish> andreaf: but when it's all said and done we should have much cleaner tempest runs at least :) 22:44:01 <andreaf> mtreinish, yep 22:44:15 <mtreinish> ok so if there aren't any other bps let's move on because we're down to ~15min 22:44:20 <andreaf> mtreinish, I was wondering if someone is working on the branchless extension? 22:44:32 <mtreinish> andreaf: salv-orlando volunteered on monday 22:44:41 <mtreinish> and I actually need to ping him about it after the meeting 22:44:51 <andreaf> mtreinish, great 22:44:52 <mtreinish> because I had some suggestions 22:45:04 <mtreinish> yeah it's awesome, because that's holding up some patches 22:45:08 <salv-orlando> mtreinish: yeah I’m actually pushing the patches now 22:45:14 <mtreinish> salv-orlando: ok cool 22:45:19 <andreaf> mtreinish, I'm happy to do reviews there if needed 22:45:42 <salv-orlando> but your suggestions are still welcome 22:45:52 <mtreinish> andreaf: cool that'll help, but most of it will probably be devstack and devstack-gate 22:46:01 <mtreinish> ok let's move on 22:46:08 <mtreinish> #topic Neutron testing 22:46:31 <mtreinish> so the big news here is salv-orlando finally got neutron full parallel gating everywhere 22:46:34 <mtreinish> \o/ 22:46:57 <mtreinish> I'm not sure what other updates there are here though 22:46:59 <andreaf> :) great job 22:47:01 <masayukig> great! 22:47:03 <salv-orlando> bad news is that today an old bug came back. It started about 24 hours ago, I will track it down tomorrow 22:47:17 <salv-orlando> https://bugs.launchpad.net/neutron/+bug/1265495 <- 5 hits in gate today 22:47:18 <uvirtbot> Launchpad bug 1265495 in neutron "Error reading SSH protocol banner" [Critical,Fix released] 22:47:29 <mtreinish> oh more ssh fun... 22:47:44 <salv-orlando> mtreinish: yeah it’s a tricky part where the connection works but auth fails 22:48:06 <mtreinish> that always feels server metadata related to me 22:48:44 <salv-orlando> Indeed that was the first thing we looked at last time. It would be fun if the metadata server is returning some other server’s ssh key 22:49:10 <mtreinish> heh, yeah I think we've seen that before 22:49:26 <mtreinish> ok are there any other neutron updates? 22:49:34 <mtreinish> otherwise let's move on 22:50:03 <mtreinish> #topic Bugs 22:50:26 <mtreinish> dkranz: is there any update on finding someone to organize a bug day? 22:50:48 <dkranz> mtreinish: I will do it. Is Tuesday Sept. 2 OK for folks? 22:51:03 <mtreinish> dkranz: I think that should work for me 22:51:26 <dkranz> mtreinish: ok, I was going to send out an email but wanted to get some feedback here 22:51:38 <andreaf> dkranz, sounds ok 22:51:59 <dkranz> We have been getting a lot less of the random stacktraces going to tempest bugs I think 22:52:10 <mtreinish> dkranz: ok yeah I think that'll work for most people put it on the ML 22:52:14 <dkranz> But I'm sure there are stil some 22:52:16 <mtreinish> dkranz: oh that would be a nice change 22:52:18 <dkranz> mtreinish: will do 22:52:36 <mtreinish> I get emails for each new bug and I've seen a few go by 22:52:48 <dkranz> yeah 22:52:50 <mtreinish> but I think most people are starting to catch on 22:52:56 <dkranz> right 22:52:59 <mtreinish> or just filing them against openstack-ci instead :) 22:53:16 <mtreinish> ok let's move on 22:53:17 <dkranz> mtreinish: at least that is more likely to be correct :) 22:53:23 <mtreinish> #topic Critical Reviews 22:53:35 <mtreinish> so does anyone have any reviews that need some extra attention? 22:53:37 <adam_g> yea! 22:53:50 <mtreinish> adam_g: ok, links? 22:53:52 <adam_g> ironic has some qa requirements that need to be fulfilled before/right around feature freeze 22:54:08 <adam_g> we need migration testing using greande from nova-bm -> ironic 22:54:14 <adam_g> #link https://review.openstack.org/#/q/topic:ironic_grenade+status:open,n,z 22:54:15 <dkranz> https://review.openstack.org/#/c/112581/ (cleanup) and the client checks 22:54:43 <adam_g> in addition to some grenade work, this is dependent on general tempest work we've been chipping away at, which is now shrunk to a small set of patches to devstack+tempest 22:54:51 <adam_g> #link https://review.openstack.org/#/q/topic:ironic_tempest+status:open,n,z 22:55:17 <mtreinish> adam_g: oh lucky you, grenade patches move slowly, and 2/4 people with +2 on it are at linuxcon this week... 22:55:18 <adam_g> the grenade sideways migration testing is based off discussion we had at the nova midcycle, and will probably progress faster when sdague is back 22:55:28 <adam_g> right 22:55:35 <mtreinish> #link https://review.openstack.org/#/c/112581/ 22:56:02 <adam_g> it would be *great* if the remaining tempest related work could get some review cycles, in the meantime 22:56:13 <clarkb> adam_g: the devstack-gate side of the sideways upgrade/migration tests is done 22:56:22 <mtreinish> adam_g: the 2 patches 22:56:24 <mtreinish> ? 22:56:25 <adam_g> clarkb, yeah, i saw that it merged. thanks 22:56:33 <clarkb> adam_g: now we need https://review.openstack.org/#/c/111853/ and its child 22:56:39 * clarkb will bug fungi for that now 22:56:58 <adam_g> mtreinish, yeah, 2 tempest patches @ https://review.openstack.org/#/q/topic:ironic_tempest+status:open,n,z and a few more devstack patches 22:57:10 <mtreinish> I have one commit this week too: 22:57:12 <mtreinish> #link https://review.openstack.org/99451 22:57:20 <mtreinish> oh 2 actually: 22:57:22 <mtreinish> #link https://review.openstack.org/111635 22:57:38 <mtreinish> adam_g: ok cool 22:57:48 <mtreinish> ok does anyone else have any reviews to bring up? 22:58:04 <mtreinish> dkranz: on that one, yeah that'll be good to have 22:58:11 <mtreinish> dkranz: I do have some initial comments already 22:58:23 <mtreinish> but I'll do a deep dive on it tomorrow 22:58:27 <dkranz> mtreinish: I was going to get to that tomorrow 22:58:43 <dkranz> mtreinish: I think it has some issues 22:58:50 <mtreinish> it's also kind of a big script, which is why I was hoping for a more iterative approach earlier 22:59:01 <mtreinish> dkranz: yeah, that's what I thought too 22:59:10 <dkranz> mtreinish: Well, it does implement the spec :) 22:59:32 <mtreinish> heh, fair enough 22:59:41 <mtreinish> ok if there aren't any other reviews 22:59:50 <mtreinish> I guess we'll call it a meeting with 20sec left 22:59:55 <dkranz> bye all 23:00:09 <mtreinish> #endmeeting