22:01:30 #startmeeting qa 22:01:30 Meeting started Thu Aug 21 22:01:30 2014 UTC and is due to finish in 60 minutes. The chair is mtreinish. Information about MeetBot at http://wiki.debian.org/MeetBot. 22:01:31 Useful Commands: #action #agreed #help #info #idea #link #topic #startvote. 22:01:33 The meeting name has been set to 'qa' 22:01:39 hi who's here today? 22:01:48 hi 22:01:49 hi 22:02:03 heh, it'll be a nice quick meeting today :) 22:02:05 #link https://wiki.openstack.org/wiki/Meetings/QATeamMeeting#Proposed_Agenda_for_August_21_2014_.282200_UTC.29 22:02:07 hi 22:02:09 ^^^ Today's agenda 22:02:25 o/ 22:02:41 ok let's get started, maybe more people will trickle in later 22:02:54 #topic Devstack now part of the QA program 22:03:06 I just wanted to make an announcement on this 22:03:14 dtroyer's governance change landed earlier this week 22:03:26 so the devstack repo is now officially part of the qa program 22:03:38 this was announced on the ML a couple of weeks ago 22:03:40 mtreinish: does that really change anything? 22:03:43 no 22:03:48 yay welcome dtroyer and devstack team to the QA program 22:03:51 but it felt news worthy :) 22:03:57 mtreinish: sure :) 22:04:15 ok anything else on this? 22:04:22 #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/041731.html 22:04:32 ^^^? 22:04:43 masayukig: yep that was the ML post 22:05:03 ok let's move on 22:05:03 ok, thanks :) 22:05:20 #topic Starting a QA liaison system (like oslo) (mtreinish) 22:05:39 so this was another topic that came up briefly on the ML 22:05:51 as a branch on the giant future of the integrated release thread 22:06:14 #link http://lists.openstack.org/pipermail/openstack-dev/2014-August/042938.html 22:06:33 I'm thinking setting up a similar liaison program to oslo might make sense for qa 22:06:53 I was hoping to get some opinions on it before I made an announcement on the ML 22:07:24 the concept is there is a dedicated person who will attend the qa meeting and champion project specific things 22:07:28 mtreinish: you mean to liason with domain experts in projects around tempest? 22:08:09 dkranz: yeah so a nova-dev volunteers and will be a goto point for anything overlapping between tempest and nova 22:08:16 or grenade and tempest 22:08:18 etc 22:08:23 mtreinish: doesn't moving functional tests to projects reduce the need for that. Not saying it is a bad idea. 22:08:25 wow can't type 22:08:44 dkranz: well only to a certain extent 22:09:06 and not necessarily, since functional testing haven't really been spun up in force 22:09:09 yet 22:09:11 mtreinish: right, but would we have 15 new people at our meeting doingnot so much? 22:09:37 well hopefully they'd be involved 22:10:07 mtreinish: I have advocated for this in the past. 22:10:18 is it restricted to meetings? 22:10:18 my thoughts were this would be especially useful with the tempest lib migration 22:10:25 andreaf: no it's more than just meetings 22:10:46 #link https://wiki.openstack.org/wiki/Oslo/ProjectLiaisons 22:10:51 mtreinish: ok, then I have no real concern 22:10:51 that's the oslo page for them 22:11:12 ok because it would be useful if I'm reviewing a cinder test patch to have a contact on cinder side to perhaps jump him and clarify a few things 22:11:27 andreaf: or ask for a review 22:12:00 Until now the message we got was "Add Core reviewers" which doesn't really work 22:12:19 andreaf, dkranz: well normally I just bug jgriffith on irc for those :) 22:12:48 but, yeah that's the idea having a dedicated person who will help us for other project use cases 22:12:57 mtreinish: so in case of review, will it be compulsory to have +1/2 from them before approval? or just their opinion would work may be through IRC etc 22:13:14 gmann_: no, it wouldn't be compulsory 22:13:32 gmann_: it's just about having a known person to contact 22:13:58 gmann_: mkoderer is signed up right now as the oslo liaison for tempest so it might make sense to ask his opinion on this too 22:14:27 it seems like a good idea to me 22:14:40 andreaf: yeah that was what I thought 22:14:44 I agree 22:14:50 agree 22:14:56 anyway I'll take it to the ML 22:15:01 and we'll go from there 22:15:21 but I think starting at the end of Juno/start of Kilo we'll probably have this in place 22:15:45 ok if there isn't anything else on this let's move on 22:15:47 mtreinish: yes, we can corner people at the summit :) 22:16:03 dkranz: heh, yeah 22:16:09 #topic Can we start using smoke tag for its intended purpose? (dkranz) 22:16:28 dkranz: the floor is yours 22:16:29 mtreinish: So I just wanted to know if there is any blocker on this ow 22:16:31 now 22:16:36 well your sub bullet 22:16:46 because master is still used for icehouse 22:16:54 The only issue I see is the icehouse neutron runs that still use smoke 22:17:29 but if we switch that job to just run network 22:17:33 I think we can get around it 22:17:43 my only concern is knowing what is valid for smoke and not 22:17:51 mtreinish: that would be good 22:18:04 mtreinish: you mean how we decide to label something as smoke? 22:18:08 dkranz: yeah 22:18:16 I think having a db of run history will be useful for figuring that out 22:18:40 which means I need to get https://review.openstack.org/108003 working soon 22:18:45 mtreinish: Yeah. One idea was to have just one test for each specific api be smoke, and no negative. 22:19:31 dkranz: yeah I was imagining a more metric based approach tests that fail above a certain freq would be good smoke tests (because they're testing common fail points) 22:19:44 dkranz: because picking one test still has the issue of which test 22:20:16 mtreinish: Sure but many will not have failed much, except for unrelated reasons 22:20:33 mtreinish: So I'm not sure how much metric info will be available 22:21:20 dkranz: it's those unrelated reasons that make it a good test, because they're tickling a common issue 22:21:35 dkranz: well that was the long running db thing I'm working on is to collect and aggregate that data over a relase or 2 22:21:42 but yeah it's still a long term thing 22:21:48 mtreinish: Well, for many fails it is just trying to create a server 22:21:55 so we'll probably need to figure out criteria in the short term 22:21:58 mtreinish: sure 22:22:17 mtreinish: I'm throwing out the "one per api" as a starting point 22:22:18 dkranz: yeah but over a long time the incidental failures can be filtered out 22:22:25 mtreinish: Because we will want at least that 22:23:03 mtreinish: so what exactly has to be done to deal with the icehouse issue? 22:23:33 dkranz: we need to add a new job definition, verify that it runs the same tests, and switch config to use it 22:23:38 mtreinish: You want to change the icehouse neutron job to just run network tests? 22:24:00 dkranz: yeah just use the regex 'network' and exclude slow should do the trick 22:24:39 mtreinish: but that won't run the same tests as the current job whic hruns smoke from all services 22:25:24 dkranz: then we need to add the attr network-legacy-test or something like that which'll be picked up by the regex 22:25:40 mtreinish: anyway, we can discuss this outside the meeting 22:25:41 or we could do network and new attr tag 22:25:44 dkranz: yeah 22:25:51 ok is there anything else on this topic? 22:25:55 no 22:26:00 ok then let's move on 22:26:14 #topic Specs Review 22:26:22 does anyone have a spec review to bring up 22:26:38 andreaf: sorry I haven't had a chance to look at the ssh auth one in detail yet 22:26:41 #link https://review.openstack.org/#/c/94741/ 22:26:50 andreaf: :) 22:26:59 mtreinish, ok that's the only one I had in mind 22:27:49 andreaf: ok, I'll try to take a look at it soon, its a good one to work on 22:27:59 but at this point I don't see it getting finished until kilo 22:28:22 I feel we've got enough open dev work for juno at this point 22:28:28 mtreinish, probably not - other bp are taking most of the time 22:28:50 ok does anyone have any other specs to bring up? 22:29:02 mtreinish, but it's a tricky one so it would be good to get some ideas before the summit 22:29:20 andreaf: heh, yeah I'll definitely look at it before summit... 22:29:46 ok let's move on then 22:29:50 #topic Blueprints 22:29:56 #link https://blueprints.launchpad.net/tempest/ 22:29:56 https://blueprints.launchpad.net/tempest/+spec/client-checks-success 22:30:06 dkranz: ok how's that going? 22:30:23 There are some patches up we should look at quickly because they suffer from rebases 22:30:38 dkranz: link? 22:30:49 Also, we chould increase the priority because this is needed before clients can be moved to tempest-lib 22:31:35 mtreinish: https://review.openstack.org/#/q/topic:bp/client-checks-success,n,z 22:31:47 #link https://review.openstack.org/#/q/topic:bp/client-checks-success,n,z 22:32:22 I gave -1 to the top one 22:32:32 dkranz: as for the prio I viewed this as more of a best effort thing which is why I made it low 22:32:59 Sure. Just noting that the tempest-lib thing needs it 22:33:00 mostly because we can update things without really changing funcitonality 22:33:17 dkranz: does it? because we still need to return the headers from the lib 22:33:23 it would just add an extra assert 22:33:41 and if we do it in pieces we can do it right before I guess too 22:33:47 sure 22:34:04 dkranz: but I'm fine bumping it if you think it's warranted 22:34:12 These patches don't stop the headers from being returned but they are ignored 22:34:12 set it at whatever you feel it should be 22:34:16 mtreinish: ok 22:34:52 so we've still got a bunch of open bps for the cycle 22:34:55 mtreinish: That's it. Just a request for reviews on these 22:35:00 and not so much time 22:35:08 will bp's be accepted after J3? 22:35:18 mtreinish: I have not had a chance to look at the cleanup patch 22:35:27 bp implementation I mean. 22:35:29 asselin: yes 22:35:29 andreaf: BTW, always feel free to just grab me on IRC as mtreinish suggested :) 22:35:48 jgriffith, will do thanks :) 22:35:50 jgriffith: heh, I was just giving you a hard time :) 22:35:50 jgriffith: I do :) 22:36:01 mtreinish: adahms :) I think it's perfectly valid :) 22:36:05 sorry to interrupt 22:36:16 asselin: we don't have a feature freeze like the other projects 22:36:30 mtreinish, I plan to continue work on https://blueprints.launchpad.net/tempest/+spec/stress-api-tracking after J3. I need to focus on cinder work due before J3. Unless there are people waiting for it 22:36:38 mtreinish, good to know 22:36:44 mostly because contributions for tempest tend to come mostly at the end of the cycle 22:36:56 after everyone else has a feature freeze and they need to implement tests 22:37:08 asselin: case and point :) 22:37:12 yup :) 22:37:26 on the high prio bps 22:37:48 masayukig, andreaf : any update on https://blueprints.launchpad.net/tempest/+spec/tempest-client-scenarios 22:37:56 I've seen a lot of patches moving on that 22:38:01 how many left do we have? 22:38:22 yeah, The (migrating) patches statuses are MERGED:2, APPROVED:2, SUBMITTED:9, NOT YET:5 now. 22:38:33 #link https://etherpad.openstack.org/p/tempest-client-scenarios 22:38:41 #link https://review.openstack.org/#/q/topic:bp/tempest-client-scenarios,n,z 22:38:46 masayukig: ok awesome, we're getting there then 22:39:02 if people could prioritize reviews on that bp, because it blocks a couple of other things 22:39:15 mtreinish, masayukig there were some more merged for related stuff like the os-networks client or the custom matcher 22:40:04 andreaf: yeah fair point that's just a list of test patches 22:40:11 ok are there any other bp updates? 22:40:21 I have the orchestration ones assigned to me but I will be off for a week and I've been busy on other stuff so I'll remove my name for now 22:40:27 mtreinish, test-accounts 22:40:40 #link http://specs.openstack.org/openstack/qa-specs/specs/test-accounts.html 22:40:52 andreaf: heh, well that still needs some work, we're probably going to have to drop setup class to get it working... 22:41:09 mtreinish: ick 22:41:21 dkranz: yeah the problem is by reusing creds if we fail we leak 22:41:27 and fail includes a skip 22:41:31 in setupclass 22:41:49 mtreinish, yes I just wanted to say that it uncovered somehow expected issues - in both setup and teardown at class level 22:42:30 even when using safe_setup and teardown is invoked, teardown often fails because of un-initialized attributes 22:42:47 and I've seen at least one case where credentials are deleted before other cleanups 22:42:50 sigh, yeah it's a real mess 22:43:11 so a lot of issues uncovered - or made more relevant - by this bp 22:43:50 andreaf: but when it's all said and done we should have much cleaner tempest runs at least :) 22:44:01 mtreinish, yep 22:44:15 ok so if there aren't any other bps let's move on because we're down to ~15min 22:44:20 mtreinish, I was wondering if someone is working on the branchless extension? 22:44:32 andreaf: salv-orlando volunteered on monday 22:44:41 and I actually need to ping him about it after the meeting 22:44:51 mtreinish, great 22:44:52 because I had some suggestions 22:45:04 yeah it's awesome, because that's holding up some patches 22:45:08 mtreinish: yeah I’m actually pushing the patches now 22:45:14 salv-orlando: ok cool 22:45:19 mtreinish, I'm happy to do reviews there if needed 22:45:42 but your suggestions are still welcome 22:45:52 andreaf: cool that'll help, but most of it will probably be devstack and devstack-gate 22:46:01 ok let's move on 22:46:08 #topic Neutron testing 22:46:31 so the big news here is salv-orlando finally got neutron full parallel gating everywhere 22:46:34 \o/ 22:46:57 I'm not sure what other updates there are here though 22:46:59 :) great job 22:47:01 great! 22:47:03 bad news is that today an old bug came back. It started about 24 hours ago, I will track it down tomorrow 22:47:17 https://bugs.launchpad.net/neutron/+bug/1265495 <- 5 hits in gate today 22:47:18 Launchpad bug 1265495 in neutron "Error reading SSH protocol banner" [Critical,Fix released] 22:47:29 oh more ssh fun... 22:47:44 mtreinish: yeah it’s a tricky part where the connection works but auth fails 22:48:06 that always feels server metadata related to me 22:48:44 Indeed that was the first thing we looked at last time. It would be fun if the metadata server is returning some other server’s ssh key 22:49:10 heh, yeah I think we've seen that before 22:49:26 ok are there any other neutron updates? 22:49:34 otherwise let's move on 22:50:03 #topic Bugs 22:50:26 dkranz: is there any update on finding someone to organize a bug day? 22:50:48 mtreinish: I will do it. Is Tuesday Sept. 2 OK for folks? 22:51:03 dkranz: I think that should work for me 22:51:26 mtreinish: ok, I was going to send out an email but wanted to get some feedback here 22:51:38 dkranz, sounds ok 22:51:59 We have been getting a lot less of the random stacktraces going to tempest bugs I think 22:52:10 dkranz: ok yeah I think that'll work for most people put it on the ML 22:52:14 But I'm sure there are stil some 22:52:16 dkranz: oh that would be a nice change 22:52:18 mtreinish: will do 22:52:36 I get emails for each new bug and I've seen a few go by 22:52:48 yeah 22:52:50 but I think most people are starting to catch on 22:52:56 right 22:52:59 or just filing them against openstack-ci instead :) 22:53:16 ok let's move on 22:53:17 mtreinish: at least that is more likely to be correct :) 22:53:23 #topic Critical Reviews 22:53:35 so does anyone have any reviews that need some extra attention? 22:53:37 yea! 22:53:50 adam_g: ok, links? 22:53:52 ironic has some qa requirements that need to be fulfilled before/right around feature freeze 22:54:08 we need migration testing using greande from nova-bm -> ironic 22:54:14 #link https://review.openstack.org/#/q/topic:ironic_grenade+status:open,n,z 22:54:15 https://review.openstack.org/#/c/112581/ (cleanup) and the client checks 22:54:43 in addition to some grenade work, this is dependent on general tempest work we've been chipping away at, which is now shrunk to a small set of patches to devstack+tempest 22:54:51 #link https://review.openstack.org/#/q/topic:ironic_tempest+status:open,n,z 22:55:17 adam_g: oh lucky you, grenade patches move slowly, and 2/4 people with +2 on it are at linuxcon this week... 22:55:18 the grenade sideways migration testing is based off discussion we had at the nova midcycle, and will probably progress faster when sdague is back 22:55:28 right 22:55:35 #link https://review.openstack.org/#/c/112581/ 22:56:02 it would be *great* if the remaining tempest related work could get some review cycles, in the meantime 22:56:13 adam_g: the devstack-gate side of the sideways upgrade/migration tests is done 22:56:22 adam_g: the 2 patches 22:56:24 ? 22:56:25 clarkb, yeah, i saw that it merged. thanks 22:56:33 adam_g: now we need https://review.openstack.org/#/c/111853/ and its child 22:56:39 * clarkb will bug fungi for that now 22:56:58 mtreinish, yeah, 2 tempest patches @ https://review.openstack.org/#/q/topic:ironic_tempest+status:open,n,z and a few more devstack patches 22:57:10 I have one commit this week too: 22:57:12 #link https://review.openstack.org/99451 22:57:20 oh 2 actually: 22:57:22 #link https://review.openstack.org/111635 22:57:38 adam_g: ok cool 22:57:48 ok does anyone else have any reviews to bring up? 22:58:04 dkranz: on that one, yeah that'll be good to have 22:58:11 dkranz: I do have some initial comments already 22:58:23 but I'll do a deep dive on it tomorrow 22:58:27 mtreinish: I was going to get to that tomorrow 22:58:43 mtreinish: I think it has some issues 22:58:50 it's also kind of a big script, which is why I was hoping for a more iterative approach earlier 22:59:01 dkranz: yeah, that's what I thought too 22:59:10 mtreinish: Well, it does implement the spec :) 22:59:32 heh, fair enough 22:59:41 ok if there aren't any other reviews 22:59:50 I guess we'll call it a meeting with 20sec left 22:59:55 bye all 23:00:09 #endmeeting