Tuesday, 2014-04-29

*** mmaglana has quit IRC00:01
*** aysyd has joined #openstack-qa00:03
*** kgriffs is now known as kgriffs|afk00:04
*** Aimon has quit IRC00:06
*** atiwari has quit IRC00:08
*** sthaha has joined #openstack-qa00:17
*** sthaha has quit IRC00:17
*** sthaha has joined #openstack-qa00:17
*** nati_uen_ has quit IRC00:30
*** nati_ueno has joined #openstack-qa00:31
*** nati_ueno has quit IRC00:31
*** mrodden has quit IRC00:40
*** nati_ueno has joined #openstack-qa00:44
*** mrodden has joined #openstack-qa00:55
openstackgerritMorgan Fainberg proposed a change to openstack-dev/devstack: Add switch to configure any capable service under apache  https://review.openstack.org/9086401:05
*** sweston has quit IRC01:06
*** torandu_ has joined #openstack-qa01:08
openstackrecheckopenstack/nova change: https://review.openstack.org/89663 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1295876, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/129587601:13
*** Aimon has joined #openstack-qa01:14
*** sweston has joined #openstack-qa01:23
*** nati_uen_ has joined #openstack-qa01:23
*** nati_ueno has quit IRC01:27
*** mmaglana has joined #openstack-qa01:27
*** psedlak has quit IRC01:28
*** mmaglana has quit IRC01:30
*** psedlak has joined #openstack-qa01:35
openstackrecheckopenstack/horizon change: https://review.openstack.org/75553 failed gate-grenade-dsvm in the gate queue with an unrecognized error01:35
*** torandu has quit IRC01:37
*** torandu has joined #openstack-qa01:39
*** adalbas has quit IRC01:49
*** catohornet has quit IRC01:52
*** markmcclain has quit IRC02:00
*** adalbas has joined #openstack-qa02:01
*** nati_uen_ has quit IRC02:02
*** sweston has quit IRC02:09
*** Sukhdev has joined #openstack-qa02:12
*** sweston has joined #openstack-qa02:18
openstackrecheckopenstack/taskflow change: https://review.openstack.org/90444 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/128681802:31
*** mriedem has quit IRC02:41
*** timello has quit IRC02:44
*** Sukhdev has quit IRC02:47
openstackrecheckopenstack/horizon change: https://review.openstack.org/87728 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error02:49
openstackrecheckopenstack/taskflow change: https://review.openstack.org/90569 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/128681802:56
openstackrecheckopenstack/horizon change: https://review.openstack.org/89635 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/1286818, check-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1263417 https://bugs.launchpad.net/bugs/128681803:05
*** Abhi_ has joined #openstack-qa03:23
*** catohornet has joined #openstack-qa03:31
*** catohornet has quit IRC03:35
openstackgerritLance Bragstad proposed a change to openstack/tempest: Default domain ID to valid UUID format  https://review.openstack.org/9077303:40
*** catohornet has joined #openstack-qa03:52
*** aysyd has quit IRC03:57
*** chandan_kumar has joined #openstack-qa04:05
*** yfried has quit IRC04:05
*** sweston has quit IRC04:12
*** morganfainberg is now known as morganfainberg_Z04:20
*** Ch00k has joined #openstack-qa04:25
*** sweston has joined #openstack-qa04:27
*** GMann has joined #openstack-qa04:33
*** sweston_ has joined #openstack-qa04:34
*** sweston has quit IRC04:34
*** Abhi_ has quit IRC04:34
*** Abhijeet has joined #openstack-qa04:35
*** mspreitz has joined #openstack-qa04:36
mspreitzI just got this complaint: The RSA host key for [review.openstack.org]:29418 has changed04:37
mspreitzIs anybody else getting this?04:38
*** harlowja is now known as harlowja_away04:56
*** Sukhdev has joined #openstack-qa05:08
*** GMann has quit IRC05:08
*** yfried has joined #openstack-qa05:18
*** coolsvap has joined #openstack-qa05:21
coolsvapQA folks, is there any documentation for configuring tempest to run against running openstack deployment,  I have configured my tempest.conf, but getting error for IMAGE_ID05:29
coolsvapdo I need to export IMAGE_ID?05:29
adam_gmspreitz, http://lists.openstack.org/pipermail/openstack-dev/2014-April/033908.html05:33
mspreitzadam_g: thanks05:34
Anjucoolsvap:  http://sagar-tech-gyan.blogspot.in/2012/04/testing-open-stack-with-devstack.html05:35
mspreitzcoolsvap: I was once told, without understanding what is going on, that I may need to include "HEAT_FETCHED_TEST_IMAGE=Fedora-i386-20-20131211.1-sda" in my localrc.  Are you doing that?05:35
mspreitzcoolsvap: I was told I would need that in order for Tempest to work.05:36
mspreitzBut in recent trials it seems I do not need that.05:36
mspreitzIt may depend on other details.05:37
coolsvapAnju: mspreitz thanks, I am not doing it against devstack05:43
coolsvapfor devstack its working for me05:43
coolsvapI have a one node openstack deployed and cloned tempest and trying to run tests against the openstack deployment05:44
*** raies has quit IRC05:50
*** Madan has joined #openstack-qa06:00
*** chandan_kumar has quit IRC06:01
*** Madan has quit IRC06:05
*** ashishsnigam has quit IRC06:06
*** ashishsnigam has joined #openstack-qa06:06
*** Madan has joined #openstack-qa06:08
*** Ch00k has quit IRC06:09
*** kashyap has left #openstack-qa06:09
openstackrecheckopenstack/trove change: https://review.openstack.org/88591 failed gate-tempest-dsvm-neutron in the gate queue with an unrecognized error06:11
*** Madan has quit IRC06:15
*** chandan_kumar has joined #openstack-qa06:16
*** jamielennox is now known as jamielennox|away06:18
*** Madan has joined #openstack-qa06:19
*** chandan_kumar has quit IRC06:22
*** mspreitz has quit IRC06:27
*** psharma has joined #openstack-qa06:35
*** sweston_ has quit IRC06:38
*** chandan_kumar has joined #openstack-qa06:40
*** catohornet has quit IRC06:43
*** evgenyf has joined #openstack-qa06:45
*** afazekas_ has joined #openstack-qa06:58
BobBallAwaysdague: Any chance you can re-push https://review.openstack.org/#/c/90660/? Seems that it was re-pushed yesterday and gate jobs started, but then I guess gerrit was restarted because the workflow+1 has disappeared!06:58
*** tkammer has joined #openstack-qa07:01
*** thomasbiege has joined #openstack-qa07:03
*** jlibosva has joined #openstack-qa07:05
*** Sukhdev has quit IRC07:08
*** thomasbiege has quit IRC07:08
openstackrecheckopenstack/nova change: https://review.openstack.org/44714 failed because of: gate-nova-python27: https://bugs.launchpad.net/bugs/126661107:17
*** thomasbiege has joined #openstack-qa07:20
*** Ch00k has joined #openstack-qa07:29
*** yrabl has joined #openstack-qa07:32
*** sweston has joined #openstack-qa07:34
*** sweston has quit IRC07:39
*** thomasbiege has quit IRC07:41
*** sthaha has quit IRC07:45
*** qba73 has joined #openstack-qa07:45
*** jordanP has joined #openstack-qa07:51
*** yrabl has quit IRC07:53
*** yrabl has joined #openstack-qa07:53
*** safchain has joined #openstack-qa07:53
*** jpich has joined #openstack-qa07:54
*** sthaha has joined #openstack-qa08:01
*** gszasz has joined #openstack-qa08:03
*** ashishsnigam has quit IRC08:04
*** giulivo has joined #openstack-qa08:11
*** yassine has joined #openstack-qa08:12
*** psharma has quit IRC08:17
*** psharma has joined #openstack-qa08:23
*** gszasz has quit IRC08:32
*** gszasz has joined #openstack-qa08:32
*** sweston has joined #openstack-qa08:34
*** sweston has quit IRC08:39
*** qba73 has quit IRC08:43
*** adalbas has quit IRC09:01
*** sthaha has quit IRC09:07
*** thomasbiege has joined #openstack-qa09:10
*** sthaha has joined #openstack-qa09:10
*** sthaha has joined #openstack-qa09:10
*** thomasbiege1 has joined #openstack-qa09:13
*** chandan_kumar has quit IRC09:14
*** chandan_kumar has joined #openstack-qa09:15
*** thomasbiege has quit IRC09:17
*** qba73 has joined #openstack-qa09:19
*** thomasbiege1 has quit IRC09:26
*** MJ has joined #openstack-qa09:27
MJhello all09:27
*** MJ is now known as Guest4611009:28
Guest46110appreciate your help over an issue of tempest09:28
Guest46110tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance_with_tags[gate,smoke] ... FAIL09:28
Guest46110i have submitted the code , jenkins buld validation failed with the above error logs09:29
Guest46110•check-tempest-dsvm-full http://logs.openstack.org/21/89121/2/check/check-tempest-dsvm-full/6fdc305 : FAILURE in 1h 13m 23s09:29
Guest46110the console.html shows .. tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_stop_terminate_instance_with_tags[gate,smoke] ... FAIL09:29
Guest46110is this known bug ?09:30
*** andreykurilin has joined #openstack-qa09:30
Guest46110i cud see that there is another similar tempest testcase which is identiied as bug09:30
Guest46110tempest.thirdparty.boto.test_ec2_instance_run.InstanceRunTest.test_run_terminate_instance[gate,smoke] ... skipped u'Skipped until Bug: 1098891 is resolved.'09:30
Guest46110could you help please09:31
*** tosky has joined #openstack-qa09:45
openstackgerritMh Raies proposed a change to openstack/tempest: VPNaas IPSec policies tests  https://review.openstack.org/4781609:47
*** mmaglana has joined #openstack-qa09:53
*** jhenner1 has joined #openstack-qa10:00
*** jhenner has quit IRC10:02
*** Guest46110 has quit IRC10:03
openstackgerritMh Raies proposed a change to openstack/tempest: Nodestate API test for power state set  https://review.openstack.org/8600210:04
*** andreaf has joined #openstack-qa10:06
*** BobBallAway is now known as BobBall10:09
andreykurilinsdague, Hi! I was advised to appeal to you.:) Can you consult me about running tests via subunit? I interested in division test discovering and execution.10:10
*** mdenny has quit IRC10:17
openstackrecheckopenstack/nova change: https://review.openstack.org/44714 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/125487210:26
openstackgerritMark Maglana proposed a change to openstack/tempest: Make sure to release the floating ip afterwards  https://review.openstack.org/7915310:26
*** jhenner1 has quit IRC10:26
*** ampeg_ has joined #openstack-qa10:36
ampeg_hey guys10:37
ampeg_I was just wondering if there's some kind of public Tempest API10:37
ampeg_rather than going to manager.py for instance in scenario tests10:37
openstackgerritAshish Chandra proposed a change to openstack-dev/devstack: Implement Ceph RadosGW for Openstack Swift compatible API with Keystone support.  https://review.openstack.org/9096510:39
*** coolsvap is now known as coolsvap|afk10:40
*** jhenner has joined #openstack-qa10:40
*** ndipanov has joined #openstack-qa10:45
*** sthaha has quit IRC10:46
*** sthaha has joined #openstack-qa10:47
*** sthaha has joined #openstack-qa10:47
*** gszasz has quit IRC10:55
*** enikanorov__ is now known as enikanorov_11:13
sdagueampeg_: can you give an example?11:35
openstackrecheckopenstack/heat change: https://review.openstack.org/90509 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error11:42
ndipanovhey sdague can we get this in? it's related to https://bugs.launchpad.net/nova/+bug/130277411:52
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Official client based auth  https://review.openstack.org/8291111:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Use auth data to fill credentials  https://review.openstack.org/8211111:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Define V3 Credentials  https://review.openstack.org/8187211:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Get credential IDs from Credentials class  https://review.openstack.org/8024611:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Use auth provider in official client manager  https://review.openstack.org/9016611:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Enforces the use of Credentials (part1)  https://review.openstack.org/7759411:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Enforces the use of Credentials (part2)  https://review.openstack.org/8211211:54
openstackgerritAndrea Frittoli  proposed a change to openstack/tempest: Access credential fields as attributes  https://review.openstack.org/8211311:54
*** ylobankov has quit IRC11:58
sdaguendipanov: should those be skipped to change the behavior instead?12:00
sdagueor do you think we're not going to test that behavior any more in the future?12:01
*** mkollaro has quit IRC12:03
ndipanovsdague, I don't think we should enable those12:03
*** mkollaro has joined #openstack-qa12:03
ndipanov1) we already test them as part of the attach/detach12:04
ndipanov2) even though they are public API - they are never meant to be used by users12:04
sdaguewell #2 is non sensible :) if it's in the public api, you have to assume it's getting used12:04
ndipanovand as such are an implementation detail with wich tmepest should not concern itself iiuc12:04
ndipanovsdague, it's not exposed in the clients12:04
sdaguethe clients != the API12:05
*** mkollaro has quit IRC12:05
*** afazekas_ has quit IRC12:05
*** mkollaro has joined #openstack-qa12:05
ndipanovsdague, if you want to go down that rabbit hole :) may I propose a different place? http://summit.openstack.org/cfp/details/29212:06
*** mkollaro has quit IRC12:06
sdaguesure12:06
sdaguethough, the API is the rest surface12:06
ndipanovsdague, anyway - testing them now would need to be a scenario12:06
*** mkollaro has joined #openstack-qa12:06
ndipanovas we need to attach the volume first12:06
ndipanovand that is already tested as part of the scenario tests12:07
sdaguendipanov: well if the clients don't expose that, then it's not testable in the scenarios. But anyway, lets get the fixes in first12:07
ndipanovhmm12:07
ndipanovwell I meant cli does not expose it12:07
openstackgerritAmit Ugol proposed a change to openstack/tempest: Added HEAT Software Config-Deploy a test  https://review.openstack.org/8979012:07
ndipanovit is in the client actually12:07
sdagueok12:07
sdaguethen #2 is even more moot :)12:07
ndipanovsdague, can't get the fixes without that12:07
ndipanovsdague, I can skip them - or move them to scenarios (useless imho and will add time to runs)12:08
sdagueso I +Aed the remove12:08
ndipanovah OK12:08
ndipanovsdague, thanks :)12:08
sdagueI realistically think we need to figure out how to test volumes better, because these races have been around forever12:09
ndipanovsdague, yes12:09
*** ylobankov has joined #openstack-qa12:10
ndipanovsdague, but some of it is really down to the cinder API... and from what I hear from cinder devs  -internal bookkeeping which is buggy too12:10
ndipanovsdague, I wonder how come people are not more on this stuff12:10
*** tkammer has quit IRC12:10
ndipanovsdague, not getting hit that often in real deployments or?12:11
*** mkollaro has quit IRC12:11
*** dkranz has joined #openstack-qa12:12
*** tosky_ has joined #openstack-qa12:13
*** tosky has quit IRC12:14
*** mkollaro has joined #openstack-qa12:17
*** afazekas_ has joined #openstack-qa12:19
sdaguendipanov: honestly, I expect that people see a fail, can't debug it, and just try again, and it works12:20
sdaguesort of like people just running recheck12:20
*** weshay has joined #openstack-qa12:20
ndipanovsdague, well - since it is sporadic probably lots of users don't see it as a show stopper as well and it never gets fixed12:22
ndipanovsdague, maybe having an upstream bug tag "race" and have a subteam dealing with that if there is enough interest?12:23
sdaguendipanov: well, that's basically what this is - http://status.openstack.org/elastic-recheck/ :)12:24
sdaguewe are currently tracking 91 race bugs12:24
*** tkammer has joined #openstack-qa12:25
ndipanovwell my biggest grief with tempest is that it makes bugs artificially important... just because it is hit in our less-than-realistic tempest runs does not mean it's a real issue to people... now having a crosssection of what users report and tempest would make more sense imho12:26
sdaguendipanov: how would you do that?12:26
ndipanovsdague, no idea :)12:26
sdagueyeh, so that's just random rant then :P12:26
ndipanovsdague, absolutely12:27
ndipanov:)12:27
sdagueif you see an issue a user hits, then turn it into a tempest test, so we can try to nail it down in the gate12:27
ndipanovsdague, but I do think that those little races cost us a lot... they get reported and a lot of time is spent chasing them12:27
ndipanovmaybe black-box all the way is not the best tool here12:28
ndipanovsdague, anyway - I will try to go through that and see any other cinder-nova ones and try to look into them12:29
*** jhenner has quit IRC12:29
sdaguesure, there is definitely more that could be done.12:30
*** aysyd has joined #openstack-qa12:30
sdaguebut I do get frustrated with folks that say "it's not realistic, so I just blindly recheck" then don't show up to help make it better. Because I think that's a giant cop out.12:31
*** weshay has quit IRC12:31
*** dims has quit IRC12:33
ndipanovsdague, right - the thing is - this particular one was actually a pretty big hole that can be seen by just reading the code - so for this kind of thing, there is no excuse... however it would be good to try to correlate gate vs customer bugs and see what is really getting hit in prod systems12:33
sdaguendipanov: so again, I ask how :)12:34
ndipanovonce we actually close all the gaping holes first :)12:34
sdaguebecause in the gate we've got volumes of data that is well structured with structured logs at DEBUG level12:34
sdagueand even with that, it takes a while to get to root issues12:34
*** jhenner has joined #openstack-qa12:35
sdaguewe're doing ~ 30k tempest runs a week12:35
sdagueI'm not sure how you'd cross product in customer bugs vs. that, because you also need to know all the times it works.12:36
ndipanovsdague, right - I get that... I actually think it's a social probelm, not a technical one - ppl don't see it as important enough to warrant fixing or even actually tracking correctly (apart from gate folks)12:36
ndipanovI think it is missguided tho because even though we are not "broken" as such12:36
ndipanovthese little instabilities add up to a ton of engineering hours (I imagine, citation needed)12:37
ndipanovand hurts the project in the long run...12:37
sdagueyep12:38
sdagueso a big part of it is probably getting people to realize these are real issues, I don't know how to message that any more than I'm doing. Suggestions welcomed. :)12:38
ndipanovbut hacking on a new feature we need for X vs chasing down races is a hard sell12:38
ndipanovsdague, well - I do think stuff gets reported12:39
ndipanovnot all of it but some of it12:39
sdaguebecause based on your initial statements you seemed to think you don't consider a lot of these real issues12:39
ndipanovsdague, no no no12:39
sdaguendipanov: sure, lots of stuff gets reported :)12:39
ndipanovsdague, however engineering time is somewhat of a limited resource hence prioriti<ation12:40
sdaguethis is also an attempt to discuss it at summit - http://junodesignsummit.sched.org/event/787ed50a56972fd994d7930e13542e0a12:40
sdaguendipanov: sure12:40
sdagueso I think you raise a good point, that we need to raise the profile of these issues, and the people fixing them, maybe that will help? (or some other social influence bit)12:40
ndipanovyes12:41
sdaguethat we've got a social issue around fixing races as much as a technical issue12:41
ndipanovI do agree 100%12:41
ndipanovI think (and this has been said before a lot)12:41
ndipanovwe need to get better at bug triage12:42
ndipanovin nova I think D&C approach has potential - but there is not enough people12:42
ndipanoveven for the sub-categories of bugs12:43
ndipanovand I think if we are going to weed out the races12:43
ndipanovwe need a team of people owning it12:43
*** tosky_ has quit IRC12:44
*** mkollaro1 has joined #openstack-qa12:45
*** mkollaro has quit IRC12:45
*** jhenner1 has joined #openstack-qa12:46
*** jhenner has quit IRC12:46
*** dims has joined #openstack-qa12:48
andreykurilinCan somebody help me with launching tempest tests via subunit?12:48
*** tkammer_ has joined #openstack-qa12:50
*** tkammer has quit IRC12:50
*** tosky_ has joined #openstack-qa12:51
*** dkranz has quit IRC12:53
*** adalbas has joined #openstack-qa12:56
*** gszasz has joined #openstack-qa12:56
*** Madan has quit IRC12:58
andreafandreykurilin: what's your question exactly? Looking at .testr.conf in tempest can give you some examples12:58
andreykurilinandreaf: For example, how can I execute `tempest.api.compute.admin.test_flavors_negative.FlavorCreateNegativeTestJSON` ?13:02
andreykurilinandreaf: I tried to do it with command "python -m subunit.run $TEST_NAME", but I did not succeed. http://paste.openstack.org/show/77696/13:03
openstackgerritA change was merged to openstack/tempest: Remove {begin,roll}_detaching volume API tests  https://review.openstack.org/9041013:07
vponomaryovandreykurilin: testr run --concurrency=1 --subunit tempest.api.compute.admin.test_flavors_negative.FlavorCreateNegativeTestJSON13:07
andreafandreykurilin, vponomaryov: to run a single test you can also use unittest instead of subunit - it shows errors at import level as well13:09
andreykurilinvponomaryov: Thanks, but I want to devide tests discovering and execution, so I can't use testr.13:12
*** yfried has quit IRC13:12
*** qba73_ has joined #openstack-qa13:13
*** weshay has joined #openstack-qa13:13
*** qba73 has quit IRC13:15
andreykurilinandreaf: `python -m unittest $TEST_NAME` returns the same result as subunit. http://paste.openstack.org/show/77700/13:15
andreafandreykurilin: you can either use a load list or list the tests directly (see python -m subunit.run --help)13:15
andreafandreykurilin: you may be missing testscenarios13:16
andreykurilinandreaf: how to enable/add them?13:17
sdagueandreykurilin: why are you trying to separate these parts out?13:19
*** mriedem has joined #openstack-qa13:19
sdagueandreykurilin: because we're definitely doing more down the path of things like testscenarios and dynamically created tests for negative testing, which means skipping around testr for the normal path is going to become less and less functional13:21
sdagueso perhaps there is something that needs to be added to testr13:21
*** mwagner_lap has quit IRC13:22
*** jgrimm has joined #openstack-qa13:24
andreykurilinandreaf, about --load-list: I don't understand how to work with this option in subunit.run, so I tried to transmit all test names as arguments, but subunit failed(ValueError: no such test method in <class 'tempest.api.compute.admin.test_flavors_negative.FlavorCreateNegativeTestJSON'>: generic_test)13:26
andreykurilinsdague, I want to launch tempest test and calculate real time of execution. For example, time of execution several tests  equals to 0.2s, but total time(with discovering) equals to ~20s.13:26
*** Obulpathi has joined #openstack-qa13:27
sdagueandreykurilin: well, total time includes setupClass13:27
sdaguewhich isn't accounted in the subunit stream13:28
*** Obulpathi has quit IRC13:28
*** Obulpathi has joined #openstack-qa13:28
sdagueI think if you back up and point at a bug or blueprint that you are trying to address, there might be better suggestions here13:29
andreafandreykurilin: discovery includes some overhead as it generates the test list dynamically  - testscenarios are running and generating negative tests and scenario tests13:29
sdagueyeh, it seems like what you really want is to inject more events into the subunit stream13:30
sdaguethen you could time account from that more accurately13:30
sdaguewhich would be helpful in general for tempest as well13:30
*** psharma has quit IRC13:31
andreykurilinsdague, tests discover takes a lot of time. Without discovering, total time of  "tempest.api.image.v2.test_images.ListImagesTest.test_index_no_params" decreased from 20s to 3.5s13:31
sdagueandreykurilin: no, it's not just discovery13:32
sdagueit's also setupClass time13:32
sdagueand in tempest.api.image.v2.test_images.ListImagesTest.test_index_no_param I don't think any reasonable amount is discovery13:33
sdagueit's all setupClass time13:33
sdaguewhich is why doing instrumenting tempest with additional subunit events is the right way to address this13:33
sdaguebecause then you can really see all the time accounting of different phases13:34
sdaguebecause today subunit is giving you timing only from setup -> teardown13:34
sdaguethe list image tests are a great example of where that gets things wrong13:34
sdaguebecause we setup a bunch of images in setupClass so we take that time hit only once13:35
andreykurilin sdague, andreaf: okay. how can I split discovery and other preparation with test execution?13:35
*** tosky_ is now known as tosky13:35
openstackrecheckopenstack/sahara change: https://review.openstack.org/90717 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1295165 https://bugs.launchpad.net/bugs/130925213:35
sdagueandreykurilin: I don't know how to split it. However lifeless explained that we could inject additional subunit events into the subunit stream13:35
sdagueso you could parse that out later to get the math on things more accurately13:36
*** rbak has joined #openstack-qa13:36
sdagueperhaps it's better to take this to the mailing list to discuss?13:36
*** markmcclain has joined #openstack-qa13:40
andreafandreykurilin: discovery alone you should be able to split13:40
andreafsdague: +1 yes this is an interesting discussion for the ML13:41
andreafandreykurilin: python -m subunit.run discover -t ./ tempest/test_discover --list > test_list13:42
*** rbak has quit IRC13:42
sdagueandreaf: right, but I don't think this solves the issue13:42
andreafthat will generate the test list in subunit format in a file13:42
sdagueI'm assuming this is rally13:42
sdagueand rally wants good time accounting in tempest13:42
sdaguewhich I think is correct13:42
sdagueso we should come up with the right way to get enough data in the subunit stream to have that13:43
andreykurilinsdague, andreaf: I will prepare a letter to mailing list a little bit later.13:44
*** jhenner1 has quit IRC13:44
sdaguecool13:44
andreykurilinsdague, yes, my problem is related with launching tempest tests from rally:)13:45
*** markmcclain has quit IRC13:45
sdagueandreykurilin: right, so your actual concern is that the subunit stream doesn't actually account for all the test run time13:46
sdaguewhich we should fix13:46
andreykurilinandreaf: I tried to do the same, but I'm not understand how to load this stream later and execute it.13:48
sdagueandreykurilin: right, but the point is, you can't really do that in a lot of cases13:49
sdagueyou can do discovery if yuo like13:49
sdaguebut that doesn't make discovery go away later13:49
sdagueat least not in all cases13:50
andreafsdague: right13:53
*** malini_afk is now known as malini13:54
andreafandreykurilin: just for sake of completeness - you can either create the subunit list and then do a load and then run, or you can create you list using testtools instead of subunit (so plain text) and then use the --load-list option13:54
ampeg_sdague, sorry for the late response13:57
ampeg_basically I'm trying to code my first scenario test13:58
*** Ch00k has quit IRC13:58
ampeg_so I'm inheriting NetworScenarioTest13:58
ampeg_from manager.py13:58
ampeg_but I was wondering if there was some API documentation for this13:58
andreykurilinsdague, split discovery from the others execution will help to achieve more correct time.13:58
*** Ch00k has joined #openstack-qa13:59
andreykurilinandreaf: yes, I can discover tests via `subunit.run discover` or by using testtools, but, presently, I can't understand how to load list of tests later. I did not found any documentation for subunit, except `subunit.run -h`13:59
*** mrodden has quit IRC13:59
*** dkranz has joined #openstack-qa14:01
sdagueandreykurilin: I disagree14:03
andreykurilinsdague: why?14:03
sdagueachieving correct time will be done by correctly putting all relevant events into the subunit stream14:03
sdagueandreykurilin: because the time to run everything is A14:04
sdaguediscovery is a piece of that, test run is a piece of it14:05
sdaguehowever you are assuming that Discovery (D) + Run (R) == A14:05
*** catohornet has joined #openstack-qa14:05
sdaguehowever when done separately, that may or may not be true14:05
sdaguebecause there are lots of layers here14:05
sdagueso, a *much better* approach is to instrument all the tempest runs, all the time14:06
sdaguethen you can pull out all the relevant times14:06
*** catohornet has quit IRC14:09
*** yfried has joined #openstack-qa14:11
*** mrodden has joined #openstack-qa14:12
andreykurilinsdague, I understand that total time consists of many components, not only discovery and test execution. Subunit results(including time) are parsed and recorded into database.14:15
sdagueandreykurilin: right, but I think you are missing the point that we *know* the subunit results today are inaccurate14:16
andreykurilinsdague, Why I want to split discovery? Simple example: some tests should be executed for one, two...ten active users. Discovery for all this tests are similar and can be launched separately, before all tests execution.14:16
sdagueand on some tests are < 10% of the test execution time14:16
sdagueso until that part is addressed, all the rest of this is going to remain wildly inaccurate14:16
sdaguewhich means it's kind of pointless. Hence why I'm trying to direct you towards solving the base problem first14:17
*** ashishsnigam has joined #openstack-qa14:20
sdagueif that was addressed, and rally was reading the subunit, we could, for instance, apply the rally analysis at the end of every gate run14:21
*** catohornet has joined #openstack-qa14:21
*** mdenny has joined #openstack-qa14:24
*** markmcclain has joined #openstack-qa14:25
andreykurilinsdague, I understand your opinion. Thanks for discussion. If new questions will appear, I will write you again.:)14:27
*** mwagner_lap has joined #openstack-qa14:31
*** yrabl has quit IRC14:34
*** kgriffs|afk is now known as kgriffs14:35
*** ndipanov has quit IRC14:36
*** bnemec has quit IRC14:39
*** tkammer_ has quit IRC14:40
*** andreaf has quit IRC14:41
*** bnemec has joined #openstack-qa14:42
*** markmcclain has quit IRC14:43
*** markmcclain has joined #openstack-qa14:44
openstackgerritAlexei Vinogradov proposed a change to openstack-dev/devstack: Add support of hbase database  https://review.openstack.org/8668314:45
*** samuelbercovici has joined #openstack-qa14:47
*** mmaglana has quit IRC14:51
*** mmaglana has joined #openstack-qa14:52
*** mmaglana has quit IRC14:56
dkranzsdague: Is there a precise definition of what is meant by "micro-versioning" somewhere? Search turned up nothing.14:59
sdaguego look at the Nova v2 vs. v3 threads14:59
dkranzsdague: k14:59
openstackgerritPavel Sedlák proposed a change to openstack/tempest: Stress runner/driver prints stats after SIGINT  https://review.openstack.org/9105914:59
openstackgerritPavel Sedlák proposed a change to openstack/tempest: Fix stress runner not stopping on first fail  https://review.openstack.org/9106014:59
*** ekhugen has joined #openstack-qa15:06
dkranzmtreinish: ping, re verify_config_script15:14
*** ashishsnigam has quit IRC15:14
dkranzmtreinish: I found it difficult to cleanly separate the "update" part from the "print messages" part15:15
dkranzmtreinish: What I have is a version where the discovery methods return the changes that need to be made.15:15
dkranzmtreinish: But control of whether print to the console happens is a little hacky15:16
dkranzmtreinish: I will push what I have as a WIP to get your feedback15:17
*** mlavalle has joined #openstack-qa15:27
*** mlavalle has quit IRC15:27
*** mlavalle has joined #openstack-qa15:30
*** TravT has joined #openstack-qa15:31
*** mmaglana has joined #openstack-qa15:40
*** afazekas_ has quit IRC15:47
*** Ch00k has quit IRC15:48
openstackgerritA change was merged to openstack/tempest: Verify delete aggregate attributes of Nova APIs  https://review.openstack.org/8947815:50
*** sweston has joined #openstack-qa15:50
*** andreaf has joined #openstack-qa15:55
*** coolsvap|afk is now known as coolsvap15:56
*** samuelbercovici has quit IRC15:57
*** qba73_ has quit IRC15:57
*** Obulpathi has quit IRC15:57
*** samuelbercovici has joined #openstack-qa15:58
openstackgerritA change was merged to openstack/tempest: Verify create/get flavor attributes of Nova APIs  https://review.openstack.org/8054316:01
openstackgerritA change was merged to openstack-dev/devstack: Only use heat image where support  https://review.openstack.org/9066016:02
*** Aimon has quit IRC16:03
*** markmcclain has quit IRC16:05
*** Obulpathi has joined #openstack-qa16:05
*** yassine has quit IRC16:07
openstackrecheckopenstack/keystone change: https://review.openstack.org/86998 failed because of: gate-tempest-dsvm-neutron: unrecognized error, gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1101147 https://bugs.launchpad.net/bugs/129587616:08
*** enikanorov__ has joined #openstack-qa16:12
*** enikanorov__ has quit IRC16:12
*** evgenyf has quit IRC16:12
*** enikanorov__ has joined #openstack-qa16:12
*** enikanorov has quit IRC16:12
*** gszasz has quit IRC16:12
*** jlibosva has quit IRC16:14
*** chandan_kumar has quit IRC16:19
*** markmcclain has joined #openstack-qa16:20
*** sweston has quit IRC16:24
*** salv-orlando_ has joined #openstack-qa16:29
*** mriedem has quit IRC16:29
*** samuelbercovici has quit IRC16:29
*** salv-orlando has quit IRC16:30
*** samuelbercovici1 has joined #openstack-qa16:30
*** salv-orlando_ is now known as salv-orlando16:30
*** samuelbercovici1 is now known as samuelbercovici16:30
*** mriedem has joined #openstack-qa16:30
*** safchain has quit IRC16:31
*** samuelbercovici has quit IRC16:32
*** markmcclain has quit IRC16:44
*** Aimon has joined #openstack-qa16:47
*** ampeg_ has quit IRC16:49
openstackrecheckopenstack/python-ceilometerclient change: https://review.openstack.org/75646 failed because of: gate-python-ceilometerclient-pypy: https://bugs.launchpad.net/bugs/1290562, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/127038216:50
*** atiwari has joined #openstack-qa16:52
*** weshay has quit IRC16:52
*** sthaha has quit IRC16:54
*** krtaylor has quit IRC16:56
*** harlowja_away is now known as harlowja16:59
*** Ch00k has joined #openstack-qa17:00
*** jlibosva has joined #openstack-qa17:00
openstackgerritEdgar Magana proposed a change to openstack/tempest: Add test for subnet gateway IPv4 and IPv6  https://review.openstack.org/6731217:08
*** mkoderer has quit IRC17:12
*** jpich has quit IRC17:16
*** morganfainberg_Z is now known as morganfainberg17:18
*** chandan_kumar has joined #openstack-qa17:20
*** Ch00k has quit IRC17:21
*** nati_ueno has joined #openstack-qa17:21
*** Ch00k has joined #openstack-qa17:21
*** Mikhail_D_ltp has joined #openstack-qa17:23
openstackgerritRob Pothier proposed a change to openstack/tempest: Add tempest test for heat resource AWS::EC2::EIP  https://review.openstack.org/8956417:24
*** weshay has joined #openstack-qa17:28
asselin__Hi, I'm trying to do a check bug here for multiple bugs, but it didn't kick off a new check. Is this related to the new gerrit or is my comment not correct? https://review.openstack.org/#/c/90449/17:31
*** openstackgerrit has quit IRC17:32
*** jordanP has quit IRC17:33
*** openstackgerrit has joined #openstack-qa17:33
*** Obulpath_ has joined #openstack-qa17:35
jlibosvasdague: dtroyer hi guys, I made another patch fixing grenade with neutron. I would be very happy if you can give any feedback: https://review.openstack.org/#/c/90785/17:38
*** Obulpathi has quit IRC17:39
*** chandan_kumar has quit IRC17:49
*** dkorn has joined #openstack-qa17:50
openstackgerritSalvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingeprint for bug 1314313  https://review.openstack.org/9111217:50
*** jlibosva has quit IRC17:58
openstackrecheckConsole logs not available after 14:18s for gate-nova-pep8 59365,30,c1e0df618:00
openstackrecheckopenstack/oslo.messaging change: https://review.openstack.org/90577 failed gate-tempest-dsvm-neutron in the gate queue with an unrecognized error18:01
*** jamielennox|away is now known as jamielennox18:03
*** jaypipes has joined #openstack-qa18:06
*** ChanServ sets mode: +o jaypipes18:06
*** jgrimm has quit IRC18:06
openstackgerritSalvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313794  https://review.openstack.org/9079818:20
openstackgerritChristian Schwede proposed a change to openstack/tempest: Tool to create a basic tempest.conf using Keystone credentials  https://review.openstack.org/9111618:21
*** jgrimm has joined #openstack-qa18:24
lifelesssdague: sounds like something I'd have said18:28
*** coolsvap is now known as coolsvap|afk18:30
*** yrabl has joined #openstack-qa18:35
sdaguelifeless: :) which part18:38
lifelesssubunit foo18:38
sdagueah, yeh18:39
sdagueI think we know it's a problem to be solved18:39
sdaguelifeless: do you have an example somewhere of subunit event injection?18:40
*** tosky has quit IRC18:41
lifelesssdague: the subunit-output CLI lets you generate a fairly arbitrary event from the commandline; doing arbitrary event injection from within the test running process - no18:41
lifelesssdague: We should get heads together in atlanta for 20m and beat this to death18:42
sdaguelifeless: sure18:42
sdagueyeh, I need to be able to inject from the tests18:42
*** Mikhail_D_ltp has left #openstack-qa18:43
openstackrecheckopenstack/taskflow change: https://review.openstack.org/90561 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1254890, gate-tempest-dsvm-neutron: unrecognized error18:58
*** rockyg has joined #openstack-qa19:01
*** morganfainberg is now known as morganfainberg_Z19:01
openstackgerritGabriel Assis Bezerra proposed a change to openstack-dev/devstack: Fixes keystone verification on APACHE_ENABLED_SERVICES variable  https://review.openstack.org/9063119:15
sdaguedkranz: per previous conversation on changes in the search syntax19:18
sdaguestatus:open NOT label:Code-Review>=0,self label:Verified>=1,jenkins NOT label:Code-Review<=-1 NOT label:Workflow<=-119:18
sdaguethat's a pretty good baseline for patches that you should comment on19:19
dkranzsdague: Thanks, that's great. Also a good template for the other stuff I had to change.19:24
sdagueyeh, I think we're going to try to turn that into a dashboard19:24
sdagueso you can get it on any project19:25
*** Obulpath_ has quit IRC19:26
*** Obulpathi has joined #openstack-qa19:26
*** Ch00k has quit IRC19:34
*** malini is now known as malini_afk19:42
*** nati_ueno has quit IRC19:42
*** yrabl has quit IRC19:42
*** morganfainberg_Z is now known as morganfainberg19:52
openstackrecheckopenstack/swift change: https://review.openstack.org/91037 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1295165 https://bugs.launchpad.net/bugs/1283522 https://bugs.launchpad.net/bugs/130925219:56
*** krtaylor has joined #openstack-qa19:58
*** rockyg has quit IRC20:00
*** nati_ueno has joined #openstack-qa20:09
*** Obulpathi has quit IRC20:10
*** nati_uen_ has joined #openstack-qa20:13
adam_gany devstack people around for a trivial? https://review.openstack.org/#/c/90821/20:14
lifelesssdague loves trivials :P20:16
*** nati_ueno has quit IRC20:17
sdague:)20:21
openstackgerritPeter Stachowski proposed a change to openstack/tempest: Added additional test to Trove (database) flavor API tests  https://review.openstack.org/8939620:22
*** Obulpathi has joined #openstack-qa20:22
*** wendar has quit IRC20:24
*** wendar has joined #openstack-qa20:24
openstackgerritPeter Stachowski proposed a change to openstack/tempest: Added Trove (database) version API tests  https://review.openstack.org/8941120:30
*** dkorn has quit IRC20:31
*** catohornet has quit IRC20:37
*** catohornet has joined #openstack-qa20:38
SlickNikHeads up: Just got done fixing our rdjenkins gerrit issues. Integration tests should now be running normally.20:41
*** adalbas has quit IRC20:53
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313851  https://review.openstack.org/9079420:56
*** catohornet has joined #openstack-qa20:56
*** openstackrecheck has quit IRC21:00
*** openstackrecheck has joined #openstack-qa21:00
*** aysyd has quit IRC21:03
*** andreaf has quit IRC21:06
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313800  https://review.openstack.org/9079721:10
*** openstackrecheck has quit IRC21:15
*** openstackrecheck has joined #openstack-qa21:15
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingerprint for bug 1313794  https://review.openstack.org/9079821:17
*** krtaylor has quit IRC21:18
*** morganfainberg is now known as morganfainberg_Z21:22
openstackrecheckopenstack/heat change: https://review.openstack.org/88499 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/129587621:23
*** openstackrecheck has quit IRC21:24
*** openstackrecheck has joined #openstack-qa21:24
*** morganfainberg_Z is now known as morganfainberg21:25
*** Obulpathi has quit IRC21:32
*** morganfainberg is now known as morganfainberg_Z21:33
*** morganfainberg_Z is now known as morganfainberg21:34
*** mwagner_lap has quit IRC21:39
jogosdague: things are not looking good today http://jogo.github.io/gate/21:46
sdaguejogo: have you identified the actual causes of fails?21:49
sdagueI honestly feel like we've been in a bad state for a few weeks, but that clean check means the gate backups aren't that bad21:50
sdagueso people aren't complaining or digging into the issues21:50
jogosdague: yeah21:50
jogolooks like there are a bunch of neutron issues21:50
jogojust approved a bunch of salv-orlando patches21:51
jogoto -er21:51
jogoe-r21:51
sdaguecool21:51
sdaguehonestly, the metadata server is in a bad state as well21:51
sdagueit stack traces all the time, but at info level, so our normal stuff doesn't pick it up21:51
sdaguehttp://logs.openstack.org/96/89396/3/check/check-tempest-dsvm-postgres-full/fb10799/logs/screen-n-api-meta.txt.gz#_2014-04-28_11_36_35_90221:52
jogooh strange21:53
jogowonder why it isn't getting picked up21:53
sdaguebecause it's stack tracing in eventlet outside of any place where there is a logger21:55
jogosdague: https://review.openstack.org/#/c/91112/21:55
jogois a big gate fail case21:55
sdagueok, I'll be honest, I had the grenade branch update bounce off the gate something like 5 times21:56
sdagueit was never neutron's fault21:56
*** weshay has quit IRC21:56
jogowell that was a big one for unclassified bugs21:57
jogoso https://bugs.launchpad.net/tempest/+bug/1295165 is still big21:57
*** dims has quit IRC21:57
sdaguewell it's mostly neutron grenade21:58
sdaguewhich is still dubious in passing21:58
jogo165 is happening outside of grenade a lot21:59
jogoor are you refering to something else21:59
openstackgerritA change was merged to openstack-infra/elastic-recheck: Add a fingeprint for bug 1314313  https://review.openstack.org/9111222:00
*** openstackrecheck has quit IRC22:02
*** openstackrecheck has joined #openstack-qa22:02
*** dosaboy has quit IRC22:03
*** dosaboy has joined #openstack-qa22:04
*** dkranz has quit IRC22:09
*** dims has joined #openstack-qa22:10
*** nati_uen_ has quit IRC22:12
*** ekhugen has quit IRC22:15
*** afazekas has quit IRC22:18
openstackrecheckopenstack/requirements change: https://review.openstack.org/89382 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1306029, gate-requirements-pypy: unrecognized error22:26
*** zigo_ has joined #openstack-qa22:35
*** mriedem has quit IRC22:37
*** zigo has quit IRC22:37
jogosdague: looks like 3 out of top 4 CI issues (all queues) are neutron related22:37
*** jgrimm has quit IRC22:37
*** nati_ueno has joined #openstack-qa22:37
*** mriedem has joined #openstack-qa22:37
*** nati_ueno is now known as 7JTAAM1ZL22:37
*** nati_ueno has joined #openstack-qa22:37
*** 7JTAAM1ZL has quit IRC22:38
*** mriedem has quit IRC22:43
*** mriedem has joined #openstack-qa22:57
*** mwagner_lap has joined #openstack-qa23:01
*** TravT has quit IRC23:03
*** TravT has joined #openstack-qa23:03
*** TravT has quit IRC23:04
*** TravT has joined #openstack-qa23:04
*** TravT has quit IRC23:04
*** TravT has joined #openstack-qa23:05
*** yamahata has quit IRC23:07
openstackrecheckConsole logs not available after 13:26s for gate-python-cinderclient-pep8 91138,1,2908c0223:24
sdagueok, good to know23:31
boris-42sdague we will start some mailing thread about it23:33
*** mrodden has quit IRC23:36
*** matsuhashi has joined #openstack-qa23:38
*** mlavalle has quit IRC23:40
openstackgerritJoe Gordon proposed a change to openstack/tempest: Skip lbaas test test_load_balancer_basic  https://review.openstack.org/9119423:44
openstackrecheckopenstack-dev/devstack change: https://review.openstack.org/86834 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error23:49
*** mrodden has joined #openstack-qa23:50
*** nati_uen_ has joined #openstack-qa23:53
*** kgriffs is now known as kgriffs|afk23:57
*** nati_ueno has quit IRC23:57

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!