*** yamahata has joined #openstack-qa | 00:00 | |
openstackrecheck | Console logs not available after 13:27s for gate-devstack-unit-tests 286084,4,7e9e312 | 00:02 |
---|---|---|
*** tmatsu has joined #openstack-qa | 00:04 | |
*** pglass has quit IRC | 00:10 | |
*** yamahata has quit IRC | 00:11 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make data_processing/baremetal use rest_client https://review.openstack.org/286914 | 00:11 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make compute keypairs_client use rest_client https://review.openstack.org/286915 | 00:11 |
openstackrecheck | Console logs not available after 13:37s for gate-tempest-dsvm-neutron-full 286672,1,71b3e91 | 00:16 |
*** mylu has quit IRC | 00:17 | |
*** zz_dimtruck is now known as dimtruck | 00:18 | |
openstackgerrit | Ghanshyam Mann proposed openstack/tempest: Add support of microversion in all compute service clients https://review.openstack.org/284414 | 00:19 |
*** mylu has joined #openstack-qa | 00:19 | |
*** mylu has quit IRC | 00:25 | |
openstackrecheck | Console logs not available after 13:32s for gate-tempest-dsvm-ceilometer-mysql-neutron-full 285602,6,38b6970 | 00:29 |
*** mylu has joined #openstack-qa | 00:32 | |
*** erlon has quit IRC | 00:36 | |
*** mylu has quit IRC | 00:37 | |
*** mylu has joined #openstack-qa | 00:41 | |
*** ajmiller has quit IRC | 00:43 | |
openstackrecheck | Console logs not available after 13:21s for gate-grenade-dsvm-neutron 286695,2,66886cb | 00:43 |
mriedem | mtreinish: i failed miserably http://logs.openstack.org/55/286855/1/experimental/gate-tempest-dsvm-neutron-dvr-multinode-full/8c4b90c/console.html#_2016-03-02_00_17_36_780 | 00:43 |
*** jlk has quit IRC | 00:46 | |
*** jlk has joined #openstack-qa | 00:46 | |
*** dimtruck is now known as zz_dimtruck | 00:46 | |
*** jlk has quit IRC | 00:46 | |
*** jlk has joined #openstack-qa | 00:46 | |
*** mylu has quit IRC | 00:47 | |
*** mylu has joined #openstack-qa | 00:47 | |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Use instance variable in ByteStreamToStreamResult init https://review.openstack.org/286925 | 00:47 |
*** Swami has quit IRC | 00:48 | |
mtreinish | mriedem: heh, it's close | 00:48 |
mtreinish | at least the code is being run :) | 00:49 |
*** rossella_s has quit IRC | 00:54 | |
*** rossella_s has joined #openstack-qa | 00:54 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-python27 253082,2,3d813cd | 00:56 |
*** luzC has quit IRC | 00:58 | |
openstackgerrit | Ghanshyam Mann proposed openstack/tempest: Add tests for compute v2.10 microversion https://review.openstack.org/277763 | 01:00 |
*** zz_dimtruck is now known as dimtruck | 01:02 | |
*** achanda has quit IRC | 01:03 | |
*** yangyapeng has joined #openstack-qa | 01:05 | |
openstackgerrit | Ghanshyam Mann proposed openstack/tempest: Add tests for compute v2.10 microversion https://review.openstack.org/277763 | 01:06 |
openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Remove unused flush from read_subunit() https://review.openstack.org/286928 | 01:09 |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python27 285469,5,8139ebc | 01:10 |
*** tochi has quit IRC | 01:15 | |
*** mylu has quit IRC | 01:19 | |
mriedem | mtreinish: probably safe to abandon this one https://review.openstack.org/#/c/157729/ | 01:23 |
mriedem | no updates since may of 2015 | 01:23 |
openstackrecheck | Console logs not available after 13:21s for gate-glance-pep8 256381,4,7c5cf74 | 01:23 |
*** cody-somerville_ has joined #openstack-qa | 01:23 | |
mtreinish | mriedem: yeah | 01:24 |
*** cody-somerville has quit IRC | 01:24 | |
*** jasonsb has joined #openstack-qa | 01:25 | |
mriedem | mtreinish: and https://review.openstack.org/#/c/164396/ | 01:25 |
mriedem | and this :) https://review.openstack.org/#/c/207621/ | 01:26 |
mriedem | just make me qa core so i can abandon all of these old patches | 01:26 |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-neutron-full 268712,3,be63a15 | 01:36 |
*** beisner has quit IRC | 01:38 | |
*** tochi has joined #openstack-qa | 01:39 | |
*** beisner has joined #openstack-qa | 01:40 | |
*** fawadkhaliq has quit IRC | 01:42 | |
*** yangyapeng has quit IRC | 01:43 | |
*** yamamoto has joined #openstack-qa | 01:45 | |
*** apoorvad has quit IRC | 01:48 | |
openstackrecheck | Console logs not available after 13:20s for gate-grenade-dsvm 276903,5,f8e66ae | 01:50 |
*** achanda has joined #openstack-qa | 01:51 | |
*** mtanino has quit IRC | 01:51 | |
*** jaypipes has quit IRC | 01:54 | |
*** pvaneck has quit IRC | 02:00 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-pep8 283171,8,808d823 | 02:03 |
*** woodster_ has quit IRC | 02:06 | |
*** rfolco_ has joined #openstack-qa | 02:08 | |
*** rfolco_ has quit IRC | 02:08 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-full 274885,12,a90a228 | 02:16 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-full 267827,11,306579f | 02:30 |
*** vhosakot has joined #openstack-qa | 02:30 | |
*** tochi has quit IRC | 02:32 | |
*** yamamoto has quit IRC | 02:36 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 262637,4,0130e15 | 02:43 |
*** mriedem has quit IRC | 02:46 | |
*** dimtruck is now known as zz_dimtruck | 02:48 | |
*** ngoracke has joined #openstack-qa | 02:51 | |
*** cody-somerville_ is now known as cody-somerville | 02:52 | |
*** cody-somerville has joined #openstack-qa | 02:52 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 273982,3,ade256d | 02:56 |
*** bkopilov has quit IRC | 02:57 | |
*** tochi has joined #openstack-qa | 02:58 | |
*** vhosakot has quit IRC | 03:01 | |
*** vhosakot has joined #openstack-qa | 03:02 | |
*** zz_dimtruck is now known as dimtruck | 03:08 | |
*** EinstCrazy has joined #openstack-qa | 03:09 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-python27 286744,1,db731d4 | 03:10 |
*** dims has quit IRC | 03:15 | |
*** dimtruck is now known as zz_dimtruck | 03:18 | |
*** weshay has quit IRC | 03:22 | |
*** yamamoto has joined #openstack-qa | 03:22 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 286745,1,aaa2b9c | 03:23 |
*** david-lyle has joined #openstack-qa | 03:24 | |
*** larainema has quit IRC | 03:27 | |
*** larainema has joined #openstack-qa | 03:29 | |
*** zz_dimtruck is now known as dimtruck | 03:33 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 03:34 | |
openstackgerrit | yalei wang proposed openstack-dev/devstack: Add pg_createcluster creating process for postgresql 9.3 https://review.openstack.org/286960 | 03:34 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make compute keypairs_client use rest_client https://review.openstack.org/286915 | 03:36 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make data_processing/baremetal use rest_client https://review.openstack.org/286914 | 03:36 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 281482,2,1b61268 | 03:37 |
*** fnaval has quit IRC | 03:39 | |
*** fawadkhaliq has joined #openstack-qa | 03:43 | |
*** dimtruck is now known as zz_dimtruck | 03:47 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 227036,41,93f572f | 03:50 |
*** fnaval has joined #openstack-qa | 03:53 | |
*** luzC has joined #openstack-qa | 03:56 | |
*** zz_dimtruck is now known as dimtruck | 04:01 | |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-full 285972,1,959b14a | 04:03 |
*** shausy has joined #openstack-qa | 04:06 | |
*** bkopilov has joined #openstack-qa | 04:13 | |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-full 275266,14,db1d7c6 | 04:17 |
*** johnsom__ has joined #openstack-qa | 04:22 | |
johnsom__ | Hi folks, question about the log archive from a gate run. Which repo has the code that archives the logs from a gate run? | 04:23 |
*** johnsom has joined #openstack-qa | 04:25 | |
johnsom__ | The octavia project boots service vms and I would like to explore the possibility of archiving the nova console logs from those service vms. | 04:26 |
*** ianw has joined #openstack-qa | 04:28 | |
openstackrecheck | Console logs not available after 13:20s for gate-nova-python27 284839,3,643aeeb | 04:30 |
*** Poornima has joined #openstack-qa | 04:31 | |
*** sridharg has joined #openstack-qa | 04:31 | |
*** cody-somerville has quit IRC | 04:33 | |
*** ngoracke has quit IRC | 04:34 | |
*** cody-somerville has joined #openstack-qa | 04:35 | |
openstackgerrit | Tim Buckley proposed openstack/openstack-health: Add CSS rules to improve table column sizes https://review.openstack.org/286979 | 04:37 |
*** oomichi_ has joined #openstack-qa | 04:40 | |
oomichi_ | masayukig: hi, can you take a look at https://review.openstack.org/#/c/286914/ again? | 04:41 |
openstackgerrit | Tim Buckley proposed openstack/openstack-health: Add CSS rules to improve table column sizes https://review.openstack.org/286979 | 04:41 |
*** yamahata has joined #openstack-qa | 04:41 | |
oomichi_ | that is a nice point, and gmann also pointed it before. | 04:41 |
*** luzC has quit IRC | 04:43 | |
oomichi_ | masayukig: so quick, thanks | 04:43 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-full 263927,9,4362be6 | 04:43 |
masayukig | oomichi_: yeah, thanks for clarification :) I feel it would be good to add a description about it in the commit message or somewhere. | 04:44 |
oomichi_ | masayukig: ok, will do it soon | 04:45 |
*** sree has joined #openstack-qa | 04:49 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make compute keypairs_client use rest_client https://review.openstack.org/286915 | 04:52 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make data_processing/baremetal use rest_client https://review.openstack.org/286914 | 04:52 |
oomichi_ | masayukig: done on ^^^ | 04:53 |
*** rossella_s has quit IRC | 04:53 | |
*** rossella_s has joined #openstack-qa | 04:54 | |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-neutron-full 284778,4,1abf5dc | 04:57 |
*** fawadkhaliq has quit IRC | 04:58 | |
*** dimtruck is now known as zz_dimtruck | 05:02 | |
openstackrecheck | Console logs not available after 13:21s for gate-keystone-pep8 285521,6,9fb716d | 05:10 |
gmann | oomichi_: I updated microversion patches please check | 05:11 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make object/orchestration clients use rest_client https://review.openstack.org/286985 | 05:13 |
oomichi_ | gmann: you have updated the commit msg? | 05:13 |
*** vhosakot has quit IRC | 05:14 | |
oomichi_ | gmann: https://review.openstack.org/#/c/277763/ ? | 05:14 |
gmann | oomichi_: commit msg? no | 05:15 |
oomichi_ | gmann: I got it now. but that still is red on jenkins | 05:16 |
gmann | oomichi_: ah just recheck | 05:16 |
gmann | oomichi_: you can check tomorrow :) | 05:16 |
gmann | oomichi_: what i told is true about your late night work there :) | 05:16 |
gmann | oomichi_: you still working :P | 05:16 |
*** amotoki has joined #openstack-qa | 05:16 | |
oomichi_ | gmann: with wine ;-) | 05:17 |
gmann | oomichi_: heh | 05:17 |
*** tmatsu has quit IRC | 05:17 | |
oomichi_ | gmann: don't we need to cap original keypair test with v2.9 ? | 05:17 |
oomichi_ | gmann: that is another point. | 05:18 |
*** salv-orl_ has joined #openstack-qa | 05:19 | |
gmann | oomichi_: you mean test_keypairs_v22 ? | 05:19 |
*** tmatsu has joined #openstack-qa | 05:19 | |
oomichi_ | gmann: when running mv tests with mv range v2.10+, the original test run against these mv also due to lack of cap. is it intentional? | 05:19 |
oomichi_ | gmann: yeah, right | 05:19 |
* gmann thinking | 05:20 | |
oomichi_ | gmann: and test_keypairs.py also | 05:21 |
gmann | oomichi_: I think we should run. cloud with 2.10+ means it has v2.2 and v2.1 version too | 05:21 |
gmann | oomichi_: in that case we should run those tests too | 05:21 |
oomichi_ | gmann: I am feeling I did put the same point before | 05:21 |
gmann | oomichi_: yea that was running the same tests for both microversion | 05:21 |
*** tmatsu has quit IRC | 05:22 | |
*** salv-orlando has quit IRC | 05:22 | |
gmann | oomichi_: in this case v2.10 tests is derived from base.BaseKeypairTest | 05:22 |
oomichi_ | gmann: because each test module inherits from base test class, not previous mv test class, right? | 05:22 |
gmann | oomichi_: which has no tests case, means no tests running twice | 05:22 |
gmann | oomichi_: yea right | 05:23 |
oomichi_ | gmann: ok, here is a point. | 05:23 |
gmann | oomichi_: if we inherit from previous mv tests then we should cap as we did for v2.2 | 05:23 |
oomichi_ | gmann: that depends on how to inherit/implement mv tests | 05:23 |
openstackrecheck | Console logs not available after 13:20s for gate-nova-pep8 265415,3,f64d83a | 05:23 |
openstackgerrit | Tim Buckley proposed openstack/openstack-health: Condense tables to improve visual consistency https://review.openstack.org/286986 | 05:24 |
oomichi_ | gmann: ok, and sorry for one more point. | 05:24 |
gmann | oomichi_: yea, based on mv tests implementation we need to decide the cap base class tests etc | 05:24 |
gmann | oomichi_: sure, it getting more clear and better :) | 05:24 |
oomichi_ | gmann: is it necessary to set "latest" on a new test class? | 05:24 |
oomichi_ | gmann: original test module doesn't specify it, and the default value of the class is the same | 05:25 |
oomichi_ | gmann: when I saw that, the above question came to me. | 05:25 |
gmann | oomichi_: ahh, nice point that is in very base class | 05:26 |
gmann | oomichi_: so we can skip that | 05:26 |
oomichi_ | gmann: can you update? | 05:27 |
gmann | oomichi_: sure, please -1 | 05:27 |
oomichi_ | gmann: +1 instead of +2 | 05:28 |
*** yfried_ has quit IRC | 05:28 | |
gmann | oomichi_: ok Thanks :) | 05:28 |
oomichi_ | gmann: that is not worth to put -1 ;-) | 05:28 |
*** tmatsu has joined #openstack-qa | 05:29 | |
openstackgerrit | Ghanshyam Mann proposed openstack/tempest: Add tests for compute v2.10 microversion https://review.openstack.org/277763 | 05:29 |
gmann | oomichi_: done | 05:29 |
oomichi_ | gmann: +2 | 05:30 |
gmann | oomichi_: Thanks. | 05:31 |
*** tmatsu has quit IRC | 05:31 | |
gmann | oomichi_: and can you check bases patches also but may be tomorrow not now, it too late for you :) | 05:32 |
*** sayalilunkad has quit IRC | 05:33 | |
oomichi_ | gmann: +2 again | 05:33 |
gmann | oomichi_: Thanks again :) | 05:34 |
*** achanda has quit IRC | 05:35 | |
*** sree_ has joined #openstack-qa | 05:35 | |
*** pcaruana has joined #openstack-qa | 05:36 | |
*** sree_ is now known as Guest49443 | 05:36 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-pep8 286772,1,9a91ece | 05:37 |
*** sree has quit IRC | 05:39 | |
*** achanda has joined #openstack-qa | 05:39 | |
*** achanda has quit IRC | 05:39 | |
*** tmatsu has joined #openstack-qa | 05:41 | |
*** tmatsu_ has joined #openstack-qa | 05:44 | |
*** sree has joined #openstack-qa | 05:45 | |
*** Guest49443 has quit IRC | 05:45 | |
*** sayalilunkad has joined #openstack-qa | 05:45 | |
*** tmatsu has quit IRC | 05:47 | |
*** pcaruana has quit IRC | 05:49 | |
*** sree has quit IRC | 05:50 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-pep8 285293,7,42371c6 | 05:50 |
*** sree has joined #openstack-qa | 05:51 | |
*** rcernin has joined #openstack-qa | 05:51 | |
*** johnsom__ has quit IRC | 05:51 | |
*** dave-mccowan has quit IRC | 05:51 | |
*** zz_dimtruck is now known as dimtruck | 05:53 | |
*** yangyapeng has joined #openstack-qa | 05:54 | |
openstackgerrit | Merged openstack-dev/devstack: Disable tempest tox venv recreation in OFFLINE mode https://review.openstack.org/285571 | 05:55 |
openstackgerrit | Merged openstack-dev/devstack: Add note about the periodic proposal job to the header https://review.openstack.org/286666 | 05:56 |
*** achanda has joined #openstack-qa | 06:00 | |
*** hoangcx has joined #openstack-qa | 06:01 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 285084,6,791f9c0 | 06:03 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Split out Neutron routers client https://review.openstack.org/260319 | 06:06 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make routers_client use **kwargs https://review.openstack.org/284412 | 06:06 |
*** sree has quit IRC | 06:11 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make object/orchestration clients use rest_client https://review.openstack.org/286985 | 06:11 |
oomichi_ | gmann: your turn ;-) https://review.openstack.org/#/c/286914/ and https://review.openstack.org/#/c/286915/3 | 06:12 |
*** binbincong has quit IRC | 06:13 | |
openstackrecheck | Console logs not available after 13:21s for experimental-tempest-dsvm-neutron-full-dibtest 274792,6,6dff40a | 06:17 |
*** dimtruck is now known as zz_dimtruck | 06:17 | |
gmann | oomichi_: i saw that, +A on both | 06:18 |
oomichi_ | gmann: thanks :) | 06:18 |
gmann | oomichi_: ll review https://review.openstack.org/286985 too after jenkin | 06:19 |
*** binbincong has joined #openstack-qa | 06:24 | |
*** gamado has joined #openstack-qa | 06:24 | |
*** sree has joined #openstack-qa | 06:25 | |
*** sree has joined #openstack-qa | 06:25 | |
*** kbaikov has quit IRC | 06:29 | |
openstackrecheck | Console logs not available after 13:20s for gate-cinder-python27 276981,13,b49a92a | 06:30 |
*** amotoki has quit IRC | 06:33 | |
*** amotoki has joined #openstack-qa | 06:34 | |
*** sridharg has quit IRC | 06:34 | |
*** armax has quit IRC | 06:35 | |
*** amotoki has quit IRC | 06:36 | |
*** krtaylor has quit IRC | 06:37 | |
*** jamielennox is now known as jamielennox|away | 06:38 | |
*** yfried_ has joined #openstack-qa | 06:38 | |
*** kbaikov has joined #openstack-qa | 06:42 | |
openstackrecheck | Console logs not available after 13:21s for experimental-tempest-dsvm-multinode-full-dibtest 240611,3,0825b71 | 06:44 |
*** hoangcx has quit IRC | 06:44 | |
*** ngoracke has joined #openstack-qa | 06:46 | |
openstackgerrit | OpenStack Proposal Bot proposed openstack-dev/devstack: Updated from generate-devstack-plugins-list https://review.openstack.org/286401 | 06:49 |
*** jamielennox|away is now known as jamielennox | 06:49 | |
*** hichihara has joined #openstack-qa | 06:49 | |
*** krtaylor has joined #openstack-qa | 06:49 | |
*** ngoracke has quit IRC | 06:50 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full-kilo 283769,5,d289d6c | 06:57 |
*** shardy has joined #openstack-qa | 07:00 | |
*** hoangcx has joined #openstack-qa | 07:02 | |
*** e0ne has joined #openstack-qa | 07:06 | |
openstackgerrit | Merged openstack/tempest: Properly handle failures during resource cleanup https://review.openstack.org/286112 | 07:07 |
*** jamielennox is now known as jamielennox|away | 07:08 | |
*** pilgrimstack1 has joined #openstack-qa | 07:09 | |
*** pilgrimstack has quit IRC | 07:10 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-releasenotes 274885,13,680293b | 07:10 |
*** Serlex has joined #openstack-qa | 07:13 | |
*** amotoki has joined #openstack-qa | 07:14 | |
*** oomichi_ has quit IRC | 07:16 | |
*** anshul has joined #openstack-qa | 07:18 | |
*** sridharg has joined #openstack-qa | 07:23 | |
openstackrecheck | Console logs not available after 13:21s for gate-glance-python34 269277,3,0032f68 | 07:24 |
*** amotoki has quit IRC | 07:35 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 284484,5,8e598c5 | 07:37 |
*** amotoki has joined #openstack-qa | 07:43 | |
*** sridhargaddam has joined #openstack-qa | 07:44 | |
*** sridharg has quit IRC | 07:44 | |
*** hoangcx has quit IRC | 07:46 | |
*** amotoki has quit IRC | 07:48 | |
openstackrecheck | Console logs not available after 13:21s for gate-devstack-gate-unittests 286136,2,f15aeb4 | 07:50 |
*** markvoelker has quit IRC | 07:58 | |
*** e0ne has quit IRC | 07:59 | |
*** neeti has joined #openstack-qa | 07:59 | |
*** sree_ has joined #openstack-qa | 08:00 | |
*** sree_ is now known as Guest54160 | 08:01 | |
*** pcaruana has joined #openstack-qa | 08:02 | |
*** achanda has quit IRC | 08:03 | |
*** sree has quit IRC | 08:03 | |
*** hichihara has quit IRC | 08:04 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 227036,42,d4af500 | 08:04 |
*** gfidente has joined #openstack-qa | 08:05 | |
*** gfidente has quit IRC | 08:05 | |
*** gfidente has joined #openstack-qa | 08:05 | |
*** achanda has joined #openstack-qa | 08:07 | |
*** zz_dimtruck is now known as dimtruck | 08:11 | |
openstackgerrit | Thomas Herve proposed openstack-infra/devstack-gate: Remove double timestamp from console logs https://review.openstack.org/286136 | 08:16 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-pep8 283770,7,2bc94dc | 08:17 |
*** achanda has quit IRC | 08:18 | |
*** jamielennox|away is now known as jamielennox | 08:25 | |
*** jlanoux has joined #openstack-qa | 08:26 | |
openstackrecheck | Console logs not available after 13:20s for gate-cinder-pep8 286795,2,d29d3c4 | 08:30 |
*** lxsli0 has joined #openstack-qa | 08:32 | |
*** tmatsu_ has quit IRC | 08:33 | |
*** yfried_ has quit IRC | 08:34 | |
*** Guest54160 has quit IRC | 08:37 | |
*** sree has joined #openstack-qa | 08:38 | |
*** lxsli0 has quit IRC | 08:38 | |
*** ifarkas has joined #openstack-qa | 08:40 | |
*** sree has quit IRC | 08:42 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 267827,12,bff2950 | 08:44 |
openstackgerrit | Yuriy Nesenenko proposed openstack/tempest: Fix test_get_service_by_host_name https://review.openstack.org/267765 | 08:45 |
*** ihrachys has joined #openstack-qa | 08:51 | |
*** hoangcx has joined #openstack-qa | 08:53 | |
*** rossella_s has quit IRC | 08:54 | |
*** dimtruck is now known as zz_dimtruck | 08:54 | |
*** yaume has joined #openstack-qa | 08:54 | |
*** rossella_s has joined #openstack-qa | 08:54 | |
*** speller has joined #openstack-qa | 08:57 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 286828,1,1fd7c47 | 08:57 |
*** apevec has joined #openstack-qa | 08:58 | |
*** markvoelker has joined #openstack-qa | 08:58 | |
*** apevec has left #openstack-qa | 09:03 | |
reedip | ajo: Ping | 09:04 |
*** yfried has joined #openstack-qa | 09:05 | |
*** lucas-dinner is now known as lucasagomes | 09:06 | |
*** hoangcx has quit IRC | 09:06 | |
openstackrecheck | Console logs not available after 13:21s for gate-grenade-dsvm 286833,2,dc74aaf | 09:10 |
*** jasonsb has quit IRC | 09:16 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-python34 260319,27,258206b | 09:24 |
*** e0ne has joined #openstack-qa | 09:28 | |
openstackgerrit | Merged openstack/eslint-config-openstack: Enable builtinGlobals for no-redeclare https://review.openstack.org/265887 | 09:33 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 284412,4,b0ed38c | 09:37 |
*** e0ne has quit IRC | 09:37 | |
*** Serlex has quit IRC | 09:39 | |
*** Serlex has joined #openstack-qa | 09:40 | |
*** jasonsb has joined #openstack-qa | 09:43 | |
*** tochi has quit IRC | 09:44 | |
*** reedip is now known as reedip_away | 09:46 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-dsvm-integration 268368,17,a88f690 | 09:51 |
*** _nadya_ has joined #openstack-qa | 09:58 | |
*** e0ne has joined #openstack-qa | 10:00 | |
*** kiran-r has joined #openstack-qa | 10:02 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 286271,1,579c106 | 10:04 |
kiran-r | Can somebody review this patch please? https://review.openstack.org/#/c/267452/ | 10:15 |
openstackrecheck | Console logs not available after 13:20s for gate-keystone-python27 286852,1,9cb94b1 | 10:17 |
*** amotoki has joined #openstack-qa | 10:18 | |
*** mvk has quit IRC | 10:20 | |
*** dtantsur|afk is now known as dtantsur | 10:25 | |
openstackrecheck | Console logs not available after 13:21s for experimental-tempest-dsvm-multinode-full-dibtest 286855,1,12ad62b | 10:31 |
*** gszasz has joined #openstack-qa | 10:31 | |
*** jordanP has joined #openstack-qa | 10:32 | |
*** mvk has joined #openstack-qa | 10:35 | |
*** doradus has joined #openstack-qa | 10:42 | |
*** doradus has quit IRC | 10:42 | |
*** doradus has joined #openstack-qa | 10:42 | |
*** _nadya__ has joined #openstack-qa | 10:43 | |
*** _nadya_ has quit IRC | 10:43 | |
openstackrecheck | Console logs not available after 13:21s for gate-glance-python27 280786,10,8da2005 | 10:44 |
*** tmatsu has joined #openstack-qa | 10:45 | |
*** ihrachys has quit IRC | 10:47 | |
*** tmatsu has quit IRC | 10:49 | |
*** yamahata has quit IRC | 10:51 | |
*** tosky has joined #openstack-qa | 10:51 | |
*** dims_ has joined #openstack-qa | 10:51 | |
*** hichihara has joined #openstack-qa | 10:54 | |
*** binbincong has quit IRC | 10:57 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 284627,6,85c1ba3 | 10:57 |
*** Abhishek has joined #openstack-qa | 10:58 | |
*** cdent has joined #openstack-qa | 10:59 | |
Abhishek | can anyone pls review : https://review.openstack.org/#/c/267452/ | 11:01 |
*** sdague has joined #openstack-qa | 11:02 | |
openstackgerrit | Merged openstack/tempest: Add release notes for upcoming release https://review.openstack.org/286695 | 11:05 |
*** tosky has quit IRC | 11:05 | |
*** _nadya__ has quit IRC | 11:10 | |
*** binbincong has joined #openstack-qa | 11:10 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 227036,43,8242371 | 11:11 |
*** tosky has joined #openstack-qa | 11:11 | |
*** salv-orlando has joined #openstack-qa | 11:19 | |
*** salv-orl_ has quit IRC | 11:22 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python27 284395,2,4dafa61 | 11:24 |
*** EinstCrazy has quit IRC | 11:24 | |
*** yangyapeng has quit IRC | 11:30 | |
*** Abhishek has quit IRC | 11:35 | |
*** Abhishek has joined #openstack-qa | 11:35 | |
*** Abhishek has quit IRC | 11:36 | |
*** Abhishek has joined #openstack-qa | 11:36 | |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-neutron-full 285901,2,464c8f3 | 11:37 |
*** Abhishek__ has joined #openstack-qa | 11:37 | |
*** yfried has quit IRC | 11:39 | |
*** yfried_ has joined #openstack-qa | 11:39 | |
*** _nadya_ has joined #openstack-qa | 11:40 | |
*** Abhishek has quit IRC | 11:41 | |
*** yamamoto has quit IRC | 11:41 | |
*** _nadya_ has quit IRC | 11:45 | |
*** Abhishek__ has quit IRC | 11:47 | |
openstackrecheck | Console logs not available after 13:23s for gate-tempest-dsvm-full-liberty 270564,3,9af801e | 11:51 |
*** ihrachys has joined #openstack-qa | 11:52 | |
*** dave-mccowan has joined #openstack-qa | 11:53 | |
*** _nadya_ has joined #openstack-qa | 11:55 | |
snikitin | andreaf: Hi, Andrea! Have you been satisfied by results of experimental jobs here? https://review.openstack.org/#/c/284621/ | 12:03 |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 209680,10,18f9da9 | 12:04 |
*** aysyd has joined #openstack-qa | 12:05 | |
andreaf | snikitin: yes I was mostly interested in the v3 only job which passed | 12:05 |
*** cdent has quit IRC | 12:06 | |
andreaf | snikitin: +A | 12:08 |
snikitin | andreaf: great! thank you | 12:08 |
*** rodrigods has quit IRC | 12:12 | |
*** rodrigods has joined #openstack-qa | 12:12 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-full 286889,1,b3e8770 | 12:18 |
*** Poornima has quit IRC | 12:20 | |
*** kiran-r has quit IRC | 12:21 | |
*** yamamoto has joined #openstack-qa | 12:24 | |
*** yamamoto_ has joined #openstack-qa | 12:25 | |
*** erlon has joined #openstack-qa | 12:25 | |
*** Liuqing has joined #openstack-qa | 12:27 | |
*** yamamoto has quit IRC | 12:29 | |
*** kiran-r has joined #openstack-qa | 12:29 | |
*** raildo-afk is now known as raildo | 12:31 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 277670,4,450e808 | 12:31 |
*** bkopilov has quit IRC | 12:31 | |
*** hichihara has quit IRC | 12:39 | |
*** yamamoto_ has quit IRC | 12:43 | |
*** rfolco has joined #openstack-qa | 12:44 | |
openstackrecheck | Console logs not available after 13:22s for gate-cinder-python27 286256,7,76722b0 | 12:44 |
*** edmondsw has joined #openstack-qa | 12:46 | |
*** markvoelker has quit IRC | 12:48 | |
*** markvoelker has joined #openstack-qa | 12:49 | |
*** lucasagomes is now known as lucas-hungry | 12:51 | |
*** rossella_s has quit IRC | 12:53 | |
*** rossella_s has joined #openstack-qa | 12:54 | |
*** fawadkhaliq has joined #openstack-qa | 12:54 | |
*** yamamoto has joined #openstack-qa | 12:55 | |
openstackrecheck | Console logs not available after 13:20s for gate-horizon-npm-run-lint 286918,2,99778de | 12:58 |
*** _nadya_ has quit IRC | 13:01 | |
*** yaume_ has joined #openstack-qa | 13:03 | |
*** yaume has quit IRC | 13:04 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 227036,44,f42987c | 13:11 |
*** yaume_ has quit IRC | 13:12 | |
*** yaume has joined #openstack-qa | 13:13 | |
*** dkranz has joined #openstack-qa | 13:15 | |
*** yangyapeng has joined #openstack-qa | 13:19 | |
*** fawadkhaliq has quit IRC | 13:21 | |
*** yangyapeng has quit IRC | 13:22 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-python34 286915,1,6d62d2a | 13:24 |
*** ngoracke has joined #openstack-qa | 13:25 | |
*** yaume has quit IRC | 13:27 | |
*** yaume has joined #openstack-qa | 13:27 | |
*** yamamoto has quit IRC | 13:28 | |
openstackgerrit | Sean Dague proposed openstack/tempest: Add LOG.info to boot from volume tests https://review.openstack.org/286634 | 13:29 |
*** ngoracke has quit IRC | 13:30 | |
*** pleia2 has quit IRC | 13:33 | |
*** yaume has quit IRC | 13:33 | |
*** yfried has joined #openstack-qa | 13:34 | |
*** yaume has joined #openstack-qa | 13:34 | |
*** yfried_ has quit IRC | 13:38 | |
sdague | andreaf / mtreinish / jordanP - https://review.openstack.org/#/c/286634/ - some additional logging in the boot from volume tests to try to make understanding the failures easier | 13:38 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 286908,1,87b45a7 | 13:38 |
jordanP | sdague, this won"t help a lot but +2 anyway. Problem is, as you said in your email, on teardown | 13:39 |
*** yamamoto has joined #openstack-qa | 13:39 | |
*** yamamoto has quit IRC | 13:39 | |
jordanP | it looks like a deadlock when the volume and the instance need to be deleted. | 13:40 |
*** yaume has quit IRC | 13:40 | |
sdague | jordanP: it's not always on teardown | 13:41 |
*** yaume has joined #openstack-qa | 13:41 | |
jordanP | ah ok then | 13:41 |
sdague | but, yes, there is *also* a teardown issue | 13:41 |
jordanP | I'll try to spend an hour on doing the cleanup explicitely in the test itself, it should help to understand what's going on in that case, hopefully | 13:42 |
*** yangyapeng has joined #openstack-qa | 13:48 | |
*** yaume has quit IRC | 13:50 | |
*** bkopilov has joined #openstack-qa | 13:51 | |
*** yaume has joined #openstack-qa | 13:51 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-python34 286914,1,e6dadea | 13:51 |
*** _nadya_ has joined #openstack-qa | 13:51 | |
*** yamamoto has joined #openstack-qa | 13:54 | |
*** yaume has quit IRC | 13:55 | |
*** yaume has joined #openstack-qa | 13:56 | |
*** Abhishek has joined #openstack-qa | 13:56 | |
*** Abhishek has quit IRC | 13:56 | |
*** sree has joined #openstack-qa | 13:59 | |
*** yaume has quit IRC | 14:00 | |
*** yaume has joined #openstack-qa | 14:01 | |
*** yamamoto has quit IRC | 14:02 | |
*** josdotso has quit IRC | 14:02 | |
*** sree has quit IRC | 14:02 | |
*** sree has joined #openstack-qa | 14:02 | |
*** lucas-hungry is now known as lucasagomes | 14:03 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 284635,2,56c3c5b | 14:04 |
*** bapalm has quit IRC | 14:05 | |
*** yaume has quit IRC | 14:05 | |
*** yaume has joined #openstack-qa | 14:06 | |
*** jbernard_ has joined #openstack-qa | 14:06 | |
*** bapalm has joined #openstack-qa | 14:06 | |
*** sree has quit IRC | 14:07 | |
*** weshay has joined #openstack-qa | 14:07 | |
*** yamamoto has joined #openstack-qa | 14:07 | |
openstackgerrit | Trevor McCasland proposed openstack-dev/devstack: Add error checking for euch.sh volume creation https://review.openstack.org/285464 | 14:10 |
*** yaume has quit IRC | 14:12 | |
*** yaume has joined #openstack-qa | 14:12 | |
*** rodrigods has quit IRC | 14:14 | |
*** rodrigods has joined #openstack-qa | 14:14 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-npm-run-test 285084,9,d6a659d | 14:18 |
*** jbernard has left #openstack-qa | 14:20 | |
*** yamamoto has quit IRC | 14:23 | |
*** yaume has quit IRC | 14:25 | |
*** Poornima has joined #openstack-qa | 14:25 | |
*** yaume has joined #openstack-qa | 14:25 | |
*** jecarey has quit IRC | 14:30 | |
*** Ravikiran_K has joined #openstack-qa | 14:31 | |
*** mriedem has joined #openstack-qa | 14:31 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full-liberty 284414,6,edfaac2 | 14:31 |
*** neeti_ has joined #openstack-qa | 14:32 | |
*** anshul has quit IRC | 14:32 | |
*** neeti has quit IRC | 14:33 | |
*** mylu has joined #openstack-qa | 14:35 | |
*** shausy has quit IRC | 14:36 | |
*** Swami has joined #openstack-qa | 14:38 | |
yfried | gmann: ping | 14:40 |
openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for volume-backed live migration abort bug 1524898 https://review.openstack.org/286725 | 14:41 |
openstack | bug 1524898 in OpenStack Compute (nova) "Volume based live migration aborted unexpectedly" [High,Confirmed] https://launchpad.net/bugs/1524898 | 14:42 |
*** anshul has joined #openstack-qa | 14:44 | |
*** Liuqing has quit IRC | 14:44 | |
*** yamamoto has joined #openstack-qa | 14:44 | |
openstackrecheck | Console logs not available after 13:21s for gate-glance-docs 280786,11,478bbe1 | 14:45 |
*** yaume_ has joined #openstack-qa | 14:45 | |
*** mylu has quit IRC | 14:46 | |
*** Swami_ has joined #openstack-qa | 14:46 | |
openstackgerrit | Merged openstack-infra/subunit2sql: Use instance variable in ByteStreamToStreamResult init https://review.openstack.org/286925 | 14:46 |
openstackgerrit | Merged openstack-infra/subunit2sql: Remove unused flush from read_subunit() https://review.openstack.org/286928 | 14:47 |
*** yaume has quit IRC | 14:49 | |
openstackgerrit | Merged openstack-infra/subunit2sql: Add API methods for getting tests by prefix https://review.openstack.org/283334 | 14:49 |
openstackgerrit | Yair Fried proposed openstack/tempest: Security Groups multi-node scenario https://review.openstack.org/259746 | 14:50 |
*** Swami has quit IRC | 14:50 | |
*** yangyapeng has quit IRC | 14:51 | |
*** yangyapeng has joined #openstack-qa | 14:52 | |
*** openstackrecheck has quit IRC | 14:54 | |
*** openstackrecheck has joined #openstack-qa | 14:54 | |
*** rbak has joined #openstack-qa | 14:55 | |
openstackgerrit | Merged openstack/tempest: Make data_processing/baremetal use rest_client https://review.openstack.org/286914 | 14:55 |
*** sshnaidm_ has joined #openstack-qa | 14:56 | |
*** yangyape_ has joined #openstack-qa | 14:56 | |
*** sshnaidm has quit IRC | 14:56 | |
*** sshnaidm_ has quit IRC | 14:56 | |
*** jecarey has joined #openstack-qa | 14:57 | |
*** yangyapeng has quit IRC | 14:57 | |
*** doradus has quit IRC | 14:58 | |
*** tosky has quit IRC | 15:00 | |
*** tosky has joined #openstack-qa | 15:01 | |
*** sridhargaddam has quit IRC | 15:03 | |
*** sshnaidm has joined #openstack-qa | 15:03 | |
*** zz_dimtruck is now known as dimtruck | 15:08 | |
*** sigmavirus24_awa is now known as sigmavirus24 | 15:08 | |
*** mtanino has joined #openstack-qa | 15:09 | |
*** tosky has quit IRC | 15:10 | |
*** tosky has joined #openstack-qa | 15:15 | |
*** austin81 has joined #openstack-qa | 15:16 | |
openstackrecheck | Console logs not available after 13:21s for gate-glance-docs 278157,4,5391edb | 15:17 |
*** yamamoto has quit IRC | 15:17 | |
*** jaypipes has joined #openstack-qa | 15:17 | |
*** ChanServ sets mode: +o jaypipes | 15:17 | |
*** mylu has joined #openstack-qa | 15:18 | |
*** yamamoto has joined #openstack-qa | 15:18 | |
*** ajmiller has joined #openstack-qa | 15:18 | |
*** yamamoto has quit IRC | 15:18 | |
*** yangyape_ has quit IRC | 15:19 | |
*** yangyapeng has joined #openstack-qa | 15:19 | |
*** yamamoto has joined #openstack-qa | 15:21 | |
*** sridharg has joined #openstack-qa | 15:23 | |
*** yangyape_ has joined #openstack-qa | 15:26 | |
*** woodster_ has joined #openstack-qa | 15:27 | |
*** sridharg has quit IRC | 15:28 | |
*** yangyape_ has quit IRC | 15:29 | |
*** yangyapeng has quit IRC | 15:30 | |
openstackrecheck | Console logs not available after 13:20s for gate-keystone-python27 285521,8,3efdef6 | 15:30 |
*** gamado has quit IRC | 15:30 | |
*** jecarey has quit IRC | 15:31 | |
*** jecarey has joined #openstack-qa | 15:31 | |
*** anshul has quit IRC | 15:36 | |
*** neeti_ has quit IRC | 15:36 | |
*** yamahata has joined #openstack-qa | 15:39 | |
*** ngoracke has joined #openstack-qa | 15:42 | |
*** yamamoto has quit IRC | 15:42 | |
*** yamamoto has joined #openstack-qa | 15:43 | |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-full-liberty 286634,2,cf8238f | 15:43 |
*** yamamoto has quit IRC | 15:43 | |
*** yamamoto has joined #openstack-qa | 15:44 | |
*** ngoracke has quit IRC | 15:47 | |
*** yfried has quit IRC | 15:49 | |
*** jlwhite has joined #openstack-qa | 15:49 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-full 287226,1,b950416 | 15:57 |
openstackgerrit | Merged openstack/eslint-config-openstack: Documented project governance https://review.openstack.org/265918 | 15:59 |
*** doradus has joined #openstack-qa | 16:03 | |
*** pglass has joined #openstack-qa | 16:03 | |
*** vhosakot has joined #openstack-qa | 16:05 | |
*** pcaruana has quit IRC | 16:05 | |
*** fawadkhaliq has joined #openstack-qa | 16:06 | |
*** Serlex has quit IRC | 16:06 | |
*** jaypipes has quit IRC | 16:08 | |
*** _nadya_ has quit IRC | 16:09 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 286915,3,3070278 | 16:10 |
*** armax has joined #openstack-qa | 16:11 | |
*** doradus has quit IRC | 16:14 | |
*** fawadkhaliq has quit IRC | 16:14 | |
*** bauzas_ has joined #openstack-qa | 16:14 | |
*** bauzas has quit IRC | 16:15 | |
*** johnthetubaguy has quit IRC | 16:15 | |
*** bauzas_ is now known as bauzas | 16:15 | |
*** johnthetubaguy has joined #openstack-qa | 16:15 | |
*** jplopezgu_ has joined #openstack-qa | 16:17 | |
*** jplopezgu_ is now known as jlopezgu | 16:18 | |
openstackgerrit | Al Miller proposed openstack-dev/devstack: Remove ~/.config/openstack/clouds.yml in clean.sh https://review.openstack.org/286898 | 16:20 |
*** luzC has joined #openstack-qa | 16:20 | |
*** jaypipes has joined #openstack-qa | 16:22 | |
*** ChanServ sets mode: +o jaypipes | 16:22 | |
*** Swami__ has joined #openstack-qa | 16:22 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python34 286601,2,c03a43d | 16:23 |
*** aalam has joined #openstack-qa | 16:25 | |
openstackgerrit | James E. Blair proposed openstack-infra/subunit2sql: Don't extend the passed in targets https://review.openstack.org/287299 | 16:26 |
*** Swami_ has quit IRC | 16:26 | |
sdague | mtreinish: is there any easy switch to flip to make the teardown non fatal? | 16:27 |
sdague | because that would at least mitigate the gate a bit | 16:27 |
*** mylu has quit IRC | 16:28 | |
mtreinish | sdague: no we don't have anything like that right now, we'd have to add one | 16:30 |
sdague | I thought i saw something about teardown fatal | 16:30 |
sdague | in a config somewhere | 16:30 |
*** mylu has joined #openstack-qa | 16:32 | |
mtreinish | I don | 16:32 |
mtreinish | 't know of anything like that | 16:32 |
*** mylu has quit IRC | 16:33 | |
*** doradus has joined #openstack-qa | 16:33 | |
openstackgerrit | Thomas Maddox proposed openstack-dev/devstack: Don't waste time zero'ing out local block image lvm volumes with nova https://review.openstack.org/215929 | 16:34 |
*** david-lyle has quit IRC | 16:36 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python34 286600,2,aa44fa6 | 16:37 |
mtreinish | sdague: unrelated: https://pypi.python.org/pypi/Tempest/ | 16:37 |
*** david-lyle has joined #openstack-qa | 16:38 | |
*** doradus has quit IRC | 16:38 | |
*** yamahata has quit IRC | 16:39 | |
*** jecarey_ has joined #openstack-qa | 16:39 | |
*** apoorvad has joined #openstack-qa | 16:40 | |
*** yfried has joined #openstack-qa | 16:41 | |
*** regXboi has joined #openstack-qa | 16:41 | |
*** vhosakot has quit IRC | 16:41 | |
*** krtaylor has quit IRC | 16:41 | |
openstackgerrit | Matthew Treinish proposed openstack/qa-specs: Move reintegrate-tempest-lib spec to implemented https://review.openstack.org/287308 | 16:41 |
*** jecarey has quit IRC | 16:42 | |
*** yamamoto has quit IRC | 16:42 | |
*** mylu has joined #openstack-qa | 16:44 | |
*** mylu has quit IRC | 16:45 | |
sdague | mtreinish: ah, it's a stress test option - https://github.com/openstack/tempest/blob/52abbd93a2ea9d4517538e3ec0d18f87c166602d/tempest/config.py#L986 | 16:45 |
*** sc68cal has joined #openstack-qa | 16:46 | |
openstackgerrit | James E. Blair proposed openstack-infra/subunit2sql: Don't extend the passed in targets https://review.openstack.org/287299 | 16:47 |
openstackgerrit | James E. Blair proposed openstack-infra/subunit2sql: Don't extend the passed in targets https://review.openstack.org/287299 | 16:47 |
sdague | jordanP: did you get around to unwinding the delete into the main method there? | 16:53 |
jordanP | nope, I didn't start. I was planning to spend an hour on it | 16:53 |
*** rossella_s has quit IRC | 16:53 | |
jordanP | but had to do something else | 16:54 |
*** rossella_s has joined #openstack-qa | 16:54 | |
*** ttx has quit IRC | 16:54 | |
*** ttx has joined #openstack-qa | 16:56 | |
sdague | ok, I wonder if we can trigger _run_cleanups directly | 16:57 |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python34 286677,1,cd40e57 | 16:57 |
mtreinish | sdague: https://docs.python.org/2/library/unittest.html#unittest.TestCase.doCleanups | 16:58 |
*** _nadya_ has joined #openstack-qa | 17:00 | |
*** Poornima has quit IRC | 17:00 | |
*** yfried has quit IRC | 17:00 | |
*** gema has quit IRC | 17:01 | |
*** speller has quit IRC | 17:01 | |
*** gema has joined #openstack-qa | 17:02 | |
sdague | mtreinish: does testtools honor that? | 17:03 |
*** luzC has quit IRC | 17:03 | |
sdague | I guess it's got inheritance | 17:03 |
sdague | so maybe | 17:03 |
*** luzC has joined #openstack-qa | 17:04 | |
mtreinish | sdague: as far as I know. Testtools should just be an extension on unittest so it shouldn't break anything in the base stdlib | 17:04 |
mtreinish | if it does that's a bug | 17:04 |
sdague | well, there is definitely an alternative path here | 17:04 |
sdague | however, I think this will bypass that | 17:04 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Split out Neutron routers client https://review.openstack.org/260319 | 17:06 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make routers_client use **kwargs https://review.openstack.org/284412 | 17:06 |
openstackgerrit | Sean Dague proposed openstack/tempest: Force cleanups into the main test method for boot from volume https://review.openstack.org/287323 | 17:07 |
sdague | also, liberty failed across the board on the other one | 17:07 |
sdague | I kind of wonder how the timing and failure graphs will look if we account teardown | 17:07 |
*** luzC has quit IRC | 17:08 | |
*** luzC has joined #openstack-qa | 17:08 | |
openstackgerrit | Merged openstack/tempest: Use isinstance instead of type https://review.openstack.org/284773 | 17:08 |
*** vhosakot has joined #openstack-qa | 17:09 | |
*** yamahata has joined #openstack-qa | 17:09 | |
mtreinish | sdague: you can use stackviz to visualize that: https://stackviz.timothyb89.org/s/74c0523f-2e13-44a0-abc0-4c8a8c99d7c7/#/0/timeline | 17:09 |
*** jlwhite_ has joined #openstack-qa | 17:10 | |
*** jlwhite has quit IRC | 17:10 | |
mtreinish | the setupclass for volume boot pattern looks like it's pretty quick | 17:10 |
sdague | but what about the teardown | 17:11 |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-full 285521,9,4509763 | 17:11 |
*** e0ne has quit IRC | 17:11 | |
sdague | because I think the issue is that teardown takes minutes | 17:11 |
sdague | so we're getting a timeout in teardown | 17:12 |
sdague | even though the test looks like it completed in less than the timeout | 17:12 |
*** pleia2 has joined #openstack-qa | 17:12 | |
*** Swami__ has quit IRC | 17:12 | |
sdague | nothing actually failed in teardown in these logs from what I can see, it's just progressing slowly | 17:12 |
*** achanda has joined #openstack-qa | 17:12 | |
mtreinish | it doesn't get added to the subunit stream and since volume boot pattern is last on both of it's workers you can't cheat and look at the gaps | 17:13 |
sdague | right | 17:13 |
sdague | the _run_cleanups never get added? | 17:13 |
sdague | it's not a real tearDown | 17:13 |
*** yangyapeng has joined #openstack-qa | 17:14 | |
*** grafuls has quit IRC | 17:14 | |
*** mylu has joined #openstack-qa | 17:14 | |
mtreinish | so the class level fixtures only get added on failures, I don't think addcleanup or the per method setups ever end up in the stream as seperate ids (since they're tied to an id already) | 17:16 |
*** dtantsur is now known as dtantsur|afk | 17:17 | |
*** rfolco has quit IRC | 17:17 | |
*** jlanoux has quit IRC | 17:18 | |
*** mylu has quit IRC | 17:19 | |
*** salv-orl_ has joined #openstack-qa | 17:19 | |
*** yangyapeng has quit IRC | 17:19 | |
sdague | mtreinish: ok, well forcing doCleanup in the main process should at least make this more clear. | 17:19 |
*** jlwhite_ has quit IRC | 17:21 | |
*** fawadkhaliq has joined #openstack-qa | 17:21 | |
jordanP | the cleanup part of this test is a bit complicated. You have to delete in that order : a server, a volume, a volume snapshort then a server and then a volume. And wait for resource deletion in the middle of each step | 17:21 |
*** salv-orlando has quit IRC | 17:22 | |
*** jaypipes has quit IRC | 17:22 | |
*** yfried has joined #openstack-qa | 17:23 | |
*** mriedem has left #openstack-qa | 17:24 | |
*** mriedem has joined #openstack-qa | 17:24 | |
*** krtaylor has joined #openstack-qa | 17:24 | |
*** achanda has quit IRC | 17:27 | |
*** kiran-r has quit IRC | 17:28 | |
openstackgerrit | Jordan Pittier proposed openstack/tempest: Scenario/manager.py: always wait when delete a cinder vol https://review.openstack.org/287336 | 17:28 |
openstackgerrit | Ihar Hrachyshka proposed openstack-dev/devstack: worlddump: request Guru Mediation reports for neutron agents https://review.openstack.org/279035 | 17:29 |
sdague | jordanP: right, that's why I think cheating is fine and just running doCleanups | 17:29 |
*** vhosakot has quit IRC | 17:29 | |
*** jasonsb has quit IRC | 17:30 | |
*** jlwhite has joined #openstack-qa | 17:30 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-pep8 287252,1,3aa15b0 | 17:30 |
*** cdent has joined #openstack-qa | 17:30 | |
*** vhosakot has joined #openstack-qa | 17:30 | |
jordanP | sdague, at the end of the test ? L143 ? | 17:31 |
*** rbak has quit IRC | 17:31 | |
jordanP | I am not sure what this will buy us. More logging ? | 17:31 |
jordanP | ah I didn't see your patch, okay then | 17:32 |
*** jasonsb has joined #openstack-qa | 17:32 | |
sdague | well, I think one of the things is the test fail rate in all the analysis tools is off because of where this processes | 17:32 |
sdague | as is the actual duration of this test | 17:32 |
sdague | subunit2sql, for instance, doesn't count a fail in _run_cleanups as a fail | 17:33 |
mtreinish | sdague: if it fails during _run_cleanups I'm pretty sure that's fatal to the test | 17:33 |
sdague | from what i can tell | 17:33 |
mtreinish | and subunit2sql will count it | 17:33 |
sdague | mtreinish: well, it reports it differently | 17:33 |
*** Swami has joined #openstack-qa | 17:33 | |
mtreinish | does it? | 17:33 |
*** rbak has joined #openstack-qa | 17:33 | |
sdague | http://status.openstack.org//openstack-health/#/ | 17:33 |
jordanP | yes, unittest doc says if the test fail in a cleanup stage, it's marked as fail. I don't know about subunit thoug | 17:34 |
mtreinish | sdague: tearDownClass is different from _run_cleanups | 17:34 |
mtreinish | _run_cleanups failures will work like you want | 17:34 |
sdague | mtreinish: then what is going on there? | 17:34 |
mtreinish | tearDownClass doesn't | 17:34 |
mtreinish | the thing that isn't accounted for with _run_cleanups is the timing | 17:35 |
mtreinish | which is arguably a subunit bug, but others might disagree with me there... | 17:35 |
sdague | I also think our timings is way off. We see all the success runs of 2.5minutes, and think our timeouts are fine | 17:35 |
mtreinish | (well more realistically the fix would be in testtools) | 17:35 |
*** boris-42 has joined #openstack-qa | 17:35 | |
sdague | but don't realize it's really more like 4.5minutes and we don't have the headroom | 17:35 |
*** jasonsb has quit IRC | 17:36 | |
*** jaypipes has joined #openstack-qa | 17:36 | |
*** ChanServ sets mode: +o jaypipes | 17:36 | |
*** Ravikiran_K has left #openstack-qa | 17:36 | |
*** doradus has joined #openstack-qa | 17:39 | |
*** jordanP has quit IRC | 17:42 | |
*** yamamoto has joined #openstack-qa | 17:43 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-python27 287253,1,0c269d3 | 17:43 |
*** achanda has joined #openstack-qa | 17:45 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make database/telemetry clients use rest_client https://review.openstack.org/287343 | 17:47 |
*** jaypipes has quit IRC | 17:47 | |
*** tosky_ has joined #openstack-qa | 17:47 | |
*** tosky has quit IRC | 17:48 | |
*** aalam has quit IRC | 17:50 | |
*** yamamoto has quit IRC | 17:50 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make database/telemetry clients use rest_client https://review.openstack.org/287343 | 17:51 |
*** aalam has joined #openstack-qa | 17:52 | |
*** rfolco has joined #openstack-qa | 17:52 | |
*** _nadya_ has quit IRC | 17:53 | |
openstackrecheck | Console logs not available after 13:20s for gate-horizon-npm-run-test 236042,51,d8b12e9 | 17:57 |
*** rbak_ has joined #openstack-qa | 17:57 | |
sdague | huh, there is a 4th instance somewhere? | 17:58 |
*** rbak has quit IRC | 17:59 | |
*** pcrews__ has joined #openstack-qa | 17:59 | |
*** tosky_ is now known as tosky | 17:59 | |
*** pcrews_ has quit IRC | 17:59 | |
sdague | what is tempest.common.compute-instance-1530232993 ? | 17:59 |
*** lucasagomes is now known as lucas-dinner | 18:01 | |
*** e0ne has joined #openstack-qa | 18:04 | |
*** austin81 has quit IRC | 18:04 | |
*** safchain has joined #openstack-qa | 18:05 | |
*** shardy has quit IRC | 18:06 | |
*** cdent has left #openstack-qa | 18:06 | |
*** rbak__ has joined #openstack-qa | 18:07 | |
*** rbak_ has quit IRC | 18:09 | |
*** luzC has quit IRC | 18:10 | |
openstackrecheck | Console logs not available after 13:20s for gate-cinder-python27 271595,18,e864faa | 18:10 |
sdague | huh, we're polling status on things which are already gone | 18:11 |
*** david-lyle has quit IRC | 18:12 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 18:12 | |
*** david-lyle has joined #openstack-qa | 18:13 | |
*** fnaval has quit IRC | 18:14 | |
*** gfidente has quit IRC | 18:14 | |
openstackgerrit | Sean Dague proposed openstack/tempest: add cleanup debug https://review.openstack.org/287360 | 18:14 |
*** ihrachys has quit IRC | 18:14 | |
*** luzC has joined #openstack-qa | 18:14 | |
*** mriedem has quit IRC | 18:18 | |
*** yaume_ has quit IRC | 18:18 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 286276,3,6d0ca33 | 18:23 |
*** doradus has quit IRC | 18:24 | |
mtreinish | sdague: ugh, how are we ending up with an instance without the test name as the random string seed? | 18:24 |
sdague | I think because it's the boot from snapshot 3rd instance | 18:25 |
*** safchain has quit IRC | 18:25 | |
*** mylu has joined #openstack-qa | 18:25 | |
mtreinish | sdague: well this: https://github.com/openstack/tempest/blob/master/tempest/common/compute.py#L56-L57 is also wrong | 18:26 |
sdague | so, more odd.... concerning | 18:26 |
mtreinish | we shouldn't ever default to that name because it makes tracking leaks really difficult | 18:26 |
sdague | we seem to be looking for deleted things multiple times | 18:26 |
sdague | like, we tear down server 3 | 18:26 |
sdague | and then after we tear down server 2 | 18:27 |
sdague | we do another poke at server 3 | 18:27 |
*** aalam has quit IRC | 18:27 | |
sdague | even though we definitely already got the server 3 404 and moved on | 18:27 |
mtreinish | sdague: so the scenario tests have an async wait add_cleanup call (which was supposed to be a timing optimization) | 18:27 |
sdague | I've been locally annotating the log, and it's weird | 18:27 |
mtreinish | where it issues a delete and then does the poll loop at the end during teardownclass | 18:28 |
sdague | mtreinish: well, it's doing it all at least twice | 18:28 |
sdague | and it's definitely not going through the resources like that | 18:28 |
*** austin81 has joined #openstack-qa | 18:28 | |
sdague | I think we should probably delete any optimizations here for clarity, honestly | 18:28 |
*** mriedem has joined #openstack-qa | 18:29 | |
*** e0ne has quit IRC | 18:29 | |
mtreinish | sdague: that dates back to like folsom or grizzly IIRC | 18:29 |
*** doradus has joined #openstack-qa | 18:29 | |
sdague | yeh | 18:30 |
*** e0ne has joined #openstack-qa | 18:30 | |
mtreinish | I remember adding it, I was refactoring something jaypipes did that created a lifo out of band in teardownclass to actually use addcleanup | 18:31 |
*** e0ne has quit IRC | 18:31 | |
sdague | well, anyway, this teardown is kind of nuts, and it's pretty hard to figure out why we're looping around again | 18:31 |
*** ddieterly has joined #openstack-qa | 18:31 | |
sdague | delete of server 2 takes 45 seconds as well ... which seems long | 18:32 |
*** e0ne has joined #openstack-qa | 18:32 | |
mtreinish | that does seem kinda long | 18:32 |
mtreinish | especially for a tiny cirros image | 18:32 |
ddieterly | if there is a devstack plugin and a gate job, how does the gate job know how to run the tempest tests if i add a tempest test plugin? what is the mechanism to get the tempest tests to run? | 18:33 |
sdague | mtreinish: right, I think the issue is we're requiring a graceful shutdown ... because nova does that by default now | 18:33 |
mtreinish | sdague: oh, yeah that could do it | 18:34 |
*** mylu has quit IRC | 18:35 | |
*** mylu has joined #openstack-qa | 18:35 | |
*** fnaval has joined #openstack-qa | 18:37 | |
openstackrecheck | Console logs not available after 13:20s for gate-cinder-pep8 283440,10,06f6b1f | 18:37 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make identity clients use rest_client https://review.openstack.org/287380 | 18:41 |
ddieterly | sdague: what is the mechanism to get tempest tests to run in a tempest test plugin? | 18:43 |
sdague | mtreinish: ok, that's pretty deeply embedded now. I thought we had fast kill but we really don't any more | 18:44 |
*** ajmiller_ has joined #openstack-qa | 18:45 | |
ddieterly | is it simply declaring tempest.test_plugins in the setup.cfg file? | 18:45 |
*** ajmiller has quit IRC | 18:48 | |
*** austin81 has left #openstack-qa | 18:50 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python34 286256,8,0b283e7 | 18:50 |
mriedem | mtreinish: so i'm not sure how i'm hitting htis http://logs.openstack.org/55/286855/1/experimental/gate-tempest-dsvm-neutron-dvr-multinode-full/8c4b90c/console.html#_2016-03-02_00_17_36_780 | 18:51 |
mriedem | because as far as i can tell, test_volume_boot_pattern is hitting similar code when calling get_server_ip(server) | 18:51 |
mriedem | i.e. https://review.openstack.org/#/c/286855/1/tempest/api/compute/admin/test_live_migration.py@93 | 18:52 |
mriedem | vs | 18:52 |
mriedem | https://github.com/openstack/tempest/blob/master/tempest/scenario/test_volume_boot_pattern.py#L113 | 18:52 |
mriedem | oooo i see | 18:53 |
mriedem | get_server_ip is different for api tests and scenario tests | 18:53 |
mriedem | sneaky sneaky | 18:53 |
sdague | yeh | 18:53 |
*** pvaneck has joined #openstack-qa | 18:53 | |
sdague | it's a bit less bonkers then it used to be | 18:54 |
sdague | but they are still different | 18:54 |
*** tosky has quit IRC | 18:55 | |
*** dwalleck has joined #openstack-qa | 18:56 | |
*** dwalleck has quit IRC | 18:56 | |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make network clients use rest_client https://review.openstack.org/287382 | 18:56 |
sdague | mriedem: also liberty went extra bonkers? | 18:58 |
sdague | https://review.openstack.org/#/c/286634/ is very suspicious that it failed on all of liberty | 18:58 |
mriedem | looking | 18:59 |
mriedem | hmmm | 18:59 |
mriedem | internap for one thing | 18:59 |
mgagne | mriedem: hi | 18:59 |
mriedem | sdague: have you run what-broke? | 18:59 |
sdague | mriedem: nope | 18:59 |
sdague | it's the boot from volume job though | 19:00 |
mriedem | yeah, i'll run what-broke | 19:00 |
mgagne | mriedem: sup with internap? =) | 19:00 |
mriedem | nothing yet | 19:00 |
sdague | mgagne: being slow... ? | 19:00 |
mgagne | sdague: any metrics? | 19:00 |
sdague | http://logs.openstack.org/76/286276/3/gate/gate-tempest-dsvm-postgres-full/6d0ca33/console.html and http://logs.openstack.org/76/286276/3/gate/gate-tempest-dsvm-neutron-full/e9b56fd/console.html timing out on boot from volume | 19:01 |
sdague | both on internap | 19:01 |
mgagne | sdague: gate is running at full capacity, you might be your own noisy neighbours | 19:01 |
mriedem | sdague: this failed differently http://logs.openstack.org/34/286634/2/check/gate-tempest-dsvm-neutron-full-liberty/0dd5886/console.html | 19:01 |
sdague | mgagne: right could be | 19:01 |
mriedem | yikes http://logs.openstack.org/34/286634/2/check/gate-tempest-dsvm-neutron-full-liberty/0dd5886/logs/screen-c-vol.txt.gz?level=TRACE | 19:01 |
sdague | but we're always going to be | 19:02 |
mriedem | http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22fixtures._fixtures.timeout.TimeoutException%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d | 19:02 |
mriedem | from that, build_node is primarily internap and ovh | 19:03 |
sdague | mriedem: yep | 19:03 |
mriedem | it's really weird that it'd be failing full on liberty though and not master | 19:03 |
sdague | honestly, I think it's just a speed issue | 19:03 |
mgagne | I prefer node_provider =) | 19:03 |
sdague | it also fails on master | 19:03 |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-python27 284848,6,968fe23 | 19:03 |
sdague | just not this time apparently | 19:04 |
sdague | I guess it was just bad dice on allocation | 19:04 |
sdague | mtreinish: do you know which timeout is killing us here? | 19:04 |
sdague | because last I looked the tempest timeout was 1200s | 19:04 |
sdague | and we're not there | 19:04 |
sdague | this is like a 600s timeout | 19:04 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Remove service_client module https://review.openstack.org/287388 | 19:07 |
mgagne | mriedem: RAX: 10(iad), OVH: 29(bsh1) + 16(gra1), INAP: 72(nyj01) | 19:08 |
sdague | hmmm.... somethign is setting it to 500 | 19:08 |
sdague | OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-500} | 19:08 |
sdague | gdi | 19:10 |
mgagne | mriedem: I'm trying to see where the timeout are occuring | 19:12 |
mgagne | I saw some related to instance connectivity tests | 19:12 |
*** ylobankov has joined #openstack-qa | 19:12 | |
*** ajmiller_ is now known as ajmiller | 19:15 | |
openstackgerrit | Sean Dague proposed openstack/tempest: bump tempest testr timeout to 1800s https://review.openstack.org/287395 | 19:15 |
sdague | that will fix this | 19:15 |
sdague | mriedem: you have a bug for it? | 19:15 |
mriedem | nope | 19:16 |
mgagne | I see a lot of ERROR logs for one of the failed connectivity tests: http://logs.openstack.org/71/270971/5/check/gate-neutron-dsvm-fullstack/0df391e/logs/TestOvsConnectivitySameNetwork.test_connectivity_GRE-and-l2pop,Ofctl_/neutron-openvswitch-agent--2016-02-25--02-25-30-454060.log.txt.gz | 19:16 |
mgagne | could it be related? | 19:16 |
sdague | mgagne: so.... it looks like the primary issue is just that we've got a timeout that's too low | 19:16 |
mgagne | like an issue that could only be triggered on slow nodes | 19:17 |
sdague | and it's in a place that I wasn't expecting | 19:17 |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python34 286116,2,021570e | 19:17 |
sdague | because I actually looked for it the other day | 19:17 |
sdague | mtreinish: so.... let me know if you are ok with https://review.openstack.org/287395 or if I should do the test scaling thing | 19:19 |
openstackgerrit | Merged openstack-infra/subunit2sql: Don't extend the passed in targets https://review.openstack.org/287299 | 19:19 |
mgagne | timeout looks to be related to test_connectivity test failures and since I see errors in logs, I'm wondering what triggers them | 19:20 |
mgagne | if all test_connectivity are timing out, this can easily pile up and make it looks like node is slow | 19:20 |
sdague | so, I think there are 2 things | 19:21 |
mgagne | I think too | 19:21 |
sdague | 1 ... in general the internap nodes are in the slower bucket for us, so the boot from volume test can individually timeout | 19:21 |
mgagne | slow passing tests and slow failing tests =) | 19:21 |
sdague | 2 ... we run out of total time | 19:22 |
*** sigmavirus24_awa is now known as sigmavirus24 | 19:22 | |
sdague | https://review.openstack.org/#/c/286276/ | 19:22 |
sdague | it's what happened there, 120 minutes wasn't long enough to get through the run | 19:22 |
mgagne | sdague: where is the time wasted? | 19:23 |
mgagne | sdague: "this is a very slow node" what does it mean? | 19:23 |
openstackgerrit | Matt Riedemann proposed openstack/tempest: DNM: ssh into active instance before and after live migration https://review.openstack.org/286855 | 19:23 |
mgagne | sdague: Running devstack: is it the bootstrapping part before tempest is run? | 19:23 |
sdague | mgagne: that is an error we throw in there for logstash | 19:23 |
sdague | because we can't do math on the logs easily | 19:24 |
mgagne | sdague: I'm sure there is more to that =) | 19:24 |
sdague | http://logs.openstack.org/76/286276/3/gate/gate-tempest-dsvm-postgres-full/6d0ca33/logs/devstacklog.txt.gz is the detailed log | 19:25 |
mgagne | I'm looking at it and trying to figure out what's slow | 19:25 |
mgagne | ~3m more to install packages (based on stats at the end) | 19:26 |
sdague | this feels like io slowness | 19:26 |
sdague | because 257s for pip_install is really high given the wheel mirror | 19:26 |
sdague | the disk io speed is just way off | 19:26 |
openstackrecheck | Console logs not available after 13:20s for gate-horizon-python27 257770,12,59822de | 19:30 |
*** mvk has quit IRC | 19:31 | |
*** vhosakot has quit IRC | 19:31 | |
*** fnaval has quit IRC | 19:32 | |
mgagne | sdague: could be very bad lock where multiple pip install are running at the same time on the same node | 19:33 |
mgagne | luck* | 19:33 |
*** vhosakot has joined #openstack-qa | 19:35 | |
*** doradus has quit IRC | 19:35 | |
sdague | mgagne: ? | 19:36 |
*** mylu has quit IRC | 19:36 | |
sdague | well, we use the io path quite a lot for lots of things | 19:36 |
*** mylu has joined #openstack-qa | 19:36 | |
sdague | that's just a good global counter for how things are doing on the io level | 19:37 |
*** davideagnello is now known as davideagnello__ | 19:40 | |
openstackrecheck | Console logs not available after 13:21s for gate-glance-python27 273982,3,87cac5d | 19:43 |
*** davideagnello__ is now known as davide85_ | 19:45 | |
mgagne | sdague: dstat isn't enabled until after packages are installed | 19:48 |
mgagne | sdague: I tried to see where the resources went | 19:48 |
mgagne | sdague: but the interesting part isn't available in the csv of dstat | 19:48 |
sdague | mgagne: no, honestly, the rest of the run is slow as well | 19:48 |
mgagne | sdague: sure but I can't analyze 2 hours of test. If I can concentrate on one part, that would be easier | 19:49 |
sdague | mgagne: right, but the point is we emit that message from devstack when the aggregate timing is odd | 19:49 |
mgagne | I know | 19:50 |
sdague | we still had 1.5 hrs after devstack ran | 19:50 |
sdague | I don't think you are going to find any particular hot spot | 19:50 |
mgagne | I think I could correlate those infos with stats I have on my side (provider) | 19:50 |
*** doradus has joined #openstack-qa | 19:50 | |
mgagne | I just enabled some stats tool so won't be able to compare until it gather a couple of data points | 19:51 |
ddieterly | anyone here able to answer some tempest plugin test questions? | 19:51 |
mgagne | sdague: do you have a test result for gate-tempest-dsvm-cells but on an other provider? I can try to compare | 19:53 |
mgagne | ok found rax results, 47m vs ~2h | 19:56 |
openstackrecheck | Console logs not available after 13:20s for experimental-tempest-dsvm-multinode-full-dibtest 286573,2,706f49b | 19:57 |
*** gszasz has quit IRC | 19:58 | |
mtreinish | sdague: we can do a blanket timeout, unless you think there is value in holding faster tests to the smaller timeout | 19:58 |
mtreinish | sdague: are we not having any issues with the build_timeout of 192s? (or whatever it is) | 20:03 |
mgagne | sdague: ok, average load at internap is just way higher than others | 20:05 |
mgagne | sdague: will see what I can do on my side about it | 20:05 |
openstackrecheck | Console logs not available after 13:21s for gate-nova-python27 286931,2,bda89b2 | 20:10 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Split out Neutron routers client https://review.openstack.org/260319 | 20:18 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Make routers_client use **kwargs https://review.openstack.org/284412 | 20:18 |
*** mylu has quit IRC | 20:22 | |
*** apoorvad has quit IRC | 20:23 | |
*** mylu has joined #openstack-qa | 20:23 | |
openstackrecheck | Console logs not available after 13:20s for gate-cinder-python34 286775,4,a867f49 | 20:24 |
*** mylu has quit IRC | 20:25 | |
*** mylu has joined #openstack-qa | 20:27 | |
*** rcernin has quit IRC | 20:36 | |
openstackrecheck | Console logs not available after 13:21s for gate-requirements-integration-dsvm 287313,1,f5e4562 | 20:37 |
*** luzC has quit IRC | 20:38 | |
sdague | mtreinish: no, this is the fixture timeout on delete | 20:40 |
sdague | mgagne: yeh, my guess is io constraints, we may just need to back off the over commit | 20:41 |
mtreinish | sdague: right, I realize that. I was just curious if we're not also tripping the per async operation timeout | 20:41 |
sdague | mtreinish: I don't believe so | 20:43 |
sdague | or if we are, it's way less than this | 20:43 |
mtreinish | ok | 20:43 |
sdague | the reality is, it's only this job that seems in trouble | 20:44 |
sdague | so we could put the test scaling bit in instead | 20:44 |
*** luzC has joined #openstack-qa | 20:44 | |
sdague | I had that pulled together the other day, but then the 1200 timeout in tox.ini threw me off that path | 20:44 |
sdague | not realizing that we're falling back to the 500 in .testr.conf | 20:44 |
*** david-lyle has quit IRC | 20:45 | |
*** david-lyle has joined #openstack-qa | 20:47 | |
*** jecarey_ has quit IRC | 20:49 | |
*** yangyapeng has joined #openstack-qa | 20:50 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-pep8 287248,2,889030e | 20:50 |
openstackgerrit | Merged openstack/tempest: Added processing /compute URL for getting list of versions https://review.openstack.org/284621 | 20:51 |
*** rossella_s has quit IRC | 20:53 | |
*** rossella_s has joined #openstack-qa | 20:54 | |
openstackgerrit | Sean Dague proposed openstack/tempest: Implement TIMEOUT_SCALING_FACTOR for tempest https://review.openstack.org/287434 | 20:55 |
sdague | mtreinish: so that's honestly probably *more* correct ^^^ | 20:55 |
mgagne | sdague: we couldn't find any io constraints on our side | 20:55 |
mgagne | sdague: we suspect CPU instead | 20:55 |
sdague | mgagne: really? interesting | 20:55 |
sdague | what's the procs on these? | 20:55 |
mgagne | sdague: rax: 9 average load. inap: 25 average load | 20:55 |
sdague | oof | 20:56 |
mgagne | yea | 20:56 |
sdague | honestly, I'm surprised it held up | 20:56 |
sdague | normally once we get above 12 we start killing the message bus | 20:56 |
mgagne | sdague: I would find a way to auto generate graphs with http://lamada.eu/dstat-graph/ from build logs very useful | 20:56 |
mtreinish | sdague: +2 | 20:56 |
sdague | mgagne: me too | 20:57 |
mgagne | sdague: that's how I found about the load average | 20:57 |
*** yangyapeng has quit IRC | 20:57 | |
sdague | mtreinish: isn't that a thing? | 20:57 |
mgagne | there is no way I can mentally parse the csv | 20:57 |
*** pvaneck has quit IRC | 20:57 | |
sdague | or is it only a partial thing | 20:57 |
mtreinish | sdague: well stackviz is the closest thing to that | 20:57 |
mgagne | sdague: but there is no license attached to the project so I don't know if we can fork it and patch it + host it | 20:57 |
*** pvaneck has joined #openstack-qa | 20:58 | |
mtreinish | it's still not integrated into the workflow | 20:58 |
sdague | mgagne: yeh, if you ask and find out, we could do a thing there | 20:58 |
mtreinish | you can do it out of band | 20:58 |
sdague | mtreinish: is there still anyone working on stackviz? | 20:58 |
timothyb89 | sdague: yup! | 20:58 |
sdague | timothyb89: ok, coolio, any thoughts on getting it into our workflow a little tighter | 20:58 |
sdague | it would be super nice to have that at the end of runs | 20:59 |
mtreinish | sdague: https://review.openstack.org/#/c/287373/ | 20:59 |
*** apoorvad has joined #openstack-qa | 20:59 | |
mtreinish | sdague: in the meantime you can use timothyb89's prototype: https://stackviz.timothyb89.org/ | 20:59 |
sdague | mtreinish: ok, starred for tomorrow | 20:59 |
timothyb89 | sdague: mtreinish beat me to it :) | 20:59 |
*** mylu has quit IRC | 21:01 | |
*** mylu has joined #openstack-qa | 21:01 | |
*** raildo is now known as raildo-afk | 21:02 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 284412,6,c0489b1 | 21:04 |
*** jecarey has joined #openstack-qa | 21:05 | |
*** jecarey has quit IRC | 21:08 | |
*** jecarey has joined #openstack-qa | 21:08 | |
*** aysyd has quit IRC | 21:09 | |
*** moravec has joined #openstack-qa | 21:09 | |
*** ngoracke has joined #openstack-qa | 21:10 | |
*** ngoracke has quit IRC | 21:14 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-pep8 283171,9,b57228d | 21:17 |
*** aunnam has joined #openstack-qa | 21:23 | |
*** anshul has joined #openstack-qa | 21:24 | |
*** anshul has quit IRC | 21:26 | |
*** dkranz has quit IRC | 21:27 | |
sdague | mtreinish: oo... is openstack health a thing again - http://status.openstack.org/openstack-health/#/ ? | 21:29 |
sdague | mtreinish: feature request on last 10 failed | 21:30 |
*** aunnam has left #openstack-qa | 21:30 | |
*** dkranz has joined #openstack-qa | 21:30 | |
openstackrecheck | Console logs not available after 13:21s for gate-grenade-dsvm-neutron 279035,4,e499b70 | 21:30 |
sdague | sort by stop time | 21:31 |
sdague | or by some time, not by test name | 21:31 |
sdague | and if we could get the last 10 in the integrated gate queue, that would be awesome | 21:31 |
*** manjeets has joined #openstack-qa | 21:32 | |
mtreinish | sdague: yeah, jeblair found the bug and fixed it with: https://review.openstack.org/#/c/287299/ | 21:33 |
sdague | nice | 21:33 |
mtreinish | so things are moving again there | 21:33 |
mtreinish | oh, did I forget to make the table sortable | 21:34 |
mtreinish | oops | 21:34 |
sdague | it sorts by test name I think | 21:34 |
sdague | it's very noticable with the current data set | 21:34 |
mtreinish | yeah, that's wrong we should default sort on one of the times, but I meant to make it selectable | 21:35 |
sdague | also, the time could probably just be Feb 2, 17:26 | 21:35 |
mtreinish | like the other tables | 21:35 |
yfried | afazekas: mtreinish: gmann: oomichi: masayukig: https://review.openstack.org/259746 could you please review? | 21:35 |
sdague | so that it was visually easier to parse | 21:35 |
sdague | almost all of those values in the time field are not human useful :) | 21:36 |
sdague | especially seconds an uS | 21:36 |
sdague | and year :) | 21:36 |
mtreinish | sdague: well when we had >1 yr of data in the db year was important | 21:37 |
mtreinish | but we're filtering < 6months now so it's not needed anymore | 21:37 |
mtreinish | sdague: if you file a bug on that, masayukig will prbably just do it when he's online | 21:37 |
mriedem | blarg | 21:38 |
mriedem | why do compute api tests have to redefine class variables for all of the clients that are already in the base class? | 21:39 |
timothyb89 | mtreinish, sdague: I can take care of it really quick | 21:39 |
mtreinish | timothyb89: go for it, I'm stuck in a pandas hole right now | 21:39 |
timothyb89 | mtreinish: that doesn't sound like much fun | 21:39 |
mtreinish | I dunno, pandas are cool :) | 21:40 |
mtreinish | mriedem: for no good reason, the only thing I can think of is 80 chars in pep8 | 21:40 |
mriedem | mtreinish: no i mean http://logs.openstack.org/55/286855/2/experimental/gate-tempest-dsvm-neutron-dvr-multinode-full/3fd9226//console.html#_2016-03-02_21_24_18_876 | 21:41 |
mriedem | https://review.openstack.org/#/c/286855/2/tempest/api/compute/admin/test_live_migration.py | 21:41 |
mriedem | i'm using some other clients that aren't stored as vars in setup_clients in the test class, but they are defined in setup_clients in the base class | 21:41 |
mriedem | so i figured i could just re-use those | 21:41 |
mtreinish | mriedem: that should work, is super() called in the test_live_migration.py setup_clients? | 21:43 |
mriedem | yup | 21:43 |
mriedem | and in BaseV2ComputeAdminTest | 21:44 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-dvr 260319,29,8a6caef | 21:44 |
mriedem | is it because this is admin? | 21:44 |
mriedem | so it has to use cls.os_adm | 21:44 |
mriedem | b/c the base class uses cls.os.* | 21:44 |
mtreinish | mriedem: well both managers should be initialized | 21:44 |
mtreinish | it might not be using the one you want to be, but it should be defined | 21:45 |
mriedem | alright, weird | 21:45 |
manjeets | hi I am trying to check if some extension is enabled in neutron or not using defined method https://github.com/openstack/tempest/blob/master/tempest/test.py#L163 | 21:45 |
mriedem | well, i guess i'll just put my head in the sand and follow the pattern already in the test | 21:46 |
manjeets | but it is not working for one of extension recently added | 21:46 |
manjeets | if want to check a extension from list returned by neutron ext-list | 21:46 |
mtreinish | mriedem: I normally prefer to do self.os_adm.client_name.call() instead of an alias | 21:46 |
*** r-mibu has quit IRC | 21:46 | |
mtreinish | it's easier to know where things come from that way | 21:46 |
*** erlon has quit IRC | 21:46 | |
*** r-mibu has joined #openstack-qa | 21:47 | |
openstackgerrit | Matthew Treinish proposed openstack/openstack-health: WIP: Add numeric run_time data for runs https://review.openstack.org/287463 | 21:48 |
*** krtaylor has quit IRC | 21:48 | |
mtreinish | timothyb89: ^^^ my pandas hole | 21:48 |
openstackgerrit | Matt Riedemann proposed openstack/tempest: DNM: ssh into active instance before and after live migration https://review.openstack.org/286855 | 21:48 |
mtreinish | although it breaks the js side, but I didn't want to lose my work | 21:48 |
*** sc68cal has quit IRC | 21:48 | |
timothyb89 | mtreinish: ah, cool, that'll be nice | 21:48 |
timothyb89 | we can always use more scatter plots :) | 21:49 |
mriedem | mtreinish: check out how much of the scenario manager i had to copy here https://review.openstack.org/#/c/286855/3/tempest/api/compute/admin/test_live_migration.py | 21:49 |
mtreinish | mriedem: how do the other api tests that ssh do it? | 21:50 |
mriedem | the validation_resources or whatever it's called | 21:50 |
mriedem | but i think those are only set if run_validation=True? | 21:50 |
mtreinish | manjeets: that function is only for configuration, it doesn't query the api at all | 21:50 |
mriedem | the scenarios tests create all of that stuff on the fly | 21:50 |
*** sc68cal has joined #openstack-qa | 21:50 | |
mtreinish | manjeets: tempest doesn't rely on the api to make a skip decision like that. It's just asking for trouble to do that | 21:51 |
oomichi | yfried: hat is gmann's turn and need to wait for his morning more 2 hours | 21:51 |
yfried | oomichi: turn? | 21:51 |
manjeets | yea i need to check configuration whether some extension is loaded or not | 21:52 |
oomichi | yfried: his score is latest now, and I'd like to see his feedback | 21:52 |
manjeets | mtreinish: | 21:52 |
yfried | oomichi: ok | 21:52 |
*** josdotso has joined #openstack-qa | 21:54 | |
*** armax has quit IRC | 21:55 | |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-python27 284973,3,9ac6bde | 21:57 |
sdague | mriedem: because they hate you | 21:58 |
mriedem | god, obviously | 21:58 |
oomichi | mtreinish: gate-tempest-dsvm-layer4 seems broken now, is someone working for that? | 21:58 |
oomichi | else I will do | 21:58 |
*** armax has joined #openstack-qa | 21:59 | |
sdague | oomichi: I don't think anyone else is looking at it, and it looks proper broken | 21:59 |
oomichi | sdague: proper broken? | 22:00 |
sdague | not just an odd race | 22:00 |
oomichi | meaning intentional broken? | 22:00 |
sdague | create_tenant has wrong number of args | 22:00 |
moravec | @mtreinish @timothyb89 is there an option on OpenStack Health to see %tests failed instead of %runs failed? | 22:04 |
*** achanda has quit IRC | 22:06 | |
moravec | unless I'm missing something, all the data is showing the number and % of the runs that fail, but a failed run can mean anything from 1 to n failed tests | 22:08 |
mtreinish | moravec: yes, you can either dive down into a per run page, or go right to: http://status.openstack.org/openstack-health/#/tests?groupKey=project&resolutionKey=hour | 22:09 |
mtreinish | moravec: and there is also a per test page: http://status.openstack.org/openstack-health/#/test/tempest.scenario.test_volume_boot_pattern.TestVolumeBootPatternV2.test_volume_boot_pattern | 22:10 |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-neutron-full 281999,2,c89d5f9 | 22:10 |
moravec | @mtreinish ok, but no summary data for the range selected? Like, over the past week what is the %tests failed for a given build_queue (e.g., periodic_qa)? | 22:11 |
openstackgerrit | Ken'ichi Ohmichi proposed openstack/tempest: Skip test_list_db_versions https://review.openstack.org/287476 | 22:12 |
mtreinish | moravec: not for a single build_queue we have that per job (well build_name is the actual key) | 22:12 |
mtreinish | moravec: like http://status.openstack.org/openstack-health/#/job/gate-tempest-dsvm-neutron-full?groupKey=project&resolutionKey=hour | 22:13 |
mtreinish | oomichi: I wasn't aware it was broken, do you have a link? | 22:13 |
*** achanda has joined #openstack-qa | 22:13 | |
moravec | @mtreinish ok, thanks, that's what I thought. Something like this, but showing the test% pass/fail (vs. runs) http://status.openstack.org/openstack-health/#/?groupKey=build_queue&resolutionKey=hour&duration=P1M | 22:14 |
oomichi | mtreinish: http://logs.openstack.org/43/287343/2/check/gate-tempest-dsvm-layer4/e07db68/logs/testr_results.html.gz | 22:14 |
*** e0ne has quit IRC | 22:14 | |
oomichi | mtreinish: I start digging it now | 22:14 |
*** mylu has quit IRC | 22:15 | |
*** mylu has joined #openstack-qa | 22:15 | |
mtreinish | oomichi: is that on your change which updates the clients to use the rest_client instead of service_client? | 22:16 |
mtreinish | I'm wondering are we seeing that failure elsewhere | 22:16 |
oomichi | mtreinish: oops, nice point | 22:17 |
*** e0ne has joined #openstack-qa | 22:17 | |
mtreinish | moravec: sure, it's doable. The problem with that is twofold though. there isn't neccesarily a consistent set of tests across all jobs on an arbitrary metadata key and its a lot of data | 22:18 |
mtreinish | like the per job page is are slowest because to pull in all the data necessary at the per test level is expensive | 22:18 |
*** weshay has quit IRC | 22:19 | |
moravec | @mtreinish ok, I see the link you sent has Test Failure Rate by build_nameā¦.that's useful. Thanks for the pointers. I was thinking the %test fail is a better view of how things are than job failures, since a single test failure marks the whole job failed. | 22:20 |
*** achanda has quit IRC | 22:20 | |
openstackgerrit | Tim Buckley proposed openstack/openstack-health: Improve failed and recent runs tables https://review.openstack.org/287478 | 22:21 |
andreaf | mtreinish: do you want me to re-spin this or can we get it in as is, and I will start the implementation? | 22:22 |
mtreinish | andreaf: what is this? | 22:23 |
andreaf | mtreinish: heh sorry it's the client manager refactor spec | 22:23 |
mtreinish | andreaf: I think we should respin the spec first | 22:24 |
openstackrecheck | Console logs not available after 13:20s for gate-tempest-dsvm-neutron-full 287355,1,4476977 | 22:24 |
andreaf | I just noticed you have comments on the resources one - I'll check them now | 22:24 |
andreaf | ok | 22:24 |
mtreinish | to see if there is buy in from the broader community, not just us :) | 22:24 |
andreaf | jswarren: hi - have you heard anything from dwalleck about his patches for the test resources spec? | 22:25 |
andreaf | mtreinish: heh, good point | 22:26 |
jswarren | andreaf: He submitted this: https://review.openstack.org/#/c/286342/2 | 22:26 |
*** e0ne has quit IRC | 22:27 | |
andreaf | jswarren: oh - ok - I missed that, thanks for the link I'll review it | 22:28 |
*** dkranz has quit IRC | 22:28 | |
openstackgerrit | Tim Buckley proposed openstack/openstack-health: Add REST API methods for test prefixes https://review.openstack.org/287482 | 22:31 |
*** edmondsw has quit IRC | 22:32 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python34 286901,2,e5cb6cc | 22:37 |
*** bkopilov has quit IRC | 22:39 | |
*** yangyapeng has joined #openstack-qa | 22:41 | |
*** yangyapeng has quit IRC | 22:46 | |
openstackgerrit | Tim Buckley proposed openstack/openstack-health: Add REST API methods for test prefixes https://review.openstack.org/287482 | 22:47 |
*** mriedem has quit IRC | 22:48 | |
*** mylu has quit IRC | 22:49 | |
*** bkopilov has joined #openstack-qa | 22:50 | |
openstackrecheck | Console logs not available after 13:20s for gate-cinder-python34 281625,6,66667af | 22:51 |
*** krtaylor has joined #openstack-qa | 22:52 | |
*** yarkot_ has joined #openstack-qa | 22:53 | |
*** sc68cal_ has joined #openstack-qa | 22:56 | |
*** dims_ has quit IRC | 22:56 | |
*** yarkot_ has quit IRC | 22:57 | |
*** sc68cal has quit IRC | 22:58 | |
*** mylu has joined #openstack-qa | 22:58 | |
openstackrecheck | Console logs not available after 13:20s for gate-swift-docs 278245,23,7c31cab | 23:04 |
openstackgerrit | Merged openstack/openstack-health: Remove workaround graph width https://review.openstack.org/285955 | 23:12 |
*** yamamoto_ has joined #openstack-qa | 23:12 | |
*** sigmavirus24 is now known as sigmavirus24_awa | 23:15 | |
*** mylu has quit IRC | 23:15 | |
*** regXboi has quit IRC | 23:17 | |
openstackrecheck | Console logs not available after 13:21s for gate-glance-python27 287349,2,7650799 | 23:17 |
*** ajmiller has quit IRC | 23:17 | |
*** ajmiller has joined #openstack-qa | 23:18 | |
*** salv-orlando has joined #openstack-qa | 23:19 | |
*** achanda has joined #openstack-qa | 23:20 | |
*** salv-orl_ has quit IRC | 23:22 | |
*** yamamoto_ has quit IRC | 23:24 | |
*** dimtruck is now known as zz_dimtruck | 23:25 | |
*** mylu has joined #openstack-qa | 23:27 | |
*** mylu has quit IRC | 23:27 | |
*** mylu has joined #openstack-qa | 23:27 | |
*** achanda has quit IRC | 23:28 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 287343,2,02ea132 | 23:31 |
*** mylu has quit IRC | 23:33 | |
*** tochi has joined #openstack-qa | 23:37 | |
*** ddieterly has quit IRC | 23:39 | |
*** pglass has quit IRC | 23:42 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-python27 287253,2,de61189 | 23:44 |
*** manjeets has left #openstack-qa | 23:47 | |
*** salv-orlando has quit IRC | 23:50 | |
*** salv-orlando has joined #openstack-qa | 23:51 | |
gmann | yfried: looks fine. | 23:52 |
*** ylobankov has quit IRC | 23:55 | |
openstackrecheck | Console logs not available after 13:20s for experimental-tempest-dsvm-multinode-full-dibtest 287363,1,5d258d3 | 23:57 |
*** ajmiller has quit IRC | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!