Thursday, 2014-07-24

*** rbak has quit IRC00:00
*** rbak has joined #openstack-qa00:00
*** atiwari has quit IRC00:01
openstackgerritSimeon Monov proposed a change to openstack/tempest: Fix exception when api_extensions is set to empty  https://review.openstack.org/10723000:03
*** rbak has quit IRC00:04
*** yamahata has joined #openstack-qa00:05
*** openstackrecheck has quit IRC00:06
*** openstackrecheck has joined #openstack-qa00:06
*** rbak has joined #openstack-qa00:07
*** rbak has quit IRC00:13
*** gmann has joined #openstack-qa00:14
*** dims has joined #openstack-qa00:14
*** salv-orlando has quit IRC00:15
*** gmann has quit IRC00:15
*** gmann has joined #openstack-qa00:15
*** aimon has quit IRC00:19
*** danfinch has quit IRC00:21
*** arnaud__ has quit IRC00:23
*** arnaud has quit IRC00:23
openstackrecheckConsole logs not available after 13:24s for gate-keystone-docs 109081,1,243004200:28
*** dmorita has joined #openstack-qa00:32
*** nati_ueno has joined #openstack-qa00:38
openstackrecheckConsole logs not available after 13:23s for check-swift-dsvm-functional 107942,4,adfda3700:42
*** arnaud has joined #openstack-qa00:46
openstackrecheckConsole logs not available after 13:25s for check-grenade-dsvm-partial-ncpu 105552,2,4a7db3300:55
*** gmann has quit IRC01:05
*** gmann has joined #openstack-qa01:05
*** fandi has quit IRC01:07
openstackrecheckConsole logs not available after 13:23s for gate-horizon-docs 101755,3,661041401:09
openstackgerritA change was merged to openstack-infra/subunit2sql: Bump oslo.config version  https://review.openstack.org/10914001:16
*** nati_ueno has quit IRC01:21
openstackrecheckConsole logs not available after 13:21s for gate-nova-python27 105351,3,adc6b6f01:22
jogosdague: thanks01:26
*** mmaglana has quit IRC01:28
*** mmaglana has joined #openstack-qa01:29
*** jecarey has joined #openstack-qa01:29
*** mwagner_lap has joined #openstack-qa01:32
openstackrecheckConsole logs not available after 13:21s for gate-keystone-pep8 96266,20,2c8125101:35
openstackgerritSean Dague proposed a change to openstack-dev/grenade: move from screen dumping to using worlddump tool  https://review.openstack.org/10912301:41
openstackgerritSean Dague proposed a change to openstack-dev/grenade: first pass of integrating javelin2 to grenade  https://review.openstack.org/9731701:41
*** nosnos has joined #openstack-qa01:43
*** kashyap has quit IRC01:45
*** kashyap has joined #openstack-qa01:47
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 107479,4,e0f48fa01:49
openstackgerritSean Dague proposed a change to openstack-dev/grenade: move from screen dumping to using worlddump tool  https://review.openstack.org/10912301:49
*** eglynn-officeafk has quit IRC01:51
*** eglynn-officeafk has joined #openstack-qa01:55
openstackgerritFeihuJiang proposed a change to openstack/tempest: Adds an orchestration API test for resource types  https://review.openstack.org/10377502:02
*** Longgeek has joined #openstack-qa02:02
openstackrecheckConsole logs not available after 13:21s for gate-nova-python27 105425,3,633796602:02
*** arnaud has quit IRC02:03
*** Longgeek has quit IRC02:07
*** yamahata has quit IRC02:08
*** yamahata has joined #openstack-qa02:09
*** arnaud has joined #openstack-qa02:13
*** dims has quit IRC02:14
openstackrecheckConsole logs not available after 13:21s for check-grenade-dsvm 108606,3,7f5f14802:15
openstackrecheckConsole logs not available after 13:21s for gate-nova-python26 103916,7,332b3da02:29
jogosdague: do you have a script to run grenade locally?02:35
*** bkopilov has quit IRC02:36
jogoI can't even get icehouse set up02:36
jogo2014-07-24 02:19:24.140 | ERROR: openstackclient.shell Exception raised: six>=1.6.002:36
*** dims has joined #openstack-qa02:40
openstackrecheckConsole logs not available after 13:21s for gate-ceilometer-python26 102353,4,c68d04002:42
*** dims has quit IRC02:44
*** pcrews has quit IRC02:45
*** adalbas has quit IRC02:55
openstackrecheckConsole logs not available after 13:21s for gate-keystone-pep8 98522,14,f82fea402:56
openstackgerritJustin Shepherd proposed a change to openstack-dev/devstack: Install gettext from apt on Trusty (Ubuntu 14.04)  https://review.openstack.org/10917903:02
*** Longgeek has joined #openstack-qa03:03
*** Longgeek has quit IRC03:07
openstackrecheckConsole logs not available after 13:22s for check-tempest-dsvm-neutron 109095,1,382532e03:09
*** kashyap has quit IRC03:11
*** nosnos has quit IRC03:15
*** kashyap_bbiab has joined #openstack-qa03:15
*** kashyap_bbiab is now known as kashyap03:20
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 106822,7,444ae1603:22
*** cmyster has joined #openstack-qa03:24
*** cmyster has quit IRC03:24
*** cmyster has joined #openstack-qa03:24
*** cmyster has quit IRC03:34
openstackrecheckConsole logs not available after 13:23s for check-grenade-dsvm 106824,7,dfa9e0003:36
*** jecarey has quit IRC03:36
*** arnaud has quit IRC03:38
*** dims has joined #openstack-qa03:41
*** cmyster has joined #openstack-qa03:42
*** cmyster has quit IRC03:42
*** cmyster has joined #openstack-qa03:42
*** dims has quit IRC03:45
openstackrecheckConsole logs not available after 13:22s for gate-nova-python26 104688,7,9df131c03:49
openstackgerritA change was merged to openstack-dev/devstack: stop leaking service as a global var  https://review.openstack.org/10908203:50
openstackgerritA change was merged to openstack-dev/devstack: add is_keystone_enabled  https://review.openstack.org/10911603:51
*** chandankumar has joined #openstack-qa03:53
*** mmaglana has quit IRC04:02
openstackrecheckConsole logs not available after 13:21s for gate-cinder-docs 106113,3,594129f04:02
*** Longgeek has joined #openstack-qa04:03
*** Longgeek has quit IRC04:08
*** yfried_ has quit IRC04:09
*** bkopilov has joined #openstack-qa04:14
openstackrecheckConsole logs not available after 13:21s for check-grenade-dsvm 98235,4,4f1ba0104:16
*** krtaylor has joined #openstack-qa04:17
*** nosnos has joined #openstack-qa04:18
openstackgerritA change was merged to openstack-dev/devstack: Fixed NO_UPDATE_REPOS variable usage  https://review.openstack.org/10859404:25
*** Poornima_ has joined #openstack-qa04:26
jogosdague: heh setuptools04:26
jogosdague: sorted out the issue04:27
jogosdague: should grenade make sure setuptools is up to date?04:27
*** oomichi has joined #openstack-qa04:28
openstackrecheckConsole logs not available after 13:23s for gate-keystone-pep8 104066,8,552d44704:29
*** nati_ueno has joined #openstack-qa04:40
*** dims has joined #openstack-qa04:41
openstackrecheckConsole logs not available after 13:24s for check-tempest-dsvm-neutron 107526,4,b3e8b6e04:43
*** SridharG has joined #openstack-qa04:44
*** dims has quit IRC04:46
*** chandankumar has quit IRC04:49
openstackrecheckConsole logs not available after 13:22s for gate-heat-docs 100420,19,422235704:56
*** arnaud has joined #openstack-qa04:59
*** chandankumar has joined #openstack-qa04:59
openstackrecheckConsole logs not available after 13:22s for gate-swift-pep8 101758,2,a2af02f05:09
*** harlowja has quit IRC05:11
*** harlowja has joined #openstack-qa05:11
openstackgerritFei Long Wang proposed a change to openstack/tempest: Add cases for Swift scenario test  https://review.openstack.org/9951405:13
*** ArxCruz has quit IRC05:22
openstackrecheckConsole logs not available after 13:22s for gate-nova-pep8 81954,13,c2e2bd605:23
*** k4n0 has joined #openstack-qa05:24
*** Anju has joined #openstack-qa05:35
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-full 96283,3,8fdd79a05:36
*** gmann has quit IRC05:38
*** dims has joined #openstack-qa05:42
*** Sudhi has joined #openstack-qa05:46
SudhiHi, can anyone please review my patch "https://review.openstack.org/#/c/108973/"05:47
*** dims has quit IRC05:47
openstackrecheckConsole logs not available after 13:22s for check-grenade-dsvm 108465,1,81aa3e005:50
*** ildikov has quit IRC06:02
openstackrecheckConsole logs not available after 13:22s for gate-keystone-pep8 104065,8,3e0588206:03
*** gmann has joined #openstack-qa06:04
*** gmann has quit IRC06:05
*** gmann has joined #openstack-qa06:06
*** arnaud has quit IRC06:06
*** Madan1 has joined #openstack-qa06:12
*** Madan1 has quit IRC06:13
openstackrecheckConsole logs not available after 13:23s for gate-nova-python26 109114,1,a65a17406:16
*** Longgeek has joined #openstack-qa06:18
*** Madan has joined #openstack-qa06:18
*** _afazekas has quit IRC06:20
*** afazekas is now known as _afazekas06:20
*** jlibosva has joined #openstack-qa06:24
*** dkorn has joined #openstack-qa06:25
*** psharma has joined #openstack-qa06:26
*** zigo has quit IRC06:27
*** ildikov has joined #openstack-qa06:29
openstackrecheckConsole logs not available after 13:24s for gate-keystone-python27 108841,4,ff12c8706:30
*** mitz has quit IRC06:30
*** mitz has joined #openstack-qa06:32
*** zigo has joined #openstack-qa06:32
*** ChanServ changes topic to "All Things QA..."06:33
*** andreaf has quit IRC06:34
*** liusheng has joined #openstack-qa06:34
*** Longgeek has quit IRC06:38
*** Longgeek has joined #openstack-qa06:39
*** SridharG has quit IRC06:39
*** harlowja is now known as harlowja_away06:42
*** dims has joined #openstack-qa06:43
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-full 109115,1,168a5dd06:43
*** enikanorov has quit IRC06:44
*** tkammer has joined #openstack-qa06:44
openstackgerritFeihuJiang proposed a change to openstack/tempest: Adds an orchestration API test for resource types  https://review.openstack.org/10377506:44
*** enikanorov has joined #openstack-qa06:44
*** dims has quit IRC06:48
*** nati_ueno has quit IRC06:50
openstackrecheckConsole logs not available after 13:23s for gate-cinder-pep8 109192,1,484017607:02
*** afazekas_ has joined #openstack-qa07:04
*** markus_z has joined #openstack-qa07:06
*** evgenyf has joined #openstack-qa07:11
*** luqas has joined #openstack-qa07:14
*** luqas has quit IRC07:15
openstackrecheckConsole logs not available after 13:23s for check-grenade-dsvm 108564,3,e40b0dd07:16
*** nmagnezi has joined #openstack-qa07:16
*** jamielennox is now known as jamielennox|away07:20
*** luqas has joined #openstack-qa07:22
*** xianghuihui has joined #openstack-qa07:23
*** eglynn_ has joined #openstack-qa07:24
*** krtaylor has quit IRC07:25
*** xianghui has quit IRC07:26
*** salv-orlando has joined #openstack-qa07:27
openstackrecheckConsole logs not available after 13:23s for gate-sahara-pep8 109196,1,43b93d407:29
*** jlanoux has joined #openstack-qa07:32
*** xianghuihuihui has joined #openstack-qa07:41
openstackrecheckConsole logs not available after 13:22s for check-grenade-dsvm 107290,8,049d21507:42
*** dims_ has joined #openstack-qa07:43
*** salv-orlando has quit IRC07:44
*** xianghuihui has quit IRC07:44
*** yfried_ has joined #openstack-qa07:47
*** xianghuihui has joined #openstack-qa07:48
*** dims_ has quit IRC07:48
*** xianghuihuihui has quit IRC07:51
*** mestery_ has joined #openstack-qa07:54
openstackgerritYair Fried proposed a change to openstack/tempest: Rename "thing" to "resource"  https://review.openstack.org/10113507:56
openstackrecheckConsole logs not available after 13:25s for check-grenade-dsvm 109194,1,454ff7b07:56
*** mestery has quit IRC07:57
*** SridharG has joined #openstack-qa07:58
*** xianghuihuihui has joined #openstack-qa08:01
*** xianghuihui has quit IRC08:03
*** andreaf has joined #openstack-qa08:05
*** safchain has joined #openstack-qa08:09
openstackrecheckConsole logs not available after 13:28s for check-tempest-dsvm-neutron-heat-slow 106939,10,ba58b3608:09
*** xianghuihuihui has quit IRC08:11
*** xianghuihui has joined #openstack-qa08:11
*** gszasz has joined #openstack-qa08:17
openstackgerritChris Dent proposed a change to openstack-infra/elastic-recheck: e-r query for intermittent failure in glance tests  https://review.openstack.org/10899308:22
openstackrecheckConsole logs not available after 13:22s for check-grenade-dsvm 107291,9,f1ac93608:23
openstackrecheckConsole logs not available after 13:22s for check-tempest-dsvm-neutron 108917,2,644f3ab08:36
openstackgerritPavel Sedlák proposed a change to openstack-dev/devstack: Without tenant isolation use static test accounts  https://review.openstack.org/10745708:39
*** jordanP has joined #openstack-qa08:41
*** dims_ has joined #openstack-qa08:44
*** salv-orlando has joined #openstack-qa08:45
*** xianghuihuihui has joined #openstack-qa08:47
*** xianghuihui has quit IRC08:48
*** dims_ has quit IRC08:49
openstackrecheckConsole logs not available after 13:22s for check-tempest-dsvm-full 108899,2,dee6cc608:50
*** gmann has quit IRC08:50
openstackgerritFeihuJiang proposed a change to openstack/tempest: Adds an orchestration API test for resource types  https://review.openstack.org/10377508:52
*** markus_z has quit IRC08:57
openstackrecheckConsole logs not available after 13:22s for gate-ceilometer-python26 104974,17,fe0423909:03
*** yfried__ has joined #openstack-qa09:07
*** amotoki has joined #openstack-qa09:08
*** yfried has joined #openstack-qa09:08
*** yfried_ has quit IRC09:10
*** yfried__ has quit IRC09:12
*** markus_z has joined #openstack-qa09:15
openstackrecheckConsole logs not available after 13:22s for gate-tempest-dsvm-large-ops 104893,7,a5747d209:17
*** evgenyf has quit IRC09:19
*** zz_johnthetubagu is now known as johnthetubaguy09:21
*** jyuso has joined #openstack-qa09:28
openstackrecheckConsole logs not available after 13:25s for check-tempest-dsvm-postgres-full 96999,6,afde47409:30
openstackgerritRavikumar Venkatesan proposed a change to openstack/tempest: Adds "user-password-update" to v3/test_users  https://review.openstack.org/9210009:41
openstackrecheckConsole logs not available after 13:23s for gate-ceilometer-pep8 106304,3,9b4625a09:44
*** ildikov has quit IRC09:44
*** dims_ has joined #openstack-qa09:45
*** Poornima_ has quit IRC09:46
*** Mmike has left #openstack-qa09:47
*** dims_ has quit IRC09:50
*** yfried has quit IRC09:54
*** evgenyf has joined #openstack-qa09:56
*** tosky has joined #openstack-qa09:56
openstackrecheckConsole logs not available after 13:28s for gate-nova-pep8 86115,12,798c3ee09:57
openstackgerritA change was merged to openstack-dev/devstack: Add DVR support to DevStack  https://review.openstack.org/10318009:59
*** yamahata has quit IRC10:00
*** sdmonov has joined #openstack-qa10:01
openstackgerritSunil G proposed a change to openstack/tempest: Fixes incorrect assertion check  https://review.openstack.org/10897310:02
openstackgerritSunil G proposed a change to openstack/tempest: Fixes incorrect assertion check  https://review.openstack.org/10923710:02
*** jlibosva has quit IRC10:07
*** k4n0 has quit IRC10:07
*** yfried has joined #openstack-qa10:10
openstackrecheckConsole logs not available after 13:23s for gate-grenade-dsvm-partial-ncpu 107168,4,e4f6b2910:10
*** markus_z has quit IRC10:13
sdaguejogo: I just run grenade.sh10:20
*** k4n0 has joined #openstack-qa10:20
*** eglynn-officeafk has quit IRC10:23
openstackrecheckConsole logs not available after 13:23s for gate-ceilometer-python26 107942,4,6ed98f110:24
*** eglynn-officeafk has joined #openstack-qa10:24
*** krtaylor has joined #openstack-qa10:33
openstackgerritRamakrishnan G proposed a change to openstack-dev/devstack: Add support for ironicclient bash completion  https://review.openstack.org/10924410:35
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-postgres-full 109201,1,6211e4910:37
openstackgerritChristian Berendt proposed a change to openstack-dev/devstack: Remove configuration parameter EXTRA_BAREMETAL_OPTS  https://review.openstack.org/10924510:38
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: Neutron default tenant network type vxlan  https://review.openstack.org/10556910:39
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: multi-node dg setup  https://review.openstack.org/10604310:39
openstackgerritChristian Berendt proposed a change to openstack-dev/devstack: Remove configuration parameter Q_DHCP_EXTRA_DEFAULT_OPTS  https://review.openstack.org/10924710:48
openstackgerritChristian Berendt proposed a change to openstack-dev/devstack: Remove configuration parameter Q_SRV_EXTRA_DEFAULT_OPTS  https://review.openstack.org/10924810:50
openstackrecheckConsole logs not available after 13:21s for gate-nova-pep8 107615,2,c410b7110:51
openstackgerritChristian Berendt proposed a change to openstack-dev/devstack: Remove configuration parameter EXTRA_OPTS  https://review.openstack.org/10924910:54
*** mestery has joined #openstack-qa10:56
*** mestery_ has quit IRC10:56
openstackgerritSean Dague proposed a change to openstack-dev/grenade: move from screen dumping to using worlddump tool  https://review.openstack.org/10912310:57
openstackgerritSean Dague proposed a change to openstack-dev/grenade: first pass of integrating javelin2 to grenade  https://review.openstack.org/9731710:57
openstackgerritSean Dague proposed a change to openstack-dev/grenade: Only upgrade services we are actually running  https://review.openstack.org/10912210:57
openstackgerritSean Dague proposed a change to openstack-dev/grenade: provide breathing room for service shutdown  https://review.openstack.org/10925010:57
openstackgerritSean Dague proposed a change to openstack-dev/grenade: make grenade logdir creation work on clean system  https://review.openstack.org/10925110:57
*** harlowja_away has quit IRC10:59
openstackgerritSunil G proposed a change to openstack/tempest: Fixes incorrect assertion check  https://review.openstack.org/10923710:59
*** Poornima_ has joined #openstack-qa10:59
*** k4n0 has quit IRC11:01
openstackgerritSean Dague proposed a change to openstack-infra/devstack-gate: address bashate style violations  https://review.openstack.org/10126811:02
openstackgerritSean Dague proposed a change to openstack-infra/devstack-gate: add tox to the tree for doing bashate checking  https://review.openstack.org/10126711:02
*** mestery_ has joined #openstack-qa11:02
*** mestery has quit IRC11:03
openstackrecheckConsole logs not available after 13:22s for check-tempest-dsvm-full 106038,3,078c6fb11:04
*** yfried_ has joined #openstack-qa11:04
*** yfried has quit IRC11:04
*** ildikov has joined #openstack-qa11:05
*** Poornima_ has quit IRC11:14
*** liusheng has quit IRC11:16
*** jyuso has quit IRC11:17
*** chandankumar has quit IRC11:17
openstackrecheckConsole logs not available after 13:22s for gate-taskflow-pep8 108572,13,a6b792911:18
*** k4n0 has joined #openstack-qa11:18
openstackgerritvarun kumar yadav proposed a change to openstack/tempest: Router  API Tests Enhancements  https://review.openstack.org/10801011:20
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-neutron-heat-slow 105069,15,15b3d6411:31
*** chandankumar has joined #openstack-qa11:32
*** Poornima_ has joined #openstack-qa11:40
openstackgerritYair Fried proposed a change to openstack/tempest: Fix manager.status_timeout exception  https://review.openstack.org/10354711:41
*** dims has joined #openstack-qa11:44
openstackrecheckConsole logs not available after 13:28s for gate-tempest-dsvm-full 91081,48,f96b1f811:44
openstackgerritSimeon Monov proposed a change to openstack/tempest: VPN/Firewall integration test scenario  https://review.openstack.org/10647311:48
openstackgerritSunil G proposed a change to openstack/tempest: Fixes incorrect assertion check  https://review.openstack.org/10925811:54
*** tosky has quit IRC11:55
*** sdmonov has quit IRC11:56
*** nmagnezi has quit IRC11:58
*** yfried__ has joined #openstack-qa11:58
openstackrecheckConsole logs not available after 13:28s for gate-nova-python26 108879,1,283e96711:58
*** tkammer has quit IRC11:59
*** cdent has joined #openstack-qa11:59
*** k4n0 has quit IRC11:59
*** cmyster has quit IRC12:00
*** yfried has joined #openstack-qa12:00
*** yfried_ has quit IRC12:01
openstackgerritA change was merged to openstack-dev/devstack: Add SERVICE_TOKEN to the sample local.conf  https://review.openstack.org/10862812:01
*** markus_z has joined #openstack-qa12:03
*** yfried__ has quit IRC12:03
*** jhenner has quit IRC12:04
*** afazekas_ has quit IRC12:07
*** nmagnezi has joined #openstack-qa12:08
*** tosky has joined #openstack-qa12:08
*** nosnos has quit IRC12:09
*** gszasz has quit IRC12:10
openstackrecheckConsole logs not available after 13:27s for check-grenade-dsvm-icehouse 101135,3,0b435a912:12
*** jhenner1 has joined #openstack-qa12:15
*** gszasz has joined #openstack-qa12:15
*** Longgeek has quit IRC12:16
*** Longgeek has joined #openstack-qa12:16
*** vrovachev has joined #openstack-qa12:16
*** Longgeek_ has joined #openstack-qa12:17
*** jhenner has joined #openstack-qa12:18
vrovachevsdague: mtreinish: masayukig: Hello, guys, please, review my patch in own time: https://review.openstack.org/#/c/71259/12:18
*** jhenner1 has quit IRC12:18
*** adalbas has joined #openstack-qa12:18
*** jhenner has quit IRC12:20
*** jhenner1 has joined #openstack-qa12:20
*** Longgeek has quit IRC12:20
*** cdent_ has joined #openstack-qa12:21
*** jhenner has joined #openstack-qa12:21
*** cdent has quit IRC12:22
*** cdent_ is now known as cdent12:22
*** yamahata has joined #openstack-qa12:23
*** afazekas_ has joined #openstack-qa12:24
*** jhenner1 has quit IRC12:24
*** dustins has joined #openstack-qa12:24
*** dustins has quit IRC12:25
*** dustins has joined #openstack-qa12:25
*** oomichi has quit IRC12:25
openstackrecheckConsole logs not available after 13:25s for gate-tempest-dsvm-full 105110,8,1b2be4912:26
*** jamielennox|away has quit IRC12:26
*** jamielennox|away has joined #openstack-qa12:29
*** ArxCruz has joined #openstack-qa12:34
*** aysyd has joined #openstack-qa12:35
*** tkammer has joined #openstack-qa12:36
*** nmagnezi has quit IRC12:36
openstackrecheckConsole logs not available after 13:26s for check-tempest-dsvm-neutron 109178,2,e98300e12:39
*** Anju has quit IRC12:39
*** ekhugen has joined #openstack-qa12:40
*** ekhugen has quit IRC12:40
*** ekhugen has joined #openstack-qa12:40
*** ekhugen has joined #openstack-qa12:41
kashyapafazekas_, Hi, I was away for last few days, I see that you're hitting this telemetry notification bug more w/ qpid -- https://bugs.launchpad.net/bugs/133675512:41
*** Poornima_ has quit IRC12:42
*** rfolco has joined #openstack-qa12:43
*** bkopilov has quit IRC12:43
openstackgerritSunil G proposed a change to openstack/tempest: Fixes incorrect assertion check  https://review.openstack.org/10925812:44
*** mflobo has joined #openstack-qa12:44
*** dmorita has quit IRC12:45
afazekas_kashyap: not just with qpid, but neutron+qpid+full-parallel hits these kind of issue more frequently than any other12:47
kashyapafazekas_, Yes, I see some further analysis there by salv-orlando and Gordon Chung12:47
afazekas_kashyap: probbaly qpid driver just have different latency, and it makes the issue more visible12:48
salv-orlandokashyap: I am looking the neutron logs to see if there is something which might trigger the ceilometer failure12:48
kashyapafazekas_, Yeah, I haven't checked after that day, just trying to see if this 'critical' bug will end up languishing too12:49
kashyapsalv-orlando, Ah, thank you.12:49
salv-orlandokashyap: from the look of it this one of those bugs like the “ssh timeout”. It just tells something which was expected to happen did not happen, but go figure out the actual root cause12:49
kashyapYeah, IIRC, eglynn from Ceilometer team said something along those lines12:50
*** nmagnezi has joined #openstack-qa12:53
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-postgres-full 105635,15,16c1bac12:53
openstackgerritEoghan Glynn proposed a change to openstack/tempest: Skip telemetry_notification_api test due to bug 1336755  https://review.openstack.org/10801112:54
*** mestery_ is now known as mestery12:55
*** bnemec has joined #openstack-qa12:56
*** dmellado has joined #openstack-qa12:56
*** eglynn_ is now known as eglynn-brb13:01
*** markus_z has quit IRC13:03
*** zigo has quit IRC13:03
*** zigo has joined #openstack-qa13:04
openstackgerritYair Fried proposed a change to openstack/tempest: Fix manager.status_timeout exception  https://review.openstack.org/10354713:06
openstackrecheckConsole logs not available after 13:22s for gate-nova-python26 101901,17,cb4107413:06
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: add support for channel config being nested  https://review.openstack.org/10870513:09
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: add support for an external message catalog  https://review.openstack.org/10870613:09
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: add message strings to recheckwatchbot.yaml  https://review.openstack.org/10928013:09
*** chandankumar has quit IRC13:10
openstackgerritnswarnkar@gmail.com proposed a change to openstack/tempest: test_neutron_resources to read template's values  https://review.openstack.org/10928313:11
*** eglynn-brb is now known as eglynn13:11
*** nmagnezi has quit IRC13:12
*** tkammer has quit IRC13:12
kashyapafazekas_, There's a temporary workaround to skip that -- https://review.openstack.org/#/c/10801113:12
kashyapthat = telemetry notification issue13:13
*** mriedem has joined #openstack-qa13:13
afazekas_kashyap: you can use a custom regexp13:13
*** dims has quit IRC13:13
*** dims has joined #openstack-qa13:14
*** mwagner_lap has quit IRC13:14
*** markus_z has joined #openstack-qa13:18
*** mestery_ has joined #openstack-qa13:18
*** Sudhi has quit IRC13:19
sdaguekashyap: can you respin that to not drop the nova skip?13:20
openstackrecheckConsole logs not available after 13:22s for gate-trove-python26 109219,1,e417f8f13:20
*** afazekas_ has quit IRC13:20
openstackrecheckopenstack/nova change: https://review.openstack.org/107143 failed because of: gate-grenade-dsvm-partial-ncpu: unrecognized error, gate-nova-python26: https://bugs.launchpad.net/bugs/1330763 https://bugs.launchpad.net/bugs/133973913:20
sdaguebecause both skips should be there13:20
*** mestery has quit IRC13:20
kashyapsdague, Looking. eglynn wrote the above change (108011), if you're referring to that13:20
*** jhenner has quit IRC13:22
*** yfried has quit IRC13:22
kashyap(I just had an old similar change -- 104516 -- that was abandoned after a discussion here, assuming a certain revert fixed it.)13:22
sdagueyeh, mostly I don't want the nova skip removed13:23
sdaguewe can add the other skip13:23
*** lbragstad has joined #openstack-qa13:23
openstackgerritSean Dague proposed a change to openstack-dev/grenade: move from screen dumping to using worlddump tool  https://review.openstack.org/10912313:24
openstackgerritSean Dague proposed a change to openstack-dev/grenade: first pass of integrating javelin2 to grenade  https://review.openstack.org/9731713:24
*** gfidente has joined #openstack-qa13:25
kashyapsdague, Understood. Just wondering, I don't think it's polite to respin someone else patch?13:26
sdagueI think it's fine13:27
kashyapOkay.13:28
openstackgerritEoghan Glynn proposed a change to openstack/tempest: Skip telemetry_notification_api test due to bug 1336755  https://review.openstack.org/10801113:28
*** tosky has quit IRC13:28
eglynnsdague: done :)13:28
*** psharma has quit IRC13:28
sdagueeglynn: oh, woops, I hit the review button the patch while you were uploading :)13:29
sdagueok, now to change back to +213:29
eglynnsdague: thank you sir!13:29
kashyapCool.13:29
*** hockeynut has joined #openstack-qa13:30
*** tosky has joined #openstack-qa13:30
sdagueeglynn: thanks for fast respin13:30
eglynnsdague: np!13:30
*** Obulpathi has joined #openstack-qa13:32
*** afazekas_ has joined #openstack-qa13:33
openstackrecheckConsole logs not available after 13:23s for gate-trove-python26 108972,5,f00458e13:34
*** oomichi has joined #openstack-qa13:34
*** jecarey has joined #openstack-qa13:34
anteayajhuthmann: yay!!13:34
*** jhenner has joined #openstack-qa13:35
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: multi-node dg setup  https://review.openstack.org/10604313:36
* afazekas_ away for 5h13:37
mtreinishandreaf: I pushed through https://review.openstack.org/109140 last night which should fix the duplicate option exception13:39
*** oomichi has quit IRC13:40
*** afazekas_ has quit IRC13:42
*** yfried has joined #openstack-qa13:43
*** Ananci has joined #openstack-qa13:46
*** jhenner has quit IRC13:47
*** hockeynut has quit IRC13:47
*** hockeynu_ has joined #openstack-qa13:47
openstackrecheckConsole logs not available after 13:23s for gate-heat-pep8 109012,2,689591a13:47
*** jhenner has joined #openstack-qa13:47
*** jhenner has quit IRC13:49
*** jhenner has joined #openstack-qa13:49
*** jhenner has quit IRC13:49
andreafmtreinish: thanks it works now13:50
andreafmtreinish: I'm try to use subunit2sql but my understanding of alembic is ~0 so it's not working. I though the migrations would magically create the DB and schema but that doesn't seem to be the case13:51
mtreinishandreaf: you need to put the db connection info in the alembic.ini13:52
*** jhenner1 has joined #openstack-qa13:52
mtreinishthen run alembic upgrade head13:52
andreafmtreinish: I tried creating an empty DB by  sqlite3 subunit.sqlite ""13:52
andreafmtreinish: ah13:52
*** jhenner1 has quit IRC13:52
andreafok13:52
*** bdperkin_gone has quit IRC13:53
mtreinishandreaf: yeah that's one of the things I want to make more clear with future changes13:53
mtreinishalthough probably just putting it in the readme is fine for now...13:53
*** jhenner has joined #openstack-qa13:53
openstackgerritYair Fried proposed a change to openstack/tempest: Fix manager.status_timeout exception  https://review.openstack.org/10354713:54
andreafmtreinish: is sqlite supposed to work?13:55
mtreinishandreaf: it did for me in local tests13:55
andreafmtreinish:     "No support for ALTER of constraints in SQLite dialect")13:55
mtreinishandreaf: oh, yeah I was only testing the first 2 migrations on sqlite13:55
*** bdperkin_gone has joined #openstack-qa13:55
*** jhenner has quit IRC13:55
mtreinishyeah alter doesn't exist on sqlite so I need to special case that migration13:56
mtreinishand do it manually13:56
mtreinishandreaf: file a bug :)13:56
*** jhenner has joined #openstack-qa13:56
*** tkammer has joined #openstack-qa13:56
*** jhenner has joined #openstack-qa13:56
*** nmagnezi has joined #openstack-qa13:56
*** mkoderer has quit IRC13:57
*** bdperkin_gone is now known as bdperkin13:57
*** mkoderer_bot has quit IRC13:58
*** bdperkin has quit IRC13:58
*** bdperkin has joined #openstack-qa13:58
andreafmtreinish: https://bugs.launchpad.net/subunit2sql/+bug/134820213:58
yfriedmtreinish: in scenario.manager._status_timeout()13:59
*** mkoderer has joined #openstack-qa13:59
yfriedmtreinish: method was written with nova-server in mind, am I right?13:59
mtreinishandreaf: ok, yeah I can push a fix for that today. Unless you wanted to give it a shot?14:00
mtreinishyfried: I'm not sure I'd have to look at the code. One sec14:00
yfriedmtreinish: it seems designed to specifically check VM's status, with the hard-coded timeouts, and exception raised14:00
*** mkoderer has quit IRC14:01
openstackrecheckConsole logs not available after 13:23s for gate-glance-python27 109230,1,b972cee14:01
*** mkoderer has joined #openstack-qa14:01
yfriedmtreinish: I want to change to be more generic. for instance - it's totally incompatible with neutron resources14:03
*** pcrews has joined #openstack-qa14:03
*** mkoderer has quit IRC14:03
mtreinishyfried: no that actually looks to be pretty generic for most of the python clients. The issue is the neutron client differs from glance,cinder,nova,keystone,etc in behavior14:04
mtreinishyfried: anyway this kind of issue is going away with: https://blueprints.launchpad.net/tempest/+spec/tempest-client-scenarios14:04
mtreinishso I wouldn't bother pushing a fix until we finish that migration14:04
yfriedmtreinish: that's why I'm asking14:04
*** dkranz has joined #openstack-qa14:05
yfriedmtreinish: side-note - how can I join that effort?14:05
yfrieddkranz: ^14:06
mtreinishyfried: talk to andreaf, masayukig, or dmorita they're the ones who are driving that forward14:07
openstackgerritAdam Gandelman proposed a change to openstack-dev/devstack: Set Tempest feature flags for Ironic  https://review.openstack.org/10930014:07
mtreinishI believe there is an etherpad in the spec that links to the tracking. The tests files are being split up one at a time14:07
mtreinishbut double check with them14:07
*** bdperkin has quit IRC14:08
*** sdmonov has joined #openstack-qa14:08
*** rbak has joined #openstack-qa14:08
*** samuelbercovici has joined #openstack-qa14:09
andreafyfried: https://etherpad.openstack.org/p/tempest-client-scenarios14:09
*** bdperkin_gone has joined #openstack-qa14:09
*** bdperkin_gone is now known as bdperkin14:10
*** bdperkin has joined #openstack-qa14:10
yfriedmtreinish: anyway about the method - take a look at the timeouts, it's designed specifically to wait for a VM creating. if I want to use it to verify port status or floatingip status, it's a14:10
yfriedmtreinish: lot of time to wait for a very quick change14:10
yfriedmtreinish: this might be a reason lbaas takes a lot of time at the gate14:10
andreafmtreinish: for now I switched to mysql :P but I can try to see how to do the index migration in sqlite14:11
*** nmagnezi has quit IRC14:11
sdaguemtreinish: given your data requirements for subunit2sql, I'd actually remove the sqlite parts entirely14:12
sdagueit's just not going to be useful for anything14:12
sdagueand causes issues with migrations14:12
yfriedandreaf: should I just put my name on a module?14:12
andreafyfried: the base class is merged but it does not contain much, if you need something more, like the waiters, you need to base your patch on https://review.openstack.org/#/c/107562/14:13
sdaguejogo: about to go for a ride, when I get back, I need to figure out how to get javelin to actually log to stdout because we're back in another oslo log blackhole failure here14:13
andreafyfried: yes you can take one and place a review link once you have it14:13
*** bdperkin has quit IRC14:14
yfriedandreaf: I'm putting my name on the network tests (network|secgroup_basic_ops) I'll start working on it sunday. ok?14:14
mtreinishandreaf: you would just need to special case the offending migration(s) in sqlite and write raw sql to manually create a new table with the altered constraint and copy the data over14:14
mtreinishsdague: yeah, running it in production, sqlite doesn't make much sense. But I thought it would be useful for testing14:15
*** bdperkin has joined #openstack-qa14:15
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 109240,1,cdaa90d14:15
*** zz_dimtruck is now known as dimtruck14:15
*** bdperkin has quit IRC14:15
*** bdperkin has joined #openstack-qa14:15
mtreinishsdague: there are no sqlite specific things in the code now (except for the default in the alembic.ini)14:15
sdaguemtreinish: honestly, I think you'll find you are just having to do a lot of work arounds for it14:15
sdaguemuch better to just tell people they actually have to have mysql installed on their laptop14:15
sdagueit will test a ton faster as well14:16
mtreinishsdague: that's a fair point14:17
mtreinishI need to update some of the documentation anyway so I can add that in there when I do that14:17
andreafyfried: sure sounds fine, just make sure you lock only the tests you're working on ;)14:19
*** mkoderer has joined #openstack-qa14:20
yfriedandreaf: I'm gonna work on them one at a time, I just think that since I wrote most of them, it's better if I'm doing the work there. If someone wants to work on them specifically, or if I'm blocking you guys somehow, just let me know.14:21
andreafmtreinish: it looks like it may be a good idea to have you test results and database on different disks14:21
andreafyfried: fair enough :)14:21
andreafyfried: thanks for contributing on this!!14:22
andreafs/on/to14:22
yfriedandreaf: I was out of touch with upstream for a few weeks, and that's seems like a great place to jump back in.14:22
mtreinishyfried: no, it takes in an arbitrary resource type from the clients and an id. I don't see any hardcoded server create timeouts in _status_timeout()14:22
mtreinishyfried: do you have a link with an example of what you're talking about14:22
mtreinishandreaf: I had issues on one machine because the disk was slow. But, it seems to be reasonably quick elsewhere14:23
yfriedmtreinish: https://github.com/openstack/tempest/blob/master/tempest/scenario/manager.py#L29214:23
yfriedmtreinish: https://github.com/openstack/tempest/blob/master/tempest/scenario/manager.py#L29214:23
mtreinishyfried: that just a CONF variable for timeout duration and wait interval14:24
*** oomichi has joined #openstack-qa14:24
mtreinishcompute is there so to not overload it14:24
mtreinishso we don't have a bunch of timeout variables in the conf file for no reason14:24
yfriedmtreinish: we could use this as default, but as I said - waiting 3min, for port status seems like overkill14:25
mtreinishyfried: if port status is quick it will come back after the first retry14:25
mtreinishit only waits 3 min if something fails14:25
yfriedmtreinish: if port status fails then we wait 3min for a failure14:25
*** jgrimm has joined #openstack-qa14:26
mtreinishyfried: well until we move to the tempest clients, then if it fails with a bad status code it raises an exception and fails fast.14:26
*** markus_z has quit IRC14:27
mtreinishit's also tunable (being a conf value) I just think it's crazy to have a separate timeout for each kind of resource14:27
yfriedmtreinish: really? nice. could you send me a link?14:27
mtreinishyfried: that's how the rest client works, if there is a 4xx or 5xx status code it raises an exception14:28
*** cpallares has joined #openstack-qa14:28
*** rwsu has joined #openstack-qa14:28
mtreinishif the timeout thing doesn't handle that the test fails...14:28
*** markus_z has joined #openstack-qa14:28
yfriedmtreinish: https://review.openstack.org/#/c/102700/14:28
openstackrecheckConsole logs not available after 13:22s for gate-swift-pep8 109243,1,b88331c14:29
yfriedmtreinish: are you saying this patch is redundant once the move is complete?14:29
andreafmtreinish: yay I got my first subunit2sql DB14:30
yfriedmtreinish: could you point me to the code you described? we have a couple of patches regarding port-status in mind, and I don't want to do unnecessary job14:30
mtreinishyfried: well, TBH it's redundant now too, jordanP is right in his comment14:30
mtreinishandreaf: cool, congrats14:31
mtreinishandreaf: was it being slow for you, because it wasn't too bad when I tested it on a rax node14:31
*** mestery_ is now known as mestery14:31
mtreinishyfried: this is what I'm talking about: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/common/rest_client.py#n43314:32
andreafmtreinish: it was on my workstation, but I didn't time it - it took a few minutes for a full tempest run14:33
yfriedmtreinish: well I had to write it because of the neutron-client issue you mentioned. I was going to use eezhova solution instead https://github.com/openstack/tempest/blob/master/tempest/scenario/test_load_balancer_basic.py#L30014:33
andreafmtreinish: I'll try again measuring how long it takes14:34
mtreinishandreaf: hmm, yeah I had perf issues on my personal workstation, but I'm having some issues with the perf of the main disk on that machine14:34
mtreinishso I assumed it was just that14:34
andreafmtreinish: btw the run_time is a sum of all tests... and it does not consider the parallelism - is that on purpose?14:34
mtreinishespecially after it took like 1.5mins for a tempest run on the rax node14:35
*** dkranz has quit IRC14:35
mtreinishandreaf: well, my assumption was that it was a combined stream for all the workers14:35
mtreinishso it was the time it took for that run to finish14:35
mtreinishthe inidvidual test_run rows have time data and the worker info is stored, so if you wanted per worker run time it would be easy to get14:36
andreafmtreinish: ok that can be added14:38
mtreinishandreaf: That would actually be a good helper method to have, return a dict of per worker run_times for run_id X14:38
*** mwagner_lap has joined #openstack-qa14:38
mtreinishandreaf: oh, nm I misunderstood what you meant, the run_time is misleading because it's summing across worker boundaries14:39
andreafmtreinish: are you running from file or pipe? I'm doing testr last --subunit | subunit-1to2 | subunit2sql ...14:39
mtreinishso the value is higher than what it actually took...14:39
mtreinishandreaf: I was doing both, to test that they both worked :)14:39
andreafmtreinish: yes that's what I meant but it's a valid in terms of "cpu time"14:40
mtreinishthe speed was about the same14:40
openstackgerritSteven Hardy proposed a change to openstack/tempest: orchestration: tolerate NotFound in wait_for_stack_status  https://review.openstack.org/10930814:40
openstackgerritSteven Hardy proposed a change to openstack/tempest: orchestration api test_stacks wait for deletion  https://review.openstack.org/10930914:40
andreafmtreinish: it may be that the worker metadata is not always there so in that case the run_time is fine14:41
mtreinishandreaf: yeah I guess it's more a matter of what we should be storing. I'm fine with it for now, because the worker-id is just metadata, so there is no guarantee that it's right14:41
mtreinishandreaf: heh, yeah14:41
andreafmtreinish: so there are several things that can be refined as we start using it , but it looks like a great start, thanks!14:42
openstackrecheckConsole logs not available after 13:22s for gate-swift-pep8 109242,1,fa92b4514:42
*** eglynn-officeafk has quit IRC14:42
mtreinishandreaf: cool, thanks for testing it out14:42
mtreinishandreaf: patches are always welcome :)14:42
andreafmtreinish: well I'm starting to integrate it in my downstream testing pipeline, so if I find issues I'll also try to fix them14:43
mtreinishcool14:43
mtreinishandreaf: I'm trying to do the same with infra: https://review.openstack.org/10800314:44
mtreinishI need to pick that up again soon14:44
andreafmtreinish: yes I've seen that it would be really really nice to have14:44
yfriedmtreinish: what if the status is not error, but simply wrong?14:47
*** andreaf has quit IRC14:47
mtreinishyfried: what like a 204 instead of 200?14:47
*** lbragstad has quit IRC14:48
mtreinishthe response status check should catch that. dkranz is working on moving those into all the clients directly instead of the tests14:48
yfriedmtreinish: no, https://bugs.launchpad.net/neutron/+bug/133470814:48
*** dkranz has joined #openstack-qa14:48
*** nati_ueno has joined #openstack-qa14:49
*** tkammer has quit IRC14:49
*** yamahata has quit IRC14:49
mtreinishyfried: well in that case it will do the full timeout interval14:50
*** oomichi has quit IRC14:50
yfriedmtreinish: but the check is still in order? right?14:50
*** eglynn-officeafk has joined #openstack-qa14:50
*** oomichi has joined #openstack-qa14:50
mtreinishyfried: I'm not following. Which check?14:52
*** oomichi has quit IRC14:54
*** oomichi_ has joined #openstack-qa14:54
*** Obulpathi has quit IRC14:55
*** markmcclain has joined #openstack-qa14:55
openstackrecheckConsole logs not available after 13:22s for gate-nova-python26 103109,7,4dadc4014:56
yfriedport status14:56
*** oomichi_ has quit IRC14:56
yfriedmtreinish: ^14:56
openstackrecheckopenstack/ceilometer change: https://review.openstack.org/107942 failed because of: gate-ceilometer-python26: https://bugs.launchpad.net/bugs/133619314:56
*** SridharG has quit IRC14:57
mtreinishyfried: are you asking if you should still add it? Sorry, I'm still not sure what your asking.14:58
*** lbragstad has joined #openstack-qa14:59
yfriedmtreinish: I'm asking if this check isn't handled by tempest clients as well14:59
mtreinishyfried: you'd have to add the check to the tempest neutron client if it's not there. But that bug was a failure to transition state15:00
mtreinishthat won't raise an exception until the timeout is reached assuming you used similar wait for status code15:01
*** chandankumar has joined #openstack-qa15:02
yfriedmtreinish: ok, tnx. sorry for bugging your15:03
yfriedyou15:03
yfriedmtreinish: btw, doing flake8 gets me a lot of these :15:04
yfriedtempest/scenario/manager.py:32:1: H302  import only modules.'from tempest import auth' does not import a module15:04
yfriedfrom tempest import auth15:04
yfriedmtreinish: but the gate doesn't have a problem with that. why is that?15:04
*** hockeynu_ has quit IRC15:05
openstackgerritYair Fried proposed a change to openstack/tempest: Adds status check for FloatingIP in scenarios  https://review.openstack.org/10270015:05
mtreinishyfried: I'd check your locally installed version of hacking, and flake815:05
mtreinishyou probably have an older version installed15:05
mtreinishand the checks are different15:05
*** hockeynut has joined #openstack-qa15:05
yfriedflake8==2.1.015:06
yfriedpyflakes==0.8.115:06
yfriedhacking==0.9.215:06
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Enable E128 ignore E129  https://review.openstack.org/9890915:07
openstackgerritMatthew Treinish proposed a change to openstack/tempest: Enable H407,H305,H307,E122 ignore E123  https://review.openstack.org/9891815:07
mtreinishyfried: what about the pep8 version15:08
mtreinishthose look correct to me15:08
yfriedpep8==1.5.615:08
mtreinishyfried: hmm dunno, auth is a module so I'm not sure what is going on15:09
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full-icehouse 106043,6,6c6711115:09
yfriedmtreinish: they all are (I'm getting a lot of those)15:09
mtreinishyfried: http://git.openstack.org/cgit/openstack-dev/hacking/tree/hacking/checks/imports.py#n2815:10
mtreinishthat's the check code15:10
openstackgerritAdam Gandelman proposed a change to openstack-dev/devstack: Set Tempest feature flags for Ironic  https://review.openstack.org/10930015:11
*** sballe has joined #openstack-qa15:11
mtreinishyfried: actually http://git.openstack.org/cgit/openstack-dev/hacking/tree/hacking/checks/imports.py#n106 is the relavent bit15:12
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for cinder timeout bug 1348204  https://review.openstack.org/10931615:14
mriedemmtreinish: jogo: sdague: ^15:14
mriedemnot great, but i need to get test class/name in the scenario timeout exceptions15:15
*** Madan has quit IRC15:15
*** lbragstad has quit IRC15:19
*** bkopilov has joined #openstack-qa15:21
mtreinishmriedem: found the issue: http://logs.openstack.org/15/109115/1/check/check-tempest-dsvm-full/168a5dd/logs/screen-c-api.txt.gz?#_2014-07-24_01_00_22_935 cinder returned a 400 on the delete (I haven't found why yet)15:22
mtreinishalso tempest should have blown up when it got an unexpected 40015:22
mtreinishso there's probably a tempest side bug15:22
mtreinishmriedem: although, I guess we can't query cinder retuning 400 on a delete request :)15:23
mriedemmtreinish: yeah there are several issues15:23
openstackrecheckConsole logs not available after 13:29s for check-tempest-dsvm-postgres-full 109240,2,8163faa15:23
mriedemmtreinish: http://logs.openstack.org/15/109115/1/check/check-tempest-dsvm-full/168a5dd/console.html#_2014-07-24_01_07_09_65815:24
mtreinishmriedem: oh, it didn't blow up because it's a scenario test and used python-cinderclient sigh...15:24
*** dkorn has quit IRC15:25
mtreinishyeah it's supposed to wait until the volume status is available before making the delete call (it even says it in the logs right above the delete call)15:27
mtreinishso that's probably the bug...15:27
mriedemmtreinish: in the scenario test15:27
mriedemi can fix that15:28
*** lbragstad has joined #openstack-qa15:28
*** danfinch has joined #openstack-qa15:28
mtreinishmriedem: yeah in that scenario test. My guess is that it's probably a sync bug in the recent addCleaup conversion, but it might not be15:30
*** xianghuihuihui has quit IRC15:30
*** xianghuihuihui has joined #openstack-qa15:30
openstackrecheckConsole logs not available after 13:22s for check-grenade-dsvm 108010,3,6a1b0f215:37
mriedemmtreinish: the error started showing up on 7/15, https://review.openstack.org/62101 merged on 7/3 so not the issue15:38
mtreinishmriedem: hmm, ok15:39
*** cpallares has quit IRC15:48
*** jlanoux has quit IRC15:48
openstackrecheckConsole logs not available after 13:24s for gate-trove-python27 109219,3,a1fad9215:50
*** nati_ueno has quit IRC15:50
*** mlavalle has joined #openstack-qa15:52
mriedemmtreinish: i think the deleting thing is a red herring, nova_volume_detach is waiting for the volume to be in 'available' state but it's detaching for a long time, i think we hit the timeout waiting for 'available' status and then on tearDown try to delete the volume15:52
mriedembut the volume isn't ready to be deleted, so we get the 2nd error on invalid volume state on the delete request15:52
mtreinishmriedem: ahh, ok that makes sense. But we should raise/fail on a timeout exception if we're when the volume doesn't go into available if we're waiting on that at some point15:54
mriedemmtreinish: we do15:55
mriedemthat's the first failure in traceback15:55
mriedemthen there are subsequent errors on teardown15:55
mriedemtaking us on this wild goose chase15:55
mriedemhence https://review.openstack.org/#/c/109316/ isn't great atm15:56
mriedemi'm going to push a tempest patch to add the test case info to the timeout exception in the scenario manager, like we do in waiters for the api tests15:56
openstackgerritSteve Baker proposed a change to openstack-dev/devstack: Ironic and baremetal install diskimage-builder with pip  https://review.openstack.org/9225715:56
openstackgerritSteve Baker proposed a change to openstack-dev/devstack: A service to install diskimage-builder  https://review.openstack.org/9225615:56
openstackgerritSteve Baker proposed a change to openstack-dev/devstack: Install heat-cfntools and heat-templates repos  https://review.openstack.org/9225815:56
openstackgerritSteve Baker proposed a change to openstack-dev/devstack: WIP enable building test image  https://review.openstack.org/9561715:56
openstackgerritSteve Baker proposed a change to openstack-dev/devstack: Build a custom fedora image for tempest heat-slow  https://review.openstack.org/9226015:56
openstackgerritSteve Baker proposed a change to openstack-dev/devstack: Utility functions for building test images  https://review.openstack.org/9561615:56
mriedemthen we can fingerprint on the timeout for the specific test15:56
mtreinishmriedem: oh, ok I was reading the wrong stack trace :)15:57
mriedemmtreinish: the timing of this lines up https://review.openstack.org/#/c/99782/ but it's not on the lvm driver15:57
mriedemso probably not the issue15:57
*** arnaud has joined #openstack-qa15:57
*** arnaud__ has joined #openstack-qa15:57
*** evgenyf has quit IRC15:58
mtreinishmriedem: yeah adding that will be useful for debug15:58
*** lbragstad has quit IRC15:58
mtreinishmriedem: although I wonder how much clearer things will become when https://blueprints.launchpad.net/tempest/+spec/tempest-client-scenarios is done?15:59
*** samuelbercovici has quit IRC16:01
openstackrecheckConsole logs not available after 13:24s for check-tempest-dsvm-full 103547,6,3f9b4ef16:04
mriedemmtreinish: i talked to jay bryant from the cinder team and he said it's probably a nova keymgr issue16:04
mriedemso it's not timing out b/c it's slow, it's timing out because the attach fails silently16:04
*** markmcclain has quit IRC16:05
mriedemwhich would probably explain why the 7/15 merges in tempest/cinder aren't showing me anytihng16:05
mtreinishmriedem: oh, cross project fun...16:06
mriedeminbreeding16:06
mtreinishmriedem: what about 7/15 merges in nova?16:07
*** dmellado has quit IRC16:07
mriedemmtreinish: digging through those now16:07
mriedemhttps://github.com/openstack/nova/commit/4a77e75b6f9b40129afb21ce8802cff5893f8de9 comes to mind16:07
*** adalbas has quit IRC16:09
*** gfidente has quit IRC16:09
mtreinishmriedem: hmm, it did have a lot of rechecks16:09
*** yfried has quit IRC16:10
*** andreaf has joined #openstack-qa16:11
*** andreaf has quit IRC16:11
*** andreaf has joined #openstack-qa16:11
openstackgerritRob Crittenden proposed a change to openstack-dev/devstack: Move SERVICE_PROTOCOL from stack.sh to stackrc  https://review.openstack.org/10933516:12
*** andreaf_ has joined #openstack-qa16:13
Longgeek_Who know the cause of this error? vif_type=binding_failed16:15
*** andreaf_ has quit IRC16:16
*** andreaf_ has joined #openstack-qa16:16
*** andreaf_ has quit IRC16:16
*** andreaf_ has joined #openstack-qa16:17
*** andreaf has quit IRC16:17
openstackrecheckConsole logs not available after 13:22s for gate-keystone-pep8 106096,4,b11002c16:18
*** SridharG has joined #openstack-qa16:18
*** mestery has quit IRC16:22
*** mestery has joined #openstack-qa16:23
*** mlavalle has quit IRC16:23
*** jlanoux has joined #openstack-qa16:23
*** adalbas has joined #openstack-qa16:23
*** Longgeek_ has quit IRC16:26
*** lbragstad has joined #openstack-qa16:29
*** Longgeek has joined #openstack-qa16:29
openstackrecheckConsole logs not available after 13:22s for gate-nova-pep8 85594,14,35ab8b616:31
*** gfidente has joined #openstack-qa16:31
*** vrovachev has quit IRC16:32
*** lbragstad has quit IRC16:34
*** masayukig has quit IRC16:40
openstackgerritRavikumar Venkatesan proposed a change to openstack/tempest: test_tokens_nocatalog to identity v3 test_tokens  https://review.openstack.org/9317216:40
*** chandankumar has quit IRC16:44
*** lbragstad has joined #openstack-qa16:44
*** afazekas_ has joined #openstack-qa16:44
openstackrecheckConsole logs not available after 13:23s for gate-nova-pep8 109267,1,c51031716:44
*** masayukig has joined #openstack-qa16:46
*** safchain has quit IRC16:51
*** gfidente has quit IRC16:51
*** bnemec has quit IRC16:52
*** nati_ueno has joined #openstack-qa16:54
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for cinder timeout bug 1348204  https://review.openstack.org/10931616:54
mriedemmtreinish: ^ redid that query with the n-cpu trace16:55
*** atiwari has joined #openstack-qa16:56
*** bnemec has joined #openstack-qa16:56
*** rwsu has quit IRC16:58
openstackrecheckConsole logs not available after 13:38s for gate-ceilometer-python26 107942,4,746e43416:58
*** eglynn has quit IRC16:59
*** gszasz has quit IRC16:59
*** lbragstad has quit IRC16:59
*** gfidente has joined #openstack-qa16:59
*** rwsu has joined #openstack-qa16:59
*** lbragstad has joined #openstack-qa17:03
*** jordanP has quit IRC17:04
*** nati_uen_ has joined #openstack-qa17:05
*** nati_ueno has quit IRC17:05
*** aimon has joined #openstack-qa17:06
*** markus_z has quit IRC17:07
*** tosky has quit IRC17:10
*** lbragstad has quit IRC17:10
*** lbragstad has joined #openstack-qa17:11
openstackrecheckConsole logs not available after 13:24s for check-tempest-dsvm-full 109268,1,d290c8517:11
*** hockeynut has quit IRC17:12
jogosdague: ping when your back17:12
*** SridharG has quit IRC17:12
sdaguejogo: pong17:15
*** lbragstad has quit IRC17:16
jogosdague: going through your grenade patch series17:16
jogowhat user do you usually run as?17:16
jogoI use a user-data file that calls devstack/tooks/creatge-stack-user then runs it as stack17:16
sdaguewhatever user it runs as17:17
*** SridharG has joined #openstack-qa17:17
*** adalbas has quit IRC17:17
*** luqas has quit IRC17:17
jogoas long as it isn't root17:18
*** jlanoux has quit IRC17:19
jogoahh now up to the one I needed https://review.openstack.org/#/c/109122/2/grenade.sh17:19
sdagueno, I don't run as root17:19
*** andreaf_ has quit IRC17:20
jogothe rax image I use adds my ssh key to root by default17:20
jogoanyway cool17:20
*** andreaf_ has joined #openstack-qa17:20
jogojust wondering17:20
*** bnemec has quit IRC17:22
*** lbragstad has joined #openstack-qa17:23
openstackrecheckConsole logs not available after 13:25s for check-tempest-dsvm-postgres-full 109266,1,4b08a9a17:25
*** markmcclain has joined #openstack-qa17:26
*** markmcclain1 has joined #openstack-qa17:29
*** markmcclain has quit IRC17:31
*** harlowja has joined #openstack-qa17:35
openstackrecheckConsole logs not available after 13:22s for check-tempest-dsvm-postgres-full 103830,4,239d09517:39
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: Neutron default tenant network type vxlan  https://review.openstack.org/10556917:40
openstackgerritafazekas proposed a change to openstack-infra/devstack-gate: multi-node dg setup  https://review.openstack.org/10604317:40
*** adalbas has joined #openstack-qa17:41
*** johnthetubaguy is now known as zz_johnthetubagu17:42
*** jhenner has quit IRC17:44
openstackrecheckConsole logs not available after 13:24s for check-tempest-dsvm-postgres-full 109288,1,8cbbf9b17:52
openstackgerritRob Crittenden proposed a change to openstack-dev/devstack: Move SERVICE_PROTOCOL from stack.sh to stackrc  https://review.openstack.org/10933517:54
*** tosky has joined #openstack-qa17:59
dkranzmtreinish: I added a third bullet to the meeting agenda. At this point I am not sure whether it will be considered controversial or not.18:01
*** rbak has quit IRC18:01
openstackrecheckConsole logs not available after 13:22s for check-tempest-dsvm-postgres-full 103976,4,019d40f18:05
openstackgerritJoe Gordon proposed a change to openstack-dev/grenade: wip: create a javelin.log instead of using the tempest.log  https://review.openstack.org/10738018:11
openstackgerritMatt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for cinder timeout bug 1348204  https://review.openstack.org/10931618:12
openstackgerritAnna Shen proposed a change to openstack/tempest: Add Trove (database) Instance API Tests  https://review.openstack.org/10719318:16
*** nati_uen_ has quit IRC18:18
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-postgres-full 109257,2,bbdb45018:19
jogogot javelin2 check to fail locally with logs to stderr18:19
*** nati_ueno has joined #openstack-qa18:19
*** SridharG has quit IRC18:19
*** rbak has joined #openstack-qa18:19
jogomriedem: http://logs.openstack.org/54/105554/2/check/check-grenade-dsvm/8762173/console.html#_2014-07-24_05_44_42_48618:20
jogohave you seen that before?18:20
*** nati_ueno has quit IRC18:20
*** markmcclain1 has quit IRC18:21
mriedemjogo: https://review.openstack.org/#/c/109316/18:23
mriedemspent my entire day on it18:23
*** ildikov has quit IRC18:24
*** ildikov_ has joined #openstack-qa18:24
jogomriedem: sweet thanks18:25
mriedemjogo: lots of stuff in the bug report18:26
mriedemjogo: thinking this might be involved https://github.com/openstack/nova/commit/d19c75c19d2de8b20e82e6de9413ba53671ad7fb18:26
*** mestery has quit IRC18:27
*** amitpp has joined #openstack-qa18:27
*** mestery has joined #openstack-qa18:28
jogosdague: I think i found the, well a, javelin2 issue18:29
jogoin create it doesn't wait till the resources are fully created18:29
jogoso when we take down old the resources aren't there yet18:29
sdaguejogo: that's entirely possible18:29
*** nati_ueno has joined #openstack-qa18:30
jogosdague: as that is what I saw when running it locally18:30
*** mlavalle has joined #openstack-qa18:30
openstackrecheckConsole logs not available after 13:22s for gate-nova-python26 109178,3,f83546318:32
sdagueso the user creates should be synchronous18:33
sdagueI think the resource create path should be fine18:34
sdaguethe real issue is if yuo check too fast18:34
jogothe create instance isn't synchronous18:34
jogonova boot is in the middle when old is turned off18:34
jogosdague: at least thats what it looked like on my box18:35
sdaguejogo: oh, really?18:35
sdaguethat's interesting18:35
sdagueok... so I guess it's time to put the polling on that18:35
sdaguegood find18:35
jogosdague: yeah, nova CLI has a  boot --poll option which is nice18:36
jogosdague: maybe add --poll options to the tempest clients?18:36
jogoor equivalent18:36
sdaguewe have wait calls18:36
jogoyeah http://git.openstack.org/cgit/openstack/tempest/tree/tempest/cmd/javelin.py#n372 doesn't appear to to wait18:36
sdagueyeh, we don't18:37
jogosdague: cool so we just add some waits in. maybe its good to document somewhere that javelin2 create commands should be blocking18:37
jogosdague: anyway lunch time  be back later18:37
*** bnemec has joined #openstack-qa18:40
*** amitpp has quit IRC18:43
*** gordc has joined #openstack-qa18:44
openstackrecheckConsole logs not available after 13:22s for gate-nova-python26 109279,1,e46cd4218:46
gordchi, re: https://bugs.launchpad.net/ceilometer/+bug/1336755, i was wondering, any hints when the XML version of the tests timeouts but JSON version passes in a few seconds?18:49
*** hockeynut has joined #openstack-qa18:51
*** mwagner_lap has quit IRC18:53
*** aimon has quit IRC18:56
*** mriedem has quit IRC18:58
*** mriedem has joined #openstack-qa18:58
openstackrecheckConsole logs not available after 13:26s for check-tempest-dsvm-full 106096,5,de75d1d19:00
*** bnemec has quit IRC19:02
ekhugenmtreinish, is more work needed for this blueprint, or are the work items not reflective of the actual status?  thanks!  https://blueprints.launchpad.net/tempest/+spec/add-service-tags19:05
openstackgerritSteve Baker proposed a change to openstack-infra/devstack-gate: Enable dib service for master, heat-slow  https://review.openstack.org/9563619:09
*** mkoderer_cloud has joined #openstack-qa19:09
*** lbragstad has quit IRC19:10
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 109290,1,87fb5e219:13
mtreinishdkranz: ok, I'll take a look19:13
dkranzmtreinish: thanks :)19:13
mtreinishekhugen: so the only thing left is to add service tags where required19:14
*** andreaf_ has quit IRC19:14
mtreinishit's a lot of iteration going through all the api tests and identifying where service tags are missing19:14
mtreinishI did one patch not too long ago that added network tags to the compute api tests19:14
mtreinishekhugen: is that something you'd be willing to take over?19:14
ekhugenmtrenish, so the volume api tests, for instance, still need that19:14
ekhugenI'd be interested in that, yes19:15
*** andreaf has joined #openstack-qa19:15
mtreinishekhugen: ok cool, go right ahead and work on it then. I don't have much time to look at it now, so you won't be stepping on anyone's toes by closing that out19:16
*** Longgeek has quit IRC19:16
ekhugenthanks mtreinish19:16
mtreinishI'm pretty sure that bp has been open for ~1 year at this point19:16
*** mkoderer has quit IRC19:21
mtreinishdkranz: I think taking that step is a bit premature, and not neccesarily the direction we were planning on taking by adding functional testing to the projects19:21
mtreinishbut we can discuss it more during the meeting19:21
*** mkoderer has joined #openstack-qa19:21
*** hockeynu_ has joined #openstack-qa19:21
mtreinishbut I need to shuffle the agenda a bit, because we need to do a bp review19:21
mtreinishsince it's j-219:21
dkranzmtreinish: ok. Are you saying we might not stop running all the existing nova api tests on swift commits, etc?19:22
dkranzmtreinish: I thought that was what Sean and Jim's message was about, at least in part.19:22
dkranzmtreinish: Perhaps the distinction is between "adding" and "moving out of tempest". Anyway, can discuss later.19:23
mtreinishdkranz: That's not what I was envisioning. We will probably remove a bunch of tests from tempest that are really functional tests19:23
mtreinishlike the CLI tests19:23
mtreinishbut I didn't plan on moving us to running a subset of tests on the tempest19:24
mtreinisherr, I should say a smaller subset19:24
mtreinishit would still be the same gate regex...19:24
*** hockeynut has quit IRC19:24
dkranzmtreinish: I was proposing that was a temporary measure but was assuming most of the tempest api tests would move to projects. Otherwise I don't see how it would help with the gate issues.19:25
dkranzmtreinish: But I don't really know what you were thinking or what you guys came up with last week.19:25
mtreinishyeah, and I don't think we want to be doing that. Otherwise there is a window where things will slip through, because our coverage decreases.19:26
mtreinishdkranz: well I may not see eye to eye with everyone on this point either19:26
mtreinishI dunno19:26
dkranzIt would not decrease on the main project19:26
dkranzmtreinish: All the nova tests would still run on nova, just not on swift19:26
mtreinishsee but the thing is nova depends on swift in the gate just a second level dependency glance uses swift for a store and nova uses those images19:26
openstackrecheckConsole logs not available after 13:25s for gate-nova-python27 109265,2,a52f6a519:27
mtreinishso we actually exercise swift more than most people realize19:27
dkranzmtreinish: Right, that's why some api tests would remain19:27
dkranzmtreinish: But we don't need the 17 different url combinations that are there now19:27
dkranzmtreinish: I am talking about increasing the budgget for relevant tests, and reducing it for not-relevant19:27
dkranzmtreinish: But I agree that is an arcnhitectural judgement call19:28
dkranzmtreinish: We should run more nova tests when gating nova patches, and fewer tests is all I mean. The budget is fixed.19:28
dkranzmtreinish: fewer tests of other projects I mean19:29
mtreinishyeah, but we're not talking about a total test budget, we're close to an hour, but still under it. What I viewed the role of functional tests is to fill the gap between unit tests and tempest19:30
mtreinishand there are probably some tempest tests that belong in that middle level and not in tempest19:31
mtreinishwhich is where we'd get reduction in total tests19:31
mtreinishI don't really see much value in splitting up which tests run on which changes19:31
*** markmcclain has joined #openstack-qa19:32
mtreinishit adds complexity and decreases how much we're exercise the whole picture19:32
mtreinishespecially if we're not augmenting things with a new test suite19:32
mtreinish(because they haven't been written yet)19:32
mtreinishthe other aspect is we don't have much data on the long term trends around these tests. We don't know which ones fail the most and which never do for more than 2 weeks19:35
mtreinishwell aside from those scenario tests that always break :)19:35
mtreinishso I would not be comfortable redistributing what gets run where without knowing how important of a test it is19:36
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-postgres-full 105231,13,62b38ba19:40
*** lbragstad has joined #openstack-qa19:41
*** lbragstad has quit IRC19:45
openstackgerritTushar Kalra proposed a change to openstack/tempest: Quota tests don't take into account a limit of -1  https://review.openstack.org/10914819:47
openstackgerritTushar Kalra proposed a change to openstack/tempest: Quota tests don't take into account a limit of -1  https://review.openstack.org/10914819:47
openstackgerritTushar Kalra proposed a change to openstack/tempest: Quota tests don't take into account a limit of -1  https://review.openstack.org/10914819:50
*** aimon has joined #openstack-qa19:51
openstackrecheckConsole logs not available after 13:21s for gate-tempest-dsvm-neutron 103326,2,ece940d19:53
*** lbragstad has joined #openstack-qa19:54
*** bnemec has joined #openstack-qa20:04
*** aimon has quit IRC20:05
openstackrecheckConsole logs not available after 13:27s for gate-tempest-dsvm-large-ops 109301,1,72cc7df20:07
openstackgerritA change was merged to openstack-dev/devstack: Remove unfubar_setuptools  https://review.openstack.org/10755320:08
*** aimon has joined #openstack-qa20:09
*** chandankumar has joined #openstack-qa20:10
*** rfolco has quit IRC20:13
jogosdague: your sleep 10 for nova-compute isn't working: http://logs.openstack.org/80/107380/11/check/check-grenade-dsvm/0006253/logs/grenade.sh.txt.gz#_2014-07-24_18_28_57_61820:14
jogohttp://logs.openstack.org/80/107380/11/check/check-grenade-dsvm/0006253/logs/grenade.sh.txt.gz#_2014-07-24_18_50_06_95920:14
mtreinishandreaf: are you around?20:15
jogosdague: hmm I wonder if nova-compute won't stop if its in the middle of something20:15
*** rfolco has joined #openstack-qa20:16
*** chandankumar has quit IRC20:17
*** lbragstad has quit IRC20:18
*** lbragstad has joined #openstack-qa20:19
jogosdague: woot http://logs.openstack.org/80/107380/11/check/check-grenade-dsvm-partial-ncpu/d82d971/logs/grenade.sh.txt.gz#_2014-07-24_18_56_03_34320:20
mtreinishjogo: use SIGKILL or SIGPWR instead of SIGINT? :)20:20
*** bnemec has quit IRC20:20
jogomtreinish: what do we use now?20:21
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-full 108011,5,a3a1fc720:21
mtreinishjogo: nice you got the javelin logs20:21
mtreinishjogo: oh, I dunno, I just assumed it was SIGINT to do it gracefully20:21
*** Sukhdev has joined #openstack-qa20:22
jogopkill -TERM -P -3055920:22
jogomtreinish: do you knwo if sdague is working on fixing up the javelin stuff?20:23
jogoif not I will take a whack at it20:23
jogodansmith: can you think of any reason why trying to kill nova-compute won't work20:24
dansmithjogo: if you're not sending -9 or something, we try to do cleanup, IIRC20:25
mtreinishjogo: I think he was planning on getting to it when he had the time. I'm not sure when that is though20:25
jogodansmith: this is what we are doing http://logs.openstack.org/80/107380/11/check/check-grenade-dsvm/0006253/logs/grenade.sh.txt.gz#_2014-07-24_18_50_01_18020:26
mtreinishjogo: yeah SIGTERM will do it gracefully too.20:26
jogodansmith: my hunch is that nova-compute gets stuck / is taking too long because its in the middle of something20:26
mtreinishso it's probably trying to cleanup or doing something while handling that20:26
jogowe are booting an instance when -TERM is called20:26
*** rbak has quit IRC20:26
jogowe wait 10 seconds20:26
jogobut somewhow that isn't enough20:26
dansmithjogo: right, TERM will try to do cleanup I think20:26
*** mwagner_lap has joined #openstack-qa20:26
dansmithKILL should kill it unless it's in D sleep in the kernel20:27
*** rbak has joined #openstack-qa20:27
jogodansmith: hmm http://logs.openstack.org/80/107380/11/check/check-grenade-dsvm/0006253/logs/old/screen-n-cpu.txt.gz logs stop right when the kill happens20:31
dansmithjogo: well, if the handling of it causes a deadlock or something, that would make sense20:33
dansmithjogo: how do you know it's not stopping?20:33
jogodansmith: http://logs.openstack.org/80/107380/11/check/check-grenade-dsvm/0006253/logs/grenade.sh.txt.gz#_2014-07-24_18_50_17_11720:35
openstackrecheckConsole logs not available after 13:25s for gate-horizon-python27 98961,2,7027a4920:35
jogodansmith: and the process list at the bottom has it20:35
dansmithokay20:35
dansmithjogo: can you SIGUSR1 it first to get it to dump the greenthread info?20:36
dansmithactually maybe before and after the TERM20:36
jogoohh good idea20:37
jogodansmith: push a grenade patch to do that on top of https://review.openstack.org/#/c/97317/1120:37
*** lbragstad has quit IRC20:38
*** dustins has quit IRC20:38
*** bnemec has joined #openstack-qa20:38
*** lbragstad has joined #openstack-qa20:38
dansmithjogo: s/^/jogo will / , right? :)20:44
jogomtreinish: what tempest function do I use to make javelin2 wait until an instance is ready20:44
*** Ananci has quit IRC20:47
mtreinishjogo: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/services/compute/json/servers_client.py#n17020:48
openstackgerritJoe Gordon proposed a change to openstack-dev/grenade: TESTING: Trigger guru meditation report on nova-compute  https://review.openstack.org/10941320:52
jogodansmith: ^ happy ?20:52
* jogo hopes his bash foo is strong today20:52
dansmithjogo: looks good to me20:53
*** dkranz has quit IRC20:56
openstackrecheckConsole logs not available after 13:26s for gate-trove-python27 109313,1,609895520:57
openstackrecheckopenstack/nova change: https://review.openstack.org/109033 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error20:58
sdaguejogo: just got back from errands, going to move to the HP office (i.e. porch) will dive back in there20:58
*** gordc has left #openstack-qa20:58
sdaguejogo: the n-cpu not stopping is just the delay thing, right?21:01
sdagueI have a quiece time patch up there now for that21:01
*** ekhugen has quit IRC21:02
*** mmaglana has joined #openstack-qa21:04
dansmithsdague: you need a definition21:04
dansmithsdague: "Hexterior Porch" or some such21:04
sdaguedansmith: nice :)21:04
dansmith(silent H)21:04
sdaguemaybe I'll need to get a banner made up21:04
*** bnemec has quit IRC21:05
*** aysyd has quit IRC21:05
mtreinishdansmith: the silent H is a nice touch :)21:05
dansmithmtreinish: you can make lots of better acronyms with silent letters :)21:05
*** rfolco has quit IRC21:05
sdaguemtreinish: so here is my current thinking, which I was doing today when trying not to stab myself doing expense reports :)21:07
*** mriedem has quit IRC21:07
sdagueinstead of unit tests for javelin2, why don't we just do real tempest tests that run it. Then we can shake out the rest of these oddities in a real devstack env.21:07
*** cdent has quit IRC21:07
sdaguemostly I was wondering where to put them in the tree21:07
jogosdague: I tried the 10 second delay patch21:08
mtreinishsdague: hmm, well that seems like a scenario to me21:08
jogoand it didn't work21:08
mtreinishsdague: since we're not testing a specific API but a workflow21:09
jogomtreinish sdague: ++21:09
jogothe schenario would be javelin -m create21:09
jogojavelin -m check21:09
jogojavelin -m destroy21:09
mtreinishsdague: although putting it there will cause some duplication because scenario is part of smoke...21:09
mtreinishbut whatever21:09
mtreinishjogo: although you probably don't need to subprocess it, just call things directly if that works21:10
sdaguemtreinish: I'd actually subprocess it21:10
sdagueso that it's run the same way we'd run it otherwise21:10
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 109306,1,d2259f721:11
mtreinishsdague: then what's the value in having that test once we get it gating with grenade?21:11
mtreinishif we're exercising things the same exact way21:11
sdaguemtreinish: because it would also include fail cases21:11
sdagueproduce bad resource files, and test the fail slices21:12
sdagueand that they provide sane output21:12
sdaguewhich... today, they don't21:12
mtreinishheh, so your proposing adding the first negative scenario tests? :)21:12
sdaguewell, honestly, this is actually tempest functional testing21:13
sdaguewhich means it should probably be another subdir21:13
sdaguebecause this isn't going to look like scenario tests at all21:13
mtreinishso now we're going to have a sub functional test suite for self testing in our integrated functional test suite?21:14
mtreinishthat's not at all confusing :)21:14
mtreinishsdague: and would you propose that we run this on every tempest job? or have a special tempest dsvm functional job for it?21:16
sdagueI think long term it should be it's own job21:17
sdagueI'd be fine with a short term add for expediency21:17
jogomtreinish sdague: we still have to get it working first21:18
sdagueyeh21:18
jogomy env got wonky so rebuilding it to do some testing21:18
sdaguemostly I'm trying to flush out the idea for sanity before diving down that path21:18
jogosdague: I think the check command should have a  lot of checks in it21:19
jogoor even better21:19
jogowait until resource is ready21:19
jogothen call the check command21:19
jogoand check should have multiple levels of checks21:19
jogolike make sure instance is in ACTIVE mode21:19
jogoetc21:19
jogosdague: to make debugging easier21:19
sdaguejogo: sure, though honestly it's got to actually put out output first21:21
sdaguedo we have that patch somewhere?21:21
jogodansmith: http://logs.openstack.org/13/109413/1/check/check-grenade-dsvm/04119ad/logs/grenade.sh.txt.gz21:21
sdaguebecause that's my rage quit right now, it fails in check, and the logs are all lost21:21
jogosdague: yes I do have that21:21
*** aimon has quit IRC21:22
sdaguejogo: where's that patch?21:22
jogosdague: http://logs.openstack.org/80/107380/11/check/check-grenade-dsvm-partial-ncpu/d82d971/logs/grenade.sh.txt.gz#_2014-07-24_18_55_59_84021:22
jogosdague: https://review.openstack.org/#/c/107380/21:22
*** bnemec has joined #openstack-qa21:24
andreafmtreinish: I am now21:24
openstackrecheckConsole logs not available after 13:23s for check-tempest-dsvm-neutron 102700,6,fcbf40521:24
jogodansmith: http://logs.openstack.org/13/109413/1/check/check-grenade-dsvm/04119ad/logs/grenade.sh.txt.gz#_2014-07-24_21_12_06_492  http://logs.openstack.org/13/109413/1/check/check-grenade-dsvm/04119ad/logs/old/screen-n-cpu.txt.gz21:24
jogodansmith: that doesn't look right21:24
andreafmtreinish: what's up?21:25
sdaguejogo: damn, that's after the timeout was added21:25
dansmithjogo: is stderr for nova in n-cpu.log?21:25
jogosdague: I tried sending a SIGUSR1 for guru meditation21:25
jogobut taht isn't working21:25
jogodansmith: hrm excellent question21:26
jogosdague: ^21:26
dansmithjogo: oh, I see, the signal killed it, which shouldn't have happened21:26
dansmithmaybe there is some config21:26
* dansmith looks21:26
sdaguedansmith: it should be21:26
sdagueit's litterally the screen log21:26
dansmithsdague: should be what?21:26
sdaguestderr21:27
dansmithoh, stderr21:27
dansmithright21:27
mtreinishandreaf: I was just wondering you were going to make it to the meeting or not21:27
*** bkopilov has quit IRC21:27
mtreinishandreaf:  because I wanted to ask about your j-2 targetted bp if you couldn't make it21:27
*** masayukig has quit IRC21:27
jogogonna add a SIGUSR1 to n-api in the same ay21:28
jogoway21:28
jogoto sanity check21:28
dansmithjogo: hrm, I don't see any config21:28
dansmithseems like it should just work21:28
andreafmtreinish: I will be in the meeting, or at least the first half of it21:28
dansmithjogo: this is grenade-old, which is icehouse?21:28
mtreinishok then I'll make sure to ask about your bp first :)21:28
andreafmtreinish: let me go and review my bp21:28
jogodansmith: that is correct21:29
mtreinishandreaf: it's more a question of status and whether it needs to be defered21:30
mtreinishwhich I think the answer to all of them is yes...21:30
openstackgerritJoe Gordon proposed a change to openstack-dev/grenade: TESTING: Trigger guru meditation report on nova-compute  https://review.openstack.org/10941321:30
jogodansmith sdague: if this doesn't work then I am doing something wrong21:30
dansmithjogo: the signal is getting to the process it seems,21:30
dansmithjogo: it's more of a matter of it not triggering the report I think21:30
jogodansmith: well the second one is21:30
jogojust like you suggested I sent two21:31
andreafmtreinish: indeed, but none of them made it for j-221:31
*** aimon has joined #openstack-qa21:31
dansmithjogo: how do you know it's the second one?21:31
jogodansmith: heh I don't but only 1 apears to make it through21:31
jogoand the second one causes nova-compute to terminate21:31
dansmithright, the first one I think :)21:31
jogoerr that one21:31
dansmiththe first one kills it I think21:31
jogooh wait maybe21:32
jogoeither way21:32
dansmithyeah21:32
jogosomething no work21:32
andreafmtreinish: but the multi-auth is quite close to be completed, and if the scenario migration goes on well I'll be able to finally close that in Juno timeframe21:32
*** masayukig has joined #openstack-qa21:32
dansmithI can't find solly anywhere, but we should catch him when we can, he'd know21:32
jogodansmith: good idea21:33
andreafmtreinish: the keystone v3 jobs depends on how fast the v3 integration goes on in Juno21:33
andreafmtreinish: client manager refactor is not even approved, so I didn't start working on it, anyways other bp come first - so may that one can be left open-ended for a while21:35
jogobbiab, its way to hot in here going to find some place cooler21:36
mtreinishandreaf: it was just the multi-auth, the others either aren't approved or not targetted for j-221:37
jogosdague: I was going to work on adding more testing to javelin2, but if you want to do that just  let me know and I will let you.21:37
andreafmtreinish: ssh-auth-strategy is really hard to say, we tried many times to enable this and it always fails because it makes the gate unstable - so perhaps we need to have the ability to switch ssh on in all tests, but we could have that in a post-merge kind of environment, hopefully collecting enough data will allow us to finally fix the issues behind it. This is very dear to me21:37
jogosdague: I am waiting for my env to get setup at the moment21:37
andreafmtreinish: ok I'll stop my spam :D21:37
mtreinishandreaf: the answer is to defer that one to j-3 and add a dep on the scenario conversion21:37
sdaguejogo: honestly, if you have momentum, go for it21:38
mtreinishandreaf: which is all I needed, I'll update the BP :)21:38
andreafok21:38
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 98918,11,4b03fa221:38
andreafjogo: ping21:39
openstackrecheckopenstack/nova change: https://review.openstack.org/109036 failed gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error21:39
*** dkranz has joined #openstack-qa21:40
sdaguejogo: then I could go look at the screen issue21:42
sdagueand figure out if I can get screen pulled out of it21:42
*** mkoderer_cloud has quit IRC21:42
*** rwsu has quit IRC21:44
*** aimon has quit IRC21:46
*** aimon has joined #openstack-qa21:47
*** bnemec has quit IRC21:47
sdaguemtreinish: I split up the patch you didn't like - https://review.openstack.org/#/q/status:open+project:openstack-infra/elastic-recheck+branch:master+topic:erchanges,n,z21:48
sdagueit would be good to land the first 2 of those, then I'll go get clark to land the right bits on config side21:48
*** gfidente has quit IRC21:48
sdaguewell, I guess they haven't been reviewed yet21:48
*** lbragstad has quit IRC21:49
mtreinisheww, they changed the launchpad openid theme...21:49
mtreinishsdague: +2 on the first 1, but I don't see the point of the second patch21:50
mtreinishwhy isn't that just part of the 3rd patch21:50
mtreinishthat file isn't used anywhere21:50
sdaguebecause it's the in tree example for what you are supposed to do21:50
sdagueI do realize that we use a different file in config21:50
mtreinishyeah, and why don't you just update that at the same time as when you add the new field21:51
sdagueok, here is how I'm thinking of the problem21:51
sdague"hey infra, I need you to land https://review.openstack.org/#/c/108719/"21:52
sdague"ok, what's all this new stuff"21:52
sdague"the example is in tree, see here"21:52
openstackrecheckConsole logs not available after 13:23s for check-grenade-dsvm 109325,1,aa4b8c721:52
sdaguebecause it represents a point where we have to stop, and land code from the other tree before we move forward21:53
mtreinishsee yeah that last step doesn't matter. We've done this kind of thing several times already with e-r and no one cares if the example is in sync with what's in the config tree.21:53
mtreinishI get the need to preempt the change in config so as to not lose the messages21:53
openstackgerritGael Chamoulaud proposed a change to openstack-dev/devstack: Fixes some typos in stack.sh  https://review.openstack.org/10942521:54
mtreinishbut there is no reason to update the sample before there is code using it21:54
openstackgerritSean Dague proposed a change to openstack-infra/elastic-recheck: add support for an external message catalog  https://review.openstack.org/10870621:54
sdaguemtreinish: ok, well that seems bad form to me, but if you want me to just collapse them, sure21:54
mtreinishoh, I don't really care (besides the stat padding... :) ) I was just curios if I was missing something21:55
sdagueit was more what I considered the right flow for consistency21:56
*** oomichi has joined #openstack-qa21:56
*** ekhugen has joined #openstack-qa21:58
mtreinishsdague: yeah, but looking at it from the opposite way you'll have a point in the tree where the example doesn't reflect reality21:59
mtreinishbut I guess it just depends on how you're looking at things21:59
*** lbragstad has joined #openstack-qa22:01
*** hockeynu_ has quit IRC22:02
*** jecarey has quit IRC22:02
*** _afazekas is now known as afazekas22:03
*** rockyg has joined #openstack-qa22:04
*** lbragsta_ has joined #openstack-qa22:04
openstackrecheckConsole logs not available after 13:25s for gate-python-openstackclient-python27 109323,1,53ec51222:06
*** lbragstad has quit IRC22:07
*** lbragsta_ has quit IRC22:08
*** markmcclain has quit IRC22:08
*** dims_ has joined #openstack-qa22:09
openstackgerritA change was merged to openstack-infra/devstack-gate: add os-loganalyze to projects list  https://review.openstack.org/10554922:09
openstackgerritA change was merged to openstack/tempest: Add base class for all credential providers  https://review.openstack.org/10734822:10
*** mlavalle has quit IRC22:11
*** dims has quit IRC22:12
*** dims_ has quit IRC22:16
openstackrecheckConsole logs not available after 13:31s for check-tempest-dsvm-full 107505,3,e5be71722:19
*** Sukhdev has quit IRC22:22
jogosdague: sounds like a plan22:26
jogoandreaf: pong22:26
andreafjogo: in Darmstad you were asking about v3 / domain testing - do you know who's driving the implementation of v3 auth for the services in devstack and what's the ETA for that22:28
jogodansmith: http://logs.openstack.org/13/109413/2/check/check-grenade-dsvm/db9572f/logs/old/screen-n-api.txt.gz22:28
*** aimon has quit IRC22:28
jogoandreaf: sorry I don't , I do know that someone is22:28
jogoI think there were comments about that on the ML22:28
andreafjogo: ok I'll dig in the ML thx22:29
jogodansmith: the guru meditation doesn't seem to work -- or I am doing it wrong22:29
jogon-api failed to start22:29
jogodansmith: I wonder if its a screen thing22:30
sdaguejogo: failed to start is typically screen22:31
jogosdague: what does it mean?22:32
sdagueit means that the box is slow22:33
sdagueand that screen -X stuff got dropped22:33
openstackrecheckConsole logs not available after 13:34s for gate-heat-pep8 109333,1,390aef722:33
jogosdague: ahh so guru meditation may have blown screen up?22:34
*** mlavalle has joined #openstack-qa22:35
sdaguejogo: that's possible22:37
mtreinishjogo: that's quite the imagery...22:37
*** jgrimm has quit IRC22:39
*** sballe has quit IRC22:39
*** nati_ueno has quit IRC22:40
*** nati_ueno has joined #openstack-qa22:40
*** dims_ has joined #openstack-qa22:42
*** jhenner has joined #openstack-qa22:43
openstackrecheckConsole logs not available after 13:21s for gate-tempest-dsvm-full 109093,1,a95d9db22:47
*** dims_ has quit IRC22:47
*** dimtruck is now known as zz_dimtruck22:52
*** rockyg has quit IRC22:52
*** jhenner has quit IRC22:53
openstackrecheckConsole logs not available after 13:23s for gate-keystone-python26 109343,1,fd91fc523:00
*** oomichi has quit IRC23:01
*** ekhugen has quit IRC23:01
andreafmtreinish: I've seen -1 on new API tests because of assert on return code - but I don't think we should mix that refactor with writing new tests23:01
*** jamielennox|away is now known as jamielennox23:01
andreafmtreinish: I'll let you think about this, I'm going to fall asleep on the sofa soon, talk to you tomorrow23:02
mtreinishandreaf: ok yeah it's a bit late for you now23:02
mtreinishwe can discuss it later23:02
*** sdmonov has quit IRC23:03
*** nati_ueno has quit IRC23:04
*** nati_ueno has joined #openstack-qa23:05
*** nati_ueno has quit IRC23:08
openstackrecheckConsole logs not available after 13:22s for gate-django_openstack_auth-python26 85291,3,ca6eff323:14
*** tosky has left #openstack-qa23:15
jogosdague: well we can try again when you have screen excised23:18
jogohmm thats not the write word23:19
jogooh it is23:19
mroddenuh... did the launchpad login page change for others too....?23:26
*** ArxCruz has quit IRC23:26
openstackrecheckConsole logs not available after 13:22s for gate-nova-python26 107435,3,404fdf023:28
mtreinishmrodden: yeah and I think that it's way uglier...23:28
mtreinishbut that's probably because I'm just used to the old one23:28
mroddenthere is ubuntu marketing all over the place23:29
mroddengross23:29
jogosdague mtreinish: does the gate env have enough resources to create a m1.medium and m1.small at the same time?23:32
jogomy dev config doesn't23:32
mtreinishjogo: how big are those flavors?23:33
mroddensmall is 1vpcu 2g usuaally23:33
mroddenmedium is 2x that23:33
mtreinishyeah so that's 6G of ram out of 8 I think23:34
mtreinishand I'm not sure what we run as overcommit in devstack for the gate23:34
jogomtreinish: maybe we should do a m1.micro so we don't have to think about it23:35
jogosdague: thoughts? ^ its on your patch23:36
mroddenthere is tiny23:36
mrodden512mb ram23:36
mroddeni use tiny's a lot...23:36
*** yamahata has joined #openstack-qa23:39
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-full 109357,2,3ccb44823:42
*** jaypipes has quit IRC23:43
*** dims_ has joined #openstack-qa23:43
*** gmann has joined #openstack-qa23:44
*** oomichi has joined #openstack-qa23:45
jogomtreinish sdague: some javelin2 tweaks23:46
jogoerr git commit hooks23:46
*** dims_ has quit IRC23:47
openstackgerritJoe Gordon proposed a change to openstack/tempest: Add logs to javelin2  https://review.openstack.org/10944823:48
openstackgerritJoe Gordon proposed a change to openstack/tempest: Add destroy_server to javelin2  https://review.openstack.org/10944923:48
openstackgerritJoe Gordon proposed a change to openstack/tempest: Make javelin2 create_server wait for server to boot  https://review.openstack.org/10945023:48
openstackgerritJoe Gordon proposed a change to openstack/tempest: Make javelin2's ping wait 60 seconds  https://review.openstack.org/10945123:48
jogothere ^23:48
openstackgerritJoe Gordon proposed a change to openstack/tempest: Run javelin2 check at the end of create  https://review.openstack.org/10945223:48
*** atiwari has quit IRC23:55
openstackrecheckConsole logs not available after 13:21s for check-tempest-dsvm-postgres-full 109013,2,e62c80623:55
*** SridharG has joined #openstack-qa23:59

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!