Monday, 2015-08-31

openstackrecheckopenstack/keystone change: https://review.openstack.org/214509 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error00:06
*** achanda has joined #openstack-qa00:06
*** dimtruck is now known as zz_dimtruck00:15
*** oomichi has joined #openstack-qa00:21
openstackgerritReedip proposed openstack/tempest: Added Test Case for BaremetalNodesClient  https://review.openstack.org/21503800:36
*** achanda has quit IRC00:44
openstackgerritReedip proposed openstack/tempest: Added Test Case for InterfacesClient  https://review.openstack.org/21813300:46
openstackgerritMatthew Treinish proposed openstack/tempest: Set default value for tempest log file  https://review.openstack.org/17358100:52
openstackrecheckopenstack-infra/devstack-gate change: https://review.openstack.org/190601 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error00:54
openstackgerritIan Wienand proposed openstack-dev/devstack: Use sudo iniset to modify /etc files  https://review.openstack.org/20424700:56
*** tochi has quit IRC01:07
*** salv-orlando has joined #openstack-qa01:07
openstackgerritGhanshyam Mann proposed openstack/tempest: Return complete resp from volumes_extensions_client  https://review.openstack.org/21616401:08
*** salv-orlando has quit IRC01:13
*** salv-orlando has joined #openstack-qa01:13
*** hoangcx has joined #openstack-qa01:17
*** zhangjn has joined #openstack-qa01:20
openstackrecheckConsole logs not available after 13:23s for gate-tempest-dsvm-postgres-full 218355,4,6fcfc8801:23
openstackrecheckopenstack-infra/devstack-gate change: https://review.openstack.org/199851 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error01:23
*** zz_dimtruck is now known as dimtruck01:24
*** salv-orlando has quit IRC01:26
*** dimtruck is now known as zz_dimtruck01:29
*** Caleb-- has quit IRC01:37
*** Caleb-- has joined #openstack-qa01:37
*** achanda has joined #openstack-qa01:37
*** TC01 has quit IRC01:37
*** zhangjn has quit IRC01:40
*** zhangjn has joined #openstack-qa01:41
*** tochi has joined #openstack-qa01:41
*** aix_ has joined #openstack-qa01:41
*** aix has quit IRC01:42
*** TC01 has joined #openstack-qa01:46
*** armax has joined #openstack-qa02:10
*** mlavalle has quit IRC02:11
*** yamamoto has quit IRC02:12
openstackrecheckopenstack/cinder change: https://review.openstack.org/216044 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full, gate-tempest-dsvm-full-ceph in the gate queue with an unrecognized error02:16
*** flwang1 has quit IRC02:18
*** flwang1 has joined #openstack-qa02:19
*** zhangjn has quit IRC02:19
*** achanda has quit IRC02:22
*** achanda has joined #openstack-qa02:24
*** salv-orlando has joined #openstack-qa02:28
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Add mac info to test_list_virtual_interfaces  https://review.openstack.org/21870502:36
*** salv-orlando has quit IRC02:41
*** Poornima has joined #openstack-qa02:55
*** yamamoto has joined #openstack-qa02:58
openstackrecheckopenstack-infra/devstack-gate change: https://review.openstack.org/201279 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error03:12
*** eglute has quit IRC03:25
*** tochi has quit IRC03:25
*** jroll has quit IRC03:26
*** jroll has joined #openstack-qa03:31
*** eglute has joined #openstack-qa03:31
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Encap netaddr-0.7.16  https://review.openstack.org/21871203:34
*** I has joined #openstack-qa03:35
*** I is now known as Guest2505903:36
*** rbak has quit IRC03:36
*** salv-orlando has joined #openstack-qa03:43
*** Guest25059 has quit IRC03:49
*** haigang has joined #openstack-qa03:49
*** cody-somerville has joined #openstack-qa03:50
*** cody-somerville has quit IRC03:50
*** cody-somerville has joined #openstack-qa03:50
*** lawrance_ has joined #openstack-qa03:51
lawrance_hello there03:51
lawrance_the tag 5 of tempest is kilo release03:54
lawrance_?03:54
*** salv-orlando has quit IRC03:56
*** tochi has joined #openstack-qa04:00
openstackrecheckopenstack/cinder change: https://review.openstack.org/216591 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full, gate-tempest-dsvm-full-ceph in the gate queue with an unrecognized error04:05
*** gamado has joined #openstack-qa04:26
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Encap netaddr-0.7.16  https://review.openstack.org/21871204:31
*** zhangjn has joined #openstack-qa04:37
*** sabeen1 has joined #openstack-qa04:40
*** sabeen1 has quit IRC04:49
*** jasonsb has joined #openstack-qa04:50
*** salv-orlando has joined #openstack-qa04:52
*** yfried|afk has quit IRC04:53
*** Sukhdev has joined #openstack-qa04:56
*** salv-orlando has quit IRC04:57
*** salv-orlando has joined #openstack-qa04:57
*** ig0r__ has joined #openstack-qa04:58
*** ig0r_ has quit IRC04:59
*** shausy has joined #openstack-qa05:01
*** salv-orlando has quit IRC05:10
openstackrecheckopenstack/nova change: https://review.openstack.org/209499 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full, gate-tempest-dsvm-nova-v21-full, gate-tempest-dsvm-full-ceph in the gate queue with an unrecognized error05:12
*** sabeen1 has joined #openstack-qa05:13
openstackgerritMD NADEEM proposed openstack/tempest: Add unit tests for migrations_client  https://review.openstack.org/21706005:22
*** haigang has quit IRC05:23
*** oomichi has quit IRC05:27
*** shausy has quit IRC05:36
*** shausy has joined #openstack-qa05:36
*** sabeen2 has joined #openstack-qa05:39
*** sabeen1 has quit IRC05:39
*** itzikb_ has joined #openstack-qa05:46
*** haigang has joined #openstack-qa05:47
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Add skip_because for bug/1490368  https://review.openstack.org/21873606:03
*** shuquan has joined #openstack-qa06:05
*** SridharG has joined #openstack-qa06:08
*** salv-orlando has joined #openstack-qa06:10
*** jasonsb has quit IRC06:10
*** davideagnello has joined #openstack-qa06:16
*** oomichi has joined #openstack-qa06:18
*** afazekas_ has joined #openstack-qa06:19
*** shuquan has quit IRC06:23
*** shuquan has joined #openstack-qa06:26
*** yfried|afk has joined #openstack-qa06:26
*** haigang has quit IRC06:27
*** armax has quit IRC06:28
*** haigang has joined #openstack-qa06:28
*** davideagnello has quit IRC06:29
*** haigang has quit IRC06:29
*** yfried has joined #openstack-qa06:30
*** yfried|afk has quit IRC06:31
*** anshul has joined #openstack-qa06:34
*** haigang has joined #openstack-qa06:36
*** zhangjn has quit IRC06:45
*** zhangjn has joined #openstack-qa06:45
*** neeti has joined #openstack-qa06:56
*** Sukhdev has quit IRC07:00
*** shuquan has quit IRC07:00
*** achanda has quit IRC07:05
*** piyush has quit IRC07:06
*** sabeen2 has quit IRC07:18
*** zhangjn has quit IRC07:28
*** k4n0 has joined #openstack-qa07:40
*** b10n1k_ has quit IRC07:44
*** safchain has joined #openstack-qa07:48
*** Caleb-- has quit IRC07:50
*** sshnaidm has joined #openstack-qa07:52
*** Caleb-- has joined #openstack-qa08:00
*** jerryz has quit IRC08:02
*** sshnaidm has quit IRC08:02
*** zhangjn has joined #openstack-qa08:10
*** shuquan has joined #openstack-qa08:12
*** hichihara has quit IRC08:13
*** sshnaidm has joined #openstack-qa08:15
*** kiran-r has joined #openstack-qa08:15
*** kbaikov has quit IRC08:23
*** shuquan has quit IRC08:24
*** shuquan has joined #openstack-qa08:24
*** salv-orlando has quit IRC08:24
*** salv-orlando has joined #openstack-qa08:25
*** shuquan has quit IRC08:25
*** rossella_s has quit IRC08:25
*** shuquan has joined #openstack-qa08:26
openstackgerritGhanshyam Mann proposed openstack/tempest: Return complete response from remaining service client  https://review.openstack.org/21876108:27
*** rossella_s has joined #openstack-qa08:27
*** hoangcx has quit IRC08:27
*** yog_ has joined #openstack-qa08:28
*** kbaikov has joined #openstack-qa08:29
*** aix_ has quit IRC08:32
*** aix has joined #openstack-qa08:33
*** jhenner has joined #openstack-qa08:34
*** shuquan has quit IRC08:38
openstackgerritTakashi NATSUME proposed openstack/tempest: Add container and disk format parameters  https://review.openstack.org/21106508:41
*** amotoki has joined #openstack-qa08:41
*** haigang has quit IRC08:44
*** haigang has joined #openstack-qa08:44
*** zhangjn has quit IRC08:45
openstackgerritGhanshyam Mann proposed openstack/tempest: Return complete response from remaining service client  https://review.openstack.org/21876108:48
*** haigang has quit IRC08:50
*** haigang has joined #openstack-qa08:52
*** e0ne has joined #openstack-qa08:59
*** tosky has joined #openstack-qa09:04
*** rossella_s has quit IRC09:05
*** hoangcx has joined #openstack-qa09:06
*** rossella_s has joined #openstack-qa09:06
*** e0ne has quit IRC09:09
*** e0ne has joined #openstack-qa09:09
openstackgerritTakashi NATSUME proposed openstack/tempest: Use container/disk format settings in glance v1 test  https://review.openstack.org/21877509:12
*** jhenner has quit IRC09:12
*** haigang has quit IRC09:20
*** zhangjn has joined #openstack-qa09:23
*** salv-orlando has quit IRC09:24
*** zhangjn has quit IRC09:27
*** haigang has joined #openstack-qa09:33
openstackgerritSergey Slipushenko proposed openstack/tempest: Added test_list_tenants non-admin test for v2, v3 api/identity  https://review.openstack.org/19270909:36
openstackgerritSergey Slipushenko proposed openstack/tempest: Added Identity API tests for user ec2 credentials  https://review.openstack.org/19458409:36
openstackgerritSergey Slipushenko proposed openstack/tempest: Added test_user_update_own_password tests for Identity v2, v3  https://review.openstack.org/19760009:36
*** ig0r_ has joined #openstack-qa09:37
*** ig0r__ has quit IRC09:39
*** zhangjn has joined #openstack-qa09:39
*** zhangjn has quit IRC09:41
*** yfried is now known as yfried|afk09:43
*** aix has quit IRC09:49
*** yfried|afk is now known as yfried09:53
*** pc_m has joined #openstack-qa09:57
*** sshnaidm has quit IRC09:57
*** pc_m_ has joined #openstack-qa09:58
*** pc_m_ has quit IRC09:59
*** pc_m has quit IRC10:01
*** lawrance_ has quit IRC10:08
*** sshnaidm has joined #openstack-qa10:10
*** pc_m has joined #openstack-qa10:12
*** hoangcx has quit IRC10:13
*** haigang has quit IRC10:14
*** haigang has joined #openstack-qa10:15
*** jhenner has joined #openstack-qa10:17
*** aix has joined #openstack-qa10:18
openstackrecheckopenstack/nova change: https://review.openstack.org/214085 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/146363110:42
openstackLaunchpad bug 1463631 in grenade "60_nova/resources.sh:106:ping_check_public fails intermittently" [Undecided,Confirmed]10:42
*** jaypipes has joined #openstack-qa10:58
*** ChanServ sets mode: +o jaypipes10:58
*** k4n0 has quit IRC10:58
*** gszasz has joined #openstack-qa11:01
openstackrecheckConsole logs not available after 13:25s for gate-tempest-dsvm-neutron-full-kilo 218761,2,c7ea1c011:09
*** e0ne has quit IRC11:10
openstackgerritTakashi NATSUME proposed openstack/tempest: Use container/disk format settings in glance v1 test  https://review.openstack.org/21877511:16
*** anshul has quit IRC11:29
openstackrecheckConsole logs not available after 13:28s for gate-tempest-dsvm-neutron-full-kilo 218775,1,f03990411:29
*** anshul has joined #openstack-qa11:30
*** jaypipes has quit IRC11:35
*** jaypipes has joined #openstack-qa11:36
*** ChanServ sets mode: +o jaypipes11:36
*** amotoki has quit IRC11:39
*** amotoki has joined #openstack-qa11:39
*** lawrancejing has joined #openstack-qa11:42
*** yfried is now known as yfried|afk11:43
*** openstackgerrit has quit IRC11:46
*** yfried|afk is now known as yfried11:47
*** openstackgerrit has joined #openstack-qa11:47
*** k4n0 has joined #openstack-qa11:48
*** mwagner_lap has quit IRC11:59
*** e0ne has joined #openstack-qa11:59
*** aysyd has joined #openstack-qa12:14
*** Poornima has quit IRC12:15
*** dkranz has joined #openstack-qa12:17
*** rfolco has joined #openstack-qa12:19
*** mhorban has quit IRC12:26
*** neeti has quit IRC12:29
*** k4n0 has quit IRC12:34
*** krtaylor has quit IRC12:34
*** mhorban has joined #openstack-qa12:36
openstackrecheckopenstack/nova change: https://review.openstack.org/217840 failed because of: gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/140329112:39
openstackLaunchpad bug 1403291 in neutron "VM's fail to receive DHCPOFFER messages" [Critical,Fix released]12:39
*** neeti has joined #openstack-qa12:39
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: added query for netaddr 0.7.16 mac addr parsing bug  https://review.openstack.org/21883412:39
*** krtaylor has joined #openstack-qa12:47
*** k4n0 has joined #openstack-qa12:48
*** afazekas__ has joined #openstack-qa12:56
*** afazekas_ has quit IRC12:58
*** dustins has joined #openstack-qa13:04
*** rossella_s has quit IRC13:05
*** rossella_s has joined #openstack-qa13:06
*** dkranz has quit IRC13:07
*** anshul has quit IRC13:08
*** sabeen1 has joined #openstack-qa13:11
*** shuquan has joined #openstack-qa13:11
*** dkranz has joined #openstack-qa13:20
*** anshul has joined #openstack-qa13:20
*** e0ne has quit IRC13:22
*** gamado has quit IRC13:24
openstackgerritMerged openstack-infra/elastic-recheck: added query for netaddr 0.7.16 mac addr parsing bug  https://review.openstack.org/21883413:24
*** neeti has quit IRC13:24
*** k4n0 has quit IRC13:27
*** anshul has quit IRC13:28
*** e0ne has joined #openstack-qa13:29
*** mtanino has joined #openstack-qa13:30
*** aalam has joined #openstack-qa13:31
openstackgerritJohn Warren proposed openstack/tempest: Full response for VolumesClient methods  https://review.openstack.org/21478113:32
*** safchain has quit IRC13:35
*** superdan is now known as dansmith13:35
*** salv-orlando has joined #openstack-qa13:36
*** dims has joined #openstack-qa13:36
openstackrecheckopenstack/nova change: https://review.openstack.org/218286 failed gate-tempest-dsvm-neutron-full in the gate queue with an unrecognized error13:37
*** anshul has joined #openstack-qa13:40
*** k4n0 has joined #openstack-qa13:40
*** k4n0 has quit IRC13:44
*** k4n0 has joined #openstack-qa13:45
openstackgerritSean Dague proposed openstack-dev/devstack: turn multi host true for nova network by default  https://review.openstack.org/21886013:45
*** tochi has quit IRC13:46
*** kiran-r has quit IRC13:48
*** neeti has joined #openstack-qa13:50
*** k4n0 has quit IRC13:50
*** mwagner_lap has joined #openstack-qa13:52
*** k4n0 has joined #openstack-qa13:57
*** shuquan has quit IRC13:57
*** anshul has quit IRC13:58
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: add query for misuse of netaddr  https://review.openstack.org/21886513:58
*** zhangjn has joined #openstack-qa13:59
*** zhangjn has quit IRC13:59
*** mriedem has joined #openstack-qa13:59
*** zhangjn has joined #openstack-qa14:00
sdaguemriedem / mtreinish - I thought the ebs test was supposed to be fixed - test_create_ebs_image_and_check_boot14:01
sdaguehttps://jenkins04.openstack.org/job/gate-tempest-dsvm-cells/975/consoleFull14:01
mriedemhrm14:02
mriedemmaybe only busted on the cells job now?14:02
mriedemalaski: didn't you have a patch up in nova for that ebs test?14:02
sdaguehmmm yep14:02
sdaguethe fail rate in cells is way to high to keep that in the current state14:02
*** Poornima has joined #openstack-qa14:03
sdaguehttp://logstash.openstack.org/#eyJzZWFyY2giOiJcIkJsb2NrIERldmljZSBNYXBwaW5nIGlzIEludmFsaWQ6IEJvb3Qgc2VxdWVuY2UgZm9yIHRoZSBpbnN0YW5jZSBhbmQgaW1hZ2UvYmxvY2sgZGV2aWNlIG1hcHBpbmcgY29tYmluYXRpb24gaXMgbm90IHZhbGlkXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0NDEwMjk3MDM1MzF914:03
mriedemyeah, i'm looking at it14:03
*** Kennan has quit IRC14:03
*** Kennan2 has joined #openstack-qa14:03
mriedemi can push a skip patch in nova for the cells job14:03
mriedemb/c we're cool like that14:03
*** malini has joined #openstack-qa14:03
mriedemit also fails in the ceph job a bit14:03
mriedemso just skipping cells won't help since nova / cinder / glance gate on the ceph bjo14:04
mriedem*job14:04
mriedemsdague: should i just push a tempest patch to skip the test?14:05
sdaguemriedem: the cells job skips are managed in the nova tree14:05
mriedemyeah14:05
*** bnemec has joined #openstack-qa14:05
mriedembut ceph isn't14:05
*** safchain has joined #openstack-qa14:05
alaskimriedem: https://review.openstack.org/#/c/218450/ should help the cells job14:05
alaskilooks like it just failed a job in the gate though14:06
sdaguealaski: yes, it failed on the bdm issue14:08
*** dkranz has quit IRC14:08
*** afazekas__ has quit IRC14:08
sdagueso, the cells job managed to exceed 50% failure rate last week - http://graphite.openstack.org/render/?from=-200hours&height=500&until=now&width=800&bgcolor=ffffff&fgcolor=000000&yMax=100&yMin=0&target=color(alias(movingAverage(asPercent(stats.zuul.pipeline.gate.job.gate-tempest-dsvm-cells.FAILURE,sum(stats.zuul.pipeline.gate.job.gate-tempest-dsvm-cells.{SUCCESS,FAILURE})),%2712hours%27),%20%27gate-tempest-dsvm-cells%27),%27orange%27)14:08
sdaguein the gate14:09
*** rbak has joined #openstack-qa14:09
*** zhangjn has quit IRC14:09
*** jroll has quit IRC14:09
*** jroll has joined #openstack-qa14:09
alaskisdague: it's a different failure at least.  it's failing on the cleanup now14:09
sdaguealaski: it's still failing on a mapping issue14:10
sdagueand it's failing at a high enough rate that it's contributing to the world wrecking14:10
sdaguemriedem: so, yes, a skip would be appreciated here14:10
mriedemalright, on the way14:10
*** boris-42 has joined #openstack-qa14:11
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tempest-lib: Fix cli.base.execute with py34  https://review.openstack.org/21887014:11
*** zz_dimtruck is now known as dimtruck14:11
*** anshul has joined #openstack-qa14:11
openstackrecheckopenstack/nova change: https://review.openstack.org/218450 failed because of: gate-tempest-dsvm-cells: https://bugs.launchpad.net/bugs/148958114:12
openstackLaunchpad bug 1489581 in OpenStack Compute (nova) "test_create_ebs_image_and_check_boot is race failing" [Critical,Triaged]14:12
openstackgerritMatt Riedemann proposed openstack/tempest: Skip ebs scenario test due to bug 1489581  https://review.openstack.org/21887114:13
openstackbug 1489581 in OpenStack Compute (nova) "test_create_ebs_image_and_check_boot is race failing" [Critical,Triaged] https://launchpad.net/bugs/148958114:13
mriedemsdague: ^14:14
mriedemsec i need to move the decorator14:16
openstackgerritMatt Riedemann proposed openstack/tempest: Skip ebs scenario test due to bug 1489581  https://review.openstack.org/21887114:16
openstackbug 1489581 in OpenStack Compute (nova) "test_create_ebs_image_and_check_boot is race failing" [Critical,Triaged] https://launchpad.net/bugs/148958114:16
*** zhangjn has joined #openstack-qa14:17
*** shausy has quit IRC14:17
*** mpaolino has joined #openstack-qa14:17
mtreinishmriedem: from tempest-lib import decorators @decorators.skip_because(bug='1489581')14:18
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: update 1490374 for tempest hits as well  https://review.openstack.org/21887614:18
mriedemmtreinish: is that necessary?14:21
mtreinishmriedem: yes, it enables tracking it with skip_tracker14:21
mriedemah, ok, updating14:21
*** zhangjn has quit IRC14:21
*** lawrancejing has quit IRC14:21
mriedemmtreinish: you should have a hacking rule in tempest that blocks testtools.skip :)14:22
*** gholler has joined #openstack-qa14:22
openstackgerritMatthew Treinish proposed openstack/tempest: Skip ebs scenario test due to bug 1489581  https://review.openstack.org/21887114:22
openstackbug 1489581 in OpenStack Compute (nova) "test_create_ebs_image_and_check_boot is race failing" [Critical,Triaged] https://launchpad.net/bugs/148958114:22
mtreinishmriedem: ^^^14:22
mtreinishmriedem: that's not a bad idea14:22
mriedemit's a great idea14:23
*** tpeoples has joined #openstack-qa14:23
mriedemi'll report a bug and rlrossit and add the check14:23
mriedem*can add14:23
mtreinishwfm14:23
*** dkranz has joined #openstack-qa14:24
*** gholler has quit IRC14:24
*** gholler has joined #openstack-qa14:24
*** gholler has quit IRC14:26
*** gholler has joined #openstack-qa14:27
*** pglass has joined #openstack-qa14:28
*** dkranz has quit IRC14:28
openstackgerritMerged openstack-infra/elastic-recheck: add query for misuse of netaddr  https://review.openstack.org/21886514:29
*** rlrossit has joined #openstack-qa14:29
openstackgerritMehdi Abaakouk (sileht) proposed openstack/tempest-lib: Fix cli.base.execute with py34  https://review.openstack.org/21887014:29
*** gholler has quit IRC14:29
*** ctrath has joined #openstack-qa14:30
*** jswarren has joined #openstack-qa14:31
*** rlrossit has left #openstack-qa14:32
*** dkranz has joined #openstack-qa14:32
*** malini has quit IRC14:33
*** malini has joined #openstack-qa14:35
sdaguemriedem: on the next ER update, the uncategorized rate hopefully drops a bunch as that will sweep up the netaddr thing14:38
*** mriedem is now known as mriedem_away14:39
mriedem_awaysdague: ok, gone for about an hour or so14:39
sdaguemriedem_away: no prob14:40
*** dkranz has quit IRC14:43
*** dkranz has joined #openstack-qa14:48
*** achanda has joined #openstack-qa14:49
*** zhangjn has joined #openstack-qa14:49
*** dims has quit IRC14:51
*** itzikb_ has quit IRC14:57
openstackgerritgordon chung proposed openstack/tempest: remove too_slow_to_test flag  https://review.openstack.org/21843215:02
*** dims has joined #openstack-qa15:03
*** ccneill has joined #openstack-qa15:04
*** hemnafk has quit IRC15:04
*** bnemec has quit IRC15:04
*** achanda has quit IRC15:05
*** dpaterson has joined #openstack-qa15:06
*** dkranz has quit IRC15:08
*** bnemec has joined #openstack-qa15:09
*** yfried has quit IRC15:11
*** dkranz has joined #openstack-qa15:11
ccarmackI wonder if anyone can help me.  I uploaded a patch set and now Intel PCI CI is sending me a failure email every minute.  https://review.openstack.org/#/c/218468/15:13
openstackgerritMerged openstack-infra/elastic-recheck: update 1490374 for tempest hits as well  https://review.openstack.org/21887615:14
*** woodster_ has joined #openstack-qa15:18
*** mylu has joined #openstack-qa15:19
*** mylu has quit IRC15:20
*** mylu has joined #openstack-qa15:20
*** mylu has quit IRC15:21
*** mylu has joined #openstack-qa15:21
*** mylu has quit IRC15:22
*** mylu has joined #openstack-qa15:22
*** mylu has quit IRC15:23
*** mylu has joined #openstack-qa15:23
*** mylu has quit IRC15:24
openstackgerritEvgeny Antyshev proposed openstack-infra/devstack-gate: Third-party rhel-based oses in devstack-gate  https://review.openstack.org/21502915:24
*** mylu has joined #openstack-qa15:24
openstackgerritMerged openstack/tempest: Skip ebs scenario test due to bug 1489581  https://review.openstack.org/21887115:24
openstackbug 1489581 in OpenStack Compute (nova) "test_create_ebs_image_and_check_boot is race failing" [Critical,Triaged] https://launchpad.net/bugs/148958115:24
*** mylu has quit IRC15:26
*** jaypipes has quit IRC15:26
*** mylu has joined #openstack-qa15:27
*** mylu has quit IRC15:27
*** mylu has joined #openstack-qa15:27
*** mylu has quit IRC15:29
*** k4n0 has quit IRC15:29
*** neeti has quit IRC15:30
*** mylu has joined #openstack-qa15:30
*** mylu has quit IRC15:31
*** david-ly_ has joined #openstack-qa15:31
*** david-ly_ is now known as david-lyle_15:33
*** mylu has joined #openstack-qa15:33
*** mylu has quit IRC15:33
*** david-lyle has quit IRC15:33
*** mylu has joined #openstack-qa15:33
*** mylu has quit IRC15:34
*** fedexo has joined #openstack-qa15:34
*** mylu has joined #openstack-qa15:34
*** mylu has quit IRC15:35
*** david-lyle_ is now known as david-lyle15:35
*** anshul has quit IRC15:35
*** mylu has joined #openstack-qa15:35
*** mylu has quit IRC15:35
*** mylu has joined #openstack-qa15:36
*** mylu has quit IRC15:37
*** yog_ has quit IRC15:37
*** mylu has joined #openstack-qa15:38
*** mylu has quit IRC15:38
*** mylu has joined #openstack-qa15:39
*** mylu has quit IRC15:39
*** Poornima has quit IRC15:39
*** malini has quit IRC15:40
*** malini has joined #openstack-qa15:40
*** mylu has joined #openstack-qa15:41
*** mylu has quit IRC15:41
ccarmackCan someone please help me, Intel PCI CI is going crazy putting failure messages into my code review https://review.openstack.org/#/c/218468/15:41
*** mylu has joined #openstack-qa15:42
*** mylu has quit IRC15:44
*** mylu has joined #openstack-qa15:44
*** mylu has quit IRC15:45
*** mylu has joined #openstack-qa15:45
*** mylu has quit IRC15:46
*** mylu has joined #openstack-qa15:47
dimsccarmack: let's take it to openstack-infra15:47
*** mylu has quit IRC15:47
*** mylu has joined #openstack-qa15:48
ccarmackdims: ok15:48
*** amotoki has quit IRC15:50
*** Sukhdev has joined #openstack-qa15:51
*** mlavalle has joined #openstack-qa15:54
*** zhangjn has quit IRC15:54
*** mylu has quit IRC15:56
*** mylu has joined #openstack-qa15:57
*** diegows has joined #openstack-qa15:58
*** mylu has quit IRC15:59
openstackgerritIevgeniia Zadorozhna proposed openstack/tempest: Added test_list_tenants non-admin test for v2, v3 api/identity  https://review.openstack.org/19270916:00
*** mylu has joined #openstack-qa16:00
*** sshnaidm has quit IRC16:00
openstackgerritIevgeniia Zadorozhna proposed openstack/tempest: Added Identity API tests for user ec2 credentials  https://review.openstack.org/19458416:00
openstackgerritIevgeniia Zadorozhna proposed openstack/tempest: Added test_user_update_own_password tests for Identity v2, v3  https://review.openstack.org/19760016:01
openstackgerritMichael Krotscheck proposed openstack/eslint-config-openstack: Enable space-in-parens  https://review.openstack.org/21827416:02
*** mylu has quit IRC16:02
*** mylu has joined #openstack-qa16:03
*** mylu has quit IRC16:04
*** sayali has quit IRC16:04
*** mylu has joined #openstack-qa16:04
*** mylu has quit IRC16:04
*** mylu has joined #openstack-qa16:06
*** mylu has quit IRC16:06
*** sayali has joined #openstack-qa16:06
*** dkranz has quit IRC16:06
*** mylu has joined #openstack-qa16:07
*** mriedem_away is now known as mriedem16:08
*** flashgordon is now known as jogo16:09
*** mfalatic has joined #openstack-qa16:09
*** dkranz has joined #openstack-qa16:10
*** mylu has quit IRC16:11
*** mylu has joined #openstack-qa16:12
*** kiran-r has joined #openstack-qa16:13
*** armax has joined #openstack-qa16:15
*** fedexo has quit IRC16:15
*** mpaolino has quit IRC16:16
*** mylu has quit IRC16:16
*** mylu has joined #openstack-qa16:17
*** SridharG has left #openstack-qa16:18
*** mylu has quit IRC16:18
*** safchain has quit IRC16:19
*** mylu has joined #openstack-qa16:19
*** mylu has quit IRC16:20
*** e0ne has quit IRC16:20
*** mylu has joined #openstack-qa16:20
*** mylu has quit IRC16:20
*** mylu has joined #openstack-qa16:22
*** jaypipes has joined #openstack-qa16:22
*** ChanServ sets mode: +o jaypipes16:22
*** Swami has joined #openstack-qa16:24
*** salv-orl_ has joined #openstack-qa16:25
*** amuller has joined #openstack-qa16:25
*** mylu has quit IRC16:25
amullermriedem: heya, about https://review.openstack.org/#/q/status:open+project:openstack-infra/elastic-recheck+branch:master,n,z16:25
amullerI have 6 changes up there for review, one is WIP until the next failure16:26
amullerfor the remaining 5, some of them use specific log queries, and some use the test name16:26
*** mylu has joined #openstack-qa16:27
amullermriedem: would appreciate guidance so we could merge them16:27
*** mylu has quit IRC16:28
*** salv-orlando has quit IRC16:28
mriedemmessage:"Wrongfully passed a connectivity check after applying" AND tags:"console" AND build_name:"gate-neutron-dsvm-functional"16:28
mriedemhas hits for your WIP one16:28
amullerlet me look in to that16:28
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: update signature for another expression  https://review.openstack.org/21894016:28
*** mylu has joined #openstack-qa16:29
amullermriedem: that actually found hits for 2 tests I was not catching in the original logstash query in that patch16:31
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: update signature for another expression  https://review.openstack.org/21894016:31
amullermriedem: so I'll change the query to that and expand the scope of the bug16:31
*** mylu has quit IRC16:31
sdagueamuller: typically we want really narrow signatures so we are sure it's a fixable bug16:31
sdaguemriedem: want to double check the update to - https://review.openstack.org/218940 ?16:32
mriedemsdague: yeah16:32
sdagueI *think* that has all the various iterations on it16:32
mriedemsdague: and on amuller's changes, yeah, it's a problem with fingerprinting the failures in the neutron functional tests16:32
sdagueheat, neutron, horizon and horizon expose it differently16:32
sdaguemriedem: because they don't capture service logs?16:32
*** mylu has joined #openstack-qa16:32
sdagueI feel like that's a bigger issue16:32
amullernay16:33
amullerneutron functional tests dont have service logs because we dont start the services16:33
amullerthe tests are run by the test code itself16:33
amullerso all you get is the console output16:33
*** mylu has quit IRC16:33
sdagueamuller: oh, I thought you were running in a devstack env?16:33
amullersdague: we use devstack to install deps, unstack.sh, then run the tests16:33
*** lifeless has quit IRC16:34
*** mylu has joined #openstack-qa16:34
sdagueamuller: so, where does the logging for the service code go?16:34
*** mylu has quit IRC16:34
amullersdague: what service?16:34
*** mylu has joined #openstack-qa16:35
amullerwe don't start any process (neutron-server, l3/ovs/dhcp/metadata agents) during the functional tests16:35
*** mylu has quit IRC16:35
amullerthe tests import code and execute methods directly16:35
amullerin the process that executes the tests16:35
*** mylu has joined #openstack-qa16:36
*** mylu has quit IRC16:36
*** mylu has joined #openstack-qa16:37
amullersdague: Did I understand your question correctly?16:37
*** mylu has quit IRC16:37
sdagueok, but that code typically have loggers in it16:37
*** malini has quit IRC16:37
sdaguewhere is that going?16:37
*** mylu has joined #openstack-qa16:38
*** gszasz has quit IRC16:38
amullerit should go to the console output16:38
*** mylu has quit IRC16:38
*** mylu has joined #openstack-qa16:39
mriedemsdague: e.g. http://logs.openstack.org/00/218000/2/check/gate-neutron-dsvm-functional/aacc154/console.html#_2015-08-29_16_40_56_66916:39
amullersdague: looks like it's only warning/info/error/trace16:39
amullerso we lose DEBUG levels even for failing tests16:39
mriedemamuller: well you'd have to configure the logger in the test runs to set debug level16:40
mriedemi think nova does16:41
*** malini has joined #openstack-qa16:41
amullerI vageuly remember discussions about that with Maru16:41
sdagueamuller: so, if it's not possible to highlight the failure at the logging level you've got there, it's probably indicative that the logging isn't sufficient for production environments to debug failures either16:41
*** mylu has quit IRC16:41
*** lifeless has joined #openstack-qa16:41
*** mylu has joined #openstack-qa16:42
amullersdague: I don't know if you can make that determination. We have not successfully performed root cause analysis yet on that test, I cannot tell you if the failure is a genuine neutron bug or a test environment failure. It's possible that Neutron does not and should not emit any issues to its logs and that the issue is at the testing framework level.16:42
*** mylu has quit IRC16:42
*** mylu has joined #openstack-qa16:43
sdagueamuller: yeh, so increasing logging to figure that out would be my next suggestion16:44
amullerI was assuming we want elastic search queries for bugs so we knew where to spend our time16:45
amullerif we're required to perform RCA on every bug beforehand we're kinda missing the point no?16:46
mriedemamuller: well, e-r queries are good so you have an idea of what your top races are16:46
amullerright16:46
*** nic has joined #openstack-qa16:46
mriedemamuller: but we generally prefer that the fingerprints come from the service logs rather than just saying 'this test is racey so fingerprint on that'16:46
mriedemamuller: because like i was saying last week, test A could be failing because of bug x or y in the actual code, so fingerprinting on the test could mask that16:47
mriedemespecially in the check queue when joe schmo's patch is introducing bug z to that test16:47
*** mylu has quit IRC16:47
*** davideagnello has joined #openstack-qa16:47
*** jerryz has joined #openstack-qa16:47
mriedembut people ignore it because they just think, well we know the test is racy16:47
mriedemyou have to think, the code is racy, not the test - unless the test doesn't make sense16:48
*** mylu has joined #openstack-qa16:48
amullergenerally I agree16:48
*** mylu has quit IRC16:48
amullerthough here in some cases we don't know16:48
*** mylu has joined #openstack-qa16:48
mriedemwell that's where sdague's point about ffdc comes in16:49
*** davideagnello has quit IRC16:49
mriedemif you can get that into the service logs it *should* help16:49
*** lutostag has quit IRC16:49
mriedemmaybe at least rule some things out16:49
*** yamahata has joined #openstack-qa16:50
*** davideagnello has joined #openstack-qa16:50
*** mylu has quit IRC16:52
*** mylu has joined #openstack-qa16:53
mriedemamuller: anyway, looks like we can fingerprint at least 2 of those bugs with the test output dump, see comments inline, i haven't gotten to the other queries yet16:53
amullermriedem: ack16:53
*** mylu has quit IRC16:53
amullerthanks16:53
*** mylu has joined #openstack-qa16:54
*** mylu has quit IRC16:54
*** mylu has joined #openstack-qa16:55
*** mylu has quit IRC16:55
*** dims has quit IRC16:55
*** mylu has joined #openstack-qa16:55
*** mylu has quit IRC16:56
*** salv-orl_ has quit IRC16:56
*** mylu has joined #openstack-qa16:56
*** mylu has quit IRC16:57
*** lutostag has joined #openstack-qa16:57
*** mylu has joined #openstack-qa16:57
*** mylu has quit IRC16:57
*** aalam has quit IRC16:58
*** mylu has joined #openstack-qa16:58
*** mylu has quit IRC16:58
*** mylu has joined #openstack-qa16:59
*** mylu has quit IRC16:59
*** mylu has joined #openstack-qa17:00
*** mylu has quit IRC17:00
*** mylu has joined #openstack-qa17:01
*** mylu has quit IRC17:02
*** mylu has joined #openstack-qa17:03
*** mylu has quit IRC17:03
*** mylu has joined #openstack-qa17:04
*** mylu has quit IRC17:05
*** rossella_s has quit IRC17:05
amullermriedem: if you look here: http://logs.openstack.org/03/218703/1/check/gate-neutron-dsvm-functional/5f40d6d/console.html, right now I'm matching on 'time.sleep(0.25)', I thought I could match on 'time.sleep(0.25)' AND async_ping17:06
amullerto be more specific17:06
amullerbut it's returning 0 results17:06
*** rossella_s has joined #openstack-qa17:06
amullermessage:"time.sleep(0.25)" AND message:"async_ping" AND tags:"console" AND build_name:"gate-neutron-dsvm-functional"17:06
amullermriedem: any idea why?17:06
*** mylu has joined #openstack-qa17:06
*** mylu has quit IRC17:06
*** mylu has joined #openstack-qa17:07
*** mylu has quit IRC17:07
*** mylu has joined #openstack-qa17:09
*** mylu has quit IRC17:09
*** mylu has joined #openstack-qa17:09
*** kiran-r has quit IRC17:10
*** mylu has quit IRC17:10
*** ccarmack has left #openstack-qa17:10
*** mylu has joined #openstack-qa17:10
*** mylu has quit IRC17:11
*** mylu has joined #openstack-qa17:11
*** mylu has quit IRC17:12
*** mylu has joined #openstack-qa17:13
*** mylu has quit IRC17:13
*** mylu has joined #openstack-qa17:14
*** mylu has quit IRC17:14
*** mylu has joined #openstack-qa17:15
*** mylu has quit IRC17:15
*** mylu has joined #openstack-qa17:15
*** mylu has quit IRC17:15
*** mylu has joined #openstack-qa17:17
*** mylu has quit IRC17:17
*** achanda has joined #openstack-qa17:18
*** nic has quit IRC17:18
mriedemamuller: console logs aren't multiline17:19
*** mylu has joined #openstack-qa17:20
amullermriedem: so the entire console output is considered a single string without newlines?17:20
amulleror, sorry, a single string with newlines17:20
mriedemevery console line is it's own thing17:20
mriedemeach timestamp17:20
*** jasonsb has joined #openstack-qa17:20
mriedemit's not like a multiline stacktrace in a service log17:20
*** aix has quit IRC17:21
amullerok17:21
amullermriedem: so it's not possible then?17:21
mriedemright17:23
*** nic has joined #openstack-qa17:24
* amuller is sad17:24
*** e0ne has joined #openstack-qa17:26
*** dims has joined #openstack-qa17:29
*** amuller is now known as amuller_afk17:29
*** dims_ has joined #openstack-qa17:30
*** dims has quit IRC17:30
*** jasonsb has quit IRC17:33
*** jasonsb has joined #openstack-qa17:33
*** e0ne has quit IRC17:36
*** jasonsb_ has joined #openstack-qa17:37
*** pvaneck has joined #openstack-qa17:37
*** jasonsb has quit IRC17:38
*** nikita_09 has joined #openstack-qa17:39
*** e0ne has joined #openstack-qa17:39
*** mylu has quit IRC17:41
*** mylu has joined #openstack-qa17:42
nikita_09mtreinish: hi ..is there any ETA of tempest tag 6 ?17:43
nikita_09like when is it likely going to happen?17:43
*** ig0r__ has joined #openstack-qa17:43
mtreinishnikita_09: I was planning to do it in 1 of 2 cases, either the liberty release, or when a project starts a gate job that uses the tempest plugin interface17:43
mtreinishthe liberty release tag will happen but it will be 7 if the plugin thing happens first17:44
nikita_09mtreinish: the later(tempest plugin interface) is a WIP or done completely?17:44
openstackgerritSean Dague proposed openstack-infra/devstack-gate: warn when workspace setup takes a long time  https://review.openstack.org/21896617:44
mtreinishnikita_09: the interface is "done" in that all the pieces we planned for should be in place17:44
mtreinishbut no project has started using it yet17:45
mtreinish(also I guess the cookiecutter repo to help init a plugin is still a wip)17:45
mtreinishbut that's not a requirement in tempest itself17:45
nikita_09ok..thats good to know..by plugin interface , you mean something like we can use it via stevedore..? im not sure i completely understand plugin :/17:46
*** ig0r_ has quit IRC17:46
mtreinishnikita_09: http://docs.openstack.org/developer/tempest/plugin.html17:46
*** VanL has joined #openstack-qa17:46
nikita_09mtreinish: thank you :)17:46
mtreinishnikita_09: for an example patch mkoderer, has been working on getting manila migrated to use the interface: https://review.openstack.org/#/c/201955/17:48
*** aix has joined #openstack-qa17:48
*** Sukhdev has quit IRC17:51
*** shayneburgess has left #openstack-qa17:59
openstackrecheckConsole logs not available after 13:30s for gate-horizon-tox-py27dj18 218815,3,ac294ed18:03
*** salv-orlando has joined #openstack-qa18:04
*** mylu has quit IRC18:06
*** davideagnello has quit IRC18:06
*** dwalleck has joined #openstack-qa18:06
*** mylu has joined #openstack-qa18:07
*** salv-orlando has quit IRC18:09
*** malini has quit IRC18:09
*** mylu has quit IRC18:11
*** mylu has joined #openstack-qa18:13
*** malini has joined #openstack-qa18:14
*** mylu has quit IRC18:16
*** hitalia has joined #openstack-qa18:16
*** mylu has joined #openstack-qa18:16
openstackrecheckConsole logs not available after 13:28s for gate-tempest-dsvm-neutron-large-ops 216732,4,1adcbcd18:19
openstackgerritSean Dague proposed openstack-infra/elastic-recheck: add signature for cinder pyopenssl issue  https://review.openstack.org/21897618:20
*** b10n1k_ has joined #openstack-qa18:21
*** mylu has quit IRC18:21
sdaguemriedem / mtreinish - https://review.openstack.org/#/c/218976/118:22
mriedemsdague: where is all of this e-r energy coming from?18:22
mriedemi like it18:22
openstackgerritSean Dague proposed openstack-infra/devstack-gate: warn when workspace setup takes a long time  https://review.openstack.org/21896618:22
sdaguemriedem: pure rage?18:23
mriedemsdague: but no gdi?18:23
sdaguehonestly, my well pump is out on my lawn right now18:23
mmmporkoh man i found the power adapter!18:23
mriedemarwen has been at it again18:24
sdagueso in a day where I'm dealing with people replacing that, it's hard to imagine concentrating on something that takes longer than tracking down bugs18:24
mmmporkit was hiding in a non obvious place.. although the spouse had said he couldn't remember 100% if he'd packed it so i assumed the worse18:24
sdagueso I figured I'd just hack away at the gate races18:24
mmmporkoh jeez wrong channel, sorry18:25
*** davideagnello has joined #openstack-qa18:25
mriedemmmmpork: that could have gone much worse :)18:26
mriedem"i don't think they'll find the body, i hid it real good"18:26
sdaguemaybe down my well, which is why it stopped working18:27
*** dwalleck has quit IRC18:27
mmmporkhahahaa18:28
*** dtantsur|afk is now known as dtantsur18:29
*** VanL has quit IRC18:31
*** vgridnev has joined #openstack-qa18:31
*** mylu has joined #openstack-qa18:32
*** ccneill has quit IRC18:34
*** salv-orlando has joined #openstack-qa18:39
openstackrecheckConsole logs not available after 13:43s for gate-cinder-python27 218407,2,f0718e318:40
*** pcm_ has joined #openstack-qa18:43
*** pc_m has quit IRC18:45
*** ig0r__ has quit IRC18:48
lifelesssdague: so looking at it18:52
lifelesssdague: thats the only edit-constraints call happening in that run18:52
lifelesssdague: is there anything special/different/unique about keystone ?18:52
sdaguenot that I know off18:52
sdagueI expect it's first18:53
sdaguelifeless: yeh, it's mostly it's the first time you are running that code18:53
sdaguethat directory is clearly there18:53
sdagueI have no idea why permissions would be different on 2 nodes18:53
openstackrecheckConsole logs not available after 13:23s for gate-horizon-python27 218594,2,b4cdcdc18:54
*** ccneill has joined #openstack-qa18:54
lifelesssudo chmod +r /usr/lib/python2.7/dist-packages/prettytable-0.7.2.egg-info/PKG-INFO <- worries me a little18:54
mriedemsdague: can you poke at this? https://review.openstack.org/#/c/216360/18:54
sdaguewe've had that forever18:54
sdaguelifeless: they completely f'ed up their permissions and it makes it ununinstallable on linux18:55
lifelesssdague: sure, still :). Anyhow I can't see a sudo chmod affecting /opt18:55
sdaguelifeless: right18:55
lifelesssdague: the binary package ?18:55
sdaguelifeless: honestly, I don't remember the full origin, it went in years ago18:56
lifelessk, anyhow, side issue18:56
lifelessso this node was devstack-trusty-rax-ord-465649118:56
*** VanL has joined #openstack-qa18:57
*** edmondsw has joined #openstack-qa18:57
lifelessdims_: your other one was devstack-trusty-rax-iad-463622618:57
lifelessand the other edit-constraints one was devstack-trusty-hpcloud-b3-463788018:58
*** amuller_afk is now known as amuller18:58
*** sabeen1 has quit IRC18:59
lifelesssdague: http://logs.openstack.org/86/218286/1/gate/gate-tempest-dsvm-neutron-full/f070942/logs/worlddump-2015-08-31-125459.txt.gz18:59
lifelesssdague: it looks like ansible is running a bunch of stuff concurrently19:00
lifelessjenkins  15731 16533  4.5  0.2  85008 19180 ?        S    12:54:54 00:00:00 /tmp/ansible/bin/python /tmp/ansible/bin/ansible all -f 5 -i /home/jenkins/workspace/gate-ceilometer-dsvm-functional-mongodb/inventory -m shell -a source '/home/jenkins/workspace/gate-ceilometer-dsvm-functional-mongodb/test_env.sh' source '/home/jenkins/workspace/gate-ceilometer-dsvm-functional-mongodb/devstack-gate/functions.sh' set -o errexit tsfilter s19:00
lifelessthis is from the dsvm-neutron-full log19:00
lifelessbut I'm not familiar with the ansible glue yet, so I might just be jumping at shadows19:01
lifelesshowever, incase I'm not, let me invoke clarkb & mordred ^19:01
clarkblifeless: yes it should run things conccurently19:02
clarkblifeless: but on most jobs its single host so serial19:02
mtreinishlifeless: shadows are scary :)19:02
clarkbonly the -multinode jobs should actually have >1 ansible thing running at a time19:02
lifelessclarkb: so while devstack is running ansible should be recloning repositories ?19:02
*** VanL has quit IRC19:02
clarkblifeless: no19:02
lifelessclarkb: that is what appears to be happening here19:02
clarkblifeless: ansible triggers cloning repos and devstack as separate steps19:02
*** dtantsur is now known as dtantsur|afk19:03
lifelessclarkb: ok, so here is where I'm at. sdague reported a weird permission denited from edit-cosntraints19:03
lifelesserm19:03
lifelessedit-constraints19:03
lifelesshttp://logstash.openstack.org/#eyJzZWFyY2giOiJcInVwcGVyLWNvbnN0cmFpbnRzLnR4dC50bXBcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQ0MTA0NjE4MTE3NX0=19:03
clarkblifeless: so those steps can run on >1 node at a time but in that job its a single node and the steps shouldn't overlap19:03
lifelessclarkb: then dims reported permission denied from another spot: http://logs.openstack.org/07/202207/18/gate/gate-neutron-lbaasv2-dsvm-api/4e704ba/console.html#_2015-08-30_04_17_05_45419:03
lifelessclarkb: and I was looking for reasons that we might get permission denied there - I went looking for dmesg output to see if the fs had gone readonly or something19:04
lifelessand found git cloning stuff in worlddump19:04
clarkblifeless: looking at that world dump I only see one git clone?19:04
lifelessyup19:05
clarkbok so where do you see them overlapping?19:05
lifelessand the ansible invoking stuff for a different test run19:05
lifelessthis is a gate-tempest-dsvm-neutron-full job, but the ansible code is running in workspace/gate-ceilometer-dsvm-functional-mongodb19:05
lifelessif you look at the timestamps the error occurs at http://logs.openstack.org/86/218286/1/gate/gate-tempest-dsvm-neutron-full/f070942/logs/devstacklog.txt.gz#_2015-08-31_12_54_59_01819:06
clarkbgotcha so thats more likely node reuse19:06
clarkband unrelated to ansible I think19:06
lifelessand ansible starts at http://logs.openstack.org/86/218286/1/gate/gate-tempest-dsvm-neutron-full/f070942/logs/syslog.txt.gz#_Aug_31_12_54_5919:06
*** yfried has joined #openstack-qa19:06
lifelessI thought we never reused nodes?19:06
clarkblifeless: we shouldn't but that looks like we ran a mongodb test on the same node as the neutron full19:07
lifelessactually no, ansible starts at http://logs.openstack.org/86/218286/1/gate/gate-tempest-dsvm-neutron-full/f070942/logs/syslog.txt.gz#_Aug_31_12_54_5519:07
lifelessclarkb: yes, that would explain ansible running concurrently :)19:07
openstackrecheckConsole logs not available after 13:23s for gate-tempest-dsvm-large-ops 218871,3,30cb7d119:07
lifelesssdague: / dims: clarkb: ok, so I'm going to go 'not an edit-constraints problem' and get my nose out- got to go do family morning stuff19:07
*** VanL has joined #openstack-qa19:08
clarkbI will dig in nodepool logs in a moment to see what and whee thinks it ran there19:08
*** malini has quit IRC19:12
clarkbaccording to nodepool it only knows about the neutron-full job19:13
*** malini has joined #openstack-qa19:14
openstackgerritMerged openstack-infra/elastic-recheck: update signature for another expression  https://review.openstack.org/21894019:15
sdagueclarkb: you're suspecting node reuse?19:16
clarkbsdague: ya, since those commands imply that two different d-g runs called ansible19:16
clarkbthe workspace path is the give away19:16
sdagueclarkb: ah, cool19:16
clarkbhowever according to nodepool there was just the one job19:16
sdagueis there anything finger printable there?19:16
clarkbI wonder (and this is a huge wonder) if two nodes got the same IP addr19:16
clarkbso looking at ipa addresses now19:17
sdagueor something we can expose in d-g as a warning if we detect that, so we can track this kind of issue19:17
*** dwalleck has joined #openstack-qa19:18
clarkbsdague: I am not sure19:18
clarkbtrying to figure out what happened still19:18
*** dwalleck has quit IRC19:20
sdaguelifeless: though, on the "back in your wheelhouse" when you get back - http://logs.openstack.org/27/211727/5/gate/gate-grenade-dsvm-neutron/ce9e23d//logs/grenade.sh.txt.gz#_2015-08-31_19_02_27_99119:20
openstackrecheckConsole logs not available after 13:27s for gate-horizon-python27 198089,2,530a54d19:21
sdagueI thought we were supposed to be imune to those kinds of mismatches in pkg_resources19:21
clarkbhttp://logs.openstack.org/22/218822/1/check/gate-ceilometer-dsvm-functional-mongodb/f3bc93b/console.html#_2015-08-31_12_54_53_117 I think thats the other job19:21
clarkbsdague: yup same IP19:23
clarkbtwo different nodes19:23
*** dwalleck has joined #openstack-qa19:26
*** malini has quit IRC19:33
openstackrecheckConsole logs not available after 13:35s for gate-keystone-python27 218481,3,f001b5d19:34
*** davideag_ has joined #openstack-qa19:35
openstackgerritJohn Warren proposed openstack/tempest: Add hacking check for testtools.skip  https://review.openstack.org/21899919:36
mtreinishandreaf: around? if you get a sec I want to get the test accounts jobs working again19:37
*** davideagnello has quit IRC19:37
mtreinishright now: https://review.openstack.org/216522 is the blocker for that19:37
openstackgerritAssaf Muller proposed openstack-infra/elastic-recheck: Add query for Neutron + functional bug 1478847  https://review.openstack.org/21845519:37
openstackbug 1478847 in neutron "test_udp_{input|output}.* fails with " AssertionError: RuntimeError not raised."" [High,In progress] https://launchpad.net/bugs/1478847 - Assigned to Jakub Libosvar (libosvar)19:37
*** salv-orlando has quit IRC19:37
*** bknudson has joined #openstack-qa19:42
*** mylu has quit IRC19:48
*** mylu has joined #openstack-qa19:48
openstackrecheckConsole logs not available after 13:25s for gate-grenade-dsvm-partial-ncpu 210689,4,59cd89c19:48
*** yfried has quit IRC19:49
*** Sukhdev has joined #openstack-qa19:49
*** salv-orlando has joined #openstack-qa19:50
*** davideagnello has joined #openstack-qa19:52
*** mylu has quit IRC19:53
openstackgerritJulien Danjou proposed openstack-dev/grenade: Fix whitespace handling in ensure_services_stopped  https://review.openstack.org/21900619:53
*** davideag_ has quit IRC19:53
*** mylu has joined #openstack-qa19:56
vgridnevhey, folks! Is there someone who can review the following change: https://review.openstack.org/#/c/214194/ ?19:57
*** VanL_ has joined #openstack-qa20:01
*** david-lyle has quit IRC20:01
*** mylu has quit IRC20:02
openstackrecheckConsole logs not available after 13:24s for gate-cinder-python27 205834,9,fb832c520:03
*** mylu has joined #openstack-qa20:03
mtreinishamuller: btw, did you see I fixed the bug you reported about the blacklist file: https://review.openstack.org/#/c/218527/20:04
mtreinishturns out there were a couple of issues there20:04
amullermtreinish: I did, didn't get a chance to play with it yet20:04
*** VanL has quit IRC20:04
amullermtreinish: also couldn't fix that patch I pushed to ostestr20:05
amullermtreinish: just can't get to it it seems20:05
mtreinishamuller: no worries, it's definitely a lower prio20:05
*** VanL_ has quit IRC20:05
*** david-lyle has joined #openstack-qa20:06
openstackgerritMerged openstack-infra/elastic-recheck: add signature for cinder pyopenssl issue  https://review.openstack.org/21897620:07
*** mylu has quit IRC20:09
*** mylu has joined #openstack-qa20:09
*** dims has joined #openstack-qa20:12
*** dims has quit IRC20:12
*** dims_ has quit IRC20:14
*** ccneill has quit IRC20:16
*** malini has joined #openstack-qa20:16
*** skiafa has quit IRC20:17
openstackgerritJohn Warren proposed openstack/tempest: Add hacking check for testtools.skip  https://review.openstack.org/21899920:17
*** dwalleck_ has joined #openstack-qa20:18
openstackrecheckConsole logs not available after 13:22s for gate-grenade-dsvm 217255,1,537225b20:19
*** dwalleck has quit IRC20:21
*** dwalleck_ has quit IRC20:22
*** austin81 has joined #openstack-qa20:25
*** vgridnev has quit IRC20:27
openstackgerritSergey Vilgelm proposed openstack/tempest: Use oslo.log library instead of system logging module  https://review.openstack.org/20960120:32
openstackrecheckConsole logs not available after 13:22s for gate-horizon-python27 218815,3,fcea59b20:33
*** svilgelm has joined #openstack-qa20:35
*** achanda has quit IRC20:38
*** achanda has joined #openstack-qa20:41
*** achanda has quit IRC20:41
*** malini has quit IRC20:41
*** achanda has joined #openstack-qa20:41
openstackrecheckConsole logs not available after 13:25s for gate-grenade-dsvm 218597,4,a918d2b20:48
*** bkopilov has quit IRC20:48
*** VanL has joined #openstack-qa20:50
*** hoangcx has joined #openstack-qa20:50
*** regXboi has joined #openstack-qa20:51
*** bkopilov has joined #openstack-qa20:53
*** Swami has quit IRC20:53
*** yamahata has quit IRC20:55
*** mwagner_lap has quit IRC20:56
*** rfolco has quit IRC20:57
*** yamahata has joined #openstack-qa20:57
*** dims has joined #openstack-qa20:57
*** bkopilov has quit IRC21:00
*** jasonsb_ has quit IRC21:01
*** hichihara has joined #openstack-qa21:01
openstackrecheckConsole logs not available after 13:26s for gate-tempest-dsvm-postgres-full 205835,8,3ba8ca421:02
*** svilgelm has left #openstack-qa21:04
*** rossella_s has quit IRC21:05
*** rossella_s has joined #openstack-qa21:06
*** regXboi has quit IRC21:10
*** svilgelm has joined #openstack-qa21:11
*** jasonsb has joined #openstack-qa21:13
*** mylu has quit IRC21:13
*** regXboi has joined #openstack-qa21:14
*** rmoats has joined #openstack-qa21:15
openstackrecheckConsole logs not available after 13:23s for gate-tempest-dsvm-full 216378,6,c4bad3321:15
*** svilgelm has quit IRC21:16
*** ylobanko_ has joined #openstack-qa21:16
*** ylobanko_ has quit IRC21:18
*** regXboi has quit IRC21:19
*** bkopilov has joined #openstack-qa21:20
*** dustins has quit IRC21:26
*** VanL has quit IRC21:27
openstackgerritMatt Riedemann proposed openstack-infra/elastic-recheck: Add query for sqla-migrate 0.9.8 bug 1490740  https://review.openstack.org/21903321:29
openstackbug 1490740 in OpenStack-Gate "sqlalchemy-migrate 0.9.8 release breaks stable/kilo because of capped pbr" [Undecided,In progress] https://launchpad.net/bugs/1490740 - Assigned to Matt Riedemann (mriedem)21:29
mriedemmtreinish: ^21:29
openstackrecheckConsole logs not available after 13:30s for gate-grenade-dsvm 218973,1,295f29221:29
*** dkranz has quit IRC21:30
mtreinishmriedem: +A21:30
*** austin81 has left #openstack-qa21:33
*** e0ne has quit IRC21:34
mriedemmtreinish: you're stable core, https://review.openstack.org/#/c/219027/21:41
mriedemwe're blocked on stable/kilo and master (grenade) until that lands and is synced21:41
openstackrecheckConsole logs not available after 13:37s for gate-cinder-python27 187270,24,dc462c821:43
*** regXboi has joined #openstack-qa21:45
mtreinishmriedem: couldn't you have just push 0.9.9 which had the right deps?21:46
mtreinishit's not like this is an upstream break where we can't push a quick release21:47
mriedemmtreinish: i guess that's fungi's point in the ML21:47
mtreinishoh, there's a ml thread?21:47
mriedemyeah, so i pushed 0.10.0 that liberty will use and so we could cap that on kilo21:47
mtreinishbut you can still push a 0.9.921:47
mriedemyeah21:47
mtreinishoh, but we'll still need to cap21:48
mtreinishbecause you pushed the higher version21:48
* mtreinish shakes fist21:48
*** regXboi has quit IRC21:48
mriedemi know i know21:48
mtreinishmriedem: +A21:48
mriedemi'm trying to think through a revert of the pbr change on a stable/kilo branch21:48
*** Sukhdev has quit IRC21:49
mriedemi can't just revert https://github.com/stackforge/sqlalchemy-migrate/commit/8252703f56d394743199e702c38f1ecc1db590e5 since that would break the py26 and py34 jobs again, i think21:49
*** rmoats has quit IRC21:49
mtreinishmriedem: can't you just leave things alone?21:50
mriedemmy thinking was, if we need stable/kilo 0.9.x changes, we create stable/kilo from 0.9.7 and then patch that and release 0.9.921:50
mriedemthat's what i'm saying21:50
mriedemand was my initial thought21:50
mriedembasically, blacklist 0.9.8 and act like it never happened21:50
mtreinishyep21:50
mtreinishkrotscheck: I'll review those js rule patches, but do realize I've not really ever used js21:51
*** Sukhdev has joined #openstack-qa21:51
mtreinishso I'm fine with whatever crazy rules, since they dont mean anything to me :)21:51
mriedemmake jogo review them21:52
krotscheckmtreinish: Honestly, I really only need you on https://review.openstack.org/#/c/218389/ and the two following.21:53
krotscheckAs for rules....21:53
krotscheckIt's a bit difficult tof rme to get people to review rules.21:53
krotscheckEspecially during feature freeze21:53
krotscheckSo I'm throwing things at the wall to see what sticks.21:54
krotscheckmtreinish: Honestly, I would prefer if those eslint rules do NOT automatically get a +A21:54
krotscheckGive more people bit of time to weigh in on them.21:54
mtreinishkrotscheck: that's fine with me, I was just saying for extra rules you can basically view me as a +2 bot21:56
mtreinishalso fwiw, I do like truthy and falsey21:56
krotscheckmtreinish: Can I replace you with a small shell script? :)21:56
mtreinishkrotscheck: yes, yes you can :)21:56
krotscheckmtreinish: Then vote that way!21:56
openstackrecheckConsole logs not available after 13:24s for gate-cinder-pep8 218750,4,6e9791821:57
krotscheckmtreinish: Serious question: How strningent are we with stylist curles in openstack?21:58
krotscheck*rule21:58
*** sabeen has joined #openstack-qa21:58
krotscheckArgh spelling on moving bus21:58
krotscheckThere's a lot of rules out there that can be dogmatic, ven restrictive.21:58
krotscheckOn one side, we can keep them off and let everyone just do whatever they want.21:59
jogomriedem: I am not sqla-migrate core am I?21:59
krotscheckBut at what point does code style switch from "That's a preference" to "We really should do this the same way"21:59
mtreinishkrotscheck: well for python we're not super strict about it, there are basic alignment rules, but there is some flexability21:59
mtreinishkrotscheck: it's really a project level decision, for the hacking stuff it was already some share culture we were encaspsulating21:59
mtreinishkrotscheck: for the js stuff, I'd say be a bit more open at first22:00
krotscheckI'm german, so I fall on the ALL THE RULES! side of things.22:00
mtreinishand lock it down as a common style evolves22:00
mtreinishkrotscheck: sure, I can understand wanting to do that :)22:00
krotscheckmakes sense to me22:00
*** hoangcx has quit IRC22:00
openstackgerritMerged openstack-infra/elastic-recheck: Add query for sqla-migrate 0.9.8 bug 1490740  https://review.openstack.org/21903322:01
openstackbug 1490740 in OpenStack-Gate "sqlalchemy-migrate 0.9.8 release breaks stable/kilo because of capped pbr" [Undecided,In progress] https://launchpad.net/bugs/1490740 - Assigned to Matt Riedemann (mriedem)22:01
*** dpaterson has quit IRC22:06
openstackrecheckConsole logs not available after 13:25s for gate-cinder-python27 212861,10,91b5c9922:10
*** pglass has quit IRC22:10
*** dimtruck is now known as zz_dimtruck22:11
*** mlavalle has quit IRC22:12
*** amuller has quit IRC22:16
openstackrecheckConsole logs not available after 13:22s for gate-tempest-dsvm-full 217415,3,1f4ad9222:24
*** edmondsw has quit IRC22:28
*** jaypipes has quit IRC22:32
*** pcm_ has quit IRC22:33
openstackrecheckConsole logs not available after 13:25s for gate-grenade-dsvm-partial-ncpu 211753,21,8ade5c022:37
*** ctrath has quit IRC22:42
openstackrecheckConsole logs not available after 13:28s for gate-tempest-dsvm-neutron-large-ops 212485,1,b2111cc22:51
*** mwagner_lap has joined #openstack-qa23:00
*** rbak has quit IRC23:01
openstackrecheckConsole logs not available after 13:22s for gate-cinder-pep8 201578,29,ef2af6123:04
*** tosky has quit IRC23:06
*** jasonsb has quit IRC23:13
openstackrecheckConsole logs not available after 13:26s for gate-grenade-dsvm 212484,1,7f3b17a23:18
openstackrecheckConsole logs not available after 13:32s for gate-grenade-dsvm 212480,1,ff31e9f23:31
openstackrecheckConsole logs not available after 13:22s for gate-cinder-pep8 218645,4,3bfb55323:44
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Make update_server use **kwargs  https://review.openstack.org/21745923:45
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Add "server" to action methods - part 2  https://review.openstack.org/21745623:45
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Do not pass wait_until to create_server()  https://review.openstack.org/21751323:45
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Remove unused 'wait_until' parameter  https://review.openstack.org/21746123:45
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Add docstring for tenant_network  https://review.openstack.org/21751223:45
openstackgerritKen'ichi Ohmichi proposed openstack/tempest: Make create_server use **kwargs  https://review.openstack.org/21746823:45
*** tochi has joined #openstack-qa23:46
*** Sukhdev has quit IRC23:51
*** diegows has quit IRC23:53
*** achanda_ has joined #openstack-qa23:58
openstackrecheckConsole logs not available after 13:23s for gate-tempest-dsvm-neutron-full 212479,1,3f7a75623:58

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!