| *** achanda has quit IRC | 00:02 | |
| *** zz_dimtruck is now known as dimtruck | 00:03 | |
| *** mhorban has quit IRC | 00:03 | |
| *** mahito has joined #openstack-qa | 00:04 | |
| *** sslypushenko has quit IRC | 00:06 | |
| *** openstack has joined #openstack-qa | 00:08 | |
| *** mhorban has joined #openstack-qa | 00:11 | |
| *** Swami has quit IRC | 00:11 | |
| *** sslypushenko has joined #openstack-qa | 00:15 | |
| *** samueldmq has quit IRC | 00:15 | |
| *** jogo has quit IRC | 00:17 | |
| *** yamamoto has quit IRC | 00:18 | |
| *** suneelb has quit IRC | 00:18 | |
| *** dims has joined #openstack-qa | 00:27 | |
| *** tsekiyam_ has joined #openstack-qa | 00:28 | |
| *** mtanino_ has joined #openstack-qa | 00:29 | |
| *** jecarey has joined #openstack-qa | 00:29 | |
| *** mtanino has quit IRC | 00:29 | |
| *** tsekiyama has quit IRC | 00:31 | |
| *** bkopilov has quit IRC | 00:32 | |
| *** tsekiyam_ has quit IRC | 00:32 | |
| *** dmorita has joined #openstack-qa | 00:43 | |
| *** davideagnello has quit IRC | 00:43 | |
| *** davideagnello has joined #openstack-qa | 00:44 | |
| *** mariza10 has quit IRC | 00:45 | |
| *** amotoki has joined #openstack-qa | 00:47 | |
| *** davideagnello has quit IRC | 00:49 | |
| *** bkopilov has joined #openstack-qa | 00:50 | |
| *** shashankhegde has quit IRC | 00:53 | |
| *** Poornima has joined #openstack-qa | 01:01 | |
| *** krtaylor has quit IRC | 01:04 | |
| *** sputnik13 has quit IRC | 01:05 | |
| *** yamamoto has joined #openstack-qa | 01:06 | |
| *** jogo has joined #openstack-qa | 01:10 | |
| *** krtaylor has joined #openstack-qa | 01:17 | |
| *** mriedem has quit IRC | 01:26 | |
| *** dboik has joined #openstack-qa | 01:36 | |
| *** Yogi has joined #openstack-qa | 01:39 | |
| *** dboik has quit IRC | 01:40 | |
| *** weshay has quit IRC | 01:44 | |
| *** dims has quit IRC | 01:46 | |
| *** patrickeast has quit IRC | 01:47 | |
| *** Nic has quit IRC | 01:47 | |
| openstackrecheck | Console logs not available after 13:24s for gate-devstack-bashate 159885,5,d74043b | 01:52 |
|---|---|---|
| *** shashankhegde has joined #openstack-qa | 02:00 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/154593 failed because of: gate-devstack-dsvm-cells: unrecognized error, gate-tempest-dsvm-full-ceph: https://bugs.launchpad.net/bugs/1449136 | 02:04 |
| openstack | Launchpad bug 1449136 in OpenStack-Gate "OpenStack pypi mirrors disconnecting connections" [Undecided,New] | 02:04 |
| *** mtanino_ has quit IRC | 02:06 | |
| *** yamahata has quit IRC | 02:06 | |
| *** Poornima has quit IRC | 02:10 | |
| *** bkopilov has quit IRC | 02:11 | |
| *** mahito has quit IRC | 02:31 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/172775 failed because of: gate-nova-tox-functional: https://bugs.launchpad.net/bugs/1282876, gate-tempest-dsvm-neutron-large-ops: unrecognized error | 02:34 |
| openstack | Launchpad bug 1282876 in OpenStack-Gate "git clone fails with "fatal: Not a git repository", "git remote update failed."" [Critical,Fix released] - Assigned to Jeremy Stanley (fungi) | 02:34 |
| *** mahito has joined #openstack-qa | 02:35 | |
| *** dboik has joined #openstack-qa | 02:43 | |
| *** dboik_ has joined #openstack-qa | 02:44 | |
| *** ygoto has joined #openstack-qa | 02:45 | |
| *** dims has joined #openstack-qa | 02:47 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/168907 failed gate-grenade-dsvm, gate-tempest-dsvm-large-ops, gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error | 02:48 |
| *** dboik has quit IRC | 02:48 | |
| *** gamado has quit IRC | 02:49 | |
| *** dims has quit IRC | 02:52 | |
| *** jyuso1 has quit IRC | 03:06 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/172896 failed gate-tempest-dsvm-large-ops in the gate queue with an unrecognized error | 03:17 |
| *** yamahata has joined #openstack-qa | 03:24 | |
| *** mahito has quit IRC | 03:29 | |
| *** mahito has joined #openstack-qa | 03:37 | |
| *** mahito has quit IRC | 03:41 | |
| *** yfried|afk is now known as yfried_ | 03:45 | |
| *** fedexo has joined #openstack-qa | 03:47 | |
| *** shashankhegde has quit IRC | 03:47 | |
| *** aopincar has joined #openstack-qa | 03:48 | |
| *** gamado has joined #openstack-qa | 03:48 | |
| *** aopincar has quit IRC | 03:50 | |
| *** dimtruck is now known as zz_dimtruck | 03:52 | |
| *** Yogi has quit IRC | 03:52 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: WIP: Add CLI tool to graph aggregate failure counts for tests https://review.openstack.org/178039 | 03:56 |
| *** yfried_ is now known as yfried|afk | 04:00 | |
| *** Yogi has joined #openstack-qa | 04:01 | |
| *** Yogi has quit IRC | 04:01 | |
| raies | mtreinish: hi | 04:04 |
| raies | mtreinish: I have one question. | 04:04 |
| *** hitalia has quit IRC | 04:06 | |
| *** bkopilov has joined #openstack-qa | 04:07 | |
| *** subscope_ has joined #openstack-qa | 04:09 | |
| *** zz_dimtruck is now known as dimtruck | 04:13 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/172910 failed gate-tempest-dsvm-large-ops, gate-tempest-dsvm-neutron-large-ops in the gate queue with an unrecognized error | 04:14 |
| *** yfried|afk is now known as yfried_ | 04:15 | |
| *** annashen has joined #openstack-qa | 04:16 | |
| *** shashankhegde has joined #openstack-qa | 04:16 | |
| *** dimtruck is now known as zz_dimtruck | 04:23 | |
| *** annashen has quit IRC | 04:23 | |
| *** annashen has joined #openstack-qa | 04:29 | |
| *** yfried_ is now known as yfried|afk | 04:31 | |
| *** Sukhdev has joined #openstack-qa | 04:37 | |
| *** yfried|afk is now known as yfried_ | 04:40 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: WIP: Add CLI tool to graph aggregate failure counts for tests https://review.openstack.org/178039 | 04:41 |
| *** mahito has joined #openstack-qa | 04:42 | |
| *** kiran-r has joined #openstack-qa | 04:42 | |
| *** annashen has quit IRC | 04:44 | |
| *** achanda has joined #openstack-qa | 04:44 | |
| *** sks has joined #openstack-qa | 04:46 | |
| *** fedexo has quit IRC | 04:53 | |
| *** fedexo has joined #openstack-qa | 04:54 | |
| *** subscope_ has quit IRC | 04:57 | |
| openstackgerrit | YAMAMOTO Takashi proposed openstack-infra/devstack-gate: Handle the case of REMAINING_TIME <= 0 https://review.openstack.org/178043 | 04:58 |
| *** annashen has joined #openstack-qa | 04:59 | |
| *** yfried_ has quit IRC | 05:01 | |
| *** madhu_ak has joined #openstack-qa | 05:02 | |
| madhu_ak | hi everybody | 05:03 |
| madhu_ak | I am trying to explore testscenarios tempest tests, just checking whether there are any testscenarios implemented in any openstack project? | 05:04 |
| *** Madan has joined #openstack-qa | 05:04 | |
| *** shashankhegde has quit IRC | 05:05 | |
| *** jyuso1 has joined #openstack-qa | 05:13 | |
| *** mwagner_lap has quit IRC | 05:15 | |
| *** annashen has quit IRC | 05:18 | |
| *** BharatK has joined #openstack-qa | 05:23 | |
| *** madhu_ak has quit IRC | 05:28 | |
| *** rushiagr_away is now known as rushiagr | 05:29 | |
| *** mwagner_lap has joined #openstack-qa | 05:32 | |
| *** fedexo has quit IRC | 05:33 | |
| *** annashen has joined #openstack-qa | 05:37 | |
| *** annashen has quit IRC | 05:38 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/167658 failed because of: gate-grenade-dsvm-ironic-sideways: unrecognized error, gate-tempest-dsvm-large-ops: https://bugs.launchpad.net/bugs/1298006 | 05:40 |
| openstack | Launchpad bug 1298006 in OpenStack-Gate "network sometimes unreachable accessing our git server" [Medium,Triaged] | 05:40 |
| *** ibiris_away is now known as ibiris | 05:41 | |
| *** itzikb has joined #openstack-qa | 05:42 | |
| *** shashankhegde has joined #openstack-qa | 05:49 | |
| *** harlowja_ is now known as harlowja_away | 05:53 | |
| *** hemnafk has quit IRC | 06:03 | |
| *** hemnafk has joined #openstack-qa | 06:03 | |
| *** SlickNik has quit IRC | 06:05 | |
| *** krotscheck has quit IRC | 06:05 | |
| *** krotscheck has joined #openstack-qa | 06:05 | |
| *** SlickNik has joined #openstack-qa | 06:06 | |
| *** achanda has quit IRC | 06:13 | |
| *** yfried_ has joined #openstack-qa | 06:15 | |
| *** shardy_z is now known as shardy | 06:15 | |
| *** shashankhegde has quit IRC | 06:19 | |
| *** shashankhegde has joined #openstack-qa | 06:20 | |
| *** SridharG has joined #openstack-qa | 06:23 | |
| *** armax has quit IRC | 06:23 | |
| *** yfried_ is now known as yfried|afk | 06:24 | |
| *** k4n0 has joined #openstack-qa | 06:24 | |
| *** anshul has joined #openstack-qa | 06:26 | |
| *** yfried|afk is now known as yfried_ | 06:28 | |
| *** anshul has quit IRC | 06:36 | |
| *** aopincar has joined #openstack-qa | 06:36 | |
| *** anshul has joined #openstack-qa | 06:36 | |
| *** Sukhdev has quit IRC | 06:37 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/177306 failed gate-devstack-dsvm-cells in the gate queue with an unrecognized error | 06:38 |
| *** e0ne has joined #openstack-qa | 06:39 | |
| *** yfried_ is now known as yfried|afk | 06:40 | |
| *** yfried|afk is now known as yfried_ | 06:44 | |
| *** ygoto has quit IRC | 06:47 | |
| *** yamamoto has quit IRC | 06:48 | |
| *** zz_dimtruck is now known as dimtruck | 06:59 | |
| *** safchain has joined #openstack-qa | 07:03 | |
| *** dimtruck is now known as zz_dimtruck | 07:09 | |
| *** zz_dimtruck is now known as dimtruck | 07:14 | |
| *** pkoniszewski has joined #openstack-qa | 07:14 | |
| *** sergsh has joined #openstack-qa | 07:19 | |
| *** dimtruck is now known as zz_dimtruck | 07:23 | |
| *** tkammer has joined #openstack-qa | 07:26 | |
| *** shashankhegde has quit IRC | 07:28 | |
| *** neeti has joined #openstack-qa | 07:30 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/176984 failed gate-tempest-dsvm-neutron-large-ops, gate-tempest-dsvm-full-ceph in the gate queue with an unrecognized error | 07:31 |
| *** pcaruana has joined #openstack-qa | 07:33 | |
| *** luqas has joined #openstack-qa | 07:33 | |
| *** yog__ has joined #openstack-qa | 07:34 | |
| *** jlanoux has joined #openstack-qa | 07:39 | |
| *** afazekas_ has joined #openstack-qa | 07:39 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/177759 failed gate-tempest-dsvm-neutron-large-ops in the gate queue with an unrecognized error | 07:40 |
| *** e0ne has quit IRC | 07:41 | |
| *** apevec has joined #openstack-qa | 07:41 | |
| *** apevec has quit IRC | 07:41 | |
| *** apevec has joined #openstack-qa | 07:41 | |
| *** gfidente has joined #openstack-qa | 07:42 | |
| *** oomichi has joined #openstack-qa | 07:43 | |
| *** yfried_ is now known as yfried|afk | 07:44 | |
| *** yfried|afk is now known as yfried_ | 07:46 | |
| *** rushiagr is now known as rushiagr_away | 07:46 | |
| *** rushiagr_away is now known as rushiagr | 07:49 | |
| *** sputnik13 has joined #openstack-qa | 07:53 | |
| *** yamahata has quit IRC | 07:56 | |
| *** mwagner_lap has quit IRC | 07:58 | |
| *** yfried_ is now known as yfried|afk | 08:00 | |
| *** Longgeek has joined #openstack-qa | 08:00 | |
| *** yfried|afk is now known as yfried_ | 08:03 | |
| openstackgerrit | Abhishek Shrivastava proposed openstack-dev/devstack: Additional Cinder backend: cloudbyte https://review.openstack.org/177677 | 08:07 |
| *** jogo has quit IRC | 08:11 | |
| *** oomichi has quit IRC | 08:13 | |
| *** Longgeek has quit IRC | 08:16 | |
| *** Longgeek has joined #openstack-qa | 08:16 | |
| *** yfried__ has joined #openstack-qa | 08:20 | |
| *** openstackgerrit has quit IRC | 08:20 | |
| *** openstackgerrit has joined #openstack-qa | 08:21 | |
| *** yfried__ has quit IRC | 08:21 | |
| *** yfried__ has joined #openstack-qa | 08:21 | |
| *** yfried_ has quit IRC | 08:22 | |
| *** aopincar has quit IRC | 08:22 | |
| *** apevec has quit IRC | 08:24 | |
| *** mrodden has quit IRC | 08:24 | |
| *** apevec has joined #openstack-qa | 08:24 | |
| *** apevec has joined #openstack-qa | 08:24 | |
| *** mrodden has joined #openstack-qa | 08:28 | |
| *** mpaolino has joined #openstack-qa | 08:33 | |
| *** mpaolino has quit IRC | 08:33 | |
| *** aopincar has joined #openstack-qa | 08:38 | |
| *** jordanP has joined #openstack-qa | 08:39 | |
| *** amotoki has quit IRC | 08:45 | |
| *** gszasz has joined #openstack-qa | 08:47 | |
| *** raies has quit IRC | 08:47 | |
| *** aopincar has quit IRC | 08:54 | |
| *** luqas has quit IRC | 08:54 | |
| *** mahito has quit IRC | 08:56 | |
| *** yfried__ has quit IRC | 09:03 | |
| *** yfried__ has joined #openstack-qa | 09:04 | |
| *** e0ne has joined #openstack-qa | 09:05 | |
| *** aix has joined #openstack-qa | 09:07 | |
| *** aopincar has joined #openstack-qa | 09:09 | |
| *** pkoniszewski has quit IRC | 09:11 | |
| *** yamamoto has joined #openstack-qa | 09:12 | |
| *** yamamoto has quit IRC | 09:16 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 09:17 | |
| *** yfried__ is now known as yfried|afk | 09:25 | |
| *** Longgeek has quit IRC | 09:28 | |
| *** yog__ has quit IRC | 09:29 | |
| *** yamamoto has joined #openstack-qa | 09:29 | |
| *** itzikb has quit IRC | 09:29 | |
| *** yog__ has joined #openstack-qa | 09:29 | |
| *** itzikb has joined #openstack-qa | 09:30 | |
| *** luqas has joined #openstack-qa | 09:30 | |
| *** itzikb has quit IRC | 09:32 | |
| *** itzikb has joined #openstack-qa | 09:33 | |
| *** Longgeek has joined #openstack-qa | 09:40 | |
| *** jogo has joined #openstack-qa | 09:43 | |
| *** yfried|afk is now known as yfried__ | 09:45 | |
| *** woodster_ has quit IRC | 09:50 | |
| *** rushiagr is now known as rushiagr_away | 09:52 | |
| *** Longgeek has quit IRC | 09:54 | |
| openstackgerrit | Mathieu Rohon proposed openstack-dev/devstack: Wait after restarting rabbitmq https://review.openstack.org/177785 | 09:54 |
| *** yfried__ is now known as yfried|afk | 09:55 | |
| *** Longgeek has joined #openstack-qa | 10:00 | |
| *** Poornima has joined #openstack-qa | 10:02 | |
| openstackgerrit | Mathieu Rohon proposed openstack-dev/devstack: Wait after restarting rabbitmq https://review.openstack.org/177785 | 10:05 |
| *** Poornima has quit IRC | 10:06 | |
| *** yfried|afk is now known as yfried__ | 10:08 | |
| *** marrusl has joined #openstack-qa | 10:09 | |
| *** dims has joined #openstack-qa | 10:12 | |
| *** Longgeek has quit IRC | 10:16 | |
| *** zigo has quit IRC | 10:29 | |
| *** cdent has joined #openstack-qa | 10:33 | |
| *** jlanoux_ has joined #openstack-qa | 10:33 | |
| *** Longgeek has joined #openstack-qa | 10:33 | |
| *** jlanoux has quit IRC | 10:33 | |
| *** sridhar has joined #openstack-qa | 10:33 | |
| *** e0ne is now known as e0ne_ | 10:33 | |
| *** zigo_ has joined #openstack-qa | 10:33 | |
| *** yamamoto has quit IRC | 10:34 | |
| *** yog__ has quit IRC | 10:35 | |
| *** e0ne_ has quit IRC | 10:35 | |
| *** yog__ has joined #openstack-qa | 10:36 | |
| *** e0ne has joined #openstack-qa | 10:39 | |
| *** jlanoux has joined #openstack-qa | 10:47 | |
| *** aix has quit IRC | 10:47 | |
| *** marcusvrn1 has joined #openstack-qa | 10:49 | |
| *** jlanoux_ has quit IRC | 10:50 | |
| *** marcusvrn has quit IRC | 10:52 | |
| openstackgerrit | Jaume Devesa proposed openstack/tempest: Skip scenario test when cinder is not available https://review.openstack.org/178132 | 10:55 |
| *** Longgeek has quit IRC | 10:56 | |
| *** nfedotov has joined #openstack-qa | 10:56 | |
| *** jamielennox is now known as jamielennox|away | 10:57 | |
| *** e0ne is now known as e0ne_ | 10:58 | |
| *** aix has joined #openstack-qa | 10:58 | |
| *** jhenner has quit IRC | 10:59 | |
| *** Longgeek has joined #openstack-qa | 11:00 | |
| openstackrecheck | Console logs not available after 13:24s for check-grenade-dsvm-partial-ncpu 177920,2,1d5ceb5 | 11:02 |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/177427 failed because of: gate-keystone-pep8: https://bugs.launchpad.net/bugs/1282876 | 11:02 |
| openstack | Launchpad bug 1282876 in OpenStack-Gate "git clone fails with "fatal: Not a git repository", "git remote update failed."" [Critical,Fix released] - Assigned to Jeremy Stanley (fungi) | 11:02 |
| *** weshay has joined #openstack-qa | 11:07 | |
| *** e0ne_ has quit IRC | 11:09 | |
| *** weshay has quit IRC | 11:09 | |
| *** weshay has joined #openstack-qa | 11:10 | |
| *** Poornima has joined #openstack-qa | 11:13 | |
| *** jhenner has joined #openstack-qa | 11:14 | |
| *** early has quit IRC | 11:15 | |
| *** Poornima has quit IRC | 11:17 | |
| *** early has joined #openstack-qa | 11:18 | |
| openstackgerrit | Dmitry Ratushnyy proposed openstack/tempest: Implementation of Scenario tests for port-security ML2 extension https://review.openstack.org/178137 | 11:24 |
| *** e0ne has joined #openstack-qa | 11:26 | |
| *** aix has quit IRC | 11:34 | |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/160856 failed gate-tempest-dsvm-large-ops, gate-devstack-dsvm-cells in the gate queue with an unrecognized error | 11:36 |
| *** yfried__ is now known as yfried|afk | 11:36 | |
| *** Raghava has joined #openstack-qa | 11:39 | |
| Raghava | please let me know the purpose of "build_timeout" and "build_interval" parameters in network section of tempest.conf. I don't see any difference when I increase/decrease the timeout | 11:40 |
| *** dims has quit IRC | 11:44 | |
| *** Madan has quit IRC | 11:46 | |
| *** yfried|afk is now known as yfried__ | 11:46 | |
| *** rushiagr_away is now known as rushiagr | 11:46 | |
| *** yamamoto has joined #openstack-qa | 11:48 | |
| openstackgerrit | Dmitry Ratushnyy proposed openstack/tempest: Implementation of Scenario tests for port-security ML2 extension https://review.openstack.org/178137 | 11:48 |
| *** rushiagr is now known as rushiagr_away | 11:48 | |
| *** dmorita has quit IRC | 11:49 | |
| *** aix has joined #openstack-qa | 11:50 | |
| *** rfolco has joined #openstack-qa | 11:50 | |
| *** dims has joined #openstack-qa | 11:51 | |
| *** rushiagr_away is now known as rushiagr | 12:06 | |
| *** pkoniszewski has joined #openstack-qa | 12:06 | |
| *** yfried__ has quit IRC | 12:06 | |
| *** mpaolino has joined #openstack-qa | 12:06 | |
| *** yfried__ has joined #openstack-qa | 12:07 | |
| *** Yogi has joined #openstack-qa | 12:11 | |
| afazekas_ | mtreinish, sdague, andreaf: FYI: I will not be around until May. 12. and I will not attend the next summit :(. | 12:11 |
| sdague | afazekas_: bummer | 12:11 |
| *** tochi has quit IRC | 12:11 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 12:11 | |
| *** dpaterson has quit IRC | 12:12 | |
| *** mwagner_lap has joined #openstack-qa | 12:13 | |
| *** shardy_ has joined #openstack-qa | 12:13 | |
| *** woodster_ has joined #openstack-qa | 12:14 | |
| *** shardy has quit IRC | 12:15 | |
| *** Yogi has quit IRC | 12:16 | |
| *** shardy_ has quit IRC | 12:19 | |
| *** shardy has joined #openstack-qa | 12:19 | |
| *** kiran-r has quit IRC | 12:29 | |
| nithyag_ | afazekas, mtrenish, andreaf: Shall we have a quick discussion on https://review.openstack.org/#/c/149229 (ssh-auth blueprint) to conclude on the logic for security group deletion? | 12:30 |
| *** jecarey has quit IRC | 12:32 | |
| *** dboik_ has quit IRC | 12:33 | |
| *** davideagnello has joined #openstack-qa | 12:33 | |
| *** mordred has quit IRC | 12:35 | |
| *** mordred has joined #openstack-qa | 12:35 | |
| *** rushiagr is now known as rushiagr_away | 12:36 | |
| andreaf | afazekas_: oh too bad | 12:37 |
| andreaf | nithyag_: I think mtreinish is still offline | 12:38 |
| *** davideagnello has quit IRC | 12:38 | |
| -openstackstatus- NOTICE: Gate is experiencing epic failures due to issues with mirrors, work is underway to mitigate and return to normal levels of sanity | 12:42 | |
| *** bkopilov has quit IRC | 12:42 | |
| *** ChanServ changes topic to "Gate is experiencing epic failures due to issues with mirrors, work is underway to mitigate and return to normal levels of sanity" | 12:42 | |
| *** samueldmq has joined #openstack-qa | 12:44 | |
| nithyag_ | andreaf: will ping again after a while, thanks | 12:49 |
| *** zz_dimtruck is now known as dimtruck | 12:50 | |
| *** ibiris is now known as ibiris_away | 12:50 | |
| *** dboik has joined #openstack-qa | 12:54 | |
| *** marcusvrn1 has quit IRC | 12:54 | |
| *** marcusvrn has joined #openstack-qa | 12:55 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/154593 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-large-ops: | 12:55 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 12:55 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-large-ops: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1286818 | 12:55 |
| *** ibiris_away is now known as ibiris | 12:57 | |
| *** aopincar has quit IRC | 12:59 | |
| *** bknudson has joined #openstack-qa | 12:59 | |
| openstackgerrit | Sean Dague proposed openstack-dev/devstack: fix warn function https://review.openstack.org/178178 | 12:59 |
| *** yamamoto has quit IRC | 13:01 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/172910 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full-ceph: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: | 13:02 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 13:02 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-large-ops: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-large-ops: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-nova-v21-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-postgres-full: | 13:02 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1286818 | 13:02 |
| *** aopincar has joined #openstack-qa | 13:02 | |
| *** pkoniszewski has quit IRC | 13:04 | |
| *** yamamoto has joined #openstack-qa | 13:04 | |
| *** mriedem has joined #openstack-qa | 13:08 | |
| *** Yogi has joined #openstack-qa | 13:09 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 13:12 | |
| openstackrecheck | openstack/swift change: https://review.openstack.org/177664 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818 | 13:15 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 13:15 |
| *** dimtruck is now known as zz_dimtruck | 13:16 | |
| openstackrecheck | openstack/glance change: https://review.openstack.org/164432 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full-ceph: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818 | 13:17 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 13:17 |
| *** dkranz has joined #openstack-qa | 13:24 | |
| *** jecarey_ has joined #openstack-qa | 13:25 | |
| *** yog__ has quit IRC | 13:26 | |
| *** sks has quit IRC | 13:26 | |
| *** yog__ has joined #openstack-qa | 13:27 | |
| *** e0ne is now known as e0ne_ | 13:28 | |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Use addCleanup in test_rebuild_server https://review.openstack.org/178187 | 13:28 |
| *** dustins has joined #openstack-qa | 13:29 | |
| *** dkranz has quit IRC | 13:29 | |
| *** aopincar has quit IRC | 13:30 | |
| openstackgerrit | Evgeny Antyshev proposed openstack/tempest: Image properties in scenario tests https://review.openstack.org/178192 | 13:30 |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from negative tests https://review.openstack.org/177745 | 13:35 |
| openstackgerrit | Sean Dague proposed openstack/tempest: adjust smoke definition to not include *all* scenario https://review.openstack.org/177852 | 13:35 |
| openstackgerrit | Sean Dague proposed openstack/tempest: trim network smoke tests https://review.openstack.org/177829 | 13:35 |
| openstackgerrit | Sean Dague proposed openstack/tempest: trim compute smoke tag https://review.openstack.org/177841 | 13:35 |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove qos tests from volume smoke tag https://review.openstack.org/177840 | 13:35 |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from admin tests https://review.openstack.org/177810 | 13:35 |
| *** itzikb has quit IRC | 13:35 | |
| *** tkammer has quit IRC | 13:36 | |
| *** e0ne_ is now known as e0ne | 13:37 | |
| *** artom has joined #openstack-qa | 13:40 | |
| *** artom has quit IRC | 13:42 | |
| *** artom has joined #openstack-qa | 13:42 | |
| *** miqui has joined #openstack-qa | 13:44 | |
| *** zz_jgrimm is now known as jgrimm | 13:47 | |
| mtreinish | afazekas_: oh man, that's too bad | 13:50 |
| mtreinish | raies: if you're still around ask away | 13:50 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/177675 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818 | 13:50 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 13:50 |
| mtreinish | nithyag_: I am around now | 13:50 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/177186 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1286818, gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-large-ops: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-full: | 13:50 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-large-ops: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-nova-v21-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1286818 | 13:50 |
| mtreinish | man apt must be going crazy in all the runs | 13:51 |
| nithyag_ | mtrenish, afazekas_, andreaf: a quick summary on using while loop instead of wait_for_resource_deletion() was to handle 409 errors as suggested by Attila | 13:52 |
| fungi | mtreinish: yeah, all the runs in rackspace anyway. their ubuntu mirror is currently in a broken state | 13:52 |
| mtreinish | fungi: :( I thought we stopped using the cloud hosted a while ago | 13:53 |
| nithyag_ | mtrenish, does it sound ok to retain the while loop with all other comments of your fixed? | 13:53 |
| mtreinish | nithyag_: what was raising a 409, it's not clear to me why doing a wait loop there would ever raise a conflict | 13:53 |
| *** BharatK has quit IRC | 13:54 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/177193 failed because of: gate-devstack-dsvm-cells: https://bugs.launchpad.net/bugs/1286818, gate-swift-dsvm-functional: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818 | 13:54 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 13:54 |
| fungi | mtreinish: the distro mirror network is inherently unstable. we've just merged a change to switch to official mirrors, but i wouldn't be surprised to see just as many failures with them over time | 13:55 |
| nithyag_ | mtrenish, if we dont have the VM associated with the security group deleted, we would get a 409 and that would need another request of security group deletion. | 13:55 |
| *** rbak has joined #openstack-qa | 13:56 | |
| nithyag_ | wait for resource deletion only waits for resource deletion and does not send another delete request | 13:56 |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from negative tests https://review.openstack.org/177745 | 13:56 |
| openstackgerrit | Sean Dague proposed openstack/tempest: not all scenario tests should be in smoke https://review.openstack.org/177852 | 13:56 |
| openstackgerrit | Sean Dague proposed openstack/tempest: trim network smoke tests https://review.openstack.org/177829 | 13:56 |
| openstackgerrit | Sean Dague proposed openstack/tempest: trim compute smoke tag https://review.openstack.org/177841 | 13:56 |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove qos tests from volume smoke tag https://review.openstack.org/177840 | 13:56 |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from admin tests https://review.openstack.org/177810 | 13:56 |
| andreaf | nithyag_, mtreinish: personally I think that if the VM is still around something is wrong in the code before the SG cleanup - and I would rather not wait for Conflict to go away | 13:56 |
| mtreinish | nithyag_: wait, what? That seems broken, you shouldn't need to make 2 secgroup delete calls if no server is using it. The resource should be independent of servers | 13:56 |
| *** armax has joined #openstack-qa | 13:56 | |
| mtreinish | if the server is still around I could see there being an issue with delete (like trying to delete a volume while it's still attached) | 13:57 |
| mtreinish | but having no server around doesn't make any sense | 13:57 |
| mtreinish | (erroring on deleting with no server around I mean) | 13:57 |
| *** Yogi2 has joined #openstack-qa | 13:58 | |
| mtreinish | andreaf: yeah, we should ensure the server is cleanedup before we ever get to the secgroup delete | 13:59 |
| nithyag_ | mtrenish, andreaf, can I assume that when we reach cleanup_validation resources that the security group will not be associated with any of the VMs? | 13:59 |
| andreaf | mtreinish, nithyag_: so I think afazekas_ comment about handling 409 was related to the case where the server cleanup does not wait for deletion to complete - trying to delete the SG would hit a conflict but once the server is gone it would work | 14:00 |
| mtreinish | nithyag_: sorry I misread what you wrote originally, I see the issue is now the vm is still around | 14:00 |
| mtreinish | andreaf: so is it just a cleanup race, because we do the delete async? | 14:00 |
| *** dustins_ has joined #openstack-qa | 14:00 | |
| *** Raghava has quit IRC | 14:00 | |
| openstackrecheck | openstack/sahara change: https://review.openstack.org/169466 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1286818 | 14:00 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 14:00 |
| *** Yogi has quit IRC | 14:01 | |
| andreaf | mtreinish: basically yes, but I think if that happens it's a bad behaviour in the test code - the delete of servers should wait for deletion | 14:01 |
| *** aopincar has joined #openstack-qa | 14:02 | |
| andreaf | mtreinish: I can see the benefit of deleting 10 servers at once and wait for all of them, but I see no benefit in deleting the servers and then deleting SGs and having the delete of SG have to wait on conflict to go away - maybe I'm missing something | 14:02 |
| afazekas_ | andreaf: Usually we does waiting, but If we have some exceptional case , the double failure just more confusing | 14:02 |
| *** dustins has quit IRC | 14:03 | |
| *** Yogi has joined #openstack-qa | 14:03 | |
| mtreinish | andreaf: I agree, we should ensure the deletes are done beforehand, whether that's an bulk async issue and wait, or just 1 delete wait | 14:04 |
| andreaf | afazekas_, we could extend the waiter to handle conflict, and log a less confusing message, and re-raise | 14:04 |
| mtreinish | afazekas_: yeah I think in an exceptional case we might want to just say things did not go as expected and fail, but that might be naive of me | 14:05 |
| afazekas_ | andreaf: Do you mean changing the regular resource delete waiter ? | 14:05 |
| *** gamado has quit IRC | 14:05 | |
| *** Yogi2 has quit IRC | 14:05 | |
| mtreinish | because presumably we'd only ever get the conflict if somethign went wrong in the delete case beforehand | 14:05 |
| *** ibiris is now known as ibiris_away | 14:06 | |
| *** shardy_ has joined #openstack-qa | 14:06 | |
| *** 77CAAQ8P9 has joined #openstack-qa | 14:06 | |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/176380 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1286818 | 14:07 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 14:07 |
| afazekas_ | mtreinish: We had many funny issues on the gate because of not waiting for delete, or attempting to delete at an unexpected short living state .. | 14:07 |
| mtreinish | fungi: heh, you must be getting pings on each test failure :) | 14:07 |
| *** marrusl has quit IRC | 14:08 | |
| *** shardy has quit IRC | 14:08 | |
| fungi | mtreinish: yep | 14:09 |
| *** pglass has joined #openstack-qa | 14:09 | |
| afazekas_ | mtreinish, andreaf: I would like to have the ssh code in ASAP, if you thing it is safe to do it without wait we can risk it :) | 14:09 |
| nithyag_ | mtrenish, afazekas_, andreaf: we can modify wait_for_resource_deletion() to pick an optional argument "retry_delete set to false by default" and handle conflicting cases. | 14:09 |
| mtreinish | afazekas_: sure, but looping the delete in those cases to try and always kill the sg in those case seems like it would just cause us to wait a full timeout | 14:09 |
| nithyag_ | not sure if that sounds good | 14:09 |
| *** yfried has joined #openstack-qa | 14:09 | |
| nithyag_ | that would need a change to tempest_lib rest_client though | 14:10 |
| mtreinish | afazekas_: as opposed to catching a 409 on the delete call, and raising a detailed error message | 14:10 |
| *** Yogi has quit IRC | 14:10 | |
| *** 77CAAQ8P9 has quit IRC | 14:11 | |
| *** yfried__ has quit IRC | 14:12 | |
| mtreinish | nithyag_: well that's what I'm not sure is behavior we want to encode anywhere. Looping deletes instead of gets waiting for a 404 seems really weird to me | 14:12 |
| afazekas_ | mtreinish: and your reason behind this is you think the server MUST be deleted before the network resource cleanup, and you would like to see the cases when it is not. Correct ? | 14:12 |
| *** shardy_ has quit IRC | 14:12 | |
| *** dboik_ has joined #openstack-qa | 14:12 | |
| *** shardy has joined #openstack-qa | 14:12 | |
| mtreinish | afazekas_: yeah, more or less. Because in the case it's not deleted it's either a tempest bug or a bug in nova, cinder, neutron, etc. | 14:13 |
| *** tkammer has joined #openstack-qa | 14:15 | |
| *** dims has quit IRC | 14:15 | |
| *** ibiris_away is now known as ibiris | 14:15 | |
| *** dims has joined #openstack-qa | 14:15 | |
| *** dboik has quit IRC | 14:16 | |
| mtreinish | nithyag_: it feels to me like pressing the elevator button non stop until it comes, as opposed to just pushing it once and periodically looking at what floor it's on | 14:16 |
| andreaf | mtreinish, afazekas_ , nithyag_ : so bottom line, can we agree on delete once, and wait on 404? | 14:17 |
| nithyag_ | andreaf: fine by me | 14:17 |
| andreaf | mtreinish, afazekas_, nithyag_ : as a separate change we can still go and improve the wait cycle to log a reasonable message on 409 and stop waiting | 14:18 |
| *** Yogi has joined #openstack-qa | 14:18 | |
| mtreinish | andreaf: wait does the GET raise the 409? I just assumed the initial delete call raised the 409 | 14:18 |
| afazekas_ | mtreinish: The initial does | 14:19 |
| *** bkopilov has joined #openstack-qa | 14:19 | |
| andreaf | mtreinish: oh, you are right - sorry I mixed up things | 14:19 |
| mtreinish | heh, ok for a second I thought I misread something | 14:20 |
| andreaf | andreaf: so we don't need to touch the waiter - even better - just log a reasonable error message if 409 is raise d:) | 14:20 |
| andreaf | mtreinish: no you are right the delete fails synchronously with 409 in case of conflict | 14:21 |
| *** rossella_s has quit IRC | 14:21 | |
| *** rossella_s has joined #openstack-qa | 14:22 | |
| *** neeti has quit IRC | 14:22 | |
| afazekas_ | andreaf: we will see what will happen, it can be OK if we already ensured the server deletion everywhere. | 14:22 |
| *** sputnik13 has quit IRC | 14:22 | |
| afazekas_ | ensured server deletion: The server(s) does not visible on the user API when we reach net sec group clean up part | 14:23 |
| andreaf | afazekas_, eh if not we will have a nice error message to find where we don't and go and fix it | 14:23 |
| *** neeti has joined #openstack-qa | 14:24 | |
| nithyag_ | mtrenish, andreaf, afazekas_: I will go ahead with delete once and wait on 404 for security group abd submit a patch. I will fix the other comments that have been provided as well. is that ok? | 14:25 |
| afazekas_ | andreaf: so be it | 14:25 |
| *** gszasz has quit IRC | 14:26 | |
| afazekas_ | nithyag_: so be it | 14:26 |
| nithyag_ | andreaf, afazekas_, mtrenish: thanks for your time | 14:26 |
| mtreinish | jogo: so it turns out bar graphs might not be too useful: http://imgur.com/gocwcIq | 14:26 |
| mtreinish | I think I'm going to have to play games with axis scales | 14:27 |
| afazekas_ | nithyag_: thanks for taking care of difficult task which is hanging around since long | 14:27 |
| *** BharatK has joined #openstack-qa | 14:28 | |
| mtreinish | jogo: that's the counts for the 2 volume boot pattern tests and the ec2 volume tests from Apr 1st | 14:28 |
| mtreinish | for the full set in the db is a bit better: http://imgur.com/mkrHEfJ (just because volume boot was skipped for long enough) | 14:29 |
| *** aopincar has quit IRC | 14:31 | |
| *** mriedem has quit IRC | 14:31 | |
| nithyag_ | afazekas_: :) yes, its high time we get this in. I will submit a patch soon | 14:31 |
| *** notmyname has quit IRC | 14:31 | |
| *** afazekas_ has quit IRC | 14:33 | |
| *** notmyname has joined #openstack-qa | 14:34 | |
| openstackrecheck | openstack/sahara change: https://review.openstack.org/173205 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-neutron-full: https://bugs.launchpad.net/bugs/1286818, gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1286818 | 14:36 |
| openstack | Launchpad bug 1286818 in OpenStack-Gate "Ubuntu package archive periodically inconsistent causing gate build failures" [Low,In progress] - Assigned to Jeremy Stanley (fungi) | 14:36 |
| *** mriedem has joined #openstack-qa | 14:36 | |
| *** gszasz has joined #openstack-qa | 14:40 | |
| *** aopincar has joined #openstack-qa | 14:45 | |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: Remove tripleo from signature https://review.openstack.org/178220 | 14:46 |
| *** marrusl has joined #openstack-qa | 14:46 | |
| *** masayukig_ has quit IRC | 14:48 | |
| openstackgerrit | Evgeny Antyshev proposed openstack/tempest: Image properties in scenario tests https://review.openstack.org/178192 | 14:48 |
| mriedem | there has to be a tempest test which does a volume attach and detach right? | 14:49 |
| *** sputnik13 has joined #openstack-qa | 14:49 | |
| *** masayukig_ has joined #openstack-qa | 14:50 | |
| mtreinish | mriedem: several | 14:50 |
| mriedem | ok :) https://bugs.launchpad.net/nova/+bug/1447989 | 14:50 |
| openstack | Launchpad bug 1447989 in OpenStack Compute (nova) "nova volume-detach volume failed due to arguments counts mismatch?" [Undecided,New] | 14:50 |
| mriedem | i'm about to call bullshit all over that | 14:50 |
| mtreinish | mriedem: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/volume/test_volumes_actions.py#n62 | 14:50 |
| *** BharatK has quit IRC | 14:51 | |
| mtreinish | mriedem: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/volume/test_volumes_actions.py#n62 | 14:51 |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Remove tripleo from signature https://review.openstack.org/178220 | 14:51 |
| mtreinish | mriedem: the best test for it though: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_stamp_pattern.py | 14:52 |
| mtreinish | has been skipped for basically forever | 14:52 |
| *** yamahata has joined #openstack-qa | 14:52 | |
| *** openstackrecheck has quit IRC | 14:55 | |
| *** openstackrecheck has joined #openstack-qa | 14:55 | |
| *** ajmiller has joined #openstack-qa | 14:55 | |
| *** dkranz has joined #openstack-qa | 14:56 | |
| *** jaypipes has quit IRC | 14:58 | |
| *** shardy_ has joined #openstack-qa | 14:58 | |
| *** nfedotov has quit IRC | 14:59 | |
| *** shardy has quit IRC | 15:00 | |
| *** sabeen has joined #openstack-qa | 15:02 | |
| *** BharatK has joined #openstack-qa | 15:03 | |
| *** sabeen2 has joined #openstack-qa | 15:03 | |
| *** mtanino has joined #openstack-qa | 15:03 | |
| *** shardy_ has quit IRC | 15:04 | |
| *** shardy has joined #openstack-qa | 15:04 | |
| *** e0ne is now known as e0ne_ | 15:04 | |
| *** e0ne_ is now known as e0ne | 15:05 | |
| sdague | mriedem: so... we're actually testing some version of 1447989 in novaclient functional tests | 15:05 |
| mriedem | sdague: mtreinish: i figured it out | 15:05 |
| mriedem | ffs | 15:05 |
| mriedem | http://git.openstack.org/cgit/openstack/cinder/tree/cinder/volume/manager.py?id=2015.1.0rc2#n892 | 15:05 |
| mriedem | cinder volume manager passes those args | 15:05 |
| mriedem | guess who doesn't have the right signature? | 15:06 |
| mriedem | http://git.openstack.org/cgit/stackforge/powervc-driver/tree/cinder-powervc/powervc/volume/driver/powervc.py#n312 | 15:06 |
| sdague | mriedem: kill them with fire? | 15:06 |
| mriedem | gdi i'd like to | 15:07 |
| *** sabeen has quit IRC | 15:07 | |
| mriedem | send me a sev 1 for a busted shit in your out of tree driver will oyu | 15:07 |
| mriedem | *you | 15:07 |
| mriedem | "Yay for forks with no CI!" | 15:07 |
| mriedem | that was in my response when i rejected the internal bug | 15:07 |
| *** rushiagr_away is now known as rushiagr | 15:09 | |
| *** sputnik13 has quit IRC | 15:09 | |
| mtreinish | mriedem: heh, I'm sure that'll go over well. Watch out you might get escalated :) | 15:09 |
| mriedem | dude, re-org, they can't touch us now | 15:09 |
| *** anshul has quit IRC | 15:10 | |
| *** aopincar has quit IRC | 15:13 | |
| *** plaurin has left #openstack-qa | 15:21 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 15:22 | |
| *** yog__ has quit IRC | 15:22 | |
| *** SridharG has left #openstack-qa | 15:24 | |
| *** tsekiyama has joined #openstack-qa | 15:29 | |
| *** ajmiller has quit IRC | 15:31 | |
| *** krtaylor has quit IRC | 15:31 | |
| *** marrusl has quit IRC | 15:31 | |
| *** zz_dimtruck is now known as dimtruck | 15:36 | |
| -openstackstatus- NOTICE: gerrit has been restarted to clear an issue with its event stream. any change events between 14:43-15:30 utc should be rechecked or have their approval votes reapplied to trigger jobs | 15:37 | |
| *** Swami has joined #openstack-qa | 15:37 | |
| *** dustins_ has quit IRC | 15:39 | |
| *** bkopilov has quit IRC | 15:44 | |
| *** anshul has joined #openstack-qa | 15:45 | |
| *** bkopilov has joined #openstack-qa | 15:45 | |
| *** k4n0 has quit IRC | 15:45 | |
| *** neeti has quit IRC | 15:45 | |
| *** mpaolino has quit IRC | 15:47 | |
| *** hemnafk is now known as hemna | 15:47 | |
| *** rushiagr is now known as rushiagr_away | 15:50 | |
| *** tkammer has quit IRC | 15:50 | |
| *** ajmiller has joined #openstack-qa | 15:50 | |
| *** yfried has quit IRC | 15:52 | |
| *** ChanServ changes topic to "All Things QA..." | 15:53 | |
| sdague | mtreinish: so... I reordered the smoke patches to put the scenario trim after the dropping of gate tags | 15:53 |
| sdague | which means the neutron icehouse jobs might actually pass | 15:53 |
| sdague | but you know, it's 30% failure rate | 15:53 |
| sdague | so they might not | 15:53 |
| *** sergsh has quit IRC | 15:53 | |
| *** krtaylor has joined #openstack-qa | 15:56 | |
| *** marrusl has joined #openstack-qa | 15:56 | |
| *** cody-somerville has quit IRC | 15:58 | |
| mtreinish | jogo: what do you think: http://blog.kortar.org/wp-content/uploads/2015/04/test-volume-boot-pattern.png (definitely improving the pretty factor) | 15:59 |
| mtreinish | sdague: ok sure, although that failure rate seems too high to just be caused by running too many tests | 15:59 |
| sdague | mtreinish: it's some of the adv ops tests failing over and over again | 16:00 |
| openstackgerrit | Monty Taylor proposed openstack-infra/devstack-gate: Put input variables into ansible inventory https://review.openstack.org/177943 | 16:01 |
| openstackgerrit | Monty Taylor proposed openstack-infra/devstack-gate: Move all the ansible calls into playbooks https://review.openstack.org/177944 | 16:01 |
| *** Yogi has quit IRC | 16:01 | |
| sdague | https://bugs.launchpad.net/nova/+bug/1323658 | 16:01 |
| openstack | Launchpad bug 1323658 in OpenStack Compute (nova) "Nova resize/restart results in guest ending up in inconsistent state with Neutron" [Medium,Confirmed] | 16:01 |
| sdague | the resize tests | 16:01 |
| sdague | it looks like people are just regularly rechecking icehouse to get past it | 16:02 |
| mtreinish | sdague: sure, but a 30% failure rate would mean we basically never land anything in tempest. I've landed a lot of patches the past couple weeks and that wasn't something I had to recheck grind on | 16:03 |
| mtreinish | it's really only been the network and infra stuff | 16:03 |
| sdague | well, that's the rolling average I got over the last 300 hours | 16:03 |
| sdague | You can see my linked graphite graph | 16:03 |
| *** gszasz has quit IRC | 16:04 | |
| sdague | my first gate tag removal patch failed the neutron icehouse job twice already | 16:05 |
| *** jlanoux has quit IRC | 16:05 | |
| sdague | 2 out of 5 runs | 16:06 |
| *** gszasz has joined #openstack-qa | 16:06 | |
| sdague | not counting the last 2 runs, which were the apt mirror pocalypse | 16:06 |
| sdague | oh, sorry, 2 out of 4 | 16:06 |
| sdague | one of those was a gate run where it wasn't run | 16:06 |
| jogo | mtreinish: neat graph, does that include success only or success and failure? | 16:07 |
| mtreinish | jogo: only successes | 16:07 |
| jogo | interesting | 16:07 |
| jogo | mtreinish: looks like we removed a lot of those outliers | 16:07 |
| jogo | I wonder what those were | 16:08 |
| *** apevec has quit IRC | 16:08 | |
| mtreinish | jogo: hp cloud nodes? | 16:09 |
| jogo | hmm possibly | 16:09 |
| *** mlavalle has joined #openstack-qa | 16:09 | |
| jogo | either way I think that is a decently useful datapoint | 16:09 |
| sdague | for stuff like this I wouldn't expect hp to be much slower | 16:09 |
| mtreinish | jogo: the node is in the metadata, if you want to pry it out of there. I had a script to do that at one point, I could dig it up | 16:11 |
| *** davideagnello has joined #openstack-qa | 16:11 | |
| jogo | mtreinish: meh, that may not be the reason | 16:11 |
| mtreinish | sdague: yeah if they were consistently >100sec slower on hp we would definitely see more spikes | 16:11 |
| jogo | mtreinish: looks like there are other things to hunt down at the moment | 16:12 |
| *** yamahata has quit IRC | 16:12 | |
| mtreinish | jogo: it could also be branches, this is everything in gate, I didn't filter on just master | 16:12 |
| sdague | honestly, the timing on this test is mostly going to be affected by what other tests are running | 16:13 |
| sdague | if another volume action is going on at the same time, it will bog down a lot | 16:13 |
| *** pcaruana has quit IRC | 16:14 | |
| jogo | sdague: good point | 16:15 |
| *** davideagnello has quit IRC | 16:15 | |
| *** kiran-r has joined #openstack-qa | 16:15 | |
| *** ajmiller has quit IRC | 16:17 | |
| openstackgerrit | Matthew Treinish proposed openstack-infra/subunit2sql: Improve run_time graph formatting https://review.openstack.org/178276 | 16:21 |
| *** safchain has quit IRC | 16:23 | |
| *** sputnik13 has joined #openstack-qa | 16:29 | |
| *** mtanino_ has joined #openstack-qa | 16:30 | |
| *** mtanino has quit IRC | 16:33 | |
| *** Nic has joined #openstack-qa | 16:33 | |
| *** Yogi has joined #openstack-qa | 16:34 | |
| *** yamahata has joined #openstack-qa | 16:37 | |
| *** marrusl has quit IRC | 16:41 | |
| *** dboik_ has quit IRC | 16:45 | |
| *** gszasz has quit IRC | 16:47 | |
| *** yfried has joined #openstack-qa | 16:51 | |
| *** patrickeast has joined #openstack-qa | 16:51 | |
| *** e0ne has quit IRC | 16:52 | |
| *** dboik has joined #openstack-qa | 16:58 | |
| *** dboik has quit IRC | 16:58 | |
| *** dustins has joined #openstack-qa | 17:00 | |
| *** davideagnello has joined #openstack-qa | 17:01 | |
| *** suneelb has joined #openstack-qa | 17:03 | |
| *** dboik has joined #openstack-qa | 17:04 | |
| *** dboik has quit IRC | 17:04 | |
| *** yfried is now known as yfried|prtially_ | 17:05 | |
| *** dboik has joined #openstack-qa | 17:05 | |
| *** marun has joined #openstack-qa | 17:07 | |
| *** ir2ivps6 has quit IRC | 17:10 | |
| openstackgerrit | Sean Dague proposed openstack-dev/devstack: don't source openrc https://review.openstack.org/178297 | 17:16 |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-neutron-full-juno 177760,1,843d467 | 17:18 |
| *** harlowja_away is now known as harlowja_ | 17:21 | |
| *** jordanP has quit IRC | 17:22 | |
| *** yfried|prtially_ is now known as yfried | 17:25 | |
| openstackgerrit | Merged openstack-dev/grenade: Make nova migrate flavor data, per Kilo->Lemming requirements https://review.openstack.org/174567 | 17:26 |
| *** e0ne has joined #openstack-qa | 17:28 | |
| *** luqas has quit IRC | 17:29 | |
| *** luqas has joined #openstack-qa | 17:29 | |
| *** Yogi2 has joined #openstack-qa | 17:31 | |
| *** jogo has quit IRC | 17:33 | |
| *** yfried is now known as yfried|away | 17:33 | |
| *** Yogi has quit IRC | 17:34 | |
| *** ir2ivps6 has joined #openstack-qa | 17:34 | |
| *** aix has quit IRC | 17:36 | |
| *** melwitt has joined #openstack-qa | 17:38 | |
| *** luqas has quit IRC | 17:42 | |
| *** yfried|away is now known as yfried | 17:43 | |
| *** jogo has joined #openstack-qa | 17:44 | |
| *** Yogi2 has quit IRC | 17:48 | |
| *** kiran-r has quit IRC | 17:51 | |
| *** dustins has quit IRC | 17:53 | |
| *** achanda has joined #openstack-qa | 17:58 | |
| *** e0ne has quit IRC | 17:58 | |
| *** ir2ivps6 has quit IRC | 17:59 | |
| *** dboik_ has joined #openstack-qa | 18:01 | |
| openstackrecheck | Console logs not available after 13:31s for check-grenade-dsvm-partial-ncpu 130526,12,3f42886 | 18:01 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/172910 failed gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error | 18:02 |
| *** weshay has quit IRC | 18:02 | |
| *** melwitt has quit IRC | 18:02 | |
| *** melwitt has joined #openstack-qa | 18:03 | |
| *** melwitt has quit IRC | 18:03 | |
| *** jamielennox|away is now known as jamielennox | 18:04 | |
| *** dboik has quit IRC | 18:04 | |
| *** pkoniszewski has joined #openstack-qa | 18:05 | |
| *** jogo has quit IRC | 18:06 | |
| *** melwitt has joined #openstack-qa | 18:06 | |
| *** weshay has joined #openstack-qa | 18:07 | |
| *** jogo has joined #openstack-qa | 18:07 | |
| *** Yogi has joined #openstack-qa | 18:13 | |
| *** gfidente is now known as gfidente|afk | 18:14 | |
| *** cdent has quit IRC | 18:17 | |
| *** madhu_ak has joined #openstack-qa | 18:20 | |
| madhu_ak | hello, where can I find testscenarios testcases in openstack project? | 18:21 |
| *** pkoniszewski has quit IRC | 18:23 | |
| *** ir2ivps6 has joined #openstack-qa | 18:24 | |
| *** hitalia has joined #openstack-qa | 18:25 | |
| *** jhenner has quit IRC | 18:26 | |
| *** dustins has joined #openstack-qa | 18:29 | |
| *** Nic has quit IRC | 18:30 | |
| *** Nic has joined #openstack-qa | 18:31 | |
| *** Yogi has quit IRC | 18:32 | |
| *** dustins has quit IRC | 18:34 | |
| *** achanda has quit IRC | 18:34 | |
| *** anshul has quit IRC | 18:34 | |
| *** achanda has joined #openstack-qa | 18:35 | |
| *** Longgeek has quit IRC | 18:37 | |
| *** yfried is now known as yfried|afk | 18:40 | |
| *** achanda has quit IRC | 18:41 | |
| *** suneelb has quit IRC | 18:43 | |
| *** suneelb has joined #openstack-qa | 18:43 | |
| openstackgerrit | Merged openstack/tempest: remove gate tag (part 1) https://review.openstack.org/177759 | 18:45 |
| mtreinish | andreaf: I'm going to push tempest-lib 0.5.0 right now | 18:46 |
| *** krtaylor has quit IRC | 18:46 | |
| *** weshay has quit IRC | 18:48 | |
| *** Rockyg has joined #openstack-qa | 18:49 | |
| *** Yogi has joined #openstack-qa | 18:51 | |
| *** weshay has joined #openstack-qa | 18:52 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Explicitly say when job times out https://review.openstack.org/178330 | 18:52 |
| *** suneelb has quit IRC | 18:53 | |
| *** suneelb has joined #openstack-qa | 18:53 | |
| mtreinish | mriedem, jogo: brace yourselves I just pushed the tempest-lib tag | 18:56 |
| * mriedem braces | 18:57 | |
| * jogo thinks about getting lunch and hiding | 18:57 | |
| mtreinish | jogo: heh, I doubt that'll be sufficient to escape the likely stable explosion | 18:58 |
| jogo | mtreinish: haha | 18:59 |
| *** e0ne has joined #openstack-qa | 18:59 | |
| *** pglass has quit IRC | 18:59 | |
| *** yfried|afk is now known as yfried | 19:00 | |
| *** pglass has joined #openstack-qa | 19:03 | |
| *** jlk has quit IRC | 19:04 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Expand fingerprint for git fetch error https://review.openstack.org/178338 | 19:04 |
| *** jaypipes has joined #openstack-qa | 19:06 | |
| *** ChanServ sets mode: +o jaypipes | 19:06 | |
| *** krtaylor has joined #openstack-qa | 19:06 | |
| *** jlk has joined #openstack-qa | 19:06 | |
| *** jlk has quit IRC | 19:07 | |
| *** jlk has joined #openstack-qa | 19:07 | |
| *** erlon has joined #openstack-qa | 19:07 | |
| openstackgerrit | Matthew Treinish proposed openstack/tempest-lib: Add release notes to docs for past few releases https://review.openstack.org/178341 | 19:09 |
| *** yfried is now known as yfried|afk | 19:10 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/elastic-recheck: Expand fingerprint for git fetch error https://review.openstack.org/178338 | 19:11 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest-lib: Update setup.cfg metadata to reflect reality https://review.openstack.org/178343 | 19:11 |
| mriedem | jogo: still a high success rate | 19:13 |
| *** hitalia has quit IRC | 19:13 | |
| mriedem | i guess the original query is too | 19:13 |
| jogo | mriedem: yeah that is expected | 19:14 |
| jogo | mriedem: the new line shows when git commands take a long time | 19:14 |
| jogo | and make things super slow | 19:14 |
| openstackgerrit | Sirushti Murugesan proposed openstack-dev/grenade: Add initial support for Heat https://review.openstack.org/86978 | 19:19 |
| *** gfidente|afk has quit IRC | 19:19 | |
| *** dboik_ has quit IRC | 19:20 | |
| *** dboik has joined #openstack-qa | 19:21 | |
| *** weshay has quit IRC | 19:31 | |
| *** rfolco has quit IRC | 19:32 | |
| *** mestery has quit IRC | 19:34 | |
| openstackgerrit | Merged openstack-dev/devstack: Set DevStack to dual stack by default in Kilo+ https://review.openstack.org/160856 | 19:36 |
| openstackrecheck | Console logs not available after 13:27s for check-grenade-dsvm 178061,1,92458a9 | 19:37 |
| sc68cal | Woo! ^^ | 19:38 |
| * sc68cal waits for everything to suddenly explode now that we're using ipv6 | 19:38 | |
| *** mestery has joined #openstack-qa | 19:39 | |
| andreaf | mtreinish: yay :) | 19:40 |
| *** rfolco has joined #openstack-qa | 19:43 | |
| mtreinish | andreaf: you might want to respin your auth delete patch, and add a depends-on to: https://review.openstack.org/#/c/178362/ | 19:43 |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Drop auth and corresponding unit tests https://review.openstack.org/166022 | 19:44 |
| *** dustins has joined #openstack-qa | 19:45 | |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Drop auth and corresponding unit tests https://review.openstack.org/166022 | 19:45 |
| andreaf | mtreinish: all done via gerrit UI | 19:45 |
| mtreinish | andreaf: heh, yeah having those is nice. Although, I started gpg signing my commits and using the ui drops that... | 19:47 |
| mtreinish | andreaf: oh, also I -1'd the patch the version in requirements was wrong | 19:47 |
| andreaf | mtreinish: oh, ok - how do you gpg sign your commits? | 19:48 |
| *** marun has quit IRC | 19:48 | |
| mtreinish | andreaf: git commit -S | 19:48 |
| andreaf | mtreinish: oh that easy :) | 19:48 |
| mtreinish | andreaf: seeing the signatures though is not default. You've got to do --show-signature on git log or git show | 19:49 |
| andreaf | mtreinish: ok | 19:50 |
| andreaf | mtreinish: still, sounds like a nice thing to do | 19:51 |
| *** madhu_ak has quit IRC | 19:51 | |
| openstackrecheck | Console logs not available after 13:33s for gate-tempest-dsvm-full 168907,5,9fa10bc | 19:51 |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Drop auth and corresponding unit tests https://review.openstack.org/166022 | 19:51 |
| *** openstackstatus has quit IRC | 19:52 | |
| andreaf | mtreinish: ^^^ | 19:53 |
| *** openstackstatus has joined #openstack-qa | 19:54 | |
| *** ChanServ sets mode: +v openstackstatus | 19:54 | |
| andreaf | mtreinish: so at the end you are not against https://review.openstack.org/#/c/176979/ as an interim solution? the TOP_LEVEL testr patch | 19:55 |
| mtreinish | andreaf: well I'm curious how it works for you, because doesn't the .testr.conf need to be in your cwd | 19:56 |
| mtreinish | but I don't really have an objection to it | 19:56 |
| *** Yogi2 has joined #openstack-qa | 19:56 | |
| andreaf | well I pull .testr.conf from git in my run folder | 19:57 |
| *** Yogi has quit IRC | 19:58 | |
| mtreinish | yeah, that doesn't seem like the right way to me | 20:00 |
| *** Yogi2 has quit IRC | 20:00 | |
| *** Yogi has joined #openstack-qa | 20:01 | |
| sdague | hogepodge: so... while bike riding I had an idea about the idempotent ids and subclasses | 20:03 |
| sdague | it's a little ugly | 20:03 |
| sdague | but it might be worth it | 20:03 |
| *** dimtruck is now known as zz_dimtruck | 20:03 | |
| mtreinish | andreaf: I think the long term plan for this is one of things we need to have a session on in vancouver. There is already one on the etherpad for it | 20:04 |
| mtreinish | andreaf: heh, also if you're copying your testr.conf around you can just set that path to whatever... | 20:04 |
| *** hitalia has joined #openstack-qa | 20:06 | |
| *** zz_dimtruck is now known as dimtruck | 20:06 | |
| *** madhu_ak has joined #openstack-qa | 20:07 | |
| hogepodge | sdague: ok | 20:08 |
| sdague | so... what if the decorator took a 2nd optional parameter | 20:09 |
| sdague | which was the class name it was supposed to apply to | 20:09 |
| sdague | things that have 2 tests by inheritance get 2 id decorators on the function | 20:09 |
| sdague | then enforce the id only shows up in a testr list once | 20:10 |
| sdague | as part of the pep8 check | 20:10 |
| hogepodge | sdague: seems like it would work, I worry about the extra effort in breaking inheritance. I don't like the parent having to know about the children | 20:11 |
| sdague | honestly the inheritance isn't really generic | 20:13 |
| sdague | it's just an easy way to duplicate across scenarios | 20:13 |
| sdague | our last layer of inheritance in tempest *really* should be done with test scenarios, not inheritance | 20:14 |
| sdague | but ... it goes back a ways in time | 20:14 |
| hogepodge | ok | 20:14 |
| hogepodge | sdague: I was thinking it would be useful to talk about this in Vancouver, so that all of the stakeholders can be in the same room | 20:15 |
| sdague | hogepodge: sure | 20:15 |
| sdague | like I said, it's an idea that would solve your problem, but it's a bit ugly | 20:15 |
| *** e0ne has quit IRC | 20:17 | |
| hogepodge | feels like all of the solutions are | 20:17 |
| hogepodge | need to go afk | 20:17 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 136567,10,249fa6f | 20:20 |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Add dstat to subnode in multinode mode https://review.openstack.org/177970 | 20:21 |
| *** dimtruck is now known as zz_dimtruck | 20:25 | |
| *** samueldmq has quit IRC | 20:29 | |
| *** dustins_ has joined #openstack-qa | 20:30 | |
| sirushti | sdague, around? a question regarding grenade. So I revised https://review.openstack.org/#/c/86978/ using the new modular resource checks but it seems like I'm missing on how to actually enable the Heat service in the gate. It seemed to work fine locally. Could you take a look at it? | 20:30 |
| *** dustins has quit IRC | 20:34 | |
| dkranz | mtreinish: You want to +A the rest of https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:smoke,n,z ? | 20:36 |
| *** e0ne has joined #openstack-qa | 20:36 | |
| dkranz | mtreinish: the only one I have an issue with is the admin. Seems to me we would want to have quick checks of all resources after upgraded and so keystone would be part of that even if admin. Operators still care about keystone as much as other stuff, don't they? And keystone tests are pretty fast. | 20:37 |
| *** e0ne has quit IRC | 20:39 | |
| mtreinish | dkranz: I thought you wanted to wait until after the tag | 20:43 |
| dkranz | mtreinish: as in my review comment, changed my mind | 20:43 |
| openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 171736,6,e7d3709 | 20:43 |
| mtreinish | dkranz: it doesn't bother me one way or the other. I think this would be a good chance to add some keystone tests that don't require admin | 20:44 |
| mtreinish | hogepodge: already started that by adding a basic token test | 20:44 |
| dkranz | mtreinish: I was really just thinking I would like to use 'smoke' for stuff too, and my "vision" for that is the same as Sean's except for wanting some basic tests around user/projects/etc. | 20:45 |
| dkranz | mtreinish: It would have been better to have a qa spec for the new meaning of smoke | 20:45 |
| mtreinish | dkranz: well I did ask sdague to write up a doc to explain what it means | 20:46 |
| dkranz | mtreinish: yes, but what he thinks it means may not be the same as others. That is the point of having a spec to review before pushing patches, no? | 20:46 |
| sdague | ok, so I did an ML post instead after conversation with dkranz yesterday | 20:46 |
| dkranz | sdague: yes, thanks for doing that | 20:46 |
| sdague | so, how about put commentary there | 20:47 |
| dkranz | sdague: I was going to but there is only one that is controversial and I wanted to push the others through | 20:47 |
| sdague | dkranz: ok | 20:47 |
| sdague | well we can have the conversation in parallel | 20:47 |
| dkranz | sdague: sure | 20:47 |
| sdague | your concern is the keystone tests? | 20:48 |
| dkranz | sdague: yes | 20:48 |
| sdague | can you specify which tests you think should remain? | 20:48 |
| dkranz | sdague: Simple ops on the keystone resources | 20:48 |
| dkranz | sdague: I thought the point was to check upgreade, at least as you wanted it | 20:48 |
| sdague | dkranz: right, but I mean comment in that patch that removes them - don't remove this | 20:48 |
| sdague | don't remove this | 20:48 |
| sdague | so specific feedback | 20:48 |
| dkranz | sdague: sure | 20:48 |
| sdague | if you do that, I'll adjust it | 20:48 |
| dkranz | sdague: ok, cool | 20:49 |
| sdague | I think it will be easier to think about as remove all, then add back these 6 or something | 20:49 |
| *** zigo_ is now known as zigo | 20:50 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Remove tracing since ansible seems to be working https://review.openstack.org/178398 | 20:50 |
| sdague | dkranz: so I want a small smoke run, which means it's good enough for sanity checking the grenade jobs, adding keystone stuff back in isn't something I'm strongly opposed to, it just felt like it wasn't needed | 20:51 |
| openstackgerrit | Joe Gordon proposed openstack/tempest: Add volume backed live migration test https://review.openstack.org/165233 | 20:51 |
| sdague | the object, database, and sahara tests need to be trimmed still as well, haven't gotten there yet | 20:51 |
| sdague | sirushti: so... other things will need to change in the gate to put heat in there, but I think we might want to make a specific heat config for this | 20:52 |
| jamielennox | dtroyer: can i kill this file: https://github.com/openstack-dev/devstack/blob/master/tools/create_userrc.sh | 20:52 |
| jamielennox | is it used anywhere outside of devstack? | 20:52 |
| jamielennox | i want to replace it with something that writes userrc files as you go for the users you just created rather than iterate through users at the end | 20:53 |
| jamielennox | or sdague ^ | 20:53 |
| dkranz | sdague: I don't disagree except that there are other use cases and the one I had in mind was checking that ops on all resources work to some extent. Making review comments now. | 20:53 |
| sdague | dkranz: yup, that's fine | 20:53 |
| sdague | jamielennox: I don't know, but honestly the last kill it with fire broke a lot of folks, so I think we should be cautious | 20:54 |
| sdague | and put something on the ML first | 20:54 |
| *** harlowja_ has quit IRC | 20:55 | |
| jamielennox | sdague: ok, yea i was surprised when i found it was in its own script in tools at all so i assumed at some point someone wanted to use it | 20:55 |
| dtroyer | jamielennox: IIRC it was an outside contribution, I don't think we use it directly but there are some who do. | 20:55 |
| dtroyer | that doesn't mean DevStack itself needs to use it though, we can go ahead with something a bit more streamlined | 20:56 |
| jamielennox | dtroyer: devstack does use it to generate eg accrc/admin/admin etc | 20:56 |
| dtroyer | right, but we don't have to continue to do that…just make the change and leave the exisitng script for a while | 20:56 |
| jamielennox | i want to write the v3 equivalent rc files, but it's a pain to include it in that script and i can't see why we run it at the end rather than just generate them as we do the accounts | 20:56 |
| jamielennox | dtroyer: ok | 20:57 |
| *** melwitt has quit IRC | 20:59 | |
| *** melwitt has joined #openstack-qa | 20:59 | |
| sdague | dtroyer: now that you are back gainfully employed, are you going to dive in on the neutron rewrite? | 21:00 |
| *** marun has joined #openstack-qa | 21:00 | |
| sdague | also, I have a stack of additional grenade cleanups - https://review.openstack.org/#/q/status:open+project:openstack-dev/grenade+branch:master+topic:modular_grenade,n,z | 21:00 |
| *** melwitt has quit IRC | 21:01 | |
| *** melwitt has joined #openstack-qa | 21:01 | |
| *** suneelb has quit IRC | 21:03 | |
| *** suneelb has joined #openstack-qa | 21:03 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-neutron-full-juno 177840,3,b13695f | 21:07 |
| sdague | dkranz: everything in https://review.openstack.org/#/c/177810/4 seems reasonable to me, I will respin that in the morning | 21:08 |
| sdague | thank you for your feedback | 21:08 |
| dkranz | sdague: thanks for doing this. It was long overdue. | 21:08 |
| *** jgrimm is now known as zz_jgrimm | 21:09 | |
| *** rfolco has quit IRC | 21:11 | |
| jamielennox | does neutron rewrite mean neutron itself or devstack - because *shudder* | 21:12 |
| *** dustins_ has quit IRC | 21:13 | |
| *** ajmiller has joined #openstack-qa | 21:17 | |
| *** artom has quit IRC | 21:19 | |
| openstackgerrit | Andrey Pavlov proposed openstack/tempest: Drop auth and corresponding unit tests https://review.openstack.org/166022 | 21:19 |
| dtroyer | sdague: I hope so, want to get an update release of OSC out probably tomorrow yet | 21:23 |
| dtroyer | jamielennox: didn't I tell you I wanted to take over the world? Shhhhhh…it'll be in Go! | 21:24 |
| jogo | morganfainberg: http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-n-api.txt.gz?level=TRACE#_2015-04-28_19_48_11_241 ! | 21:24 |
| dtroyer | seriously, its just DevStack's lib/neutron Refactor of Sanity | 21:24 |
| jogo | morganfainberg: what is keystoneclient spewing log spam about now ? | 21:25 |
| mtreinish | jogo, morganfainberg: oh yeah I was going to point that out at some point too, I saw similar in the cinder logs | 21:25 |
| mtreinish | jamielennox: ^^^ you can fix that right :) | 21:26 |
| jogo | not surprised | 21:26 |
| jogo | mtreinish: this is is why ops hate us | 21:26 |
| jogo | jamielennox: ^ | 21:26 |
| mtreinish | jogo: not us, just morganfainberg :) | 21:26 |
| jogo | mtreinish: :) | 21:27 |
| *** Yogi has quit IRC | 21:27 | |
| jamielennox | jogo: it's to do with cinderclient | 21:27 |
| jamielennox | though it's probably too verbose in ksc | 21:27 |
| openstackgerrit | Merged openstack/tempest: remove gate tag (part 2) https://review.openstack.org/177760 | 21:29 |
| openstackgerrit | Merged openstack/tempest: remove gate tag (part 3) https://review.openstack.org/177761 | 21:29 |
| mtreinish | jamielennox: this is what I was referring to seeing before: http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-c-api.txt.gz?level=TRACE | 21:29 |
| mtreinish | similar but different | 21:29 |
| jogo | jamielennox: nova doesn't use KSC at least not directly | 21:30 |
| openstackgerrit | Merged openstack/tempest: Switch all uses of ConfigParser to import from six https://review.openstack.org/176785 | 21:30 |
| openstackgerrit | Merged openstack/tempest: Switch all uses of ConfigParser to import from six https://review.openstack.org/176785 | 21:30 |
| jamielennox | mtreinish: hmm, that's not the one i was thinking of | 21:30 |
| mtreinish | actually that's worse, it looks like it logs that on every incoming request | 21:30 |
| mtreinish | ^^^ oh a double merge I must have good luck today :) | 21:30 |
| jamielennox | jogo: it does if you use session, like for example neutron talking to nova | 21:31 |
| * mtreinish feels like that means the gerrit stream is going to explode again | 21:31 | |
| openstackgerrit | Merged openstack-dev/devstack: fix warn function https://review.openstack.org/178178 | 21:32 |
| jogo | jamielennox: err I was thinking OSC, you are right | 21:32 |
| jamielennox | mtreinish: so i haven't seen those messages - are they new? | 21:34 |
| *** ajmiller_ has joined #openstack-qa | 21:34 | |
| jamielennox | i mean i've seen the messages, but they shouldn't be showing up like thta | 21:34 |
| mtreinish | jamielennox: I noticed them yesterday (or friday) when I was debugging a volume fail | 21:35 |
| mtreinish | don't know how long they've been showing up | 21:35 |
| * mtreinish checks logstash | 21:35 | |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Drop auth and corresponding unit tests https://review.openstack.org/166022 | 21:36 |
| jogo | been around for at least a week | 21:36 |
| *** dkranz has quit IRC | 21:36 | |
| *** ajmiller has quit IRC | 21:37 | |
| jogo | jamielennox: and the cinder log version http://logs.openstack.org/70/177970/3/check/check-tempest-dsvm-neutron-full/ff7b99b//logs/screen-c-api.txt.gz?level=TRACE is also terribad | 21:38 |
| mtreinish | jogo: that's what we were just talking about, it's different and worse | 21:38 |
| jogo | jamielennox morganfainberg: you will need another release of kestonemiddleware and keystoneclient for Kilo IMHO | 21:38 |
| mtreinish | jamielennox: it looks like it started to peak on the 22nd | 21:39 |
| mtreinish | 3.3M warnings emitted in the past 10 days | 21:39 |
| jogo | mtreinish: at least it isn't glance http://logs.openstack.org/70/177970/3/check/check-tempest-dsvm-neutron-full/ff7b99b//logs/screen-g-api.txt.gz?level=TRACE | 21:39 |
| mtreinish | jogo: heh, yeah. I'm still trying to figure out why glance has an http client to talk to itself | 21:40 |
| mtreinish | and 404ing when it does... | 21:40 |
| jogo | mtreinish: yeah :) | 21:40 |
| mtreinish | flaper87: ^^^ | 21:40 |
| *** mlavalle has quit IRC | 21:41 | |
| jogo | mtreinish: so much fail | 21:41 |
| mtreinish | jogo: heh, well I'm having fun making graphs: https://i.imgur.com/0zg4Rdh.png | 21:43 |
| jamielennox | jogo: so that one i'm not sure about Unable to find authentication token in headers AFAIK nothing has changed in middleware that would cause that to happen where it wouldn't before | 21:44 |
| mtreinish | heh, my favorite thing is the variance in the gate is so large the std dev part goes really negative | 21:45 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 178376,1,db393c2 | 21:46 |
| mtreinish | jamielennox: hmm, well I'm positive we've not changed anything in tempest around tokens in the headers either, and most of those warnings are coming from tempest requests afaict | 21:52 |
| mtreinish | and why just cinder... | 21:53 |
| jamielennox | mtreinish: so i'm spinning up a devstack now to test it but the message is fairly self explanitory https://github.com/openstack/keystonemiddleware/blob/master/keystonemiddleware/auth_token/__init__.py#L680 | 21:54 |
| jamielennox | there is no X-Auth-Token in the request headers | 21:54 |
| jamielennox | it should probably be downgraded from a warn | 21:54 |
| jogo | jamielennox: cool | 21:54 |
| jogo | mtreinish: what happens if you remove outliers? | 21:54 |
| mtreinish | jamielennox: oh, you know what I don't think the reqs are from tempest | 21:56 |
| mtreinish | I can't find any of the req-id from the warns in the tempest logs | 21:56 |
| jamielennox | mtreinish: if they are from cinder client that would make sense | 21:56 |
| *** mlavalle has joined #openstack-qa | 21:56 | |
| mtreinish | so it might be nova talking to cinder that's screwing up | 21:56 |
| mtreinish | jamielennox: yep | 21:56 |
| *** dboik has quit IRC | 21:56 | |
| jamielennox | mtreinish: because the same release that caused the endpoint warnings might have broken the cinder shell | 21:57 |
| mtreinish | jogo: hmm, not sure. I can hard code a filter in to test | 21:57 |
| jamielennox | oh - nova -> cinder? that would be bad | 21:57 |
| mtreinish | jamielennox: well I don't think anything talks to cinder besides nova and tempest | 21:57 |
| *** pglass has quit IRC | 21:58 | |
| jamielennox | i just mean if it's a bad cinderclient we can fix that, if it's nova that would be harder | 21:58 |
| *** Swami has quit IRC | 21:59 | |
| jamielennox | are the tests actually failing or is it just writing a lot of logs ? | 21:59 |
| mtreinish | jamielennox: just logging too much | 21:59 |
| openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm 172906,3,4f62098 | 21:59 |
| mtreinish | the runs are still passing | 21:59 |
| mtreinish | jamielennox: well nova only talks to cinder via cinderclient | 22:00 |
| jamielennox | that's weird - why would the logging have started being a problem now | 22:00 |
| mtreinish | was there a cinderclient release recently? | 22:00 |
| jamielennox | yea | 22:00 |
| jamielennox | 015-04-21 | 22:00 |
| jamielennox | 2015-04-21 | 22:00 |
| mtreinish | oh, that kinda lines up with the spike | 22:01 |
| jamielennox | thingee!! | 22:01 |
| jogo | jamielennox: even though this doesn't cause anything to break, IMHO it would look really bad if this ended up in Kilo | 22:01 |
| jamielennox | jogo: so from auth_token we need to downgrade that log message - any suggestions> | 22:01 |
| jamielennox | otherwise we need to figure out what's actually calling it, because AFAIK it's been a warn forever | 22:02 |
| *** thingee has joined #openstack-qa | 22:02 | |
| mtreinish | thingee: so you missed the backscroll, but it looks like the latest cinderclient release is causing: http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-c-api.txt.gz?level=TRACE#_2015-04-28_19_58_02_882 | 22:02 |
| mtreinish | that gets logged on every incoming request that uses cinderclient | 22:03 |
| *** miqui has quit IRC | 22:03 | |
| mtreinish | jamielennox: sigh, yeah this where having the reverse req-id mapping would be super useful | 22:03 |
| jamielennox | reverse? | 22:03 |
| mtreinish | but I'm pretty sure it has to be nova, because nothing else would be calling cinder | 22:03 |
| mtreinish | jamielennox: basically nova logs the req-id returned from cinder | 22:03 |
| mtreinish | right now they get returned by cinder and nova drops them | 22:04 |
| jamielennox | oh, right - we could just log it from auth_token | 22:04 |
| thingee | mtreinish: I'll take a look in a sec...about to push to get rid of discovery warnings | 22:04 |
| thingee | push a fix* | 22:04 |
| *** harlowja has joined #openstack-qa | 22:04 | |
| thingee | in cinderclient | 22:04 |
| jamielennox | mtreinish: or we could log it from session and that would do it as well | 22:05 |
| mtreinish | jamielennox: sure somewhere, but all we need is a way to correlate nova's req-id with the cinder calls it makes | 22:06 |
| mtreinish | so we can trace an operation through each step | 22:06 |
| mtreinish | I thought there was an openstack-spec for this somewhere | 22:06 |
| Rockyg | there is | 22:06 |
| jamielennox | i saw this on a review recently - why does the request id change? surely what you pass to nova is the same as what nova passes to cinder? otherwise it's pointless | 22:06 |
| Rockyg | https://review.openstack.org/#/c/156508/ i think | 22:07 |
| mtreinish | jamielennox: no, the req-id's are uniquely generated on each incoming request, and returned in the resp headers | 22:07 |
| jamielennox | mtreinish: that's pointless | 22:07 |
| jogo | jamielennox: downgrade and figure out what is calling it | 22:08 |
| jogo | jamielennox: lets do both? | 22:08 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Set pipefail when running tsfilter https://review.openstack.org/178437 | 22:08 |
| *** jecarey_ has quit IRC | 22:08 | |
| mtreinish | jamielennox: well, only for the cross-service request case. In the general case it lets you track internal single service operations which are all async and multithreaded | 22:09 |
| *** melwitt has quit IRC | 22:09 | |
| *** melwitt has joined #openstack-qa | 22:09 | |
| jamielennox | mtreinish: keeping the same req id would do that as well | 22:09 |
| mtreinish | jamielennox: sure | 22:09 |
| mtreinish | there were sec (which I never understood, it's just an identifier) and collision concerns (which doesn't make much sense since they should be uuids) for letting the caller set the id | 22:10 |
| mtreinish | at least in HK when this was first discussed | 22:10 |
| mtreinish | maybe the wind has changed direction | 22:11 |
| bknudson | you don't want a huge identifier at least. | 22:11 |
| mtreinish | bknudson: it's already a uuid4, and that's fine | 22:11 |
| jamielennox | i don't *think* there's a sec issue, it will only show up in logs which a user can't see, and even if there is a collision it just makes it confusing to debug that single request | 22:11 |
| bknudson | if the client generates the ID they might not pick uuid4 | 22:11 |
| jamielennox | bknudson: right, but if the client picked '1' every time it just makes there own logs more confusing | 22:12 |
| bknudson | you're right they will only screw themselves. | 22:12 |
| mtreinish | jamielennox: unless they follow the glanceclient and requires the user to provide the req-id and everyone uses 1 | 22:13 |
| bknudson | but you also don't want them to pick '1'*1024*1024*1024 | 22:13 |
| mtreinish | :) | 22:13 |
| openstackrecheck | Console logs not available after 13:30s for gate-horizon-jshint 178227,12,edde696 | 22:13 |
| thingee | mtreinish: ok back | 22:13 |
| *** zz_jgrimm has quit IRC | 22:13 | |
| mtreinish | jamielennox: but you could just make it opt-in, and only have the services exploit it | 22:13 |
| mtreinish | and then put a doc saying don't do this unless you know what your doing | 22:14 |
| jogo | I found one caller | 22:14 |
| jogo | http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-n-api.txt.gz#_2015-04-28_19_45_53_070 | 22:14 |
| jamielennox | mtreinish: so if it's not present from the client then the first service should generate one from the rest of the request | 22:14 |
| jogo | http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-c-api.txt.gz?#_2015-04-28_19_58_02_882 | 22:14 |
| jogo | looks like it is cinderclient | 22:15 |
| mtreinish | jamielennox: well there is already oslo.middleware to generate one for incoming requests, but yeah | 22:15 |
| mtreinish | jogo: yep | 22:16 |
| *** zz_jgrimm has joined #openstack-qa | 22:16 | |
| jogo | here it is http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-n-api.txt.gz#_2015-04-28_19_45_51_792 | 22:16 |
| jogo | but I see an X-Auth-Token | 22:17 |
| mtreinish | is that (SHA1) just a logging artifact or is that what it sends over the wire? | 22:18 |
| *** suneelb has quit IRC | 22:18 | |
| jogo | AFAIK its just logging | 22:18 |
| jogo | that would be really funny if we sent that over | 22:18 |
| jogo | jamielennox: ^ | 22:18 |
| jogo | thingee: ^ | 22:18 |
| *** suneelb has joined #openstack-qa | 22:18 | |
| mtreinish | jogo: heh, but I was thinking that's an invalid token if we did send it | 22:18 |
| jamielennox | mtreinish: yea, it's a SHA1 of the token so as not to have them in logs | 22:19 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Set pipefail when running tsfilter https://review.openstack.org/178437 | 22:19 |
| *** bknudson has quit IRC | 22:19 | |
| *** melwitt has quit IRC | 22:22 | |
| *** melwitt has joined #openstack-qa | 22:22 | |
| jogo | so there is something else happening here | 22:24 |
| jogo | search req-3bbeb5dd-a092-4c67-9dd8-4b3a823c57d1 | 22:24 |
| jogo | in http://logs.openstack.org/60/177760/1/gate/gate-tempest-dsvm-full/fd8bb99//logs/screen-c-api.txt.gz?#_2015-04-28_19_58_02_882 | 22:24 |
| jogo | the same request ID is being used twice | 22:24 |
| mtreinish | jogo: so filtering it to < 225 sec it just looks the same: https://i.imgur.com/O0uVkXi.png | 22:25 |
| mtreinish | which really isn't a surprise given the variability in the gate | 22:25 |
| mtreinish | it just makes it look fuzzier because the smaller scale variations are more noticeable | 22:26 |
| mtreinish | jogo: yeah I notied that, but the bad auth token gets looked twice too | 22:26 |
| openstackrecheck | Console logs not available after 13:26s for gate-tempest-dsvm-large-ops 178398,1,0d4dfa5 | 22:26 |
| jogo | do we have some replay happening? | 22:26 |
| *** Rockyg has quit IRC | 22:27 | |
| jogo | thingee: ^ | 22:27 |
| jogo | Assuming the req-id is generated serverside keystone | 22:27 |
| jogo | and not passed in somehow | 22:27 |
| mtreinish | jogo: no, it would be generated server side cinder | 22:28 |
| mtreinish | keystone has it's own request id | 22:28 |
| jogo | this would point to a bug in cinder server | 22:28 |
| jogo | mtreinish: I thought cinder but wrote keystone | 22:28 |
| mtreinish | heh, ok | 22:28 |
| mtreinish | yeah, I dunno logging the 401 twice is weird | 22:29 |
| jogo | jamielennox: switching gears for a moment, the Failed to contact the endpoint' line | 22:30 |
| jogo | are you working on resolving that? and does it sound like a keystoneclient error? | 22:30 |
| mtreinish | jogo: oh search for that query path that's raising the 401, something is really weird, because if I'm not mistaken the second part is the tenant id | 22:30 |
| mtreinish | which should be different for each test | 22:30 |
| jamielennox | jogo: that will be fixed by: https://review.openstack.org/#/c/178439/2 | 22:30 |
| mtreinish | but it's not | 22:31 |
| jogo | jamielennox: ahh excellent | 22:32 |
| jogo | jamielennox thingee: and the " Unable to find authentication token in headers" issue | 22:32 |
| jogo | any ideas of where that comes from? | 22:33 |
| *** Swami has joined #openstack-qa | 22:33 | |
| * jogo wonders if it is something in the cinder wsgi pipeline | 22:33 | |
| jamielennox | jogo: still looking | 22:34 |
| jamielennox | i was waiting for tempest to stop on my ipdb but then remembered it's parallel | 22:34 |
| thingee | jogo: not yet. was resolving the second issue you just brought up | 22:35 |
| jogo | thingee: ack. I am thinking something is causing a replay in the pipeline | 22:36 |
| jogo | wsgi pipeline | 22:36 |
| jogo | I wonder if its osprofiler? | 22:36 |
| mtreinish | jogo: oh, heh. There is that crazy traceback from the unit tests... | 22:38 |
| mtreinish | that could very well be part of the issue | 22:38 |
| mtreinish | jogo: that's easy enough to test | 22:38 |
| *** zz_jgrimm has quit IRC | 22:39 | |
| openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-full 85249,18,4be20dd | 22:40 |
| mtreinish | jogo: https://review.openstack.org/#/c/178445/ | 22:40 |
| thingee | mtreinish: thanks subscribing now | 22:41 |
| jogo | testing it locally | 22:41 |
| mtreinish | jogo: oh, I've been trying to kill my dependence on rax since I'm losing the free credit | 22:41 |
| mtreinish | I probably should setup vms on my desktop... | 22:41 |
| mtreinish | so locally for me right now is the gate :) | 22:42 |
| *** davideagnello has quit IRC | 22:42 | |
| jamielennox | surely hp has some public cloud you could use | 22:42 |
| jogo | I am using HP cloud | 22:42 |
| thingee | mtreinish: start asking dreamhost :) | 22:42 |
| *** jecarey has joined #openstack-qa | 22:42 | |
| mtreinish | jamielennox: I have enough quota for 1 server | 22:42 |
| mtreinish | well one server I can actually run devstack on | 22:43 |
| mtreinish | I have to ask for more quotas, just been lazy because rax works | 22:43 |
| *** davideagnello has joined #openstack-qa | 22:43 | |
| mtreinish | thingee: heh, that's an idea | 22:43 |
| *** mriedem is now known as mriedem_away | 22:44 | |
| jamielennox | red hat for the win there | 22:44 |
| *** timello has quit IRC | 22:44 | |
| *** heyongli has quit IRC | 22:45 | |
| *** maurosr has quit IRC | 22:45 | |
| thingee | Intel donates me some crazy boxes, so I just turn those into hypervisors. | 22:45 |
| thingee | $ cat /proc/cpuinfo | grep -c processor | 22:46 |
| thingee | 56 | 22:46 |
| mtreinish | thingee: heh, yeah 56 threads in a single box would be awesome | 22:47 |
| *** timello has joined #openstack-qa | 22:47 | |
| *** maurosr has joined #openstack-qa | 22:47 | |
| jamielennox | thingee: oh! i bet it's the discovery requests | 22:48 |
| *** davideagnello has quit IRC | 22:48 | |
| *** heyongli has joined #openstack-qa | 22:49 | |
| *** zz_jgrimm has joined #openstack-qa | 22:49 | |
| thingee | huh, yeah | 22:49 |
| jamielennox | so the request to list versions is unauthenticated by default, so every one of the 'couldn't contact' messages will have a corresponding 'no auth token' on the api | 22:50 |
| *** Swami_ has joined #openstack-qa | 22:50 | |
| jogo | jamielennox: how come the req-id is the same though? | 22:50 |
| jogo | in cinder server | 22:50 |
| jamielennox | i'm not sure on that, unless the req-id is being passed in from the client | 22:51 |
| jogo | thingee: ^ | 22:51 |
| *** dboik has joined #openstack-qa | 22:53 | |
| *** Swami has quit IRC | 22:53 | |
| mtreinish | jamielennox: they wouldn't be | 22:53 |
| openstackrecheck | Console logs not available after 13:28s for check-tempest-dsvm-neutron-full 177920,2,4a6c73b | 22:53 |
| mtreinish | because that mechanism doesn't exist | 22:54 |
| *** dboik_ has joined #openstack-qa | 22:54 | |
| *** rbak has quit IRC | 22:54 | |
| mtreinish | and the req-id in oslofmt logs comes from oslo context (which is why keystone doesn't log req-id in it's log file yet) | 22:54 |
| thingee | yes, I'm not aware of anything | 22:54 |
| *** timello has quit IRC | 22:55 | |
| *** ekhugen has quit IRC | 22:55 | |
| jogo | so I think that is part of the answer actually | 22:55 |
| *** ekhugen has joined #openstack-qa | 22:56 | |
| jogo | 2015-04-28 19:39:03.793 1314 WARNING keystonemiddleware.auth_token [-] Unable to find authentication token in headers | 22:56 |
| jogo | "GET /v1/956e65d4518f4611ba40b7b199535d31 HTTP/1.1" 401 257 0.002910 | 22:56 |
| jogo | that has no req-id | 22:56 |
| *** davideagnello has joined #openstack-qa | 22:56 | |
| jogo | and is the right REST call I think | 22:56 |
| *** timello has joined #openstack-qa | 22:56 | |
| *** sabeen2 has quit IRC | 22:57 | |
| jamielennox | yep, there's no other reason for a call to GET /v1/project_id other than it tried to do discovery based on what was in the service catalog | 22:57 |
| *** dboik has quit IRC | 22:57 | |
| jogo | so that is part of the puzzle | 22:57 |
| jamielennox | afaik there is no response at the base url - thingee? | 22:57 |
| andreaf | mtreinish, sdague: a quick review to fix a tempest bug if you have a moment: https://review.openstack.org/#/c/178187 | 22:58 |
| jamielennox | also i'd love to see a proposal for cinder and nova to scrap project_id from urls, either 301 them from /v2/project_id to /v2/ or handled internally | 22:59 |
| thingee | jamielennox: base should return available versions | 23:00 |
| jamielennox | thingee: GET / should and GET /v1/ should but what about GET /v1/project_id | 23:00 |
| jogo | it looks like all the Unable to find authentication token in headers lines are related to the discovery junk | 23:01 |
| jogo | why req-ids are being re-used I have no idea | 23:02 |
| jamielennox | so my devstack is still reloading with this patch: https://review.openstack.org/#/c/178439/ | 23:02 |
| jamielennox | but i guess the test will be the tempest logs for ^ | 23:02 |
| mtreinish | jogo, thingee: looking at the cinder code, I'm not sure removing osprofiler from the pipeline will be enough to stop the interaction if it is the cause | 23:04 |
| mtreinish | it looks like it latches on regardless of your pipeline settings | 23:04 |
| jogo | mtreinish: haha I don't think its osprofiler anymore either | 23:04 |
| jogo | thingee jamielennox: http://paste.openstack.org/show/210951/ | 23:04 |
| jamielennox | jogo: using that patch? | 23:05 |
| jogo | jamielennox: no | 23:05 |
| thingee | jogo: that patch should get rid of that | 23:05 |
| thingee | jamielennox: pretty sure that would result in a 404 | 23:06 |
| thingee | route would fail | 23:06 |
| openstackrecheck | Console logs not available after 13:24s for check-grenade-dsvm 177664,2,2a7b8a3 | 23:07 |
| jamielennox | jogo: so that patch fixes the "cinder list" messages for me, but i'm still seeing the errors on the backend | 23:07 |
| thingee | jamielennox: yeah just verified /v1/tenant_id gives a 404 | 23:08 |
| jamielennox | ok | 23:08 |
| *** weshay has joined #openstack-qa | 23:09 | |
| jamielennox | so it can only be caused by discovery | 23:09 |
| thingee | jamielennox: I agree that moving away from version/tenant urls would be good | 23:09 |
| jamielennox | thingee: i don't know how hard it would be - it might just be a middleware that can be installed after auth_token | 23:09 |
| *** dboik_ has quit IRC | 23:11 | |
| jogo | jamielennox: the patch didn't help at all for me | 23:11 |
| jogo | to be clear the patch being: https://review.openstack.org/#/c/178439/ | 23:11 |
| thingee | jogo: ps1 worked for me | 23:11 |
| *** dboik has joined #openstack-qa | 23:11 | |
| thingee | let me try ps2 | 23:12 |
| jogo | yup PS1 worked for me as well | 23:12 |
| jogo | PS2 not at all | 23:13 |
| *** hitalia has quit IRC | 23:14 | |
| jamielennox | for calling "cinder list" or for tempest? | 23:14 |
| jogo | jamielennox: cinder list | 23:14 |
| jogo | that is all I tested | 23:14 |
| jogo | for tempest did you restart nova and everything else to get the new client? | 23:15 |
| thingee | yeah ps2 doesn't not work when I have OS_VOLUME_API_VERSION set to 2. | 23:15 |
| thingee | works when I have it set to 1 | 23:15 |
| *** zz_jgrimm has quit IRC | 23:18 | |
| *** timello has quit IRC | 23:19 | |
| *** timello has joined #openstack-qa | 23:20 | |
| *** zz_jgrimm has joined #openstack-qa | 23:21 | |
| openstackrecheck | Console logs not available after 13:28s for check-grenade-dsvm-neutron 177840,3,927e791 | 23:23 |
| *** armax has quit IRC | 23:23 | |
| *** hemna is now known as hemnafk | 23:25 | |
| jogo | have to head out. sounds like you guys have a handle on this bug | 23:25 |
| *** bknudson has joined #openstack-qa | 23:27 | |
| jamielennox | thingee: oops - i've got volume2 in https://review.openstack.org/#/c/178439/2/cinderclient/client.py instead of volumev2 | 23:27 |
| thingee | ah ha | 23:27 |
| thingee | not even I saw that | 23:27 |
| thingee | I got bored and started eating an apple. | 23:28 |
| mtreinish | jamielennox: what stands out more to me there is the python 2.5 workaround | 23:28 |
| jamielennox | i have ipdb statements everywhere and i stared at it for a minute before i saw it | 23:28 |
| jamielennox | thingee: ok for me to re-push? | 23:29 |
| thingee | yes | 23:29 |
| *** dims_ has joined #openstack-qa | 23:29 | |
| *** hitalia has joined #openstack-qa | 23:29 | |
| *** erlon has quit IRC | 23:31 | |
| *** dims has quit IRC | 23:31 | |
| *** jogo has quit IRC | 23:33 | |
| *** jogo has joined #openstack-qa | 23:34 | |
| jamielennox | cool, that solves both cases for me - going afk for a bit | 23:35 |
| *** mlavalle has quit IRC | 23:37 | |
| thingee | jogo: ^ | 23:38 |
| openstackrecheck | Console logs not available after 13:22s for gate-cinder-pylint 177335,3,e6d274f | 23:39 |
| *** hitalia has quit IRC | 23:42 | |
| *** hitalia has joined #openstack-qa | 23:45 | |
| openstackrecheck | Console logs not available after 13:24s for check-grenade-dsvm 178332,2,fef957c | 23:58 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!
