| openstackrecheck | openstack/swift change: https://review.openstack.org/80095 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1262906 | 00:00 |
|---|---|---|
| *** mrodden has quit IRC | 00:02 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80097 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1288918and https://bugs.launchpad.net/bugs/1248757and https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1265495 | 00:03 |
| *** markmcclain has quit IRC | 00:04 | |
| openstackrecheck | openstack/trove change: https://review.openstack.org/77160 failed check-grenade-dsvm because of: https://bugs.launchpad.net/bugs/1272417 | 00:05 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/68195 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 00:07 |
| *** malini_afk is now known as malini | 00:10 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test caching-forwarding nameserver https://review.openstack.org/80045 | 00:11 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/74544 failed gate-nova-pep8, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full because of: https://bugs.launchpad.net/bugs/1249065and https://bugs.launchpad.net/bugs/1251448and https://bugs.launchpad.net/bugs/1273386 | 00:13 |
| *** rhsu has quit IRC | 00:13 | |
| *** malini is now known as malini_afk | 00:14 | |
| *** openstackrecheck has quit IRC | 00:15 | |
| *** openstackrecheck has joined #openstack-qa | 00:15 | |
| clarkb | jogo: ^ that restart happened because e-r updated | 00:16 |
| clarkb | jogo: you should see new data over the next hour | 00:16 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/70348 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 00:18 |
| *** malini_afk is now known as malini | 00:18 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 00:20 |
| *** jasondotstar has joined #openstack-qa | 00:22 | |
| *** mmaglana has joined #openstack-qa | 00:23 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/78024 failed check-tempest-dsvm-neutron, gate-keystone-python33 because of: https://bugs.launchpad.net/bugs/1249065and https://bugs.launchpad.net/bugs/1248757 | 00:24 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80106 failed gate-tempest-pep8, gate-tempest-python27, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1248757 | 00:26 |
| *** matsuhashi has joined #openstack-qa | 00:27 | |
| *** zhikunliu has joined #openstack-qa | 00:28 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/79727 failed gate-keystone-python33, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254890 | 00:29 |
| *** EmilienM has quit IRC | 00:30 | |
| *** sdague has quit IRC | 00:30 | |
| *** nati_uen_ has joined #openstack-qa | 00:31 | |
| jogo | mtreinish: https://review.openstack.org/#/c/68626/7/tempest/api/network/admin/test_l3_agent_scheduler.py isn't stable | 00:34 |
| *** nati_ueno has quit IRC | 00:34 | |
| *** EmilienM has joined #openstack-qa | 00:35 | |
| *** sdague has joined #openstack-qa | 00:35 | |
| *** jasondotstar has quit IRC | 00:36 | |
| openstackgerrit | A change was merged to openstack/tempest: Fix RemoteClient usage in scenario tests https://review.openstack.org/77221 | 00:38 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79660 failed gate-tempest-dsvm-neutron, gate-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 00:38 |
| jogo | sdague: https://bugs.launchpad.net/tempest/+bug/1291694 | 00:40 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1291694 https://review.openstack.org/80129 | 00:44 |
| *** jasondotstar has joined #openstack-qa | 00:45 | |
| jogo | clarkb: http://status.openstack.org/elastic-recheck/data/uncategorized.html updated thanks | 00:46 |
| *** rhsu has joined #openstack-qa | 00:47 | |
| *** mmaglana has quit IRC | 00:47 | |
| openstackrecheck | openstack/python-openstackclient change: https://review.openstack.org/80035 failed check-tempest-dsvm-neutron, gate-python-openstackclient-python33 because of: https://bugs.launchpad.net/bugs/1248757 | 00:48 |
| *** mmaglana has joined #openstack-qa | 00:48 | |
| *** nati_uen_ has quit IRC | 00:50 | |
| *** nati_ueno has joined #openstack-qa | 00:50 | |
| openstackgerrit | Joe Gordon proposed a change to openstack/tempest: Revert "Adds L3 agent test case to test_l3_agent_scheduler.py" https://review.openstack.org/80132 | 00:51 |
| jogo | mtreinish sdague ^ | 00:51 |
| *** mkoderer has quit IRC | 00:52 | |
| *** mmaglana has quit IRC | 00:52 | |
| *** cyeoh has quit IRC | 00:53 | |
| *** cyeoh has joined #openstack-qa | 00:54 | |
| *** mkoderer has joined #openstack-qa | 00:54 | |
| openstackrecheck | openstack/python-novaclient change: https://review.openstack.org/71846 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1248757 | 00:54 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/80116 failed check-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1288918 | 00:54 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79843 failed gate-neutron-docs, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1272417and https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1273386 | 00:55 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/80114 failed gate-heat-pep8, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1248757 | 00:56 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79801 failed gate-nova-pep8, gate-nova-pylint, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1272417 | 00:59 |
| *** sweston has joined #openstack-qa | 01:00 | |
| *** bill_az_ has quit IRC | 01:03 | |
| *** otherwiseguy has joined #openstack-qa | 01:03 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79251 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 01:06 |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify service attributes through Nova os-flavor-access API https://review.openstack.org/80135 | 01:10 |
| *** matsuhashi has quit IRC | 01:11 | |
| *** sthaha has joined #openstack-qa | 01:13 | |
| *** sthaha has joined #openstack-qa | 01:13 | |
| openstackrecheck | openstack/heat change: https://review.openstack.org/71512 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1248757 | 01:14 |
| *** matsuhashi has joined #openstack-qa | 01:14 | |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "create a server" API response attributes https://review.openstack.org/77517 | 01:15 |
| *** dkranz has joined #openstack-qa | 01:17 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 01:19 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/71511 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 01:20 |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify service attributes through Nova os-flavor-access API https://review.openstack.org/80135 | 01:20 |
| openstackgerrit | Brant Knudson proposed a change to openstack-dev/devstack: Keystone run db_version https://review.openstack.org/80137 | 01:24 |
| openstackgerrit | Brant Knudson proposed a change to openstack-dev/devstack: Keystone run db_version https://review.openstack.org/80137 | 01:26 |
| *** mkollaro has quit IRC | 01:27 | |
| *** rhsu has quit IRC | 01:28 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test caching-forwarding nameserver https://review.openstack.org/80045 | 01:29 |
| *** fungi has quit IRC | 01:29 | |
| *** matsuhashi has quit IRC | 01:31 | |
| *** matsuhashi has joined #openstack-qa | 01:31 | |
| *** matsuhashi has quit IRC | 01:32 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80097 failed gate-nova-python26, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1254872 | 01:32 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79458 failed gate-neutron-python26, gate-neutron-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: | 01:34 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1273386and https://bugs.launchpad.net/bugs/1101147and https://bugs.launchpad.net/bugs/1248757 | 01:34 |
| *** matsuhashi has joined #openstack-qa | 01:35 | |
| *** mspreitz has joined #openstack-qa | 01:40 | |
| *** dkranz has quit IRC | 01:41 | |
| mspreitz | I have just installed DevStack; if I want to run tempest do I have to do it now or can I do some other work first? That is, does tempest have pre-conditions that I might violate by working on something else first? | 01:42 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/68195 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 01:43 |
| *** ekhugen has joined #openstack-qa | 01:45 | |
| openstackgerrit | A change was merged to openstack-dev/grenade: Change default target from stable/icehouse to master https://review.openstack.org/79982 | 01:45 |
| *** rhsu has joined #openstack-qa | 01:46 | |
| *** nosnos has joined #openstack-qa | 01:46 | |
| *** ekhugen has quit IRC | 01:50 | |
| *** adalbas has joined #openstack-qa | 01:52 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/78091 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 01:58 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/75253 failed gate-tempest-dsvm-neutron-pg, gate-tempest-dsvm-neutron-pg because of: https://bugs.launchpad.net/bugs/1253896 | 02:00 |
| *** rfolco has joined #openstack-qa | 02:04 | |
| *** mriedem has quit IRC | 02:05 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80132 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1262153 | 02:05 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "update a server" API response attributes https://review.openstack.org/80147 | 02:07 |
| *** weshay has quit IRC | 02:10 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67382 failed gate-neutron-python27, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288312 | 02:13 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Split up stop_nova to match start_nova https://review.openstack.org/80017 | 02:15 |
| *** adalbas has quit IRC | 02:15 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/73802 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1288918and https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1273386 | 02:15 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "update a server" API response attributes https://review.openstack.org/80147 | 02:17 |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify service attributes through Nova os-quota-sets API https://review.openstack.org/80150 | 02:19 |
| *** malini is now known as malini_afk | 02:20 | |
| openstackgerrit | A change was merged to openstack/tempest: Skip Nova API attribute tests if an XML response https://review.openstack.org/79871 | 02:25 |
| *** vkozhukalov has quit IRC | 02:25 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79256 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error | 02:25 |
| openstackrecheck | openstack-infra/devstack-gate change: https://review.openstack.org/80045 failed check-grenade-dsvm, check-tempest-dsvm-full, check-dg-tempest-dsvm-full, check-dg-tempest-dsvm-full-reexec, gate-tempest-dsvm-large-ops, gate-tempest-dsvm-neutron-large-ops, check-swift-dsvm-functional, check-tempest-dsvm-full-grizzly, check-tempest-dsvm-neutron-grizzly, check-devstack-dsvm-cells-havana, check-tempest- | 02:27 |
| openstackrecheck | dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1286963 | 02:27 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "update a server" API response attributes https://review.openstack.org/80147 | 02:28 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "create a server" API response attributes https://review.openstack.org/77517 | 02:28 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Validate Volume attributes of Nova POST & GET API https://review.openstack.org/78593 | 02:39 |
| openstackrecheck | Console logs not available after 13:32s for gate-tempest-pep8 80135,2,5b5e765 | 02:41 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80136 failed gate-neutron-pep8, gate-neutron-python26, gate-neutron-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and | 02:43 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1273386 | 02:43 |
| *** matsuhashi has quit IRC | 02:45 | |
| openstackrecheck | openstack/python-novaclient change: https://review.openstack.org/76451 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 | 02:45 |
| *** matsuhashi has joined #openstack-qa | 02:45 | |
| *** matsuhashi has quit IRC | 02:46 | |
| *** mwagner_lap has joined #openstack-qa | 02:46 | |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/76787 failed check-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254890 | 02:46 |
| *** matsuhashi has joined #openstack-qa | 02:46 | |
| *** matsuhashi has quit IRC | 02:47 | |
| *** matsuhashi has joined #openstack-qa | 02:47 | |
| *** matsuhashi has quit IRC | 02:48 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/77300 failed gate-nova-python26, gate-nova-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254890 | 02:48 |
| *** matsuhashi has joined #openstack-qa | 02:49 | |
| *** matsuhashi has quit IRC | 02:49 | |
| *** matsuhashi has joined #openstack-qa | 02:50 | |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/79943 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 02:51 |
| *** giulivo has quit IRC | 02:53 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Validate Volume attributes of Nova POST & GET API https://review.openstack.org/78593 | 02:57 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Validate Volume attributes of Nova POST & GET API https://review.openstack.org/78593 | 02:59 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79882 failed check-tempest-dsvm-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254890 | 03:00 |
| openstackrecheck | openstack/python-ceilometerclient change: https://review.openstack.org/58051 failed gate-python-ceilometerclient-python27 because of: https://bugs.launchpad.net/bugs/1272417 | 03:05 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/78091 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896 | 03:10 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80142 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 03:12 |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/79726 failed gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254872 | 03:13 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Network fwaas API test https://review.openstack.org/78857 | 03:17 |
| *** nati_ueno has quit IRC | 03:21 | |
| openstackrecheck | openstack-dev/pbr change: https://review.openstack.org/79855 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 03:25 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/73673 failed gate-cinder-pep8, gate-cinder-python27 because of: https://bugs.launchpad.net/bugs/1270382 | 03:31 |
| *** fungi has joined #openstack-qa | 03:32 | |
| openstackgerrit | anju Tiwari proposed a change to openstack/tempest: Creating same name of security group https://review.openstack.org/80170 | 03:33 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Verify the list volume attributes of Nova APIs https://review.openstack.org/78950 | 03:37 |
| *** mmaglana has joined #openstack-qa | 03:37 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/73935 failed check-tempest-dsvm-neutron, gate-keystone-python33, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 03:46 |
| *** matsuhashi has quit IRC | 03:49 | |
| *** vkozhukalov has joined #openstack-qa | 03:51 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/77517 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv because of: https://bugs.launchpad.net/bugs/1254872 | 03:52 |
| *** rfolco has quit IRC | 03:56 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Network Firewall API https://review.openstack.org/79868 | 03:58 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79529 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 04:03 |
| *** mmaglana has quit IRC | 04:05 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67382 failed gate-neutron-python26, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288312 | 04:10 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80001 failed gate-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 04:12 |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/79726 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron, gate-keystone-python33, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 04:14 |
| *** harlowja is now known as harlowja_away | 04:20 | |
| *** afazekas has joined #openstack-qa | 04:21 | |
| *** dmorita has joined #openstack-qa | 04:21 | |
| openstackgerrit | jun xie proposed a change to openstack/tempest: Test for neutron api with provider extension https://review.openstack.org/79864 | 04:23 |
| *** harlowja_away is now known as harlowja | 04:24 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/78857 failed check-grenade-dsvm, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1240256 | 04:30 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80136 failed gate-neutron-python26, gate-neutron-python27, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 04:31 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Making 'body' param optional for resp validation https://review.openstack.org/80174 | 04:32 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/80169 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1253896 | 04:37 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79880 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1265495and https://bugs.launchpad.net/bugs/1248757 | 04:42 |
| *** rms_13 has joined #openstack-qa | 04:43 | |
| rms_13 | Anyone from devstack-core: Ping | 04:44 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80170 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1248757 | 04:45 |
| *** matsuhashi has joined #openstack-qa | 04:47 | |
| *** otherwiseguy has quit IRC | 04:47 | |
| rms_13 | Can someone please pick up this? https://review.openstack.org/#/c/78042/ | 04:47 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/78941 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 04:54 |
| *** jasondotstar has quit IRC | 04:57 | |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "get a server" API response attributes https://review.openstack.org/80175 | 04:58 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "get a server" API response attributes https://review.openstack.org/80175 | 05:03 |
| *** amotoki has joined #openstack-qa | 05:06 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: Rename all Savanna usages to Sahara https://review.openstack.org/79353 | 05:07 |
| openstackgerrit | A change was merged to openstack/tempest: Improve ServerCfnInitTestJSON asserts and debugging https://review.openstack.org/69241 | 05:07 |
| openstackgerrit | A change was merged to openstack/tempest: Fix test by waiting to lbaas entity delete https://review.openstack.org/74121 | 05:07 |
| openstackgerrit | A change was merged to openstack/tempest: Stop volume leaking test_server_rescue_negative https://review.openstack.org/78312 | 05:07 |
| *** rhsu has quit IRC | 05:14 | |
| *** GMann has quit IRC | 05:14 | |
| *** rhsu1 has joined #openstack-qa | 05:14 | |
| openstackrecheck | Console logs not available after 13:34s for gate-tempest-docs 77517,20,95eaf4b | 05:16 |
| *** rms_13 has quit IRC | 05:16 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/62771 failed gate-neutron-python26, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288312and https://bugs.launchpad.net/bugs/1273386 | 05:17 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80019 failed gate-tempest-dsvm-neutron, gate-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1273386 | 05:17 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/80103 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1249065 | 05:18 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/79504 failed gate-cinder-python26, check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1272417and https://bugs.launchpad.net/bugs/1248757 | 05:20 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/79791 failed gate-cinder-python27 because of: https://bugs.launchpad.net/bugs/1291108 | 05:21 |
| *** GMann has joined #openstack-qa | 05:23 | |
| *** rhsu1 has quit IRC | 05:28 | |
| *** sthaha has quit IRC | 05:30 | |
| *** yfried has quit IRC | 05:31 | |
| openstackgerrit | A change was merged to openstack/tempest: Add agents tests for Nova v3 API https://review.openstack.org/74611 | 05:36 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: JSON Schema validation for attach volume attribute https://review.openstack.org/79919 | 05:39 |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/77375 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254890and https://bugs.launchpad.net/bugs/1254872 | 05:42 |
| openstackgerrit | Chen Xiao proposed a change to openstack/tempest: Add cinder tempest cases for encryption-type https://review.openstack.org/72561 | 05:47 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79675 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1273386and https://bugs.launchpad.net/bugs/1249065and https://bugs.launchpad.net/bugs/1248757 | 05:48 |
| *** harlowja is now known as harlowja_away | 05:53 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80175 failed check-tempest-dsvm-neutron, gate-tempest-pep8, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885and | 06:03 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1248757 | 06:03 |
| *** zhikunliu has quit IRC | 06:05 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/77517 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1254890and | 06:06 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1257885and https://bugs.launchpad.net/bugs/1254872 | 06:06 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Verify "get az list" API response attributes https://review.openstack.org/77807 | 06:07 |
| *** zhikunliu has joined #openstack-qa | 06:07 | |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify service attributes through Nova os-flavor-access API https://review.openstack.org/80135 | 06:08 |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify service attributes through Nova os-flavor-access API https://review.openstack.org/80135 | 06:12 |
| openstackgerrit | Ann Kamyshnikova proposed a change to openstack/tempest: Verify more information in floating ip tests https://review.openstack.org/63723 | 06:15 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80136 failed gate-neutron-python26, gate-neutron-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1273386 | 06:16 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80177 failed gate-neutron-pep8, check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1254872 | 06:18 |
| openstackgerrit | Ann Kamyshnikova proposed a change to openstack/tempest: Verify more information in floating ip tests(part2) https://review.openstack.org/71251 | 06:19 |
| *** yfried has joined #openstack-qa | 06:23 | |
| *** nati_ueno has joined #openstack-qa | 06:24 | |
| *** lukego has joined #openstack-qa | 06:27 | |
| *** psharma has joined #openstack-qa | 06:33 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/79284 failed gate-tempest-dsvm-neutron, gate-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1248757 | 06:33 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Validate of Nova APIs Floating IPs attributes https://review.openstack.org/80186 | 06:35 |
| *** vponomaryov1 has joined #openstack-qa | 06:35 | |
| openstackrecheck | openstack/trove change: https://review.openstack.org/77588 failed gate-trove-python27, gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1265672and https://bugs.launchpad.net/bugs/1254890and https://bugs.launchpad.net/bugs/1254872 | 06:41 |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify service attributes through Nova os-quota-sets API https://review.openstack.org/80150 | 06:45 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79919 failed check-grenade-dsvm, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 06:46 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Stress action for volume attach verify https://review.openstack.org/77196 | 06:50 |
| *** ken1ohmichi has joined #openstack-qa | 06:53 | |
| cyeoh | ken1ohmichi: hi! | 06:56 |
| *** zhikunliu has quit IRC | 06:56 | |
| ken1ohmichi | hi cyeoh | 06:57 |
| *** zhikunliu has joined #openstack-qa | 06:57 | |
| ken1ohmichi | cyeoh: your idea is nice :-) | 06:57 |
| cyeoh | ken1ohmichi: cool - it only occurred to me it might be a better place after seeing that other patch to make the body optional | 06:58 |
| cyeoh | should mean a lot less code to modify too I think | 06:59 |
| ken1ohmichi | cyeoh: yes, right. and we need more discussion for doing it. | 06:59 |
| cyeoh | ken1ohmichi: would you like me to try doing a patch to modify the services test to see what it would look like? | 07:00 |
| ken1ohmichi | cyeoh: yes, of course:) | 07:01 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/63060 failed gate-cinder-python27, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1291108and https://bugs.launchpad.net/bugs/1254872 | 07:01 |
| cyeoh | ken1ohmichi: cool I'll give it a go... | 07:02 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/55987 failed check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1273386and https://bugs.launchpad.net/bugs/1265495 | 07:02 |
| ken1ohmichi | cyeoh: sorry, I should go out now. I forgot the office meeting. see you later | 07:03 |
| *** ken1ohmichi has quit IRC | 07:03 | |
| openstackrecheck | openstack/heat change: https://review.openstack.org/67638 failed gate-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 07:08 |
| *** chandan_kumar has joined #openstack-qa | 07:10 | |
| *** swlivesay has joined #openstack-qa | 07:11 | |
| *** swlivesay has left #openstack-qa | 07:11 | |
| *** vkozhukalov has quit IRC | 07:16 | |
| *** morganfainberg is now known as morganfainberg_Z | 07:18 | |
| openstackgerrit | Daisuke Morita proposed a change to openstack/tempest: Add parametric tests of Swift object API https://review.openstack.org/67072 | 07:22 |
| openstackgerrit | Chen Xiao proposed a change to openstack/tempest: Add test case for cinder snapshot https://review.openstack.org/65210 | 07:23 |
| raies | afazekas: hi | 07:26 |
| openstackgerrit | rendh proposed a change to openstack/tempest: Add more role,tenant to user and delete them https://review.openstack.org/72547 | 07:26 |
| openstackgerrit | Chen Xiao proposed a change to openstack/tempest: Add cinder tempest cases for encryption-type https://review.openstack.org/72561 | 07:26 |
| openstackgerrit | rendh proposed a change to openstack/tempest: Add more role,tenant to user and delete them https://review.openstack.org/72547 | 07:30 |
| *** matsuhashi has quit IRC | 07:32 | |
| *** matsuhashi has joined #openstack-qa | 07:32 | |
| *** nosnos has quit IRC | 07:32 | |
| *** nosnos_ has joined #openstack-qa | 07:32 | |
| *** jlibosva has joined #openstack-qa | 07:39 | |
| *** mspreitz has left #openstack-qa | 07:42 | |
| raies | afazekas: https://review.openstack.org/#/c/78857/ is done as per your comment | 07:42 |
| openstackgerrit | Marc Koderer proposed a change to openstack/tempest: Fix issue with pep8 gate job https://review.openstack.org/80201 | 07:47 |
| mkoderer | cyeoh: afazekas: sdague: mtreinish: we have currently no pep8 check in the gate. Can we merge https://review.openstack.org/#/c/80201/ before anything else | 07:50 |
| openstackgerrit | Christopher Yeoh proposed a change to openstack/tempest: Move verification of service attributes into service client https://review.openstack.org/80202 | 07:50 |
| mkoderer | no pep8 for tempest | 07:51 |
| afazekas | raies: isn't the response a list of dicts ? ,and you just need to find one list element by id, and do the asserts only on that row ? | 07:52 |
| afazekas | items = [ m for m in fw_rules if m['id'] == fw_rule['id'] ] | 07:54 |
| afazekas | assert len of items is 1 | 07:54 |
| afazekas | use the first items[0] for asserts | 07:55 |
| afazekas | mkoderer: So you need to suppress an otherwise valid exception ? | 07:59 |
| afazekas | LOG.critical ? | 07:59 |
| mkoderer | afazekas: that's correct, I don't have any other idea to fix it.. since we are on the import level | 08:04 |
| mkoderer | afazekas: I didn't wanted to confuse ppl looking at pep8 logs.. but you want me to change the level? | 08:05 |
| afazekas | I never liked the inspection feature of the flake8 | 08:05 |
| * afazekas once tried to kill it ,but it wasn't worked on OSX, and I can't reproduce the issue without OSX | 08:06 | |
| mkoderer | cyeoh: regarding your comment.. the scenario test lib doesn't support it currently | 08:07 |
| afazekas | mkoderer: I do not have any other quick fix idea, but I think log level should be increased | 08:07 |
| mkoderer | afazekas: all right I will increase it | 08:07 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79227 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error | 08:10 |
| *** saju_m has joined #openstack-qa | 08:12 | |
| openstackgerrit | Marc Koderer proposed a change to openstack/tempest: Fix issue with pep8 gate job https://review.openstack.org/80201 | 08:14 |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/77333 failed gate-keystone-python33, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254872 | 08:17 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Volume xml client translate json attribute names https://review.openstack.org/78345 | 08:21 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Detecting unexpectedly not deleted resources https://review.openstack.org/78251 | 08:23 |
| *** tkammer has joined #openstack-qa | 08:26 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/76379 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, gate-tempest-dsvm-neutron-large-ops, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: | 08:29 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288427and https://bugs.launchpad.net/bugs/1273386 | 08:29 |
| *** vponomaryov1 has left #openstack-qa | 08:29 | |
| *** andreaf has joined #openstack-qa | 08:30 | |
| afazekas | mkoderer: /me is waiting for jenkins | 08:34 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: unskip test_stamp_pattern https://review.openstack.org/52740 | 08:41 |
| *** jordanP has joined #openstack-qa | 08:41 | |
| openstackrecheck | openstack/swift change: https://review.openstack.org/79772 failed gate-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 08:42 |
| *** jpich has joined #openstack-qa | 08:45 | |
| *** jasondotstar has joined #openstack-qa | 09:02 | |
| *** matsuhashi has quit IRC | 09:07 | |
| *** safchain has joined #openstack-qa | 09:09 | |
| *** nosnos has joined #openstack-qa | 09:11 | |
| *** nosnos_ has quit IRC | 09:11 | |
| *** matsuhashi has joined #openstack-qa | 09:12 | |
| lukego | I am falling into a pit of dispair while trying to make devstack-vm-gate.sh do what I want. I have many copies of this file, which seem to be overwritten at unpredictable times, which makes it hard to experiment with changes. if anybody would like to help that would be very welcome :) | 09:14 |
| lukego | I'm following Jay's procedure and basically just trying to add a line or two to localrc. | 09:15 |
| *** giulivo has joined #openstack-qa | 09:16 | |
| *** nmagnezi has joined #openstack-qa | 09:17 | |
| *** nati_ueno has quit IRC | 09:21 | |
| *** rossella_s has joined #openstack-qa | 09:21 | |
| *** luqas has joined #openstack-qa | 09:25 | |
| *** yassine has joined #openstack-qa | 09:27 | |
| *** rossella_ has joined #openstack-qa | 09:28 | |
| *** yassine has quit IRC | 09:28 | |
| *** yassine has joined #openstack-qa | 09:28 | |
| *** rossella_s has quit IRC | 09:28 | |
| *** chandan_kumar has quit IRC | 09:30 | |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/76762 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1253896 | 09:31 |
| *** zhikunliu has quit IRC | 09:31 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/77807 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885and | 09:32 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1254872 | 09:32 |
| *** BobBall has joined #openstack-qa | 09:36 | |
| openstackrecheck | openstack/python-novaclient change: https://review.openstack.org/67074 failed check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 09:38 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/62108 failed gate-nova-python26, gate-nova-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1249065and https://bugs.launchpad.net/bugs/1248757 | 09:39 |
| raies | afazekas: hi during nova boot with scheduler hint can you suggest xml request body ?? | 09:41 |
| raies | afazekas: I have done with xml and tried too many ways for xml but no correct guess for xml his found | 09:41 |
| raies | afazekas: so please suggest xml request body | 09:42 |
| raies | afazekas: reference doc http://docs.openstack.org/grizzly/openstack-compute/admin/content/scheduler-filters.html | 09:42 |
| *** matsuhashi has quit IRC | 09:44 | |
| afazekas | raies: https://github.com/openstack/nova/blob/master/nova/tests/api/openstack/compute/test_servers.py#L3224 | 09:44 |
| *** chandan_kumar has joined #openstack-qa | 09:46 | |
| *** yrabl has joined #openstack-qa | 09:46 | |
| *** matsuhashi has joined #openstack-qa | 09:47 | |
| raies | afazekas: thanks I am trying on it | 09:47 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/77517 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: | 09:50 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1248757and https://bugs.launchpad.net/bugs/1257885and https://bugs.launchpad.net/bugs/1265495 | 09:50 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80186 failed check-tempest-dsvm-postgres-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885and https://bugs.launchpad.net/bugs/1254872 | 09:51 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/79969 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1248757 | 09:51 |
| openstackrecheck | openstack/glance change: https://review.openstack.org/80182 failed gate-glance-python26, gate-glance-python27, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 09:52 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80142 failed check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 10:00 |
| *** jasondotstar has quit IRC | 10:00 | |
| openstackrecheck | openstack/python-keystoneclient change: https://review.openstack.org/80193 failed check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1249065and https://bugs.launchpad.net/bugs/1254872 | 10:02 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/56184 failed check-grenade-dsvm, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, gate-tempest-dsvm-neutron-large-ops, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: | 10:03 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288427and https://bugs.launchpad.net/bugs/1273386 | 10:03 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67281 failed gate-neutron-python27, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1288312and https://bugs.launchpad.net/bugs/1273386 | 10:04 |
| BobBall | sdague: here? | 10:06 |
| *** luqas has quit IRC | 10:07 | |
| mkoderer | cyeoh: are you around? | 10:14 |
| cyeoh | mkoderer: hi | 10:14 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/65210 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 10:14 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/78488 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896 | 10:14 |
| mkoderer | cyeoh: did you had a look to lib scenario.. since I already tried to move it to setupClass | 10:15 |
| cyeoh | mkoderer: hrm maybe I missed it - I did a grep for generate_scenario and had a look at a few of those testcases | 10:16 |
| cyeoh | which one in particular was a problem? | 10:16 |
| mkoderer | cyeoh: the thing is that lib scenario registers test to the test loader | 10:17 |
| mkoderer | cyeoh: and this must be done while import time | 10:17 |
| *** yfujioka has quit IRC | 10:18 | |
| raies | afazekas: hi | 10:18 |
| cyeoh | mkoderer: where is the code for that one? | 10:18 |
| raies | afazekas: when json tried using - | 10:18 |
| raies | afazekas: curl -i 'http://<IP>:8774/v2/2023f70489c444f3a146ca9573954fa4/servers' -X POST -H "X-Auth-Project-Id: demo" -H "User-Agent: python-novaclient" -H "Content-Type: application/json" -H "Accept: application/json" -H "X-Auth-Token: <token> -d '{"os:scheduler_hints": {"build_near_host_ip": "192.168.1.1", "cidr": "/24"}, "server": {"min_count": 1, "flavorRef": "3", "name": "server-1", "imageRef": "eebc9c6d-182c-435f-80e4-9122 | 10:18 |
| raies | request is successful | 10:18 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79660 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1253896and https://bugs.launchpad.net/bugs/1280464 | 10:18 |
| raies | afazekas: but during xml I tried - | 10:18 |
| mkoderer | cyeoh: give me a sec | 10:19 |
| raies | afazekas: - | 10:19 |
| raies | "curl -i 'http://<IP>:8774/v2/2023f70489c444f3a146ca9573954fa4/servers' -X POST -H "X-Auth-Project-Id: demo" -H "User-Agent: python-novaclient" -H "Content-Type: application/xml" -H "Accept: application/xml" -H "X-Auth-Token: <token> -d '<server><flavorRef>42</flavorRef><name>ServersTestXML-instance-1231428186</name><imageRef>eebc9c6d-182c-435f-80e4-912201616507</imageRef>/><os:scheduler_hints <build_near_host_ip>127.0.0.1</buil | 10:19 |
| raies | afazekas: then 400 Bad request occurs | 10:19 |
| *** kmartin has quit IRC | 10:20 | |
| mkoderer | cyeoh: https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_flavors_negative.py#L24 | 10:20 |
| raies | please check and suggest me exact xml requets | 10:20 |
| *** GMann has quit IRC | 10:20 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80198 failed gate-nova-python26, gate-nova-python27, check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896 | 10:23 |
| mkoderer | cyeoh: https://pypi.python.org/pypi/testscenarios/ we are using load_tests which is a variable interpreted by the test loader | 10:23 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80202 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 10:24 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/76533 failed gate-neutron-python26, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288312 | 10:24 |
| cyeoh | mkoderer: ok and then testscenarios looks in each class for a scenarios attribute? | 10:25 |
| mkoderer | cyeoh: yep this is the reason we have to introduce a class for each schema | 10:26 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/59907 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1265672and https://bugs.launchpad.net/bugs/1257885 | 10:26 |
| mkoderer | cyeoh: I already start working on the lib to find a better way | 10:26 |
| mkoderer | cyeoh: but this isn't a quick fix ;) | 10:26 |
| cyeoh | mkoderer: ah ok I see now. Thanks! | 10:26 |
| cyeoh | yea getting pep8 working in the short term is important. | 10:27 |
| cyeoh | mkoderer: thanks for explaining it to me... | 10:27 |
| mkoderer | cyeoh: yep thanks for asking.. I really want to find the best possible way ;) | 10:27 |
| sdague | BobBall: am now | 10:28 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/74300 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 10:30 |
| BobBall | sdague: https://review.openstack.org/#/c/74112/ - you suggested having config passed into d-g; do you have a preferred way of doing this? | 10:30 |
| BobBall | simple way would just be to have known variable names (e.g. XENAPI_DOM0_IP) that are checked if the driver is xenapi | 10:30 |
| BobBall | probably with the DEVSTACK_GATE prefix, but that gets very long... DEVSTACK_GATE_XENAPI_DOM0_IP ;) | 10:31 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/49227 failed gate-neutron-pep8, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1248757 | 10:32 |
| sdague | BobBall: so those values will be true for any xenserver environment ? | 10:32 |
| raies | afazekas: you there ?? | 10:32 |
| BobBall | well, no - but d-g needs to run in a very specialised environment for XS since it must run as a domU under an existing XS host | 10:32 |
| *** mkollaro has joined #openstack-qa | 10:32 | |
| BobBall | it's all down to XS running openstack in a domU rather than KVM which runs it in the equivalent of dom0 | 10:33 |
| sdague | BobBall: devstack-gate is supposed to be more generic | 10:33 |
| sdague | that's my objection | 10:33 |
| afazekas | raies: xmlns:os="http://docs.openstack.org/compute/ext/scheduler-hints/api/v2 not in your xml request | 10:34 |
| sdague | because fungi has scripts that deploy d-g into random guests, for instance, and I'm working on vagrant using d-g | 10:34 |
| sdague | so you can't put site specific config in there | 10:34 |
| BobBall | Ah - it's not site-specific | 10:34 |
| afazekas | raies: invalid xml <os:scheduler_hints <build_near_host_ip> | 10:34 |
| BobBall | Each VM in RAX is running XenServer, with the fixed IP of 192.168.33.2 - each of those VMs then has a sub-VM which is where d-g runs | 10:35 |
| sdague | BobBall: so all xenserver implementations will always look like this | 10:35 |
| sdague | BobBall: that's site specific | 10:35 |
| BobBall | it's the way it's set up in the config repository, sure | 10:35 |
| afazekas | xml test site, this was the first from a google search: http://www.xmlvalidation.com/ | 10:36 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/devstack-gate: Temp enable savanna service for sahara tests. https://review.openstack.org/80237 | 10:36 |
| sdague | mkoderer: hold on in the pep8 thing please, I want to look at that | 10:36 |
| BobBall | sdague: http://git.openstack.org/cgit/openstack-infra/config/tree/modules/openstack_project/files/nodepool/scripts/convert_node_to_xenserver.sh#n487 | 10:36 |
| BobBall | Anyway - as I said - I'm very happy to have this configurable | 10:37 |
| BobBall | I was just asking if you had a preferred / recommended method | 10:37 |
| sdague | BobBall: so it feels like this probably needs to come out into a localrc or something | 10:37 |
| sdague | like a d-g localrc | 10:37 |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/77333 failed gate-keystone-python33, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254872 | 10:37 |
| sdague | and a big README about that whole setup in the d-g repo | 10:37 |
| raies | afazekas: One more thing I have noticed during json create I checked response - | 10:37 |
| BobBall | but it's all driven on env vars currently | 10:38 |
| *** yfried has quit IRC | 10:38 | |
| mkoderer | sdague: jenkins is nearly done | 10:38 |
| BobBall | the whole of d-g I mean, isn't it? | 10:38 |
| SergeyLukjanov | sdague, we've landed devstack integration renaming for sahara | 10:38 |
| mkoderer | sdague: yes please have a look maybe you have a good idea ;) | 10:38 |
| sdague | BobBall: yes | 10:38 |
| SergeyLukjanov | sdague, yesterday and missed service enable for savanna => tempest think that savanna isn't available | 10:39 |
| sdague | ok, but, right now I need to in order, look at this pep8 thing, because that's blocking tempest, then help SergeyLukjanov land tests :) | 10:39 |
| BobBall | so would you be happy with DEVSTACK_GATE_XENAPI_DOM0_IP/DOMU_IP/PASSWORD ? | 10:39 |
| *** yfried has joined #openstack-qa | 10:39 | |
| sdague | BobBall: sure | 10:39 |
| sdague | yes, that would be better | 10:39 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/76379 failed check-grenade-dsvm, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, gate-tempest-dsvm-neutron-large-ops, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: | 10:39 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1280464and https://bugs.launchpad.net/bugs/1288427 | 10:40 |
| raies | afazekas: When I tried dusing json request body with one json accept header and once xml request header | 10:40 |
| BobBall | OK. I will work on that. | 10:40 |
| sdague | mostly we shouldn't have ip addresses hard coded in both nodepool and d-g | 10:40 |
| raies | afazekas: http://www.fpaste.org/84962/70714613/ here you can check output | 10:40 |
| sdague | they should come from some common place | 10:40 |
| SergeyLukjanov | sdague, please, take a look on https://review.openstack.org/80237 when have some time | 10:40 |
| sdague | mkoderer: how did the pep8 issue land? | 10:40 |
| BobBall | Agreed - but that can be passed to d-g through the env-var, so I think this is a step in the right direction. | 10:40 |
| raies | afazekas: I think in response "OS-SCH-HNT:scheduler_hints" this key should be available but it is not seen in output | 10:41 |
| raies | afazekas: while output response is 202 and OK | 10:41 |
| mkoderer | sdague: pep8 simply not run with an exception but in jenkins it was flagged as success | 10:41 |
| raies | afazekas: wht you say on this | 10:41 |
| sdague | mkoderer: explain | 10:41 |
| mkoderer | sdague: I am searching an example | 10:41 |
| SergeyLukjanov | sdague, https://review.openstack.org/80237 is needed because of https://github.com/openstack-dev/devstack/blob/master/lib/tempest#L359; to rename savanna in tempest tests we need to release new client version with sahara cli | 10:41 |
| mkoderer | sdague: https://review.openstack.org/#/c/79256/ http://logs.openstack.org/56/79256/1/gate/gate-tempest-pep8/088cc12/console.html | 10:42 |
| afazekas | raies: the key names on XML are different and usually prefixed with ns | 10:42 |
| mkoderer | pep8 fails but it's still a success | 10:42 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/78853 failed gate-neutron-python26, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 10:43 |
| mkoderer | sdague: I didn't fix this thingee but I wanted to fix the issue why it occours | 10:43 |
| mkoderer | sdague: mtreinish found that last night and ask me to fix it | 10:43 |
| afazekas | raies: probably you need to add something here: https://github.com/openstack/tempest/blob/master/tempest/services/compute/xml/servers_client.py#L125 | 10:43 |
| *** luqas has joined #openstack-qa | 10:44 | |
| mkoderer | sdague: seems that the pep8 job does has a wrong result code if something went wrong | 10:45 |
| sdague | mkoderer: ok, so we need to figure that out | 10:46 |
| mkoderer | sdague: sure | 10:46 |
| raies | afazekas: as per http://docs.openstack.org/api/openstack-compute/2/content/POST_os-scheduler-hints-v2_createServer_v2__tenant_id__servers_ext-os-scheduler-hints.html you can see that I think "OS-SCH-HNT:scheduler_hints" is key not related to diskconfig | 10:46 |
| sdague | mkoderer: but we are doing this lazy loading for a reason | 10:47 |
| sdague | and I don't want to start putting that exception back in there | 10:47 |
| sdague | mkoderer: so can you actually work on the tox.ini change that will fail us correctly here | 10:48 |
| afazekas | nraies: ot related to diskconfig, I just pointed there because that part contains xml "key" replacing. Are you sure you do not get any "key" or element which contains the same information ? Can you show us the full xml response ? | 10:49 |
| mkoderer | sdague: I can have a look but anyway it's not easy to find a solution for this particular issue | 10:49 |
| sdague | and I'll try to sort out getting around the pep8 | 10:49 |
| openstackgerrit | Bob Ball proposed a change to openstack-infra/devstack-gate: Support XenServer in the gate https://review.openstack.org/80241 | 10:50 |
| SergeyLukjanov | sdague, thank you! | 10:50 |
| mkoderer | sdague: so jenkins gave it's +1 https://review.openstack.org/#/c/80201/ .. should we merge it to have a quick fix? | 10:51 |
| sdague | mkoderer: no, I don't want to do that | 10:51 |
| mkoderer | sdague: just comment it... I will have a look to the tox stuff | 10:52 |
| sdague | because we went through all this effort to get lazy loading conf to get us out of these kinds of situations | 10:52 |
| *** mwagner_lap has quit IRC | 10:52 | |
| sdague | mkoderer: so walk me through this | 10:52 |
| raies | afazekas: First of all I dont know xml request body, I tried with json request body | 10:52 |
| sdague | how testscenarios works | 10:53 |
| mkoderer | sdague: all right | 10:53 |
| sdague | I see we have a magically load_tests in test_flavors_negative.py | 10:53 |
| mkoderer | sdague: yep we have this on all negative tests working we the framework | 10:54 |
| *** tosky has joined #openstack-qa | 10:54 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/78251 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-nv, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm- | 10:54 |
| openstackrecheck | sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1285833and https://bugs.launchpad.net/bugs/1257885 | 10:54 |
| raies | afazekas: when I tried using json request body with Accept header as JSON | 10:54 |
| raies | afazekas: http://www.fpaste.org/84965/39470804/ | 10:54 |
| mkoderer | sdague: test scenario expects the scenario variable in place on module import AFAIK | 10:55 |
| sdague | mkoderer: where does that get called? | 10:55 |
| sdague | or asked for | 10:55 |
| mkoderer | sdague: https://pypi.python.org/pypi/testscenarios/ | 10:55 |
| mkoderer | sdague: search for load_tests | 10:55 |
| raies | afazekas: when I tried using xml request body http://www.fpaste.org/84968/39470814/ | 10:56 |
| *** mkollaro has quit IRC | 10:58 | |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Object storage tests to use default auth_provider https://review.openstack.org/80244 | 10:59 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Configuration for domain name https://review.openstack.org/80245 | 10:59 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Get credential IDs from Credentials class https://review.openstack.org/80246 | 10:59 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Enforces the use of Credentials https://review.openstack.org/77594 | 10:59 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Defines a Credentials class https://review.openstack.org/74387 | 10:59 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/76567 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885and https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 10:59 |
| sdague | mkoderer: and all that load_tests can only run on import? | 10:59 |
| openstackrecheck | openstack/python-glanceclient change: https://review.openstack.org/67383 failed gate-python-glanceclient-pep8, check-tempest-dsvm-full, gate-python-glanceclient-python33 because of: https://bugs.launchpad.net/bugs/1254872 | 11:00 |
| raies | afazekas: have you seen ?? | 11:00 |
| mkoderer | sdague: I already tried to move it to setUpClass but this didn't work | 11:00 |
| openstackrecheck | openstack/python-heatclient change: https://review.openstack.org/80034 failed check-tempest-dsvm-full, gate-python-heatclient-python33 because of: https://bugs.launchpad.net/bugs/1254890 | 11:01 |
| raies | during sever output scheduler hint should be present I think and here it is not | 11:01 |
| raies | afazekas: so I think it is problem | 11:01 |
| afazekas | I do not see the </server> | 11:01 |
| sdague | I wonder if lifeless is still awake | 11:01 |
| mkoderer | sdague: other idea.. I can disable the config variable in the meanwhile since currently there is no other implementation for other negative generators | 11:01 |
| cyeoh | sdague: so I'd guess it has to happen at import time because the testloader relies on it | 11:02 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/52740 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885and | 11:02 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1254872and https://bugs.launchpad.net/bugs/1248757 | 11:02 |
| sdague | cyeoh: yeh, that's just a little sucky, but such it is | 11:02 |
| cyeoh | yep :-( | 11:02 |
| sdague | so if we put an actual pep8 error in there, this fails correctly | 11:04 |
| mkoderer | sdague: flake8 doesn't change it's exit code in case such an error | 11:04 |
| sdague | mkoderer: it does on a real pep8 error | 11:04 |
| sdague | I just did it | 11:04 |
| sdague | afazekas: do you still have your alternate implement for module detection for hacking? | 11:05 |
| mkoderer | sdague: yes sure but not for this exception | 11:06 |
| sdague | that doesn't do the import | 11:06 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Network fwaas API test https://review.openstack.org/78857 | 11:06 |
| sdague | mkoderer: sure | 11:06 |
| openstackgerrit | jun xie proposed a change to openstack/tempest: Test for neutron api with provider extension https://review.openstack.org/79864 | 11:06 |
| raies | afazekas: no no </server> is also present | 11:06 |
| raies | I missed in copy paste | 11:06 |
| afazekas | sdague: It was merged once, but it was changed after the OSX incompatibility issue reported | 11:06 |
| raies | afazekas: | 11:06 |
| raies | afazekas: I have basically two problems 1. I dont know exact xml REQUEST body | 11:07 |
| sdague | afazekas: so the osx incompatibility just meant it didn't detect the error there, right? | 11:07 |
| sdague | or did it explode in a funny way | 11:07 |
| mkoderer | sdague: so the pep8 check is working in general but not for all negative tests using the framework | 11:07 |
| raies | afazekas: 2. During json REQUEST body expected 'OS-SCH-HNT:scheduler_hints' key is not seen in response | 11:08 |
| afazekas | sdague: I can't remember, I think it failed because one of the 'imp' mothods returned with an unexpected format | 11:08 |
| sdague | afazekas: can you did up the commit? | 11:08 |
| raies | afazekas: So I think you are mixing both two problems | 11:08 |
| sdague | I'd like to actually make us stop importing modules in that check, because it causes tons of other problems | 11:09 |
| sdague | not just this one | 11:09 |
| sdague | mkoderer: ok, +A | 11:09 |
| *** mkollaro has joined #openstack-qa | 11:09 | |
| mkoderer | sdague: I found something else.... | 11:10 |
| mkoderer | sdague: http://paste.openstack.org/show/73367/ | 11:11 |
| *** matsuhashi has quit IRC | 11:11 | |
| mkoderer | sdague: it doesn't occur in the negative files it self.. it's due to the facte that we use inheritance in /tempest/api/compute/admin/test_flavors_negative_xml.py | 11:12 |
| afazekas | sdague: nova 5dd5625d976603bcba8152c0476290d0aa81f5d7 the last version I remember to work, some special rare import style like frozen .. etc, is not considered | 11:12 |
| *** matsuhashi has joined #openstack-qa | 11:12 | |
| sdague | mkoderer: ok, so that should actually get fixed | 11:14 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Network Firewall API https://review.openstack.org/79868 | 11:14 |
| mkoderer | sdague: I will do so.. | 11:14 |
| sdague | mkoderer: you may find you don't need the except there | 11:15 |
| mkoderer | sdague: I will have a look and if it's not needed I simply remove it again | 11:15 |
| sdague | afazekas: thanks | 11:15 |
| mkoderer | sdague: afazekas: cyeoh: thanks for helping me with that | 11:16 |
| sdague | yep | 11:16 |
| *** dmorita has quit IRC | 11:19 | |
| afazekas | sdague: http://fpaste.org/84971/47095351/ this style if import was on exception list (nova did it with the logging), but it would be detectable just be parsing the from/import statements | 11:19 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Network Firewall API https://review.openstack.org/79868 | 11:20 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79826 failed gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254890 | 11:21 |
| *** jasondotstar has joined #openstack-qa | 11:22 | |
| *** jamielennox is now known as jamielennox|away | 11:22 | |
| BobBall | sdague: For your consideration at some point: https://review.openstack.org/#/c/80241/ | 11:23 |
| *** yfried has quit IRC | 11:25 | |
| *** yfried has joined #openstack-qa | 11:26 | |
| *** Madan has joined #openstack-qa | 11:27 | |
| openstackrecheck | openstack/python-openstackclient change: https://review.openstack.org/79040 failed check-tempest-dsvm-full, gate-python-openstackclient-python33 because of: https://bugs.launchpad.net/bugs/1273908 | 11:28 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/79628 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 11:29 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/73234 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, gate-tempest-dsvm-neutron-large-ops, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: | 11:32 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1280464 | 11:32 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/67638 failed check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1258848and https://bugs.launchpad.net/bugs/1254872 | 11:32 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: only care about unclassifieds in gate queue https://review.openstack.org/80066 | 11:32 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: fix formatting on 'and' https://review.openstack.org/80083 | 11:32 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/80225 failed gate-ceilometer-python26, gate-ceilometer-python27 because of: https://bugs.launchpad.net/bugs/1258682 | 11:33 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint for bug 1291694 https://review.openstack.org/80129 | 11:33 |
| *** openstackrecheck has quit IRC | 11:34 | |
| *** openstackrecheck has joined #openstack-qa | 11:35 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/79901 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 | 11:36 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67382 failed gate-neutron-python27, check-grenade-dsvm, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1270382 and https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1288312 | 11:39 |
| *** jgrimm has joined #openstack-qa | 11:39 | |
| afazekas | raies: if it really messing from the json response after a proper request, you should create a bug | 11:40 |
| afazekas | But looks like it passes the on the nova unittests: https://github.com/openstack/nova/blob/master/nova/tests/api/openstack/compute/test_servers.py#L3246 | 11:41 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/67164 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-nv, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm- | 11:49 |
| openstackrecheck | sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 and https://bugs.launchpad.net/bugs/1248757 | 11:50 |
| *** _sweston_ has joined #openstack-qa | 11:50 | |
| *** tosky has quit IRC | 11:50 | |
| openstackrecheck | openstack/heat change: https://review.openstack.org/71512 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1251521 | 11:50 |
| openstackgerrit | Pranali Deore proposed a change to openstack/tempest: Modified test case for nova keypair https://review.openstack.org/79551 | 11:51 |
| *** Akshik has joined #openstack-qa | 11:51 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova unit test bug 1291620 https://review.openstack.org/80091 | 11:52 |
| Akshik | Hi,I have a basic question i have a 3 node setup [controller, compute, and network]. what i want to know is should the nova.conf same across controller and compute | 11:52 |
| *** sweston has quit IRC | 11:52 | |
| openstackgerrit | Pranali Deore proposed a change to openstack/tempest: Modified test case for nova security group https://review.openstack.org/79551 | 11:54 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79418 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 11:54 |
| *** openstackrecheck has quit IRC | 11:55 | |
| *** dkranz has joined #openstack-qa | 11:55 | |
| *** openstackrecheck has joined #openstack-qa | 11:55 | |
| openstackgerrit | Pranali Deore proposed a change to openstack/tempest: Modified test case for nova keypair https://review.openstack.org/79549 | 11:57 |
| *** dkranz has quit IRC | 11:57 | |
| *** dkranz has joined #openstack-qa | 11:58 | |
| *** weshay has joined #openstack-qa | 12:03 | |
| *** tosky has joined #openstack-qa | 12:04 | |
| *** matsuhashi has quit IRC | 12:11 | |
| *** matsuhashi has joined #openstack-qa | 12:12 | |
| openstackrecheck | Console logs not available after 13:24s for gate-neutron-docs 79285,5,92c36af | 12:12 |
| *** nosnos has quit IRC | 12:15 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a footprint for Bug 1291957 https://review.openstack.org/80262 | 12:15 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67490 failed gate-neutron-pep8, gate-neutron-python26, gate-neutron-python27, check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1253896 and | 12:15 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 12:15 |
| *** sthaha has joined #openstack-qa | 12:16 | |
| *** sthaha has joined #openstack-qa | 12:16 | |
| *** matsuhashi has quit IRC | 12:16 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79151 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 and https://bugs.launchpad.net/bugs/1248757 | 12:16 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/80243 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 12:17 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/77381 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 and https://bugs.launchpad.net/bugs/1254872 | 12:18 |
| openstackrecheck | openstack/swift change: https://review.openstack.org/80095 failed gate-tempest-dsvm-neutron in the gate queue with an unrecognized error | 12:18 |
| afazekas | raies: looks like it is a bug.. | 12:24 |
| *** aysyd has joined #openstack-qa | 12:24 | |
| *** matsuhashi has joined #openstack-qa | 12:26 | |
| *** luqas has quit IRC | 12:27 | |
| *** dims_ has quit IRC | 12:30 | |
| *** dims_ has joined #openstack-qa | 12:33 | |
| *** Akshik has quit IRC | 12:37 | |
| *** Akshik has joined #openstack-qa | 12:37 | |
| *** GMann has joined #openstack-qa | 12:38 | |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Account failure to the tearDown insted of setUp on shared https://review.openstack.org/54318 | 12:38 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80245 failed check-tempest-dsvm-postgres-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 12:41 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/77594 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-nv, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm- | 12:42 |
| openstackrecheck | sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 12:42 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/78857 failed gate-tempest-dsvm-large-ops, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1270382 and https://bugs.launchpad.net/bugs/1257885 | 12:43 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/71245 failed gate-cinder-python26 because of: https://bugs.launchpad.net/bugs/1291108 | 12:44 |
| *** andreaf has quit IRC | 12:44 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/62038 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 12:44 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80244 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1253896 and https://bugs.launchpad.net/bugs/1254890 and | 12:45 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1257885 and https://bugs.launchpad.net/bugs/1254872 | 12:45 |
| *** jlibosva has quit IRC | 12:46 | |
| *** jlibosva has joined #openstack-qa | 12:46 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79864 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1257885 | 12:50 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80001 failed check-grenade-dsvm, check-grenade-dsvm-partial-ncpu, check-devstack-dsvm-neutron, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1272417 and https://bugs.launchpad.net/bugs/1257885 | 12:51 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79971 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 12:54 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/79388 failed gate-cinder-python26 because of: https://bugs.launchpad.net/bugs/1291108 | 13:00 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/75253 failed gate-neutron-python27 because of: https://bugs.launchpad.net/bugs/1288312 | 13:01 |
| *** GMann has quit IRC | 13:03 | |
| *** dkranz has quit IRC | 13:07 | |
| *** adalbas has joined #openstack-qa | 13:09 | |
| *** rfolco has joined #openstack-qa | 13:09 | |
| *** mriedem has joined #openstack-qa | 13:12 | |
| openstackrecheck | openstack/python-ceilometerclient change: https://review.openstack.org/80256 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1291694 and https://bugs.launchpad.net/bugs/1248757 | 13:13 |
| *** psharma has quit IRC | 13:16 | |
| *** dkranz has joined #openstack-qa | 13:21 | |
| openstackgerrit | Elena Ezhova proposed a change to openstack/tempest: Add tests for binding extended attributes for ports https://review.openstack.org/64271 | 13:24 |
| mriedem | mtreinish: this passes jenkins now https://review.openstack.org/#/c/79958/ | 13:27 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80261 failed gate-neutron-pep8, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 13:28 |
| *** otherwiseguy has joined #openstack-qa | 13:30 | |
| fungi | afazekas: mriedem: have either of you noticed anything which could potentially account for the 10-hour backlog in the gate currently? new bug maybe? i poked around in the e-r page but nothing really jumped out at me | 13:33 |
| mriedem | fungi: just got in so haven't heard anything | 13:33 |
| mriedem | dansmith: ^? | 13:33 |
| fungi | same for me, so coming at this fresh | 13:33 |
| mriedem | sdague: ^? | 13:33 |
| openstackrecheck | openstack/python-keystoneclient change: https://review.openstack.org/79474 failed gate-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1291694 | 13:34 |
| sdague | fungi: I sent an email to -dev on it | 13:34 |
| sdague | I think the dramatic uptick in 1248757 is a big part of this problem | 13:35 |
| fungi | sdague: aha! my eye jumped right to the neutron testing message and i overlooked yours | 13:35 |
| sdague | also, if you look down all the network related tests, they all jumped about the same time | 13:35 |
| sdague | test_add_list_remove_router_on_l3_agent is unstable | 13:36 |
| sdague | Attempts to verify guests are running via SSH fails. SSH connection to guest does not work. | 13:36 |
| sdague | they all show upticks on Tuesday | 13:36 |
| sdague | http://status.openstack.org/elastic-recheck/gate.html | 13:36 |
| fungi | ick | 13:37 |
| fungi | that suggests some second-degree interaction | 13:37 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Wait properly for server deletion in test_volumes_actions https://review.openstack.org/80280 | 13:37 |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a footprint for Bug 1291992 https://review.openstack.org/80281 | 13:38 |
| *** jasondotstar has quit IRC | 13:38 | |
| sdague | anyway, I'm actually not able to chase this one, as I have some "actual work" I've been punting all week that I need to dive into. | 13:39 |
| sdague | sadly enough | 13:39 |
| sdague | SergeyLukjanov: one thing, could you adjust the fingerprint on the django bug to not match sahara runs? | 13:40 |
| sdague | the spike there is a false one, and it would be good to drop it | 13:40 |
| SergeyLukjanov | sdague, yup, I'm now working on it | 13:40 |
| sdague | SergeyLukjanov: thanks | 13:41 |
| SergeyLukjanov | sdague, will it rebuild stats in e-r? | 13:41 |
| sdague | yeh, that runs on cron hourly | 13:41 |
| SergeyLukjanov | sdague, ok | 13:41 |
| SergeyLukjanov | sdague, sorry about that | 13:41 |
| SergeyLukjanov | sdague, we've missed dj16 usage in horizon while starting renaming savanna | 13:42 |
| sdague | no worries, it's kind of a bad fingerprint because we aren't indexing the horizon console log | 13:42 |
| sdague | ok, no worries | 13:42 |
| sdague | SergeyLukjanov: oh, is it actually an issue? | 13:42 |
| sdague | I just assumed it was a bad fingerprint | 13:42 |
| SergeyLukjanov | sdague, yup, it was an issue in savanna-dashboard | 13:42 |
| *** matsuhashi has quit IRC | 13:42 | |
| SergeyLukjanov | sdague, incompatibility with dj16 | 13:42 |
| SergeyLukjanov | sdague, but it's in sahara-dashnoard, not in the horizon | 13:43 |
| sdague | ok | 13:43 |
| sdague | yeh, so maybe if you could separate the issues | 13:43 |
| sdague | somehow, that would be great | 13:44 |
| SergeyLukjanov | sdague, yup, I'm now looking into logs, it's easy to separate by horizon screen logs | 13:44 |
| SergeyLukjanov | sdague, but as you said - we're not indexing it ;) | 13:44 |
| *** YorikSar_ has quit IRC | 13:45 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67382 failed gate-neutron-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1288312 and https://bugs.launchpad.net/bugs/1248757 | 13:46 |
| sdague | SergeyLukjanov: well fixing the indexing problem would be good :) | 13:46 |
| *** dkranz has quit IRC | 13:47 | |
| SergeyLukjanov | sdague, ok, looking | 13:47 |
| *** ekhugen has joined #openstack-qa | 13:48 | |
| SergeyLukjanov | sdague, oh, it's not indexing due to the non-standart timestamp format? | 13:49 |
| *** YorikSar has joined #openstack-qa | 13:49 | |
| *** dkranz has joined #openstack-qa | 13:49 | |
| sdague | SergeyLukjanov: yeh, you'd need to build that into the index | 13:50 |
| sdague | Or figure out if we have a ts format for that already | 13:50 |
| mriedem | sdague: fungi: just read scrollback, we've been failing interanlly on test_add_list_remove_router_on_l3_agent for the last 2 days also | 13:50 |
| sdague | mriedem: right, I think that is indicative of the fact that something regressed in neutron | 13:50 |
| sdague | all the network related tests started failing more | 13:51 |
| SergeyLukjanov | sdague, heh, I'm quite unfamiliar how is the indexing working... will take a look on logs | 13:52 |
| *** Madan has quit IRC | 13:56 | |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/devstack: change the vmdk to the one used in VMWare ci https://review.openstack.org/80101 | 13:57 |
| dansmith | sdague: check-tempest-dsvm-neutron is parallel? | 13:57 |
| sdague | yes | 13:57 |
| dansmith | has it changed in number of threads recently ? | 13:58 |
| sdague | nope | 13:58 |
| sdague | that's controlled by d-g | 13:58 |
| sdague | by same algorithm as all the jobs | 13:58 |
| dansmith | so, the fail on that thing you raised is nova calling to neutron and getting no connection, | 13:59 |
| dansmith | meaning connection refused I imagine | 13:59 |
| dansmith | but it might be a timeout | 13:59 |
| dansmith | almost seems like neutron either never starts, or falls over at some point | 13:59 |
| dansmith | oh, extra debug: it's a timeout | 13:59 |
| *** malini_afk is now known as malini | 14:00 | |
| dansmith | yeah, seems to work, then timeout, then work for a while and then timeout | 14:00 |
| dansmith | so I don't think this is a race, and I don't think it's related to nova-neutron interaction, but looks more like just something changed on the neutron side that is causing us to get low-level connection failures, | 14:01 |
| dansmith | like maybe its API is overloaded or something | 14:01 |
| dansmith | or blocked | 14:01 |
| sdague | dansmith: could be, hence why I raised it with neutron and nova tag, because I expect it's going to take a bunch of deep dive into it | 14:01 |
| dansmith | yeah, just providing some analysis | 14:02 |
| sdague | yep, definitely, if you could make sure to put that in the bug, that would be very cool | 14:02 |
| dansmith | yeah | 14:02 |
| *** beisner has quit IRC | 14:02 | |
| sdague | because I'm not sure how much I'll be able to chase people on this one today, as I've put off a bunch of things I was supposed to work on earlier in the week, and need to make some progress on them :) | 14:03 |
| sdague | appreciate you jumping in here | 14:03 |
| dansmith | yeah, not asking you to, but the neutron channel seems dead right now, | 14:03 |
| dansmith | so wanted to say things where people would hear them | 14:03 |
| sdague | yeh, salv-orlando is usually awake right now, but I agree, it looked pretty dead this morning | 14:04 |
| *** Akshik has quit IRC | 14:04 | |
| *** YorikSar has quit IRC | 14:04 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a footprint for bug1283522 https://review.openstack.org/80286 | 14:05 |
| *** YorikSar has joined #openstack-qa | 14:05 | |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/elastic-recheck: Improve fingerprint for Django 1.6 not supported issue https://review.openstack.org/80287 | 14:05 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack-infra/elastic-recheck: Add fingerprint for bug 1291953 https://review.openstack.org/80288 | 14:05 |
| SergeyLukjanov | sdague, ^^ | 14:05 |
| SergeyLukjanov | savanna/sahara-dashboard is installed only in savanna/sahara jobs, so, let's ignore them | 14:06 |
| salv-orlando | dansmith: we have a bug open against lock wait timeouts. | 14:07 |
| *** yfried has quit IRC | 14:07 | |
| dansmith | salv-orlando: okay, so you know what this is? or should I still post my discussion above? | 14:07 |
| salv-orlando | We were initially looking at it only for the full job, which is not yet voting, but it seems these failures have crept in the gate checks and then are probably now affecting the smoke job as well. | 14:07 |
| salv-orlando | dansmith: posting that discussion (which I am reading now) will be probably useful anyway | 14:08 |
| dansmith | salv-orlando: okie | 14:08 |
| salv-orlando | dansmit: what you see from nova is always just "unable to connect to neutron. maximum attempts reached" or something like that. is that what you seE? | 14:09 |
| *** amotoki has quit IRC | 14:09 | |
| dansmith | salv-orlando: when it fails, yes, but it succeeds just a little bit later, and then fails again | 14:11 |
| dansmith | salv-orlando: http://logs.openstack.org/28/79628/4/check/check-tempest-dsvm-neutron/11f8293/logs/screen-n-cpu.txt.gz#_2014-03-13_10_55_41_409 | 14:11 |
| salv-orlando | yeah perhaps 50 seconds later? | 14:11 |
| dansmith | salv-orlando: less than that | 14:12 |
| dansmith | salv-orlando: 35 seconds in the case I'm looking at | 14:12 |
| salv-orlando | dansmith; yes it makes sense, the 50 second timeout is on the neutron side, you should not see it reflected on the nova side. I'm checking the neutron side of the logs you posted | 14:13 |
| *** adalbas has quit IRC | 14:13 | |
| dansmith | salv-orlando: the client tries several times, so that would cover up part of the 50 seconds | 14:13 |
| salv-orlando | dansmith: so I have a lock wait timeout 25 seconds after the nova failure, and is for an operation which was not initiated by nova (requested by an agent) | 14:15 |
| dansmith | salv-orlando: so nova timed out waiting for the log that the agent was holding? | 14:15 |
| salv-orlando | what is happening here is that because how eventlet and sqlalchemy locks work, both transactions (the one initiated from nova and the one from the agent stall) | 14:16 |
| dansmith | yeah | 14:16 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80276 failed gate-neutron-python27, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1288312 | 14:16 |
| salv-orlando | they both stall. When the one initiated from the agent times out, it relinquishes control to the green thread running the nova transaction | 14:16 |
| salv-orlando | which succeeds but then it's too late because nova already timed out | 14:17 |
| dansmith | they stall because of a lock ordering problem? | 14:17 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: remove duplicate bug https://review.openstack.org/80295 | 14:17 |
| *** markmcclain has joined #openstack-qa | 14:17 | |
| *** markmcclain has quit IRC | 14:17 | |
| salv-orlando | they stall because they operate on the same object, the same port resource in this case. In theory this is a normal situation, mysql is able to serialise transactions. However, if you put eventlet in the picture, things can go crazy. | 14:18 |
| salv-orlando | dansmith: nova had this issue too, and was solved by introducing API objects, I think | 14:19 |
| dansmith | salv-orlando: hmm, not sure about that :) | 14:19 |
| *** markmcclain has joined #openstack-qa | 14:19 | |
| mriedem | salv-orlando: is this related? https://review.openstack.org/79443 | 14:20 |
| dansmith | salv-orlando: is this a lock on a port_id or what? | 14:20 |
| mriedem | salv-orlando: or this https://review.openstack.org/78399 | 14:20 |
| *** luqas has joined #openstack-qa | 14:20 | |
| salv-orlando | mriedem: 79443 that was for an issue causing a traceback because a sub transaction raised an exception; unrelated I think | 14:20 |
| openstackgerrit | Ivan-Zhu proposed a change to openstack/tempest: share test_servers between v2/v3 api tests https://review.openstack.org/80297 | 14:20 |
| salv-orlando | 78399 not sure… I think that was for l3 agent interaction. I don't think that's for lock wait problems. | 14:21 |
| *** saju_m has quit IRC | 14:22 | |
| salv-orlando | dansmith: basically from what I read nova updates a port, and concurrently the agent tries to update the same port, and they both start a transaction involving updating a given port_id. So far there is nothing strange in it | 14:22 |
| dansmith | salv-orlando: right, that should be fine | 14:23 |
| salv-orlando | dansmith: note that I've seen this happening with both PUT and DELETE requests from nova,. but I guess this is just a detail | 14:23 |
| *** shwetaap has joined #openstack-qa | 14:23 | |
| afazekas | Looks like Neutron reschedules the router to the same (we have only one l3-agent) after delete | 14:24 |
| dansmith | salv-orlando: the agent isn't going through the external neutron API is it? | 14:24 |
| dansmith | where is the lock that they both hold? | 14:24 |
| dansmith | *eventlet lock, that is | 14:24 |
| salv-orlando | yes that's an eventlet lock | 14:24 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80275 failed gate-nova-pep8, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1253896 and https://bugs.launchpad.net/bugs/1265495 | 14:24 |
| salv-orlando | from SQL perspective it's all fine. | 14:24 |
| *** adalbas has joined #openstack-qa | 14:25 | |
| dansmith | salv-orlando: the agent and the API are on different nodes though, right? | 14:25 |
| *** jasondotstar has joined #openstack-qa | 14:25 | |
| dansmith | or at least different services, I guess they're on the same node in devstack.. | 14:25 |
| salv-orlando | dansmith: yes they are (same node but different processes on the gate). However the agent calls a RPC on the neutron server | 14:25 |
| mriedem | in tempest it'd be single node wouldn't it? | 14:25 |
| mkoderer | jogo: ping | 14:26 |
| salv-orlando | dansmith: I said this happened in nova a while ago because of this - http://eavesdrop.openstack.org/meetings/db/2013/db.2013-02-07-22.02.log.html | 14:26 |
| dansmith | salv-orlando: so they synchronize at the backend? | 14:26 |
| salv-orlando | dansmit: not sure what you mean here | 14:26 |
| salv-orlando | dansmith: ^ ^ | 14:26 |
| dansmith | salv-orlando: that was an eventlet threadpooling bug | 14:26 |
| dansmith | salv-orlando: not related to locks in any way I don't think | 14:27 |
| salv-orlando | dansmith: I definetely agree | 14:27 |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a footprint for bug 1283522 https://review.openstack.org/80286 | 14:27 |
| openstackgerrit | Elena Ezhova proposed a change to openstack/tempest: Add api tests for load balancer's VIPs and health monitors https://review.openstack.org/66796 | 14:27 |
| dansmith | salv-orlando: something about being able to trigger a deadlock in the mysql driver when eventlet thread pooling | 14:27 |
| dansmith | salv-orlando: is that what you think is happening here? | 14:27 |
| salv-orlando | it manifests as a SQL lock wait timeout error, but it's an eventlet deadlock | 14:27 |
| dansmith | salv-orlando: the way we "fix" that is just not using threadpooling | 14:27 |
| dansmith | we have lots of database worker processes instead | 14:28 |
| salv-orlando | dansmith: we were talking about doing something similar as well as separating the processes handling API and RPC requests. | 14:28 |
| * salv-orlando actually API requests are technically RPCs too ;) | 14:29 | |
| salv-orlando | anyway, this did not happen because we had other issues to chase, such as the ovs agent taking ages to configure a port. | 14:30 |
| salv-orlando | so at the moment we're adopting a disgusting approach | 14:30 |
| dansmith | salv-orlando: so are you setting max_pool_size? | 14:30 |
| salv-orlando | which is to force eventlet to relinquish control during mysql transaction | 14:30 |
| salv-orlando | it defaults to 10 | 14:30 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Do not assert router l3 agent association https://review.openstack.org/80301 | 14:30 |
| salv-orlando | there was a discussion some time ago. | 14:31 |
| mkoderer | mtreinish: are you around? | 14:31 |
| dansmith | salv-orlando: yeah, that's not good :( | 14:31 |
| salv-orlando | dansmith: on performance vs reliability. Because we turned that off. And then it went back on, but I'm not sure when that happened | 14:31 |
| openstackgerrit | Ivan-Zhu proposed a change to openstack/tempest: share test_servers between v2/v3 api tests https://review.openstack.org/80297 | 14:31 |
| salv-orlando | I can try and push a patch for disabling db connection pooling | 14:31 |
| *** mrodden has joined #openstack-qa | 14:32 | |
| salv-orlando | dansmith: and we'll see performance-wise how it affects the jobs in terms of runtime. This might give us a rough measure. | 14:32 |
| dansmith | salv-orlando: yeah, it might cause timeouts just because of performance | 14:32 |
| dansmith | salv-orlando: when comstud is around today, we should talk to him because he knows a lot about this | 14:33 |
| *** rdekel has joined #openstack-qa | 14:33 | |
| dansmith | he has a patch for eventlet to fix the actual problem, but it's been stalled for a long time | 14:33 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/79628 failed check-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254872 | 14:33 |
| mtreinish | mkoderer: yeah | 14:33 |
| *** gtt116 has joined #openstack-qa | 14:34 | |
| mtreinish | mriedem: +A | 14:34 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 14:34 |
| mriedem | mtreinish: thanks | 14:34 |
| mkoderer | mtreinish: I tried to drill down the problem with the pep8 check | 14:34 |
| mkoderer | mtreinish: I think we have a bug in hacking since flake8 doesn't change it's result code in case on an import error | 14:35 |
| mkoderer | mtreinish: additionally I found out that the config call isn't the root cause | 14:36 |
| *** bill_az_ has joined #openstack-qa | 14:37 | |
| *** otherwiseguy has quit IRC | 14:37 | |
| mkoderer | mtreinish: it seems to be the inheritance in test_flavors_negative_xml.py | 14:37 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/64271 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 14:37 |
| *** shwetaap has quit IRC | 14:37 | |
| *** andreaf has joined #openstack-qa | 14:37 | |
| mkoderer | mtreinish: btw I can't make it to the meeting today again :( | 14:38 |
| *** shwetaap has joined #openstack-qa | 14:39 | |
| mtreinish | mkoderer: ok, maybe next week (although it's kind of late for you then) | 14:39 |
| mtreinish | mkoderer: what do you mean it's the inheritance | 14:40 |
| mkoderer | mtreinish: https://github.com/openstack/tempest/blob/master/tempest/api/compute/admin/test_flavors_negative_xml.py#L24 | 14:41 |
| mkoderer | mtreinish: if I run a flake8 -v with no config I get: http://paste.openstack.org/show/73367/ | 14:42 |
| mkoderer | mtreinish: so it doesn't complain about the negative tests using the framework | 14:42 |
| mkoderer | mtreinish: it complains about the _xml one... | 14:42 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/80225 failed gate-ceilometer-python26, gate-ceilometer-python27 because of: https://bugs.launchpad.net/bugs/1258682 | 14:43 |
| salv-orlando | dansmith: yes, it's a good idea. So far we've been forcing eventlet to yield within transactions. And part of the problem is that some neutron transactions are awfully long. | 14:43 |
| mkoderer | mtreinish: I will write a mail to the ML and hopefully come up with a better fix tomorrow | 14:43 |
| *** nati_ueno has joined #openstack-qa | 14:43 | |
| dansmith | salv-orlando: yeah, that sounds like a recipe for disaster :) | 14:43 |
| mkoderer | mtreinish: https://review.openstack.org/#/c/80201/ this one will at least workaround it.. although it's not really nice | 14:43 |
| salv-orlando | dansmith: not a recipe for disaster, but neither a solution. A bit like when you putting duct tape on a leaking pipe rather than calling a plumber | 14:44 |
| salv-orlando | eventually it's going to explode on your face | 14:44 |
| dansmith | salv-orlando: heh, explode on your face == disaster :) | 14:45 |
| mtreinish | mkoderer: yeah that should work, but it's not pretty | 14:45 |
| *** yrabl has quit IRC | 14:46 | |
| mkoderer | mtreinish: yep.. but the good thing is pep8 was woring on the gate.. just this one file wasn't checked :) | 14:46 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Do not assert router l3 agent association https://review.openstack.org/80301 | 14:46 |
| *** gtt116 has left #openstack-qa | 14:47 | |
| *** comstud has joined #openstack-qa | 14:49 | |
| comstud | dansmith: ping | 14:49 |
| dansmith | salv-orlando: the man has arrived | 14:49 |
| openstackgerrit | Ivan-Zhu proposed a change to openstack/tempest: share test_servers between v2/v3 api tests https://review.openstack.org/80297 | 14:49 |
| dansmith | comstud: http://eavesdrop.openstack.org/irclogs/%23openstack-qa/%23openstack-qa.2014-03-13.log | 14:49 |
| comstud | ok | 14:50 |
| dansmith | 2014-03-13T14:07:00 | 14:50 |
| dansmith | comstud: it sounds like they have the same db connection pooling thing we experienced under high load, | 14:50 |
| comstud | well | 14:50 |
| dansmith | comstud: and they're trying to defeat it by yielding in the middle of transactions and stuff | 14:50 |
| *** otherwiseguy has joined #openstack-qa | 14:50 | |
| salv-orlando | dansmith, comstud: I'm not going anywhere | 14:50 |
| comstud | i'm not sure I recall what you're talking about | 14:50 |
| comstud | unless you mean 'thread pool' | 14:50 |
| comstud | which you shouldn't use | 14:50 |
| comstud | because eventlet is broken | 14:51 |
| comstud | but "db connection pooling" problem itself doesn't sound familiar | 14:51 |
| dansmith | that's what I'm referring to, perhaps we're confusing terms, but I expect it's the same | 14:51 |
| comstud | ok | 14:51 |
| salv-orlando | comstud: what we've been experiencing for a while now is that two transaction which should be easily serialised deadlock because of eventlet | 14:51 |
| comstud | How are you using eventlet w/ db calls? | 14:51 |
| salv-orlando | and I think that when moving to openstack.common.db we enabled db connection pooling | 14:52 |
| comstud | sec | 14:52 |
| salv-orlando | ok | 14:52 |
| comstud | an eyelash and my eye are fighting right now | 14:52 |
| comstud | phew, the eye one... close one | 14:53 |
| dansmith | comstud: try not using the big long fake ones and it'll be less of a problem | 14:53 |
| comstud | my real ones are too long | 14:53 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 14:54 |
| comstud | salv-orlando: Are you using the 'use_tpool' setting from openstack.common.db.api? | 14:54 |
| salv-orlando | nope not that one | 14:54 |
| comstud | ok good | 14:54 |
| dansmith | bah | 14:54 |
| comstud | because we have an outstanding bug with eventlet when that's turned on | 14:54 |
| dansmith | salv-orlando: can we confirm that it's not on though? | 14:54 |
| salv-orlando | dansmith: checking logs on gate | 14:55 |
| salv-orlando | comstud: the issue causing the lock wait timeout is not caused by thread pooling for connections | 14:55 |
| salv-orlando | pretty much this error might happen when two requests, issues concurrently, try to update the same resource | 14:56 |
| *** shwetaap1 has joined #openstack-qa | 14:56 | |
| salv-orlando | calling them R1 and R2, they are served by 2 green threads, say T1 and T2 | 14:56 |
| salv-orlando | when R1 comes in, T1 serves it. Then at some point R2 comes (and T1 is in the middle of a transaction) | 14:57 |
| salv-orlando | T2 starts a transaction as well, and wait for T1 to complete its transaction, but never yields again | 14:57 |
| comstud | salv-orlando: dansmith said something about you yielding in the middle of transactions | 14:57 |
| comstud | That would cause a problem itself | 14:57 |
| salv-orlando | comstud: that's how we "fix" the issue so far | 14:57 |
| comstud | that would not fix it | 14:57 |
| *** shwetaap1 has quit IRC | 14:58 | |
| salv-orlando | comstud: that why "fix" was quoted ;) | 14:58 |
| comstud | imagine GT1 starts transaction, does something that causes db to lock a table | 14:58 |
| comstud | you yield to GT2 | 14:58 |
| comstud | which does the same thing | 14:58 |
| comstud | GT1 is going to completely lock up the process | 14:58 |
| comstud | in the db call trying to get the lock | 14:58 |
| comstud | i mean GT2 | 14:58 |
| comstud | because GT1's db connection ahs the lock | 14:58 |
| salv-orlando | I agree. The added yields actually add entropy | 14:59 |
| comstud | well I'd rip that out immediately | 14:59 |
| comstud | it's masking the real problem | 14:59 |
| comstud | and creating a 2nd problem | 14:59 |
| comstud | I can't really help debug until that's gone :) | 14:59 |
| comstud | because any logs you show me could be *that* problem | 15:00 |
| salv-orlando | I can push a patch for removing the existing greenthread.sleep calls | 15:00 |
| comstud | honestly, I'd guess that would make your problem worse | 15:00 |
| comstud | so, here's another possibility.. | 15:00 |
| *** shwetaap1 has joined #openstack-qa | 15:01 | |
| salv-orlando | listening | 15:01 |
| comstud | after you fix that. any LOG()s in the middle of transactions can cause a GT switch | 15:01 |
| comstud | when 2 greenthreads try to log at the same time | 15:01 |
| dansmith | they grab the log lock | 15:01 |
| comstud | yes | 15:02 |
| dansmith | boom | 15:02 |
| comstud | (we may have this problem in nova actually :) | 15:02 |
| *** chandan_kumar has quit IRC | 15:02 | |
| *** chandan_kumar has joined #openstack-qa | 15:02 | |
| locke105 | is that the python stdlib log module that would grab the lock? | 15:03 |
| comstud | use_tpool=True would fix that problem (assuming that eventlet was actually thread safe :) | 15:03 |
| *** YorikSar has quit IRC | 15:03 | |
| comstud | the stdlib logging module uses a threading.Lock | 15:03 |
| comstud | which eventlet wraps | 15:03 |
| salv-orlando | right. I forgot about logging. | 15:03 |
| comstud | you *could* try to NOT monkey patch threading | 15:03 |
| locke105 | i c | 15:03 |
| comstud | monkey_patch(threading=False) or whatever | 15:04 |
| comstud | and that would fix the LOG() problem | 15:04 |
| locke105 | that would probably explode for other reasons | 15:04 |
| salv-orlando | comstud: I think we went already there in the past, but had a lot of other problems | 15:04 |
| comstud | Yeah, it may break other things | 15:04 |
| comstud | so, i'd remove your sleep()s | 15:04 |
| comstud | and comment out your LOG calls | 15:05 |
| salv-orlando | I think we might try to not log anymore in the middle of a transaction, but I just hope this won't become a mammoth sized task | 15:05 |
| comstud | and see what happens | 15:05 |
| comstud | btw, what error do you see | 15:05 |
| comstud | 'Lock wait timeout' from mysql ? | 15:05 |
| salv-orlando | yeah | 15:05 |
| comstud | yeah | 15:05 |
| *** YorikSar has joined #openstack-qa | 15:05 | |
| comstud | So another thing... | 15:05 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/78986 failed gate-cinder-python26 because of: https://bugs.launchpad.net/bugs/1291108 | 15:06 |
| comstud | Or 2 other things :) | 15:06 |
| salv-orlando | as many things as you like | 15:06 |
| *** shwetaap has quit IRC | 15:06 | |
| comstud | 1) You should retry the transaction on those :) You can use a nice little decorator we created in oslo | 15:06 |
| comstud | 2) There's another case that can cause those... | 15:06 |
| comstud | if your transaction does a SELECT and then an UPDATE | 15:06 |
| comstud | but really it would only happen if you run more than 1 copy | 15:07 |
| *** mwagner_lap has joined #openstack-qa | 15:07 | |
| comstud | I think it was devananda that explained that one to me once | 15:07 |
| devananda | ? | 15:07 |
| comstud | I could be remembering wrong :) | 15:08 |
| salv-orlando | we don't on the gate, but it's a common deployment. I am however in the process of removing the select..update situations | 15:08 |
| comstud | devananda: lock wait timeouts... SELECT+UPDATE ? | 15:08 |
| devananda | select .. for upate is pretty terrible as far as performance goes. AND can lead to deadlocks | 15:08 |
| salv-orlando | because they tend to cause rollback in transactions which should not be rolled back | 15:08 |
| devananda | and lock wait timeouts | 15:08 |
| devananda | yep | 15:08 |
| salv-orlando | ah sorry you meant SELECT FOR UPDATE? | 15:08 |
| comstud | phew, my memory is intact | 15:08 |
| comstud | Now, we actually still have a bunch of this in nova... | 15:09 |
| salv-orlando | yeah, we have plenty of those in our IPAM logic. I think somebody was looking at a non-locking approach, but then nothing happened | 15:09 |
| comstud | and we kludge it with the retry decorator | 15:09 |
| salv-orlando | comstud: but is that or the lock wait timeout or the DbDeadlock? | 15:09 |
| devananda | salv-orlando: if you do two separate statements: SELECT .. WHERE X; UPDATE ... WHERE X; and both are in teh same transaction and the transaction-isolation level is at least REPEATABLE-READ, you should be safe | 15:09 |
| devananda | however | 15:09 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79626 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1272417 and https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1253896 | 15:09 |
| devananda | well. no. not entirely | 15:10 |
| devananda | what's the problem exactly? | 15:10 |
| comstud | salv-orlando: They are similar issues... I think the decorator handles both, but let me check | 15:10 |
| salv-orlando | devananda: com stud was just walking me through root causes for the lock wait timeout error | 15:10 |
| comstud | salv-orlando: It actually looks like maybe we're not retrying on 'lock wait timeout' | 15:11 |
| comstud | and the _retry_on_deadlock decorator is only in nova (in nova/db/sqlalchemy/api.py) | 15:11 |
| devananda | salv-orlando: ah. short version -- probably some other trx locked that row (either with UPDATE or SELECT..FOR UPDATE) and didn't COMMIT. | 15:12 |
| comstud | anyway, if you're only running 1 process... | 15:12 |
| comstud | I suspect the real problem maybe the LOG statements | 15:12 |
| comstud | may be | 15:12 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/79373 failed gate-tempest-dsvm-full, gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error | 15:12 |
| salv-orlando | comstud: I agree that's what might be affecting gate tests. | 15:12 |
| comstud | yeah, i'd remove your sleeps and comment out the LOGs for now and see if that helps | 15:13 |
| comstud | The other thing you can do... | 15:13 |
| dansmith | comstud: yeah, we have a few logs in transactions, but only error paths and not many | 15:13 |
| * dansmith just looked | 15:13 | |
| comstud | is create a decorator that locks a semaphore | 15:13 |
| comstud | and put it on all of your db api calls :) | 15:13 |
| jogo | mkoderer: pong | 15:13 |
| comstud | That might be a better solution for now | 15:14 |
| comstud | so that you can keep your LOGs | 15:14 |
| comstud | And it doesn't cause any perf problems because they all run serially anyway :-/ | 15:14 |
| mkoderer | jogo: hey, I already sent a mail to the ML about my issue | 15:14 |
| openstackgerrit | A change was merged to openstack/tempest: Extend network debug infos on network failures https://review.openstack.org/76643 | 15:15 |
| openstackgerrit | A change was merged to openstack/tempest: Fix InvalidInstanceID.NotFound handling https://review.openstack.org/75385 | 15:15 |
| salv-orlando | comstud: I actually thought about that as well | 15:15 |
| markmcclain | semaphore might be a good low hanging fruit way to address this | 15:15 |
| comstud | I think that's what I'd do... now that I thought about it | 15:15 |
| mkoderer | jogo: I had some issues with the pep8 check today. maybe you could give me some feedback for hacking part | 15:15 |
| mkoderer | jogo: https://review.openstack.org/#/c/79256/, http://logs.openstack.org/56/79256/1/gate/gate-tempest-pep8/088cc12/console.html | 15:16 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Get credential IDs from Credentials class https://review.openstack.org/80246 | 15:16 |
| openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Enforces the use of Credentials https://review.openstack.org/77594 | 15:16 |
| *** kmartin has joined #openstack-qa | 15:17 | |
| *** jecarey has joined #openstack-qa | 15:18 | |
| salv-orlando | markmcclain: we should look at which methods in db_base_plugin should be guarded. Perhaps just a single global semaphore should be enough | 15:18 |
| mkoderer | jogo: IMHO flake8 should have a different result code if somehing like this happends | 15:18 |
| salv-orlando | comstud, devananda, dansmith: thanks all for you help! | 15:18 |
| dansmith | salv-orlando: all I did was point to the smart people.. for that, you're most welcome :) | 15:18 |
| jogo | salv-orlando: https://review.openstack.org/#/c/80301/ | 15:19 |
| salv-orlando | jogo: is that an alternative for the refer? | 15:19 |
| salv-orlando | revert | 15:19 |
| salv-orlando | I've looked at the original patch and I agree that assertion was not valid | 15:20 |
| sdague | salv-orlando: more importantly, how did the original patch pass? | 15:20 |
| jogo | salv-orlando: yeah | 15:20 |
| jogo | sdague: pure luck? | 15:20 |
| comstud | salv-orlando: np, let me know how it goes :) | 15:20 |
| sdague | because it only seemed to fail once over 9 revs | 15:20 |
| jogo | mkoderer: so this is an issue of should modules be importable and the answer is yes | 15:21 |
| sdague | salv-orlando: if you are +1 on the assert change in https://review.openstack.org/#/c/80301/ I'll hit +A | 15:21 |
| jogo | except we don't have a hacking requirement on that | 15:21 |
| salv-orlando | sdague: could it be that it passed before we rearranged tempest jobs and they were not running with parallelism? | 15:21 |
| salv-orlando | sdague: I am +1 | 15:21 |
| sdague | salv-orlando: no, this was always running in parallel | 15:21 |
| mkoderer | jogo: yep modules should be importable and if not it should fail | 15:21 |
| sdague | I mean jobs move around | 15:21 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/72845 failed gate-cinder-python26, gate-grenade-dsvm, gate-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1291108 and https://bugs.launchpad.net/bugs/1254872 | 15:22 |
| mkoderer | jogo: I am not quite sure why the result code is 0 at the end of the run | 15:22 |
| sdague | jogo: so on that issue, I think we need to rethink how the modules hacking test works | 15:22 |
| jogo | mkoderer: so I agree, I have to double check the hacking code but I think we changed that recently ... perhaps for the next release | 15:22 |
| sdague | because it currently means you have to pip install *all* of requirements.txt and test-requirements.txt to run pep8 | 15:22 |
| jogo | sdague mkoderer I would prefer to discuss this at a later date | 15:23 |
| mkoderer | jogo: yep I saw that there were some changes in.. I will test tomorrow if they are really fixing it | 15:23 |
| sdague | jogo: sure | 15:23 |
| mkoderer | jogo: sure I wrote a mail to the ML | 15:23 |
| *** nati_ueno has quit IRC | 15:23 | |
| mkoderer | jogo: but I will be off in IRC | 15:23 |
| jogo | mkoderer sdague: want to cut a hacking release soon with what is in, but only when we our back out of firedrill mode | 15:24 |
| sdague | jogo: there is an alternative way to do this, that just doesn't work on Mac, but we could detect and not care | 15:24 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/78614 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 15:24 |
| markmcclain | salv-orlando: yeah we should be able to find the ones we need | 15:24 |
| sdague | jogo: that's fine, it can be an early juno thing | 15:24 |
| openstackgerrit | Jakub Libosvar proposed a change to openstack-dev/grenade: Support for upgrading neutron https://review.openstack.org/58695 | 15:24 |
| jogo | sdague: well I haven't done a hacking cut yet because I didn't want to distract from getting icehouse out the door | 15:25 |
| jogo | anyway | 15:25 |
| sdague | yep | 15:25 |
| jogo | 1248757 scary | 15:25 |
| jogo | that is a massive spike | 15:25 |
| *** shwetaap1 has quit IRC | 15:25 | |
| sdague | jogo: honestly, I think all of these are coupled | 15:26 |
| *** shwetaap has joined #openstack-qa | 15:26 | |
| sdague | something in the last 2 days made neutron less reliable | 15:26 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79971 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 15:26 |
| jogo | sdague: that makes sense | 15:27 |
| *** nmagnezi has quit IRC | 15:27 | |
| *** yfried has joined #openstack-qa | 15:27 | |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Do not assert router l3 agent association https://review.openstack.org/80301 | 15:28 |
| jogo | Bug 1248757, Bug 1253896 ,Bug 1265495 all have spikes and are all ssh | 15:29 |
| sdague | afazekas: man, you just reset the clock on that patch by about 2 hrs | 15:30 |
| *** markmcclain has quit IRC | 15:30 | |
| sdague | jogo: ssh is just proxy for "network doesn't work" | 15:30 |
| salv-orlando | jogo: bug 1253896 is the one that will never go away; I will look at the spike soon (unless somebody here volunteers) | 15:31 |
| sdague | salv-orlando: so is there anything suspect in the neutron changelogs that you think could have impacted this? | 15:31 |
| salv-orlando | sdague: actually the only suspect change I can think of is the change that should have prevented situations where a vm started without network! | 15:32 |
| jogo | dansmith: ^ are we using your new nova/neutron code? | 15:33 |
| salv-orlando | jogo: beyond the bug on test_l3_agent_scheduler for which you filed the e-r query, I think I've created now bugs and e-r queries for most uncategorized neutron failures. I'm just working on a last one where apparently a port appears out of nowhere. | 15:33 |
| dansmith | jogo: nope, it's blocked on the current issues in the gate | 15:33 |
| salv-orlando | dasmith, jogo: right | 15:33 |
| salv-orlando | I gave it for granted that once approved, it merged | 15:33 |
| jogo | dansmith: hehe, I am a little relived to hear that because I would have been disappointed if that code broke things | 15:34 |
| afazekas | https://review.openstack.org/#/c/66375/ this can fix some connectivity issue | 15:34 |
| dansmith | jogo: me too :) | 15:34 |
| salv-orlando | afazekas: yes, that's for all the test minimum basic scenario falures | 15:34 |
| salv-orlando | or any failure where the network drops after a reboot | 15:34 |
| dansmith | jogo: the current thing is actually nova timing out trying to make a request, so it would neither help nor hurt the current thing | 15:34 |
| jogo | dansmith: thats what I thought, but glad we can 100% rule it out for now | 15:35 |
| salv-orlando | dansmith: I had a brain far thinking that the nova/neutron patches merged and the db event hanlder hook we installed was causing this issue | 15:35 |
| salv-orlando | s/brain far/brain fart | 15:35 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: remove duplicate bug https://review.openstack.org/80295 | 15:37 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/65892 failed gate-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1249065 | 15:37 |
| *** openstackrecheck has quit IRC | 15:38 | |
| *** openstackrecheck has joined #openstack-qa | 15:38 | |
| *** jasondotstar has quit IRC | 15:38 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/61037 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-postgres-nv because of: https://bugs.launchpad.net/bugs/1248757 | 15:39 |
| afazekas | Caught error: Connection to neutron failed: Maximum attempts reached is frequent in the n-api logs again | 15:40 |
| *** vkozhukalov has joined #openstack-qa | 15:41 | |
| jogo | I didn't see any obvious devstack/devstack gate patches to change any config | 15:43 |
| salv-orlando | sdague: I've lost a bit track of things. When was parallelism enabled on the neutron jobs running in the gate checks? | 15:44 |
| jogo | have you looked at the load of q-svc in gate? | 15:44 |
| sdague | salv-orlando: so the occurance of this exception seems to profile well with the rise | 15:44 |
| sdague | http://logstash.openstack.org/#eyJzZWFyY2giOiJcIkludmFsaWQ6IEV4cGVjdGVkIGFjdGl2ZSBwb29sXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjEzOTQ3MjU0MDA2ODJ9 | 15:44 |
| sdague | salv-orlando: at least a month ago | 15:44 |
| openstackgerrit | Steve Martinelli proposed a change to openstack-dev/devstack: Change IDENTITY_API_VERSION to v3 https://review.openstack.org/80320 | 15:45 |
| jogo | sdague: yeah that does match up pretty well | 15:45 |
| salv-orlando | sdague: thanks I'm just trying to identify the root cause of the spike in lock wait timeout errors seen in the past 36 hours | 15:45 |
| jogo | Optimize floating IP status update | 15:46 |
| jogo | I1e74fb8d90de09b53d330f499b93073f19a5b9bf | 15:46 |
| jogo | sounds very vague | 15:46 |
| salv-orlando | It sounds terribly suspicious mostly because of the author ;) | 15:46 |
| jogo | oh hehe, in that case | 15:47 |
| afazekas | salv-orlando: This time of monkey_patching can be too late https://github.com/openstack/neutron/blob/master/neutron/server/__init__.py#L37 , and the eventlet.monkey_patch does not patches the mysql module by default | 15:47 |
| jogo | I didn't have the authors up (was looking at the openstack/openstack repo) | 15:47 |
| salv-orlando | there I'm just doing only an update query rather than doing select and then update | 15:47 |
| salv-orlando | but I think it's either that or this: http://git.openstack.org/cgit/openstack/neutron/commit/?id=665352f169a7d1e7eca590d466241cc313adb7c3 | 15:49 |
| jogo | salv-orlando: those are the only two patches in the right time window | 15:49 |
| openstackrecheck | openstack/glance change: https://review.openstack.org/76927 failed check-grenade-dsvm, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1248757 | 15:49 |
| salv-orlando | one touches a function invoked by a RPC handler, and the other a function called in the midst of delete ports operations. | 15:50 |
| salv-orlando | I can easily revert both in one patch and see how it goes. | 15:50 |
| jogo | what about Ie2971bfae1fecc97086b58cd943c321feb0aba04 | 15:50 |
| jogo | salv-orlando: I see no reason to not try reverting both in a patch -- but this bug is infrequent enough that just a few runs may not be enough data | 15:52 |
| openstackgerrit | Jakub Libosvar proposed a change to openstack-dev/grenade: Support for upgrading neutron https://review.openstack.org/58695 | 15:52 |
| salv-orlando | jogo: agreed I was thinking of running it about 50 times pushing 10 identical changes and running each 5 times | 15:53 |
| salv-orlando | but perhaps that would be asking too much to the gate | 15:53 |
| salv-orlando | I mean to the infrastructure | 15:53 |
| salv-orlando | I'll then wait just a little for markmcclain to come back on line and then discuss with him the best course of action | 15:54 |
| *** vkozhukalov has quit IRC | 15:54 | |
| jogo | salv-orlando: what about reverting both patches separately and seeing if either gets any hits for the bug? | 15:58 |
| salv-orlando | jogo: yes that would work as well | 15:58 |
| jogo | that may make it easier to isolate if a single patch caused this to fail | 15:58 |
| salv-orlando | jogo: cool I will take care of that. | 16:00 |
| *** mmaglana has joined #openstack-qa | 16:03 | |
| *** jecarey has quit IRC | 16:07 | |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/79962 failed gate-ceilometer-python26, gate-ceilometer-python27, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1291694 | 16:08 |
| *** markmcclain has joined #openstack-qa | 16:08 | |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/79918 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 16:08 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/79943 failed check-grenade-dsvm, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1272417 and https://bugs.launchpad.net/bugs/1253896 | 16:10 |
| *** yassine has quit IRC | 16:14 | |
| *** jecarey has joined #openstack-qa | 16:17 | |
| *** mrodden has quit IRC | 16:18 | |
| jaypipes | mtreinish: hola. hey, I'm looking for a good regex to use for DEVSTACK_TEMPEST_REGEX for lukego to run only Neutron API tests in tempest. could you assist? | 16:19 |
| jaypipes | sdague: any idea on ^^? unfortunately, I'm unfamiliar with testr's regex matching compared to nose | 16:22 |
| *** rdekel has quit IRC | 16:23 | |
| *** vkozhukalov has joined #openstack-qa | 16:26 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 16:29 |
| *** yassine has joined #openstack-qa | 16:29 | |
| *** mrodden has joined #openstack-qa | 16:31 | |
| *** jasondotstar has joined #openstack-qa | 16:32 | |
| *** YorikSar_ has joined #openstack-qa | 16:32 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 16:33 |
| openstackgerrit | A change was merged to openstack/tempest: Do not assert router l3 agent association https://review.openstack.org/80301 | 16:34 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80244 failed gate-tempest-pep8, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1258682 | 16:35 |
| *** YorikSar has quit IRC | 16:36 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Temp enable savanna service for sahara tests. https://review.openstack.org/80237 | 16:36 |
| afazekas | jaypipes: api[.]network | 16:37 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/78069 failed gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1288918 | 16:38 |
| *** otherwiseguy has quit IRC | 16:40 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/66796 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-ironic-postgres-nv because of: https://bugs.launchpad.net/bugs/1291694 | 16:42 |
| lukego | jaypipes: hm I seem to have a syntax error in my regex? can have missed it in the log before. I see this now: | 16:47 |
| lukego | 17:45:35 /opt/stack/new/devstack-gate/devstack-vm-gate.sh: line 330: [[: tempest.api.network: syntax error: invalid arithmetic operator (error token is ".api.network") | 16:47 |
| lukego | when set to: tempest.api.network | 16:47 |
| *** shwetaap has quit IRC | 16:48 | |
| openstackgerrit | jun xie proposed a change to openstack/tempest: Test for neutron api with provider extension https://review.openstack.org/79864 | 16:48 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80320 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-heat-slow, check-devstack-dsvm-cells, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, gate-tempest-dsvm-large-ops, gate-tempest-dsvm-neutron-large-ops, check-tempest-dsvm-cells-full, check-grenade-dsvm- | 16:49 |
| openstackrecheck | partial-ncpu, check-devstack-dsvm-neutron, check-tempest-dsvm-ironic-nv, check-tempest-dsvm-ironic-postgres-nv, check-tempest-dsvm-sahara-full-nv, check-tempest-dsvm-sahara-postgres-nv, check-tempest-dsvm-sahara-neutron-nv because of: https://bugs.launchpad.net/bugs/1209086 | 16:49 |
| afazekas | lukego: do you have link to your change ? | 16:51 |
| *** shwetaap has joined #openstack-qa | 16:51 | |
| lukego | afazekas: afaik I'm running the upstream openstack-infra code without modification | 16:52 |
| lukego | md5sum: cbf47a4f8516cb3234f652777e4997ba /opt/stack/new/devstack-gate/devstack-vm-gate.sh | 16:52 |
| afazekas | " if [[ "$DEVSTACK_GATE_TEMPEST_REGEX" -ne "" ]] ; then" so this is the actual line ? | 16:52 |
| afazekas | == | 16:53 |
| lukego | looks like it. that's line 330 of my /opt/stack/new/devstack-gate/devstack-vm-gate.sh | 16:54 |
| afazekas | == | 16:54 |
| lukego | oh s/-ne/==/ ? | 16:54 |
| *** otherwiseguy has joined #openstack-qa | 16:54 | |
| lukego | alas, I have not succeeded in experimenting with changes to this script, my hacked versions always end up getting clobbered by a pristine upstream copy somehow when going through the Jenkins workflow | 16:54 |
| afazekas | = or == is the valid string comparator op | 16:54 |
| *** otherwiseguy has quit IRC | 16:55 | |
| lukego | != in this case perhaps | 16:55 |
| openstack | lukego: Error: "=" is not a valid command. | 16:55 |
| afazekas | :) yes | 16:55 |
| lukego | I don't know any way to test that change at the moment, beyond somebody pushing it upstream | 16:55 |
| afazekas | '-n' with one arg | 16:56 |
| afazekas | BTW: you should escape the '.' in the regexp, but probably it does not matters now | 16:57 |
| *** otherwiseguy has joined #openstack-qa | 16:57 | |
| openstackrecheck | openstack/python-keystoneclient change: https://review.openstack.org/80193 failed gate-python-keystoneclient-python27 because of: https://bugs.launchpad.net/bugs/1272417 | 16:59 |
| mtreinish | jaypipes: well for the neutron gate we run smoke | 16:59 |
| mtreinish | but just 'network' should be fine too that will pick up the api tests and all the network service tagged tests | 16:59 |
| mtreinish | which include all the scenarios | 17:00 |
| afazekas | mtg time | 17:00 |
| openstackgerrit | Steven Hardy proposed a change to openstack-dev/devstack: heat add HEAT_DEFERRED_AUTH option https://review.openstack.org/80002 | 17:01 |
| lukego | afazekas: thanks very much for the diagnosi | 17:01 |
| *** tkammer has quit IRC | 17:01 | |
| openstackgerrit | Steven Hardy proposed a change to openstack-dev/devstack: heat configure auth_token for v3 keystone https://review.openstack.org/80341 | 17:01 |
| lukego | jaypipes: so! afazekas is right, devstack-vm-gate.sh regex support is broken, it's using integer compare (-ne) on strings and so it will never apply the regexp | 17:02 |
| mtreinish | lukego: oops that's my fault | 17:03 |
| mtreinish | do you want to file a bug and push out a fix | 17:03 |
| lukego | mtreinish: any chance of a quick fix? | 17:03 |
| mtreinish | sure I can do it | 17:03 |
| lukego | thanks! | 17:04 |
| *** rhsu has joined #openstack-qa | 17:05 | |
| *** mlavalle has joined #openstack-qa | 17:05 | |
| lukego | mtreinish: I think that s/-ne/!=/ is what we need btw | 17:06 |
| openstackgerrit | Aleksandr Gordeev proposed a change to openstack-dev/devstack: Integration testing preparation for Ironic https://review.openstack.org/70348 | 17:09 |
| *** morganfainberg_Z is now known as morganfainberg | 17:09 | |
| *** harlowja_away is now known as harlowja | 17:10 | |
| openstackrecheck | openstack-infra/devstack-gate change: https://review.openstack.org/80045 failed check-grenade-dsvm, check-tempest-dsvm-full, check-tempest-dsvm-neutron-heat-slow, check-devstack-dsvm-cells, check-dg-tempest-dsvm-full, check-dg-tempest-dsvm-full-reexec, check-tempest-dsvm-neutron-full, gate-tempest-dsvm-large-ops, gate-tempest-dsvm-neutron-large-ops, check-swift-dsvm-functional, check-tempest-dsvm- | 17:12 |
| openstackrecheck | neutron-grizzly, check-tempest-dsvm-postgres-full-grizzly, check-tempest-dsvm-cells-grizzly, check-tempest-dsvm-neutron-havana, check-devstack-dsvm-cells-havana, check-tempest-dsvm-sahara-postgres-nv because of: https://bugs.launchpad.net/bugs/1286963 | 17:12 |
| openstackrecheck | openstack/swift change: https://review.openstack.org/80095 failed check-grenade-dsvm, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1248757 | 17:12 |
| *** asselin has joined #openstack-qa | 17:14 | |
| *** BobBall is now known as BobBall_Away | 17:14 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79458 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1254872 | 17:16 |
| asselin | Hello, I'm new to working on tempest. I'm getting auth errors after running for an hour. Any tips on how to force a re-auth? | 17:20 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/70429 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu, check-tempest-dsvm-sahara-full-nv because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1249065 and https://bugs.launchpad.net/bugs/1226412 and https://bugs.launchpad.net/bugs/1254872 | 17:20 |
| openstackrecheck | openstack/python-cinderclient change: https://review.openstack.org/74234 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 17:21 |
| *** sthaha has quit IRC | 17:22 | |
| *** rhsu has quit IRC | 17:22 | |
| *** rms_13 has joined #openstack-qa | 17:24 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Make IRC bot list which failures were seen in which job. https://review.openstack.org/78790 | 17:25 |
| openstackgerrit | jun xie proposed a change to openstack/tempest: Test for neutron api with provider extension https://review.openstack.org/79864 | 17:26 |
| *** rhsu has joined #openstack-qa | 17:26 | |
| *** jordanP has quit IRC | 17:28 | |
| *** luqas has quit IRC | 17:28 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/67382 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1253896 and https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1248757 | 17:29 |
| *** luqas has joined #openstack-qa | 17:30 | |
| asselin | Hi fungi, I'm new to working on tempest. I'm getting auth errors after running for an hour. Any tips on how to force a re-auth? | 17:31 |
| jaypipes | lukego: did you chat with markmcclain about what tests the neutron external CI systems should run? | 17:31 |
| lukego | No. I noticed the opendaylight was running the neutron API ones though. | 17:31 |
| markmcclain | lukego: that is a good start | 17:33 |
| *** luqas has quit IRC | 17:33 | |
| markmcclain | ideally the full run would be best | 17:33 |
| sdague | asselin: it's currently qa meeting, so people will tend to be back here in 30 mins or so | 17:34 |
| asselin | sdague, thanks | 17:34 |
| lukego | markmcclain: thanks for the tip. main reason I want to restrict right now is to make the turnaround time on a successful run faster, for debugging | 17:35 |
| *** atiwari has joined #openstack-qa | 17:37 | |
| *** yassine has quit IRC | 17:38 | |
| *** _sweston_ has quit IRC | 17:40 | |
| *** nithyag has joined #openstack-qa | 17:41 | |
| lifeless | sdague: I was not. I am now | 17:41 |
| *** [1]eezhova has joined #openstack-qa | 17:42 | |
| fungi | asselin: i'm not sure what you mean by auth errors or force a re-auth... but i also don't really use tempest, so the other people in here might have a better idea of what you're talking about | 17:50 |
| *** atiwari has quit IRC | 17:51 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova unit test bug 1291605 https://review.openstack.org/80087 | 17:53 |
| *** rossella_ has quit IRC | 17:54 | |
| *** mmaglana has quit IRC | 17:55 | |
| *** YorikSar_ is now known as YorikSar | 17:55 | |
| *** dkranz has quit IRC | 17:56 | |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/79755 failed check-grenade-dsvm, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 17:56 |
| openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm 73694,10,218541a | 17:56 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/79256 failed gate-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254872 | 17:56 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/79278 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1249065 | 17:56 |
| asselin | Tempest gurus: I'm running into auth failures about 1 hour after a tempest run of some new stress tests I'm creating. Any tips on how to fix that? http://paste.openstack.org/show/73442/ | 17:56 |
| *** jpich has quit IRC | 17:56 | |
| *** dkranz has joined #openstack-qa | 17:57 | |
| *** YorikSar has joined #openstack-qa | 17:57 | |
| dkranz | mtreinish: I got booted out | 17:57 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 17:57 |
| *** eezhova_ has joined #openstack-qa | 17:58 | |
| afazekas | andreaf, dkranz, sdague, mtreinish: if just by having a domain admin account configured is enough for running temepst as non-admin for everyone, we could delete a lot of logic/code | 17:59 |
| *** openstackrecheck has quit IRC | 17:59 | |
| openstackgerrit | Aniket Dandekar proposed a change to openstack/tempest: Add subnet-gateway tests in api network tests https://review.openstack.org/71397 | 17:59 |
| *** openstackrecheck has joined #openstack-qa | 18:00 | |
| *** [1]eezhova has left #openstack-qa | 18:00 | |
| sdague | afazekas: which code? | 18:00 |
| andreaf | afazekas: that's enough for tenant isolation | 18:00 |
| afazekas | All related to the not isolated runs | 18:00 |
| sdague | afazekas: sure, though that makes v3 mandatory | 18:01 |
| andreaf | afazekas: I'm not sure about other admin tests, where you need to be nova admin, neutron admin etc | 18:01 |
| sdague | which I'm not entirely comfortable with | 18:01 |
| afazekas | What are the remaining reason after a domin admin account to keep the non -isolated code path ? | 18:01 |
| *** shwetaap has quit IRC | 18:01 | |
| *** mmaglana has joined #openstack-qa | 18:01 | |
| sdague | I think it's a good addition, I just don't want to drop the other code yet | 18:01 |
| afazekas | Not now :), but for example in the next cycle | 18:02 |
| sdague | mtreinish: on asselin's issue, are we handling keystone token expiration ? | 18:02 |
| sdague | afazekas: well, not until keystone drops v2 :) | 18:02 |
| andreaf | the overall identity admin is a critical account to use, if you are in a non-disposable cloud, perhaps we need some more role granularity in devstack / tempest | 18:03 |
| asselin | sdague, afazekas, i found it here...but not sure why it's not working tempest.auth.KeystoneV2AuthProvider.is_expired | 18:03 |
| andreaf | at least something like a service admin, which is not an identity admin | 18:03 |
| mtreinish | sdague: I thought we did in the rest client | 18:03 |
| sdague | mtreinish: I know keystone recently dropped token expiration to 1 hr | 18:04 |
| mtreinish | oh and I guess andreaf's refactor would have moved it | 18:04 |
| sdague | so that would explain the behavior being hit | 18:04 |
| andreaf | mtreinish: my code in auth.py checks for expired tokens | 18:04 |
| sdague | so we have a test for that in the gate? | 18:04 |
| sdague | to make sure we are doing that right ourselves? | 18:05 |
| asselin | sdague, andreaf, mtreinish seems the code should be more conservative to e.g. refresh 1 minute before, instead of == at the time | 18:06 |
| andreaf | sdague: I think there is an identity test which deletes a token and tries to auth | 18:06 |
| lukego | mtreinish: I can file that bug/fix. thanks for the offer, but I need the sense of progress :-) | 18:06 |
| andreaf | asselin: yes that makes sense | 18:06 |
| mtreinish | sdague: it doesn't look like there is an expiration test (or an auth unit test for expiration) | 18:06 |
| mtreinish | lukego: ok you can go ahead and do it | 18:07 |
| mtreinish | it's nice low hanging fruit | 18:07 |
| andreaf | mtreinish: right the unit test is missing, I put a comment on the patch for auth unit tests, but I didn't submit a new patchset for that | 18:07 |
| andreaf | mtreinish: I can do so | 18:07 |
| sdague | andreaf: that would be great | 18:07 |
| andreaf | mtreinish: I will do so | 18:07 |
| sdague | thanks | 18:08 |
| asselin | thank you, if possible, let me know the bug # so I can pull it in my env | 18:08 |
| openstackgerrit | jun xie proposed a change to openstack/tempest: Test for neutron api with provider extension https://review.openstack.org/79864 | 18:08 |
| andreaf | asselin: it will be part of the unit test bp | 18:08 |
| andreaf | asselin: re checking exp mroe conservative | 18:09 |
| asselin | andreaf, ok | 18:09 |
| afazekas | asselin: the token should be renewed before the expiration | 18:09 |
| andreaf | asselin: do we need a configurable time delta there | 18:09 |
| asselin | I don't think it needs to be configurable. just large enough for the token to be still valid by the time the request is sent....which happens 'soon' after the check | 18:10 |
| andreaf | afazekas, asselin: ok, a fixed 60s should be enough, and still not cause too many keystone requests | 18:11 |
| asselin | andreaf, agree | 18:12 |
| andreaf | so I'll submit to patches, on to renew the token 60s in advance, and a second one to unit test that | 18:13 |
| *** mmaglana has quit IRC | 18:13 | |
| asselin | andreaf, sounds good to me! thanks! | 18:13 |
| *** nati_ueno has joined #openstack-qa | 18:14 | |
| andreaf | https://bugs.launchpad.net/tempest/+bug/1292145 | 18:15 |
| lukego | mtreinish: do I use git-review or a github pull request to send a fix to devstack-gate? I filed a bug here: https://bugs.launchpad.net/openstack-community/+bug/1292138 | 18:16 |
| sdague | lukego: git-review | 18:17 |
| lukego | sdague: the repo I cloned is on github. was that the wrong one? | 18:17 |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron-full 79962,6,6aba54f | 18:18 |
| mtreinish | lukego: https://wiki.openstack.org/wiki/Gerrit_Workflow | 18:18 |
| sdague | lukego: no, we mirror it to github as well | 18:18 |
| sdague | we just don't use pull requests | 18:18 |
| *** shwetaap has joined #openstack-qa | 18:18 | |
| *** atiwari has joined #openstack-qa | 18:18 | |
| sdague | we have multiple git mirrors to make it easy for people to get it | 18:18 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80340 failed gate-neutron-pep8, gate-neutron-python26, check-grenade-dsvm, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1254872 and | 18:19 |
| openstackrecheck | https://bugs.launchpad.net/bugs/1288312 and https://bugs.launchpad.net/bugs/1248757 | 18:19 |
| lukego | sdague: thanks for clarifying | 18:19 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/71474 failed check-tempest-dsvm-neutron, check-grenade-dsvm-partial-ncpu, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 18:19 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/72528 failed check-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254872 | 18:20 |
| andreaf | sdague: if you have a moment to check https://review.openstack.org/#/c/79212/ and https://review.openstack.org/#/c/79314/ - I've not changed layout.yaml before, I'd welcome feedback | 18:21 |
| andreaf | sdague: about experimental keystone v3 test run | 18:21 |
| openstackgerrit | John Dickinson proposed a change to openstack/tempest: fix race condition in tempurl test https://review.openstack.org/80108 | 18:21 |
| andreaf | sdague: also do you have pointers to the change that lost the admin tests, and its fix? | 18:22 |
| openstackgerrit | Luke Gorrie proposed a change to openstack-infra/devstack-gate: Fixed DEVSTACK_GATE_TEMPEST_REGEX. https://review.openstack.org/80359 | 18:23 |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: Setup the correct ec2 manifest path https://review.openstack.org/71474 | 18:24 |
| sdague | andreaf: it was devstack | 18:24 |
| sdague | andreaf: this is the fix - https://review.openstack.org/#/c/78931/ | 18:25 |
| andreaf | sdague: ok I see, stupid me, thanks for fixing that | 18:25 |
| sdague | yeh, I missed it on your original review as well | 18:26 |
| sdague | I believe I was the one that +Aed the review that broke that | 18:26 |
| sdague | it happens | 18:26 |
| sdague | we are all only human | 18:26 |
| *** jlibosva has quit IRC | 18:26 | |
| afazekas | sdague: I have removed the fixup part, the devstack gate will pick the 2.27 , which looks like a good version ^ | 18:31 |
| andreaf | mtreinish: ping | 18:32 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/80196 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 18:33 |
| mtreinish | andreaf: what's up | 18:36 |
| andreaf | mtreinish: I split my multi-auth patches a bit more, but two of them are still relatively large - I'd like to know if you think they are reviewable as they are | 18:37 |
| mtreinish | sdague: when you get a chance I'd like to get this merged before we start adding more unit tests: https://review.openstack.org/#/c/77269/ | 18:37 |
| mtreinish | andreaf: let me take a look | 18:37 |
| andreaf | https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:bp/multi-keystone-api-version-tests,n,z | 18:37 |
| mtreinish | andreaf: yeah I'd say they are but it'll probably take some time | 18:38 |
| mtreinish | the smaller and more consumable it is the faster it will get reviewed | 18:38 |
| andreaf | mtreinish: I agree - but in same cases you'd end up writing code just to be able to split up the patch which is additional code to be reviewed | 18:39 |
| openstackrecheck | openstack/glance change: https://review.openstack.org/80026 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 | 18:40 |
| andreaf | mtreinish: if you see something which can be easily split which I missed please let me know :) | 18:40 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/80343 failed gate-ceilometer-pep8, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 18:41 |
| *** safchain has quit IRC | 18:42 | |
| openstackgerrit | Jay Pipes proposed a change to openstack/tempest: Override oslotest time mock if need timing events https://review.openstack.org/80106 | 18:51 |
| openstackgerrit | A change was merged to openstack/tempest: Fix issue with pep8 gate job https://review.openstack.org/80201 | 19:02 |
| openstackgerrit | A change was merged to openstack/tempest: Validate Volume attributes of Nova POST & GET API https://review.openstack.org/78593 | 19:02 |
| *** tosky has quit IRC | 19:08 | |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-full 77546,2,5c0b7a7 | 19:08 |
| openstackrecheck | openstack/swift change: https://review.openstack.org/80095 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1249065 | 19:08 |
| *** marun has quit IRC | 19:10 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: remove additional f18 references https://review.openstack.org/79308 | 19:12 |
| *** adalbas has quit IRC | 19:13 | |
| *** vkozhukalov has quit IRC | 19:18 | |
| *** mwagner_lap has quit IRC | 19:19 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80261 failed gate-neutron-pep8, check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1248757 | 19:19 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/79949 failed gate-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254872 | 19:20 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80001 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1254872 | 19:20 |
| *** marun has joined #openstack-qa | 19:21 | |
| *** adalbas has joined #openstack-qa | 19:25 | |
| openstackgerrit | Malini Kamalambal proposed a change to openstack-dev/devstack: Disable journaling for mongodb https://review.openstack.org/74088 | 19:27 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80354 failed gate-neutron-pep8, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 19:27 |
| *** malini has left #openstack-qa | 19:28 | |
| dkranz | sdague: I just noticed that if you do 'nova net-list' as the demo user with devstack running nova-network, it shows nothing. Do you know if that is expected? | 19:30 |
| dkranz | sdague: I also noticed that tempest has zero tests for the apis called by 'nova net-*' | 19:31 |
| sdague | I do not | 19:31 |
| dkranz | sdague: ok thanks. I'll ask around. | 19:31 |
| sdague | dkranz: that doesn't entirely surprise me either | 19:31 |
| dkranz | sdague: It certainly didn't surprise me after seeing it return nothing :) | 19:32 |
| *** nati_ueno has quit IRC | 19:33 | |
| *** atiwari has quit IRC | 19:35 | |
| *** beisner has joined #openstack-qa | 19:38 | |
| openstackrecheck | openstack/swift change: https://review.openstack.org/71704 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1248757 | 19:39 |
| *** andreaf has quit IRC | 19:47 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/70457 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 19:57 |
| openstackrecheck | openstack/glance change: https://review.openstack.org/80358 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1288918 | 20:00 |
| openstackgerrit | Sergey Lukjanov proposed a change to openstack/tempest: Rename Savanna to Sahara https://review.openstack.org/80382 | 20:00 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79254 failed check-grenade-dsvm, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 20:01 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/80225 failed gate-ceilometer-python26, gate-ceilometer-python27, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1258682 | 20:04 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/79504 failed check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1288918 | 20:05 |
| openstackrecheck | openstack/python-heatclient change: https://review.openstack.org/79487 failed gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254872 | 20:05 |
| *** adalbas has quit IRC | 20:06 | |
| *** atiwari has joined #openstack-qa | 20:07 | |
| maurosr | /bu12 | 20:14 |
| maurosr | sorry ^ | 20:14 |
| *** adalbas has joined #openstack-qa | 20:19 | |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/70348 failed check-tempest-dsvm-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1272417 and https://bugs.launchpad.net/bugs/1254890 | 20:21 |
| *** shwetaap has quit IRC | 20:29 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/80293 failed check-tempest-dsvm-neutron, gate-keystone-python33 because of: https://bugs.launchpad.net/bugs/1248757 | 20:30 |
| *** vkozhukalov has joined #openstack-qa | 20:32 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80284 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1258848 and https://bugs.launchpad.net/bugs/1254872 | 20:32 |
| *** shwetaap has joined #openstack-qa | 20:35 | |
| *** shwetaap has quit IRC | 20:37 | |
| *** nati_ueno has joined #openstack-qa | 20:51 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/75253 failed gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1254872 | 20:53 |
| *** jamielennox|away is now known as jamielennox | 20:55 | |
| *** shwetaap has joined #openstack-qa | 21:07 | |
| jaypipes | does tempest have any testing at all for nova cells? | 21:08 |
| *** lukego has quit IRC | 21:08 | |
| clarkb | jaypipes: as far as the gate goes, no. cells don't pass tempest beyond grizzly | 21:09 |
| clarkb | jaypipes: some subset may work (eg the subset grizzly runs) | 21:09 |
| mtreinish | jaypipes: there isn't any testing of the cells api in tempest | 21:10 |
| jaypipes | k, thx guys, didn't think so. | 21:10 |
| *** giulivo has quit IRC | 21:11 | |
| jogo | sdague salv-orlando: any updates on the neutron spike | 21:11 |
| *** yassine has joined #openstack-qa | 21:13 | |
| jogo | because everything is still backed up :/ | 21:15 |
| *** safchain has joined #openstack-qa | 21:20 | |
| *** safchain has quit IRC | 21:20 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/80382 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1226412 and https://bugs.launchpad.net/bugs/1254872 | 21:20 |
| *** mmaglana has joined #openstack-qa | 21:20 | |
| openstackgerrit | James E. Blair proposed a change to openstack-infra/devstack-gate: WIP: test caching-forwarding nameserver https://review.openstack.org/80045 | 21:21 |
| *** mlavalle has quit IRC | 21:22 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/79525 failed gate-keystone-python26, gate-keystone-python27, gate-keystone-python33, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 | 21:23 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79254 failed check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 21:25 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80177 failed gate-neutron-pep8, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 21:25 |
| openstackgerrit | Adam Gandelman proposed a change to openstack-dev/devstack: Move libvirt install + setup to functions-libvirt https://review.openstack.org/80404 | 21:27 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Enable run_ssh and add floatingip + keypair auth https://review.openstack.org/50337 | 21:28 |
| *** adalbas has quit IRC | 21:28 | |
| *** ekhugen has quit IRC | 21:29 | |
| *** markmcclain has quit IRC | 21:31 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80388 failed gate-neutron-pep8, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 | 21:32 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/77916 failed check-tempest-dsvm-neutron, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 21:33 |
| salv-orlando | jogo: we're checking the reverts… unfortunately even a spin in the check queue takes 1 hours | 21:36 |
| salv-orlando | I am working also on a patch based on comstud's advice | 21:36 |
| jogo | salv-orlando: thanks, is there anything I can do/facilate to help things along? | 21:38 |
| *** rhsu1 has joined #openstack-qa | 21:40 | |
| *** rhsu has quit IRC | 21:42 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/74300 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1254872 | 21:44 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/75980 failed gate-cinder-python26, gate-cinder-python27, check-tempest-dsvm-neutron-heat-slow because of: https://bugs.launchpad.net/bugs/1272417 | 21:45 |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/80225 failed gate-ceilometer-python26, gate-ceilometer-python27, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1258682 and https://bugs.launchpad.net/bugs/1248757 | 21:48 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/75291 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1248757 | 21:49 |
| openstackgerrit | A change was merged to openstack/tempest: Make test_volume_quotas force tenant isolation https://review.openstack.org/79958 | 21:50 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80101 failed gate-tempest-dsvm-full because of: https://bugs.launchpad.net/bugs/1288918 | 21:50 |
| *** sweston has joined #openstack-qa | 21:50 | |
| *** mriedem has quit IRC | 21:54 | |
| *** rockyg has joined #openstack-qa | 21:55 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/77802 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1248757 | 21:59 |
| *** eezhova_ has quit IRC | 22:00 | |
| openstackgerrit | Salvatore Orlando proposed a change to openstack-infra/elastic-recheck: Add a footprint for bug 1283522 https://review.openstack.org/80286 | 22:03 |
| openstackgerrit | Adam Gandelman proposed a change to openstack-dev/devstack: Move libvirt install + setup to functions-libvirt https://review.openstack.org/80404 | 22:04 |
| openstackrecheck | openstack/python-keystoneclient change: https://review.openstack.org/79474 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 22:09 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80354 failed gate-neutron-pep8, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1254872 | 22:15 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/80397 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 22:16 |
| *** mmaglana has quit IRC | 22:28 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add unit basic unit tests for tenant_isolation https://review.openstack.org/80416 | 22:29 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/80399 failed check-tempest-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1248757 | 22:30 |
| *** shwetaap has quit IRC | 22:31 | |
| openstackgerrit | Armando Migliaccio proposed a change to openstack/tempest: Fix error trace induced by dhcp test actions https://review.openstack.org/80418 | 22:34 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/71397 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 22:35 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/78361 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-cells-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 22:37 |
| *** shwetaap has joined #openstack-qa | 22:44 | |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/80343 failed check-tempest-dsvm-postgres-full, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1254872 | 22:44 |
| *** mkollaro has quit IRC | 22:46 | |
| *** jgrimm has quit IRC | 22:47 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/79018 failed gate-neutron-python27, check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1288312 and https://bugs.launchpad.net/bugs/1248757 | 22:49 |
| *** aysyd has quit IRC | 22:52 | |
| *** shwetaap has quit IRC | 22:54 | |
| *** sweston has quit IRC | 22:55 | |
| *** dims_ has quit IRC | 22:58 | |
| *** shwetaap has joined #openstack-qa | 23:00 | |
| openstackrecheck | openstack/swift change: https://review.openstack.org/65347 failed check-tempest-dsvm-full, check-tempest-dsvm-neutron-full, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1254872 and https://bugs.launchpad.net/bugs/1248757 | 23:01 |
| openstackgerrit | A change was merged to openstack/tempest: Make tempest accounts independent from devstack https://review.openstack.org/77602 | 23:02 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/78744 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full, check-tempest-dsvm-neutron, check-grenade-dsvm-neutron, check-devstack-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1248757 | 23:02 |
| *** adalbas has joined #openstack-qa | 23:03 | |
| *** mriedem has joined #openstack-qa | 23:04 | |
| openstackgerrit | A change was merged to openstack/tempest: Configure the ec2 zone explicitly https://review.openstack.org/75411 | 23:05 |
| *** mriedem has quit IRC | 23:05 | |
| openstackrecheck | openstack/trove change: https://review.openstack.org/79838 failed gate-trove-python27 in the gate queue with an unrecognized error | 23:05 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/48255 failed check-tempest-dsvm-neutron, check-tempest-dsvm-neutron-full because of: https://bugs.launchpad.net/bugs/1251521 and https://bugs.launchpad.net/bugs/1248757 | 23:06 |
| *** mriedem has joined #openstack-qa | 23:07 | |
| *** mkollaro has joined #openstack-qa | 23:08 | |
| *** shwetaap has quit IRC | 23:11 | |
| *** dims_ has joined #openstack-qa | 23:11 | |
| *** samuelbercovici has joined #openstack-qa | 23:12 | |
| *** jgrimm has joined #openstack-qa | 23:15 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80413 failed check-tempest-dsvm-neutron-pg, check-tempest-dsvm-neutron-pg, check-grenade-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1288918 and https://bugs.launchpad.net/bugs/1280464 | 23:15 |
| *** shwetaap has joined #openstack-qa | 23:15 | |
| *** atiwari_ has joined #openstack-qa | 23:17 | |
| *** atiwari_ has quit IRC | 23:18 | |
| *** atiwari_ has joined #openstack-qa | 23:18 | |
| *** jasondotstar has quit IRC | 23:18 | |
| *** atiwari has quit IRC | 23:19 | |
| *** jasondotstar has joined #openstack-qa | 23:22 | |
| *** shwetaap has quit IRC | 23:22 | |
| *** rockyg has quit IRC | 23:24 | |
| *** samuelbercovici1 has joined #openstack-qa | 23:24 | |
| *** samuelbercovici has quit IRC | 23:27 | |
| *** samuelbercovici1 is now known as samuelbercovici | 23:27 | |
| *** masayukig has joined #openstack-qa | 23:27 | |
| *** ken1ohmichi has joined #openstack-qa | 23:27 | |
| openstackrecheck | Console logs not available after 13:27s for gate-tempest-pep8 80418,1,84f0eb5 | 23:30 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/80404 failed check-tempest-dsvm-full, check-tempest-dsvm-postgres-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1254872 | 23:30 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/78042 failed check-tempest-dsvm-full, check-grenade-dsvm-partial-ncpu because of: https://bugs.launchpad.net/bugs/1288918 | 23:31 |
| jogo | sdague: ping https://review.openstack.org/#/c/80027/ -- that should be the last match needed to get the partial-ncpu job working. I was hoping to get this in as soon as possible | 23:31 |
| jogo | (this week) | 23:31 |
| sdague | jogo: will look in the morning | 23:31 |
| jogo | great, I am running a recheck to confirm it works. so morning is perfect | 23:32 |
| sdague | with luck, it will be merged before you wake up | 23:32 |
| * jogo makes a mental note to sleep in | 23:32 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80356 failed check-tempest-dsvm-neutron-heat-slow, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1249065 | 23:32 |
| *** atiwari_ has quit IRC | 23:32 | |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/80354 failed gate-neutron-pep8, gate-neutron-python27, check-tempest-dsvm-neutron-full, check-grenade-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1288312 | 23:34 |
| *** GMann has joined #openstack-qa | 23:35 | |
| *** yfujioka has joined #openstack-qa | 23:36 | |
| *** rhsu has joined #openstack-qa | 23:36 | |
| GMann | Hi dkranz - I have updated the patch can you please have a look - https://review.openstack.org/#/c/79251/ | 23:37 |
| *** jasondot_ has joined #openstack-qa | 23:38 | |
| *** KurtMartin has joined #openstack-qa | 23:39 | |
| *** YorikSar_ has joined #openstack-qa | 23:40 | |
| *** jasondotstar has quit IRC | 23:41 | |
| *** YorikSar has quit IRC | 23:41 | |
| *** markmcclain has joined #openstack-qa | 23:45 | |
| *** rhsu1 has quit IRC | 23:45 | |
| *** kmartin has quit IRC | 23:45 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/58372 failed check-tempest-dsvm-full, gate-keystone-python33, check-tempest-dsvm-postgres-full because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1254872 | 23:46 |
| *** adalbas has quit IRC | 23:46 | |
| openstackgerrit | Armando Migliaccio proposed a change to openstack/tempest: Fix error trace induced by dhcp test actions https://review.openstack.org/80418 | 23:49 |
| openstackrecheck | openstack/neutron change: https://review.openstack.org/75253 failed gate-neutron-python26, check-tempest-dsvm-full, check-grenade-dsvm-neutron because of: https://bugs.launchpad.net/bugs/1254890 and https://bugs.launchpad.net/bugs/1280464 and https://bugs.launchpad.net/bugs/1288312 | 23:50 |
| jogo | salv-orlando: https://review.openstack.org/#/c/80286/ mind if I fix that one my self and push it | 23:51 |
| salv-orlando | jogo: yeah go ahead. I would have never though an e-r query would have taken that many patch sets! | 23:54 |
| *** sweston has joined #openstack-qa | 23:57 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!