jogo | dtroyer: ping https://bugs.launchpad.net/bugs/1331274 | 00:05 |
---|---|---|
jogo | sdague: said you had an idea about that one | 00:05 |
jogo | something about services not being killed properly | 00:08 |
*** melwitt has left #openstack-qa | 00:09 | |
*** melwitt has joined #openstack-qa | 00:09 | |
melwitt | jogo: hi, on https://bugs.launchpad.net/grenade/+bug/1349617 I was thinking if it would be reasonable to make a new log file for paramiko debug logs so we can look at details of ssh connections in tests? | 00:11 |
cyeoh | if there are any cores around could they have a look at https://review.openstack.org/#/c/112445/ (it was waiting for the nova change to be +2'd - which it has. And it has one +2 itself already | 00:18 |
*** sballe_ has quit IRC | 00:22 | |
mtreinish | cyeoh: +A | 00:23 |
cyeoh | mtreinish: thanks! | 00:23 |
*** dmorita has joined #openstack-qa | 00:25 | |
dtroyer | jogo: that's what I just proposed (again). First thing is to make services start/stop reliably without screen, then we see what is left for that bug | 00:31 |
*** sballe has joined #openstack-qa | 00:31 | |
*** yamahata has joined #openstack-qa | 00:34 | |
*** yamahata has quit IRC | 00:36 | |
*** mrodden has quit IRC | 00:36 | |
*** yamahata has joined #openstack-qa | 00:36 | |
*** mrodden has joined #openstack-qa | 00:37 | |
jogo | melwitt: yeah sounds like a good idea | 00:43 |
jogo | melwitt: thanks for digging into that bug | 00:43 |
jogo | dtroyer: cool, link? | 00:43 |
dtroyer | https://review.openstack.org/#/c/117339/ | 00:44 |
melwitt | jogo: :) I think I'm not helping much but I'm trying. I'm checking into how to add the paramiko logging so I can put up a patch for it | 00:44 |
jogo | melwitt: awesome | 00:47 |
jogo | dtroyer: cool | 00:49 |
jogo | dtroyer: btw I wonder if https://review.openstack.org/#/c/116484/ is related | 00:49 |
jogo | rather the bug underneat | 00:49 |
jogo | h | 00:49 |
dtroyer | jogo: there's a good chance it is | 00:49 |
jogo | so that would nead a nova fix? | 00:50 |
jogo | can you update the bug with your comments | 00:50 |
dtroyer | I doubt it…if that bug happens outside devstack then it isn't related | 00:50 |
dtroyer | what I'm changing is how we launch services inside devstack, and how we get the pid of those to kill them later | 00:51 |
dtroyer | (only when USE_SCREEN=False) | 00:51 |
*** gmann has joined #openstack-qa | 00:52 | |
jogo | but we use screen in grenade | 00:52 |
jogo | where this bug is happening | 00:52 |
dtroyer | only because of not being able to stop services reliably | 00:52 |
jogo | ahh, so with this we can drop screen in grenade | 00:52 |
dtroyer | that's the idea | 00:52 |
jogo | ahh cool | 00:53 |
*** suneelb has quit IRC | 01:11 | |
openstackgerrit | Hirofumi Ichihara proposed a change to openstack-dev/devstack: Fix keystone auth parameter to neutron conf https://review.openstack.org/118512 | 01:11 |
*** dims has joined #openstack-qa | 01:11 | |
*** mwagner_lap has joined #openstack-qa | 01:14 | |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Migrate baremetal_basic_ops to tempest clients https://review.openstack.org/117673 | 01:16 |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Add baremetal tests for get node by instance/port https://review.openstack.org/118513 | 01:16 |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/grenade: Set required DSG variables (DO NOT MERGE) https://review.openstack.org/118515 | 01:20 |
*** catohornet has quit IRC | 01:23 | |
*** dims has quit IRC | 01:23 | |
*** dims has joined #openstack-qa | 01:23 | |
*** dims_ has joined #openstack-qa | 01:26 | |
*** zz_dimtruck is now known as dimtruck | 01:26 | |
*** dims has quit IRC | 01:28 | |
*** mestery has quit IRC | 01:31 | |
*** mestery has joined #openstack-qa | 01:31 | |
*** sballe has quit IRC | 01:32 | |
*** mestery_ has joined #openstack-qa | 01:35 | |
*** mestery has quit IRC | 01:37 | |
*** yjiang5 has quit IRC | 01:42 | |
gmann | coolsvap: ping | 01:47 |
*** melwitt has quit IRC | 01:48 | |
gmann | coolsvap: one update needed in https://review.openstack.org/#/c/90306/ . please update that. | 01:48 |
gmann | coolsvap: Sorry i forgot to notice that earlier. My bad.... | 01:49 |
*** Sukhdev has joined #openstack-qa | 01:50 | |
*** pcrews has quit IRC | 01:50 | |
*** Sukhdev has quit IRC | 01:51 | |
*** pcrews has joined #openstack-qa | 01:52 | |
*** nosnos has joined #openstack-qa | 01:52 | |
*** dims_ has quit IRC | 02:03 | |
*** dims has joined #openstack-qa | 02:03 | |
*** adalbas has quit IRC | 02:07 | |
*** dims has quit IRC | 02:08 | |
*** mestery_ is now known as mestery | 02:09 | |
*** salv-orlando has quit IRC | 02:10 | |
*** dims has joined #openstack-qa | 02:13 | |
*** nosnos has quit IRC | 02:16 | |
*** pcrews has quit IRC | 02:28 | |
*** bkopilov has quit IRC | 02:35 | |
*** mriedem has quit IRC | 02:42 | |
*** harlowja is now known as harlowja_away | 02:52 | |
*** jyuso has joined #openstack-qa | 02:53 | |
openstackgerrit | Matthew Treinish proposed a change to openstack-infra/subunit2sql: Add subunit2sql-db-manage utility https://review.openstack.org/118418 | 02:54 |
*** gmann has quit IRC | 02:55 | |
*** dims has quit IRC | 02:56 | |
*** dims has joined #openstack-qa | 02:57 | |
*** Madan has joined #openstack-qa | 03:00 | |
*** yjiang5 has joined #openstack-qa | 03:01 | |
*** dims has quit IRC | 03:01 | |
*** jecarey_ has joined #openstack-qa | 03:04 | |
openstackgerrit | YAMAMOTO Takashi proposed a change to openstack-dev/devstack: ofagent: Switch to IptablesFirewallDriver https://review.openstack.org/108915 | 03:16 |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/grenade: Set required DSG variables (DO NOT MERGE) https://review.openstack.org/118515 | 03:18 |
*** cmyster has quit IRC | 03:21 | |
*** dimtruck is now known as zz_dimtruck | 03:26 | |
*** Poornima has joined #openstack-qa | 03:42 | |
*** gmann_ has quit IRC | 03:51 | |
*** Sukhdev has joined #openstack-qa | 03:52 | |
*** gmann has joined #openstack-qa | 03:53 | |
*** bkopilov has joined #openstack-qa | 04:00 | |
*** melwitt has joined #openstack-qa | 04:06 | |
openstackgerrit | Masayuki Igawa proposed a change to openstack/tempest: Migrate test_server_cfn_init to tempest clients https://review.openstack.org/117714 | 04:09 |
*** melwitt1 has joined #openstack-qa | 04:13 | |
*** melwitt has quit IRC | 04:16 | |
*** SridharG has joined #openstack-qa | 04:24 | |
*** SridharG has quit IRC | 04:28 | |
*** Ch00k has joined #openstack-qa | 04:30 | |
*** zz_dimtruck is now known as dimtruck | 04:32 | |
openstackgerrit | Masayuki Igawa proposed a change to openstack/tempest: Fix unstable assertion in test_cinder_endpoints https://review.openstack.org/117742 | 04:34 |
*** ianw has quit IRC | 04:37 | |
*** ianw has joined #openstack-qa | 04:37 | |
*** jecarey_ has quit IRC | 04:41 | |
*** nosnos has joined #openstack-qa | 04:50 | |
*** amotoki has joined #openstack-qa | 04:56 | |
*** yfried_ has quit IRC | 05:00 | |
*** dimtruck is now known as zz_dimtruck | 05:00 | |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Remove resolved fingerprints https://review.openstack.org/117916 | 05:05 |
*** NuclearW0rm has quit IRC | 05:07 | |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/grenade: Set required DSG variables (DO NOT MERGE) https://review.openstack.org/118515 | 05:09 |
*** chandankumar has joined #openstack-qa | 05:13 | |
openstackgerrit | Swapnil Kulkarni proposed a change to openstack/tempest: Adds the clients and tests for CINDER QoS V1 & V2 APIs https://review.openstack.org/90306 | 05:15 |
*** neeti has joined #openstack-qa | 05:17 | |
*** enikanorov__ has quit IRC | 05:23 | |
*** Sukhdev has quit IRC | 05:52 | |
*** SridharG has joined #openstack-qa | 05:52 | |
*** rdekel has joined #openstack-qa | 05:53 | |
openstackgerrit | varun kumar yadav proposed a change to openstack/tempest: Test to update port with CIDR value https://review.openstack.org/113861 | 05:56 |
*** Ch00k has quit IRC | 06:00 | |
*** yjiang5 has quit IRC | 06:08 | |
*** raies has joined #openstack-qa | 06:13 | |
openstackgerrit | A change was merged to openstack/tempest: Fix "mutable" object as default value https://review.openstack.org/100132 | 06:17 |
*** k4n0 has joined #openstack-qa | 06:18 | |
*** afazekas has joined #openstack-qa | 06:18 | |
raies | cyeoh: hi | 06:21 |
cyeoh | raies: hi | 06:21 |
raies | cyeoh: I am installing openstack multi compute node | 06:22 |
raies | node 1 - controller, node 2 - compute 1 , node 3 - compute 2 | 06:22 |
raies | cyeoh: controller install is successful | 06:22 |
raies | compute install is also successful on both compute nodes | 06:23 |
raies | but | 06:23 |
raies | during nova network installation (legacy mode) | 06:23 |
raies | when I tried to install on both compute nodes | 06:23 |
raies | then besides it was successfull, but during network create Networklist InstanceList not supported resulted | 06:24 |
raies | again I remove nova network from compute 2 | 06:24 |
raies | and only install there nova-compute | 06:24 |
raies | and tried to install compute + nova-network on compute 1 | 06:25 |
raies | then following error is encountered in the logs - | 06:25 |
raies | cyeoh: http://pastebin.com/zM9bxDFV | 06:25 |
raies | cyeoh: looking ?? | 06:26 |
cyeoh | I don't know - looks like you have a set of packages for your install that is not self consistent | 06:27 |
cyeoh | but I can't say I know what is really going on with your setup | 06:28 |
raies | okks | 06:29 |
raies | np | 06:29 |
*** flaper87|afk is now known as flaper87 | 06:30 | |
*** yfried_ has joined #openstack-qa | 06:34 | |
*** tkammer has joined #openstack-qa | 06:36 | |
*** cmyster has joined #openstack-qa | 06:40 | |
*** luqas has joined #openstack-qa | 06:41 | |
*** evgenyf has joined #openstack-qa | 06:43 | |
*** aimon has quit IRC | 06:54 | |
*** jhenner has quit IRC | 06:57 | |
*** ildikov has joined #openstack-qa | 06:58 | |
*** chandankumar has quit IRC | 07:07 | |
raies | cyeoh: hi | 07:11 |
raies | anyone -- how to make nova version same across all nodes? | 07:13 |
*** chandan_kumar has joined #openstack-qa | 07:13 | |
*** nmagnezi has joined #openstack-qa | 07:14 | |
*** markus_z has joined #openstack-qa | 07:14 | |
*** jhenner has joined #openstack-qa | 07:19 | |
*** e0ne has joined #openstack-qa | 07:19 | |
*** qba73 has joined #openstack-qa | 07:20 | |
raies | anyone ^^ | 07:20 |
*** flaper87 is now known as flaper87|afk | 07:26 | |
openstackgerrit | A change was merged to openstack-infra/devstack-gate: Add oslo.utils and oslo.concurrency https://review.openstack.org/117307 | 07:28 |
*** shardy_afk is now known as shardy | 07:29 | |
*** lyxus has quit IRC | 07:29 | |
*** lyxus has joined #openstack-qa | 07:31 | |
raies | gmann: hi | 07:31 |
raies | have installed openstack multinode | 07:32 |
raies | gmann: but on controller nova service-list does not show compute service info | 07:32 |
*** yfried_ has quit IRC | 07:32 | |
raies | while compute service is running on compute node | 07:32 |
raies | gmann: any idea | 07:33 |
*** chandan_kumar has quit IRC | 07:33 | |
raies | gmann: It was issue related to amqp, when connected there is error like IncompatibleObjectVersion: Version 1.8 of InstanceList is not supported | 07:35 |
*** yfried has joined #openstack-qa | 07:35 | |
*** wendar_ is now known as wendar | 07:36 | |
*** jhenner has quit IRC | 07:37 | |
*** e0ne has quit IRC | 07:38 | |
*** e0ne has joined #openstack-qa | 07:38 | |
*** flaper87|afk is now known as flaper87 | 07:39 | |
*** marun_ has quit IRC | 07:41 | |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/110110 | 07:42 |
gmann | raies: hi | 07:42 |
*** jlibosva has joined #openstack-qa | 07:43 | |
raies | gmann: sorry problem resolved :) | 07:43 |
gmann | raies: :). what was the issue? | 07:43 |
raies | actually issue was during multinode installation we were unable to acces compute from controller | 07:44 |
raies | gmann: and somethis InstanceList version support isseue was there | 07:44 |
raies | :) | 07:44 |
*** jhenner has joined #openstack-qa | 07:45 | |
gmann | raies: ohk. So how you fixed that | 07:45 |
*** tkammer has quit IRC | 07:46 | |
raies | yes actually there was a conflict of with devstack packages | 07:46 |
*** jlanoux has joined #openstack-qa | 07:46 | |
raies | I removed them and it is working nw | 07:46 |
raies | :) | 07:46 |
*** chandan_kumar has joined #openstack-qa | 07:47 | |
openstackgerrit | A change was merged to openstack-dev/devstack: Add myself to the MAINTAINERS file for Swift https://review.openstack.org/117969 | 07:52 |
*** safchain has joined #openstack-qa | 07:53 | |
*** jhenner has quit IRC | 07:58 | |
*** gszasz has joined #openstack-qa | 07:59 | |
openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/110110 | 08:01 |
*** Ch00k has joined #openstack-qa | 08:01 | |
*** tkammer has joined #openstack-qa | 08:02 | |
gmann | raies: cool | 08:07 |
*** catohornet has joined #openstack-qa | 08:16 | |
*** ZZelle has quit IRC | 08:18 | |
*** ZZelle has joined #openstack-qa | 08:19 | |
*** gszasz has quit IRC | 08:22 | |
*** melwitt1 has quit IRC | 08:24 | |
*** oomichi has quit IRC | 08:25 | |
*** luqas has quit IRC | 08:28 | |
*** chandan_kumar has quit IRC | 08:29 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 08:30 | |
masayukig | Should we stop to approve patches of Tempest except critical bug fixes for now? Because we should preserve the gate queue as ttx said and Tempest doesn't have 'Feature Freeze'. | 08:32 |
*** tkammer has quit IRC | 08:34 | |
*** salv-orlando has joined #openstack-qa | 08:36 | |
*** Longgeek has joined #openstack-qa | 08:39 | |
*** ildikov_ has joined #openstack-qa | 08:44 | |
*** tkammer has joined #openstack-qa | 08:46 | |
*** ildikov has quit IRC | 08:47 | |
*** mpaolino has joined #openstack-qa | 08:51 | |
*** amotoki has quit IRC | 08:52 | |
*** Longgeek has quit IRC | 08:53 | |
*** Longgeek has joined #openstack-qa | 08:53 | |
yfried | masayukig: +1 | 08:53 |
*** andreaf has joined #openstack-qa | 08:53 | |
yfried | masayukig: seems like the gate is stuck as it is. it's now almost 24hr for a single merge. no reason adding new stuf | 08:54 |
yfried | stuff | 08:54 |
yfried | gmann: remember our talk yesterday about large_ops and connectivity checks? | 08:55 |
gmann | yfried: yes | 08:55 |
*** gszasz has joined #openstack-qa | 08:56 | |
gmann | masayukig: Agree. +1 | 08:56 |
yfried | gmann: so I'm testing it with 10 VMs. it's 66sec for no connectivity and 120sec with connectivity. though I think that it's not a 1:2 issue, but rather X+60sec issue since it's ~60 sec for a VM to achieve connectivity. and all VMs are doing this in parallel, so they should all be ready by the time the first few are ready | 08:58 |
*** masayukig has quit IRC | 09:00 | |
*** masayukig has joined #openstack-qa | 09:00 | |
yfried | gmann: I wonder how does it run with 100 VMs at the gate. quota is 10 and you need tenant id to change it. how do you change quota for all tenants | 09:01 |
gmann | yfried: I see. +~60 sec should not be issue i think. | 09:02 |
*** luqas has joined #openstack-qa | 09:04 | |
yfried | gmann: I can't check it on large scale without changing quotas for all tenants. how do I do that? | 09:04 |
yfried | gmann: nm - figured it out. gonna run some numbers now :) | 09:09 |
gmann | yfried: thats need to be changed in nova. if you want to tests with 100 that can be done by changing - https://github.com/openstack/nova/blob/master/nova/quota.py | 09:09 |
gmann | yfried: how ? by quota change in nova? | 09:10 |
yfried | gmann: nova.conf has default quotas | 09:10 |
yfried | that's the link you just sune | 09:10 |
yfried | sent | 09:10 |
gmann | yfried: just give try. but with 100 VM quota it might need more ram and other resources in your dev environment | 09:12 |
yfried | gmann: my setup is a baremetal machine and not a VM | 09:13 |
gmann | yfried: ohk | 09:13 |
*** aix has joined #openstack-qa | 09:13 | |
gmann | yfried: i think if that work with large scale VM number. its a good to enhance that tests with connectivity check. | 09:14 |
gmann | yfried: have to go. ll catch u later. best of luck :) | 09:15 |
*** cdent has joined #openstack-qa | 09:15 | |
*** neeti has quit IRC | 09:18 | |
*** neeti has joined #openstack-qa | 09:21 | |
*** jhenner has joined #openstack-qa | 09:21 | |
*** mkollaro has joined #openstack-qa | 09:21 | |
openstackgerrit | Santosh Kumar proposed a change to openstack/tempest: Check port's status also in show port tests. https://review.openstack.org/116771 | 09:21 |
*** Longgeek has quit IRC | 09:24 | |
*** evgenyf has quit IRC | 09:27 | |
*** Longgeek has joined #openstack-qa | 09:27 | |
*** evgenyf has joined #openstack-qa | 09:28 | |
*** e0ne has quit IRC | 09:35 | |
*** jhenner has quit IRC | 09:36 | |
*** dmorita has quit IRC | 09:37 | |
*** marun_ has joined #openstack-qa | 09:39 | |
sergeysh | hi | 09:39 |
sergeysh | I'll appreciate your review in very short patches, please: https://review.openstack.org/#/c/118424/, https://review.openstack.org/#/c/117258/ | 09:39 |
*** neeti has quit IRC | 09:40 | |
*** e0ne has joined #openstack-qa | 09:41 | |
*** Santosh_ has joined #openstack-qa | 09:49 | |
*** neeti has joined #openstack-qa | 09:51 | |
Santosh_ | hi there... | 09:51 |
*** tosky has joined #openstack-qa | 09:52 | |
Santosh_ | gmann: hi | 09:55 |
nmagnezi | masayukig, thanks for the review! (https://review.openstack.org/#/c/117454/) --> will make sure to follow the commit msg guidelines in my next patches | 10:02 |
*** jhenner has joined #openstack-qa | 10:05 | |
*** jhenner has quit IRC | 10:10 | |
*** yamahata has quit IRC | 10:11 | |
*** jhenner has joined #openstack-qa | 10:16 | |
coolsvap | cyeoh, hi | 10:19 |
cyeoh | coolsvap: hi | 10:20 |
coolsvap | cyeoh, can you please have a look at https://review.openstack.org/90306 and https://review.openstack.org/115871 | 10:20 |
cyeoh | coolsvap: I'll put them on the list.... | 10:21 |
coolsvap | cyeoh, thanks | 10:21 |
cyeoh | I'd already commented on 115871 btw | 10:22 |
*** e0ne has quit IRC | 10:24 | |
*** Longgeek has quit IRC | 10:31 | |
*** Longgeek has joined #openstack-qa | 10:32 | |
*** jhenner has quit IRC | 10:33 | |
openstackgerrit | A change was merged to openstack/tempest: Migrate test_network_basic_ops to tempest clients https://review.openstack.org/111719 | 10:33 |
masayukig | nmagnezi: you're welcome :) | 10:34 |
openstackgerrit | A change was merged to openstack/tempest: Add a check_telemetry method to javelin https://review.openstack.org/102354 | 10:34 |
*** jlanoux has quit IRC | 10:35 | |
*** Longgeek_ has joined #openstack-qa | 10:36 | |
*** masayukig has quit IRC | 10:39 | |
*** Longgeek has quit IRC | 10:39 | |
*** e0ne has joined #openstack-qa | 10:39 | |
*** Poornima has quit IRC | 10:54 | |
*** jhenner has joined #openstack-qa | 10:56 | |
*** Poornima has joined #openstack-qa | 10:57 | |
openstackgerrit | Nir Magnezi proposed a change to openstack/tempest: Adds status check for router Port in scenarios https://review.openstack.org/117454 | 10:59 |
*** fungi has quit IRC | 11:03 | |
*** comstud has quit IRC | 11:03 | |
*** dims has joined #openstack-qa | 11:06 | |
*** jyuso has quit IRC | 11:08 | |
*** comstud has joined #openstack-qa | 11:08 | |
*** marun_ has quit IRC | 11:10 | |
*** marun has joined #openstack-qa | 11:10 | |
*** dims has quit IRC | 11:12 | |
*** dims has joined #openstack-qa | 11:13 | |
*** Poornima has quit IRC | 11:14 | |
*** dims_ has joined #openstack-qa | 11:14 | |
*** fungi has joined #openstack-qa | 11:17 | |
*** dims has quit IRC | 11:17 | |
openstackgerrit | Santosh Kumar proposed a change to openstack/tempest: Check port's status also in show port tests https://review.openstack.org/116771 | 11:20 |
*** mpaolino has quit IRC | 11:21 | |
*** Poornima has joined #openstack-qa | 11:23 | |
*** cdent has quit IRC | 11:26 | |
openstackgerrit | Flavio Percoco proposed a change to openstack-dev/devstack: Set configs for glance_store https://review.openstack.org/118609 | 11:28 |
*** dims_ has quit IRC | 11:29 | |
*** dims has joined #openstack-qa | 11:29 | |
*** masayukig has joined #openstack-qa | 11:31 | |
*** dims has quit IRC | 11:34 | |
*** dims has joined #openstack-qa | 11:37 | |
*** jlanoux has joined #openstack-qa | 11:39 | |
*** masayukig_mobi has joined #openstack-qa | 11:40 | |
*** mmedvede has joined #openstack-qa | 11:40 | |
*** masayukig_mobi has quit IRC | 11:42 | |
*** yamahata has joined #openstack-qa | 11:45 | |
*** masayukig_mobi has joined #openstack-qa | 11:46 | |
*** masayukig has quit IRC | 11:48 | |
*** masayukig has joined #openstack-qa | 11:48 | |
*** dims has quit IRC | 11:50 | |
*** dims has joined #openstack-qa | 11:50 | |
*** dims_ has joined #openstack-qa | 11:51 | |
*** Longgeek_ has quit IRC | 11:53 | |
*** masayukig_mobi has quit IRC | 11:53 | |
*** Longgeek has joined #openstack-qa | 11:54 | |
*** dims has quit IRC | 11:54 | |
*** Longgeek_ has joined #openstack-qa | 11:54 | |
*** marun_ has joined #openstack-qa | 11:56 | |
*** yamahata has quit IRC | 11:58 | |
*** Longgeek has quit IRC | 11:58 | |
*** marun has quit IRC | 11:59 | |
*** yamahata has joined #openstack-qa | 12:00 | |
*** jlanoux_ has joined #openstack-qa | 12:00 | |
*** jlanoux has quit IRC | 12:01 | |
*** markmcclain has joined #openstack-qa | 12:02 | |
*** markmcclain has quit IRC | 12:02 | |
*** HenryG is now known as HenryG_afk | 12:04 | |
*** markmcclain has joined #openstack-qa | 12:04 | |
*** tkammer has quit IRC | 12:04 | |
*** aysyd has joined #openstack-qa | 12:16 | |
*** nosnos has quit IRC | 12:16 | |
*** nosnos has joined #openstack-qa | 12:17 | |
openstackgerrit | Roman Bogorodskiy proposed a change to openstack/tempest: Add coraid_repository_key config variable https://review.openstack.org/118619 | 12:17 |
*** Madan has quit IRC | 12:18 | |
*** jlanoux has joined #openstack-qa | 12:18 | |
*** Longgeek_ has quit IRC | 12:19 | |
*** tkammer has joined #openstack-qa | 12:20 | |
*** jlanoux_ has quit IRC | 12:21 | |
*** nosnos has quit IRC | 12:21 | |
*** adalbas has joined #openstack-qa | 12:21 | |
*** dkranz has joined #openstack-qa | 12:21 | |
*** dims_ has quit IRC | 12:29 | |
*** dims has joined #openstack-qa | 12:30 | |
*** Longgeek has joined #openstack-qa | 12:30 | |
*** cnesa10 has quit IRC | 12:30 | |
*** e0ne has quit IRC | 12:31 | |
*** dims_ has joined #openstack-qa | 12:32 | |
*** evgenyf has quit IRC | 12:34 | |
*** dims has quit IRC | 12:34 | |
*** e0ne has joined #openstack-qa | 12:34 | |
*** cdent has joined #openstack-qa | 12:36 | |
*** tkammer has quit IRC | 12:37 | |
ekhugen | andreaf, I saw your comments on https://review.openstack.org/#/c/118469/ I was putting that in because I picked up the bug. I'm fine with it waiting until the test-accounts bp goes in, if the bug will be closed with that? Thanks. | 12:40 |
andreaf | ekhugen: ok - I can the bug id to my patch so it's tracked there | 12:44 |
ekhugen | andreaf, okay, I'll abandon the patch then, that's what you'd prefer? | 12:44 |
*** freyes has quit IRC | 12:45 | |
*** dims_ has quit IRC | 12:45 | |
andreaf | ekhugen: just one question - the LP bug, is that an actual bug or just a nice improvement to have? Does it cause any actual failure? | 12:45 |
*** freyes has joined #openstack-qa | 12:46 | |
andreaf | ekhugen: if it does we should fix that before the test-accounts bp lands because that may take still a bit of time | 12:46 |
*** dims has joined #openstack-qa | 12:46 | |
ekhugen | andreaf, no the lp bug was just to streamline the tenant isolation code, no actual bug | 12:46 |
andreaf | ekhugen: ok than I'm fine with you abandoning your patch | 12:47 |
andreaf | ekhugen: thanks for working on that anyways | 12:48 |
ekhugen | andreaf, you don't need any help with the bp, do you? I was looking for something new to work on | 12:48 |
andreaf | ekhugen: the bp is basically done, only it uncovered other shortcomings in tempest code, so it can't be merged until such issues are solved or bypassed somehow | 12:49 |
mkollaro | is there some way to disable s3 tests? they get run even if swift is disabled | 12:49 |
andreaf | ekhugen: I'm working on a patch for the bypass approach | 12:49 |
*** tkammer has joined #openstack-qa | 12:49 | |
andreaf | ekhugen, and the actual solution is a brand new bp | 12:49 |
andreaf | ekhugen, which will need volunteers once the spec is reviewed / approved | 12:50 |
andreaf | https://blueprints.launchpad.net/tempest/+spec/resource-cleanup | 12:50 |
ekhugen | andreaf, okay, I'll keep an eye on that bp, thanks! and look for something else in the meantime | 12:51 |
andreaf | ekhugen: thank you! | 12:51 |
andreaf | mkollaro: that sounds like a bug | 12:54 |
*** cdent has quit IRC | 12:56 | |
*** vponomaryov has quit IRC | 12:57 | |
*** aix has quit IRC | 12:58 | |
mkollaro | andreaf: but maybe it is fixed already...I'm forced to use a 5 month old version of tempest | 12:59 |
*** dkranz has quit IRC | 12:59 | |
mkollaro | in any case, does anybody have an idea how to debug the "S3ResponseError: S3ResponseError: 412 Precondition Failed" error in create_bucket? because I have no idea where to go from there | 13:00 |
*** enikanorov has joined #openstack-qa | 13:01 | |
*** aix has joined #openstack-qa | 13:01 | |
mkollaro | the boto client class has create_bucket in ALLOWED_METHODS and that's it...I don't know where the implementation is | 13:01 |
*** timello_ has joined #openstack-qa | 13:02 | |
*** bkopilov has quit IRC | 13:04 | |
*** zz_dimtruck is now known as dimtruck | 13:07 | |
*** timello_ has quit IRC | 13:09 | |
*** chandankumar has joined #openstack-qa | 13:12 | |
*** toure has joined #openstack-qa | 13:15 | |
*** dkranz has joined #openstack-qa | 13:15 | |
*** enikanorov__ has joined #openstack-qa | 13:17 | |
*** timello_ has joined #openstack-qa | 13:18 | |
*** enikanorov has quit IRC | 13:19 | |
*** mriedem has joined #openstack-qa | 13:21 | |
openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Add Credentials Provider factory https://review.openstack.org/107685 | 13:29 |
openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Force use of Accounts for test purposes https://review.openstack.org/114578 | 13:29 |
openstackgerrit | Andrea Frittoli proposed a change to openstack/tempest: Add a NotLockingAccounts credentials provider https://review.openstack.org/118635 | 13:29 |
andreaf | mtreinish: so here's my alternative to go ahead and merge the test-accounts work - preserving the functionality of the existing serial runs | 13:32 |
andreaf | mtreinish: let me know what you think. The only other problem left to solve is the migration from configured accounts to YAML file | 13:33 |
*** dpaterson has joined #openstack-qa | 13:34 | |
andreaf | mtreinish: see https://review.openstack.org/118635 and https://review.openstack.org/114578 - and I've dropped the safe_setup patch from the list for now | 13:34 |
mtreinish | andreaf: hmm, well we could handle that in code and just mark the old cred options as deprecated somehow | 13:35 |
mtreinish | like if there isn't a yaml file option specified pull it from config the old way | 13:36 |
mtreinish | but still use the new mechanism for grabbing the creds | 13:36 |
andreaf | mtreinish: yes that sounds like a good option | 13:36 |
andreaf | mtreinish: else apart from the periodic job tests will stop working for people not using isolated creds | 13:37 |
mtreinish | andreaf: yeah we can't really break that without advanced notice because I know it's used a lot | 13:38 |
andreaf | mtreinish: so in this way only part of the new code is actually in use but we can close the bp and work on the resource cleanup one, and avoid endless rebases on test accounts | 13:38 |
*** Ch00k has quit IRC | 13:41 | |
mtreinish | andreaf: I'm thinking we should probably keep it open until we actually get it working, just retarget it and mark it as dependent on resource cleanup bp | 13:41 |
*** Ch00k has joined #openstack-qa | 13:42 | |
*** malini has left #openstack-qa | 13:42 | |
*** e0ne has quit IRC | 13:43 | |
openstackgerrit | Roman Bogorodskiy proposed a change to openstack/tempest: Add coraid_repository_key config variable https://review.openstack.org/118619 | 13:44 |
*** e0ne has joined #openstack-qa | 13:50 | |
openstackgerrit | Roman Bogorodskiy proposed a change to openstack/tempest: Add coraid_repository_key config variable https://review.openstack.org/118619 | 13:52 |
*** dimtruck is now known as zz_dimtruck | 13:53 | |
andreaf | mtreinish: I think the advantages of merging it now would be: 1) start making people aware of the YAML file for credentials and allow for migration 2) merge the code that cleans up all the checks about tenant isolation in the code 3) avoid rebase nightmares on this | 13:56 |
*** e0ne has quit IRC | 13:56 | |
*** cdent has joined #openstack-qa | 13:56 | |
*** dhellmann has quit IRC | 13:57 | |
*** dhellmann has joined #openstack-qa | 13:58 | |
*** e0ne has joined #openstack-qa | 13:58 | |
mtreinish | andreaf: I was just talking about the bp | 13:59 |
mtreinish | I agree merging things now makes sense | 13:59 |
*** nsaje_ has joined #openstack-qa | 14:00 | |
*** rdekel has quit IRC | 14:00 | |
*** nsaje_ is now known as nsaje | 14:00 | |
andreaf | mtreinish: ah ok - yes it makes sense to keep the bp open | 14:02 |
andreaf | mtreinish: ok let me know what you think about the NotLockingAccounts - can you work on the fallback to credentials in conf, or shall I do that? | 14:03 |
*** pcrews has joined #openstack-qa | 14:04 | |
mtreinish | andreaf: if you could tackle it that'd probably be quicker right now | 14:05 |
mtreinish | I might be able to get to it this afternoon | 14:06 |
andreaf | mtreinish: I'll try later tonight else tomorrow | 14:07 |
*** chandankumar has quit IRC | 14:07 | |
*** yfried has quit IRC | 14:09 | |
*** rbak has joined #openstack-qa | 14:09 | |
*** zz_jgrimm is now known as jgrimm | 14:09 | |
*** zz_dimtruck is now known as dimtruck | 14:10 | |
andreaf | mtreinish: about the scenario migration bp - there are a few scenario tests which are not run in the gate right now - I would vote for migrating them anyways, so we can cleanup all the code related to OfficialClients and close the bp | 14:14 |
mtreinish | andreaf: yeah I think we should migrate them too, but we need to show they work | 14:15 |
mtreinish | probably we should add WIP test patch on top of those which drop the skips so we can verify they run correctly | 14:15 |
mtreinish | IIRC most of the skips were added because of unreliability in the gate at scale but a one off run should be reliable enough (assuming things didn't bitrot) | 14:16 |
*** chandankumar has joined #openstack-qa | 14:16 | |
andreaf | mtreinish: ok | 14:17 |
andreaf | mtreinish: I don't see the skips in the code :S | 14:19 |
mtreinish | andreaf: heh, which scenario tests are we skipping in the gate right now? | 14:20 |
andreaf | according to https://etherpad.openstack.org/p/tempest-client-scenarios it's test_stamp_pattern.py , the baremetal one and the autoscaling one | 14:21 |
andreaf | so I guess baremetal scenario only runs in jobs with ironic virt driver enabled | 14:22 |
mtreinish | andreaf: yep, and the autoscaling one only get's run on the heat slow job | 14:23 |
openstackgerrit | vigneshvar proposed a change to openstack-dev/devstack: Reset no_proxy when the node is installed behind a proxy https://review.openstack.org/117985 | 14:23 |
mtreinish | and test_stamp_pattern has a skip because of bug 1205344: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_stamp_pattern.py#n153 | 14:23 |
andreaf | ok so that's the only one actually skipped | 14:24 |
andreaf | mtreinish, i fixed the etherpad | 14:26 |
*** andreaf has quit IRC | 14:28 | |
openstackgerrit | Matthew Treinish proposed a change to openstack-infra/subunit2sql: Update pep8 ignore list https://review.openstack.org/118650 | 14:33 |
*** jecarey has joined #openstack-qa | 14:33 | |
openstackgerrit | Flavio Percoco proposed a change to openstack-dev/devstack: Set configs for glance_store https://review.openstack.org/118609 | 14:33 |
sdague | mtreinish: why _hash in - https://review.openstack.org/#/c/114636/12 | 14:35 |
*** HenryG_afk is now known as HenryG | 14:36 | |
sdague | flaper87: there are still some hard tabs in your patch | 14:36 |
*** ildikov has joined #openstack-qa | 14:36 | |
sdague | so it's going to fail again | 14:36 |
flaper87 | sdague: mmh, damn. let me fix that | 14:36 |
flaper87 | sdague: thanks for the heads up | 14:36 |
mtreinish | sdague: hmm, I didn't push it that way? andreaf did you change that? | 14:36 |
mtreinish | oh, he's offline now... | 14:36 |
openstackgerrit | Flavio Percoco proposed a change to openstack-dev/devstack: Set configs for glance_store https://review.openstack.org/118609 | 14:37 |
*** ildikov_ has quit IRC | 14:40 | |
*** ildikov has quit IRC | 14:41 | |
sdague | flaper87: cool, +2 | 14:42 |
flaper87 | sdague: thanks :) | 14:42 |
flaper87 | one more +2 to go (and wait for jenkins) | 14:42 |
sdague | are all the rest of the glance_store conversion patches for devstack / devstack-gate sorted? | 14:43 |
sdague | or are there things I can look at there to help review | 14:43 |
*** yfried has joined #openstack-qa | 14:44 | |
*** Santosh has joined #openstack-qa | 14:46 | |
openstackgerrit | Alexei Vinogradov proposed a change to openstack-dev/devstack: Add support of hbase database https://review.openstack.org/86683 | 14:46 |
*** atiwari has joined #openstack-qa | 14:47 | |
flaper87 | sdague: that's the only one I have, thanks | 14:48 |
sdague | ok, cool. | 14:48 |
openstackgerrit | vigneshvar proposed a change to openstack-dev/devstack: Reset no_proxy when the node is installed behind a proxy https://review.openstack.org/117985 | 14:50 |
*** mriedem has quit IRC | 15:00 | |
*** mriedem has joined #openstack-qa | 15:01 | |
*** neeti has quit IRC | 15:01 | |
*** neeti has joined #openstack-qa | 15:02 | |
*** Santosh has quit IRC | 15:04 | |
*** markus_z has quit IRC | 15:11 | |
*** SridharG has left #openstack-qa | 15:24 | |
flaper87 | ianw: hey, when you get a chance, could you take a look at? https://review.openstack.org/#/c/118609/ | 15:24 |
*** Sukhdev has joined #openstack-qa | 15:26 | |
*** aix has quit IRC | 15:29 | |
*** qba73 has quit IRC | 15:31 | |
openstackgerrit | Sean Dague proposed a change to openstack-dev/devstack: don't install boto from packages https://review.openstack.org/118661 | 15:32 |
*** amotoki has joined #openstack-qa | 15:34 | |
*** Ch00k has quit IRC | 15:37 | |
*** yamahata has quit IRC | 15:39 | |
*** andreaf has joined #openstack-qa | 15:39 | |
*** johnthetubaguy is now known as zz_johnthetubagu | 15:40 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 15:41 | |
*** tkammer has quit IRC | 15:42 | |
*** andreaf_ has joined #openstack-qa | 15:42 | |
*** andreaf has quit IRC | 15:45 | |
*** neeti has quit IRC | 15:46 | |
openstackgerrit | A change was merged to openstack-infra/subunit2sql: Update pep8 ignore list https://review.openstack.org/118650 | 15:47 |
*** aix has joined #openstack-qa | 15:48 | |
*** Poornima has quit IRC | 15:49 | |
*** aimon has joined #openstack-qa | 15:53 | |
*** mlavalle has joined #openstack-qa | 15:53 | |
openstackgerrit | Dolph Mathews proposed a change to openstack-infra/elastic-recheck: Add query for critical keystone regression https://review.openstack.org/118668 | 15:54 |
*** mlavalle has quit IRC | 15:54 | |
*** mlavalle has joined #openstack-qa | 15:55 | |
*** chandankumar has quit IRC | 15:55 | |
*** gszasz has quit IRC | 15:59 | |
*** enikanorov has joined #openstack-qa | 16:00 | |
*** mdenny has joined #openstack-qa | 16:01 | |
*** Longgeek has quit IRC | 16:02 | |
*** sweston_ has joined #openstack-qa | 16:03 | |
*** cbkyeoh has joined #openstack-qa | 16:03 | |
*** eglynn__ has joined #openstack-qa | 16:04 | |
ZZelle | mtreinish, hi | 16:05 |
*** tristanC_ has joined #openstack-qa | 16:06 | |
*** afazekas has quit IRC | 16:07 | |
*** masayukig_ has joined #openstack-qa | 16:07 | |
*** dosaboy_ has joined #openstack-qa | 16:07 | |
*** k4n0_ has joined #openstack-qa | 16:08 | |
*** sweston has quit IRC | 16:08 | |
*** cyeoh has quit IRC | 16:08 | |
*** swartulv has quit IRC | 16:08 | |
*** k4n0 has quit IRC | 16:08 | |
*** sileht has quit IRC | 16:08 | |
*** rossella_s has quit IRC | 16:08 | |
*** eglynn has quit IRC | 16:08 | |
*** tristanC has quit IRC | 16:08 | |
*** danfinch has quit IRC | 16:08 | |
*** enikanorov_ has quit IRC | 16:08 | |
*** freyes has quit IRC | 16:08 | |
*** StevenK has quit IRC | 16:08 | |
*** luqas has quit IRC | 16:08 | |
*** maurosr has quit IRC | 16:08 | |
*** dosaboy has quit IRC | 16:08 | |
*** masayukig has quit IRC | 16:08 | |
*** krtaylor has quit IRC | 16:08 | |
*** johnthetubaguy has quit IRC | 16:08 | |
*** cbkyeoh is now known as cyeoh | 16:08 | |
*** sweston_ is now known as sweston | 16:08 | |
*** tristanC_ is now known as tristanC | 16:08 | |
*** luqas has joined #openstack-qa | 16:09 | |
*** rossella_s has joined #openstack-qa | 16:09 | |
*** johnthetubaguy has joined #openstack-qa | 16:09 | |
*** maurosr has joined #openstack-qa | 16:09 | |
*** danfinch has joined #openstack-qa | 16:09 | |
*** krtaylor has joined #openstack-qa | 16:10 | |
*** freyes has joined #openstack-qa | 16:10 | |
*** sileht has joined #openstack-qa | 16:11 | |
*** krtaylor has quit IRC | 16:11 | |
*** swartulv has joined #openstack-qa | 16:12 | |
*** StevenK has joined #openstack-qa | 16:13 | |
*** andreaf_ is now known as andreaf | 16:17 | |
mtreinish | ZZelle: hi | 16:18 |
*** chandankumar has joined #openstack-qa | 16:19 | |
ZZelle | mtreinish, about https://review.openstack.org/103498 | 16:19 |
ZZelle | mtreinish, , because specific attributes (depending) on the deployment) are required to create flat/vlan/gre/vxlan networks | 16:20 |
ZZelle | mtreinish, i can not enable their testing by default | 16:20 |
mtreinish | ZZelle: that makes sense, but at the same time we need verification that the code works now and will continue to work moving forward | 16:21 |
ZZelle | mtreinish, for local networks, vmware does not support it and is not always available (depends on the deployment config/plugin) | 16:21 |
mtreinish | and if it's not run as part of a ci job we don't have that | 16:21 |
ZZelle | mtreinish, the must i can do is to try create a local network and if it succeed, test local networks otherwise skip tests | 16:22 |
*** mwagner_lap has quit IRC | 16:22 | |
ZZelle | mtreinish, that's why every test inherits from the same | 16:22 |
mtreinish | ZZelle: I'd also touch base with sc68cal, because at one point we had discussed having an extra devstack configs for testing provider networks. | 16:26 |
mtreinish | I don't remember all the details of that discussion though | 16:26 |
*** safchain has quit IRC | 16:28 | |
ZZelle | If local network support is enabled, we can check "must" of the code except the wrapper for flat/vlan/gre/vxlan tests which seems removable using testscenario | 16:31 |
ZZelle | mtreinish, and parse_raw_spec will be parsely covered (but i can add a unit test to check its behavior) | 16:32 |
mtreinish | ZZelle: yes, that's one of the advantages of using testscenarios :) | 16:32 |
*** jlanoux has quit IRC | 16:33 | |
*** tkammer has joined #openstack-qa | 16:33 | |
mtreinish | ZZelle: yeah that should provide decent enough coverage to move forward on that patch | 16:33 |
ZZelle | the same trouble will append with its son change as it need multiple distinct network segment specs (which is not possible with only local networks) to test everything | 16:35 |
*** krtaylor has joined #openstack-qa | 16:37 | |
*** luqas has quit IRC | 16:37 | |
*** jlibosva has quit IRC | 16:38 | |
*** mriedem1 has joined #openstack-qa | 16:40 | |
openstackgerrit | vigneshvar proposed a change to openstack-dev/devstack: Reset no_proxy when the node is installed behind a proxy https://review.openstack.org/117985 | 16:42 |
*** dkranz has quit IRC | 16:43 | |
*** mriedem has quit IRC | 16:44 | |
*** arnaud has joined #openstack-qa | 16:45 | |
*** tkammer has quit IRC | 16:48 | |
*** chandankumar has quit IRC | 16:48 | |
*** suneelb has joined #openstack-qa | 16:51 | |
*** johnthetubaguy is now known as zz_johnthetubagu | 16:58 | |
*** bkopilov has joined #openstack-qa | 16:59 | |
*** marun_ has quit IRC | 17:00 | |
*** yjiang5 has joined #openstack-qa | 17:02 | |
*** gema has quit IRC | 17:02 | |
*** tosky has quit IRC | 17:04 | |
*** dkranz has joined #openstack-qa | 17:09 | |
*** amotoki has quit IRC | 17:10 | |
*** marun_ has joined #openstack-qa | 17:14 | |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for critical keystone regression https://review.openstack.org/118668 | 17:14 |
*** mwagner_lap has joined #openstack-qa | 17:16 | |
*** yjiang5 has quit IRC | 17:18 | |
*** harlowja_away is now known as harlowja | 17:20 | |
*** yjiang5 has joined #openstack-qa | 17:23 | |
*** gema has joined #openstack-qa | 17:25 | |
openstackgerrit | Adam Gandelman proposed a change to openstack-infra/devstack-gate: Set required devstack variables for Ironic+grenade https://review.openstack.org/116761 | 17:26 |
*** Sukhdev has quit IRC | 17:26 | |
*** yjiang5 has quit IRC | 17:28 | |
e0ne | hi all! could you review https://review.openstack.org/#/c/116290/, please? | 17:30 |
*** chandankumar has joined #openstack-qa | 17:30 | |
adam_g | mtreinish, goodbye DEVSTACK_GATE_TEMPEST_REGEX https://review.openstack.org/116761 | 17:31 |
*** yjiang5 has joined #openstack-qa | 17:33 | |
mtreinish | adam_g: did you mean: https://review.openstack.org/#/c/118700/1 :) | 17:34 |
mtreinish | but, it's good to finally get rid of using that | 17:34 |
adam_g | mtreinish, oh yea :) | 17:34 |
*** e0ne has quit IRC | 17:35 | |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/grenade: Set required DSG variables for Ironic DO NOT MERGE https://review.openstack.org/118515 | 17:35 |
*** tkammer has joined #openstack-qa | 17:38 | |
*** chandankumar has quit IRC | 17:46 | |
openstackgerrit | A change was merged to openstack/tempest: Removes test_force_delete_server_invalid_state negative test https://review.openstack.org/112445 | 17:46 |
*** pcrews has quit IRC | 17:47 | |
*** morganfainberg is now known as morganfainberg_Z | 17:53 | |
*** arnaud has quit IRC | 17:53 | |
*** mkollaro has quit IRC | 17:54 | |
clarkb | the ERRORs in log check is being tripped in places in the gate https://jenkins02.openstack.org/job/gate-grenade-dsvm/12009/console how do we typically treat those problems? is that open a bug against the services that logged the ERRORs? | 17:58 |
*** chandankumar has joined #openstack-qa | 17:59 | |
*** aix has quit IRC | 18:00 | |
mtreinish | clarkb: yeah that's how I've always handled it | 18:01 |
mtreinish | although I think I've seen that n-cond bug error in the logs before | 18:01 |
mtreinish | mriedem1: ^^^ did you open a bug for it? | 18:01 |
*** weshay has joined #openstack-qa | 18:03 | |
mriedem1 | mtreinish: clarkb: was talking with anteaya about that last week, it's a known bug 1353131 | 18:03 |
mriedem1 | the query we have is on the n-cond log, not console log | 18:04 |
mtreinish | mriedem1: yeah, that's probably the right place for the query | 18:04 |
clarkb | mriedem1: great, I just wasn't sure how we were dealing with those and agreed it should be in the n-cond log not console log | 18:05 |
mriedem1 | having said that, we frequently fail on the log scrub check | 18:05 |
mriedem1 | but e-r doesn't hit it | 18:05 |
clarkb | mriedem1: is that because the scrub check is failign on additional things? | 18:06 |
mriedem1 | http://logs.openstack.org/77/118477/1/gate/gate-grenade-dsvm/dc695eb/logs/new/screen-n-cond.txt.gz#_2014-09-03_17_10_01_100 | 18:06 |
mriedem1 | the e-r query should be hitting on that | 18:06 |
mriedem1 | this is the query: message:"Failed to commit reservations" AND tags:"screen-n-cond.txt" AND loglevel:"ERROR" | 18:06 |
*** mriedem1 is now known as mriedem | 18:07 | |
clarkb | hrm there are INFO lines for that file but not the ERROR | 18:09 |
mriedem | http://logs.openstack.org/77/118477/1/gate/gate-grenade-dsvm/dc695eb/logs/new/screen-n-cond.txt.gz?level=ERROR#_2014-09-03_17_10_01_100 | 18:09 |
mriedem | ? | 18:09 |
clarkb | sorry in logstash | 18:09 |
mriedem | ah | 18:09 |
*** marun_ has quit IRC | 18:09 | |
clarkb | we may have a parsing problem | 18:10 |
*** e0ne has joined #openstack-qa | 18:10 | |
clarkb | (it does show up in the console log which has much looser parsing) | 18:10 |
mriedem | oh, and for some reason if you query on the failure in the console log, we don't get 100% failure in logstash, i think that's why i've left it out | 18:11 |
mriedem | are there some jobs that don't have the log scrub check? | 18:11 |
mtreinish | mriedem: hmm, I thought it would be on all the dsvm jobs | 18:11 |
mriedem | .2% success | 18:12 |
mtreinish | mriedem: hmm, which jobs does it succeed on? Sounds like a gap that needs to plugged | 18:13 |
mriedem | gate-rally-dsvm-rally | 18:13 |
mtreinish | oh, that's not gating so it doesn't matter | 18:13 |
mriedem | and e-r doesn't filter out voting/non-voting jobs from what i remember | 18:13 |
mtreinish | nope, not unless you do it in the query | 18:14 |
mriedem | right | 18:14 |
mtreinish | we bake that into the e-r queries | 18:14 |
mriedem | anyway, i rechecked that change | 18:15 |
mriedem | and i think the oslo.db change will be approved this week | 18:15 |
mtreinish | oh, cool | 18:15 |
clarkb | neutron doesn't fail on the scrubber | 18:16 |
clarkb | so all of the neutron tempest jobs | 18:16 |
clarkb | anyways it looks like I need to do logstash debugging :/ hopefully I can figure out why we aren't getting that ERROR but moeny is on multiline blowing up | 18:17 |
*** mspreitz has joined #openstack-qa | 18:17 | |
*** masayukig_ has quit IRC | 18:18 | |
clarkb | mriedem: mtreinish: you could do something like AND build_status:FAILURE as a hack to only get the failed jobs when checking the scrubbed list | 18:19 |
clarkb | since not all jobs fail on it, but those that do fail almost certainly fail at least partially due to the scrubber | 18:19 |
*** adalbas has quit IRC | 18:22 | |
mtreinish | clarkb: wouldn't that just mask cases where the bug occured? Isn't the issue that the scrubber isn't failing where it does everywhere else | 18:23 |
clarkb | mtreinish: hrm good point so I guess tracking it on the success side is fine | 18:23 |
*** tkammer has quit IRC | 18:23 | |
mtreinish | clarkb: hmm, looking at the jjb configs only one dsvm job, the periodic stress job, turns off the scrubber faliures | 18:24 |
mtreinish | so if it hits an error in a designated clean log it should fail everywhere | 18:25 |
*** masayukig has joined #openstack-qa | 18:25 | |
clarkb | oh we only got the old side logs not the new side logs. ok starting to make progress on this | 18:25 |
clarkb | mtreinish: neutron does it | 18:25 |
clarkb | mtreinish: its done in devstack gate | 18:25 |
*** chandankumar has quit IRC | 18:26 | |
clarkb | oh nevermind all that code was rewritten at some point | 18:26 |
clarkb | I swear I have seen neutron jobs pass with non whitelisted errors though | 18:27 |
mtreinish | clarkb: only some of the log files are designated to fail | 18:27 |
clarkb | wow ok | 18:27 |
*** arnaud has joined #openstack-qa | 18:27 | |
clarkb | so we have multiple levels of whitelisting? | 18:28 |
mtreinish | a bunch are too dirty so even if there are un-whitelisted errors things still pass | 18:28 |
*** pcrews has joined #openstack-qa | 18:28 | |
mtreinish | clarkb: yeah, it's an artifact of having too many things slip through when it was an explicit log message whitelist | 18:28 |
mtreinish | so we switched to a per log file based whitelist | 18:28 |
clarkb | I see. Would it be inappropriate to suggest we stop with the massive spam in logs then? | 18:28 |
clarkb | it is extremely confusing | 18:29 |
mtreinish | we probably should remove the non-whitelisted errors | 18:29 |
mtreinish | yeah I agree | 18:29 |
mtreinish | it definitely is, I only realized it the other day when I was looking at one of these failures | 18:29 |
clarkb | but it also looks like logstash is derping pretty good. I picked a random tempest job and dont have n-cond logs in logstash for that one either | 18:30 |
clarkb | so I need to look into that | 18:31 |
*** adalbas has joined #openstack-qa | 18:34 | |
clarkb | but first I need food and caffeine. will dig into logstash stuff this afternoon | 18:39 |
mtreinish | clarkb: I'm working on cleaning up the output from the log tool, should have a fix up in a few min | 18:42 |
clarkb | mtreinish: awesome and thank you. That will probably help people when debugging (I know I probably attributed dirty logs to some failures inappropriately) | 18:43 |
*** e0ne has quit IRC | 18:47 | |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/devstack: Move apache config creation to start_keystone() https://review.openstack.org/118740 | 18:48 |
mtreinish | heh, testing the script out on coffee shop wifi probably wasn't the best idea... | 18:52 |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/devstack: Move apache config creation to start_keystone() https://review.openstack.org/118740 | 18:53 |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/devstack: Move apache config creation to start_keystone() https://review.openstack.org/118740 | 18:54 |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/grenade: Set FILES in upgrade-keystone https://review.openstack.org/118742 | 18:55 |
adam_g | clarkb, re mod wsgi ^ | 18:55 |
*** mkollaro has joined #openstack-qa | 18:59 | |
adam_g | dtroyer, any thoughts there? i think we need to merge the grenade change before the devstack one (assuming its approved) | 18:59 |
openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Make output from check_logs less verbose https://review.openstack.org/118744 | 19:01 |
mtreinish | clarkb: ^^^ | 19:01 |
*** e0ne has joined #openstack-qa | 19:05 | |
mtreinish | dkranz: ^^^ you probably would be interested in that change too | 19:08 |
*** dmellado has quit IRC | 19:08 | |
mtreinish | clarkb: sample output: http://paste.openstack.org/show/105382/ | 19:10 |
mtreinish | I'm wondering if I should change the wording a bit | 19:11 |
*** dmellado has joined #openstack-qa | 19:11 | |
mtreinish | or maybe group it into 2 sets, those which have errors but didn't cause a failure and those that caused it to fail | 19:12 |
clarkb | mtreinish: or even 'n-cpu log file has errors\nfoo-log log file has errors\nother-log log file has errors and is not allowed to have them" | 19:12 |
clarkb | or and is not whitelisted or similar | 19:12 |
clarkb | so that the listing is compact and direct | 19:12 |
*** e0ne has quit IRC | 19:16 | |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Migrate baremetal_basic_ops to tempest clients https://review.openstack.org/117673 | 19:16 |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Add baremetal tests for get node by instance/port https://review.openstack.org/118513 | 19:16 |
*** e0ne has joined #openstack-qa | 19:19 | |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Remove PYTHONHASHSEED=0 from tox.ini https://review.openstack.org/116744 | 19:20 |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Make unit tests not depend on random hash seed https://review.openstack.org/116741 | 19:20 |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Update Heat CLI test to pass with random hash https://review.openstack.org/116786 | 19:20 |
mtreinish | clarkb: yeah I like that, is this good: http://paste.openstack.org/show/105384/ | 19:20 |
clarkb | yup | 19:21 |
*** Anju has joined #openstack-qa | 19:21 | |
openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Make output from check_logs less verbose https://review.openstack.org/118744 | 19:22 |
mtreinish | clarkb: ^^^ | 19:22 |
mtreinish | adam_g: did you need to rebase the random hashseed stack again? | 19:24 |
*** marun_ has joined #openstack-qa | 19:24 | |
clarkb | mtreinish: lgtm. you might also want to check the grenade switch I am not convinced it is working | 19:24 |
adam_g | mtreinish, looked that way | 19:24 |
clarkb | mtreinish: ya, we updated the DEVSTACK_GATE_GRENADE values | 19:24 |
adam_g | config sample changes | 19:24 |
clarkb | mtreinish: the check should be is_grenade = os.environ.get('DEVSTACK_GATE_GRENADE', None) != None (though you can drop the != None if you don't need True False values | 19:25 |
mtreinish | adam_g: did we change config values in that tree? It shouldn't have caused a conflict | 19:26 |
mtreinish | clarkb: yeah I was just looking at that, do you want to push out a patch for that? | 19:26 |
clarkb | mtreinish: sure | 19:26 |
clarkb | mtreinish: do you have a preference for truthiness values? | 19:26 |
mtreinish | clarkb: heh, not really | 19:27 |
openstackgerrit | Matt Riedemann proposed a change to openstack-infra/devstack-gate: Deprecate API_RATE_LIMIT setting for Tempest runs https://review.openstack.org/116001 | 19:27 |
salv-orlando | mtreinish: I am battling with bug 1349617, but I have a ridicoulous proposal for improving gate conditions | 19:28 |
mtreinish | uh oh | 19:28 |
salv-orlando | mtreinish: basically the failure happens always in two tests which are part of the advanced network scenario | 19:28 |
* mtreinish is on the edge of his seat | 19:28 | |
salv-orlando | stop_start and resize | 19:28 |
*** sabeen has joined #openstack-qa | 19:28 | |
salv-orlando | can we skip them until the bug if fixed | 19:28 |
salv-orlando | the downside is that those test are also the only way to repro them | 19:29 |
mtreinish | salv-orlando: that's not a guaranteed fix, it could just move the failure | 19:29 |
mtreinish | but we can give it a try if you think those tests are passing reliably enough for right now | 19:29 |
salv-orlando | mtreinish: which is also true. But I can try to push a patch for it, check it a reasonable number of times, and then make a call | 19:29 |
mtreinish | s/are/aren't | 19:30 |
salv-orlando | mtreinish: ok. also, if we see the failure moving somewhere else it might give us some more clues. | 19:30 |
adam_g | mtreinish, hm my top patch was failing pep8 check_uptodate.sh | 19:30 |
salv-orlando | mtreinish: thanks anyway for the advice. | 19:30 |
mtreinish | salv-orlando: yeah it's worth giving a shot at least | 19:31 |
openstackgerrit | Clark Boylan proposed a change to openstack/tempest: Properly detect grenade in check_logs script https://review.openstack.org/118753 | 19:31 |
clarkb | mtreinish: ^ | 19:31 |
mtreinish | clarkb: LGTM | 19:32 |
mtreinish | adam_g: hmm whatever, if it makes it work that's all that counts | 19:33 |
mtreinish | adam_g: wait, unless do you think a random hash was causing the check to fail? | 19:34 |
mtreinish | by generating a different order for the config options in the temp file it creates? | 19:34 |
*** ZZelle_ has joined #openstack-qa | 19:34 | |
ZZelle_ | mtreinish, one question | 19:34 |
mtreinish | because the check script just does a diff between the file in etc and one generated when it's run | 19:35 |
adam_g | mtreinish, oh man that woudl be awesome | 19:35 |
adam_g | mtreinish, let me run the pep8 check in a loop | 19:35 |
mtreinish | adam_g: that would explain why only the top patch was failing | 19:35 |
mtreinish | ZZelle_: sure, shoot | 19:36 |
adam_g | mtreinish, good call. 2/5 checks failed locally :| | 19:36 |
ZZelle_ | mtreinish, if i use testscenarios for provider network tests, should i put them in tempest.scenario package? I would say because it seems not to feat tempest.scenario README.rst description | 19:37 |
adam_g | mtreinish, i suppose running the only pep8 check with PYTHONHASHSEED=0 is better than running them all like that | 19:37 |
mtreinish | dhellmann: ^^^ any thoughts? I know that the config generator tool is being replaced in newer oslo.config versions | 19:37 |
*** torandu has quit IRC | 19:37 | |
mtreinish | adam_g: yeah I think that's what we'll have to do moving forward | 19:37 |
mtreinish | and we can get a fix in oslo long term | 19:38 |
adam_g | mtreinish, ill revise that top patch | 19:38 |
mtreinish | ZZelle_: well if you don't think it's a scenario test then you shouldn't put it in tempest.scenario | 19:38 |
mtreinish | testscenarios is just a library you can use it anywhere | 19:38 |
mtreinish | take a look at the autogenerated negative api tests, they use testscenarios to generate tests in tempest.api | 19:39 |
ZZelle_ | mtreinish, ok, i am discovering testscenarios, so i am lost ... for the moment | 19:39 |
mtreinish | ZZelle_: yeah, it's a bit confusing because scenario is an overloaded term here | 19:39 |
mtreinish | ZZelle_: let me pull up a couple of examples in the repo for you | 19:39 |
mtreinish | ZZelle_: so http://git.openstack.org/cgit/openstack/tempest/tree/tempest/api/compute/flavors/test_flavors_negative.py with http://git.openstack.org/cgit/openstack/tempest/tree/tempest/test.py#n402 is an example of where it's used in the api tests | 19:41 |
mtreinish | although that's more complex than how you'll have to do it | 19:42 |
openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Remove PYTHONHASHSEED=0 from tox.ini https://review.openstack.org/116744 | 19:42 |
*** marun_ has quit IRC | 19:42 | |
mtreinish | ZZelle_: a bit simpler of an example from the scenario tests is: http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/utils.py and http://git.openstack.org/cgit/openstack/tempest/tree/tempest/scenario/test_server_basic_ops.py | 19:43 |
*** weshay has quit IRC | 19:46 | |
*** sballe has joined #openstack-qa | 19:47 | |
ZZelle_ | mtreinish, thanks, i think i get the mysterious (for me) ... scenario loading | 19:50 |
mtreinish | ZZelle_: nah, it's mysterious for pretty much everyone at first. It's kinda like black magic unless you've written code that uses it before. | 19:52 |
*** timello_ has quit IRC | 19:53 | |
*** mkollaro has quit IRC | 19:54 | |
*** marun_ has joined #openstack-qa | 19:55 | |
lifeless | o/ | 19:57 |
*** e0ne has quit IRC | 20:06 | |
mtreinish | lifeless: \o | 20:08 |
*** Anju has quit IRC | 20:13 | |
openstackgerrit | vigneshvar proposed a change to openstack-dev/devstack: Reset no_proxy when the node is installed behind a proxy https://review.openstack.org/117985 | 20:14 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add fingerprint for bug 1363103 https://review.openstack.org/117923 | 20:19 |
*** sballe has quit IRC | 20:28 | |
openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/elastic-recheck: Work toward Python 3.4 support and testing https://review.openstack.org/118767 | 20:28 |
*** cdent has quit IRC | 20:32 | |
*** mwagner_lap has quit IRC | 20:34 | |
*** nmagnezi has quit IRC | 20:37 | |
*** cnesa10 has joined #openstack-qa | 20:37 | |
*** adalbas has quit IRC | 20:43 | |
*** sballe has joined #openstack-qa | 20:47 | |
*** yfried has quit IRC | 20:50 | |
*** markmcclain1 has joined #openstack-qa | 20:53 | |
*** markmcclain has quit IRC | 20:54 | |
*** toure has quit IRC | 20:57 | |
openstackgerrit | Mikhail S Medvedev proposed a change to openstack/tempest: Add neutron service tag https://review.openstack.org/118886 | 20:57 |
*** Sukhdev has joined #openstack-qa | 21:01 | |
*** markmcclain1 has quit IRC | 21:01 | |
*** markmcclain has joined #openstack-qa | 21:01 | |
*** aysyd has quit IRC | 21:09 | |
openstackgerrit | David Kranz proposed a change to openstack/tempest: Move success response checking to the token client https://review.openstack.org/117621 | 21:10 |
*** zz_johnthetubagu has quit IRC | 21:10 | |
*** zz_johnthetubagu has joined #openstack-qa | 21:11 | |
*** zz_johnthetubagu is now known as johnthetubaguy | 21:11 | |
*** dpaterson has quit IRC | 21:16 | |
*** dencaval has quit IRC | 21:16 | |
*** melwitt has joined #openstack-qa | 21:17 | |
*** melwitt has quit IRC | 21:22 | |
*** melwitt has joined #openstack-qa | 21:23 | |
openstackgerrit | Salvatore Orlando proposed a change to openstack/tempest: Some scenario tests do not delete network resources https://review.openstack.org/118891 | 21:23 |
*** dhellmann is now known as dhellmann_ | 21:28 | |
mtreinish | salv-orlando: ^^^ I don't think that'll work, addCleanup doesn't work in setupClass... | 21:30 |
mtreinish | oh, missed your comment, nm | 21:30 |
salv-orlando | yeah I know… I pushed it from the wrong machine | 21:31 |
salv-orlando | I initially did that, then moved the patch to another machine and fixed it, but then pushed from the 1st machine | 21:31 |
salv-orlando | mtreinish: anyway, does it make sense? I don’t think it’s related to any gate issue, but still | 21:31 |
*** krtaylor has quit IRC | 21:32 | |
mtreinish | salv-orlando: yeah, it's actually a bigger issue right now. It doesn't just happen in the scenario tests | 21:33 |
openstackgerrit | Salvatore Orlando proposed a change to openstack/tempest: Some scenario tests do not delete network resources https://review.openstack.org/118891 | 21:33 |
salv-orlando | mtreinish: from what I gather is a collateral of moving away from project client libraries to tempest clients? | 21:34 |
mtreinish | salv-orlando: it's the rationale behind the safe_setup (or whatever it's called) decorator | 21:34 |
salv-orlando | mtreinish: oh. So I think in the medium term we should just stop creating network resources in isolated_creds? | 21:34 |
mtreinish | salv-orlando: well it's actually more of an issue caused by using unittest to run the tests. When something explodes it just kinda of stays there | 21:35 |
mtreinish | long term we need to fix it: https://review.openstack.org/#/c/118352/ | 21:35 |
mtreinish | salv-orlando: can we get away with that? I thought we needed to allocate networks if we created a new tenant | 21:36 |
*** nsaje has quit IRC | 21:37 | |
salv-orlando | mtreinish: yes we do - it’s just a bit strange that a class called isolated_creds defines networks for tenants. However that’s fine for the time being. I thought this was somehting for safe_setup. But I see safe_setup is pretty much the addCleanup equivalent for setupClass | 21:38 |
mtreinish | salv-orlando: oh, I see what you're saying about the artifact of the client migration now after looking at the second rev of your patch | 21:40 |
salv-orlando | mtreinish: some other scenario “manager” classes are either not using anymore net resources created by isolated creds or cleaning them up | 21:41 |
salv-orlando | anyway, I think the proposed blueprint makes sense as a medium term solution. | 21:41 |
mtreinish | salv-orlando: heh, so it's just the name, or did you want to split isolated_networks off into a separate class? | 21:42 |
mtreinish | salv-orlando: yeah hopefully we'll be able to make good progress on that soon | 21:42 |
salv-orlando | I think NeutronScenarioTest is already not using anymore the net resources in IsolatedCreds. If that’s correct and is the general direction then we should not worry about it anymore. | 21:43 |
mtreinish | salv-orlando: well there is only one test class using NeutronScenarioTest right now and it call set_network_resources() without any args | 21:45 |
mtreinish | but I thought that was the norm for neutron tests, because if they needed networking it was explicitly part of the test to create it | 21:45 |
mtreinish | I feel like I'm missing some detail here... | 21:46 |
salv-orlando | mtreinish: I seem to recall we did set_network_resources in a way that without params creates no resource | 21:46 |
*** mriedem has quit IRC | 21:46 | |
mtreinish | salv-orlando: yep, that's how it works | 21:46 |
*** andreaf has quit IRC | 21:51 | |
*** jecarey has quit IRC | 22:03 | |
clarkb | OSError: [Errno 24] Too many open files may be the root of our logstash trouble | 22:07 |
*** shardy is now known as shardy_z | 22:07 | |
*** openstack has joined #openstack-qa | 22:11 | |
mtreinish | clarkb: heh, yeah that might cause a problem | 22:11 |
*** dkranz has quit IRC | 22:19 | |
*** dkranz has joined #openstack-qa | 22:20 | |
*** markmcclain has quit IRC | 22:20 | |
*** dimtruck is now known as zz_dimtruck | 22:26 | |
*** jgrimm is now known as zz_jgrimm | 22:29 | |
*** krtaylor has joined #openstack-qa | 22:39 | |
*** tkammer has joined #openstack-qa | 22:41 | |
*** masayukig has quit IRC | 22:44 | |
*** masayukig has joined #openstack-qa | 22:50 | |
*** arnaud has quit IRC | 22:51 | |
*** dims has quit IRC | 22:52 | |
*** dims has joined #openstack-qa | 22:52 | |
*** marun_ has quit IRC | 22:54 | |
*** tkammer has quit IRC | 22:56 | |
*** dims has quit IRC | 22:57 | |
*** rbak has quit IRC | 22:58 | |
*** Sukhdev has quit IRC | 23:00 | |
*** masayukig has quit IRC | 23:07 | |
*** masayukig has joined #openstack-qa | 23:09 | |
*** masayukig_mobi has joined #openstack-qa | 23:10 | |
*** masayukig_mobi has quit IRC | 23:13 | |
*** mlavalle has quit IRC | 23:15 | |
*** openstackstatus has quit IRC | 23:19 | |
*** openstackstatus has joined #openstack-qa | 23:20 | |
*** ChanServ sets mode: +v openstackstatus | 23:20 | |
*** mlavalle_ has joined #openstack-qa | 23:21 | |
openstackgerrit | Adam Gandelman proposed a change to openstack-dev/grenade: Set required DSG variables for Ironic DO NOT MERGE https://review.openstack.org/118515 | 23:22 |
openstackgerrit | Salvatore Orlando proposed a change to openstack/tempest: Test for helping root cause bug 1349617 https://review.openstack.org/118925 | 23:22 |
*** ianw has quit IRC | 23:25 | |
*** ianw has joined #openstack-qa | 23:25 | |
*** coolsvap has quit IRC | 23:29 | |
*** atiwari has quit IRC | 23:34 | |
*** hemna is now known as hemna__ | 23:36 | |
*** coolsvap has joined #openstack-qa | 23:40 | |
gmann | Santosh_: hi | 23:43 |
*** mwagner_lap has joined #openstack-qa | 23:45 | |
*** torandu has joined #openstack-qa | 23:52 | |
*** mlavalle_ has quit IRC | 23:54 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!