| openstackrecheck | Console logs not available after 13:33s for check-tempest-dsvm-neutron 69047,22,e854882 | 00:02 |
|---|---|---|
| jogo | wow a good munber of these gate bugs are infra | 00:05 |
| *** mlavalle has quit IRC | 00:07 | |
| *** salv-orlando has quit IRC | 00:07 | |
| clarkb | jogo: which ones? | 00:07 |
| jogo | http://status.openstack.org/elastic-recheck/gate.html | 00:07 |
| jogo | Bug 1324932 Bug 1286635 | 00:08 |
| jogo | and two that we may not really be bugs: | 00:08 |
| *** aimon has joined #openstack-qa | 00:08 | |
| jogo | Bug 1260654 Bug 1284371 | 00:08 |
| clarkb | I think some of them are misclassified | 00:08 |
| clarkb | https://bugs.launchpad.net/openstack-ci/+bug/1324932 is a devstack bug | 00:08 |
| clarkb | https://bugs.launchpad.net/openstack-ci/+bug/1286635 is a dupe of that bug | 00:10 |
| clarkb | oh I see need force yes because of the mirrors ugh | 00:10 |
| clarkb | but yeah those are dups and should be consolidated | 00:10 |
| jogo | oh yeah those spikes are identical | 00:10 |
| jogo | as are the number of hits | 00:11 |
| clarkb | I would expect to see less of 1284371 now that we are on 1.1 | 00:11 |
| clarkb | (assuming 1.1 is more stable than 1.0 when it comes to deleting our nodes) | 00:12 |
| *** dims__ has joined #openstack-qa | 00:12 | |
| *** mlavalle has joined #openstack-qa | 00:12 | |
| clarkb | and https://bugs.launchpad.net/openstack-ci/+bug/1260654 shouldn't actually fail changes, zuul should rerun those jobs | 00:13 |
| *** jaypipes has joined #openstack-qa | 00:13 | |
| *** ChanServ sets mode: +o jaypipes | 00:13 | |
| * clarkb does a spot check | 00:13 | |
| *** mmaglana has quit IRC | 00:13 | |
| *** dims_ has quit IRC | 00:14 | |
| mtreinish | jogo: I think I found a new one http://logs.openstack.org/57/97757/1/gate/gate-tempest-dsvm-full/c447c4d/logs/screen-n-cpu.txt.gz?level=TRACE | 00:14 |
| mtreinish | which trace should I use the circular one or the unexpected task state? | 00:14 |
| clarkb | yup spot check confirms | 00:14 |
| *** dims__ is now known as dims | 00:15 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Bug 1324932 is a duplicate of 1286635 https://review.openstack.org/97992 | 00:16 |
| jogo | clarkb: ^ dedup, pushing a patch up to hide 654 | 00:16 |
| clarkb | jogo: if you need commit message info, zuul recognizes that class of failure as an instance where it should retrigger jobs and it does this | 00:17 |
| clarkb | jogo: its not perfect but it works | 00:17 |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-neutron 96124,2,fb2e72f | 00:17 |
| openstackrecheck | openstack/oslo.messaging change: https://review.openstack.org/94972 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 00:17 |
| mtreinish | clarkb: well I just approved the first one, removes are an easy +A | 00:18 |
| jogo | mtreinish: which stacktrace is the bad one there hehe | 00:19 |
| mtreinish | they both look bad to me | 00:19 |
| jogo | I don't think they both cause failures though | 00:20 |
| jogo | I remember seeing one of those before | 00:20 |
| jogo | don't know which | 00:20 |
| *** sthaha has quit IRC | 00:20 | |
| mtreinish | well we have a nova unit test query for the unexpected state one | 00:20 |
| mtreinish | let me check logstash to see if there are success hits on either | 00:20 |
| jogo | clarkb: for the failed mirrors could we just update devstack to try apt-get update twice? | 00:21 |
| jogo | if the first fails | 00:21 |
| clarkb | jogo: that may work around it if you put a short delay in there | 00:21 |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Suppress the graph for bug 1260654 https://review.openstack.org/97993 | 00:22 |
| mtreinish | jogo: looks like circular hits on success | 00:23 |
| jogo | mtreinish: yeah that sounds like what I saw, which is confusing | 00:24 |
| jogo | I think I filed a bug for that | 00:24 |
| mtreinish | jogo: hmm the unexpected state shows up everywhere on runs too | 00:28 |
| mtreinish | but not with the expected rebuilding state | 00:28 |
| mtreinish | so I guess that's the query | 00:28 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Bug 1324932 is a duplicate of 1286635 https://review.openstack.org/97992 | 00:29 |
| jogo | mtreinish: I think I added the expected task state to the debug message for this very purpose | 00:30 |
| *** yamahata has joined #openstack-qa | 00:30 | |
| openstackrecheck | Console logs not available after 13:23s for gate-cinder-pep8 97880,4,9157728 | 00:33 |
| *** dmorita has joined #openstack-qa | 00:35 | |
| openstackrecheck | openstack/python-troveclient change: https://review.openstack.org/90462 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 00:36 |
| *** mlavalle has quit IRC | 00:37 | |
| *** openstackrecheck has quit IRC | 00:37 | |
| *** openstackrecheck has joined #openstack-qa | 00:37 | |
| *** sthaha has joined #openstack-qa | 00:37 | |
| *** mlavalle has joined #openstack-qa | 00:37 | |
| *** rbak has quit IRC | 00:39 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Add query for bug 1304184 https://review.openstack.org/97995 | 00:39 |
| mtreinish | jogo: ^^^ I think that should do it | 00:39 |
| *** annegentle has quit IRC | 00:40 | |
| mtreinish | the guy's fix for that bug I used won't fix it completely I think | 00:40 |
| *** mwagner_lap has joined #openstack-qa | 00:40 | |
| *** openstackrecheck has quit IRC | 00:41 | |
| *** rbak has joined #openstack-qa | 00:42 | |
| jogo | clarkb mtreinish: how is your bash? what does NO_UPDATE_REPOS do in http://git.openstack.org/cgit/openstack-dev/devstack/tree/functions-common#n890 | 00:44 |
| jogo | mtreinish: s/queries/query/g | 00:45 |
| mtreinish | well I fail... | 00:45 |
| mtreinish | jogo: I think it supposed to give you an override to avoid running apt-get update | 00:45 |
| jogo | mtreinish: ahh | 00:46 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Add query for bug 1304184 https://review.openstack.org/97995 | 00:47 |
| *** zhikunliu has joined #openstack-qa | 00:48 | |
| *** safchain has quit IRC | 00:50 | |
| *** rwsu has quit IRC | 00:51 | |
| *** rbak has quit IRC | 00:52 | |
| openstackgerrit | Chris Jones proposed a change to openstack-dev/devstack: Reorder network creation and Neutron agent startup https://review.openstack.org/97999 | 00:55 |
| mtreinish | cyeoh: why only a +1 on https://review.openstack.org/#/c/97834/ ? | 00:57 |
| openstackgerrit | Joe Gordon proposed a change to openstack-dev/devstack: Add a single retry in to apt-get update https://review.openstack.org/98000 | 00:58 |
| jogo | mtreinish: FWIW I only noticed because the tests failed | 00:58 |
| jogo | which means they work | 00:58 |
| jogo | clarkb: ^ devstack workaround idea | 00:58 |
| mtreinish | jogo: well that's a good thing I guess. I just assumed the unit tests didn't do much for adding queries | 01:00 |
| mtreinish | because I never green lit the functional test in the gate | 01:00 |
| jogo | mtreinish: they just check the syntax | 01:00 |
| *** mrodden has quit IRC | 01:01 | |
| *** mriedem has joined #openstack-qa | 01:04 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1304184 https://review.openstack.org/97995 | 01:06 |
| jogo | is this a failure: http://logs.openstack.org/28/86228/5/gate/gate-grenade-dsvm/9104c80/logs/old/devstacklog.txt.gz#_2014-06-04_21_23_27_588 | 01:06 |
| *** openstackrecheck has joined #openstack-qa | 01:07 | |
| jogo | clarkb: ^ | 01:07 |
| *** rbak has joined #openstack-qa | 01:10 | |
| clarkb | jogo: yes | 01:12 |
| clarkb | jogo: sdague wrote a fix already | 01:12 |
| *** GMann has quit IRC | 01:12 | |
| clarkb | devstack is going to stop pip installing from sdist and use get-pip.py instead which we cache on the test slaves | 01:12 |
| mriedem | jogo: i opened an infra bug on that today | 01:14 |
| mriedem | jogo: clarkb: https://bugs.launchpad.net/openstack-ci/+bug/1326539 | 01:14 |
| mriedem | suppose it should be a devstack bug instead | 01:15 |
| mriedem | i was in a hurry | 01:15 |
| *** ekhugen has joined #openstack-qa | 01:15 | |
| jogo | clarkb: so should we just prioritize his patch? | 01:15 |
| clarkb | jogo: probably | 01:15 |
| clarkb | its a simple fix | 01:15 |
| *** rbak has quit IRC | 01:16 | |
| jogo | cool lets do it then | 01:16 |
| mriedem | jogo: the bug report has a query in it, but it's not great | 01:17 |
| mriedem | not sure if there are other failures with the same thing but for different issues | 01:17 |
| jogo | mriedem: link? | 01:17 |
| mriedem | jogo: see above | 01:18 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/96773 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error | 01:19 |
| jogo | mriedem: yeah that query is pretty short not sure about it | 01:20 |
| mriedem | jogo: that's why i didn't push it up | 01:22 |
| jogo | clarkb: here is sean's patch https://review.openstack.org/#/c/97912/ | 01:22 |
| clarkb | yup it should have my +1 | 01:23 |
| jogo | clarkb: https://etherpad.openstack.org/p/June-gate | 01:23 |
| jogo | lets track the work somewhere | 01:23 |
| *** yamahata has quit IRC | 01:26 | |
| *** yamahata has joined #openstack-qa | 01:26 | |
| jogo | clarkb: https://review.openstack.org/#/c/97845/ is good to go as well | 01:27 |
| jogo | can that be bumped | 01:27 |
| * jogo goes to infra to ask | 01:27 | |
| *** fbo_away is now known as fbo | 01:37 | |
| *** sthaha has quit IRC | 01:38 | |
| *** sthaha has joined #openstack-qa | 01:38 | |
| *** coolsvap is now known as coolsvap|afk | 01:38 | |
| *** levi_b has joined #openstack-qa | 01:40 | |
| *** levi_b_ has joined #openstack-qa | 01:40 | |
| *** levi_b_ has quit IRC | 01:40 | |
| *** flwang has joined #openstack-qa | 01:42 | |
| *** TravT has quit IRC | 01:42 | |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/devstack-gate: Force ntpdate before we start the timer https://review.openstack.org/97924 | 01:43 |
| *** aysyd has quit IRC | 01:44 | |
| *** levi_b has quit IRC | 01:44 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-dev/grenade: Fix errors in stop-services from missing includes https://review.openstack.org/95072 | 01:45 |
| *** nosnos has joined #openstack-qa | 01:46 | |
| *** sivaji has joined #openstack-qa | 01:46 | |
| flwang | hi guys, a question about tempest scenarios test, does it make any sense if we use it for end to end functional test? | 01:46 |
| openstackrecheck | Console logs not available after 13:54s for gate-nova-python26 97969,2,92c051f | 01:47 |
| flwang | my team are trying to implement some e2e test, I just wanna confirm if tempest scenarios test is a good stuff to leverage, thanks | 01:47 |
| sivaji | Hi How to join and contribute to openstack? | 01:48 |
| flwang | sivaji: download code, and fix a bug, then upsteam, bingo! | 01:48 |
| sivaji | before that any registration is required? | 01:49 |
| oomichi | sivaji: welcome:) https://wiki.openstack.org/wiki/How_To_Contribute is goot to read. | 01:50 |
| openstackgerrit | Jeremy Stanley proposed a change to openstack-infra/devstack-gate: Force ntpdate before we start the timer https://review.openstack.org/97924 | 01:51 |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-neutron 97713,1,5019d7a | 02:00 |
| *** ekhugen has quit IRC | 02:01 | |
| openstackrecheck | Console logs not available after 13:34s for check-tempest-dsvm-neutron 97996,1,98b4edc | 02:14 |
| *** fbo is now known as fbo_away | 02:16 | |
| *** dkranz has quit IRC | 02:17 | |
| openstackgerrit | A change was merged to openstack/tempest: Wait for server activations for the updated servers https://review.openstack.org/97496 | 02:23 |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/qa-specs: Add qa spec for bp/nova-v2.1-api-tests https://review.openstack.org/96661 | 02:29 |
| cyeoh | mtreinish: oh it all looked good to me, but I know I haven't been to a QA meeting for a while and missed the design summit discussions so I've been leaning on being on the conservative side on reviews. | 02:31 |
| mtreinish | cyeoh: ok that's fine, no worries. I was just wondering if there was something missing. | 02:31 |
| cyeoh | mtreinish: cool - I'll make sure I leave an explanation next time I do that sort of thing | 02:33 |
| *** sivaji has quit IRC | 02:33 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 97757,1,a47a5aa | 02:35 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-neutron 97997,3,a6ff57a | 02:49 |
| *** levi_b has joined #openstack-qa | 02:53 | |
| *** dims has quit IRC | 02:53 | |
| *** dims has joined #openstack-qa | 02:55 | |
| *** Abhijeet has joined #openstack-qa | 02:56 | |
| *** levi_b has quit IRC | 03:01 | |
| *** Madan has joined #openstack-qa | 03:02 | |
| cyeoh | cbs2004 | 03:09 |
| *** mriedem has quit IRC | 03:11 | |
| *** coolsvap|afk is now known as coolsvap | 03:13 | |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-postgres-full 98006,1,78c8188 | 03:17 |
| *** jaypipes has quit IRC | 03:33 | |
| *** dustins has joined #openstack-qa | 03:35 | |
| *** dustins has quit IRC | 03:35 | |
| openstackrecheck | Console logs not available after 13:22s for gate-grenade-dsvm 92739,2,7a59c75 | 03:38 |
| *** dims_ has joined #openstack-qa | 03:39 | |
| *** dims has quit IRC | 03:39 | |
| *** nosnos has quit IRC | 03:43 | |
| openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 97842,1,841e35c | 03:51 |
| *** Mike77 has joined #openstack-qa | 04:02 | |
| openstackrecheck | Console logs not available after 13:26s for gate-nova-python26 98011,2,2af444a | 04:05 |
| *** harlowja is now known as harlowja_away | 04:07 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Optimization and response status code update https://review.openstack.org/88233 | 04:07 |
| *** raies has joined #openstack-qa | 04:08 | |
| raies | _afazekas: hi | 04:12 |
| *** GMann has joined #openstack-qa | 04:12 | |
| *** Sukhdev has joined #openstack-qa | 04:16 | |
| *** yfried has quit IRC | 04:17 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-full 98006,2,9dad163 | 04:19 |
| *** sivaji has joined #openstack-qa | 04:20 | |
| *** dims_ has quit IRC | 04:20 | |
| sivaji | How to see the existing bugs on Nova project | 04:20 |
| clarkb | sivaji: https://bugs.launchpad.net/nova | 04:21 |
| *** Longgeek has joined #openstack-qa | 04:28 | |
| openstackrecheck | Console logs not available after 13:30s for gate-nova-python27 97529,3,1b90692 | 04:32 |
| *** SridharG has joined #openstack-qa | 04:35 | |
| sivaji | How to create a test setup | 04:36 |
| *** matsuhashi has joined #openstack-qa | 04:39 | |
| *** psharma has joined #openstack-qa | 04:41 | |
| *** nosnos has joined #openstack-qa | 04:41 | |
| *** rdekel has joined #openstack-qa | 04:46 | |
| openstackrecheck | Console logs not available after 13:37s for check-tempest-dsvm-postgres-full 96947,1,5524753 | 04:47 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/97598 failed because of: gate-tempest-dsvm-postgres-full: unrecognized error, gate-grenade-dsvm: unrecognized error, gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 04:48 |
| *** k4n0 has joined #openstack-qa | 05:02 | |
| *** Longgeek has quit IRC | 05:06 | |
| *** tkammer has joined #openstack-qa | 05:06 | |
| *** Longgeek has joined #openstack-qa | 05:07 | |
| *** yfried has joined #openstack-qa | 05:07 | |
| *** Abhijeet has quit IRC | 05:13 | |
| *** Abhijeet has joined #openstack-qa | 05:13 | |
| *** marun_afk is now known as marun | 05:29 | |
| *** coolsvap is now known as coolsvap|afk | 05:38 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Force ntpdate before we start the timer https://review.openstack.org/97924 | 05:41 |
| *** Sukhdev has quit IRC | 05:41 | |
| *** mdenny has quit IRC | 05:51 | |
| *** mdenny has joined #openstack-qa | 05:51 | |
| *** coolsvap|afk is now known as coolsvap | 05:52 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/97757 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1254890 https://bugs.launchpad.net/bugs/1315095, gate-tempest-dsvm-full: unrecognized error | 05:54 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Compute "cells" basic API test https://review.openstack.org/90337 | 05:55 |
| *** oomichi has quit IRC | 06:02 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: WIP: Nova V2.1 API test using testscenarios https://review.openstack.org/98039 | 06:05 |
| *** nsaje has quit IRC | 06:06 | |
| *** nsaje has joined #openstack-qa | 06:08 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/qa-specs: Add qa spec for bp/nova-v2.1-api-tests https://review.openstack.org/96661 | 06:09 |
| *** marun is now known as marun_afk | 06:14 | |
| _afazekas | raies: good morning | 06:17 |
| _afazekas | sivaji: 1 get devstack https://github.com/openstack-dev/devstack, 2. create a 'stack' user. 3. create /opt/stack/new owned by 'stack'. 4. get a localrc file from the logs http://logs.openstack.org/32/97532/3/check/check-tempest-dsvm-neutron/495fb0b/logs/ and copy into the devstack directory . 5. remove the ERROR_ON_CLONE=True and USE_SCREEN=False lines from the localrc. 6. execute the stack.sh | 06:23 |
| *** Longgeek has quit IRC | 06:27 | |
| *** evgenyf has joined #openstack-qa | 06:29 | |
| *** zhikunliu has quit IRC | 06:33 | |
| *** zhikunliu has joined #openstack-qa | 06:34 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-postgres-full 98032,1,2f07d74 | 06:37 |
| openstackrecheck | openstack/python-glanceclient change: https://review.openstack.org/97126 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1320617 | 06:44 |
| *** yrabl has joined #openstack-qa | 06:47 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Optimization and response status code update https://review.openstack.org/88233 | 06:48 |
| *** Longgeek has joined #openstack-qa | 06:58 | |
| openstackgerrit | Andrey Pavlov proposed a change to openstack/tempest: fix usage of allow_tenant_isolation https://review.openstack.org/97139 | 06:58 |
| openstackgerrit | Andrey Pavlov proposed a change to openstack/tempest: Add 'Member' role to all created users https://review.openstack.org/96460 | 06:58 |
| *** luqas has joined #openstack-qa | 06:59 | |
| *** GMann has quit IRC | 07:00 | |
| *** Longgeek has quit IRC | 07:02 | |
| *** Albert_13 has joined #openstack-qa | 07:04 | |
| openstackgerrit | Huruifeng proposed a change to openstack/tempest: Add tests for neutron firewall resource https://review.openstack.org/94590 | 07:07 |
| *** Mike77 has quit IRC | 07:09 | |
| *** catohornet has quit IRC | 07:12 | |
| *** oomichi has joined #openstack-qa | 07:16 | |
| *** jordanP has joined #openstack-qa | 07:21 | |
| *** luqas has quit IRC | 07:25 | |
| *** Albert_13 has quit IRC | 07:25 | |
| *** jlibosva has joined #openstack-qa | 07:25 | |
| *** luqas has joined #openstack-qa | 07:26 | |
| *** Albert_13 has joined #openstack-qa | 07:28 | |
| *** eezhova has quit IRC | 07:32 | |
| *** bkopilov has joined #openstack-qa | 07:32 | |
| *** luqas has quit IRC | 07:32 | |
| *** Albert_13 has quit IRC | 07:32 | |
| *** luqas has joined #openstack-qa | 07:34 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/88392 failed gate-tempest-dsvm-postgres-full, gate-grenade-dsvm in the gate queue with an unrecognized error | 07:39 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/96293 failed because of: gate-tempest-dsvm-postgres-full: unrecognized error, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1254890 https://bugs.launchpad.net/bugs/1315095, gate-tempest-dsvm-neutron-large-ops: https://bugs.launchpad.net/bugs/1286635 | 07:41 |
| *** Albert_13 has joined #openstack-qa | 07:44 | |
| *** andreaf has joined #openstack-qa | 07:50 | |
| *** salv-orlando has joined #openstack-qa | 07:51 | |
| *** gszasz_ has joined #openstack-qa | 07:52 | |
| *** Longgeek has joined #openstack-qa | 07:55 | |
| *** pbrooko has joined #openstack-qa | 07:57 | |
| *** chen has joined #openstack-qa | 08:03 | |
| *** zhikunliu has quit IRC | 08:04 | |
| *** Longgeek has quit IRC | 08:10 | |
| *** Longgeek has joined #openstack-qa | 08:11 | |
| *** eezhova has joined #openstack-qa | 08:15 | |
| *** Longgeek has quit IRC | 08:16 | |
| *** safchain has joined #openstack-qa | 08:16 | |
| *** Longgeek has joined #openstack-qa | 08:19 | |
| chen | hello, anyone around ??? I'm trying to install devstack on 2-nodes. Basically following http://devstack.org/guides/multinode-lab.html. While the controller looks well. but the compute node get many issues. | 08:20 |
| chen | This is my local.conf for compute node: | 08:20 |
| chen | http://paste.openstack.org/show/82898/ | 08:20 |
| chen | After I run "./stack.sh" | 08:20 |
| chen | I get errors. | 08:20 |
| chen | For neutron-openvswitch-agent, | 08:20 |
| chen | the first issue is it complains about br-int not exist. | 08:20 |
| chen | So I create br-int by hand, | 08:20 |
| chen | Then, it complains about "Unable to connect to AMQP server" | 08:20 |
| chen | Because services on controller node works well, so I first throught it is caused by iptables. | 08:20 |
| chen | but after I run "sudo iptables -I INPUT 1 -p tcp --dport 5672" on controller node. | 08:20 |
| chen | The issue is still there. | 08:20 |
| chen | Anyone know why ?? | 08:20 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/94436 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 08:23 |
| *** mlavalle has quit IRC | 08:28 | |
| *** luqas has quit IRC | 08:30 | |
| *** mkoderer has quit IRC | 08:32 | |
| *** cdent has joined #openstack-qa | 08:47 | |
| *** oomichi has quit IRC | 09:00 | |
| *** mpavlase has joined #openstack-qa | 09:00 | |
| openstackgerrit | Elena Ezhova proposed a change to openstack/tempest: Handle BadStatusLine exception in test_load_balancer_basic https://review.openstack.org/98065 | 09:07 |
| *** luqas has joined #openstack-qa | 09:07 | |
| *** tosky has joined #openstack-qa | 09:12 | |
| *** evgenyf has quit IRC | 09:18 | |
| *** mkollaro has joined #openstack-qa | 09:26 | |
| *** evgenyf has joined #openstack-qa | 09:32 | |
| *** oomichi has joined #openstack-qa | 09:32 | |
| openstackrecheck | openstack/keystone change: https://review.openstack.org/95263 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 https://bugs.launchpad.net/bugs/1326510 | 09:33 |
| *** mkoderer has joined #openstack-qa | 09:53 | |
| *** Abhijeet has quit IRC | 09:53 | |
| *** ildikov has quit IRC | 10:07 | |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/grenade: allow setting of TEMPEST_CONCURENCY https://review.openstack.org/97966 | 10:08 |
| *** yamahata has quit IRC | 10:10 | |
| *** Albert_13 has quit IRC | 10:18 | |
| *** pbrooko has quit IRC | 10:22 | |
| *** dmorita has quit IRC | 10:30 | |
| openstackgerrit | Dina Belova proposed a change to openstack/qa-specs: Add spec for the bp:add-ceilometer-scenario-tests https://review.openstack.org/97253 | 10:30 |
| openstackgerrit | Dina Belova proposed a change to openstack/qa-specs: Add spec for the bp:add-basic-ceilometer-tests https://review.openstack.org/97252 | 10:30 |
| *** matsuhashi has quit IRC | 10:31 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/97757 failed because of: gate-grenade-dsvm: unrecognized error, gate-nova-python26: https://bugs.launchpad.net/bugs/1311778 | 10:33 |
| *** matsuhashi has joined #openstack-qa | 10:34 | |
| *** leseb has joined #openstack-qa | 10:36 | |
| *** leseb has quit IRC | 10:40 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/devstack-gate: set concurrency for grenade https://review.openstack.org/98080 | 10:47 |
| *** matsuhashi has quit IRC | 11:06 | |
| *** matsuhashi has joined #openstack-qa | 11:06 | |
| *** matsuhashi has quit IRC | 11:10 | |
| *** rbak has joined #openstack-qa | 11:11 | |
| *** leseb has joined #openstack-qa | 11:11 | |
| openstackgerrit | Feodor Tersin proposed a change to openstack-dev/devstack: Parse openstackclient output to get EC2 and S3 urls. https://review.openstack.org/95500 | 11:13 |
| *** rbak has quit IRC | 11:14 | |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Compute "cells" basic API test https://review.openstack.org/90337 | 11:14 |
| *** yamahata has joined #openstack-qa | 11:15 | |
| *** nosnos has quit IRC | 11:20 | |
| *** matsuhashi has joined #openstack-qa | 11:24 | |
| *** matsuhashi has quit IRC | 11:24 | |
| *** matsuhashi has joined #openstack-qa | 11:24 | |
| *** mmaglana has joined #openstack-qa | 11:27 | |
| *** luqas has quit IRC | 11:28 | |
| *** matsuhashi has quit IRC | 11:29 | |
| *** evgenyf has quit IRC | 11:34 | |
| openstackgerrit | Amit Ugol proposed a change to openstack/tempest: Added Heat Software Config-Deploy API test https://review.openstack.org/89790 | 11:35 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: don't reuse servers in test_server_actions https://review.openstack.org/97842 | 11:37 |
| *** lbragstad has quit IRC | 11:38 | |
| *** amotoki has quit IRC | 11:41 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326754 https://review.openstack.org/98085 | 11:47 |
| *** afazekas has joined #openstack-qa | 11:48 | |
| *** sthaha has quit IRC | 11:51 | |
| *** Madan has quit IRC | 11:53 | |
| *** k4n0 has quit IRC | 11:54 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: be opinionated: only use get-pip.py https://review.openstack.org/97912 | 11:55 |
| *** cdent_ has joined #openstack-qa | 11:59 | |
| *** cdent has quit IRC | 12:00 | |
| *** cdent_ is now known as cdent | 12:00 | |
| *** Madan has joined #openstack-qa | 12:07 | |
| *** Longgeek has quit IRC | 12:12 | |
| *** Longgeek has joined #openstack-qa | 12:12 | |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/97912 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 12:13 |
| openstackgerrit | Zu Qiang proposed a change to openstack-dev/devstack: Reset no_proxy when the node is installed behind a proxy https://review.openstack.org/94988 | 12:13 |
| *** coolsvap is now known as coolsvap|afk | 12:15 | |
| *** mriedem has joined #openstack-qa | 12:15 | |
| boris-42 | sdague around? | 12:17 |
| boris-42 | sdague I just looked at the timeout exception | 12:18 |
| boris-42 | sdague and probably the main reason in keystone default setup that we are using in our gates | 12:18 |
| openstackgerrit | Amit Prakash Pandey proposed a change to openstack/tempest: Bug # 1326451 https://review.openstack.org/98092 | 12:21 |
| *** sivaji has quit IRC | 12:23 | |
| *** rbak has joined #openstack-qa | 12:24 | |
| *** sthaha has joined #openstack-qa | 12:30 | |
| *** sthaha has quit IRC | 12:30 | |
| *** sthaha has joined #openstack-qa | 12:30 | |
| *** psharma has quit IRC | 12:33 | |
| *** weshay has joined #openstack-qa | 12:37 | |
| *** ekhugen has joined #openstack-qa | 12:38 | |
| *** jchai has joined #openstack-qa | 12:39 | |
| *** dkranz has joined #openstack-qa | 12:42 | |
| vrovachev | sdague: Hi :) | 12:43 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Refine queries for tripleo jobs https://review.openstack.org/98096 | 12:43 |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: set concurrency for grenade https://review.openstack.org/98080 | 12:44 |
| openstackgerrit | A change was merged to openstack-dev/grenade: allow setting of TEMPEST_CONCURENCY https://review.openstack.org/97966 | 12:44 |
| *** adalbas has joined #openstack-qa | 12:45 | |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/devstack: make get-pip.py download failures more clear https://review.openstack.org/98098 | 12:47 |
| *** lbragstad has joined #openstack-qa | 12:47 | |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/97639 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 12:48 |
| vrovachev | sdague: thank you so much :) | 12:50 |
| sdague | vrovachev: no prob, thanks for iterating on it | 12:55 |
| *** luqas has joined #openstack-qa | 12:57 | |
| *** jchai has quit IRC | 12:59 | |
| *** jchai has joined #openstack-qa | 13:00 | |
| *** eglynn_ has joined #openstack-qa | 13:04 | |
| eglynn_ | sdague: quick question ... I don't fully understand your -1 on the ceilometer grenade coverage patch: https://review.openstack.org/94468 | 13:06 |
| eglynn_ | sdague: seems EmilienM has already landed a patch to enable ceilo in the gate: https://review.openstack.org/96518 | 13:07 |
| eglynn_ | sdague: ... or are we talking about two different aspects of gate service config? | 13:07 |
| sdague | eglynn: we're also talking about sequencing | 13:07 |
| sdague | my -1 was from way before EmilienM's change | 13:07 |
| DinaBelova | eglynn_, this -1 is for the 5th patch set... and there is 6th already published... that's something that makes me crazy in new Gerrit | 13:08 |
| sdague | DinaBelova: it doesn't reset on commit message changes | 13:08 |
| sdague | I think we can change that | 13:08 |
| eglynn_ | sdague: a-ha, ok, got it! ... so now there are no remaining blockers to that grenade coverage landing? | 13:08 |
| mtreinish | mriedem: any ideas on this one: http://logs.openstack.org/39/97639/2/gate/gate-tempest-dsvm-full/6e2a9e4/console.html#_2014-06-05_12_06_52_960 | 13:09 |
| eglynn_ | sdague: (in terms of other changes that would need to get in first) | 13:09 |
| DinaBelova | sdague, oh, it'll be cool - as sometimes we have -1s for unreadable commit messages... and then you still have these -1s even if it was actually fixed :D | 13:09 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova resize/migrate bug 1326778 https://review.openstack.org/98106 | 13:09 |
| sdague | eglynn_: not directly, though right now with the state of the gate, I'll probably not get around to this stuff this week | 13:09 |
| mtreinish | mriedem: it looks like the api is getting a MigrationNotFound exception, but I can't find anything raising it in the logs | 13:09 |
| mriedem | sdague: ^ | 13:09 |
| mtreinish | and nothing stood out to me traceing through the code | 13:09 |
| mriedem | mtreinish: yeah, see the e-r query i just posted | 13:10 |
| sdague | DinaBelova: yeh, honestly, if you could figure out what gerrit change is needed to reset the votes on commit message change, you would be my hero | 13:10 |
| sdague | because that's been biting me a bunch on many projects | 13:10 |
| mtreinish | mriedem: yeah that's what I was seeing in the logs, I was trying to track where the exception was coming from | 13:10 |
| eglynn_ | sdague: fair enough, just asking as we've targeted that grenade BP at juno-1 | 13:10 |
| eglynn_ | sdague: (... but looks like modulo gate blockages, that targetting should still hold) | 13:10 |
| mriedem | mtreinish: assuming confirm_resize in compute manager | 13:11 |
| DinaBelova | sdague, yes, sir! | 13:11 |
| DinaBelova | will try to solve this :) | 13:11 |
| sdague | DinaBelova: thank you | 13:12 |
| mtreinish | yeah, that's where I thought too. But, before it starts doing anything it gets the migration by id and has a catch for MigrationNotFound. Which means it's probably a race removing it... | 13:12 |
| sdague | oh, so nova is actually swallowing the useful part? | 13:13 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova resize/migrate bug 1326778 https://review.openstack.org/98106 | 13:14 |
| mtreinish | sdague: that's what it looks like, I think compute should be logging the exception | 13:18 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Suppress the graph for bug 1260654 https://review.openstack.org/97993 | 13:18 |
| *** Madan has quit IRC | 13:20 | |
| *** early has quit IRC | 13:21 | |
| *** openstackrecheck has quit IRC | 13:22 | |
| *** openstackrecheck has joined #openstack-qa | 13:22 | |
| *** samuelmz has quit IRC | 13:24 | |
| *** early has joined #openstack-qa | 13:24 | |
| *** Obulpathi has joined #openstack-qa | 13:31 | |
| *** yfried has quit IRC | 13:31 | |
| *** dkorn has joined #openstack-qa | 13:34 | |
| dkranz | anteaya, sdague, mtreinish : It looks like I will be able to go to MSP for the neutron thing. | 13:35 |
| dkranz | sdague, mtreinish : Also that I will not be able to go to Frankfort even if the QA date is changed. | 13:36 |
| sdague | ok cool | 13:37 |
| mtreinish | dkranz: ok, that's too bad about Germany | 13:38 |
| anteaya | dkranz: yay for being able to attend the neutron thing, sad face for missing Germany | 13:38 |
| dkranz | mtreinish: Yeah, an internal meeting got scheduled for that week | 13:38 |
| anteaya | dkranz: I asked mestery what I needed to do to prepare for the neutron thing and he said show up | 13:39 |
| anteaya | so I'm going to show up | 13:39 |
| *** Obulpathi has quit IRC | 13:39 | |
| dkranz | mtreinish: I think if we are going to do this in the future we need to plan more ahead so we can put it on our calendars and have a better chance of more people being able to attend. | 13:39 |
| mestery | dkranz anteaya: Two QA folks for Neutron mid-cycle now? | 13:39 |
| dkranz | mestery: Who is the other? | 13:40 |
| anteaya | mestery: well dkranz is qa I'm infra | 13:40 |
| mtreinish | dkranz: that's fair, this was kind of an afterthought this cycle | 13:41 |
| anteaya | but 2 non-neutron folk willing to show up and do what we can to help | 13:41 |
| mestery | anteaya dkranz: Sorry, anteaya sometimes I think of you as both QA and infra. :) | 13:41 |
| dkranz | mtreinish: I know and am not complaining | 13:41 |
| anteaya | mestery: ah okay, funny that, not how I self-describe | 13:41 |
| anteaya | but I'll take it as a compliment | 13:41 |
| mestery | anteaya: It was for sure. :) | 13:41 |
| anteaya | thanks | 13:41 |
| mestery | mtreinish: Planning mid-cycles is a part time job nowadays, isn't it? :) | 13:41 |
| anteaya | mestery: so you know we are not +Aing anything right now, right? | 13:42 |
| mtreinish | mestery: yeah, pretty much :) | 13:42 |
| mestery | anteaya: Yes, I've seen the gate for the past few days ... | 13:42 |
| anteaya | mestery: you read sdague's email of about an hour ago? | 13:42 |
| mestery | mtreinish: I'm planning two, so I feel your pain :( | 13:42 |
| dkranz | mtreinish: The weather in Boston is lovely in January I hear. | 13:42 |
| mtreinish | dkranz: we did Montreal this past Jan., so I'm thinking Southern Hemisphere for the K one. :) | 13:43 |
| anteaya | :D | 13:45 |
| anteaya | now that you have a longer travel leash | 13:45 |
| mtreinish | anteaya: yeah exactly :) | 13:46 |
| *** TravT has joined #openstack-qa | 13:49 | |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: tracking bug for connection issues between HP and RAX https://review.openstack.org/98112 | 13:59 |
| *** ekhugen has quit IRC | 14:00 | |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/devstack: make get-pip.py download failures more clear https://review.openstack.org/98098 | 14:01 |
| *** jordanP has quit IRC | 14:02 | |
| *** jordanP has joined #openstack-qa | 14:02 | |
| *** tkammer has quit IRC | 14:04 | |
| sdague | dkranz: https://review.openstack.org/#/c/97842/ - the failures are due to the top issue listed here - https://etherpad.openstack.org/p/gatetriage-june2014 | 14:05 |
| sdague | there was a substantial nova regression on resize that snuck through last night | 14:05 |
| sdague | which is being reverted right now | 14:06 |
| dkranz | sdague: ah, ok, thanks. Quite a concidence! | 14:06 |
| sdague | yeh... it's been an interesting 24hrs | 14:06 |
| dkranz | sdague: Anything I can do to help, specifically? | 14:07 |
| sdague | almost every fix for a race that I've proposed has managed to get killed by a different race | 14:07 |
| dkranz | sdague: ouch | 14:07 |
| sdague | dkranz: just what I put on the list, help categorize the fails in the gate, and help work on any of the race issues | 14:07 |
| *** malini has joined #openstack-qa | 14:07 | |
| dkranz | sdague: ok, will do | 14:08 |
| *** jordanP has quit IRC | 14:08 | |
| *** yfried has joined #openstack-qa | 14:10 | |
| *** Obulpathi has joined #openstack-qa | 14:14 | |
| *** ekhugen has joined #openstack-qa | 14:15 | |
| *** jhenner has quit IRC | 14:16 | |
| *** andreaf has quit IRC | 14:16 | |
| *** tkammer has joined #openstack-qa | 14:20 | |
| sdague | mriedem: commented - https://review.openstack.org/#/c/98112/ | 14:21 |
| *** aysyd has joined #openstack-qa | 14:22 | |
| sdague | maurosr: you got time to find me a bug for this - https://review.openstack.org/#/c/97940/ ? I never registered one, I was just trying to fix the failures that will preventing other races from getting it | 14:24 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/tempest: Add Tests for Message & Claim APIs https://review.openstack.org/94944 | 14:25 |
| mtreinish | sdague: http://www.yaml.org/spec/1.2/spec.html section 6.5, >- is line folding while > is block folding | 14:25 |
| mtreinish | but don't ask me what the distinction is | 14:25 |
| mtreinish | because I have no clue what those diagrams are supposed to me | 14:25 |
| mtreinish | an | 14:25 |
| sdague | heh | 14:25 |
| maurosr | sdague: let me see if there are any, otherwise I can fill as todo for the future | 14:26 |
| mriedem | crazy emacs | 14:27 |
| sdague | oh, so I think > preserves more of the whitespace | 14:27 |
| maurosr | sdague: https://bugs.launchpad.net/tempest/+bug/1322011 | 14:27 |
| mriedem | sdague: otherwise, that query seems realy generic | 14:27 |
| maurosr | I'll add more than the stack trace there later | 14:28 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326820 https://review.openstack.org/98124 | 14:28 |
| sdague | mriedem: it does | 14:30 |
| sdague | but run it | 14:30 |
| sdague | it's actually pretty specific | 14:30 |
| mriedem | sdague: i did, yeah, 4 hits | 14:30 |
| sdague | it's 4 hits in 7 days | 14:30 |
| sdague | 2 jobs | 14:30 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: don't die on cleanup of volume https://review.openstack.org/97940 | 14:33 |
| *** Longgeek has quit IRC | 14:33 | |
| *** jgrimm has joined #openstack-qa | 14:35 | |
| sdague | maurosr: https://review.openstack.org/#/c/97940/ I updated with bug, but due to current gerrit behavior it doesn't reset vote | 14:37 |
| maurosr | sdague: done | 14:37 |
| sdague | mriedem: also, is jsbryant within whacking up the side of the head distance? | 14:37 |
| mriedem | sdague: hmmm, maybe | 14:38 |
| *** tkammer has quit IRC | 14:38 | |
| mriedem | sdague: what for? | 14:38 |
| sdague | his feedback on https://review.openstack.org/#/c/97940/ | 14:38 |
| sdague | PS1 | 14:38 |
| sdague | https://review.openstack.org/#/c/97940/1/tempest/api/volume/test_volumes_get.py | 14:39 |
| mriedem | ha | 14:39 |
| mriedem | that's all it takes to get hit around here? | 14:39 |
| sdague | I mean... wat | 14:39 |
| mriedem | brb | 14:39 |
| *** dims_ has joined #openstack-qa | 14:44 | |
| *** evgenyf has joined #openstack-qa | 14:44 | |
| *** jaypipes has joined #openstack-qa | 14:45 | |
| *** ChanServ sets mode: +o jaypipes | 14:45 | |
| *** samuelbercovici has joined #openstack-qa | 14:47 | |
| *** ekhugen has quit IRC | 14:48 | |
| *** Obulpathi has quit IRC | 14:51 | |
| *** Obulpathi has joined #openstack-qa | 14:52 | |
| *** coolsvap|afk is now known as coolsvap | 14:53 | |
| *** jgrimm has quit IRC | 14:53 | |
| *** kgriffs|afk is now known as kgriffs | 14:55 | |
| *** tkammer has joined #openstack-qa | 14:55 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/98108 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 14:57 |
| *** ekhugen has joined #openstack-qa | 15:00 | |
| mkoderer | mtreinish: I will send a detailed plan for the mid cycle summit to jim and you | 15:01 |
| mkoderer | mtreinish: could you have a look and give feedback EOB today? We need to fix the room reservations :) | 15:02 |
| *** mpavlase has left #openstack-qa | 15:02 | |
| sdague | mtreinish: what every happened with your capture failure changes? | 15:02 |
| *** bkopilov has quit IRC | 15:02 | |
| mkoderer | sdague: you would be very welcome to join the mid cycle meetup btw ;) | 15:03 |
| sdague | mkoderer: yeh, that's my plan now | 15:03 |
| mkoderer | sdague: ok, cool :) just because of the german beer? :D | 15:04 |
| *** rbak has quit IRC | 15:06 | |
| mkoderer | dkranz: did you already had a chance to have a look to the profiling topic? | 15:07 |
| *** dustins has joined #openstack-qa | 15:07 | |
| mkoderer | dkranz: I did for some secs... if you don't mind I could propose a qa-spec | 15:07 |
| mriedem | sdague: alternative query for https://review.openstack.org/#/c/98112/ - thoughts? http://goo.gl/GtGTXR | 15:07 |
| *** atiwari has joined #openstack-qa | 15:08 | |
| *** jchai is now known as jchai_afk | 15:08 | |
| sdague | mriedem: I'm fine with that if you want to push it as an update | 15:09 |
| mtreinish | mkoderer: sure go ahead and send it to me, but I think jeblair is out until next week | 15:09 |
| mtreinish | I'll give it my feedback asap | 15:09 |
| mtreinish | sdague: it's still up for review | 15:09 |
| mkoderer | mtreinish: done | 15:09 |
| sdague | mtreinish: url ? | 15:10 |
| mtreinish | I got it working, but we'll need a devstack-gate change to capture the new failure location | 15:10 |
| sdague | mtreinish: ok, can you do a rev for that | 15:10 |
| mtreinish | sdague: https://review.openstack.org/#/c/97894/ | 15:10 |
| mtreinish | sdague: sure, one sec | 15:10 |
| sdague | there is this cinder backup service not showing up on new | 15:10 |
| *** tkammer has quit IRC | 15:10 | |
| sdague | issues that just bounced out the promoted item | 15:10 |
| dkranz | mkoderer: I did not have a chance. Just back from travel. I will take a look at what you propose. | 15:13 |
| *** jchai_afk is now known as jchai | 15:14 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Added query for bug 1323524 https://review.openstack.org/98146 | 15:16 |
| mtreinish | sdague: oh wait you meant the devstack failures not the tempest failrue reorg | 15:17 |
| mtreinish | I think that already merged | 15:17 |
| mtreinish | let me double check | 15:17 |
| mtreinish | sdague: yeah it did: https://review.openstack.org/#/c/91965/ | 15:17 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for bug 1321826 https://review.openstack.org/98149 | 15:21 |
| *** coolsvap is now known as coolsvap|afk | 15:23 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for bug 1321826 https://review.openstack.org/98149 | 15:23 |
| mriedem | sdague: so how do i categorize something that doesn't have a bug? open one? | 15:27 |
| mriedem | http://logs.openstack.org/13/94813/6/gate/gate-nova-pep8/a567f14/console.html#_2014-06-04_05_22_25_603 | 15:27 |
| mriedem | things got caught with this pep8 check in the gate | 15:27 |
| sdague | mriedem: yes | 15:27 |
| mriedem | i mean it's already fixed | 15:27 |
| mriedem | just need a bug for tracking? | 15:27 |
| sdague | yes, need a bug for tracking | 15:27 |
| mriedem | ok | 15:27 |
| sdague | that's actually the Spamaps hole | 15:27 |
| mriedem | i don't want to talk about Spamaps' hole | 15:28 |
| mriedem | oh right... | 15:28 |
| sdague | where during a reenqueue it uplifted a jenkins -1ed patch into the gate because a child patch had passing tests | 15:28 |
| mriedem | yup | 15:29 |
| mriedem | just saw pep8 had 14.8% classification rate | 15:29 |
| mriedem | which you know is odd in the gate | 15:29 |
| sdague | :) | 15:29 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Treat tripleo builds as openstack projects https://review.openstack.org/98154 | 15:32 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova pep8 gate fail bug 1326848 https://review.openstack.org/98156 | 15:34 |
| mriedem | sdague: ^ there you go | 15:34 |
| mriedem | ceilometer unit tests are unhappy | 15:35 |
| *** coolsvap|afk is now known as coolsvap | 15:36 | |
| *** weshay has quit IRC | 15:39 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326754 https://review.openstack.org/98085 | 15:39 |
| *** yrabl has quit IRC | 15:39 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for bug 1324885 https://review.openstack.org/98159 | 15:40 |
| openstackgerrit | Malini Kamalambal proposed a change to openstack/tempest: Add Tests for Message & Claim APIs https://review.openstack.org/94944 | 15:40 |
| *** mrodden has joined #openstack-qa | 15:41 | |
| *** rdekel has quit IRC | 15:43 | |
| mkoderer | dkranz: ok I'll do that tomorrow | 15:44 |
| dkranz | mkoderer: great. Sorry I cannot make it in July. We are going to schedule further out next time. | 15:44 |
| *** jchai is now known as jchai_afk | 15:45 | |
| *** jchai_afk is now known as jchai | 15:45 | |
| mkoderer | dkranz: ok sure np | 15:46 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova pep8 gate fail bug 1326848 https://review.openstack.org/98156 | 15:46 |
| *** afazekas has quit IRC | 15:47 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer test_ordination_assign timeout bug 1326855 https://review.openstack.org/98160 | 15:47 |
| mriedem | another one ^ | 15:47 |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/devstack-gate: Capture tempest failure files https://review.openstack.org/98162 | 15:48 |
| mriedem | sdague: mtreinish: if you know how to get multiple hits from the same traceback let me know | 15:48 |
| mtreinish | mriedem: if it's python logged (not on the console) it should be treated as a single line | 15:48 |
| mriedem | mtreinish: ah, this is console | 15:48 |
| mriedem | unit test fail | 15:48 |
| mriedem | so that's why it doesn't work | 15:48 |
| mtreinish | yeah then no luck there | 15:48 |
| mriedem | https://review.openstack.org/#/c/98160/ | 15:48 |
| mriedem | ok so i filter on gate queue only | 15:49 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Treat tripleo builds as openstack projects https://review.openstack.org/98154 | 15:49 |
| mtreinish | mriedem: anteaya beat you to it: https://review.openstack.org/#/c/98124/1 | 15:49 |
| mriedem | different test | 15:50 |
| mtreinish | seriously, I only read to the class name | 15:50 |
| mtreinish | sigh... | 15:50 |
| mriedem | gawd! | 15:50 |
| mriedem | there are 3 ceilometer ut fails right now | 15:50 |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Refine queries for tripleo jobs https://review.openstack.org/98096 | 15:51 |
| anteaya | more than 3 | 15:51 |
| anteaya | I just found another | 15:51 |
| anteaya | tests.alarm.test_rpc.TestRPCAlarmPartitionCoordination.test_ordination_assign | 15:51 |
| mriedem | anteaya: i already got that one | 15:51 |
| mriedem | https://bugs.launchpad.net/ceilometer/+bug/1326855 | 15:51 |
| anteaya | great | 15:51 |
| *** che-arne has joined #openstack-qa | 15:51 | |
| mriedem | doing test_ordination_allocate now | 15:51 |
| anteaya | done it | 15:51 |
| mriedem | anteaya: link? | 15:52 |
| anteaya | sorry I am not adding my e-r queries to the bug report | 15:52 |
| *** openstackrecheck has quit IRC | 15:52 | |
| anteaya | https://review.openstack.org/#/c/98146/1/queries/1323524.yaml | 15:52 |
| *** openstackrecheck has joined #openstack-qa | 15:52 | |
| mriedem | context in the commit message would be nice :) | 15:52 |
| anteaya | I don't know what to put | 15:53 |
| anteaya | I find the fail message and create a query | 15:53 |
| mriedem | for unit test fails, i at least put the project | 15:53 |
| mriedem | here is an example https://review.openstack.org/#/c/98160/ | 15:53 |
| anteaya | I can add something to the fail message | 15:53 |
| anteaya | kk, thanks | 15:53 |
| mriedem | also, see what i'm doing there about filtering on gate only | 15:53 |
| mriedem | because it could fail in check queue on someone's crappy change | 15:53 |
| anteaya | should I add that to my queries as well? | 15:55 |
| mriedem | what? | 15:55 |
| mriedem | build_queue:"gate"? | 15:55 |
| mriedem | i think so | 15:55 |
| anteaya | yes, okay | 15:56 |
| *** aimon has quit IRC | 15:56 | |
| anteaya | and you escape the : in your message | 15:56 |
| anteaya | FAIL\: | 15:56 |
| mriedem | b/c i'm paranoid | 15:56 |
| anteaya | so I should probably do that too | 15:56 |
| *** coolsvap is now known as coolsvap|afk | 15:56 | |
| anteaya | I could stand to be more paranoid | 15:56 |
| anteaya | I'll fix my queries | 15:56 |
| mtreinish | it works fine either way (escaped or not) | 15:56 |
| mriedem | famous last words! | 15:57 |
| anteaya | the only unclassified ut ceilometer failure I still haven't done anythign with is http://logs.openstack.org/88/95488/3/gate/gate-ceilometer-python26/f6976dc/console.html | 15:57 |
| anteaya | if you want to do it while I fix my queries | 15:57 |
| anteaya | mriedem: on the first line you also have: query: >- | 15:58 |
| anteaya | I have just been putting query: > | 15:59 |
| anteaya | no hyphen | 15:59 |
| mtreinish | mriedem: I can understand the paranoia, I always add a . when I run commands in a shell just to make sure I didn't screw up paths... | 16:00 |
| mriedem | anteaya: i was just copying other queries | 16:00 |
| mriedem | picked up sdague's emacs stuff i guess | 16:00 |
| anteaya | well I can do worse than copy you | 16:01 |
| *** tmcpeak has joined #openstack-qa | 16:01 | |
| anteaya | so I will copy you | 16:01 |
| *** levi_b has joined #openstack-qa | 16:01 | |
| mriedem | devananda: lifeless: 0% classification rate on ironic UT failures in the gate http://status.openstack.org/elastic-recheck/data/uncategorized.html | 16:01 |
| *** marun_afk is now known as marun | 16:02 | |
| devananda | mriedem: what does "classification rate" mean? | 16:02 |
| mriedem | devananda: means there are race bugs in the gate for those projects which aren't fingerprinted in elastic-recheck | 16:03 |
| mriedem | so e-r doesn't report back on patches when there are fails on known bugs | 16:03 |
| mriedem | jgriffith: same for cinder UT in the gate ^ | 16:04 |
| mriedem | and glance, if there are glance peeps | 16:04 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer keystone mock bug 1324885 https://review.openstack.org/98159 | 16:05 |
| mriedem | and horizon and sahar | 16:05 |
| mriedem | *sahara | 16:05 |
| mriedem | damn | 16:05 |
| mriedem | yesterday was bug day for nova, today is e-r classification day | 16:05 |
| *** weshay has joined #openstack-qa | 16:06 | |
| mriedem | anteaya: so you don't have anything for this one yet right? | 16:07 |
| mriedem | http://logs.openstack.org/88/95488/3/gate/gate-ceilometer-python26/f6976dc/console.html#_2014-06-02_08_52_21_679 | 16:07 |
| *** _afazekas is now known as afazekas | 16:08 | |
| anteaya | mriedem: correct, I have nothing for that one | 16:08 |
| *** dkorn has quit IRC | 16:08 | |
| mriedem | anteaya: ok, opening bug - looks like a problem on setup with cliff | 16:09 |
| * anteaya nods | 16:09 | |
| *** mmaglana has quit IRC | 16:10 | |
| *** mmaglana has joined #openstack-qa | 16:11 | |
| anteaya | it test_ordination_presence the same bug report as test_ordination_allocate? | 16:13 |
| anteaya | lp seems to think they are dups | 16:13 |
| anteaya | I don't know | 16:13 |
| *** levi_b has quit IRC | 16:13 | |
| mriedem | i don't know anything about the ceilometer unit tests | 16:14 |
| mriedem | are they going down common code paths? | 16:14 |
| mriedem | anteaya: in one of the bug reports someone from the ceilometer team was saying something about OS_TEST_TIMEOUT valus changing recently, or testr.conf | 16:14 |
| SergeyLukjanov | mriedem, is something bad with sahara? | 16:14 |
| sdague | dkranz: can I get your +A on this - https://review.openstack.org/#/c/97842/ | 16:14 |
| mriedem | SergeyLukjanov: http://status.openstack.org/elastic-recheck/data/uncategorized.html#gate-sahara-python27 | 16:14 |
| mriedem | SergeyLukjanov: need a bug report and an e-r query | 16:15 |
| SergeyLukjanov | mriedem, I'll take a look on it | 16:15 |
| *** mmaglana has quit IRC | 16:16 | |
| mriedem | SergeyLukjanov: thanks | 16:16 |
| mriedem | fingprinting UT failures in the gate should be easy | 16:16 |
| mriedem | those are the lowest of hanging fruit for e-r queries imo | 16:16 |
| SergeyLukjanov | mriedem, it's just pypi.o.o connection failed | 16:16 |
| SergeyLukjanov | mriedem, yup, I have some exp of e-r queries | 16:17 |
| mriedem | SergeyLukjanov: hmmm, we should still have a bug | 16:17 |
| mriedem | maybe infra or devstack bug? idk, seems like something that won't be fixed | 16:17 |
| anteaya | kk, I'll proceed with my fingerprint patch and when those that know ceilometer get around to it, they can review my bug report | 16:17 |
| mriedem | http://logs.openstack.org/92/96592/4/gate/gate-sahara-python27/a2bfa8d/console.html#_2014-06-04_01_18_39_211 | 16:17 |
| SergeyLukjanov | mriedem, IIRC there was an issue for it, hm... | 16:18 |
| *** openstackgerrit has quit IRC | 16:20 | |
| *** openstackgerrit has joined #openstack-qa | 16:21 | |
| *** jgrimm has joined #openstack-qa | 16:21 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer test_ordination_presence bug 1326820 https://review.openstack.org/98124 | 16:21 |
| mriedem | anteaya: this actually just looks like a latent py26 setuptools issue | 16:21 |
| mriedem | http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiVHlwZUVycm9yOiBleHBlY3RpbmcgYSBieXRlcyBvYmplY3QsIGdvdCB1bmljb2RlXCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIgQU5EIGJ1aWxkX3N0YXR1czpcIkZBSUxVUkVcIiBBTkQgYnVpbGRfcXVldWU6XCJnYXRlXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MDE5ODQ4Mjc1MjMsIm1vZGUiOiIiLCJhbmFseXplX2ZpZWxkIjoiIn0= | 16:21 |
| *** aimon__ has joined #openstack-qa | 16:22 | |
| *** weshay has quit IRC | 16:22 | |
| mriedem | mordred: this is a long shot, but is this something pbr could deal with? http://goo.gl/tu63Sl | 16:23 |
| mordred | mriedem: pbr can deal with anything | 16:23 |
| mriedem | nice | 16:23 |
| mordred | (looking) | 16:23 |
| mriedem | some goofy py26 setuptools race | 16:24 |
| mordred | mriedem: do we have a link to a patch with the full traceback? | 16:25 |
| *** rbak has joined #openstack-qa | 16:25 | |
| mriedem | mordred: http://logs.openstack.org/88/95488/3/gate/gate-ceilometer-python26/f6976dc/console.html | 16:25 |
| *** luqas has quit IRC | 16:25 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/98108 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 16:26 |
| anteaya | mriedem: might that be left over from when setuptools broke everything? | 16:26 |
| mriedem | anteaya: maybe, it does spike on 6/2 and then it's gone | 16:26 |
| anteaya | it happedn in the right time frame | 16:27 |
| anteaya | we might be looking at a ghost | 16:27 |
| mriedem | different query | 16:28 |
| mriedem | https://review.openstack.org/#/c/97221/1/queries/1325514.yaml | 16:28 |
| anteaya | ah | 16:28 |
| mriedem | maybe i can just adjust the other query | 16:29 |
| mriedem | first i need to fill my gut with bbq | 16:29 |
| mriedem | bbiab | 16:29 |
| devananda | mriedem: looking at addign some E-R queries for ironic now | 16:29 |
| mriedem | devananda: great, thanks | 16:29 |
| *** che-arne has quit IRC | 16:29 | |
| devananda | mriedem: granted the last 24hr have really only seen ironic fail due to https://bugs.launchpad.net/ironic/+bug/1326312 | 16:30 |
| devananda | as our checka nd gate queues are blocked by a nova chaneg that landed yesterday | 16:30 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Added query for ceilometer test_ordination_allocate bug 1323524 https://review.openstack.org/98146 | 16:32 |
| *** evgenyf has quit IRC | 16:32 | |
| *** mmaglana has joined #openstack-qa | 16:34 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer periodic notifier ut bug 1321826 https://review.openstack.org/98149 | 16:37 |
| *** safchain has quit IRC | 16:37 | |
| openstackgerrit | Derek Higgins proposed a change to openstack-infra/elastic-recheck: Refine queries for tripleo jobs https://review.openstack.org/98096 | 16:37 |
| anteaya | mriedem: bbq sounds like a good choice | 16:37 |
| *** asselin_ has quit IRC | 16:38 | |
| *** weshay has joined #openstack-qa | 16:41 | |
| *** asselin has joined #openstack-qa | 16:43 | |
| *** andreaf has joined #openstack-qa | 16:44 | |
| *** mlavalle has joined #openstack-qa | 16:44 | |
| *** coolsvap|afk is now known as coolsvap | 16:48 | |
| *** samuelbercovici has quit IRC | 16:49 | |
| *** harlowja_away is now known as harlowja | 16:50 | |
| *** jaypipes has quit IRC | 16:50 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326312 https://review.openstack.org/98175 | 16:50 |
| anteaya | I'll work on gate-cinder-python26 : 5 Uncategorized Fails. 0.0% Classification Rate (5 Total Fails) next | 16:53 |
| devananda | mriedem: i think that ^ covers all ironic failures on http://status.openstack.org/elastic-recheck/data/uncategorized.html#gate-tempest-dsvm-virtual-ironic except for the last two (one transient package mirror failure, one was already fixed) | 16:54 |
| *** ylobankov has joined #openstack-qa | 16:54 | |
| devananda | mriedem: does logstash index the py26/py27 jobs? | 16:59 |
| *** jlibosva has quit IRC | 16:59 | |
| *** rbak has quit IRC | 17:01 | |
| *** gszasz_ has quit IRC | 17:03 | |
| openstackgerrit | afazekas proposed a change to openstack-dev/devstack: [Fedora] Install dnsmasq on n-cpu less host https://review.openstack.org/98178 | 17:04 |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Added query for cinder out-of-date conf.sample bug 1326890 https://review.openstack.org/98180 | 17:18 |
| *** leseb has quit IRC | 17:21 | |
| anteaya | the out-of-date conf.sample query should categorize the cinder ut failures | 17:22 |
| openstackgerrit | Zu Qiang proposed a change to openstack-dev/devstack: Reset no_proxy when the node is installed behind a proxy https://review.openstack.org/94988 | 17:23 |
| mriedem | devananda: logstash doesn't index subunit logs, but does the console log for unit test jobs | 17:24 |
| devananda | mriedem: ack. it's probably not needed -- turns out those are all results of the same bug that's broken our gate | 17:24 |
| anteaya | I'll go here next: gate-glance-python26 : 6 Uncategorized Fails. 0.0% Classification Rate (6 Total Fails) | 17:25 |
| anteaya | mriedem: got another "got unicode" message, glance this time: http://logs.openstack.org/56/95656/5/gate/gate-glance-python26/147f699/console.html | 17:28 |
| mriedem | anteaya: yeah, this hits several projects https://bugs.launchpad.net/pbr/+bug/1326878 | 17:28 |
| mriedem | anteaya: so i just need to get a query up for that | 17:29 |
| openstackrecheck | openstack/sahara change: https://review.openstack.org/97450 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 17:29 |
| mriedem | or modify the one that's already up there | 17:29 |
| anteaya | k | 17:30 |
| *** coolsvap is now known as coolsvap|afk | 17:30 | |
| *** sthaha has quit IRC | 17:31 | |
| *** leseb has joined #openstack-qa | 17:33 | |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326312 https://review.openstack.org/98175 | 17:34 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for setuptools py26 unicode bug 1326878 https://review.openstack.org/98187 | 17:36 |
| mriedem | anteaya: ^ that's the one | 17:36 |
| mriedem | for the setuptools unicode issue | 17:36 |
| *** dosaboy has quit IRC | 17:38 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Added query for glance _authorized_upload_ bug 1326899 https://review.openstack.org/98189 | 17:39 |
| *** Sukhdev has joined #openstack-qa | 17:39 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Added query for cinder out-of-date conf.sample bug 1326890 https://review.openstack.org/98180 | 17:41 |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: tracking bug for connection issues between HP and RAX https://review.openstack.org/98112 | 17:41 |
| mriedem | sdague: ^ updated your query | 17:41 |
| *** dosaboy has joined #openstack-qa | 17:41 | |
| *** dkorn has joined #openstack-qa | 17:44 | |
| anteaya | mriedem: I reviewed it, +1 | 17:44 |
| anteaya | I think that takes care of ceilometer, cinder and glance ut | 17:44 |
| anteaya | I am going to go for a walk, should be back in about an hour | 17:44 |
| *** SlickNik has quit IRC | 17:45 | |
| *** dkorn has left #openstack-qa | 17:46 | |
| *** masayukig has quit IRC | 17:46 | |
| anteaya | I did ask in -neutron to see if anyone was helping categorize failures and got crickets | 17:46 |
| *** SlickNik has joined #openstack-qa | 17:46 | |
| *** leseb has quit IRC | 17:47 | |
| mriedem | anteaya: thanks for the help | 17:48 |
| *** masayukig has joined #openstack-qa | 17:51 | |
| mriedem | devananda: small updated needed here: https://review.openstack.org/#/c/98175/ | 17:51 |
| *** markmcclain has joined #openstack-qa | 17:52 | |
| *** marun has quit IRC | 17:56 | |
| *** marun has joined #openstack-qa | 17:56 | |
| devananda | mriedem: just change the filename? your second comment seems to suggest putting it all on one line | 17:57 |
| *** levi_b has joined #openstack-qa | 17:57 | |
| *** levi_b has joined #openstack-qa | 17:58 | |
| mriedem | devananda: use tags rather than filename | 17:58 |
| mriedem | that's the only change | 17:58 |
| devananda | k | 17:58 |
| mriedem | tags:"screen-n-sch.txt" | 17:58 |
| devananda | ack | 17:58 |
| mriedem | otherwise logs/* only hits on tempest jobs, misses grenade | 17:58 |
| devananda | i''ll improve the message a bit too, while editing | 17:58 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for ceilometer test_ordination_presence bug 1326820 https://review.openstack.org/98124 | 17:58 |
| devananda | ahh, good to know | 17:58 |
| sdague | afazekas: so I'm going to violently disagree with you | 18:00 |
| openstackgerrit | Devananda van der Veen proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326312 https://review.openstack.org/98175 | 18:00 |
| *** vrovachev has quit IRC | 18:01 | |
| *** rockyg has joined #openstack-qa | 18:01 | |
| sdague | I look at a lot of logs, and have been, and we are failing on delete alone on volumes quite a bit, and servers some | 18:01 |
| *** leseb has joined #openstack-qa | 18:01 | |
| afazekas | sdague: I just would like to see evidence, before declaring war against teardowns | 18:01 |
| afazekas | was it the only failure on the test run ? | 18:02 |
| sdague | afazekas: yes, it was the only failure | 18:02 |
| afazekas | volume deletion fails if it is in-use | 18:02 |
| sdague | afazekas: if you want evidence, devote the next 2 days to only working on gate bugs | 18:02 |
| afazekas | Those type of failures has another excpetion | 18:02 |
| sdague | I don't have the time to spoon feed you that | 18:02 |
| sdague | or the patience | 18:02 |
| afazekas | I will not be here on the next 4 days :( | 18:03 |
| afazekas | Almost on every morning I check the recheck pages | 18:03 |
| sdague | I realize I am being pretty short about this, but there are a very small number of folks actually working through these issues | 18:03 |
| sdague | afazekas: not everything is on there | 18:04 |
| afazekas | sdague: I know, lot of issue missing | 18:04 |
| sdague | anyway, a different missing piece, which would actually be really helpful | 18:05 |
| afazekas | On several days I just report 3 bugs just from the failures from my change-sets | 18:05 |
| sdague | is analysis of which tests are most likely | 18:05 |
| sdague | to fail | 18:05 |
| *** Sukhdev has quit IRC | 18:05 | |
| sdague | because I think realistically there are a few hot spots | 18:05 |
| andreaf | sdague: +1 | 18:05 |
| *** leseb has quit IRC | 18:05 | |
| mriedem | we have the test class/test case in the failure now right? | 18:05 |
| sdague | and if we prioritized those tests for some extra sanitization and logging, it would go a huge way | 18:06 |
| mriedem | so we should be able to analyze that easier | 18:06 |
| sdague | mriedem: yeh, I think all the data is out there | 18:06 |
| mriedem | cool | 18:06 |
| afazekas | Long time ago I saw a pattern of the n-cpu gots the action and does nothing | 18:06 |
| andreaf | sdague: sepcifically when analizing failures it would be great to know if a failed test is common failure or not | 18:06 |
| *** cdent has quit IRC | 18:06 | |
| sdague | someone just needs to build the analysis | 18:06 |
| sdague | andreaf: sure | 18:06 |
| *** openstackrecheck has quit IRC | 18:06 | |
| *** openstackrecheck has joined #openstack-qa | 18:06 | |
| sdague | but even right now, which tempest tests failed would let us know what we should be spending time making more debugable | 18:07 |
| sdague | sort of test profiling | 18:07 |
| sdague | because right now I'm ad hocing that | 18:07 |
| sdague | which is fine | 18:07 |
| sdague | but we should do better | 18:07 |
| andreaf | sdague: I'm not familiar enough with logstash , can we get this kind of info from there or do we need something else? | 18:07 |
| sdague | I don't know if logstash would have everything we need | 18:08 |
| andreaf | the list of failures is in console.html which is indexed right? | 18:08 |
| *** nati_ueno has joined #openstack-qa | 18:08 | |
| sdague | andreaf: yeh | 18:08 |
| andreaf | what would be really really useful is a test result repository | 18:08 |
| afazekas | teardown issues normally does not happens alone. The volume deletion in all cases in tearDownFoo can be enhanced by a detach | 18:08 |
| sdague | afazekas: nope, not in all cases | 18:08 |
| andreaf | so not only .testreporisory which is dropped after each run | 18:09 |
| sdague | andreaf: sure | 18:09 |
| andreaf | a proper data store where we can run analysis on top, do correlations and so | 18:09 |
| afazekas | BTW: nowadays the tempest log does not contains the request/repose details, it is same cases makes more difficult to debug | 18:10 |
| andreaf | for instance, we need also the ability to put events in like known outages on rax or hpcloud | 18:10 |
| andreaf | which would generate a lot of false failures | 18:10 |
| sdague | andreaf: well er does an ok job for that | 18:10 |
| sdague | anyway, got to drop for a big | 18:10 |
| *** Sukhdev has joined #openstack-qa | 18:12 | |
| *** kgriffs is now known as kgriffs|afk | 18:12 | |
| *** bkopilov has joined #openstack-qa | 18:14 | |
| openstackrecheck | openstack/heat change: https://review.openstack.org/94780 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1326778 | 18:15 |
| *** eglynn is now known as eglynn-office | 18:17 | |
| *** bkopilov has quit IRC | 18:18 | |
| afazekas | BTW: as I remember both testfixtrures and testresources are acts on teardown | 18:18 |
| afazekas | If we want something to act inside the test case implicitly we need to override the run method | 18:19 |
| afazekas | sdague: https://review.openstack.org/#/c/96090/ , https://review.openstack.org/#/c/97532/ | 18:22 |
| *** markmcclain has quit IRC | 18:24 | |
| afazekas | when I return from the vacation, probably I will look into the heat slow failures (I have more suspected reason), and https://bugs.launchpad.net/nova/+bug/1320617 | 18:25 |
| sdague | cool | 18:26 |
| sdague | yeh, we need someone running point on the heat job | 18:27 |
| sdague | because otherwise it's going to stay non voting forever | 18:27 |
| afazekas | 1. issue is the cloud image needs to be tweeked to be more reliable without para-virt timer | 18:28 |
| afazekas | no_timer_check notsc lpj=n kernel parameters | 18:28 |
| *** Mike77 has joined #openstack-qa | 18:29 | |
| *** andreaf has quit IRC | 18:30 | |
| *** catohornet has joined #openstack-qa | 18:30 | |
| afazekas | sdague: are you ok with skipping the fwaas tests ? | 18:31 |
| afazekas | sdague: do we want to retry the connection after the EOF exception ? | 18:32 |
| *** rockyg has left #openstack-qa | 18:33 | |
| *** Mike77 has quit IRC | 18:34 | |
| dkranz | boris-42: ping | 18:35 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Added query for cinder out-of-date conf.sample bug 1326890 https://review.openstack.org/98180 | 18:36 |
| sdague | afazekas: I'm happy turning off any of the layer 7 neutron tests, they haven't been very reliable | 18:38 |
| *** openstackrecheck has quit IRC | 18:38 | |
| sdague | so fwaas lbaas | 18:38 |
| *** openstackrecheck has joined #openstack-qa | 18:38 | |
| sdague | I sort of feel like those need to live in some other holding pen for a while | 18:38 |
| mtreinish | sdague: I support that | 18:39 |
| mtreinish | mestery: ^^^ | 18:39 |
| *** nati_ueno has quit IRC | 18:40 | |
| sdague | maybe that's a good place to force the issue and make them come through the neutron graduation process | 18:40 |
| mestery | mtreinish sdague: Are these ones causing problems? Reading the back scroll, just got back from lunch, apologies. | 18:40 |
| * mtreinish stepping out, time to eat | 18:41 | |
| mkoderer | sdague: could you give me feedback on https://review.openstack.org/#/c/84645/ (safe teardown). We might enhance it that it really helps | 18:42 |
| sdague | mestery: yes | 18:43 |
| mkoderer | sdague: if you consider it as useful I will write a spec and unit tests ;) | 18:43 |
| sdague | Bug 1326510 - tempest.api.network.test_fwaas_extensions.FWaaSExtensionTestJSON fails with 409 | 18:43 |
| sdague | 39 fails in 24hrs / 291 fails in 14 days | 18:43 |
| sdague | mestery: lbaas has had similar issues recently | 18:43 |
| sdague | though it might be under control now | 18:43 |
| mestery | sdague: We have someone looking into the FWaaS issue now, but it's turning out to be hard to reproduce, there was some hope they may have a solution today. | 18:43 |
| *** ylobankov has quit IRC | 18:44 | |
| mestery | sdague: Regarding LBaaS, I know they have had issues in the past. | 18:44 |
| sdague | the issue is there are a lot of moving parts here | 18:44 |
| mestery | sdague: +1 | 18:44 |
| sdague | and there is already a regular issue with neutron being able to reliably build a network for a compute | 18:44 |
| lifeless | mtreinish: right, I've been proposing finger prints recently, but certainly more are needed | 18:44 |
| afazekas | I would like to keep the lbaas for while | 18:44 |
| mestery | sdague: Which issue is that? The "ssh timeout" issues we're debugging now? | 18:45 |
| sdague | mestery: yep | 18:45 |
| sdague | that bug we've been tracking for a year | 18:45 |
| sdague | afazekas: sure, that's fine | 18:45 |
| mestery | sdague: Still looking at that one today, the thing is it manifests itself as "ssh timeout", and salv-orlando and I think it could be a few different things. | 18:45 |
| mestery | sdague: But no doubt, it's been a problem, and we're doing our best to sort that out. | 18:45 |
| sdague | afazekas: I was just saying in general, I feel like the layer 7 services in neutron aren't incredibly useful when layer 3 still is very racy | 18:46 |
| sdague | especially from a repeatable validation pov | 18:46 |
| salv-orlando | sdague: frankly I don’t think this time we have to with a neutron l3 race. I wish we had the tools to prove it quickly though. | 18:47 |
| dkranz | mkoderer: Could you check my last comment in the client-check-success-code qa-spec? | 18:47 |
| sdague | salv-orlando: ok | 18:47 |
| salv-orlando | Said that, I agree with your statement on l7 services | 18:47 |
| sdague | salv-orlando: what tools do you think you need to prove it, even if not quickly? | 18:48 |
| mkoderer | dkranz: sure, ok if I'll do it tomorrow morning? | 18:49 |
| dkranz | mkoderer: yup | 18:49 |
| afazekas | sdague: https://review.openstack.org/#/c/90782/ | 18:49 |
| sdague | mriedem: look at ER, I think Bug 1263417 - Devstack failed to resolve domain name during tempest tests triggered by gerrit | 18:49 |
| sdague | and Bug 1286818 - Ubuntu mirror periodically inconsistent causing gate build failures | 18:49 |
| sdague | or dup | 18:49 |
| sdague | are dup | 18:49 |
| *** nati_ueno has joined #openstack-qa | 18:50 | |
| *** ildikov has joined #openstack-qa | 18:55 | |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Retry ssh connection an all exception https://review.openstack.org/98212 | 18:56 |
| *** Obulpathi has quit IRC | 18:59 | |
| afazekas | salv-orlando: One thing I do not like in the neutron code, it is difficult to see is the event-let monkey patch called in time. https://github.com/openstack/nova/blob/master/nova/cmd/__init__.py collecting the entry modules to one package and doing the monkey patch in __init__.py seams more reliable, than calling it in the main after all module import | 19:01 |
| anteaya | mriedem: back | 19:01 |
| anteaya | mriedem: what should I work on next/ | 19:01 |
| anteaya | ? | 19:02 |
| anteaya | mriedem: do you want me to remove line 5 of this patch? https://review.openstack.org/#/c/98189/1/queries/1326899.yaml | 19:02 |
| salv-orlando | afazekas: that could be done too. Sounds a reasonable thing to do | 19:03 |
| *** adalbas has quit IRC | 19:04 | |
| *** jchai is now known as jchai_afk | 19:04 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: tracking bug for connection issues between HP and RAX https://review.openstack.org/98112 | 19:05 |
| afazekas | BTW: all multi-threaded (or eventleted) python project needs to be more careful about double fd close issues | 19:06 |
| *** nati_ueno has quit IRC | 19:07 | |
| *** Obulpathi has joined #openstack-qa | 19:07 | |
| *** openstackrecheck has quit IRC | 19:08 | |
| *** openstackrecheck has joined #openstack-qa | 19:08 | |
| jogo | mriedem: removing old er queries so we don't overload elasticsearch, is safe to be marked as clsed? https://bugs.launchpad.net/tempest/+bug/1254772 no hits in 14 days | 19:09 |
| *** jaypipes has joined #openstack-qa | 19:09 | |
| *** ChanServ sets mode: +o jaypipes | 19:09 | |
| jogo | mriedem: I am assuming it is, if not just re-open it | 19:10 |
| anteaya | I'll look at these now: gate-heat-python26 : 4 Uncategorized Fails. 55.6% Classification Rate (9 Total Fails) | 19:11 |
| *** leseb has joined #openstack-qa | 19:11 | |
| openstackrecheck | openstack/heat change: https://review.openstack.org/94755 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1286635 | 19:12 |
| *** levi_b has quit IRC | 19:14 | |
| *** stevebaker has quit IRC | 19:15 | |
| *** stevebaker has joined #openstack-qa | 19:15 | |
| *** leseb has quit IRC | 19:16 | |
| afazekas | sdague: do you know why this job does not runs tempest ? http://logs.openstack.org/90/96090/2/check/check-devstack-dsvm-cells/053c58a/console.html.gz | 19:16 |
| afazekas | cells | 19:16 |
| sdague | afazekas: because tempest doesn't work with cells | 19:16 |
| anteaya | those all fail with a multitude of tests, I have a ping into heat to gain some insight into the ball of string | 19:17 |
| *** mlavalle has quit IRC | 19:17 | |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/96781 failed because of: gate-ceilometer-python26: unrecognized error, gate-ceilometer-python27: https://bugs.launchpad.net/bugs/1326820 | 19:18 |
| mriedem | anteaya: per https://review.openstack.org/#/c/98189/ - it's not showing up in the gate, it shows up in the check queue on one glance patch | 19:20 |
| mriedem | anteaya: but shows up in the e-r uncategorized bugs page, so i'm a bit confused there | 19:21 |
| mriedem | seems like the e-r uncategorized bugs page is wrong | 19:21 |
| anteaya | mriedem: hmmmm | 19:22 |
| mriedem | jogo: on bug 1254772, wondering if the fingerprint chnaged | 19:24 |
| mriedem | *changed | 19:24 |
| *** coolsvap|afk has quit IRC | 19:25 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Remove some queries with no hits https://review.openstack.org/98224 | 19:26 |
| jogo | mriedem: if it did, we have a new bug and query for it | 19:27 |
| mriedem | jogo: yeah most likely | 19:28 |
| jogo | ^ cleanup some old er fingerprints patch to make clarkb happy | 19:29 |
| *** coolsvap|afk has joined #openstack-qa | 19:32 | |
| mriedem | jogo: maybe you fixed that problem with this? https://review.openstack.org/#/c/26166/ | 19:33 |
| mriedem | jogo: reviewing for clarkb's happiness | 19:33 |
| mriedem | jogo: nevermind on that old patch, that was 4/4/13 :) | 19:34 |
| jogo | mriedem: see https://bugs.launchpad.net/swift/+bug/1310597 | 19:35 |
| jogo | oh woops wrong bug | 19:35 |
| *** afazekas is now known as _afazekas_slp | 19:37 | |
| mriedem | jogo: https://bugs.launchpad.net/tempest/+bug/1254772 is still happening, i need to update the old query | 19:39 |
| mriedem | i broke the query back on 2/6 | 19:39 |
| mriedem | only 3 months ago :P | 19:39 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Add README section about API stability https://review.openstack.org/98230 | 19:45 |
| mriedem | jogo: the last one here: https://review.openstack.org/#/c/98224/ | 19:48 |
| mriedem | test was skipped, so i guess we just consider that the same as closed | 19:48 |
| mtreinish | mriedem: don't close the bug for a skip test unless it's been fixed... | 19:49 |
| mriedem | mtreinish: the bug is open | 19:49 |
| mtreinish | oh, nm you were saying just delete the query | 19:49 |
| mriedem | yeah | 19:49 |
| mriedem | which makes sense to me | 19:49 |
| mriedem | but had to think about | 19:49 |
| mriedem | it | 19:49 |
| mtreinish | yeah that does | 19:49 |
| mriedem | jogo: you mind looking at this one? https://review.openstack.org/#/c/98160/ | 19:51 |
| lifeless | I have a random thought | 19:51 |
| lifeless | why don't we put the queries in the bug description | 19:51 |
| *** nati_ueno has joined #openstack-qa | 19:51 | |
| mtreinish | lifeless: we thought about that, it was something we talked about in HK | 19:52 |
| mriedem | they can get big and ugly for one thing | 19:52 |
| mtreinish | and decided not to do it | 19:52 |
| mtreinish | I can't remember why | 19:52 |
| lifeless | one way that might work - e-r exports the query into the bug below a ===== line | 19:52 |
| lifeless | then you still have code review etc | 19:52 |
| lifeless | but if the query is removed, folk looking at the bug can tell e-r isn't checking it | 19:52 |
| *** levi_b has joined #openstack-qa | 19:53 | |
| mriedem | bug description in launchpad or the one line abstract? | 19:53 |
| *** yrabl has joined #openstack-qa | 19:56 | |
| lifeless | the big full text field | 19:57 |
| *** levi_b has quit IRC | 19:57 | |
| *** dkranz has quit IRC | 19:57 | |
| *** masayukig has quit IRC | 19:59 | |
| *** jchai_afk is now known as jchai | 19:59 | |
| mriedem | anteaya: https://review.openstack.org/#/c/98149/ needs a fix | 20:02 |
| mriedem | (message:"foo" OR message:"bar") | 20:02 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for ceilometer keystone mock bug 1324885 https://review.openstack.org/98159 | 20:02 |
| *** masayukig has joined #openstack-qa | 20:04 | |
| *** weshay has quit IRC | 20:04 | |
| openstackgerrit | Armando Migliaccio proposed a change to openstack/tempest: Add API tests for Neutron DVR extension https://review.openstack.org/98235 | 20:04 |
| anteaya | mriedem: () coming up | 20:05 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1326312 https://review.openstack.org/98175 | 20:06 |
| *** levi_b has joined #openstack-qa | 20:07 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for ceilometer periodic notifier ut bug 1321826 https://review.openstack.org/98149 | 20:07 |
| *** flwang has quit IRC | 20:07 | |
| *** nati_ueno has quit IRC | 20:08 | |
| *** openstackrecheck has quit IRC | 20:08 | |
| *** openstackrecheck has joined #openstack-qa | 20:08 | |
| *** yrabl_ has joined #openstack-qa | 20:10 | |
| *** adalbas has joined #openstack-qa | 20:11 | |
| *** Sukhdev has quit IRC | 20:11 | |
| *** malini is now known as malini_afk | 20:12 | |
| *** weshay has joined #openstack-qa | 20:12 | |
| *** yrabl has quit IRC | 20:13 | |
| *** flwang has joined #openstack-qa | 20:17 | |
| *** devananda has quit IRC | 20:19 | |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/grenade: add a more comprehensive check-sanity https://review.openstack.org/98241 | 20:19 |
| openstackrecheck | Console logs not available after 14:05s for check-tempest-dsvm-neutron 91081,15,3db114a | 20:23 |
| *** yrabl has joined #openstack-qa | 20:27 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for oslo.messaging unit test race bug 1283926 https://review.openstack.org/98244 | 20:27 |
| *** andreaf has joined #openstack-qa | 20:27 | |
| *** yrabl_ has quit IRC | 20:30 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for heat assert isinstance bug 1326946 https://review.openstack.org/98245 | 20:31 |
| *** ajo|mac has joined #openstack-qa | 20:34 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Added query for ceilometer test_ordination_allocate bug 1323524 https://review.openstack.org/98146 | 20:35 |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-neutron-pg 73226,6,bd8a26a | 20:36 |
| *** openstackrecheck has quit IRC | 20:37 | |
| *** openstackrecheck has joined #openstack-qa | 20:37 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Remove some queries with no hits https://review.openstack.org/98224 | 20:38 |
| openstackgerrit | A change was merged to openstack/tempest: don't reuse servers in test_server_actions https://review.openstack.org/97842 | 20:39 |
| *** eglynn_ has quit IRC | 20:45 | |
| openstackgerrit | Anita Kuno proposed a change to openstack-infra/elastic-recheck: Add query for heat assert isinstance bug 1326946 https://review.openstack.org/98245 | 20:46 |
| *** weshay has quit IRC | 20:51 | |
| *** openstackrecheck has quit IRC | 20:52 | |
| *** openstackrecheck has joined #openstack-qa | 20:53 | |
| *** yrabl has quit IRC | 20:55 | |
| *** mlavalle has joined #openstack-qa | 20:58 | |
| *** mlavalle has quit IRC | 20:58 | |
| *** mlavalle has joined #openstack-qa | 20:59 | |
| *** stevebaker has quit IRC | 21:00 | |
| *** stevebaker has joined #openstack-qa | 21:00 | |
| *** aysyd has quit IRC | 21:02 | |
| jogo | lifeless: if we have it automatically wired up I am all for it | 21:03 |
| jogo | we usually try to use related-bug: in e-r to at least got the commit comment, and only remove queries when the bug is resolved one way or another | 21:04 |
| jogo | mriedem: about to go and review all outstanding e-r patches FYI | 21:05 |
| mriedem | jogo: go nuts, i'm going to try and be in the nova meeting | 21:06 |
| * mriedem lurks | 21:06 | |
| *** malini_afk is now known as malini | 21:06 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for oslo.messaging unit test race bug 1283926 https://review.openstack.org/98244 | 21:07 |
| *** openstackrecheck has quit IRC | 21:08 | |
| *** openstackrecheck has joined #openstack-qa | 21:08 | |
| lifeless | jogo: so I was thinking for queryfiles in queryfiles, bug#= basename(query); ... | 21:10 |
| lifeless | jogo: could wire it in as a bug attachment rather than in the text | 21:10 |
| mtreinish | oh, now I remember why we didn't do that before | 21:10 |
| mtreinish | there wouldn't be any review of the queries that wayt | 21:10 |
| *** levi_b has quit IRC | 21:14 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for heat assert isinstance bug 1326946 https://review.openstack.org/98245 | 21:17 |
| *** andreaf has quit IRC | 21:17 | |
| jogo | mtreinish: ahh right, that was an explicit desire out of e-r | 21:18 |
| jogo | to be a reviewed list | 21:19 |
| jogo | lifeless: ^ | 21:19 |
| *** ekhugen has quit IRC | 21:20 | |
| anteaya | how long after a query is merged before http://status.openstack.org/elastic-recheck/data/uncategorized.html is updated? | 21:21 |
| *** nati_ueno has joined #openstack-qa | 21:22 | |
| lifeless | mtreinish: there would in what I'm suggesting | 21:22 |
| lifeless | mtreinish: because I' suggesting e-r export the query to the bug (and delete it when the query is removed from e-r) | 21:23 |
| *** openstackrecheck has quit IRC | 21:23 | |
| *** openstackrecheck has joined #openstack-qa | 21:23 | |
| mtreinish | lifeless: oh, I misread I thought you were proposing writing the queries in the bug and using that as the source | 21:23 |
| mtreinish | but you're saying publish them to the bugs | 21:24 |
| mtreinish | that wouldn't be hard | 21:24 |
| *** mkollaro has quit IRC | 21:31 | |
| mtreinish | mriedem: https://review.openstack.org/#/c/98187/ doesn't hit all the py26 jobs? | 21:31 |
| mtreinish | and why are we checking for stackalytics that shouldn't be in the gate | 21:32 |
| mriedem | mtreinish: i just listed the jobs that it failed on if you remove the build_name filters | 21:32 |
| mriedem | the bug is already fixed | 21:32 |
| mriedem | this is for classification | 21:32 |
| mriedem | open to other ideas on that one, wasn't really sure | 21:32 |
| anteaya | I do wonder why stackalytics is in the gate | 21:33 |
| mtreinish | no I'm fine with that as the query - stackalytics | 21:33 |
| mtreinish | I was just curious | 21:33 |
| *** lbragstad has quit IRC | 21:34 | |
| *** mkollaro has joined #openstack-qa | 21:34 | |
| jogo | anteaya: an hour or two, I don't remember exactly. | 21:34 |
| anteaya | why is it in the gate, though | 21:34 |
| anteaya | jogo: k, thanks | 21:34 |
| jogo | gate != integrated gate in | 21:35 |
| jogo | on that page | 21:35 |
| jogo | until recently we didn't have that data | 21:35 |
| anteaya | oh so that might be stackalytics gate | 21:35 |
| anteaya | ah | 21:35 |
| anteaya | cool thanks | 21:35 |
| jogo | I think we list what is part of integrated gate or not in logstash now | 21:36 |
| anteaya | do we | 21:36 |
| anteaya | more things I need to learn about logstash | 21:36 |
| mtreinish | jogo: but we definitely shouldn't be running the bot on stackforge projects in the gerrit stream | 21:36 |
| *** dustins has quit IRC | 21:37 | |
| jogo | so we don't record integrated gate or not in ER yet | 21:37 |
| jogo | but we should be able to now | 21:37 |
| jogo | clarkb: ^ | 21:37 |
| jogo | mtreinish: we don't run the bot on anything that isn't part of what we think the integrated gate is. but we build the uncategorized graph from everything | 21:37 |
| jogo | the bot checks if a dsvm job is in the jenkins comment, if so its in the integrated gate | 21:38 |
| jogo | we can't use that logic easily inthe uncategorized page | 21:38 |
| mtreinish | yeah that's what I thought it did | 21:38 |
| *** nati_uen_ has joined #openstack-qa | 21:39 | |
| jogo | so if we can get zuul to add 'Change queue: integrated' to ES | 21:39 |
| *** nati_ueno has quit IRC | 21:39 | |
| mriedem | ok so remove stackalytics is what i'm hearing | 21:40 |
| mriedem | from e-r | 21:40 |
| *** masayukig has quit IRC | 21:40 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for setuptools py26 unicode bug 1326878 https://review.openstack.org/98187 | 21:41 |
| *** levi_b has joined #openstack-qa | 21:45 | |
| *** Obulpathi has quit IRC | 21:45 | |
| *** masayukig has joined #openstack-qa | 21:46 | |
| *** levi_b has quit IRC | 21:49 | |
| *** jchai has quit IRC | 21:50 | |
| *** dims__ has joined #openstack-qa | 21:51 | |
| *** dims_ has quit IRC | 21:53 | |
| *** malini is now known as malini_afk | 21:54 | |
| jogo | mtreinish: so can you actually take a whack at making the uncategorized page ignore stackforge jobs? | 21:55 |
| dims__ | jogo, which ones are stackforge jobs? | 21:59 |
| *** malini_afk is now known as malini | 22:00 | |
| mtreinish | jogo: I'm doing it right now | 22:00 |
| mtreinish | I should have a WIP shortly | 22:00 |
| jogo | mtreinish: woot | 22:02 |
| *** ajo|mac has quit IRC | 22:03 | |
| *** stevebaker has quit IRC | 22:03 | |
| *** stevebaker has joined #openstack-qa | 22:03 | |
| flwang | hi is there anyway to enable/disable scenario test? thanks | 22:06 |
| *** dpaterson has joined #openstack-qa | 22:11 | |
| openstackgerrit | Mathieu Gagné proposed a change to openstack-dev/devstack: Update setuptools to latest for .dist-info support https://review.openstack.org/98263 | 22:12 |
| *** jgrimm has quit IRC | 22:16 | |
| openstackgerrit | Kyle Mestery proposed a change to openstack/tempest: Revert "Fail fast during advanced networking test" https://review.openstack.org/97245 | 22:18 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for setuptools py26 unicode bug 1326878 https://review.openstack.org/98187 | 22:18 |
| openstackrecheck | Console logs not available after 14:09s for check-grenade-dsvm 86483,17,bf364dd | 22:20 |
| *** openstackrecheck has quit IRC | 22:22 | |
| *** openstackrecheck has joined #openstack-qa | 22:22 | |
| *** stevebaker has quit IRC | 22:28 | |
| *** stevebaker has joined #openstack-qa | 22:28 | |
| openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326754 https://review.openstack.org/98085 | 22:34 |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron 98235,1,9f220fb | 22:37 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1326754 https://review.openstack.org/98085 | 22:38 |
| *** levi_b has joined #openstack-qa | 22:39 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack-infra/elastic-recheck: Add a real check for the integrated gate jobs https://review.openstack.org/98264 | 22:40 |
| mtreinish | jogo: ^^^ That's the first attempt, but I haven't tested it yet | 22:41 |
| mtreinish | but, it's pep8 clean | 22:41 |
| jogo | mtreinish: neato | 22:41 |
| mtreinish | err, no its not right | 22:43 |
| jogo | mtreinish: so the idea may not work | 22:43 |
| jogo | as in tripleo and ironic jobs aren't part of integrated gate | 22:43 |
| jogo | nor are nova-specs | 22:43 |
| jogo | maybe we should just see if its for a project under the openstack git org? | 22:44 |
| *** levi_b has quit IRC | 22:44 | |
| jogo | or rather !stackforge | 22:44 |
| jogo | mtreinish: oh heh thats what we try now | 22:45 |
| *** tmcpeak has quit IRC | 22:45 | |
| openstackgerrit | Clark Boylan proposed a change to openstack-infra/devstack-gate: Put cached files in old/ dirs too https://review.openstack.org/98265 | 22:45 |
| jogo | mtreinish: so this may have been a big wild goose chase ^_^, everything in http://status.openstack.org/elastic-recheck/data/uncategorized.html is in incubation or integrated | 22:45 |
| jogo | mtreinish: I wonder if the issue is actually the graphs themselves | 22:46 |
| mtreinish | jogo: then where did stackalytics come from? | 22:46 |
| jogo | mtreinish: good question | 22:47 |
| jogo | mriedem: ^^ | 22:47 |
| mtreinish | I didn't actually look at anything I just saw mriedem's query | 22:47 |
| mtreinish | so I assumed I needed to do something | 22:47 |
| jogo | mtreinish: me too | 22:47 |
| jogo | I think its the graphs | 22:47 |
| jogo | elastic_recheck/cmd/uncategorized_fails.py: if re.search("(^openstack/|devstack|grenade)", result.project): | 22:47 |
| mtreinish | yeah that's the line I removed for the fancy broken checker | 22:48 |
| jogo | mriedem: we don't limit the graph qeuries it looks like | 22:48 |
| jogo | mtreinish: haha | 22:48 |
| mtreinish | jogo: do we want to limit hits for the graphs? | 22:49 |
| mtreinish | they're just to show spikes | 22:49 |
| mtreinish | amd broader trends to be perfect counts | 22:49 |
| jogo | mtreinish: hmm I think we should right? | 22:49 |
| jogo | not sure | 22:49 |
| jogo | for example if we have a broken dependency issue in a stackforge job we don't really care | 22:49 |
| jogo | but not sure | 22:50 |
| * jogo feels silly | 22:50 | |
| mriedem | jogo: mtreinish: the stackalytics hit showed up when looking at a query for a different integrated project's unit test fails | 22:51 |
| mriedem | probably glance or cinder | 22:51 |
| mriedem | ran the query w/o build_name, then found it was hitting more than what i was expecting | 22:51 |
| mtreinish | jogo: it's just been one of those weeks | 22:51 |
| jogo | mtreinish: it sure is | 22:52 |
| mriedem | jogo: there is an e-r query up right now for an unclassified bug for glance unit tests on the uncategorized page and when you query the failure, it only shows up in the check queue on a single glance patch | 22:52 |
| jogo | mriedem: ahh so the graph would have been wrong | 22:52 |
| jogo | or loose | 22:52 |
| *** openstackrecheck has quit IRC | 22:52 | |
| *** openstackrecheck has joined #openstack-qa | 22:52 | |
| mriedem | this one https://review.openstack.org/#/c/98189/ | 22:52 |
| mriedem | http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiRkFJTFxcOiBnbGFuY2UudGVzdHMudW5pdC52MS50ZXN0X2FwaS5UZXN0R2xhbmNlQVBJLnRlc3RfYWRkX2NvcHlfZnJvbV9pbWFnZV9hdXRob3JpemVkX3VwbG9hZF9pbWFnZV9hdXRob3JpemVkXCIgQU5EIHRhZ3M6XCJjb25zb2xlXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjYwNDgwMCIsImdyYXBobW9kZSI6ImNvdW50IiwidGltZSI6eyJ1c2VyX2ludGVydmFsIjowfSwic3RhbXAiOjE0MDE5OTEyMTE0MjJ9 | 22:52 |
| jogo | mriedem: one hit meh | 22:52 |
| mtreinish | jogo: yeah just loose, but I honestly am not sure it's the end of the world, it wouldn't be off by that much comparing the throughput between openstack vs stackforge | 22:52 |
| mriedem | yeah, but it showed up on uncategorized page and it's a check queue failure | 22:52 |
| mriedem | i thought only gate queue failures got on the uncategorized page | 22:53 |
| jogo | mtreinish: yeah I agree | 22:53 |
| jogo | mriedem: yes only gate failres are supposed to show up there | 22:53 |
| mriedem | that glance one is check only | 22:53 |
| mriedem | so not sure why that shows up | 22:53 |
| mtreinish | jogo, mriedem: yeah: http://git.openstack.org/cgit/openstack-infra/elastic-recheck/tree/elastic_recheck/cmd/uncategorized_fails.py#n61 | 22:54 |
| *** jamielennox|away is now known as jamielennox | 22:54 | |
| jogo | are classificatio nrate is 83% :) | 22:54 |
| mriedem | a lot of that is the last 2 days | 22:55 |
| jogo | mriedem: maybe there is a single gate failure from over a week ago? | 22:55 |
| jogo | mriedem: well know we just have the bugs to fix | 22:55 |
| mtreinish | jogo: and I wonder how many of the remaining 17% are the pip thing which we can't really query for | 22:56 |
| jogo | mtreinish: yeah, although I haven't found any false positives for that one | 22:56 |
| jogo | mtreinish: but I think a lot | 22:56 |
| sdague | dtroyer: so what did I do wrong - https://review.openstack.org/#/c/98209/2/lib/nova - as it's not jumping out at me | 22:57 |
| dtroyer | once nova has started an instance, the ip/routes are moved from the physical interface to a bridge, usually br100. | 22:57 |
| dtroyer | running stack.sh a second time will set GUEST_INTERFACE_DEFAULT to '' | 22:58 |
| dtroyer | that might be OK, might not | 22:58 |
| sdague | mriedem: even in 14 days it wasn't in gate? | 22:58 |
| sdague | dtroyer: oh... right | 22:58 |
| mriedem | sdague: running that query | 23:00 |
| mriedem | but logstash is dying on me | 23:00 |
| sdague | yeh | 23:00 |
| sdague | 14 days ends up being sad panda through kibana | 23:01 |
| openstackgerrit | Miguel Lavalle proposed a change to openstack/tempest: VPNaaS API Tests Enhancements https://review.openstack.org/83627 | 23:02 |
| sdague | dtroyer: how about - ip link | grep 'state UP' | awk '{print $2}' | sed 's/://' | 23:03 |
| dtroyer | that's much better | 23:04 |
| dtroyer | I like the way that'll get interfaces in the order they were created | 23:05 |
| dtroyer | much morelikely to be useful | 23:05 |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/devstack: a more proper default nic selection criteria https://review.openstack.org/98272 | 23:07 |
| sdague | well, I was still going to sort it | 23:07 |
| sdague | you want me to drop the sort? | 23:07 |
| dtroyer | if you sort it alpha, br100 falls before eth0 | 23:08 |
| sdague | but I filtered that | 23:08 |
| dtroyer | yeah, was just checking that | 23:08 |
| sdague | I'm also ok dropping the sort | 23:08 |
| sdague | with the 'state UP' filter | 23:08 |
| sdague | it would do the right thing on all my machines by default | 23:09 |
| sdague | none of which have eth0 | 23:09 |
| sdague | it's a mix of em1 and eth1 | 23:09 |
| dtroyer | I like having it look in the order the interfaces were created | 23:09 |
| sdague | ok | 23:09 |
| sdague | let me fix that | 23:09 |
| *** sthaha has joined #openstack-qa | 23:09 | |
| *** sthaha has quit IRC | 23:09 | |
| *** sthaha has joined #openstack-qa | 23:09 | |
| mriedem | sdague: bingo | 23:09 |
| dtroyer | bridges, etc, should always come after physical interfaces unless modules are being manipulated | 23:09 |
| mriedem | 5/25 | 23:09 |
| openstackgerrit | Sean Dague proposed a change to openstack-dev/devstack: a more proper default nic selection criteria https://review.openstack.org/98272 | 23:10 |
| mriedem | logstash comes through | 23:10 |
| mriedem | jogo: mtreinish: the glance UT thing was in ht egate | 23:10 |
| sdague | dtroyer: sure, I'm still going to filter those out though | 23:10 |
| sdague | because that's never the right answer | 23:10 |
| dtroyer | ok…dinnertime… | 23:10 |
| sdague | yeh, me too | 23:11 |
| *** mlavalle has quit IRC | 23:14 | |
| *** malini is now known as malini_afk | 23:15 | |
| flwang | sdague: ping | 23:17 |
| flwang | mriedem: hey there | 23:17 |
| flwang | sdague: would you mind taking a look at this bp https://blueprints.launchpad.net/tempest/+spec/swift-scenario-enhancement ? | 23:17 |
| mtreinish | mriedem: ok everything is right in the world now | 23:17 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Added query for glance _authorized_upload_ bug 1326899 https://review.openstack.org/98189 | 23:18 |
| flwang | mtreinish: it would be nice if you can take a look at this https://blueprints.launchpad.net/tempest/+spec/swift-scenario-enhancement | 23:18 |
| mriedem | flwang: hi | 23:18 |
| flwang | i'm going to enhance the swift scenario test | 23:18 |
| flwang | mriedem: long time no see, just wanna say hi :) | 23:19 |
| mriedem | technically 'no see' is still a long time | 23:19 |
| mtreinish | flwang: you need to push a qa-spec for that work. See: https://github.com/openstack/qa-specs/blob/master/README.rst | 23:19 |
| mriedem | was looking for tempest-specs | 23:20 |
| mriedem | i guess you guys are being cool and not using codenames | 23:20 |
| flwang | mtreinish: sure, will do | 23:20 |
| mtreinish | mriedem: well it's grenade too | 23:20 |
| mriedem | makes sense | 23:21 |
| flwang | yep, I suppose it should be tempest-specs? | 23:21 |
| mriedem | no | 23:21 |
| mtreinish | although grenade lp doesn't have bps so it's a little weird | 23:21 |
| mriedem | flwang: there was a ML thread on the specs repos naming | 23:21 |
| flwang | so finally we pick the project name instead of codename, is it? | 23:21 |
| mtreinish | flwang: no the repo name is qa-specs: https://review.openstack.org/#/q/status:open+project:openstack/qa-specs,n,z | 23:21 |
| flwang | mtreinish: got it, thanks for the clarification | 23:21 |
| *** SridharG has quit IRC | 23:22 | |
| *** openstackrecheck has quit IRC | 23:22 | |
| *** openstackrecheck has joined #openstack-qa | 23:23 | |
| flwang | mtreinish: maybe a stupid question, is there anyway to disable all the other tests but scenario test? | 23:24 |
| mtreinish | flwang: just run with a regex filter = 'scenario' | 23:26 |
| mtreinish | so testr run --parallel scenario | 23:26 |
| mtreinish | that will only run the tests that match the filter scenario | 23:26 |
| mtreinish | or you can use a negative regex to exclude the scenario tests and run everything else | 23:27 |
| flwang | mtreinish: cool, thanks. btw, I'm going to implement some end to end functional test, do you think scenario is a good place for that? | 23:27 |
| mtreinish | there are some good examples in the tox.ini | 23:27 |
| *** mrodden has quit IRC | 23:28 | |
| *** SlickNik has quit IRC | 23:28 | |
| *** SlickNik has joined #openstack-qa | 23:29 | |
| *** SlickNik has quit IRC | 23:30 | |
| mtreinish | flwang: probably, maybe take a look at to see what's a good fit: http://docs.openstack.org/developer/tempest/field_guide/ | 23:32 |
| flwang | mtreinish: thanks, I assume it's | 23:33 |
| *** giulivo has quit IRC | 23:34 | |
| *** mrodden has joined #openstack-qa | 23:43 | |
| *** tosky has quit IRC | 23:44 | |
| *** jaypipes has quit IRC | 23:52 | |
| *** aimon__ has quit IRC | 23:52 | |
| *** mmaglana has quit IRC | 23:54 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/95879 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 23:55 |
| *** atiwari has quit IRC | 23:57 | |
| *** levi_b has joined #openstack-qa | 23:58 | |
| *** SlickNik has joined #openstack-qa | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!