| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-neutron 101717,3,4fd30a1 | 00:01 |
|---|---|---|
| *** yamahata has quit IRC | 00:13 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-neutron 98935,10,c826d16 | 00:15 |
| *** matsuhashi has joined #openstack-qa | 00:21 | |
| *** markmcclain1 has quit IRC | 00:22 | |
| *** nati_ueno has joined #openstack-qa | 00:23 | |
| *** ajo|mac_ has quit IRC | 00:27 | |
| *** dmorita has joined #openstack-qa | 00:31 | |
| *** markmcclain has joined #openstack-qa | 00:36 | |
| anteaya | dtroyer: I created a patch to draw the devstack docs into gerrit: https://review.openstack.org/#/c/102069/1 | 00:37 |
| anteaya | dtroyer: I marked it wip, do review | 00:37 |
| anteaya | my sense of it is I probably have the publishing jobs all wrong | 00:37 |
| anteaya | dtroyer: which of these would be the source repo we would import for docs? https://github.com/search?p=1&q=devstack&ref=cmdform&type=Repositories | 00:38 |
| openstackrecheck | Console logs not available after 13:27s for check-tempest-dsvm-neutron 97975,12,793c924 | 00:39 |
| *** markmcclain has quit IRC | 00:43 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/101900 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1333410, gate-tempest-dsvm-neutron: unrecognized error, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1333410 | 00:46 |
| *** yamahata has joined #openstack-qa | 00:58 | |
| openstackrecheck | Console logs not available after 13:27s for gate-horizon-python26 94706,28,e90c093 | 01:05 |
| *** ildikov has quit IRC | 01:06 | |
| *** bdperkin has quit IRC | 01:07 | |
| *** morganfainberg has quit IRC | 01:07 | |
| *** morganfainberg_Z is now known as morganfainberg | 01:07 | |
| *** jyuso has joined #openstack-qa | 01:11 | |
| *** bdperkin has joined #openstack-qa | 01:15 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 98924,2,ec65bbb | 01:18 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 97257,11,af9d649 | 01:32 |
| openstackgerrit | Zhi Kun Liu proposed a change to openstack/tempest: Enable some volumes v2 tests by sharing codes https://review.openstack.org/100193 | 01:36 |
| *** gfidente has quit IRC | 01:38 | |
| *** mriedem has joined #openstack-qa | 01:38 | |
| openstackgerrit | Zhi Kun Liu proposed a change to openstack/tempest: Sharing codes for volumes list tests https://review.openstack.org/100128 | 01:39 |
| *** Sukhdev has quit IRC | 01:41 | |
| *** nosnos has joined #openstack-qa | 01:44 | |
| openstackrecheck | Console logs not available after 13:26s for check-grenade-dsvm 101186,2,cc06a8e | 01:47 |
| *** salv-orlando has quit IRC | 01:49 | |
| *** GMann has joined #openstack-qa | 01:49 | |
| *** nati_ueno has quit IRC | 01:53 | |
| openstackrecheck | Console logs not available after 13:23s for gate-heat-python26 92406,6,ec8da42 | 02:00 |
| *** mmaglana has joined #openstack-qa | 02:10 | |
| openstackrecheck | Console logs not available after 13:22s for gate-nova-python26 84597,2,56f34e2 | 02:14 |
| openstackgerrit | Yuiko Takada proposed a change to openstack/tempest: Verify create agent attributes of V2/V3 APIs https://review.openstack.org/84344 | 02:27 |
| openstackrecheck | Console logs not available after 13:23s for gate-heat-python26 92421,4,d09932d | 02:28 |
| *** adalbas has quit IRC | 02:30 | |
| *** mestery has quit IRC | 02:31 | |
| *** mestery has joined #openstack-qa | 02:31 | |
| *** pcrews has quit IRC | 02:34 | |
| openstackrecheck | openstack/python-keystoneclient change: https://review.openstack.org/74599 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1333410, gate-tempest-dsvm-neutron: unrecognized error, gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1333410 | 02:37 |
| openstackrecheck | openstack/swift change: https://review.openstack.org/99124 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 02:40 |
| dtroyer | anteaya: I thought we were talking about handling this like the docs in the code repos built with sphinx. I moved the gh-pages branch into the master devstack repo in https://review.openstack.org/101668 and updated tools/build_docs.sh to do the actual build. | 02:41 |
| *** mmaglana has quit IRC | 02:45 | |
| annegentle | dtroyer: sounds full of win to me | 02:47 |
| *** mestery_ has joined #openstack-qa | 02:52 | |
| *** nosnos has quit IRC | 02:54 | |
| *** nosnos has joined #openstack-qa | 02:54 | |
| *** mestery has quit IRC | 02:55 | |
| *** nosnos has quit IRC | 02:59 | |
| jogo | sdague: sent out a hacking email to the ML | 02:59 |
| mrodden | failed every unit test in bash8, still passed pep8 checks | 03:01 |
| * mrodden fistpump | 03:01 | |
| *** mriedem has quit IRC | 03:02 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/102015 failed gate-requirements-integration-dsvm in the gate queue with an unrecognized error | 03:02 |
| openstackgerrit | Vishal kumar mahajan proposed a change to openstack/tempest: Add three new CLI tests of keystone https://review.openstack.org/100784 | 03:03 |
| *** nosnos has joined #openstack-qa | 03:05 | |
| *** dims__ has quit IRC | 03:13 | |
| openstackgerrit | Fei Long Wang proposed a change to openstack/tempest: Add scenario test for image service https://review.openstack.org/100140 | 03:13 |
| *** flwang has joined #openstack-qa | 03:15 | |
| flwang | zhikunliu: ping | 03:15 |
| zhikunliu | flwang: hi | 03:17 |
| flwang | zhikunliu: as for this comments https://review.openstack.org/#/c/100140/8//COMMIT_MSG I'm a little bit confused | 03:17 |
| flwang | zhikunliu: mtreinish said it's not necessary creating a spec, so does that mean the blue print is not necessary as well? | 03:18 |
| *** matsuhashi has quit IRC | 03:18 | |
| *** matsuhashi has joined #openstack-qa | 03:19 | |
| zhikunliu | flwang: yes. a single test case or small case don't need create blueprint. | 03:19 |
| flwang | zhikunliu: but it's a totally new scenario test for Glance | 03:20 |
| flwang | zhikunliu: i'm ok, TBH, just wanna know the right way | 03:21 |
| openstackrecheck | openstack/heat change: https://review.openstack.org/101928 failed gate-heat-python26, gate-heat-python27 in the gate queue with an unrecognized error | 03:22 |
| zhikunliu | flwang: It seems have a bp about scenario tests | 03:22 |
| *** matsuhashi has quit IRC | 03:24 | |
| flwang | zhikunliu: https://blueprints.launchpad.net/tempest?searchtext=scenario | 03:24 |
| flwang | as you see, the ceilometer blueprint is similar as this glance one | 03:26 |
| flwang | but it's creating spec and blueprint | 03:26 |
| flwang | so I'm really confused about the process of tempest | 03:26 |
| flwang | zhikunliu: so, I would say it is not a blocker to deserve a -1, how do you think? :) | 03:28 |
| zhikunliu | flwang: glance spec https://review.openstack.org/#/c/99857 looks like a spec for a single test case | 03:33 |
| *** nosnos has quit IRC | 03:34 | |
| flwang | zhikunliu: okay | 03:34 |
| zhikunliu | flwang: I think you need modify it to provide a simple image scenario tests plan like ceilometer | 03:35 |
| *** nosnos has joined #openstack-qa | 03:35 | |
| flwang | zhikunliu: I can just remove the blueprint from the commit message | 03:35 |
| flwang | zhikunliu: is it ok for you? | 03:35 |
| zhikunliu | flwang: do you have other images tests? | 03:36 |
| zhikunliu | flwang: for scenario | 03:36 |
| flwang | no, just this one | 03:36 |
| zhikunliu | flwang: if so, that's fine to remove blueprint | 03:36 |
| flwang | zhikunliu: cool | 03:37 |
| openstackgerrit | Fei Long Wang proposed a change to openstack/tempest: Add scenario test for image service https://review.openstack.org/100140 | 03:38 |
| *** nosnos has quit IRC | 03:39 | |
| *** dims__ has joined #openstack-qa | 03:39 | |
| flwang | zhikunliu: btw, it would be nice if you can take a look at this https://review.openstack.org/#/c/99514/ when you're most convenient, thanks | 03:40 |
| *** dims__ has quit IRC | 03:41 | |
| *** dpaterson has quit IRC | 03:53 | |
| *** Madan has joined #openstack-qa | 04:01 | |
| *** davidlenwell_ has joined #openstack-qa | 04:04 | |
| *** davidlenwell has quit IRC | 04:05 | |
| *** SlickNik has quit IRC | 04:05 | |
| *** Madan has joined #openstack-qa | 04:06 | |
| *** SlickNik has joined #openstack-qa | 04:06 | |
| *** matsuhashi has joined #openstack-qa | 04:10 | |
| *** Madan has quit IRC | 04:10 | |
| *** raies has joined #openstack-qa | 04:11 | |
| *** bkopilov has joined #openstack-qa | 04:12 | |
| openstackgerrit | ajay proposed a change to openstack/tempest: Add three new CLI tests of sahara https://review.openstack.org/100784 | 04:13 |
| *** jamielennox is now known as jamielennox|away | 04:13 | |
| *** SridharG has joined #openstack-qa | 04:15 | |
| openstackgerrit | Joe Mills proposed a change to openstack-dev/devstack: Set network_scheduler_driver for Midonet plugin https://review.openstack.org/102100 | 04:17 |
| *** nosnos has joined #openstack-qa | 04:19 | |
| *** raies has quit IRC | 04:22 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-full 102041,3,6d2527b | 04:22 |
| *** Madan has joined #openstack-qa | 04:23 | |
| *** raies has joined #openstack-qa | 04:26 | |
| *** yfried_ has quit IRC | 04:27 | |
| *** raies has quit IRC | 04:27 | |
| *** raies has joined #openstack-qa | 04:27 | |
| *** raies has quit IRC | 04:31 | |
| *** raies has joined #openstack-qa | 04:33 | |
| *** psharma has joined #openstack-qa | 04:39 | |
| *** morganfainberg_L has joined #openstack-qa | 04:41 | |
| *** dims has joined #openstack-qa | 04:42 | |
| *** dims has quit IRC | 04:47 | |
| *** pcrews has joined #openstack-qa | 04:49 | |
| openstackrecheck | openstack/python-glanceclient change: https://review.openstack.org/101903 failed because of: gate-tempest-dsvm-postgres-full: unrecognized error, gate-grenade-dsvm: unrecognized error, gate-tempest-dsvm-neutron: unrecognized error, gate-python-glanceclient-python27: https://bugs.launchpad.net/bugs/1282876 | 04:50 |
| *** mlavalle has quit IRC | 04:50 | |
| *** nati_ueno has joined #openstack-qa | 04:58 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/85937 failed because of: gate-nova-python27: https://bugs.launchpad.net/bugs/1282876, gate-nova-pep8: unrecognized error | 05:00 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/101849 failed because of: gate-nova-python27: unrecognized error, gate-nova-pep8: https://bugs.launchpad.net/bugs/1282876 | 05:05 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/95105 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error | 05:06 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/99373 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/921858, gate-nova-docs: unrecognized error, gate-tempest-dsvm-full: unrecognized error | 05:07 |
| *** ajc_ has joined #openstack-qa | 05:07 | |
| openstackgerrit | Ian Wienand proposed a change to openstack-dev/devstack: Handle failure of openstack endpoint https://review.openstack.org/102104 | 05:08 |
| openstackrecheck | openstack/python-keystoneclient change: https://review.openstack.org/99745 failed gate-python-keystoneclient-python26, gate-python-keystoneclient-python27, gate-tempest-dsvm-neutron-large-ops in the gate queue with an unrecognized error | 05:13 |
| openstackrecheck | openstack/cinder change: https://review.openstack.org/64230 failed gate-cinder-pep8, gate-cinder-python26, gate-tempest-dsvm-full in the gate queue with an unrecognized error | 05:15 |
| *** k4n0 has joined #openstack-qa | 05:21 | |
| openstackgerrit | Vishal kumar mahajan proposed a change to openstack/tempest: Add three new CLI tests of keystone https://review.openstack.org/102109 | 05:27 |
| *** Harika_ has joined #openstack-qa | 05:28 | |
| *** amitpp has joined #openstack-qa | 05:29 | |
| *** SridharG has quit IRC | 05:30 | |
| *** ildikov has joined #openstack-qa | 05:30 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-neutron 98356,5,8b9937d | 05:30 |
| *** Madan has quit IRC | 05:35 | |
| *** coolsvap|afk is now known as coolsvap | 05:35 | |
| *** SridharG has joined #openstack-qa | 05:35 | |
| *** rwsu has quit IRC | 05:41 | |
| *** dims has joined #openstack-qa | 05:43 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full 98715,13,7c12b88 | 05:44 |
| *** Madan has joined #openstack-qa | 05:44 | |
| *** dims has quit IRC | 05:47 | |
| *** morganfainberg_L has quit IRC | 05:51 | |
| *** bkopilov has quit IRC | 05:58 | |
| openstackgerrit | Zhi Kun Liu proposed a change to openstack/tempest: Sharing codes for volumes list tests https://review.openstack.org/100128 | 05:59 |
| *** bkopilov has joined #openstack-qa | 05:59 | |
| *** Longgeek has joined #openstack-qa | 06:01 | |
| *** Longgeek has quit IRC | 06:02 | |
| *** Longgeek has joined #openstack-qa | 06:03 | |
| *** danfinch has quit IRC | 06:08 | |
| *** evgenyf has joined #openstack-qa | 06:12 | |
| *** raies has quit IRC | 06:12 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 102103,1,95587c2 | 06:12 |
| openstackgerrit | A change was merged to openstack/tempest: Verify create agent attributes of V2/V3 APIs https://review.openstack.org/84344 | 06:15 |
| *** harlowja is now known as harlowja_away | 06:20 | |
| *** coolsvap is now known as coolsvap|afk | 06:24 | |
| *** k4n0 has quit IRC | 06:27 | |
| *** coolsvap|afk is now known as coolsvap | 06:28 | |
| openstackgerrit | Vishal kumar mahajan proposed a change to openstack/tempest: Add three new CLI tests of keystone https://review.openstack.org/102109 | 06:32 |
| openstackgerrit | ajay proposed a change to openstack/tempest: Add three new CLI tests of sahara https://review.openstack.org/100784 | 06:35 |
| openstackgerrit | Alok Kumar Maurya proposed a change to openstack/tempest: Allowed Address Pair API Tests Enhancements https://review.openstack.org/101522 | 06:39 |
| *** k4n0 has joined #openstack-qa | 06:39 | |
| *** dims has joined #openstack-qa | 06:43 | |
| *** gfidente has joined #openstack-qa | 06:44 | |
| *** dims has quit IRC | 06:48 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-neutron 82126,11,280b808 | 06:56 |
| *** psharma has quit IRC | 07:08 | |
| *** openstackgerrit has quit IRC | 07:10 | |
| *** Madan1 has joined #openstack-qa | 07:10 | |
| *** psharma has joined #openstack-qa | 07:11 | |
| *** Madan has quit IRC | 07:13 | |
| *** tkammer has joined #openstack-qa | 07:14 | |
| *** nati_ueno has quit IRC | 07:15 | |
| *** afazekas is now known as __afazekas | 07:17 | |
| *** catohornet has quit IRC | 07:20 | |
| *** salv-orlando has joined #openstack-qa | 07:26 | |
| *** SridharG has quit IRC | 07:29 | |
| *** SridharG has joined #openstack-qa | 07:29 | |
| *** ilyashakhat has quit IRC | 07:31 | |
| *** beekneemech has joined #openstack-qa | 07:32 | |
| *** bnemec has quit IRC | 07:32 | |
| *** eezhova has quit IRC | 07:32 | |
| *** asvechnikov has quit IRC | 07:32 | |
| *** ilyashakhat has joined #openstack-qa | 07:33 | |
| *** asvechnikov has joined #openstack-qa | 07:33 | |
| *** eezhova has joined #openstack-qa | 07:34 | |
| *** raies has joined #openstack-qa | 07:34 | |
| *** Manishanker has joined #openstack-qa | 07:38 | |
| *** andreaf has joined #openstack-qa | 07:39 | |
| *** qba73 has joined #openstack-qa | 07:40 | |
| openstackrecheck | Console logs not available after 13:24s for check-grenade-dsvm-partial-ncpu 102109,2,6db3932 | 07:42 |
| *** tkammer has quit IRC | 07:44 | |
| *** dims_ has joined #openstack-qa | 07:44 | |
| *** afazekas has joined #openstack-qa | 07:46 | |
| *** dims_ has quit IRC | 07:49 | |
| *** jpich has joined #openstack-qa | 07:55 | |
| *** gszasz has joined #openstack-qa | 07:59 | |
| *** jordanP has joined #openstack-qa | 08:01 | |
| *** tkammer has joined #openstack-qa | 08:02 | |
| *** gszasz has quit IRC | 08:04 | |
| *** safchain has joined #openstack-qa | 08:04 | |
| *** amitpp has quit IRC | 08:12 | |
| *** jordanP has quit IRC | 08:15 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-full 102103,2,425e5a5 | 08:17 |
| salv-orlando | afazekas: do you want to chat about neutron full job failures over IRC or maybe would it be better for you if I put all the data I have together in a mailing list post and you read them from there? | 08:17 |
| afazekas | salv-orlando: Both works for me | 08:22 |
| afazekas | I would like to know what is leading extra reason which causes higher failure rate on the neutron-full then we see on the neutron-smoke or on the novanet-full | 08:23 |
| *** tkammer has quit IRC | 08:23 | |
| salv-orlando | afazekas: first, there has been a spike in the past 24 hours in all neutorn jobs. We are looking at that separately, but for the sake of the full job I have looked in detail at about 50 failures between june 19th and june 20. | 08:27 |
| *** jordanP has joined #openstack-qa | 08:27 | |
| salv-orlando | For rough numbers I have rates for the past 24h, 48th and 7days | 08:28 |
| *** gszasz has joined #openstack-qa | 08:28 | |
| salv-orlando | If you look at rough failure numbers in the past 7 days you have 25.51%, 48hours 32.11% and 24 hours 44.68% (the spike) | 08:29 |
| salv-orlando | these numbers alone are horrible. But keep in mind this is the check queue, not the gate queue. To give a more reasonable estimate I calculate the ratio between check and gate failure rate for the smoke job and applied it to the full job, obtaining: | 08:30 |
| salv-orlando | what might be regarded as an “estimate” of gate failure ratio for the full job… 24 hours 20.78% 48 hours 16.98% 7 days 10.16% | 08:31 |
| salv-orlando | while these numbers are not terrifiyng they’re still too high. So let’s move to the failure that affect the full job, which is the result of the detailed analysis over 50 failures | 08:32 |
| salv-orlando | of these about 1/3 where due to bad patches, tempest tests still under development (and hence failing), infra issues or temporary failures such a timeout when downloading packages | 08:33 |
| *** jlibosva has joined #openstack-qa | 08:37 | |
| *** tkammer has joined #openstack-qa | 08:39 | |
| *** yrabl has joined #openstack-qa | 08:41 | |
| *** dims_ has joined #openstack-qa | 08:45 | |
| afazekas | so ~50 failure/week on tempest changes can be bad patches | 08:46 |
| afazekas | But the full neutron job fails 123 times / week | 08:47 |
| *** salv-orlando_ has joined #openstack-qa | 08:47 | |
| *** salv-orlando_ has quit IRC | 08:48 | |
| *** salv-orlando_ has joined #openstack-qa | 08:48 | |
| *** salv-orlando has quit IRC | 08:49 | |
| *** salv-orlando_ is now known as salv-orlando | 08:49 | |
| salv-orlando | afazekas: rather then annoying you with a lot of numbers, let say that there are three bugs causing the full job failure rate to be higher than the smoke’s | 08:49 |
| *** dims_ has quit IRC | 08:50 | |
| salv-orlando | one has to do with a race on servier resize causing the VM to fail to get to active state - and I’m triaging that one | 08:50 |
| salv-orlando | another is a strange failure in test_server_addresses | 08:50 |
| salv-orlando | and the last if an error we’re seeing in the load balancing scenario - which actually affects also the smoke job | 08:51 |
| afazekas | salv-orlando: the badStatusLine issue can be fixed, but it is about 10 failure per week | 08:52 |
| salv-orlando | yes I guess 10/week is also pretty mich the sme rate I saw on the full job | 08:53 |
| afazekas | https://bugs.launchpad.net/tempest/+bug/1326607 | 08:53 |
| salv-orlando | ah good you have a bug for that already | 08:53 |
| afazekas | is the 'servier resize' has higher failure rate with neutron than with n-net ? | 08:54 |
| openstackrecheck | Console logs not available after 13:35s for check-tempest-dsvm-postgres-full-icehouse 102128,1,127f9a7 | 08:55 |
| afazekas | Do we have bug and/or elastic recheck for the test_server_addresses and to the 'servier resize ' ? | 08:57 |
| *** cdent has joined #openstack-qa | 08:57 | |
| salv-orlando | afazekas: I do not see this kind of failures in gate queue so I’d say it does not affect nova-network | 08:59 |
| afazekas | Do you have a link to a failure with this type ? | 09:00 |
| *** yrabl has quit IRC | 09:05 | |
| *** dmorita has quit IRC | 09:07 | |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-full 98715,13,13b7a7d | 09:08 |
| *** evgenyf has quit IRC | 09:20 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-neutron 101826,2,2efef4b | 09:27 |
| afazekas | salv-orlando: which are the actually failing server_addresses and resize test cases ? | 09:37 |
| *** Manishanker has quit IRC | 09:37 | |
| *** enikanorov_ has quit IRC | 09:37 | |
| *** enikanorov_ has joined #openstack-qa | 09:38 | |
| salv-orlando | afazekas: test_server_addresses fails in setupclass -> http://logs.openstack.org/14/100014/8/check/check-tempest-dsvm-neutron-full/2bec88e/logs/testr_results.html.gz | 09:38 |
| salv-orlando | for the resize test case failure it’s in test_resize_server_confirm most of the times (it’s a smoke test strangely) | 09:39 |
| salv-orlando | and also once only in shelve_unshelve. | 09:39 |
| salv-orlando | but I think I have root caused that - out of curiosity, do checks in the gate queue use the same level of parallelism as the check queue? | 09:40 |
| salv-orlando | afazekas: the failure during resize is caused indirectly by neutron/nova notification mechanism: http://logs.openstack.org/84/100784/3/check/check-tempest-dsvm-neutron-full/70b8aac/logs/testr_results.html.gz | 09:41 |
| afazekas | salv-orlando: AFAIK that funny message is caused by the ERROR status | 09:41 |
| *** asvechnikov has quit IRC | 09:41 | |
| salv-orlando | afazekas: I would not focus on that 400 error | 09:41 |
| *** amitpp has joined #openstack-qa | 09:41 | |
| salv-orlando | perhaps that’s a minor bug in n-API or tempest | 09:41 |
| salv-orlando | what is going on is described in... | 09:42 |
| afazekas | https://review.openstack.org/#/c/101415/ | 09:42 |
| * salv-orlando scrambels to find bug report among hundreds of browsers windows open | 09:42 | |
| openstackrecheck | Console logs not available after 13:22s for gate-nova-python26 94296,18,df818fc | 09:42 |
| salv-orlando | ah cool so 101415 is for this 400 error we see, I guess? because the server errs with a message stating that body length is not an integer | 09:43 |
| *** evgenyf has joined #openstack-qa | 09:44 | |
| salv-orlando | so I have a reasonable suspect that this is the bug causing these failures in test_server_actions: https://bugs.launchpad.net/nova/+bug/1329546 | 09:44 |
| salv-orlando | afazekas: ^^ | 09:44 |
| *** Longgeek has quit IRC | 09:45 | |
| *** Longgeek has joined #openstack-qa | 09:45 | |
| salv-orlando | This was found against an external CI, but then I found the same pattern also on neutron full job failures | 09:45 |
| *** dims_ has joined #openstack-qa | 09:46 | |
| *** SridharG has quit IRC | 09:46 | |
| *** Longgeek has quit IRC | 09:46 | |
| *** Longgeek has joined #openstack-qa | 09:47 | |
| *** openstackgerrit has joined #openstack-qa | 09:47 | |
| afazekas | http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOiBcImZhaWxlZCB0byByZWFjaCBWRVJJRllfUkVTSVpFIHN0YXR1c1wiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiJhbGwiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDAzNjAzMTQ0MTU3fQ== It is really more frequent on neutron | 09:48 |
| *** psharma has quit IRC | 09:51 | |
| *** dims_ has quit IRC | 09:51 | |
| *** SridharG has joined #openstack-qa | 09:51 | |
| afazekas | https://bugs.launchpad.net/nova/+bug/1329546/comments/1 # you need to use # before the bug number | 09:51 |
| *** asvechnikov has joined #openstack-qa | 09:51 | |
| *** Longgeek has quit IRC | 09:52 | |
| salv-orlando | afazekas: Good to know | 09:52 |
| *** Longgeek has joined #openstack-qa | 09:53 | |
| *** tosky has joined #openstack-qa | 09:53 | |
| salv-orlando | afazekas: the fact that only when neutron is enabled nova waits for a notification even before spawning might explain higher neutron failure rate? | 09:53 |
| salv-orlando | I have however only an hypothesis on why the full job is affected the most - and that is that higher load on neutron causes a delay in VIF wiring which is then more likely to trigger bug 1329546 | 09:53 |
| salv-orlando | only 4 of 71 failures where non neutron -> 94.4% of occurences are with neutron | 09:54 |
| salv-orlando | And I’m quite sure that if look at non-neutron failures we’d find out those were specific patches’ problems | 09:55 |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-postgres-full 102145,1,f0190e8 | 09:56 |
| kashyap | Folks that run distros with, I see these kind of Stack traces when I restart OpenvSwitch -- https://bugzilla.kernel.org/show_bug.cgi?id=67751 | 09:56 |
| kashyap | I wonder if anyone else notices these. (I'm running Kernel -- 3.16.0-0.rc1.git1.1.fc21; OVS -- openvswitch-2.0.1-1.fc20.x86_64) | 09:57 |
| afazekas | kashyap: Is the kernel in debug mode? Does it cause a real issue ? | 10:00 |
| *** Madan1 has quit IRC | 10:01 | |
| kashyap | afazekas, Let me check if it's debug mode or not | 10:01 |
| afazekas | kashyap: #4 said it is just warning | 10:02 |
| kashyap | afazekas, Yeah, I remember talking to Tgraf about it. But now - why I see this is -- | 10:02 |
| kashyap | I rebooted my 2-node virtual setup w/ nested KVM, when I rebooted Control node | 10:02 |
| kashyap | Neutron/networking just fell apart, was restarting services and examining logs, and I stumbled on this | 10:03 |
| kashyap | afazekas, And no - the Kernel doesn't have debugging enabled | 10:04 |
| afazekas | kashyap: it is f21 rawhide ? | 10:05 |
| kashyap | afazekas, Yes | 10:05 |
| kashyap | F21 == Rawhide (F21 is not branched from Rawhide yet) | 10:05 |
| kashyap | afazekas, Meanwhile, have you seen this yet - https://review.openstack.org/#/c/102104/ | 10:05 |
| kashyap | The above was preventing from a successful DevStack setup (minimal - Nova, Neutron, Keystone, Glance). I gave my +1 after testing and noted my comment, please review and see if it looks good to you. | 10:06 |
| openstackrecheck | Console logs not available after 13:23s for gate-glance-python27 102057,1,20b588b | 10:09 |
| *** yrabl has joined #openstack-qa | 10:15 | |
| openstackgerrit | Bob Ball proposed a change to openstack-dev/devstack: Use curl to download from github.com https://review.openstack.org/102173 | 10:17 |
| openstackgerrit | Mh Raies proposed a change to openstack/tempest: Missing node driver interface validation API https://review.openstack.org/88522 | 10:17 |
| *** asselin has quit IRC | 10:17 | |
| *** nosnos has quit IRC | 10:19 | |
| afazekas | kashyap: did you see the issue only on rawhide ? | 10:19 |
| kashyap | afazekas, The OVS issue? | 10:19 |
| *** nosnos has joined #openstack-qa | 10:19 | |
| kashyap | afazekas, If so, I'm using F20, with Kernel, virtualization and OpenStack packages from Rawhide | 10:19 |
| afazekas | is it icehouse or juno ? | 10:20 |
| kashyap | IceHouse | 10:20 |
| kashyap | afazekas, The OVS issue seems like a corner case, since there's already a bug, I'll keep an eye. Don't want to consume your time on it | 10:21 |
| kashyap | Thank you. | 10:21 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full-icehouse 101381,2,95770a3 | 10:23 |
| afazekas | kashyap: BTW: do you know when the juno expected to be packaged ? | 10:23 |
| *** matsuhashi has quit IRC | 10:23 | |
| *** matsuhashi has joined #openstack-qa | 10:24 | |
| *** nosnos has quit IRC | 10:24 | |
| afazekas | salv-orlando: We will see after 1329546 solved | 10:24 |
| kashyap | afazekas, Well, at-least for Fedora/Rawhide they should be in once a milestone release is cut | 10:24 |
| *** amitpp has quit IRC | 10:24 | |
| salv-orlando | afazekas: yes, I hope to get some review cycles from nova core team | 10:24 |
| *** HenryG has quit IRC | 10:25 | |
| kashyap | afazekas, So, for Juno, M2 (24-JUL), packages should be in Rawhide in a week once the tarballs are ready on tarballs.openstack.org | 10:25 |
| *** matsuhashi has quit IRC | 10:28 | |
| openstackgerrit | A change was merged to openstack/tempest: Bump baremetal power/assoc/unprovision timeouts https://review.openstack.org/99226 | 10:32 |
| *** mkollaro has joined #openstack-qa | 10:33 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Fix availability zone client in compute admin test https://review.openstack.org/101798 | 10:35 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-neutron 101821,4,6007cc2 | 10:37 |
| openstackgerrit | Bob Ball proposed a change to openstack-dev/devstack: Use curl to download from github.com https://review.openstack.org/102173 | 10:37 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add bug for git.o.o going away https://review.openstack.org/102181 | 10:44 |
| *** dims_ has joined #openstack-qa | 10:46 | |
| *** jyuso has quit IRC | 10:46 | |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Skip quota-out tests temporary to change error codes https://review.openstack.org/102128 | 10:47 |
| *** salv-orlando has quit IRC | 10:48 | |
| kashyap | afazekas, Is this still valid -- https://review.openstack.org/#/c/98178/ ? | 10:48 |
| * kashyap bbiab, stepping out for a few | 10:48 | |
| afazekas | kashyap: yes | 10:50 |
| openstackrecheck | Console logs not available after 13:23s for gate-trove-python26 97510,11,2f8d414 | 10:50 |
| afazekas | kashyap: those things are packaged separately in another distros as well. | 10:52 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: add bug for git.o.o going away https://review.openstack.org/102181 | 10:55 |
| raies | afazekas: hi what will you say about this bug | 10:58 |
| raies | https://bugs.launchpad.net/neutron/+bug/1333103 | 10:58 |
| afazekas | kashyap: how frequently you can see that kernel message ? | 11:00 |
| afazekas | raies: Does not seams to be critical | 11:02 |
| raies | afazekas: I think message can be improved, anyways u better suggest | 11:03 |
| *** Longgeek has quit IRC | 11:03 | |
| openstackrecheck | Console logs not available after 13:22s for gate-heat-python26 101926,3,5d216c2 | 11:03 |
| *** Manishanker has joined #openstack-qa | 11:04 | |
| afazekas | kashyap: Can you reproduce it with a vanilla linus kernel ? | 11:05 |
| *** openstackrecheck has quit IRC | 11:08 | |
| *** openstackrecheck has joined #openstack-qa | 11:08 | |
| kashyap | afazekas, Hey, I'm just back, reading the scroll | 11:13 |
| *** SridharG has quit IRC | 11:15 | |
| kashyap | afazekas, I saw that OVS call trace after months. Let me login to that env if I can reproduce it again | 11:15 |
| kashyap | Can't reproduce it as of now. And, I don't know if it's worth testing w/ Linus' Kernel as I don't see it consistently | 11:18 |
| *** Longgeek has joined #openstack-qa | 11:18 | |
| *** yamahata has quit IRC | 11:18 | |
| *** vrovachev has joined #openstack-qa | 11:19 | |
| openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Fix "mutbale" object as default value https://review.openstack.org/100132 | 11:19 |
| *** Madan has joined #openstack-qa | 11:23 | |
| raies | jordanP: hi | 11:25 |
| raies | jordanP: you have given -1 at https://review.openstack.org/#/c/100142/ can you please specify reason | 11:26 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: added neutron flavor of the sqla bug https://review.openstack.org/102189 | 11:26 |
| openstackrecheck | Console logs not available after 13:31s for gate-cinder-python26 101794,2,9a3be4c | 11:27 |
| *** coolsvap is now known as coolsvap|afk | 11:27 | |
| openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: Adds "user-password-update" to v3/test_users https://review.openstack.org/92100 | 11:27 |
| *** k4n0 has quit IRC | 11:28 | |
| openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: test_tokens_nocatalog to identity v3 test_tokens https://review.openstack.org/93172 | 11:33 |
| *** Longgeek has quit IRC | 11:38 | |
| *** Longgeek has joined #openstack-qa | 11:38 | |
| *** Longgeek_ has joined #openstack-qa | 11:39 | |
| *** salv-orlando has joined #openstack-qa | 11:39 | |
| openstackrecheck | Console logs not available after 13:22s for gate-nova-pep8 61753,29,e12efca | 11:40 |
| *** Longgeek has quit IRC | 11:43 | |
| kashyap | afazekas, W/ current DevStack git on F20, why does Glance say 403 forbidden when attempting to upload a new image -- http://paste.openstack.org/show/84777/ | 11:43 |
| kashyap | (Note - I'm able to invoke other Glance commands just fine) | 11:44 |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: added neutron flavor of the sqla bug https://review.openstack.org/102189 | 11:45 |
| jordanP | raies, I've removed my -1, I don't want to argue about small nits anymore :) | 11:50 |
| *** openstackrecheck has quit IRC | 11:53 | |
| *** weshay has joined #openstack-qa | 11:57 | |
| afazekas | kashyap: are you admin ? | 11:59 |
| kashyap | afazekas, I used the openrc file from devstack, whatever it has | 11:59 |
| afazekas | kashyap: probbaly non admins cannot create public images | 12:00 |
| afazekas | "publicize_image": "role:admin", | 12:01 |
| afazekas | in the default galnce policy.json | 12:01 |
| kashyap | afazekas, Where are DevStack admin creds? I thought openrc should suffice | 12:01 |
| afazekas | source accrc/admin/admin | 12:01 |
| afazekas | kashyap: you can omit the '--is-public true ' | 12:02 |
| kashyap | afazekas, Ah, okay, let me try that | 12:02 |
| *** mkollaro has quit IRC | 12:03 | |
| afazekas | if you enable the h-api service you will get a f20 image uploaded | 12:03 |
| *** dkranz has joined #openstack-qa | 12:03 | |
| kashyap | What is h-api? Heat-api? | 12:03 |
| oomichi | SergeyLukjanov: could you take a look at https://review.openstack.org/#/c/102149/ ? | 12:04 |
| oomichi | SergeyLukjanov: that is necessary for testing nova v3 api. | 12:04 |
| kashyap | afazekas, Yep, thank you, it's the admin creds. Silly me | 12:05 |
| SergeyLukjanov | oomichi, hey, approved | 12:09 |
| oomichi | SergeyLukjanov: so fast, thank you so much :-) | 12:10 |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/tempest: Added cinder and neutron notifications tests for ceilometer https://review.openstack.org/67164 | 12:10 |
| *** k4n0 has joined #openstack-qa | 12:15 | |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Remove double-dash from the names https://review.openstack.org/100086 | 12:19 |
| openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: test_tokens_nocatalog to identity v3 test_tokens https://review.openstack.org/93172 | 12:28 |
| *** adalbas has joined #openstack-qa | 12:32 | |
| openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: test_tokens_nocatalog to identity v3 test_tokens https://review.openstack.org/93172 | 12:34 |
| *** jaypipes has quit IRC | 12:41 | |
| *** ekhugen has joined #openstack-qa | 12:41 | |
| *** GMann has quit IRC | 12:43 | |
| *** jaypipes has joined #openstack-qa | 12:43 | |
| *** ChanServ sets mode: +o jaypipes | 12:43 | |
| *** mwagner_lap has quit IRC | 12:45 | |
| *** ajc_ has quit IRC | 12:50 | |
| *** Madan1 has joined #openstack-qa | 12:51 | |
| *** Madan has quit IRC | 12:54 | |
| *** oomichi has quit IRC | 12:57 | |
| openstackgerrit | Dina Belova proposed a change to openstack-dev/devstack: Fix Swift config for Ceilometer usage https://review.openstack.org/102216 | 12:58 |
| openstackgerrit | Bartosz Górski proposed a change to openstack-dev/devstack: Adds support for multi-region https://review.openstack.org/79310 | 12:59 |
| openstackgerrit | Bob Ball proposed a change to openstack-dev/devstack: WIP: Do not merge https://review.openstack.org/102217 | 12:59 |
| *** Matt1 has joined #openstack-qa | 13:00 | |
| *** mkollaro has joined #openstack-qa | 13:02 | |
| afazekas | kashyap: FYI: http://patchwork.openvswitch.org/patch/3358/ | 13:05 |
| kashyap | afazekas, Looking | 13:06 |
| SergeyLukjanov | mtreinish, ping | 13:08 |
| *** bkopilov has quit IRC | 13:08 | |
| kashyap | afazekas, Thanks, it was from 3 months ago. Hopefully it's in the version of OVS I'm using from Rawhide | 13:09 |
| SergeyLukjanov | mtreinish, I've approved https://review.openstack.org/#/c/102149/ today and it conflicts with https://review.openstack.org/#/c/101915/1 | 13:09 |
| SergeyLukjanov | mtreinish, both of them solves the issue, do you prefer to revert https://review.openstack.org/#/c/102149/ ? | 13:10 |
| mtreinish | SergeyLukjanov: yeah lets revert the config change, because it won't work with just that | 13:10 |
| mtreinish | because the variables are mismatched between devstack-gate-wrap and devastack-gate | 13:10 |
| SergeyLukjanov | mtreinish, yup, I see | 13:12 |
| SergeyLukjanov | mtreinish, reverted | 13:12 |
| *** andreaf has quit IRC | 13:13 | |
| *** dims_ has quit IRC | 13:13 | |
| *** dustins has joined #openstack-qa | 13:13 | |
| *** dims_ has joined #openstack-qa | 13:14 | |
| *** SridharG has joined #openstack-qa | 13:14 | |
| *** Longgeek has joined #openstack-qa | 13:21 | |
| *** Longgeek_ has quit IRC | 13:22 | |
| *** Harika_ has quit IRC | 13:25 | |
| *** k4n0 has quit IRC | 13:26 | |
| *** mriedem has joined #openstack-qa | 13:29 | |
| anteaya | dtroyer: yes, my patch was mostly a placeholder for discussion, I'm glad it is redundant and the docs are already in gerrit, thanks | 13:33 |
| *** SridharG has quit IRC | 13:33 | |
| anteaya | dtroyer: yes, I wasn't sure how to handle the doc build and wanted a jumping off point for the discussion so we made actual decisions since we are all in agreement and so far hand-wavey on implementation | 13:33 |
| *** Madan1 has quit IRC | 13:33 | |
| *** Manishanker has quit IRC | 13:35 | |
| openstackgerrit | Bob Ball proposed a change to openstack-dev/devstack: XenAPI: Tail output from stack.sh https://review.openstack.org/102217 | 13:38 |
| *** Obulpathi has joined #openstack-qa | 13:42 | |
| *** mestery_ is now known as mestery | 13:42 | |
| *** Obulpathi has quit IRC | 13:44 | |
| dkranz | sdague, mtreinish : I think Dave addressed Sean's last issue with https://review.openstack.org/#/c/91725/ (qa spec for detect leak/cleanup) | 13:44 |
| *** Obulpathi has joined #openstack-qa | 13:44 | |
| *** SridharG has joined #openstack-qa | 13:46 | |
| *** jasondotstar has joined #openstack-qa | 13:47 | |
| *** mkollaro1 has joined #openstack-qa | 13:59 | |
| *** mkollaro has quit IRC | 13:59 | |
| *** tkammer has quit IRC | 14:02 | |
| *** rbak has joined #openstack-qa | 14:02 | |
| *** HenryG has joined #openstack-qa | 14:04 | |
| *** beekneemech is now known as bnemec | 14:04 | |
| *** salv-orlando has quit IRC | 14:05 | |
| *** malini1 has joined #openstack-qa | 14:06 | |
| mtreinish | dkranz: reviewed, it's almost there. I'll probably be +A on the next rev | 14:08 |
| dkranz | mtreinish: k. Looking at your comments, I'm not sure we are clear about the level of detail these qa-specs should have about specifics of implementation. | 14:10 |
| mtreinish | dkranz: yeah I would have been fine with it at that level of detail. It was more the inconsistency between using a JSON file vs config | 14:11 |
| dkranz | mtreinish: yes, good point | 14:12 |
| *** ajo|mac has joined #openstack-qa | 14:12 | |
| *** mwagner_lap has joined #openstack-qa | 14:12 | |
| *** Longgeek has quit IRC | 14:18 | |
| *** tkammer has joined #openstack-qa | 14:18 | |
| *** Longgeek has joined #openstack-qa | 14:19 | |
| *** jpich has quit IRC | 14:23 | |
| openstackgerrit | Stanislaw Pitucha proposed a change to openstack-dev/devstack: Always check keystone using http https://review.openstack.org/99433 | 14:23 |
| *** mlavalle has joined #openstack-qa | 14:26 | |
| *** mlavalle has quit IRC | 14:27 | |
| *** SridharG has quit IRC | 14:27 | |
| *** yrabl has quit IRC | 14:28 | |
| *** dkranz has quit IRC | 14:31 | |
| *** mlavalle has joined #openstack-qa | 14:32 | |
| *** sabeen has joined #openstack-qa | 14:33 | |
| *** jpich has joined #openstack-qa | 14:35 | |
| *** andreaf has joined #openstack-qa | 14:37 | |
| *** SridharG has joined #openstack-qa | 14:37 | |
| sdague | chmouel, you want to weigh in on - https://review.openstack.org/#/c/101870/ ? | 14:41 |
| *** kgriffs|afk is now known as kgriffs_ | 14:43 | |
| *** markmcclain has joined #openstack-qa | 14:45 | |
| *** mkollaro1 has quit IRC | 14:46 | |
| *** dkranz has joined #openstack-qa | 14:46 | |
| *** mkollaro has joined #openstack-qa | 14:47 | |
| *** markmcclain1 has joined #openstack-qa | 14:47 | |
| chmouel | let me have a look | 14:47 |
| sdague | chmouel: thanks | 14:48 |
| sdague | I'm hoping that will help reduce the gate errors related to apt pulls | 14:49 |
| *** markmcclain has quit IRC | 14:50 | |
| *** tkammer has quit IRC | 14:50 | |
| *** mlavalle has quit IRC | 14:52 | |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/tempest: Added cinder and neutron notifications tests for ceilometer https://review.openstack.org/67164 | 14:52 |
| *** catohornet has joined #openstack-qa | 14:54 | |
| chmouel | sdague: that make sense | 14:54 |
| *** kgriffs_ is now known as kgriffs | 14:57 | |
| *** morganfainberg_L has joined #openstack-qa | 14:58 | |
| sdague | chmouel: thanks much | 14:59 |
| chmouel | i wonder if that bug ever got fixed tho https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=119544 | 15:00 |
| *** nati_ueno has joined #openstack-qa | 15:00 | |
| *** marun has joined #openstack-qa | 15:01 | |
| *** yrabl has joined #openstack-qa | 15:02 | |
| *** ajo|mac has quit IRC | 15:02 | |
| sdague | chmouel: good question | 15:03 |
| afazekas | sdague: https://review.openstack.org/#/c/101415/ | 15:03 |
| *** SridharG has quit IRC | 15:04 | |
| *** tkammer has joined #openstack-qa | 15:06 | |
| sdague | afazekas: isn't there a nova call that gets the error state information out of the db? | 15:07 |
| afazekas | sdague: I am not aware of it, the normal `nova show` responses contains the failure reason | 15:08 |
| *** ewindisch has quit IRC | 15:11 | |
| sdague | afazekas: can we get the nova show output back into the logs instead? | 15:11 |
| jordanP | sdague, could you have a look at : https://review.openstack.org/#/c/97107/ You -1ed the thing but the author provided a reply | 15:11 |
| jordanP | plz | 15:11 |
| *** dkranz has quit IRC | 15:12 | |
| *** ewindisch has joined #openstack-qa | 15:12 | |
| sdague | jordanP: but why are all the 167 | 15:12 |
| sdague | 167 iniset $CEILOMETER_CONF keystone_authtoken auth_host $KEYSTONE_AUTH_HOST iniset $CEILOMETER_CONF keystone_authtoken auth_host $KEYSTONE_AUTH_HOST168 | 15:12 |
| sdague | 168 iniset $CEILOMETER_CONF keystone_authtoken auth_port $KEYSTONE_AUTH_PORT iniset $CEILOMETER_CONF keystone_authtoken auth_port $KEYSTONE_AUTH_PORT169 | 15:12 |
| sdague | 169 iniset $CEILOMETER_CONF keystone_authtoken auth_protocol $KEYSTONE_AUTH_PROTOCOL iniset $CEILOMETER_CONF keystone_authtoken auth_protocol $KEYSTONE_AUTH_PROTOCOL | 15:12 |
| sdague | still there | 15:12 |
| sdague | most of the other cleanup patches pulled those parts | 15:12 |
| afazekas | sdague: How ? As part of the exception ? | 15:13 |
| sdague | afazekas: sure | 15:13 |
| sdague | or just as a log message | 15:13 |
| *** yfried_ has joined #openstack-qa | 15:14 | |
| jordanP | sdague, those lines are taken care of by Jamie's patch. The patch #97107 adds a new line, this new line is not intented for keystonemiddleware | 15:15 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-dev/devstack: Update Apache + mod_wsgi deployment mechanisms https://review.openstack.org/101611 | 15:16 |
| jordanP | sdague, in ceilometer, for some reason, the credentials are in two distrincts section service_credentials and keystone_authtoken | 15:16 |
| sdague | jordanP: ok, that seems really error prone | 15:17 |
| sdague | and makes no sense | 15:17 |
| jordanP | maybe it makes sense but we haven't find it yet | 15:17 |
| sdague | I'm happy if one is deprecating the other, but I think that having both is something that we should consider a high priority fix | 15:17 |
| sdague | instead of just working around it in devstack | 15:17 |
| jordanP | it's how ceilometer works... | 15:17 |
| sdague | well, it's pretty bad. I'd really like to figure out what the long term ceilometer plan is there before we just land this. | 15:19 |
| sdague | because specifying keystone url twice, in 2 different ways, is really bad | 15:20 |
| *** mriedem has quit IRC | 15:21 | |
| morganfainberg_L | wait ceilometer needs keystone url specified twice? | 15:21 |
| *** Manishanker has joined #openstack-qa | 15:22 | |
| *** psharma has joined #openstack-qa | 15:22 | |
| sdague | morganfainberg_L: yeh, that's what I'm questioning | 15:22 |
| sdague | https://review.openstack.org/#/c/97107/ | 15:22 |
| morganfainberg_L | sdague: that seems awful | 15:22 |
| morganfainberg_L | sdague: thanks for questioning it! | 15:22 |
| morganfainberg_L | jordanP: why does ceilometer need this? | 15:23 |
| morganfainberg_L | jordanP: if you know | 15:23 |
| jordanP | morganfainberg_L, I'll check | 15:24 |
| *** dkranz has joined #openstack-qa | 15:24 | |
| morganfainberg_L | jordanP: i think i have an idea, but i'd rather not guess on it. | 15:24 |
| morganfainberg_L | jordanP: thanks :) | 15:24 |
| *** tkammer has quit IRC | 15:25 | |
| jordanP | sileht, ^^ :p | 15:26 |
| sdague | afazekas / dkranz: https://review.openstack.org/#/c/102000/ this hopefully makes that test far more predictable | 15:27 |
| jordanP | I am not sure why they didn't use the same parameter everywhere, but by having a quick look a the code, the proposed patch seems required. service_credentials.os_auth_url is accessed several times | 15:29 |
| *** tosky has quit IRC | 15:30 | |
| dkranz | sdague: ok | 15:31 |
| *** coolsvap|afk is now known as coolsvap | 15:31 | |
| sileht | sdague, jordanP you have the creds used to poll API and the creds used to validate API token | 15:32 |
| jordanP | thanks sileht ! | 15:32 |
| afazekas | sdague: would't be better to wait and try it multiple times ? | 15:32 |
| sdague | afazekas: the issue is we're actually failing with a 404 on the list | 15:33 |
| *** marun has quit IRC | 15:33 | |
| sileht | jordanP, sdague not sure the patch is needed, the default is the env var 'OS_AUTH_URL' | 15:34 |
| jordanP | ah.... | 15:34 |
| sileht | or http://localhost:5000/v2.0 is OS_AUTH_URL does not exists | 15:34 |
| sdague | sileht: so I'm still not sure I understand why the auth url is specified twice though | 15:35 |
| sileht | jordanP, but it can be usefull in case you run a devstack multi hosts with ceilometer not on the keystone node | 15:35 |
| afazekas | sdague: ok | 15:35 |
| sdague | I can sort of understand there being 2 different creds, though that seems odd | 15:35 |
| jordanP | sileht, yep that what the commits message try to say i believe | 15:35 |
| jordanP | Longgeek's usecase | 15:35 |
| sdague | sileht: that's my real issue. Specifying auth url twice, in 2 different ways, is really problematic. Is there a high priority bug in ceilometer to join that together to one? | 15:36 |
| sileht | sdague, I try to remember the use case behind that | 15:36 |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-dev/devstack: Update Apache + mod_wsgi deployment mechanisms https://review.openstack.org/101611 | 15:36 |
| *** yrabl has quit IRC | 15:36 | |
| *** marun has joined #openstack-qa | 15:37 | |
| dkranz | mtreinish: Are we flat-out rejecting any test that requires two compute nodes? | 15:38 |
| *** cdent_ has joined #openstack-qa | 15:38 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-dev/devstack: Enable Keystone under Apache by default https://review.openstack.org/100747 | 15:40 |
| *** cdent has quit IRC | 15:40 | |
| *** cdent_ is now known as cdent | 15:40 | |
| *** SridharG has joined #openstack-qa | 15:40 | |
| *** mriedem has joined #openstack-qa | 15:40 | |
| *** cdent has quit IRC | 15:42 | |
| *** cdent has joined #openstack-qa | 15:43 | |
| *** HenryG has quit IRC | 15:43 | |
| mtreinish | dkranz: yeah, we've been rejecting anything that can't run in the gate or some other ci | 15:44 |
| mtreinish | dkranz: do you have an link? | 15:44 |
| dkranz | mtreinish: https://review.openstack.org/#/c/77816 | 15:44 |
| dkranz | mtreinish: I am -1ing it for other reasons but wanted to clarify our stance | 15:44 |
| dkranz | mtreinish: This sort of test is valuable. | 15:45 |
| sileht | sdague, service_credentials since the begining of ceilometer, keystone_authtoken have been added when authentification have been added to ceilometer api, so we can at least open a bug to see if other devs have a real use case behind this separation | 15:45 |
| dkranz | mtreinish: I agree with our policy here. | 15:45 |
| dkranz | mtreinish: I guess we are waiting for triple-o in the gate to solve this? | 15:46 |
| mtreinish | dkranz: yeah having a multinode deployments for testing opens up all sorts of good stuff. | 15:46 |
| mtreinish | well that, or infra starts doing multinode | 15:47 |
| *** qba73 has quit IRC | 15:47 | |
| jordanP | morganfainberg_L, regarding the patch https://review.openstack.org/#/c/101611/ . Will it change where and how the log are outputed ? I mean if we change the default, it'd be great if the logs remained accessible in let's say this folder http://logs.openstack.org/11/101611/2/check/check-tempest-dsvm-neutron-full/6a911e8/logs/ | 15:47 |
| mtreinish | dkranz: or maybe we can convince someone to setup a third party multinode system | 15:48 |
| *** asselin has joined #openstack-qa | 15:48 | |
| mtreinish | it's really whatever happens first | 15:48 |
| *** vrovachev has quit IRC | 15:48 | |
| openstackgerrit | Pranav Salunke proposed a change to openstack/tempest: Fixes Scoping bugs https://review.openstack.org/95358 | 15:49 |
| dkranz | mtreinish: I think it will be triple-o. I think from the perspective of both RH and HP, who are providing hardware for triple-o, that *is* the third party ci, except that it won't be third-party when really turned on. | 15:50 |
| *** atiwari has joined #openstack-qa | 15:50 | |
| dkranz | mtreinish: The unfortunate part for us is that it is "blocked" on install issues :) | 15:50 |
| sdague | dkranz: honestly, tripleo is at least a year away from that | 15:51 |
| sdague | tripleo doesn't run tempest today in their ci | 15:51 |
| sdague | so there is going to be a long path to get that working | 15:51 |
| dkranz | sdague: Yes, but I imagine the focus on triple-o is part of the reason HP does not have third-party CI of the sort we are talking about, no? | 15:52 |
| sdague | dkranz: don't know | 15:52 |
| sdague | there are the seeds to be able to do this in the existing infra system, it will just take someone a few weeks to pull it all together | 15:53 |
| dkranz | sdague: that was also said two summits ago... | 15:53 |
| dkranz | sdague: I'd love to see it happen | 15:53 |
| sdague | yeh, someone needs to do the last bits. Nodepool can allocate 2 nodes now for a test, and include ssh keys that let you get between them | 15:54 |
| sdague | anyway, lunch time | 15:54 |
| dkranz | sdague: yup | 15:54 |
| openstackgerrit | Pranav Salunke proposed a change to openstack/tempest: Fixes Scoping bugs https://review.openstack.org/95358 | 15:54 |
| mriedem | mtreinish: just push these through yourself :) https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:update-oslo-config,n,z | 15:57 |
| *** markmcclain1 has quit IRC | 15:57 | |
| *** yamahata has joined #openstack-qa | 15:59 | |
| openstackgerrit | A change was merged to openstack-infra/devstack-gate: Fix typo in DEVSTACK_GATE_TEMPEST_NOVA_V3_API https://review.openstack.org/101915 | 15:59 |
| openstackgerrit | Pranav Salunke proposed a change to openstack/tempest: Fixes Scoping bugs https://review.openstack.org/95358 | 15:59 |
| *** evgenyf has quit IRC | 16:00 | |
| openstackgerrit | Kyle Mestery proposed a change to openstack-dev/devstack: Don't install openvswitch-datapath-dkms on newer kernels https://review.openstack.org/102284 | 16:09 |
| *** aimon_ has quit IRC | 16:10 | |
| jogo | what happend with grenade? http://jogo.github.io/gate/ | 16:15 |
| openstackgerrit | afazekas proposed a change to openstack/tempest: Replace the console output on Error with fault a message https://review.openstack.org/101415 | 16:17 |
| *** nati_ueno has quit IRC | 16:17 | |
| afazekas | http://www.fpaste.org/112666/40362665/ XML/JSON and not available case when the q-agt was killed | 16:18 |
| *** andreaf has quit IRC | 16:18 | |
| *** andreaf has joined #openstack-qa | 16:19 | |
| *** jordanP has quit IRC | 16:22 | |
| *** afazekas has quit IRC | 16:24 | |
| *** salv-orlando has joined #openstack-qa | 16:26 | |
| openstackgerrit | Pranav Salunke proposed a change to openstack/tempest: Fixes Scoping bugs https://review.openstack.org/95358 | 16:26 |
| jogo | mtreinish: ping re: tempest CLI test patches | 16:27 |
| jogo | there are two seperate people working on them and there seems to be now over all plan. they are a pain to review | 16:30 |
| jogo | and not sure if a spec is needed or not | 16:30 |
| *** salv-orlando_ has joined #openstack-qa | 16:36 | |
| *** yfried_ has quit IRC | 16:36 | |
| *** yfried_ has joined #openstack-qa | 16:37 | |
| *** salv-orlando has quit IRC | 16:37 | |
| *** salv-orlando_ is now known as salv-orlando | 16:37 | |
| *** __afazekas is now known as afazekas | 16:41 | |
| *** aimon_ has joined #openstack-qa | 16:50 | |
| *** ajo|mac has joined #openstack-qa | 16:54 | |
| *** harlowja_away is now known as harlowja | 16:55 | |
| *** cdent_ has joined #openstack-qa | 16:58 | |
| *** cdent has quit IRC | 17:00 | |
| *** cdent_ is now known as cdent | 17:00 | |
| *** k4n0 has joined #openstack-qa | 17:01 | |
| *** sabeen has quit IRC | 17:03 | |
| *** ajo|mac has quit IRC | 17:04 | |
| *** mkollaro has quit IRC | 17:11 | |
| *** rockyg has joined #openstack-qa | 17:12 | |
| *** jpich has quit IRC | 17:14 | |
| *** mkollaro has joined #openstack-qa | 17:16 | |
| *** mmaglana has joined #openstack-qa | 17:22 | |
| *** cdent has quit IRC | 17:24 | |
| *** rwsu has joined #openstack-qa | 17:25 | |
| mtreinish | mriedem: unlike sdague I don't mind waiting... | 17:29 |
| mriedem | zing | 17:30 |
| mtreinish | well it's a not so subtle hint to get him to review it :) | 17:30 |
| mriedem | just make me core for a day | 17:30 |
| sdague | jogo: it's all apt issues | 17:33 |
| sdague | from what I can see. | 17:33 |
| *** ildikov has quit IRC | 17:34 | |
| jogo | ahh | 17:34 |
| jogo | so more network | 17:34 |
| sdague | unfortunately ES is really backed up | 17:34 |
| mtreinish | jogo: hmm, we normally only require specs for larger testing efforts. So having a test for improving the cli test coverage would be fine | 17:34 |
| sdague | so we aren't showing them in ER yet | 17:34 |
| jogo | yeah, are logs getting too big again? | 17:34 |
| mtreinish | but that doesn't mean the piecemeal efforts would collaborate because of it | 17:34 |
| jogo | mtreinish: true, I cannot tell if this a larger effort or to piecemeal efforts | 17:35 |
| *** openstackgerrit has quit IRC | 17:35 | |
| sdague | jogo: no, rax network / node issues | 17:36 |
| jogo | sdague: oh ER is being hurt by rax network as well | 17:36 |
| jogo | wow neutron really sucks ;) | 17:36 |
| *** openstackgerrit has joined #openstack-qa | 17:36 | |
| *** Manishanker has quit IRC | 17:39 | |
| *** Manishanker has joined #openstack-qa | 17:40 | |
| openstackgerrit | ajay proposed a change to openstack/tempest: Improve nova CLI existing tests https://review.openstack.org/101767 | 17:40 |
| *** andreaf has quit IRC | 17:40 | |
| jogo | mtreinish: ^ is an example | 17:40 |
| jogo | https://review.openstack.org/#/q/owner:ajayyadavmdu%2540gmail.com+status:open,n,z | 17:41 |
| jogo | and https://review.openstack.org/#/q/owner:mh.raies%2540nectechnologies.in+status:open,n,z | 17:41 |
| openstackgerrit | ajay proposed a change to openstack/tempest: Improve nova CLI existing tests https://review.openstack.org/101767 | 17:44 |
| mtreinish | yeah they're stepping on each others toes, they should be collaborating. I don't know if either of them are on irc | 17:44 |
| *** kgriffs is now known as kgriffs|afk | 17:45 | |
| *** sabeen has joined #openstack-qa | 17:45 | |
| mtreinish | jogo: I'll reach out to them and figure something out | 17:45 |
| jogo | mtreinish: thanks, I sorta gave up o it | 17:45 |
| jogo | I tried to say work with the other person etc | 17:46 |
| jogo | looks like tempest land is fun | 17:46 |
| mtreinish | heh, it looks like one of them is using a bp but never opened a spec... | 17:48 |
| mtreinish | raies: did you open: https://blueprints.launchpad.net/tempest/+spec/missing-nova-cli-tests ? | 17:49 |
| jogo | yeah and the other isn't | 17:51 |
| *** k4n0 has quit IRC | 17:53 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: retry apt operations to avoid network issues https://review.openstack.org/101870 | 17:53 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Use curl to download from github.com https://review.openstack.org/102173 | 17:54 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Handle failure of openstack endpoint https://review.openstack.org/102104 | 17:54 |
| *** rgrober has joined #openstack-qa | 17:54 | |
| *** rockyg has quit IRC | 17:57 | |
| openstackgerrit | ajay proposed a change to openstack/tempest: Improve nova CLI existing tests https://review.openstack.org/101767 | 17:58 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Add an environment variable to enable extensions in keystone https://review.openstack.org/95778 | 18:00 |
| openstackgerrit | A change was merged to openstack-dev/devstack: XenAPI: Tail output from stack.sh https://review.openstack.org/102217 | 18:00 |
| openstackgerrit | A change was merged to openstack/tempest: provide enough time to do expires in obj tests https://review.openstack.org/102000 | 18:00 |
| openstackgerrit | A change was merged to openstack-dev/devstack: Bump EPEL release rpm for RHEL 7 to 7.0-2 https://review.openstack.org/101875 | 18:00 |
| *** rgrober has quit IRC | 18:01 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 18:01 | |
| *** kgriffs|afk has quit IRC | 18:02 | |
| *** atiwari has quit IRC | 18:03 | |
| *** harlowja has quit IRC | 18:04 | |
| *** harlowja has joined #openstack-qa | 18:07 | |
| *** psharma has quit IRC | 18:11 | |
| *** mkollaro has quit IRC | 18:11 | |
| *** dims_ has quit IRC | 18:13 | |
| *** jlibosva has quit IRC | 18:14 | |
| *** ajo|mac has joined #openstack-qa | 18:14 | |
| *** dims_ has joined #openstack-qa | 18:14 | |
| *** psharma has joined #openstack-qa | 18:18 | |
| *** atiwari has joined #openstack-qa | 18:23 | |
| sdague | dtroyer: so how you feeling about the retry loop for landing? - https://review.openstack.org/#/c/101004/ | 18:24 |
| *** yfried_ has quit IRC | 18:24 | |
| *** harlowja has joined #openstack-qa | 18:26 | |
| *** harlowja has quit IRC | 18:26 | |
| *** harlowja has joined #openstack-qa | 18:26 | |
| *** harlowja has joined #openstack-qa | 18:27 | |
| *** harlowja has joined #openstack-qa | 18:28 | |
| *** harlowja has joined #openstack-qa | 18:29 | |
| *** harlowja has joined #openstack-qa | 18:29 | |
| *** ajo|mac has quit IRC | 18:33 | |
| clarkb | btw I think we have merged our first couple commits using trusty for tempest | 18:33 |
| *** psharma has quit IRC | 18:33 | |
| clarkb | so \o/ | 18:33 |
| clarkb | I am really impressed you guys had the whole stack working before we even spun up test nodes | 18:33 |
| *** devananda has joined #openstack-qa | 18:33 | |
| *** harlowja has joined #openstack-qa | 18:34 | |
| devananda | raies: hi! we are moving all the ironic API tests to an /admin subdir with https://review.openstack.org/#/c/100989/2 | 18:34 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/qa-specs: Add a section for submitting specs for adding tests https://review.openstack.org/102314 | 18:35 |
| mtreinish | sdague, dkranz: ^^^ do you have any opinions on that? | 18:35 |
| devananda | raies: I dont think it'll conflict -- none of your patches that I see are addign new files -- but wanted you to be aware of this change | 18:36 |
| mtreinish | clarkb: awesome | 18:36 |
| sdague | clarkb: :) | 18:36 |
| *** harlowja has joined #openstack-qa | 18:39 | |
| *** harlowja has quit IRC | 18:39 | |
| *** yfried has joined #openstack-qa | 18:39 | |
| *** harlowja has joined #openstack-qa | 18:40 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/qa-specs: Add a section for submitting specs for adding tests https://review.openstack.org/102314 | 18:41 |
| *** harlowja has joined #openstack-qa | 18:42 | |
| adam_g | what are peoples thoughts on making admin-level API calls from non-admin API tests as part of setup/setupClass? | 18:42 |
| sdague | adam_g: examples? | 18:42 |
| clarkb | https://jenkins04.openstack.org/job/gate-tempest-dsvm-postgres-full/8335/ is our first failure on trusty in the gate | 18:42 |
| clarkb | sdague: mtreinish ^ probably worth double checking that it isn't due to trusty | 18:43 |
| sdague | clarkb: I've seen that before | 18:43 |
| clarkb | ok I won't worry then | 18:43 |
| sdague | it's definitely not trusty unique | 18:43 |
| clarkb | :) just letting you all know as I see them | 18:43 |
| clarkb | also so excited | 18:43 |
| sdague | yep, good call | 18:43 |
| sdague | yeh, this is goodness :) | 18:44 |
| adam_g | sdague, https://review.openstack.org/#/c/94439/11/tempest/scenario/manager.py i'm working on the API tests running against ironic clouds with finite resources. i need to reimplement that somewhere on the API side anytime an instance is spawned. | 18:44 |
| sdague | fortunately it's going so smoothly, as I'm mostly out of pocket the rest of the day with farm pickup, tc, and then a new tech meetup that I'm speaking at | 18:44 |
| sdague | adam_g: sorry, I don't have the full context. But need to run away at the moment, maybe we can sync tomorrow on it. | 18:47 |
| mtreinish | adam_g: we don't split off tests with admin from the rest of scenario (at least not yet) so it should be fine but why does it have to be in setUp? scenario tests should have like nothing in setup | 18:47 |
| adam_g | sdague, yeah, hit me up. i may put up something ugly in the maentime | 18:47 |
| adam_g | mtreinish, this isn't the scenario tests, compute API | 18:48 |
| mtreinish | adam_g: the link was from the scenario manager... | 18:48 |
| *** HenryG has joined #openstack-qa | 18:48 | |
| adam_g | mtreinish, sorry, that was an example of what i need to have happen in the API tests | 18:48 |
| *** ajo|mac has joined #openstack-qa | 18:49 | |
| sdague | adam_g: I think we need to not do that in API. If there is an issue on "don't allocated > n computes ever" that's probably something else we need to explore | 18:49 |
| sdague | because that seems like what you actually are talking about | 18:49 |
| adam_g | sdague, its not just setting an upper bound. its "dont run instance until n-scheduler has run its periodic task to reclaim resources from the instance that was deleted in previous test" :P) | 18:50 |
| sdague | yeh, so that's really getting pretty grey box. | 18:51 |
| sdague | and is going to fail in really odd ways | 18:51 |
| adam_g | the same would be true of a libvirt setup with only enough memory for one or two instances | 18:51 |
| sdague | yeh, this seems like a much bigger issue to ponder. Because just hacking that in there is going to have some odd side effects I think. | 18:52 |
| adam_g | sdague, what side effects are you thinking? it'd essentially be checking the same thing that the scheduler would be checking during spawn. | 18:55 |
| mtreinish | adam_g: I'm honestly not sure that we should care about that in tempest. It seems like a weird edge case to try and protect against. | 18:55 |
| mtreinish | if you don't have enough resources to keep up with tempest then failing seems appropiate to me | 18:55 |
| *** krtaylor has quit IRC | 18:56 | |
| mtreinish | you can limit things with running serially which shouldn't make too many resources at the same time | 18:56 |
| openstackgerrit | A change was merged to openstack/tempest: HEAD doesn't return a body https://review.openstack.org/101900 | 18:56 |
| *** krtaylor has joined #openstack-qa | 19:01 | |
| *** morganfainberg_L has quit IRC | 19:01 | |
| *** morganfainberg_L has joined #openstack-qa | 19:01 | |
| *** morganfainberg_L has quit IRC | 19:02 | |
| adam_g | mtreinish, we already do run serially. this isn't so much an issue of scaling (that is a separate thing i need to look at), its giving the cloud under test enough time to catch up with the tests. | 19:02 |
| *** morganfainberg_L has joined #openstack-qa | 19:02 | |
| *** yamahata__ has joined #openstack-qa | 19:02 | |
| *** yamahata_ has quit IRC | 19:03 | |
| mtreinish | adam_g: yeah I don't think we should protect against that in tempest. It's the same basic thing, as a deployer you would just set rate limiting to prevent too many requests. | 19:04 |
| mtreinish | Think of tempest as any other api consumer. As a user of an api (even an admin) IMO it's pretty unrealistic to assume that they'll check the state of available resources before making api calls. | 19:06 |
| mtreinish | like what your proposing, something like checking quota left maybe not | 19:06 |
| dtroyer | sdague: I'm good with it, was hoping someone else might want to chime in… | 19:07 |
| openstackgerrit | Kyle Mestery proposed a change to openstack-dev/devstack: Don't install openvswitch-datapath-dkms on newer kernels https://review.openstack.org/102284 | 19:07 |
| *** Manishanker has quit IRC | 19:10 | |
| *** ildikov has joined #openstack-qa | 19:11 | |
| adam_g | mtreinish, right, i generally agree for the real-world deployer use cases, but i'd argue things aren't so black and white in the context of tempest + devstack slaves... at least if the goal is to get baremetal testing running there as if it were standard libvirt | 19:14 |
| *** dims_ has quit IRC | 19:15 | |
| *** dims_ has joined #openstack-qa | 19:16 | |
| *** cdent has joined #openstack-qa | 19:18 | |
| dkranz | mtreinish: +2 from me | 19:21 |
| mtreinish | adam_g: I get that, but at least from soley a tempest review perspective we have to think about both gating and "real-world" at the same time because tempest has to work in both cases. Which I agree are often contradictory things, like here. | 19:21 |
| mtreinish | dkranz: cool, thanks | 19:21 |
| mtreinish | adam_g: I just think in this case we should find another way to get this working in the gate instead of introducing an admin check to the nonadmin api tests | 19:21 |
| *** Sukhdev has joined #openstack-qa | 19:22 | |
| mtreinish | because it raises a bigger question about how would things be handled with limited resources, like what we're discussing, outside of the gate | 19:23 |
| mtreinish | or is just not a concern everywhere else | 19:23 |
| adam_g | yeah | 19:23 |
| adam_g | i think barmetal/ironic deployers will have resource limits much closer to the floor, especially if testing non-idle clouds | 19:24 |
| *** ajo|mac has quit IRC | 19:24 | |
| *** gszasz has quit IRC | 19:26 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-dev/devstack: Add keystone middleware to devstack https://review.openstack.org/102326 | 19:27 |
| *** yfried has quit IRC | 19:30 | |
| *** yfried has joined #openstack-qa | 19:30 | |
| *** harlowja has quit IRC | 19:31 | |
| *** jasondotstar has quit IRC | 19:33 | |
| *** rockyg has joined #openstack-qa | 19:33 | |
| *** mmaglana has quit IRC | 19:36 | |
| *** harlowja has joined #openstack-qa | 19:37 | |
| *** harlowja_ has joined #openstack-qa | 19:39 | |
| *** yfried has quit IRC | 19:40 | |
| *** yfried has joined #openstack-qa | 19:41 | |
| *** harlowja has quit IRC | 19:42 | |
| *** malini1 has quit IRC | 19:46 | |
| *** hockeynu_ has joined #openstack-qa | 19:50 | |
| openstackgerrit | David Shrewsbury proposed a change to openstack/tempest: Add ironic instance rebuild advanced test https://review.openstack.org/94439 | 19:51 |
| *** salv-orlando has quit IRC | 19:51 | |
| *** salv-orlando has joined #openstack-qa | 19:53 | |
| *** cdent_ has joined #openstack-qa | 19:53 | |
| *** cdent has quit IRC | 19:54 | |
| *** cdent_ is now known as cdent | 19:54 | |
| *** hockeynu_ has quit IRC | 19:55 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-dev/devstack: Add keystone middleware to devstack https://review.openstack.org/102326 | 19:58 |
| sdague | dtroyer: yeh, we seem to be light on the review front of late | 19:59 |
| *** Sukhdev has quit IRC | 20:01 | |
| openstackgerrit | David Kranz proposed a change to openstack/tempest: Add 'id' to the server group cleanup list, not body https://review.openstack.org/102333 | 20:06 |
| dkranz | mtreinish: I found (and fixed) a bug that was dumping stacktraces into the tempest log. I was thinking we should fail on ERRORs in tempest log just like others but not sure it is such a great idea. Any opinion? | 20:17 |
| mtreinish | dkranz: I'm in favor of doing that, but one issue might be skip exceptions sometime leave a stack trace | 20:20 |
| mtreinish | but it doesn't seem like they always do | 20:20 |
| dkranz | mtreinish: There might be some cleanup involved before it could be turned on but it seems the right thing to do. | 20:21 |
| dkranz | mtreinish: I'll try to get some time to do that. | 20:21 |
| *** mwagner_lap has quit IRC | 20:23 | |
| *** davidlenwell_ is now known as davidlenwell | 20:28 | |
| *** ajo|mac has joined #openstack-qa | 20:32 | |
| *** dkranz has quit IRC | 20:38 | |
| *** adalbas has quit IRC | 20:39 | |
| openstackgerrit | Morgan Fainberg proposed a change to openstack-infra/devstack-gate: Add keystonemiddleware to devstack-gate https://review.openstack.org/102340 | 20:41 |
| mriedem | mtreinish: think i figured out my check_uptodate.sh issue, | 20:45 |
| mriedem | tempest doesn't require stevedore | 20:45 |
| mriedem | which is a runtime requirement for the generator.py | 20:45 |
| mtreinish | mriedem: does it work without setting the modulepath but with stevedore? Or is this a separate issue? | 20:46 |
| mriedem | mtreinish: https://github.com/openstack/tempest/blob/master/tempest/openstack/common/config/generator.py#L32 | 20:46 |
| mtreinish | either way add it to test-requirements | 20:46 |
| mriedem | mtreinish: looks like it's used for -l for libs is used | 20:46 |
| mriedem | but yeah i'll push up a patch | 20:47 |
| mriedem | mtreinish: heat has the same issue | 20:47 |
| mriedem | mrodden: ^ | 20:47 |
| mtreinish | yeah so that's actually a secondary issue because we don't run it with '-l' | 20:47 |
| *** nati_ueno has joined #openstack-qa | 20:48 | |
| *** mmaglana has joined #openstack-qa | 20:49 | |
| mriedem | mtreinish: well it blows up on the import | 20:50 |
| mriedem | https://gist.github.com/mriedem/777f2784a390a2a4621b | 20:50 |
| mtreinish | yeah I guess that's true | 20:50 |
| mriedem | mtreinish: was going to use requirements.txt since that's what other projects like nova/cinder/oslo use, but if it's only used in pep8 checks i guess test-requirements.txt makes more sense? | 20:52 |
| mtreinish | yeah test-requirements is the right place in tempest. The other projects are actually using stevedore | 20:53 |
| mtreinish | here it's just for the pep8 check | 20:53 |
| openstackgerrit | Matt Riedemann proposed a change to openstack/tempest: Require stevedore for check_uptodate.sh https://review.openstack.org/102346 | 20:57 |
| *** nati_ueno has quit IRC | 21:02 | |
| *** nati_ueno has joined #openstack-qa | 21:02 | |
| *** HenryG has quit IRC | 21:05 | |
| *** malini1 has joined #openstack-qa | 21:06 | |
| openstackgerrit | Kyle Mestery proposed a change to openstack-dev/devstack: Don't install openvswitch-datapath-dkms on newer kernels https://review.openstack.org/102284 | 21:08 |
| flwang | mtreinish: ping | 21:10 |
| *** mlavalle has joined #openstack-qa | 21:10 | |
| flwang | sdague: ping | 21:10 |
| *** mlavalle has quit IRC | 21:12 | |
| flwang | I think there is an interesting topic to discuss | 21:13 |
| *** dims__ has joined #openstack-qa | 21:13 | |
| mtreinish | flwang: hey | 21:14 |
| flwang | mtreinish: hey there, except the test policy I sent the mail list, there is another interesting topic I'd like to discuss with you guys, would you mind taking a look at this first https://review.openstack.org/#/c/101805/ ? See the Jordon's comments at last | 21:15 |
| *** mlavalle has joined #openstack-qa | 21:16 | |
| openstackgerrit | Andrey Kurilin proposed a change to openstack/qa-specs: Automate tempest config generation https://review.openstack.org/94473 | 21:16 |
| *** dims_ has quit IRC | 21:16 | |
| flwang | mtreinish: I'm adding an 'attach volume' case for the cinder scenario test since we're leveraging tempest scenario to do end-to-end test after upgrade, patching, etc | 21:17 |
| flwang | mtreinish: in the case, i'm trying to make sure the volume is working or not instead of just checking the volume status from openstack view, but Jordon said it's not openstack's responsibility | 21:18 |
| openstackgerrit | Andrey Kurilin proposed a change to openstack/qa-specs: Automate tempest config generation https://review.openstack.org/94473 | 21:18 |
| mtreinish | flwang: I think that case is tested in other places in the scenario though. (test_stamp_pattern I think) | 21:18 |
| flwang | mtreinish: so I'd like to know what's your opinion | 21:18 |
| mtreinish | it doesn't really fit in the scope of that test case | 21:18 |
| mtreinish | flwang: functional verification is definitely part of what we do tempest | 21:19 |
| mtreinish | I have a draft reply to your ML thread that I need to finish up | 21:19 |
| flwang | mtreinish: awesome, it would be nice if you can help clarify the scope of scenario test | 21:20 |
| flwang | flwang: I think scenario test of tempest is great to be leveraged for end-to-end functional test by the consumers | 21:21 |
| flwang | I will review the test_stamp_pattern | 21:22 |
| mrodden | sdague: did you have any issues here https://review.openstack.org/#/c/101508/3 i rebased it so wanted to give you a chance to look over it quick | 21:22 |
| flwang | mtreinish: seems the test_stamp_pattern has covered the case I added, fantastic, I will abandon mine, thanks for pointing out that one | 21:23 |
| *** aimon_ has quit IRC | 21:24 | |
| mtreinish | mrodden: oh, thanks for the reminder I want to add that to tempest. | 21:24 |
| flwang | I didn' realized the scope from the file name ;) | 21:24 |
| *** Obulpathi has quit IRC | 21:25 | |
| *** malini1 has quit IRC | 21:26 | |
| mtreinish | flwang: sure np, I think we mention something about the functional testing in the scenario test field guide | 21:27 |
| *** ajo|mac has quit IRC | 21:27 | |
| mtreinish | but if you think we need an update to the docs feel free to push a patch | 21:27 |
| flwang | mtreinish: got it, thanks a lot | 21:27 |
| *** ajo|mac has joined #openstack-qa | 21:28 | |
| flwang | mtreinish: it would be nice if you can take a look at this https://review.openstack.org/#/c/99514/ when you're most convenient, thanks | 21:28 |
| *** Obulpathi has joined #openstack-qa | 21:29 | |
| *** Sukhdev has joined #openstack-qa | 21:29 | |
| mrodden | mtreinish: at py34 to tox? | 21:29 |
| mrodden | err add* | 21:29 |
| mtreinish | mrodden: yeah, I've been running py34 on arch for a while now so I've been meaning to update the tox for tempest | 21:30 |
| mrodden | arch | 21:31 |
| mrodden | nice | 21:31 |
| mtreinish | although I think just switching 33 to 34 makes more sense for the tempest tox.ini | 21:31 |
| *** ekhugen has quit IRC | 21:31 | |
| *** Obulpathi has quit IRC | 21:31 | |
| mrodden | right | 21:31 |
| mrodden | latest 3.x anyway | 21:31 |
| openstackgerrit | Chris Dent proposed a change to openstack/tempest: Add a check_telemetry method to javelin https://review.openstack.org/102354 | 21:32 |
| clarkb | sdague: mtreinish do nova unittests seem flakier than normal? | 21:32 |
| clarkb | there have been a couple gate resets due to them today | 21:33 |
| clarkb | eg https://jenkins05.openstack.org/job/gate-nova-python26/3097/console | 21:33 |
| mtreinish | clarkb: I haven't been watching it too closely today | 21:33 |
| clarkb | which is centos6 so not trusty related | 21:33 |
| mtreinish | jogo: ^^^ | 21:33 |
| mtreinish | clarkb: so far no trusty issues right? That's pretty awesome | 21:35 |
| clarkb | mtreinish: other than breaking rally, pecan, and ipa testing nothing | 21:35 |
| clarkb | and I have mostly fixed those issues I think | 21:35 |
| mtreinish | although I've been running trusty on my dev snapshots since it was released and I don't think I've hit any issues | 21:35 |
| clarkb | ya that is what sdague said too | 21:35 |
| mtreinish | clarkb: yeah but who cares about those projects :) | 21:35 |
| *** harlowja_ has quit IRC | 21:36 | |
| *** harlowja has joined #openstack-qa | 21:36 | |
| *** SridharG has quit IRC | 21:38 | |
| *** enikanorov_ has quit IRC | 21:38 | |
| *** enikanorov_ has joined #openstack-qa | 21:38 | |
| *** cdent has quit IRC | 21:39 | |
| *** dkorn has quit IRC | 21:40 | |
| *** dkorn has joined #openstack-qa | 21:40 | |
| *** aimon has joined #openstack-qa | 21:44 | |
| *** mriedem has quit IRC | 21:49 | |
| *** rockyg has quit IRC | 21:53 | |
| *** nati_ueno has quit IRC | 21:53 | |
| *** nati_ueno has joined #openstack-qa | 21:54 | |
| *** dustins has quit IRC | 21:57 | |
| openstackgerrit | Kyle Mestery proposed a change to openstack-dev/devstack: Don't install openvswitch-datapath-dkms on newer kernels https://review.openstack.org/102284 | 21:59 |
| openstackgerrit | Fei Long Wang proposed a change to openstack/tempest: Add cases for Swift scenario test https://review.openstack.org/99514 | 22:00 |
| *** aimon has quit IRC | 22:03 | |
| *** jaypipes has quit IRC | 22:11 | |
| clarkb | https://jenkins01.openstack.org/job/gate-tempest-dsvm-postgres-full/11477/console that one of the known failures? | 22:26 |
| *** HenryG has joined #openstack-qa | 22:27 | |
| jogo | mtreinish: thanks for looking into the CLI test stuff | 22:28 |
| jogo | mtreinish: it looks like we hit a rare but long running race condition in the unit tests today | 22:29 |
| jogo | clarkb: ^ | 22:29 |
| jogo | clarkb: http://jogo.github.io/gate/ although thats a lof of nova CLI fail | 22:29 |
| jogo | clarkb: Bug 1311778 | 22:30 |
| jogo | that looks like the only bug in the nova unit tests in the gate right now. at least in the logstash data which is old I think | 22:32 |
| *** Longgeek has quit IRC | 22:32 | |
| *** IngoH has quit IRC | 22:35 | |
| *** yamahata has quit IRC | 22:35 | |
| *** ajo|mac has quit IRC | 22:37 | |
| openstackgerrit | Steve Baker proposed a change to openstack/tempest: Repeat stack delete until NotFound https://review.openstack.org/99766 | 22:39 |
| *** boris-42 has quit IRC | 22:46 | |
| *** yamahata has joined #openstack-qa | 22:47 | |
| *** rbak has quit IRC | 23:01 | |
| *** boris-42 has joined #openstack-qa | 23:02 | |
| *** atiwari has quit IRC | 23:03 | |
| *** yamahata has quit IRC | 23:08 | |
| *** hockeynu_ has joined #openstack-qa | 23:18 | |
| *** mwagner_lap has joined #openstack-qa | 23:25 | |
| *** hockeynu_ has quit IRC | 23:26 | |
| *** gfidente has quit IRC | 23:28 | |
| *** Sukhdev has quit IRC | 23:29 | |
| *** GMann has joined #openstack-qa | 23:39 | |
| *** jaypipes has joined #openstack-qa | 23:43 | |
| *** ChanServ sets mode: +o jaypipes | 23:43 | |
| *** oomichi has joined #openstack-qa | 23:45 | |
| *** jamielennox|away is now known as jamielennox | 23:47 | |
| *** yamahata has joined #openstack-qa | 23:57 | |
| *** harlowja_ has joined #openstack-qa | 23:58 | |
| *** GMann has quit IRC | 23:58 | |
| *** safchain has quit IRC | 23:58 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!