| *** Obulpathi has quit IRC | 00:00 | |
| *** mmaglana has quit IRC | 00:05 | |
| openstackrecheck | Console logs not available after 13:24s for gate-nova-python27 102645,4,4df6b70 | 00:07 |
|---|---|---|
| *** atiwari has quit IRC | 00:07 | |
| *** ajo|mac_ has quit IRC | 00:13 | |
| openstackrecheck | Console logs not available after 13:24s for gate-nova-pep8 93860,1,ab6242d | 00:20 |
| *** dims__ has quit IRC | 00:25 | |
| *** dmorita has joined #openstack-qa | 00:26 | |
| *** zhikunliu1 has joined #openstack-qa | 00:35 | |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm-partial-ncpu 103950,1,c917146 | 00:39 |
| *** dims__ has joined #openstack-qa | 00:43 | |
| openstackrecheck | Console logs not available after 13:34s for check-tempest-dsvm-postgres-full 94294,20,d1fd02f | 00:53 |
| *** TravT|2 has quit IRC | 00:57 | |
| *** xianghui has joined #openstack-qa | 00:58 | |
| *** nati_ueno has quit IRC | 00:59 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Remove unnecessary log message https://review.openstack.org/104045 | 01:00 |
| *** nati_ueno has joined #openstack-qa | 01:01 | |
| openstackrecheck | Console logs not available after 13:26s for check-tempest-dsvm-postgres-full 103924,1,d41e467 | 01:07 |
| *** nati_ueno has quit IRC | 01:09 | |
| *** nati_ueno has joined #openstack-qa | 01:10 | |
| *** mitz_ has joined #openstack-qa | 01:11 | |
| *** zhikunliu1 has quit IRC | 01:15 | |
| *** Sukhdev has quit IRC | 01:20 | |
| openstackrecheck | Console logs not available after 13:21s for check-grenade-dsvm 94295,23,e0c1d17 | 01:20 |
| *** Sukhdev has joined #openstack-qa | 01:22 | |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 103951,1,002ef54 | 01:33 |
| openstackrecheck | Console logs not available after 13:21s for check-grenade-dsvm 87080,3,e0cd875 | 01:47 |
| openstackgerrit | Dean Troyer proposed a change to openstack-dev/devstack: Generate recent changes list from git log https://review.openstack.org/104051 | 01:48 |
| *** nosnos has joined #openstack-qa | 01:49 | |
| *** mriedem has quit IRC | 01:53 | |
| *** nosnos has quit IRC | 01:55 | |
| *** nosnos has joined #openstack-qa | 01:55 | |
| *** rodrigods_ has joined #openstack-qa | 01:56 | |
| *** dkranz has quit IRC | 01:59 | |
| openstackrecheck | Console logs not available after 13:21s for gate-ceilometer-pep8 103956,1,db9fa07 | 02:00 |
| *** nati_ueno has quit IRC | 02:07 | |
| *** nati_ueno has joined #openstack-qa | 02:10 | |
| openstackrecheck | Console logs not available after 13:21s for check-grenade-dsvm 99805,2,a17aa9f | 02:13 |
| *** nati_ueno has quit IRC | 02:21 | |
| *** nati_ueno has joined #openstack-qa | 02:21 | |
| *** zhiyan_ is now known as zhiyan | 02:22 | |
| *** nati_ueno has quit IRC | 02:22 | |
| *** xianghui has quit IRC | 02:24 | |
| openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm-partial-ncpu 103238,1,0f5384c | 02:27 |
| *** rodrigods_ has quit IRC | 02:27 | |
| *** arnaud has quit IRC | 02:29 | |
| openstackgerrit | FeihuJiang proposed a change to openstack/tempest: Adds an orchestration API test for resource types https://review.openstack.org/103775 | 02:30 |
| *** zhiyan is now known as zhiyan_ | 02:30 | |
| openstackrecheck | Console logs not available after 13:21s for gate-glance-pep8 103952,1,69166b8 | 02:40 |
| *** zhiyan_ is now known as zhiyan | 02:42 | |
| *** Sukhdev has quit IRC | 02:46 | |
| *** harlowja is now known as harlowja_away | 02:50 | |
| *** ajc_ has joined #openstack-qa | 02:50 | |
| openstackrecheck | Console logs not available after 13:21s for check-grenade-dsvm 103234,1,09d894f | 02:54 |
| *** mlavalle has joined #openstack-qa | 02:58 | |
| *** nati_ueno has joined #openstack-qa | 03:02 | |
| *** xianghui has joined #openstack-qa | 03:05 | |
| openstackrecheck | Console logs not available after 13:21s for gate-devstack-bashate 103180,6,a256e85 | 03:07 |
| *** dims__ has quit IRC | 03:09 | |
| *** mwagner_ has quit IRC | 03:15 | |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm 99225,3,6bcb4e6 | 03:20 |
| *** zhiyan is now known as zhiyan_ | 03:22 | |
| *** zhiyan_ is now known as zhiyan | 03:24 | |
| *** mwagner_ has joined #openstack-qa | 03:27 | |
| *** nosnos has quit IRC | 03:27 | |
| *** zhiyan has quit IRC | 03:30 | |
| openstackrecheck | Console logs not available after 13:21s for gate-nova-python26 88750,21,28b1efb | 03:34 |
| *** zhiyan has joined #openstack-qa | 03:37 | |
| *** gfidente has quit IRC | 03:39 | |
| *** zhiyan is now known as zhiyan_ | 03:39 | |
| *** zhiyan_ is now known as zhiyan | 03:40 | |
| *** SridharG has joined #openstack-qa | 03:45 | |
| *** TravT has joined #openstack-qa | 03:45 | |
| *** zhiyan is now known as zhiyan_ | 03:46 | |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 103962,1,fdf4f06 | 03:47 |
| *** zhiyan_ is now known as zhiyan | 03:50 | |
| *** arnaud has joined #openstack-qa | 03:50 | |
| *** coolsvap|afk is now known as coolsvap | 03:53 | |
| *** Poornima has joined #openstack-qa | 03:59 | |
| openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 92412,6,286650c | 04:00 |
| *** pcrews has quit IRC | 04:02 | |
| *** TravT has quit IRC | 04:03 | |
| *** yfried_ has quit IRC | 04:10 | |
| *** Sukhdev has joined #openstack-qa | 04:12 | |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm 87551,5,49a29c0 | 04:14 |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 95678,6,7bc45b1 | 04:27 |
| *** dims__ has joined #openstack-qa | 04:37 | |
| openstackrecheck | Console logs not available after 13:22s for gate-glance-python27 103328,2,c849d26 | 04:40 |
| *** nosnos has joined #openstack-qa | 04:42 | |
| *** k4n0 has joined #openstack-qa | 04:42 | |
| *** dims__ has quit IRC | 04:42 | |
| *** arnaud has quit IRC | 04:47 | |
| *** arnaud has joined #openstack-qa | 04:49 | |
| *** yfried_ has joined #openstack-qa | 04:53 | |
| *** psharma has joined #openstack-qa | 04:53 | |
| openstackrecheck | Console logs not available after 13:22s for gate-glance-python26 103959,2,fc60ea9 | 04:54 |
| *** dims__ has joined #openstack-qa | 05:02 | |
| *** dims__ has quit IRC | 05:07 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 103332,2,1e2e23d | 05:07 |
| *** Madan has joined #openstack-qa | 05:10 | |
| *** marun is now known as marun_afk | 05:16 | |
| openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds list users tests in v3 https://review.openstack.org/93704 | 05:17 |
| *** amitpp has joined #openstack-qa | 05:21 | |
| openstackrecheck | Console logs not available after 13:25s for gate-ceilometer-pep8 103967,2,b63b0b4 | 05:21 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Fixes ephemeral disk flavor test for non-virt https://review.openstack.org/104069 | 05:21 |
| *** rdekel has joined #openstack-qa | 05:21 | |
| *** afazekas_ has joined #openstack-qa | 05:21 | |
| *** rwsu has quit IRC | 05:24 | |
| *** gema_ is now known as gema | 05:33 | |
| openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-postgres-full 103234,1,b55e1ba | 05:34 |
| *** amitpp has quit IRC | 05:35 | |
| *** chandan_kumar has joined #openstack-qa | 05:38 | |
| *** amitpp has joined #openstack-qa | 05:40 | |
| *** bkopilov_ has quit IRC | 05:44 | |
| *** bkopilov has joined #openstack-qa | 05:44 | |
| openstackrecheck | Console logs not available after 13:26s for check-grenade-dsvm 99296,2,5cfc0dc | 05:48 |
| openstackgerrit | A change was merged to openstack/tempest: Fix "create a flavor" tests for strong validation https://review.openstack.org/103763 | 05:57 |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full-icehouse 100143,3,3e2e91f | 06:01 |
| *** dims__ has joined #openstack-qa | 06:03 | |
| *** nati_ueno has quit IRC | 06:03 | |
| *** dims__ has quit IRC | 06:08 | |
| *** evgenyf has joined #openstack-qa | 06:10 | |
| *** Sukhdev has quit IRC | 06:11 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-postgres-full 103952,2,a0a202d | 06:15 |
| *** nati_ueno has joined #openstack-qa | 06:16 | |
| *** praveenkumar has joined #openstack-qa | 06:16 | |
| openstackgerrit | A change was merged to openstack/qa-specs: Add a section for submitting specs for adding tests https://review.openstack.org/102314 | 06:26 |
| openstackrecheck | Console logs not available after 13:26s for gate-tempest-dsvm-full 100058,2,bc1a7be | 06:28 |
| openstackrecheck | Console logs not available after 13:36s for check-grenade-dsvm 92412,6,2a77416 | 06:42 |
| *** mitz_ has quit IRC | 06:48 | |
| *** mitz_ has joined #openstack-qa | 06:50 | |
| *** nati_ueno has quit IRC | 06:51 | |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm 101605,2,f815c06 | 06:55 |
| *** jlanoux has joined #openstack-qa | 06:59 | |
| openstackgerrit | A change was merged to openstack/tempest: Add a hacking check to block scenario manager in api tests https://review.openstack.org/96164 | 07:00 |
| *** pcrews has joined #openstack-qa | 07:01 | |
| *** joy_ has joined #openstack-qa | 07:04 | |
| joy_ | hi enikanorov | 07:05 |
| joy_ | I am installing openstack 2 node using standard openstack guid | 07:05 |
| joy_ | when I reach nova setting on controller node I found /etc/nova/nova.conf and others are not present | 07:06 |
| joy_ | how to deal with it | 07:06 |
| joy_ | anyone arround | 07:07 |
| openstackgerrit | Sridhar Gaddam proposed a change to openstack/qa-specs: Spec: Add Provider Networking Scenario tests https://review.openstack.org/104098 | 07:07 |
| openstackrecheck | Console logs not available after 13:23s for gate-keystone-pep8 103983,1,7103881 | 07:08 |
| joy_ | afazekas_: hi | 07:11 |
| *** luqas has joined #openstack-qa | 07:11 | |
| *** jordanP has joined #openstack-qa | 07:14 | |
| *** pcrews has quit IRC | 07:17 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 103967,3,3fc1ac9 | 07:22 |
| *** ylobankov has quit IRC | 07:23 | |
| *** jlibosva has joined #openstack-qa | 07:24 | |
| *** ajo|mac has joined #openstack-qa | 07:24 | |
| afazekas_ | joy_: hi | 07:24 |
| *** afazekas_ is now known as afazekas | 07:25 | |
| joy_ | afazekas: I am installing openstack 2 node one is compute and other controller | 07:25 |
| joy_ | afazekas: I refered doc this - http://docs.openstack.org/havana/install-guide/install/apt/content/nova-controller.html | 07:25 |
| afazekas | installer ? | 07:25 |
| openstackgerrit | Sridhar Gaddam proposed a change to openstack/qa-specs: Spec: Add Provider Networking Scenario tests https://review.openstack.org/104098 | 07:26 |
| joy_ | afazekas: I removed /etc/nova/ directory and tried then on controller | 07:26 |
| joy_ | how to get back nova directory | 07:27 |
| afazekas | Try to reinstall the packages | 07:28 |
| joy_ | as procedures of the doc tells that nova.conf is required but I dont have /etc/nova. how to get /etc/nova | 07:28 |
| joy_ | I tried - apt-get install nova-novncproxy novnc nova-api \ nova-ajax-console-proxy nova-cert nova-conductor \ nova-consoleauth nova-doc nova-scheduler \ python-novaclient | 07:28 |
| *** mzoeller has joined #openstack-qa | 07:28 | |
| joy_ | but no /etc/nova is found | 07:28 |
| afazekas | http://packages.ubuntu.com/search?searchon=contents&keywords=nova.conf&mode=exactfilename&suite=trusty&arch=any | 07:30 |
| *** arnaud has quit IRC | 07:31 | |
| afazekas | AFAIK #openstack channel is for the users | 07:31 |
| joy_ | afazekas: thanks for link, I had tried these command and error displayed as - http://pastebin.com/8tdBTynE | 07:32 |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm-partial-ncpu 103238,1,6361ab2 | 07:35 |
| afazekas | joy_: I meamo is to only apt based system I have at the moment. Probably I am not the best person for solving apt/dependency/repo issues. If you are developer or QE/QE you can try openstack in vm by http://devstack.org/ | 07:37 |
| *** leseb has joined #openstack-qa | 07:38 | |
| afazekas | http://askubuntu.com/questions/402326/how-to-manually-run-sudo-dpkg-configure-a # guess | 07:43 |
| openstackgerrit | Sridhar Gaddam proposed a change to openstack/tempest: Add IPv4/IPv6 Provider Network Scenario tests https://review.openstack.org/100143 | 07:43 |
| openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 102397,3,9334e54 | 07:49 |
| *** gfidente has joined #openstack-qa | 07:50 | |
| *** lanoux_ has joined #openstack-qa | 07:50 | |
| *** ildikov has joined #openstack-qa | 07:51 | |
| *** jlanoux has quit IRC | 07:54 | |
| *** jpich has joined #openstack-qa | 07:56 | |
| *** ylobankov has joined #openstack-qa | 08:01 | |
| openstackrecheck | Console logs not available after 13:23s for gate-cinder-python27 103549,3,6d06fe7 | 08:02 |
| *** dims__ has joined #openstack-qa | 08:06 | |
| *** dims__ has quit IRC | 08:11 | |
| *** yfried_ has quit IRC | 08:12 | |
| *** yfried_ has joined #openstack-qa | 08:13 | |
| *** yfried_ has quit IRC | 08:13 | |
| *** yfried_ has joined #openstack-qa | 08:13 | |
| *** amitpp has quit IRC | 08:14 | |
| *** oomichi has quit IRC | 08:14 | |
| *** amitpp has joined #openstack-qa | 08:14 | |
| *** mkollaro has joined #openstack-qa | 08:14 | |
| openstackrecheck | Console logs not available after 13:21s for gate-swift-python27 102598,7,5b857e6 | 08:15 |
| *** yfried_ has quit IRC | 08:18 | |
| *** yfried has joined #openstack-qa | 08:18 | |
| *** safchain has joined #openstack-qa | 08:25 | |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Skip quota-out tests temporary to change error codes https://review.openstack.org/104112 | 08:28 |
| openstackrecheck | Console logs not available after 13:23s for gate-sahara-python26 103691,2,baa0b9a | 08:29 |
| openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Skip quota-out tests temporary to change error codes https://review.openstack.org/104112 | 08:31 |
| *** mkollaro has quit IRC | 08:35 | |
| *** qba73 has joined #openstack-qa | 08:39 | |
| *** yfried has quit IRC | 08:42 | |
| openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm 92412,6,77a5203 | 08:42 |
| *** MIDENN_ has quit IRC | 08:47 | |
| *** mdenny has joined #openstack-qa | 08:47 | |
| *** tosky has joined #openstack-qa | 08:50 | |
| openstackrecheck | Console logs not available after 13:25s for gate-tempest-dsvm-neutron-heat-slow 102743,1,9f6aa6d | 08:56 |
| *** gema has quit IRC | 09:00 | |
| *** gema has joined #openstack-qa | 09:02 | |
| *** leseb has quit IRC | 09:03 | |
| *** Longgeek has joined #openstack-qa | 09:03 | |
| *** Longgeek has quit IRC | 09:03 | |
| *** leseb has joined #openstack-qa | 09:03 | |
| *** leseb has quit IRC | 09:04 | |
| *** leseb has joined #openstack-qa | 09:04 | |
| *** Longgeek has joined #openstack-qa | 09:04 | |
| *** dims__ has joined #openstack-qa | 09:07 | |
| openstackrecheck | Console logs not available after 13:24s for gate-keystonemiddleware-python26 104000,2,561687b | 09:09 |
| *** dmorita has quit IRC | 09:11 | |
| *** dims__ has quit IRC | 09:12 | |
| *** Madan has quit IRC | 09:17 | |
| openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds list users tests in v3 https://review.openstack.org/93704 | 09:17 |
| *** leseb has quit IRC | 09:17 | |
| *** zz_johnthetubagu is now known as johnthetubaguy | 09:17 | |
| *** gszasz has joined #openstack-qa | 09:17 | |
| *** leseb has joined #openstack-qa | 09:18 | |
| *** mkollaro has joined #openstack-qa | 09:19 | |
| *** Madan has joined #openstack-qa | 09:22 | |
| openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm-partial-ncpu 104001,1,1284b8b | 09:22 |
| kashyap | afazekas, Hi, when you get a moment (finally, all voting jobs for this are successful) -- https://review.openstack.org/#/c/102490/ | 09:23 |
| kashyap | afazekas, It has acks from Sean and Dean. Will this get merged w/o again acking explicitly? | 09:24 |
| *** andreaf has joined #openstack-qa | 09:24 | |
| *** leseb has quit IRC | 09:27 | |
| *** leseb has joined #openstack-qa | 09:27 | |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/devstack-gate: Add option to enable profiling https://review.openstack.org/104123 | 09:33 |
| afazekas | kashyap: 102490 in the gate pipeline http://status.openstack.org/zuul/ | 09:33 |
| openstackgerrit | Sergey Skripnick proposed a change to openstack-infra/devstack-gate: Add option to enable profiling https://review.openstack.org/104123 | 09:35 |
| openstackrecheck | Console logs not available after 13:25s for gate-heat-pep8 96947,4,ffd19ed | 09:36 |
| *** Madan has quit IRC | 09:38 | |
| *** Madan has joined #openstack-qa | 09:38 | |
| *** liusheng is now known as liusheng-leave | 09:39 | |
| *** MIDENN_ has joined #openstack-qa | 09:40 | |
| *** mdenny has quit IRC | 09:40 | |
| openstackgerrit | Noboru Iwamatsu proposed a change to openstack-dev/devstack: Check Apache version in Horizon config https://review.openstack.org/104125 | 09:44 |
| *** mlavalle has quit IRC | 09:47 | |
| openstackrecheck | Console logs not available after 13:21s for gate-nova-python26 102645,4,0f5cbed | 09:49 |
| *** MIDENN_ has quit IRC | 09:51 | |
| *** mdenny has joined #openstack-qa | 09:52 | |
| *** mkoderer has quit IRC | 09:52 | |
| *** qba73 has quit IRC | 09:54 | |
| *** qba73 has joined #openstack-qa | 09:54 | |
| *** Madan1 has joined #openstack-qa | 09:55 | |
| *** Madan has quit IRC | 09:58 | |
| openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 100271,7,716541d | 10:03 |
| *** Longgeek has quit IRC | 10:03 | |
| *** Longgeek has joined #openstack-qa | 10:03 | |
| *** leseb has quit IRC | 10:05 | |
| *** leseb has joined #openstack-qa | 10:06 | |
| *** Longgeek has quit IRC | 10:07 | |
| *** dims__ has joined #openstack-qa | 10:08 | |
| *** kbaikov has joined #openstack-qa | 10:10 | |
| kbaikov | hello | 10:11 |
| *** dims__ has quit IRC | 10:13 | |
| *** leseb has quit IRC | 10:13 | |
| *** leseb has joined #openstack-qa | 10:13 | |
| openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-full-icehouse 103367,2,9387c21 | 10:16 |
| *** joy_ has quit IRC | 10:20 | |
| *** qba73_ has joined #openstack-qa | 10:20 | |
| *** Longgeek has joined #openstack-qa | 10:21 | |
| *** qba73 has quit IRC | 10:22 | |
| *** leseb has quit IRC | 10:24 | |
| *** leseb has joined #openstack-qa | 10:25 | |
| *** dkorn has quit IRC | 10:26 | |
| openstackrecheck | Console logs not available after 13:25s for gate-trove-pep8 94484,11,0611611 | 10:29 |
| *** dims__ has joined #openstack-qa | 10:32 | |
| *** SridharG has quit IRC | 10:42 | |
| openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 104000,2,49ed56b | 10:43 |
| openstackgerrit | Longgeek proposed a change to openstack/qa-specs: Update from global requirements https://review.openstack.org/103388 | 10:44 |
| openstackgerrit | Longgeek proposed a change to openstack/qa-specs: Update from global requirements https://review.openstack.org/103388 | 10:49 |
| *** mdenny has quit IRC | 10:49 | |
| *** mdenny has joined #openstack-qa | 10:49 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Restrict 1336172 query on build_name https://review.openstack.org/103933 | 10:54 |
| openstackrecheck | Console logs not available after 13:23s for gate-nova-python26 104008,1,4d682e6 | 10:56 |
| *** ildikov_ has joined #openstack-qa | 10:57 | |
| *** asselin has quit IRC | 10:59 | |
| *** asselin has joined #openstack-qa | 10:59 | |
| *** ildikov has quit IRC | 11:00 | |
| *** openstackrecheck has quit IRC | 11:02 | |
| *** openstackrecheck has joined #openstack-qa | 11:03 | |
| *** mkollaro has quit IRC | 11:05 | |
| *** Longgeek has quit IRC | 11:10 | |
| *** Longgeek has joined #openstack-qa | 11:10 | |
| kashyap | sdague, Morning, you got a sec here? | 11:16 |
| sdague | sure | 11:16 |
| kashyap | I think I made a mistake here -- I should run "reverify bug 1336172" here -- https://review.openstack.org/#/c/102490/ | 11:17 |
| kashyap | Isn't it? | 11:17 |
| kashyap | Since there are approvals from core, now it has to pass all the "Gate" jobs. | 11:17 |
| kashyap | I'm now investigating that Postgres failure meanwhile (it's a bug in telemetry, checking for an existing one before I file another) | 11:18 |
| sdague | yeh, reverify is fine | 11:18 |
| kashyap | Okay, thank you. | 11:19 |
| sdague | I actually think we might need to remove that test | 11:19 |
| sdague | it requires a very fast poll interval, which we don't do any more | 11:19 |
| kashyap | The telemetry test? | 11:20 |
| sdague | yeh | 11:21 |
| sdague | DinaBelova: do you know about that test? | 11:22 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/101882 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1273292 | 11:22 |
| kashyap | I'm first wondering if a bug is worth it to file that trace. I don't see an existing thing in the launchpad | 11:22 |
| DinaBelova | sdague, kashyap, sorry, lost a little bit my mind - which test? | 11:22 |
| kashyap | http://logs.openstack.org/90/102490/4/gate/gate-tempest-dsvm-postgres-full/d931fa7/console.html | 11:22 |
| sdague | test_telemetry_notification_api.py | 11:22 |
| sdague | trying to figure out what it's waiting for, or when that would happen | 11:23 |
| *** coolsvap is now known as coolsvap|afk | 11:23 | |
| *** leseb has quit IRC | 11:23 | |
| DinaBelova | sdague, kashyap, a-ha... | 11:23 |
| *** leseb has joined #openstack-qa | 11:23 | |
| *** lanoux_ has quit IRC | 11:23 | |
| DinaBelova | well, it's waiting to grab the notifications coming from the nova to ceilometer | 11:24 |
| *** Poornima has quit IRC | 11:25 | |
| DinaBelova | in this concrete situation no notification has come for the ceilo expected to be caught from nova instance... | 11:25 |
| DinaBelova | although I'm really not sure why this could happen | 11:25 |
| kashyap | DinaBelova, A bug is worth to investigate the above time out further? | 11:26 |
| DinaBelova | kashyap, yes, indeed this should be investigated, we had several gate time out errors (not connected with ceilo itself in the unit tests gate, so this one is new) | 11:27 |
| kashyap | DinaBelova, Alright, I can file one if you prefer, and can make any edits/add additional details | 11:28 |
| DinaBelova | kashyap, yes, please, and add the link to the openstack-ceilometer channel | 11:28 |
| DinaBelova | all ceilo folks have mid cycle meetup right now | 11:29 |
| DinaBelova | so we can react slow | 11:29 |
| kashyap | DinaBelova, I already did post the log in the channel, will post the bug. Thanks for the heads up. | 11:29 |
| DinaBelova | please don't become angry on us :) | 11:29 |
| kashyap | nosnos, I don't assume malice by default. I've been around IRC for a while, even if I don't get a response, I won't take offense :-) | 11:29 |
| kashyap | (_If_ I get a response, I consider it a bonus) | 11:30 |
| *** leseb has quit IRC | 11:30 | |
| *** leseb has joined #openstack-qa | 11:31 | |
| *** luqas has quit IRC | 11:34 | |
| sdague | kashyap: do we have an ER signature for that? | 11:35 |
| kashyap | sdague, Not yet. I can add one, and add Diana & you for review if you prefer | 11:35 |
| sdague | kashyap: that would be great | 11:36 |
| sdague | yeh, this is new - http://logstash.openstack.org/#eyJzZWFyY2giOiJcIlNhbXBsZSBmb3IgbWV0cmljOm1lbW9yeSB3aXRoIHF1ZXJ5OlwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDA0MzAwOTU3MDE2fQ== | 11:36 |
| kashyap | Heh, whole last week I ended up chasing totally different issues than the ones I intended. But that's the way of OpenStack I guess :-) | 11:36 |
| kashyap | That's the bug - https://bugs.launchpad.net/ceilometer/+bug/1336755 | 11:38 |
| sdague | they only thing in ceilometer I see changing around that window is the osprofiler add | 11:38 |
| kashyap | DinaBelova, Writing an Elastic Recheck query for the above, you can review it when you get time. | 11:38 |
| sdague | I'm trying to figure out if there is a devstack or tempest change that might have impacted it | 11:38 |
| *** mkoderer has joined #openstack-qa | 11:40 | |
| kashyap | The last change related to Ceilo in Devstack is ths commit, which is about a support for vsphere 846609b627bff979ce767dd9ad00daa46a150342 | 11:41 |
| sdague | yeh, I doubt it's that | 11:43 |
| sdague | it's kind of impossible to figure out what's going on in ceilometer from the logs | 11:43 |
| sdague | for instance - http://logs.openstack.org/90/102490/4/gate/gate-tempest-dsvm-postgres-full/d931fa7/logs/screen-ceilometer-acentral.txt.gz is just stack traces | 11:43 |
| sdague | but I don't think any of them are related, I think that's just normal ceilometer stack tracing all time time | 11:44 |
| *** eglynn has joined #openstack-qa | 11:44 | |
| sdague | http://logs.openstack.org/90/102490/4/gate/gate-tempest-dsvm-postgres-full/d931fa7/logs/screen-ceilometer-collector.txt.gz - 50 MB of debug which is mostly amqp debug, which should probably be turned off | 11:45 |
| *** mkollaro has joined #openstack-qa | 11:46 | |
| kashyap | Yeah, turning off needless log messages will certainly help the eyes while staring at the logs | 11:46 |
| *** samuelbercovici has joined #openstack-qa | 11:46 | |
| sdague | yeh, so as far as I can tell ceilometer isn't functioning correctly at all right now as it doesn't seem to have the credentials it needs | 11:46 |
| sdague | so all the polling isn't working | 11:47 |
| sdague | http://logs.openstack.org/90/102490/4/gate/gate-tempest-dsvm-postgres-full/d931fa7/logs/screen-ceilometer-acentral.txt.gz | 11:47 |
| kashyap | sdague, Meanwhile, I'm restricting that ER signature for build name (and project) ? | 11:47 |
| sdague | kashyap: no, I'd keep it with just tags:console | 11:47 |
| kashyap | Okay | 11:47 |
| kashyap | sdague, Maybe a dev-list thread worth of the topic you raised here re: Ceilometer logging | 11:48 |
| sdague | yeh, I've raised it before. It's hard to energize to bring it up again, as there wasn't much uptake the last time. | 11:48 |
| eglynn | sdague: credentials for which polling cycle, nova-api? | 11:49 |
| sdague | actually, neutron client | 11:50 |
| sdague | which is funny, because neutron isn't enabled in this config | 11:50 |
| sdague | http://logs.openstack.org/90/102490/4/gate/gate-tempest-dsvm-postgres-full/d931fa7/logs/screen-ceilometer-acentral.txt.gz | 11:50 |
| * eglynn looking | 11:50 | |
| *** amitpp has quit IRC | 11:50 | |
| sdague | also this - http://logs.openstack.org/90/102490/4/gate/gate-tempest-dsvm-postgres-full/d931fa7/logs/screen-ceilometer-alarm-evaluator.txt.gz | 11:51 |
| sdague | so basically it looks like even in a successful run ceilometer logs exist in one of 2 states: | 11:51 |
| eglynn | sdague: those failures seem somehow worse that just incorrect credentials in the ceil config | 11:51 |
| sdague | 1) completely stacktraces | 11:52 |
| eglynn | "/opt/stack/new/python-novaclient/novaclient/openstack/common/network_utils.py" ... if address[0] == '[': ... IndexError: string index out of range | 11:52 |
| sdague | 2) 50 MB of amqp DEBUG with nothing else | 11:52 |
| *** SridharG has joined #openstack-qa | 11:52 | |
| sdague | eglynn: yeh, that's true | 11:52 |
| sdague | it's hard to see that one in the middle of all the other stack traces though :) | 11:53 |
| *** markmcclain has joined #openstack-qa | 11:53 | |
| eglynn | sdague: does the keystone service catalog config looks sane? | 11:53 |
| sdague | I don't know, but everything else passed in that run | 11:54 |
| openstackgerrit | Kashyap Chamarthy proposed a change to openstack-infra/elastic-recheck: Add query for Ceilometer timeout - bug 1336755 https://review.openstack.org/104160 | 11:54 |
| sdague | so, honestly, right now, after looking at these logs, I'm less concerned about the specific failure, that will come back later and we can deal with it | 11:55 |
| sdague | I'm more concerned that the ceilometer logs are pretty in decypherable | 11:55 |
| sdague | and that screen-ceilometer-collector.txt.gz is larger than all the nova logs combined | 11:56 |
| eglynn | ok we can dial that AMQP logging also, as we did previously on the publication path in ceilo | 11:57 |
| *** ajc_ has quit IRC | 11:58 | |
| *** tkammer has joined #openstack-qa | 11:58 | |
| eglynn | sdague: is that failure pattern "CommandError: Unable to determine the Keystone version to authenticate with using the given auth_url: " occuring for any of the other services? | 11:58 |
| *** nosnos has quit IRC | 11:59 | |
| *** markmcclain1 has joined #openstack-qa | 11:59 | |
| eglynn | sdague: ... i.e. is it ceilo specific? ... or a general keystone problem that impacting across the board? | 11:59 |
| *** markmcclain has quit IRC | 11:59 | |
| *** nosnos has joined #openstack-qa | 11:59 | |
| DinaBelova | kashyap, did this failure appear today? | 11:59 |
| sdague | eglynn: what I know is that we see that in ceilometer logs even in a successful run | 11:59 |
| sdague | which means lots of other things were talking to keystone correctly | 12:00 |
| kashyap | DinaBelova, 26 hits in last 7 days -- all failures | 12:00 |
| *** Shrews has quit IRC | 12:00 | |
| eglynn | kashyap: so its transient failure? | 12:00 |
| sdague | I haven't gone down this rabbit hole any further, I was mostly trying to figure out if there was a clear reason it was happening | 12:00 |
| sdague | the specific fail that kashyap is looking at is a timeout issue | 12:01 |
| kashyap | eglynn, I should admit, I haven't really deeply investigated it. I only encountered it in a Jenkins test failure, and wanted to report it if it wasn't already | 12:01 |
| kashyap | Right, of what Sean said ^ | 12:01 |
| *** jd__ has joined #openstack-qa | 12:01 | |
| *** Shrews has joined #openstack-qa | 12:02 | |
| *** markmcclain1 has quit IRC | 12:02 | |
| eglynn | sdague, kashyap: does the timing of the issue line up with this patch landing? https://review.openstack.org/#/c/97107 | 12:02 |
| *** Longgeek_ has joined #openstack-qa | 12:02 | |
| * kashyap clicks | 12:03 | |
| sdague | nope | 12:03 |
| sdague | the timing issue started about 24hrs ago | 12:03 |
| kashyap | eglynn, ^ More authoritative source. | 12:03 |
| sdague | http://logstash.openstack.org/#eyJzZWFyY2giOiJcIlNhbXBsZSBmb3IgbWV0cmljOm1lbW9yeSB3aXRoIHF1ZXJ5OlwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiI2MDQ4MDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDA0MzAwOTU3MDE2fQ== | 12:03 |
| sdague | the closest ceilometer patch I see is the osprofiler one | 12:04 |
| *** nosnos has quit IRC | 12:04 | |
| jd__ | no way it would do that | 12:04 |
| jd__ | it's likely a change outside Ceilometer :( | 12:04 |
| *** Longgeek has quit IRC | 12:06 | |
| sdague | ok, so it's important to separate out that we're talking about 2 things | 12:06 |
| sdague | 1 think, a ton of ceilometer auth failures | 12:06 |
| sdague | that is *unrelated* to tempest failures | 12:06 |
| sdague | it happens all the time | 12:06 |
| sdague | however it was discovered when I tried to look at the actual test failure and look at the ceilometer logs | 12:07 |
| *** Longgeek_ has quit IRC | 12:07 | |
| *** Longgeek has joined #openstack-qa | 12:08 | |
| sdague | the 2nd thing is a test timeout, and the test is undocumented, so I don't know what other system level dependencies it requires to work. | 12:08 |
| sdague | like if this test just needs to be skipped now that we aren't tight looping the polling any more | 12:08 |
| sdague | though that change was weeks ago | 12:08 |
| kashyap | If the test is undocumented, and is causing failures, it's fair to disable it till it has proper docs, etc so it gives clues when it ends up being the culprit? | 12:10 |
| *** gszasz has quit IRC | 12:10 | |
| kashyap | (s/it's/is it) | 12:10 |
| sdague | well, they are all kind of undocumented, we wanted to start fixing that, but it's hard when you keep chasing failures :) | 12:10 |
| *** samuelbercovici1 has joined #openstack-qa | 12:10 | |
| sdague | but this is in an area that I think the qa core team is unfamiliar with | 12:11 |
| eglynn | sdague: it's a notification test so unrelated to the polling cycle | 12:11 |
| sdague | eglynn: ok | 12:11 |
| eglynn | sdague: but fine to skip for now, so that we can get to the bottom of the root cause without blocking the gate | 12:11 |
| sdague | eglynn: well, it's intermitent, and I think a low enough frequency that we can give it some time | 12:12 |
| sdague | if we skip it, you won't have any more data on it | 12:12 |
| sdague | I just wanted to make sure it's non a systemic issue related to the polling change, which means it *couldn't* work reliably | 12:12 |
| eglynn | sdague: k, leave it in place for now and we'll need to look into any relevant changes to the nova notification logic | 12:12 |
| sdague | cool | 12:12 |
| boris-42 | sdague pls review my patch | 12:13 |
| boris-42 | sdague it's just 2 lines of code and it's super simple... | 12:13 |
| eglynn | sdague: on the keystone auth failures in the ceilo logs ... just to confirm these have been intermittent for a long time without surfacing in actual test failures, right? | 12:13 |
| *** samuelbercovici has quit IRC | 12:13 | |
| *** samuelbercovici1 is now known as samuelbercovici | 12:13 | |
| boris-42 | sdague https://review.openstack.org/#/c/103835/ | 12:13 |
| sdague | eglynn: no, those seem to be 100% of the time on all runs | 12:13 |
| boris-42 | sdague it's really high priority stuff for a lot of projects | 12:13 |
| sdague | eglynn: but it doesn't surface as failures | 12:14 |
| eglynn | sdague: ok, I'll look into that auth failure in the alarm evaluator and also extend the alarming test to always surface the failure | 12:17 |
| sdague | eglynn: cool, thanks | 12:17 |
| eglynn | sdague: ... it'll be a bit of an async effort though as we're currently at mid-cycle meetup | 12:17 |
| sdague | yep, that's fine | 12:17 |
| eglynn | sdague: cool, thank you sir! | 12:18 |
| boris-42 | sdague pleaasse=) | 12:18 |
| sdague | the overall logging might be a good thing to bring up in conversation there, it would be cool if the ceilo logs were a little easier to sift through as someone not expert in the code | 12:18 |
| sdague | boris-42: seriously, if you don't stop pinging me constantly I'm just going to -2 all your patches. This is freaking rediculous. | 12:19 |
| eglynn | sdague: fair point about the logging guidelines, we'll add that to the agenda | 12:19 |
| boris-42 | sdague could you explain me one thing and I'll stop=) | 12:19 |
| boris-42 | sdague the thing is why small changes can't be reviewed fast, I mean okay when something is very big, or has impact some, or it's discussable, I understand that it can, should take some time | 12:20 |
| boris-42 | sdague but it's totally unclear for me, why it's happening for small patches :( | 12:21 |
| sdague | boris-42: because there are a million small changes. I have over 1000 patches I could be +2ing right now. | 12:21 |
| boris-42 | sdague so probably you need some help? | 12:21 |
| boris-42 | sdague with reviwing them ? | 12:22 |
| boris-42 | sdague more cores | 12:22 |
| sdague | so I prioritize things that help us with the gate | 12:22 |
| *** mdenny has quit IRC | 12:22 | |
| sdague | and people that help us with the gate | 12:22 |
| sdague | and non of this does | 12:22 |
| boris-42 | sdague okay could we talk about profiler and how it will help | 12:22 |
| boris-42 | sdague with the gates | 12:22 |
| boris-42 | sdague cause I really don't get it why you think that it won't help with them | 12:22 |
| *** mdenny has joined #openstack-qa | 12:22 | |
| *** gszasz has joined #openstack-qa | 12:23 | |
| kashyap | boris-42, I think he's trying to say: If he's constantly bombarded w/ requests to drop him from debugging Gate issues (which are more serious), he may not be able to take a look at even trivial patches | 12:24 |
| sdague | boris-42: it doesn't help any of our existing issues. And you've demonstrated over and over again that you aren't interested in actually helping with existing issues if it's not your pet project. So, honestly, this stuff is low priority to me. And you constantly pinging is not useful. | 12:24 |
| boris-42 | sdague I am just resolving the same task that you are doing | 12:24 |
| boris-42 | sdague but from other side | 12:24 |
| afazekas | sdague: I assume it is cached: https://review.openstack.org/#/c/102793/ | 12:25 |
| boris-42 | sdague I really appreciate that you are keeping everything work | 12:25 |
| boris-42 | sdague but I don't think that this approach is optimal for long term stuff | 12:25 |
| boris-42 | sdague so I am spending a lot of time for long term solutions | 12:25 |
| sdague | boris-42: well we have a difference of opinion. If you want more energy helping review your stuff, spending some time on solving the existing issues we've got would be helpful. That's how I get more time for reviewing code. | 12:26 |
| boris-42 | sdague yep we have difference of opinion, but i am not saying that your work is not important, and you are demonstrating that we in Rally are doing shit with out osprofiler, which makes me sad | 12:28 |
| kashyap | boris-42, I don't see that implication in his words here | 12:29 |
| boris-42 | kashyap it's not the first time that we are speaking about this* | 12:29 |
| *** jlanoux has joined #openstack-qa | 12:29 | |
| kashyap | He's clearly trying to say -- Imminent Gate issues (that not many are spending time to debug) that are blocking _current_ patches in queue are more important | 12:29 |
| sdague | kashyap: yes, exactly | 12:29 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/103036 | 12:30 |
| *** ekhugen has joined #openstack-qa | 12:30 | |
| boris-42 | sdague kashyap probably the issue is that my work is "low priority", and it shouldn't be .. | 12:31 |
| boris-42 | sdague but ok sorry for pinging you and hope that someday you won't think that it's low priority bs.. | 12:32 |
| sdague | afazekas: so, actually, I the trove folks did this good thing where they put the image up on tarballs.openstack.org | 12:35 |
| sdague | which I actually think is what we probably want to do with the fedora images as well | 12:35 |
| sdague | because that should handle the mirror issue | 12:35 |
| sdague | I'm not sure how they did that exactly | 12:36 |
| sdague | but asking in infra would be good | 12:36 |
| openstackgerrit | Mikhail Durnosvistov proposed a change to openstack/tempest: Expand baremetal nodes coverage `POST` https://review.openstack.org/74065 | 12:36 |
| openstackgerrit | Mikhail Durnosvistov proposed a change to openstack/tempest: Expand baremetal nodes coverage `GET` https://review.openstack.org/71476 | 12:36 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/103036 | 12:39 |
| afazekas | sdague: AFAIK trove needs special image which contains a trove agent and several extra service packages installed | 12:41 |
| *** k4n0 has quit IRC | 12:41 | |
| sdague | afazekas: sure, but in posting to tarballs.openstack.org, they don't have the chance of download failure | 12:42 |
| *** BobBall_ has joined #openstack-qa | 12:42 | |
| sdague | because the fedora mirrors are very unreliable | 12:42 |
| *** BobBall has quit IRC | 12:42 | |
| afazekas | sdague: Do you have a logstash query for fedora mirror issues ? | 12:43 |
| sdague | afazekas: nope | 12:43 |
| *** Poornima has joined #openstack-qa | 12:43 | |
| *** leseb has quit IRC | 12:45 | |
| *** leseb has joined #openstack-qa | 12:45 | |
| *** adalbas has joined #openstack-qa | 12:45 | |
| afazekas | sdague: A logstash query would be good evidence to the Fedora infra they need to check/do something, because at my location it works so I cannot confirm the issue | 12:46 |
| sdague | afazekas: sure, but I've got about 20 issues people want more evidence of, so unless you have a time machine, you won't get that out of me :) | 12:46 |
| sdague | anyway, I'm telling you what I've seen | 12:47 |
| *** BobBall_ has quit IRC | 12:47 | |
| *** BobBall has joined #openstack-qa | 12:47 | |
| *** BobBall is now known as Guest60921 | 12:47 | |
| sdague | and a way we could get around it | 12:47 |
| kashyap | afazekas, Yes. | 12:48 |
| afazekas | I remember I created an LP entry with the failure, but I can't find that bug anymore | 12:48 |
| kashyap | (About evidence of mirror issues.) | 12:48 |
| kashyap | Kevin Fenzi from Fedora infra keeps on asking if OpenStack upstream has any mirror issues, and specific evidence | 12:48 |
| kashyap | He's ready to jump on it to act | 12:48 |
| sdague | so the way to test that is to add a new image to the devstack job that couldn't be cached in a path that it would download from the mirrors directly | 12:49 |
| sdague | the last time we did that, we'd fail 2 out of 10 tests in a run based on a mirror fail | 12:50 |
| kashyap | sdague, You have a URL to that patch from your mail archives? If it's too much hassle, leave it, I'll search | 12:50 |
| sdague | kashyap: I don't, this was what fell out of adding the heat image | 12:50 |
| * kashyap goes to look at DevStack code to see if he could cook up a patch to test the above, thanks for the idea, Sean | 12:51 | |
| sdague | it's why that whole work around block exists in devstack | 12:51 |
| *** luqas has joined #openstack-qa | 12:51 | |
| sdague | remember, that if we add that image, we're downloading fedora image 25k times a week | 12:52 |
| sdague | and iirc, the mirror infrastructure redirects you to a random mirror | 12:52 |
| *** dims__ has quit IRC | 12:52 | |
| sdague | and some of those would die part way through the download | 12:53 |
| kashyap | I see, | 12:53 |
| *** dims__ has joined #openstack-qa | 12:53 | |
| sdague | realistically, I don't think it's an issue we should try to solve | 12:53 |
| sdague | because unless all fedora mirrors are commiting to 99.999% uptime, it's going to remain a problem | 12:54 |
| kashyap | So, stressing the infra to download the image 25K imes, is that useful path to go to test that w/ an image from a new location? | 12:54 |
| *** weshay has joined #openstack-qa | 12:54 | |
| sdague | it's the same reason we don't clone from github | 12:54 |
| kashyap | Hmm, how can we ask random volunteers to guarantee five nines up-time? | 12:54 |
| sdague | kashyap: right, exactly | 12:55 |
| sdague | which is why I don't think we want to bother solving that | 12:55 |
| sdague | instead, if you want to add an image to devstack, host it in infra | 12:55 |
| kashyap | Like you suggested above | 12:55 |
| sdague | yep | 12:55 |
| kashyap | tarballs.openstack.org/fedora | 12:55 |
| kashyap | ? | 12:55 |
| sdague | sure | 12:55 |
| kashyap | Hmm, that's an interesting idea | 12:55 |
| kashyap | I'll ask on #openstack-infra what do they think? | 12:55 |
| sdague | yep | 12:56 |
| sdague | that would be great | 12:56 |
| kashyap | Or should I raise a mailing list thread? | 12:56 |
| sdague | honestly, I think #openstack-infra is fine | 12:56 |
| kashyap | Alright, I'll update here | 12:56 |
| sdague | the reason I bring this up again, is we actually screwed up in adding the trove image, so it wasn't cached in nodepool | 12:56 |
| sdague | but we didn't notice for days | 12:56 |
| sdague | because we were 100% reliably downloading it during the runs | 12:57 |
| sdague | which was very different then when we tried this with the fedora mirrors | 12:57 |
| *** jlibosva has quit IRC | 12:57 | |
| kashyap | I see, thanks for the context, I don't have the background of trove issues | 12:57 |
| *** dkranz has joined #openstack-qa | 12:58 | |
| afazekas | BTW: we have download and DNS issues even with openstack infra, I do not know we had more issue with the fedora image download or not | 12:58 |
| * afazekas I had to hurry for a meeting | 12:58 | |
| kashyap | afazekas, Okay, trying doesn't hurt I guess | 12:58 |
| kashyap | (If infra folks agree) | 12:58 |
| sdague | afazekas: ok, so that entire workaround (the staging) in devstack was because of fedora mirror failures | 12:58 |
| sdague | sorry if I hadn't surfaced that more | 12:59 |
| *** jlibosva has joined #openstack-qa | 12:59 | |
| *** BobBall_ has joined #openstack-qa | 13:00 | |
| * afazekas re mtg rescheduled | 13:00 | |
| *** leseb has quit IRC | 13:01 | |
| *** leseb has joined #openstack-qa | 13:01 | |
| *** mdenny has quit IRC | 13:01 | |
| *** BobBall_ is now known as GnomeKing | 13:01 | |
| afazekas | kashyap: FIXME the fedora mirror's pick a different server based on the requester IP (geo location) | 13:02 |
| *** tkammer has quit IRC | 13:03 | |
| kashyap | afazekas, That's in nodepool git repo? I don't find it in devstack git | 13:03 |
| *** chandan_kumar is now known as chandankumar | 13:03 | |
| *** Guest60921 has quit IRC | 13:03 | |
| afazekas | kashyap: This change touching the related code: https://review.openstack.org/#/c/102793/ | 13:05 |
| kashyap | afazekas, I just posted that question in #openstack-infra to host the image on tarballs.openstack. . . | 13:05 |
| afazekas | kashyap: Round robin style DNS entry detected for dl.fedoraproject.org | 13:05 |
| afazekas | kashyap: ok | 13:05 |
| *** psharma has quit IRC | 13:09 | |
| kashyap | afazekas, Please read the scroll on #openstack-infra | 13:09 |
| openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: fix fingerprint to deal with n-obj hit https://review.openstack.org/104185 | 13:10 |
| *** Madan1 has quit IRC | 13:11 | |
| *** evgenyf has quit IRC | 13:11 | |
| *** xianghui has quit IRC | 13:12 | |
| afazekas | sdague: It is already cached, so we can switch to Fedora-x86_64-20-20131211.1-sda by https://review.openstack.org/#/c/102793/ without mirror/download issue risk. | 13:13 |
| sdague | afazekas: ok | 13:13 |
| *** GnomeKin1 has joined #openstack-qa | 13:14 | |
| *** samuelbercovici1 has joined #openstack-qa | 13:17 | |
| *** GnomeKing has quit IRC | 13:17 | |
| *** tkammer has joined #openstack-qa | 13:18 | |
| *** malini1 has joined #openstack-qa | 13:18 | |
| *** samuelbercovici has quit IRC | 13:20 | |
| *** samuelbercovici1 is now known as samuelbercovici | 13:20 | |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack-dev/devstack: Ironic: Setup/Configure iPXE https://review.openstack.org/99677 | 13:22 |
| *** mriedem has joined #openstack-qa | 13:22 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: fix fingerprint to deal with n-obj hit https://review.openstack.org/104185 | 13:23 |
| *** GnomeKin1 is now known as Bobba | 13:24 | |
| *** evgenyf has joined #openstack-qa | 13:28 | |
| *** xianghui has joined #openstack-qa | 13:29 | |
| openstackgerrit | Ray Chen proposed a change to openstack-dev/devstack: The VG_DEV should be set before delete the loop device. https://review.openstack.org/104193 | 13:29 |
| *** openstackrecheck has quit IRC | 13:32 | |
| *** openstackrecheck has joined #openstack-qa | 13:33 | |
| *** pcrews has joined #openstack-qa | 13:37 | |
| *** jlibosva has quit IRC | 13:38 | |
| *** jlibosva has joined #openstack-qa | 13:39 | |
| *** Obulpathi has joined #openstack-qa | 13:39 | |
| mtreinish | sdague: on https://review.openstack.org/#/c/98918/ for your comments I actually screwed up, the rule passes if it's a left aligned paren. | 13:41 |
| mtreinish | the ugliness/confusion is coming from the fact that there are >=5 fields for each of those assigns | 13:41 |
| mtreinish | I can still disable E123, I'm just not sure whether it's better to leave it on because it works well everywhere else | 13:43 |
| mtreinish | hmm, actually by default in pep8 that rule is ignored because there isn't consensus around it (at least according to the docs) | 13:44 |
| mtreinish | so I'll just turn it off | 13:44 |
| mtreinish | jogo_away: ^^^ why do we enable that then? | 13:45 |
| *** Obulpathi has quit IRC | 13:45 | |
| *** Obulpathi has joined #openstack-qa | 13:45 | |
| *** leseb has quit IRC | 13:46 | |
| *** jordanP has quit IRC | 13:50 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable H407,H305,H307,E122 ignore E123 https://review.openstack.org/98918 | 13:54 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/103913 failed gate-tempest-dsvm-postgres-full, gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error | 13:57 |
| *** amitpp has joined #openstack-qa | 13:57 | |
| *** jordanP has joined #openstack-qa | 13:59 | |
| *** amitpp has quit IRC | 14:00 | |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable E128 ignore E129 https://review.openstack.org/98909 | 14:02 |
| openstackgerrit | Matthew Treinish proposed a change to openstack/tempest: Enable H407,H305,H307,E122 ignore E123 https://review.openstack.org/98918 | 14:02 |
| *** rbak has joined #openstack-qa | 14:07 | |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/102490 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 14:09 |
| *** rwsu has joined #openstack-qa | 14:10 | |
| *** rbak has quit IRC | 14:10 | |
| *** jlibosva has quit IRC | 14:11 | |
| *** jlibosva has joined #openstack-qa | 14:11 | |
| *** salv-orlando has joined #openstack-qa | 14:14 | |
| *** rbak has joined #openstack-qa | 14:14 | |
| *** leseb has joined #openstack-qa | 14:18 | |
| mriedem | mtreinish: do you remember the tempest utility script that you were referring to in the ML yesterday on unit testing and entry points? | 14:18 |
| mriedem | i've got my unit tests working in nova for my script but not the entry point yet | 14:18 |
| mtreinish | mriedem: yeah anything in tempest.cmd the first one was verify_tempest_config.py | 14:19 |
| openstackgerrit | David Paterson proposed a change to openstack/qa-specs: Add spec for bp:post-run-cleanup https://review.openstack.org/91725 | 14:19 |
| mtreinish | mriedem: I was thinking just adding a subcommand in nova-manage makes a lot of sense to me too | 14:19 |
| mtreinish | because it is a db thing | 14:19 |
| mtreinish | I mean it makes sense besides the fact that nova-manage sucks | 14:20 |
| *** afazekas has quit IRC | 14:21 | |
| *** yfried has joined #openstack-qa | 14:22 | |
| *** evgenyf has quit IRC | 14:25 | |
| openstackgerrit | Dean Troyer proposed a change to openstack-dev/devstack: Support multiple Cinder backend types https://review.openstack.org/100271 | 14:25 |
| *** hockeynu_ has joined #openstack-qa | 14:25 | |
| mriedem | mtreinish: yeah i wanted to avoid adding things to nova-manage | 14:25 |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/tempest: Add glance notifications tests for ceilometer https://review.openstack.org/71259 | 14:26 |
| *** hockeyn__ has joined #openstack-qa | 14:26 | |
| *** hockeynu_ has quit IRC | 14:30 | |
| *** evgenyf has joined #openstack-qa | 14:30 | |
| *** sabeen2 has joined #openstack-qa | 14:34 | |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack-dev/devstack: Ironic: Setup/Configure iPXE https://review.openstack.org/99677 | 14:34 |
| *** evgenyf has quit IRC | 14:35 | |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/tempest: Add glance notifications tests for ceilometer https://review.openstack.org/71259 | 14:37 |
| *** sabeen2 has quit IRC | 14:37 | |
| *** sabeen1 has quit IRC | 14:38 | |
| *** sabeen2 has joined #openstack-qa | 14:38 | |
| *** mkollaro has quit IRC | 14:40 | |
| *** Bobba is now known as BobBall | 14:40 | |
| *** Poornima has quit IRC | 14:41 | |
| *** jgrimm has joined #openstack-qa | 14:44 | |
| *** andreaf has quit IRC | 14:51 | |
| *** hockeynu_ has joined #openstack-qa | 14:51 | |
| *** coolsvap|afk is now known as coolsvap | 14:53 | |
| *** dims__ has quit IRC | 14:55 | |
| *** hockeyn__ has quit IRC | 14:55 | |
| *** hockeynu_ has quit IRC | 14:55 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/104009 failed because of: gate-nova-python26: https://bugs.launchpad.net/bugs/1311778 | 14:56 |
| *** hockeynu_ has joined #openstack-qa | 14:56 | |
| *** k4n0 has joined #openstack-qa | 15:01 | |
| *** hockeynu_ has quit IRC | 15:02 | |
| *** krtaylor has quit IRC | 15:05 | |
| *** jaypipes-GOOAL has quit IRC | 15:06 | |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: update iptables rules for more useful debugging https://review.openstack.org/104220 | 15:13 |
| *** xianghui has quit IRC | 15:18 | |
| openstackgerrit | A change was merged to openstack-dev/devstack: Switch to Fedora-x86_64-20-20140618-sda https://review.openstack.org/102793 | 15:19 |
| *** dims__ has joined #openstack-qa | 15:23 | |
| mriedem | mtreinish: zhikunliu figured out why i wasn't getting nova --version values back for my cli skip decorator, | 15:25 |
| mriedem | merge_stderrr=True | 15:25 |
| *** jlibosva has quit IRC | 15:25 | |
| mriedem | just like the nova-manage version tests | 15:25 |
| *** dims__ has quit IRC | 15:27 | |
| *** dims__ has joined #openstack-qa | 15:27 | |
| kashyap | Err, can I delete a comment in LP? -- I erroneously made it in a wrong bug :-( | 15:27 |
| * kashyap adds a note to disregard it | 15:28 | |
| openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-postgres-full 103880,1,0eb9e58 | 15:33 |
| *** rdekel has quit IRC | 15:35 | |
| openstackgerrit | Matt Riedemann proposed a change to openstack/tempest: Add min_client_version decorator for CLI tests https://review.openstack.org/100031 | 15:36 |
| *** mzoeller has quit IRC | 15:36 | |
| *** yfried has quit IRC | 15:40 | |
| *** freyes has joined #openstack-qa | 15:41 | |
| openstackgerrit | Lucas Alvares Gomes proposed a change to openstack-dev/devstack: Ironic: Setup/Configure iPXE https://review.openstack.org/99677 | 15:42 |
| *** yfried has joined #openstack-qa | 15:43 | |
| *** aimon has quit IRC | 15:43 | |
| *** SridharG has quit IRC | 15:43 | |
| *** afazekas has joined #openstack-qa | 15:44 | |
| *** aimon has joined #openstack-qa | 15:44 | |
| *** chandankumar has quit IRC | 15:44 | |
| *** markmcclain has joined #openstack-qa | 15:45 | |
| *** leseb has quit IRC | 15:46 | |
| *** leseb has joined #openstack-qa | 15:46 | |
| *** leseb has quit IRC | 15:46 | |
| *** leseb has joined #openstack-qa | 15:47 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/103036 | 15:47 |
| *** samuelbercovici1 has joined #openstack-qa | 15:50 | |
| openstackrecheck | Console logs not available after 13:30s for check-tempest-dsvm-neutron 102851,4,b6bd1aa | 15:50 |
| *** nati_ueno has joined #openstack-qa | 15:50 | |
| *** k4n0 has quit IRC | 15:52 | |
| *** samuelbercovici has quit IRC | 15:53 | |
| *** samuelbercovici1 is now known as samuelbercovici | 15:53 | |
| mtreinish | mriedem: heh, nice | 15:54 |
| openstackgerrit | Boris Pavlovic proposed a change to openstack-dev/devstack: Add ability to enable OSprofiler https://review.openstack.org/103835 | 15:55 |
| openstackrecheck | openstack/sahara change: https://review.openstack.org/101845 failed gate-sahara-pep8 in the gate queue with an unrecognized error | 15:55 |
| *** Sukhdev has joined #openstack-qa | 15:58 | |
| *** aimon has quit IRC | 16:01 | |
| *** tkammer has quit IRC | 16:07 | |
| *** arnaud has joined #openstack-qa | 16:08 | |
| *** ildikov has joined #openstack-qa | 16:09 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 102633,5,2b6219a | 16:09 |
| *** ildikov|2 has joined #openstack-qa | 16:11 | |
| *** ildikov has quit IRC | 16:11 | |
| *** coolsvap is now known as coolsvap|afk | 16:12 | |
| *** ildikov_ has quit IRC | 16:12 | |
| *** ekhugen has quit IRC | 16:14 | |
| *** jlanoux has quit IRC | 16:14 | |
| yfried | dkranz: ping | 16:15 |
| yfried | mtreinish: any chance you could drive this thing home? https://review.openstack.org/#/c/62101/ | 16:16 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/98470 failed because of: gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1298472 | 16:18 |
| *** atiwari has joined #openstack-qa | 16:19 | |
| *** mriedem has quit IRC | 16:19 | |
| *** jordanP has quit IRC | 16:21 | |
| *** SridharG has joined #openstack-qa | 16:25 | |
| *** krtaylor has joined #openstack-qa | 16:30 | |
| *** ildikov|2 has quit IRC | 16:33 | |
| *** nati_ueno has quit IRC | 16:33 | |
| *** eglynn has quit IRC | 16:34 | |
| *** leseb has quit IRC | 16:34 | |
| *** leseb has joined #openstack-qa | 16:34 | |
| *** Obulpathi has quit IRC | 16:36 | |
| dkranz | yfried: pong | 16:37 |
| yfried | dkranz: I brought this spec up in meeting 2 weeks ago https://review.openstack.org/#/c/101232/ | 16:37 |
| yfried | dkranz: I was told to wait and let people take a look | 16:38 |
| dkranz | yfried: Yes, we have had problems getting reviews. There is a spec review day next week where they will all be done. | 16:39 |
| yfried | dkranz: it's been 2 weeks and no review. I was hoping I could get it discussed tomorrow | 16:39 |
| dkranz | yfried: Please put it on the agenda. | 16:39 |
| *** leseb has quit IRC | 16:39 | |
| yfried | dkranz: for tomorrow? | 16:40 |
| dkranz | yfried: sure | 16:40 |
| *** samuelbercovici has quit IRC | 16:42 | |
| yfried | dkranz: what's with the bp section? we have specs now, so why is it still there? | 16:42 |
| dkranz | yfried: For tracking purposes. bp is horrible for writing and reviewing a spec, but gerrit is horrible for tracking | 16:43 |
| dkranz | yfried: For example, when you put a reference to a blueprint in a commit message it automatically shows up in the blueprint | 16:44 |
| *** hemna__ is now known as hemna | 16:45 | |
| openstackrecheck | Console logs not available after 13:51s for check-tempest-dsvm-full 104011,2,b186146 | 16:46 |
| *** jpich has quit IRC | 16:47 | |
| *** Obulpathi has joined #openstack-qa | 16:48 | |
| *** markmcclain has quit IRC | 16:48 | |
| *** zhiyan is now known as zhiyan_ | 16:50 | |
| *** aimon has joined #openstack-qa | 16:50 | |
| *** zhiyan_ is now known as zhiyan | 16:51 | |
| *** harlowja_away is now known as harlowja | 16:52 | |
| *** markmcclain has joined #openstack-qa | 16:58 | |
| *** ekhugen has joined #openstack-qa | 16:58 | |
| *** markmcclain has quit IRC | 16:58 | |
| *** markmcclain has joined #openstack-qa | 16:59 | |
| *** jamielennox is now known as jamielennox|away | 17:02 | |
| *** Longgeek has quit IRC | 17:02 | |
| *** qba73_ has quit IRC | 17:03 | |
| *** leseb has joined #openstack-qa | 17:04 | |
| *** mmaglana has joined #openstack-qa | 17:06 | |
| *** qba73 has joined #openstack-qa | 17:06 | |
| *** zhiyan is now known as zhiyan_ | 17:08 | |
| *** leseb has quit IRC | 17:09 | |
| openstackrecheck | Console logs not available after 13:27s for check-grenade-dsvm-partial-ncpu 102359,3,cd262a2 | 17:11 |
| *** qba73 has quit IRC | 17:12 | |
| *** leseb has joined #openstack-qa | 17:13 | |
| *** qba73 has joined #openstack-qa | 17:14 | |
| *** qba73 has quit IRC | 17:15 | |
| *** johnthetubaguy is now known as zz_johnthetubagu | 17:15 | |
| *** luqas has quit IRC | 17:16 | |
| *** qba73 has joined #openstack-qa | 17:17 | |
| openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-neutron 98457,5,8c0e11b | 17:24 |
| *** mriedem has joined #openstack-qa | 17:25 | |
| *** leseb has quit IRC | 17:30 | |
| *** leseb has joined #openstack-qa | 17:31 | |
| *** rbak_ has joined #openstack-qa | 17:32 | |
| *** rbak has quit IRC | 17:32 | |
| mtreinish | yfried: I'm too close to the patch. I'd rather have another core +A it just so I don't miss something | 17:32 |
| *** leseb has quit IRC | 17:35 | |
| *** mwagner_ has quit IRC | 17:35 | |
| yfried | mtreinish: understood. Could you please get another core member to take a look? dkranz did, so that takes afazekas out of the question | 17:37 |
| openstackrecheck | Console logs not available after 13:25s for gate-tempest-dsvm-full 103913,4,d3ae4f2 | 17:38 |
| *** Obulpathi has quit IRC | 17:40 | |
| *** leseb has joined #openstack-qa | 17:45 | |
| asselin | anyone have an optionion where the api tracking code should go? line 102 https://review.openstack.org/#/c/97589/4/specs/stress-api-tracking.rst | 17:45 |
| asselin | s/optionion/opinion/ | 17:47 |
| mriedem | sdague: so i guess my nova change hit ssh timeout bug 1298472, and the fingerprint shows up in the console log, but e-r never reported on it | 17:48 |
| mriedem | i haven't seen e-r reporting on things as much lately it seems | 17:48 |
| *** luqas has joined #openstack-qa | 17:48 | |
| mriedem | well, i take that back | 17:49 |
| mriedem | it's just spotty | 17:49 |
| *** openstackgerrit has quit IRC | 17:49 | |
| sdague | mriedem: you see ^^^^ | 17:49 |
| sdague | <openstackrecheck> Console logs not available after 13:25s for gate-tempest-dsvm-full 103913,4,d3ae4f2 | 17:49 |
| sdague | that means ES is backlogged | 17:49 |
| sdague | and we give up after a while | 17:50 |
| *** openstackgerrit has joined #openstack-qa | 17:50 | |
| sdague | under the assumption that real time is less useful 15 minutes after results hit | 17:50 |
| *** leseb has quit IRC | 17:50 | |
| sdague | we're doing hard deadline scheduling on that | 17:50 |
| mriedem | minutes or seconds? | 17:50 |
| sdague | 13m 25s | 17:51 |
| mriedem | ok, 13:25s is confusing w/o the m | 17:51 |
| mriedem | anyway, ok | 17:51 |
| sdague | also at the bottom of http://status.openstack.org/zuul/ there is now the ES work chart | 17:51 |
| sdague | right now we're about 15k events behind | 17:51 |
| sdague | if we're more than a few hundred behind we start missing deadlines | 17:52 |
| mriedem | so i've seen that graph several times, and what you're saying is we want a queue of 0? | 17:52 |
| mriedem | optimally? | 17:53 |
| sdague | mriedem: yep | 17:53 |
| sdague | we have like 25 workers, so anything < 100 is fine | 17:54 |
| sdague | clarkb could give you a more informed opinion | 17:54 |
| sdague | but > 10k... not fine | 17:54 |
| mriedem | ok, the graph is so tiny it looks like 0 workers | 17:54 |
| sdague | yeh | 17:54 |
| sdague | they fall off when we're hitting backups | 17:54 |
| mriedem | and between 100 jobs in the queue and 10K that's hard to tell in that graph | 17:54 |
| sdague | clarkb is trying to rotated out the volumes to ssd volumes | 17:54 |
| clarkb | we have 20 nodes with 3 workers each basically one cpu per worker and logstash | 17:55 |
| sdague | but we seem to get caught in io rate limitting | 17:55 |
| *** Shrews has quit IRC | 17:55 | |
| mriedem | yeah i've heard about the bad neighbors and such | 17:55 |
| clarkb | you can always grab the graphs on the status page and manipulate them | 17:55 |
| mriedem | dicks doing backups :) | 17:55 |
| mriedem | clarkb: is that here? | 17:55 |
| mriedem | http://graphite.openstack.org/ | 17:55 |
| clarkb | mriedem: yes, but I find it easier to just right click on graph, copy image url, paste into new tab and go | 17:55 |
| clarkb | so I have a variety of those that show me different things | 17:56 |
| clarkb | like in the last 5 minutes the backlog dropped 7.5k events | 17:56 |
| mriedem | clarkb: ok, copy and edit image is new to me | 17:56 |
| mriedem | pro tips | 17:56 |
| clarkb | and just about every one of our 60 workers has been in use constantly for the last ~40 minutes | 17:57 |
| clarkb | mriedem: ya the graphite api is relatively well documented and I find it easier to just edit urls after looking at docs than to use that dashboard they have | 17:57 |
| mriedem | clarkb: ok, exploding the image in another tab is useful, good to know | 17:57 |
| *** Sukhdev has quit IRC | 17:58 | |
| *** arnaud has quit IRC | 17:59 | |
| clarkb | sdague: mriedem: just looking at things this morning we seem to be healthier than yesterday with the first ssd volume in | 18:00 |
| sdague | clarkb: cool | 18:00 |
| clarkb | we have spiked a bit recently but we are catching up | 18:00 |
| sdague | though it does tend to recover overnight | 18:00 |
| clarkb | yes | 18:00 |
| sdague | yeh, I'd really like to figure out what the spikes corolate to | 18:00 |
| clarkb | ++ | 18:00 |
| clarkb | the easiest correlation is the to jobs run graph | 18:01 |
| clarkb | we spiked the number of jobs we ran. I don't think that is causation though. My hunch is more jobs means higher chance for hitting the bad stuff | 18:01 |
| clarkb | and not just more stuff being a problem | 18:01 |
| *** malini2 has joined #openstack-qa | 18:01 | |
| clarkb | I need to figure out a good way to monitor that | 18:01 |
| sdague | mtreinish: so, I'm honestly starting to question the wisdom of using the official clients in scenarios | 18:02 |
| sdague | because they really aren't built for debuggability here, and jumping between their call interface and the tempest client is a reasonable context switch | 18:03 |
| *** malini2 has quit IRC | 18:03 | |
| openstackrecheck | Console logs not available after 13:28s for check-grenade-dsvm 101351,3,627f261 | 18:03 |
| *** malini2 has joined #openstack-qa | 18:03 | |
| *** malini2 has quit IRC | 18:04 | |
| *** malini2 has joined #openstack-qa | 18:04 | |
| *** malini1 has quit IRC | 18:05 | |
| *** nati_ueno has joined #openstack-qa | 18:05 | |
| mtreinish | sdague: heh, well maybe that's something we should tell dtroyer to fix... :) | 18:05 |
| mtreinish | but I agree, they're not as nice to work with | 18:06 |
| mtreinish | espeically when you factor in that they don't all have the same interface | 18:06 |
| *** Obulpathi has joined #openstack-qa | 18:07 | |
| mtreinish | I think the theory is that the scenarios are supposed to be end user simulators makes sense to me. But, I guess we can raise migrating them to the tempest clients in a meeting | 18:08 |
| sdague | right, the problem is there is 0 stepwise verification in them | 18:09 |
| sdague | honestly, my curren suspicion is this is a security group race | 18:09 |
| openstackgerrit | David Kranz proposed a change to openstack/tempest: First part of moving success response checking to identity client https://review.openstack.org/104290 | 18:11 |
| *** Shrews has joined #openstack-qa | 18:12 | |
| mtreinish | sdague: hmm, yeah I hadn't thought about that. It'd be hard to code around that if it's doing multistep operations inside the client. | 18:13 |
| mtreinish | I guess in the mean time to test we could switch sec groups to the tempest client | 18:13 |
| sdague | yeh | 18:14 |
| openstackgerrit | A change was merged to openstack-dev/grenade: Ensure upgrade testing tests same deployment methods https://review.openstack.org/100764 | 18:14 |
| openstackgerrit | Adam Gandelman proposed a change to openstack/tempest: Fixes ephemeral disk flavor test for non-virt https://review.openstack.org/104069 | 18:15 |
| *** dkorn has joined #openstack-qa | 18:15 | |
| *** marun_afk is now known as marun | 18:18 | |
| *** amitpp has joined #openstack-qa | 18:19 | |
| *** Obulpathi has quit IRC | 18:31 | |
| *** sirushti has left #openstack-qa | 18:32 | |
| dkranz | mtreinish: Just to verify, we are now skipping nova v3 tests in the gate, but are allowing changes to the tests that are no longer running. | 18:32 |
| mtreinish | dkranz: you need to run check experimental on changes to v3 tests | 18:34 |
| mtreinish | just like an incubated project | 18:34 |
| openstackrecheck | Console logs not available after 13:30s for check-tempest-dsvm-neutron 102490,4,81659ec | 18:34 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/101883 failed because of: gate-tempest-dsvm-neutron: unrecognized error, gate-tempest-dsvm-full: unrecognized error, gate-tempest-dsvm-neutron-large-ops: unrecognized error, gate-grenade-dsvm: unrecognized error, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1298472, gate-tempest-dsvm-neutron-heat-slow: unrecognized error | 18:36 |
| openstackrecheck | openstack/swift change: https://review.openstack.org/97615 failed gate-swift-python26 in the gate queue with an unrecognized error | 18:36 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/104021 failed gate-nova-python26 in the gate queue with an unrecognized error | 18:36 |
| *** dims__ has quit IRC | 18:37 | |
| *** SridharG has quit IRC | 18:41 | |
| *** amitpp has quit IRC | 18:41 | |
| *** Obulpathi has joined #openstack-qa | 18:42 | |
| dkranz | mtreinish: ah, ok | 18:43 |
| dkranz | mtreinish: Did we announce that? | 18:43 |
| mtreinish | I think we mentioned it in the meeting | 18:43 |
| mtreinish | I don't think we took it to the ML | 18:43 |
| dkranz | mtreinish: I will do that if you don't want to. | 18:44 |
| *** nati_ueno has quit IRC | 18:44 | |
| *** mlavalle has joined #openstack-qa | 18:44 | |
| mtreinish | dkranz: sure, thanks that's one less thing I have to take to the ML | 18:45 |
| *** nati_ueno has joined #openstack-qa | 18:45 | |
| dkranz | mtreinish: ok, np. So the nova team is running check experimental on such changes too? | 18:47 |
| *** dims__ has joined #openstack-qa | 18:47 | |
| mtreinish | dkranz: dunno, I'd assume so, but I didn't think there were really any v3 patches in motion | 18:48 |
| *** Obulpathi has quit IRC | 18:48 | |
| mtreinish | I'd check with oomichi and cyeoh | 18:48 |
| dkranz | mtreinish: k. I'll send to [nova][qa] | 18:48 |
| dkranz | mtreinish: and they can chime in | 18:48 |
| *** Obulpathi has joined #openstack-qa | 18:49 | |
| *** nati_uen_ has joined #openstack-qa | 18:53 | |
| openstackgerrit | afazekas proposed a change to openstack-infra/elastic-recheck: Add query for pypi index fetching issues https://review.openstack.org/104298 | 18:54 |
| *** gszasz has quit IRC | 18:56 | |
| *** nati_ueno has quit IRC | 18:56 | |
| adam_g | mtreinish, sdague would you guys be available in a bit or tomorrow to go over some goals for the rest of the cycle WRT improvements to ironic CI? i know theres been on-and-off chatter among different people over the last couple weeks, i'd like to collect thoughts in an etherpad so we're all on the same page on what needs to happen | 18:57 |
| adam_g | devananda, Shrews ^ | 18:57 |
| devananda | adam_g: I need to step away briefly a few times today, but can be flexible. Pretty much any time in the next ~5hr is good | 18:58 |
| devananda | adam_g: tomorrow before noon PST is also good | 18:58 |
| mtreinish | adam_g: tomorrow would be better for me. The only constraint is the qa meeting a 1pm est | 18:59 |
| adam_g | devananda, cool. tomorrow our AM may work better all around | 18:59 |
| *** harlowja is now known as harlowja_away | 18:59 | |
| Shrews | adam_g: available tomorrow as well | 19:00 |
| adam_g | great! | 19:00 |
| openstackgerrit | Doug Hellmann proposed a change to openstack-dev/devstack: Install oslo.i18n from source https://review.openstack.org/104300 | 19:00 |
| sdague | my availability is pretty iffy tomorrow, as I've got to drive to vermont at some point. So I'm not sure I can commit. | 19:00 |
| adam_g | sdague, okay, well maybe we can just collect thoughts on a pad and you can add some thoughts when you are back? | 19:01 |
| *** weshay has quit IRC | 19:01 | |
| sdague | sure | 19:02 |
| sdague | mtreinish: is the PTL though, he should have lots of thoughts :) | 19:02 |
| adam_g | thoughts will be had | 19:02 |
| *** harlowja_away is now known as harlowja | 19:03 | |
| devananda | adam_g: thanks. I'll try to be online ~8am PST | 19:05 |
| openstackrecheck | Console logs not available after 13:26s for check-grenade-dsvm 101901,4,f77b82f | 19:10 |
| *** luqas has quit IRC | 19:12 | |
| *** leseb has joined #openstack-qa | 19:14 | |
| openstackgerrit | David Kranz proposed a change to openstack/tempest: First part of moving success response checking to identity client https://review.openstack.org/104290 | 19:15 |
| *** weshay has joined #openstack-qa | 19:17 | |
| openstackgerrit | Chris Dent proposed a change to openstack/tempest: Add a check_telemetry method to javelin https://review.openstack.org/102354 | 19:19 |
| openstackgerrit | Sean Dague proposed a change to openstack/tempest: update iptables rules for more useful debugging https://review.openstack.org/104220 | 19:19 |
| *** Sukhdev has joined #openstack-qa | 19:21 | |
| openstackrecheck | Console logs not available after 13:27s for check-grenade-dsvm-partial-ncpu 102064,8,5564541 | 19:23 |
| openstackrecheck | openstack/horizon change: https://review.openstack.org/86125 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 19:30 |
| *** malini2 has quit IRC | 19:31 | |
| *** mmaglana has quit IRC | 19:33 | |
| *** dims__ has quit IRC | 19:33 | |
| *** mmaglana has joined #openstack-qa | 19:34 | |
| *** dkorn has quit IRC | 19:35 | |
| *** Obulpathi has quit IRC | 19:35 | |
| *** Obulpathi has joined #openstack-qa | 19:36 | |
| *** leseb has quit IRC | 19:38 | |
| *** leseb has joined #openstack-qa | 19:38 | |
| openstackgerrit | Kashyap Chamarthy proposed a change to openstack-infra/elastic-recheck: Add query for Ceilometer timeout - bug 1336755 https://review.openstack.org/104160 | 19:40 |
| *** leseb has quit IRC | 19:43 | |
| *** dims has joined #openstack-qa | 19:44 | |
| openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for Ceilometer timeout - bug 1336755 https://review.openstack.org/104160 | 19:46 |
| *** dims_ has joined #openstack-qa | 19:48 | |
| *** safchain has quit IRC | 19:48 | |
| openstackgerrit | A change was merged to openstack/tempest: botoclient does not honor disable_ssl_certificate_validation https://review.openstack.org/87367 | 19:49 |
| *** leseb has joined #openstack-qa | 19:49 | |
| *** dims has quit IRC | 19:49 | |
| *** dkranz has quit IRC | 19:49 | |
| kashyap | mriedem, Heya, when you get a sec later, just hit the above (104160) Ceilometer -> Nova notification bug - http://logs.openstack.org/90/102490/4/check/check-tempest-dsvm-full/2a84e03/console.html , please take a look if you think the revised query is reasonable now | 19:52 |
| * kashyap hits the hay for now | 19:52 | |
| *** openstackrecheck has quit IRC | 19:53 | |
| mriedem | kashyap: already approved it | 19:53 |
| *** openstackrecheck has joined #openstack-qa | 19:53 | |
| kashyap | Thank you. You're a bit too fast for my now hibernated brain :-) | 19:53 |
| *** dims_ has quit IRC | 19:54 | |
| *** dims has joined #openstack-qa | 19:54 | |
| *** qba73 has quit IRC | 19:58 | |
| *** dims_ has joined #openstack-qa | 19:58 | |
| *** dims has quit IRC | 20:00 | |
| *** dims_ has quit IRC | 20:05 | |
| *** dims has joined #openstack-qa | 20:06 | |
| *** harlowja is now known as harlowja_away | 20:09 | |
| *** weshay has quit IRC | 20:12 | |
| *** weshay has joined #openstack-qa | 20:12 | |
| *** Sukhdev has quit IRC | 20:13 | |
| openstackrecheck | Console logs not available after 13:27s for gate-nova-pep8 103837,3,03a9477 | 20:27 |
| openstackrecheck | openstack/keystonemiddleware change: https://review.openstack.org/104027 failed because of: gate-tempest-dsvm-postgres-full: https://bugs.launchpad.net/bugs/1336755 https://bugs.launchpad.net/bugs/1311066 | 20:28 |
| *** harlowja_away is now known as harlowja | 20:39 | |
| openstackgerrit | Greg Lucas proposed a change to openstack-dev/devstack: Stop trove services in unstack https://review.openstack.org/104338 | 20:43 |
| *** marun is now known as marun_afk | 20:43 | |
| *** Sukhdev has joined #openstack-qa | 20:50 | |
| *** krtaylor has quit IRC | 20:50 | |
| *** marun_afk is now known as marun | 20:52 | |
| openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-postgres-full 101055,6,903d253 | 20:53 |
| *** dims has quit IRC | 20:55 | |
| *** Sukhdev_ has joined #openstack-qa | 20:57 | |
| *** atiwari has quit IRC | 20:58 | |
| *** atiwari has joined #openstack-qa | 20:59 | |
| *** Sukhdev has quit IRC | 21:00 | |
| *** dims has joined #openstack-qa | 21:00 | |
| *** yfried has quit IRC | 21:00 | |
| *** Sukhdev_ has quit IRC | 21:02 | |
| *** yfried has joined #openstack-qa | 21:03 | |
| *** adalbas has quit IRC | 21:06 | |
| *** atiwari has quit IRC | 21:06 | |
| *** dims_ has joined #openstack-qa | 21:13 | |
| *** Obulpathi has quit IRC | 21:14 | |
| *** mmaglana has quit IRC | 21:14 | |
| *** dims has quit IRC | 21:15 | |
| *** dims_ has quit IRC | 21:18 | |
| *** mmaglana has joined #openstack-qa | 21:21 | |
| *** ajo|mac has quit IRC | 21:21 | |
| openstackgerrit | Vadim Rovachev proposed a change to openstack/tempest: Add glance notifications tests for ceilometer https://review.openstack.org/71259 | 21:24 |
| *** markmcclain has quit IRC | 21:25 | |
| *** mmaglana has quit IRC | 21:26 | |
| *** dims has joined #openstack-qa | 21:31 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 103226,4,f5d01c9 | 21:34 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/99231 failed because of: gate-grenade-dsvm: unrecognized error, gate-grenade-dsvm-partial-ncpu: https://bugs.launchpad.net/bugs/1298472 | 21:35 |
| *** leseb has quit IRC | 21:49 | |
| *** leseb has joined #openstack-qa | 21:50 | |
| *** sweston has joined #openstack-qa | 21:52 | |
| *** leseb has quit IRC | 21:54 | |
| *** jamielennox|away is now known as jamielennox | 21:55 | |
| openstackrecheck | Console logs not available after 13:23s for gate-tempest-dsvm-neutron 103731,2,ce356fc | 21:55 |
| *** gfidente has quit IRC | 21:59 | |
| *** mmaglana has joined #openstack-qa | 22:06 | |
| *** rodrigods_ has joined #openstack-qa | 22:08 | |
| *** gszasz has joined #openstack-qa | 22:17 | |
| *** mlavalle has quit IRC | 22:23 | |
| *** rodrigods_ has quit IRC | 22:36 | |
| *** jgrimm has quit IRC | 22:37 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 104343,1,17613c2 | 22:45 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/103835 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 22:45 |
| *** rbak_ has quit IRC | 22:52 | |
| *** krtaylor has joined #openstack-qa | 22:56 | |
| *** rodrigods_ has joined #openstack-qa | 23:08 | |
| *** rodrigods_ has quit IRC | 23:11 | |
| *** tosky has quit IRC | 23:18 | |
| *** Sukhdev has joined #openstack-qa | 23:22 | |
| *** aimon has quit IRC | 23:24 | |
| *** sabeen2 has quit IRC | 23:26 | |
| *** gszasz has quit IRC | 23:29 | |
| *** atiwari has joined #openstack-qa | 23:42 | |
| *** oomichi has joined #openstack-qa | 23:45 | |
| *** gmann has joined #openstack-qa | 23:46 | |
| *** nati_uen_ has quit IRC | 23:49 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/104304 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 23:54 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/tempest: Updated from global requirements https://review.openstack.org/103036 | 23:59 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!