| *** shashankhegde has joined #openstack-qa | 00:14 | |
| *** shashankhegde has quit IRC | 00:20 | |
| *** rbak has joined #openstack-qa | 00:22 | |
| *** rbak has quit IRC | 00:25 | |
| *** pcrews has joined #openstack-qa | 00:25 | |
| *** dmorita has joined #openstack-qa | 00:29 | |
| *** dims_ has quit IRC | 00:34 | |
| *** mahito has joined #openstack-qa | 00:35 | |
| *** Poornima has joined #openstack-qa | 00:37 | |
| *** salv-orlando has joined #openstack-qa | 00:43 | |
| *** salv-orlando has quit IRC | 00:50 | |
| *** hichihara has joined #openstack-qa | 00:51 | |
| *** sputnik13 has quit IRC | 00:53 | |
| *** jaypipes has joined #openstack-qa | 00:56 | |
| *** ChanServ sets mode: +o jaypipes | 00:56 | |
| *** haigang has joined #openstack-qa | 00:56 | |
| *** rbak has joined #openstack-qa | 00:59 | |
| *** gamado has quit IRC | 01:04 | |
| *** rbak has quit IRC | 01:06 | |
| *** achanda has joined #openstack-qa | 01:09 | |
| *** jecarey has quit IRC | 01:14 | |
| *** jecarey has joined #openstack-qa | 01:15 | |
| *** vozcelik has joined #openstack-qa | 01:15 | |
| *** Trozz_ has quit IRC | 01:16 | |
| *** Trozz has joined #openstack-qa | 01:17 | |
| *** haigang has quit IRC | 01:18 | |
| *** fungi has quit IRC | 01:18 | |
| *** ttx has quit IRC | 01:18 | |
| *** fungi has joined #openstack-qa | 01:21 | |
| *** ttx has joined #openstack-qa | 01:22 | |
| *** Yogi has joined #openstack-qa | 01:25 | |
| *** yamamoto_ has joined #openstack-qa | 01:30 | |
| *** yamamoto_ has quit IRC | 01:32 | |
| *** rbak has joined #openstack-qa | 01:32 | |
| *** yamamoto has quit IRC | 01:33 | |
| *** armax has joined #openstack-qa | 01:33 | |
| *** rbak has quit IRC | 01:35 | |
| *** jyuso1 has quit IRC | 01:36 | |
| *** haigang has joined #openstack-qa | 01:38 | |
| *** haigang has quit IRC | 01:40 | |
| *** haigang has joined #openstack-qa | 01:41 | |
| *** jyuso1 has joined #openstack-qa | 01:42 | |
| *** haigang has quit IRC | 01:49 | |
| *** sputnik13 has joined #openstack-qa | 01:52 | |
| *** haigang has joined #openstack-qa | 01:53 | |
| *** Yogi has quit IRC | 02:00 | |
| *** achanda has quit IRC | 02:02 | |
| *** rbak has joined #openstack-qa | 02:16 | |
| *** armax has quit IRC | 02:24 | |
| *** rbak has quit IRC | 02:28 | |
| *** Longgeek has quit IRC | 02:28 | |
| *** Longgeek has joined #openstack-qa | 02:33 | |
| *** sputnik13 has quit IRC | 02:36 | |
| *** salv-orlando has joined #openstack-qa | 02:46 | |
| *** Poornima has quit IRC | 02:51 | |
| *** shashankhegde has joined #openstack-qa | 02:51 | |
| *** wuhg has joined #openstack-qa | 02:52 | |
| *** Poornima has joined #openstack-qa | 02:54 | |
| *** salv-orlando has quit IRC | 02:56 | |
| *** haigang has quit IRC | 02:56 | |
| *** haigang has joined #openstack-qa | 02:57 | |
| *** salv-orlando has joined #openstack-qa | 03:02 | |
| *** salv-orlando has quit IRC | 03:04 | |
| *** woodster_ has joined #openstack-qa | 03:05 | |
| *** rbak has joined #openstack-qa | 03:10 | |
| *** Poornima has quit IRC | 03:11 | |
| *** Poornima has joined #openstack-qa | 03:12 | |
| *** rbak has quit IRC | 03:15 | |
| *** rbak has joined #openstack-qa | 03:16 | |
| *** rbak has quit IRC | 03:16 | |
| *** armax has joined #openstack-qa | 03:21 | |
| *** armax has quit IRC | 03:23 | |
| *** annashen has joined #openstack-qa | 03:48 | |
| *** annashen has quit IRC | 04:10 | |
| *** Longgeek has quit IRC | 04:17 | |
| *** Longgeek has joined #openstack-qa | 04:18 | |
| *** davideagnello has joined #openstack-qa | 04:24 | |
| *** davideagnello has quit IRC | 04:28 | |
| *** yamamoto has joined #openstack-qa | 04:31 | |
| *** Madan has joined #openstack-qa | 04:42 | |
| *** coolsvap|afk is now known as coolsvap | 04:47 | |
| *** raies has joined #openstack-qa | 04:50 | |
| raies | mtreinish: hi | 04:50 |
|---|---|---|
| *** yfried__ has quit IRC | 04:56 | |
| raies | dkranz: hi | 05:00 |
| *** gamado has joined #openstack-qa | 05:10 | |
| *** Trozz has quit IRC | 05:11 | |
| *** Trozz has joined #openstack-qa | 05:12 | |
| *** annashen has joined #openstack-qa | 05:29 | |
| *** subscope_ has joined #openstack-qa | 05:30 | |
| *** subscope_ has quit IRC | 05:31 | |
| *** BharatK has joined #openstack-qa | 05:31 | |
| *** achanda has joined #openstack-qa | 05:39 | |
| *** annashen has quit IRC | 05:42 | |
| *** ajo has joined #openstack-qa | 05:43 | |
| *** masayukig has quit IRC | 05:44 | |
| *** sweston has quit IRC | 05:44 | |
| *** early` has quit IRC | 05:44 | |
| *** bkopilov has quit IRC | 05:44 | |
| *** sweston has joined #openstack-qa | 05:45 | |
| *** masayukig has joined #openstack-qa | 05:46 | |
| *** salv-orlando has joined #openstack-qa | 05:46 | |
| *** early has joined #openstack-qa | 05:47 | |
| *** annashen has joined #openstack-qa | 05:49 | |
| *** jhenner has joined #openstack-qa | 05:50 | |
| *** annashen has quit IRC | 05:53 | |
| *** salv-orlando has quit IRC | 05:54 | |
| *** apevec has joined #openstack-qa | 06:01 | |
| *** bkopilov has joined #openstack-qa | 06:03 | |
| *** kiran-r has joined #openstack-qa | 06:03 | |
| *** Poornima has quit IRC | 06:04 | |
| *** Longgeek has quit IRC | 06:07 | |
| *** rdekel has joined #openstack-qa | 06:07 | |
| *** Longgeek has joined #openstack-qa | 06:09 | |
| *** itzikb has joined #openstack-qa | 06:10 | |
| *** rdekel has joined #openstack-qa | 06:10 | |
| *** Poornima has joined #openstack-qa | 06:12 | |
| *** k4n0 has joined #openstack-qa | 06:14 | |
| *** SridharG has joined #openstack-qa | 06:17 | |
| *** haigang has quit IRC | 06:21 | |
| *** masayukig has quit IRC | 06:21 | |
| *** afazekas has joined #openstack-qa | 06:21 | |
| *** masayukig has joined #openstack-qa | 06:23 | |
| *** BadCub_Hiding has quit IRC | 06:24 | |
| *** e0ne has joined #openstack-qa | 06:24 | |
| *** BadCub_Hiding has joined #openstack-qa | 06:25 | |
| *** e0ne has quit IRC | 06:28 | |
| *** Sukhdev has joined #openstack-qa | 06:29 | |
| kiran-r | How to use verify_tempest_config.py?? | 06:30 |
| *** neeti has joined #openstack-qa | 06:32 | |
| *** Longgeek_ has joined #openstack-qa | 06:34 | |
| *** Sukhdev has quit IRC | 06:34 | |
| *** annashen has joined #openstack-qa | 06:35 | |
| *** yfried__ has joined #openstack-qa | 06:36 | |
| *** Longgeek has quit IRC | 06:37 | |
| *** woodster_ has quit IRC | 06:40 | |
| *** salv-orlando has joined #openstack-qa | 06:42 | |
| *** salv-orlando has quit IRC | 06:44 | |
| *** annashen has quit IRC | 06:45 | |
| *** ttx has quit IRC | 06:47 | |
| *** ttx has joined #openstack-qa | 06:47 | |
| *** SridharG has left #openstack-qa | 06:52 | |
| *** Bsony has joined #openstack-qa | 06:56 | |
| *** anshul has joined #openstack-qa | 06:59 | |
| *** safchain has joined #openstack-qa | 07:03 | |
| *** yog__ has joined #openstack-qa | 07:06 | |
| *** mpaolino has joined #openstack-qa | 07:08 | |
| *** shardy_ has joined #openstack-qa | 07:09 | |
| *** aopincar has joined #openstack-qa | 07:10 | |
| *** gamado has quit IRC | 07:11 | |
| *** shardy has quit IRC | 07:11 | |
| *** gema has joined #openstack-qa | 07:12 | |
| *** gema has quit IRC | 07:12 | |
| *** yog__ has quit IRC | 07:13 | |
| *** gema has joined #openstack-qa | 07:13 | |
| *** shardy_ has quit IRC | 07:14 | |
| *** shardy has joined #openstack-qa | 07:15 | |
| *** freyes has quit IRC | 07:16 | |
| *** yog__ has joined #openstack-qa | 07:17 | |
| *** freyes has joined #openstack-qa | 07:20 | |
| *** Poornima has quit IRC | 07:21 | |
| *** Poornima has joined #openstack-qa | 07:22 | |
| *** SridharG has joined #openstack-qa | 07:26 | |
| *** ibiris_away is now known as ibiris | 07:31 | |
| *** tkammer has joined #openstack-qa | 07:34 | |
| *** salv-orlando has joined #openstack-qa | 07:36 | |
| *** gillliard is now known as gilliard | 07:39 | |
| *** Poornima has quit IRC | 07:39 | |
| *** jlanoux has joined #openstack-qa | 07:40 | |
| *** Bsony has quit IRC | 07:46 | |
| *** Longgeek_ has quit IRC | 07:54 | |
| *** vaidy_ril has quit IRC | 07:58 | |
| *** freyes has quit IRC | 07:59 | |
| *** vaidy has joined #openstack-qa | 07:59 | |
| *** freyes has joined #openstack-qa | 08:01 | |
| *** Longgeek has joined #openstack-qa | 08:03 | |
| *** shashankhegde has quit IRC | 08:03 | |
| *** flashgordon has quit IRC | 08:06 | |
| *** openstackstatus has joined #openstack-qa | 08:06 | |
| *** ChanServ sets mode: +v openstackstatus | 08:06 | |
| -openstackstatus- NOTICE: Restarting gerrit because it stopped sending events (ETA 15 mins) | 08:10 | |
| *** sergsh has joined #openstack-qa | 08:11 | |
| *** jordanP has joined #openstack-qa | 08:12 | |
| *** openstackgerrit has quit IRC | 08:13 | |
| *** achanda has quit IRC | 08:16 | |
| *** openstackgerrit has joined #openstack-qa | 08:16 | |
| *** freyes has quit IRC | 08:18 | |
| *** vponomaryov has quit IRC | 08:21 | |
| *** freyes has joined #openstack-qa | 08:22 | |
| openstackgerrit | Abhishek Shrivastava proposed openstack-dev/devstack: Additional Cinder backend: cloudbyte https://review.openstack.org/177677 | 08:22 |
| *** vponomaryov has joined #openstack-qa | 08:23 | |
| *** yog__ has quit IRC | 08:32 | |
| *** freyes has quit IRC | 08:32 | |
| *** shardy_ has joined #openstack-qa | 08:33 | |
| *** shardy has quit IRC | 08:34 | |
| *** freyes has joined #openstack-qa | 08:36 | |
| *** Poornima has joined #openstack-qa | 08:37 | |
| *** shardy_ has quit IRC | 08:38 | |
| *** shardy has joined #openstack-qa | 08:39 | |
| *** yog__ has joined #openstack-qa | 08:46 | |
| *** tosky has joined #openstack-qa | 08:47 | |
| *** gfidente has joined #openstack-qa | 08:50 | |
| *** gfidente has quit IRC | 08:50 | |
| *** gfidente has joined #openstack-qa | 08:50 | |
| *** aix has joined #openstack-qa | 08:58 | |
| *** achanda has joined #openstack-qa | 08:59 | |
| *** e0ne has joined #openstack-qa | 08:59 | |
| *** mahito has quit IRC | 09:00 | |
| *** flashgordon has joined #openstack-qa | 09:06 | |
| *** e0ne has quit IRC | 09:10 | |
| openstackgerrit | garyk proposed openstack-dev/devstack: VMware: add support for simple DVS https://review.openstack.org/177607 | 09:13 |
| *** e0ne has joined #openstack-qa | 09:13 | |
| openstackgerrit | Merged openstack/tempest: Allow SSH instance with username and password in scenario cases https://review.openstack.org/168253 | 09:27 |
| *** dmellado has quit IRC | 09:28 | |
| *** sks has joined #openstack-qa | 09:29 | |
| *** gszasz has joined #openstack-qa | 09:32 | |
| *** Longgeek has quit IRC | 09:33 | |
| *** Longgeek has joined #openstack-qa | 09:34 | |
| *** rossella_ has joined #openstack-qa | 09:34 | |
| *** yog__ has quit IRC | 09:38 | |
| *** yamamoto has quit IRC | 09:41 | |
| *** gamado has joined #openstack-qa | 09:41 | |
| *** gamado has quit IRC | 09:45 | |
| *** yog__ has joined #openstack-qa | 09:51 | |
| *** Poornima has quit IRC | 09:54 | |
| *** yog__ has quit IRC | 09:54 | |
| *** achanda has quit IRC | 09:55 | |
| *** achanda has joined #openstack-qa | 10:00 | |
| *** yfried__ is now known as yfried|afk | 10:03 | |
| openstackgerrit | Merged openstack/tempest-lib: Fix ValueError in subunit_trace https://review.openstack.org/174882 | 10:03 |
| openstackgerrit | Merged openstack/tempest-lib: Switch from using custom urlparse import logic to six https://review.openstack.org/176731 | 10:03 |
| openstackgerrit | Merged openstack/tempest: Merge server response schema into one file https://review.openstack.org/162875 | 10:04 |
| *** yfried|afk is now known as yfried__ | 10:06 | |
| *** yog__ has joined #openstack-qa | 10:08 | |
| *** sks has quit IRC | 10:08 | |
| *** achanda has quit IRC | 10:11 | |
| *** dims has joined #openstack-qa | 10:14 | |
| *** Longgeek has quit IRC | 10:14 | |
| openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron-full 163706,10,4bd3649 | 10:17 |
| *** Longgeek has joined #openstack-qa | 10:26 | |
| *** hichihara has quit IRC | 10:30 | |
| openstackrecheck | Console logs not available after 13:26s for check-grenade-dsvm-neutron 176898,1,be95963 | 10:31 |
| *** samueldmq has joined #openstack-qa | 10:32 | |
| *** Yogi has joined #openstack-qa | 10:57 | |
| *** yamamoto has joined #openstack-qa | 10:59 | |
| *** gszasz has quit IRC | 10:59 | |
| *** gszasz has joined #openstack-qa | 10:59 | |
| *** Poornima has joined #openstack-qa | 11:01 | |
| *** yamamoto has quit IRC | 11:02 | |
| *** yamamoto has joined #openstack-qa | 11:02 | |
| *** tkammer has quit IRC | 11:02 | |
| *** sks has joined #openstack-qa | 11:03 | |
| *** afazekas has quit IRC | 11:05 | |
| *** aopincar has quit IRC | 11:05 | |
| *** yfried has joined #openstack-qa | 11:06 | |
| *** yfried__ has quit IRC | 11:07 | |
| *** e0ne is now known as e0ne_ | 11:07 | |
| *** e0ne_ is now known as e0ne | 11:07 | |
| *** Yogi has quit IRC | 11:11 | |
| *** marcusvrn has joined #openstack-qa | 11:11 | |
| *** marcusvrn1 has joined #openstack-qa | 11:12 | |
| *** marcusvrn has quit IRC | 11:15 | |
| *** tkammer has joined #openstack-qa | 11:16 | |
| *** safchain has quit IRC | 11:18 | |
| *** e0ne is now known as e0ne_ | 11:19 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/176234 failed because of: gate-tempest-dsvm-neutron: https://bugs.launchpad.net/bugs/1323658 | 11:19 |
| openstack | Launchpad bug 1323658 in OpenStack Compute (nova) "Nova resize/restart results in guest ending up in inconsistent state with Neutron" [Medium,Confirmed] | 11:19 |
| *** aopincar has joined #openstack-qa | 11:20 | |
| *** afazekas has joined #openstack-qa | 11:21 | |
| *** Poornima has quit IRC | 11:22 | |
| *** cdent has joined #openstack-qa | 11:23 | |
| *** salv-orlando has quit IRC | 11:23 | |
| *** dkranz has quit IRC | 11:25 | |
| *** dkranz has joined #openstack-qa | 11:25 | |
| *** coolsvap is now known as coolsvap|afk | 11:27 | |
| *** e0ne_ has quit IRC | 11:29 | |
| *** mpaolino has quit IRC | 11:30 | |
| *** safchain has joined #openstack-qa | 11:30 | |
| *** eantyshev has joined #openstack-qa | 11:34 | |
| eantyshev | Hi! I need a reviewer on https://review.openstack.org/#/c/173875/ | 11:37 |
| *** salv-orlando has joined #openstack-qa | 11:45 | |
| *** dkranz has quit IRC | 11:47 | |
| *** mwagner_lap has quit IRC | 11:48 | |
| *** chmouel has quit IRC | 11:48 | |
| *** gamado has joined #openstack-qa | 11:48 | |
| *** dmorita has quit IRC | 11:49 | |
| *** dkranz has joined #openstack-qa | 11:49 | |
| *** chmouel has joined #openstack-qa | 11:50 | |
| *** afazekas has quit IRC | 11:54 | |
| *** dkranz has quit IRC | 11:55 | |
| *** tkammer has quit IRC | 11:55 | |
| *** dkranz has joined #openstack-qa | 11:56 | |
| *** chmouel_ has joined #openstack-qa | 11:57 | |
| *** chmouel has quit IRC | 11:57 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/176871 failed gate-tempest-dsvm-postgres-full, gate-tempest-dsvm-neutron-large-ops in the gate queue with an unrecognized error | 11:57 |
| *** sileht has quit IRC | 11:59 | |
| *** e0ne has joined #openstack-qa | 11:59 | |
| *** Longgeek has quit IRC | 12:00 | |
| *** aopincar has quit IRC | 12:01 | |
| *** woodster_ has joined #openstack-qa | 12:01 | |
| *** yfried_ has joined #openstack-qa | 12:03 | |
| *** Madan has quit IRC | 12:03 | |
| kiran-r | Hello! | 12:05 |
| kiran-r | how to verify the tempest.conf file? | 12:05 |
| *** Longgeek has joined #openstack-qa | 12:05 | |
| kiran-r | Would the tempest/cmd/verify_tempest_config.py work.? | 12:05 |
| *** openstackgerrit has quit IRC | 12:06 | |
| *** yfried has quit IRC | 12:06 | |
| *** openstackgerrit has joined #openstack-qa | 12:06 | |
| *** afazekas has joined #openstack-qa | 12:07 | |
| *** tkammer has joined #openstack-qa | 12:08 | |
| *** sileht has joined #openstack-qa | 12:09 | |
| *** dkranz has quit IRC | 12:09 | |
| *** dkranz has joined #openstack-qa | 12:11 | |
| samueldmq | kiran-r, hi, you want to know where tempest.conf is located, is it ? | 12:11 |
| kiran-r | samueldmq: Hi! | 12:12 |
| kiran-r | samueldmq: I am pretty familiar with tempest. :) I wanted to know if there is a script to verify if my tempest.conf is properly configured. | 12:13 |
| *** weshay has joined #openstack-qa | 12:13 | |
| samueldmq | kiran-r, ah k :) | 12:13 |
| openstackgerrit | Sean Dague proposed openstack-dev/grenade: clean up upgrade scripts https://review.openstack.org/177334 | 12:13 |
| samueldmq | kiran-r, hmm, so maybe I have a question yo you then :p | 12:13 |
| samueldmq | kiran-r, are you familiar with tempest + identity v3 ? | 12:13 |
| kiran-r | samueldmq: Go ahead! =) | 12:14 |
| *** aopincar has joined #openstack-qa | 12:14 | |
| samueldmq | kiran-r, in fact I 'd like to check with you ... | 12:14 |
| samueldmq | kiran-r, to get it working with identity v3 auth I am doing : | 12:14 |
| samueldmq | kiran-r, auth_version = v3 and | 12:15 |
| samueldmq | kiran-r, api_v3 = true | 12:15 |
| samueldmq | kiran-r, that's all, right ? | 12:15 |
| kiran-r | yes | 12:15 |
| samueldmq | kiran-r, great! | 12:16 |
| kiran-r | samueldmq: could you paste the identity subsection? | 12:16 |
| samueldmq | kiran-r, I am using the default from devstack, just changing this above .. | 12:16 |
| samueldmq | kiran-r, but let me copy it | 12:16 |
| openstackgerrit | Julien Danjou proposed openstack-dev/hacking: tests: allow to run the same test for different code https://review.openstack.org/152108 | 12:16 |
| kiran-r | samueldmq: cool! | 12:17 |
| *** mpaolino has joined #openstack-qa | 12:18 | |
| *** mpaolino has quit IRC | 12:18 | |
| samueldmq | kiran-r, http://paste.openstack.org/show/208215/ | 12:18 |
| andreaf | samueldmq, kiran-r: there are n-net and neutron identity v3 jobs defined in tempest experimental queue, you can also check the generated tempest.conf for them, e.g. http://logs.openstack.org/81/153681/40/experimental/check-tempest-dsvm-keystonev3-full/a7526e2/logs/tempest_conf.txt.gz | 12:19 |
| samueldmq | andreaf, hi, I have been looking for you o/ | 12:20 |
| samueldmq | andreaf, yes I needed that ! ^ | 12:20 |
| andreaf | samueldmq: hi - I was on holidays for a week :) | 12:20 |
| samueldmq | andreaf, I remember to have talked to you days ago and you said me there were identity v3 experimental jobs anywhere | 12:20 |
| samueldmq | andreaf, and I thought it was fungi that had talked about this with me ^ | 12:21 |
| samueldmq | :p | 12:21 |
| samueldmq | andreaf, oh, this keeps me sane | 12:21 |
| samueldmq | morganfainberg, ^ | 12:21 |
| kiran-r | samueldmq: auth_version = v3 | 12:21 |
| kiran-r | samueldmq: in identity. | 12:21 |
| samueldmq | kiran-r, yeah thx | 12:22 |
| kiran-r | samueldmq: rest of them is good to go. :) | 12:22 |
| samueldmq | andreaf, I will let keystone folks know | 12:22 |
| kiran-r | samueldmq: welcome. | 12:22 |
| *** dkranz has quit IRC | 12:22 | |
| *** bknudson has quit IRC | 12:31 | |
| *** chmouel_ is now known as chmouel | 12:33 | |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from negative tests https://review.openstack.org/177745 | 12:35 |
| *** rfolco has joined #openstack-qa | 12:35 | |
| *** aopincar has quit IRC | 12:36 | |
| *** toure has quit IRC | 12:36 | |
| *** Longgeek has quit IRC | 12:36 | |
| *** openstackgerrit has quit IRC | 12:37 | |
| *** openstackgerrit has joined #openstack-qa | 12:37 | |
| *** Longgeek has joined #openstack-qa | 12:39 | |
| *** rossella_s has quit IRC | 12:44 | |
| *** rossella_ has quit IRC | 12:44 | |
| *** rossella_s has joined #openstack-qa | 12:44 | |
| *** miqui has joined #openstack-qa | 12:45 | |
| *** jecarey has quit IRC | 12:46 | |
| *** BharatK has quit IRC | 12:47 | |
| *** dpaterson has joined #openstack-qa | 12:48 | |
| openstackgerrit | Sergey Slipushenko proposed openstack/tempest: Test UUID duplication fix https://review.openstack.org/165921 | 12:49 |
| *** sileht has quit IRC | 12:52 | |
| *** bknudson has joined #openstack-qa | 12:56 | |
| samueldmq | andreaf, ping - I am back to ask something | 12:57 |
| samueldmq | andreaf, so you run all the tempest tests using identity v3 with that experimental job? | 12:58 |
| andreaf | yes | 12:59 |
| andreaf | samueldmq: just to be clear, we run identity v2 tests - using v2, identity v3 tests using v3 and all other tests using v3 tokens | 12:59 |
| andreaf | samueldmq: it would be enough to set api_v2 to disabled in tempest.conf to stop the identity v2 tests from running | 13:00 |
| samueldmq | andreaf, oh great! that's what I was going to do | 13:00 |
| samueldmq | andreaf, run tests for the services using v3 only | 13:01 |
| samueldmq | andreaf, and that works right now ? | 13:01 |
| andreaf | samueldmq: yes | 13:01 |
| samueldmq | andreaf, oh o/ | 13:01 |
| samueldmq | samueldmq, how can I test your gate job ? | 13:01 |
| *** yog__ has quit IRC | 13:01 | |
| samueldmq | andreaf, ^ | 13:02 |
| samueldmq | I was talking to myself ... | 13:02 |
| andreaf | the v3 jobs are executed nightly | 13:02 |
| andreaf | and they are in the experimental of tempest | 13:02 |
| andreaf | so if you have a tempest change you can leave a comment "check experimental" | 13:03 |
| andreaf | samueldmq: ^^ | 13:03 |
| samueldmq | andreaf, a tempest change or any change in gferrit ? | 13:03 |
| andreaf | samueldmq: I imagine you will be working on devstack changes mainly, so you may want to add those into devstack experimental as well | 13:03 |
| samueldmq | andreaf, ah ok, that's right | 13:04 |
| samueldmq | andreaf, I want to completely disable keystone v2 and then run tempest tests | 13:04 |
| samueldmq | andreaf, to ensure services are not using the v2 api | 13:04 |
| andreaf | samueldmq: right, so you may want to add the two v3 jobs in http://git.openstack.org/cgit/openstack-infra/project-config/tree/zuul/layout.yaml#n3451 | 13:05 |
| andreaf | samueldmq: so in your devstack change (or stack of changes) were you disable v2, you can then do "check experimental" and run the v3 job | 13:06 |
| samueldmq | andreaf, hmm .. why your job is not listed there ? | 13:07 |
| andreaf | samueldmq: you need to make your disabling of v2 subject to the DEVSTACK_GATE_KEYSTONE_V3 flag though, or else all other tests will fail | 13:07 |
| *** yog__ has joined #openstack-qa | 13:07 | |
| samueldmq | andreaf, so I got it, you specify experimental jobs per projects, and then run check experimental in that specific project changes | 13:07 |
| andreaf | samueldmq: yes the experimental pipeline is triggered by leaving a "check experimental" comment in gerrit on a change | 13:08 |
| andreaf | samueldmq: so you can basically run a set of jobs on demand | 13:08 |
| samueldmq | andreaf, nice .. thanks for this, I am not familiar with infra yet | 13:08 |
| *** Yogi has joined #openstack-qa | 13:14 | |
| *** dustins has joined #openstack-qa | 13:14 | |
| samueldmq | andreaf, do you have the link of the gerrit change you made that job in project-config ? | 13:15 |
| samueldmq | andreaf, I wanted to see to write my own | 13:16 |
| *** dims has quit IRC | 13:17 | |
| *** dims has joined #openstack-qa | 13:18 | |
| andreaf | samueldmq: https://review.openstack.org/#/c/168113/ | 13:18 |
| samueldmq | andreaf, thanks | 13:19 |
| *** yamahata has quit IRC | 13:21 | |
| *** openstackgerrit has quit IRC | 13:21 | |
| *** openstackgerrit has joined #openstack-qa | 13:22 | |
| *** mriedem has joined #openstack-qa | 13:29 | |
| *** marcusvrn1 has quit IRC | 13:30 | |
| *** neeti has quit IRC | 13:31 | |
| *** superdan is now known as dansmith | 13:31 | |
| *** ttx has left #openstack-qa | 13:31 | |
| *** sileht has joined #openstack-qa | 13:32 | |
| *** mestery_ is now known as mestery | 13:32 | |
| *** dboik has quit IRC | 13:34 | |
| *** marcusvrn1 has joined #openstack-qa | 13:36 | |
| *** artom has joined #openstack-qa | 13:42 | |
| *** morganfainberg has quit IRC | 13:44 | |
| *** jroll has quit IRC | 13:44 | |
| *** jroll has joined #openstack-qa | 13:44 | |
| sdague | mtreinish: so... I decided to start trimming the smoke tag, though before that I decided it's time to just kill the gate tag with fire - https://review.openstack.org/#/q/status:open+project:openstack/tempest+branch:master+topic:smoke,n,z | 13:46 |
| sdague | that's probably worth fast approve if you are up to it | 13:46 |
| *** sergsh has quit IRC | 13:46 | |
| *** morganfainberg has joined #openstack-qa | 13:47 | |
| -openstackstatus- NOTICE: gerrit has been restarted to clear a problem with its event stream. change events between 13:09 and 13:36 utc should be rechecked or have approval votes reapplied as needed to trigger jobs | 13:47 | |
| *** jecarey has joined #openstack-qa | 13:49 | |
| mtreinish | sdague: I +2d them, I'll +A (unless someone beats me to it) when it gets the all clear from jenkins | 13:52 |
| *** aopincar has joined #openstack-qa | 13:54 | |
| *** mkoderer has quit IRC | 13:55 | |
| *** dboik has joined #openstack-qa | 13:55 | |
| *** tochi has quit IRC | 13:55 | |
| *** mwagner_lap has joined #openstack-qa | 13:57 | |
| *** mpaolino has joined #openstack-qa | 13:58 | |
| mtreinish | afazekas: on 176787 did you have a suggestion for a raise syntax that works on both py2 and 3? | 13:58 |
| *** mkoderer has joined #openstack-qa | 13:58 | |
| *** rossella_s has quit IRC | 13:59 | |
| *** anshul has quit IRC | 14:01 | |
| *** ibiris is now known as ibiris_away | 14:01 | |
| *** ibiris_away is now known as ibiris | 14:03 | |
| *** sergsh has joined #openstack-qa | 14:04 | |
| *** toure has joined #openstack-qa | 14:04 | |
| *** dkranz has joined #openstack-qa | 14:06 | |
| *** paul_glass has joined #openstack-qa | 14:06 | |
| afazekas | mtreinish: Not yet, can you put that one to the end of the series or to outside to the series ? | 14:06 |
| *** paul_glass is now known as Guest31161 | 14:06 | |
| mtreinish | afazekas: sure I can reorder the patches | 14:07 |
| mtreinish | one sec | 14:07 |
| *** Guest31161 has quit IRC | 14:07 | |
| *** pglass has joined #openstack-qa | 14:08 | |
| *** pglass is now known as Guest59146 | 14:08 | |
| *** apevec has quit IRC | 14:08 | |
| *** marcusvrn1 has quit IRC | 14:09 | |
| afazekas | mtreinish, six.reraise ?? | 14:10 |
| mtreinish | afazekas: hmm, yeah that might do it. I'll rework the branch after the approved ones go through the gate and give that a try | 14:10 |
| *** rbak has joined #openstack-qa | 14:14 | |
| *** dmellado has joined #openstack-qa | 14:14 | |
| *** vozcelik has quit IRC | 14:15 | |
| openstackgerrit | Merged openstack/tempest: Switch all uses of StringIO to use it from six https://review.openstack.org/176783 | 14:15 |
| andreaf | hogepodge: hi - I have a question regarding some of the tests in the defcore list | 14:16 |
| *** mtanino has joined #openstack-qa | 14:17 | |
| *** rbak_ has joined #openstack-qa | 14:17 | |
| openstackgerrit | Merged openstack/tempest: Add a paragraph on the use of tempest_roles option https://review.openstack.org/174675 | 14:17 |
| openstackgerrit | Merged openstack/tempest: Fix non-locking test accounts doc section https://review.openstack.org/176026 | 14:17 |
| openstackgerrit | Merged openstack/tempest: Cleanup some details about the accounts files https://review.openstack.org/177274 | 14:18 |
| *** rbak has quit IRC | 14:19 | |
| *** BadCub_Hiding is now known as BadCub | 14:19 | |
| andreaf | mtreinish: do you know what's going on with this change: https://review.openstack.org/#/c/153681/ ? | 14:20 |
| andreaf | mtreinish: the gerrit rebase fails, but a local rebase has no merge conflict | 14:20 |
| *** eantyshev has quit IRC | 14:22 | |
| *** salv-orl_ has joined #openstack-qa | 14:22 | |
| openstackgerrit | Mathieu Rohon proposed openstack-dev/devstack: Wait after restarting rabbitmq https://review.openstack.org/177785 | 14:23 |
| mtreinish | andreaf: it could just be a jgit screw up | 14:23 |
| mtreinish | I've seen that happen before | 14:23 |
| *** salv-orlando has quit IRC | 14:23 | |
| andreaf | mtreinish: so I need to rebase and re upload? | 14:24 |
| mtreinish | andreaf: I think so | 14:24 |
| mtreinish | if you do that I can just fast approve it | 14:24 |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Initial class creds creation in test base class https://review.openstack.org/153681 | 14:25 |
| *** marcusvrn has joined #openstack-qa | 14:25 | |
| *** ChuckC_ has joined #openstack-qa | 14:25 | |
| *** ChuckC has quit IRC | 14:26 | |
| andreaf | mtreinish: ^^^ | 14:26 |
| openstackgerrit | Andrea Frittoli proposed openstack/tempest: Remove unused methods https://review.openstack.org/168366 | 14:26 |
| mtreinish | andreaf: heh, so much for a nice round rev number :) | 14:26 |
| andreaf | mtreinish: I was thinking the same :D | 14:27 |
| *** ChuckC_ has quit IRC | 14:30 | |
| hogepodge | andreaf ok | 14:30 |
| *** sputnik13 has joined #openstack-qa | 14:30 | |
| mtreinish | andreaf: also, when you get a sec I thought you might want to play with: https://review.openstack.org/#/q/status:open+project:openstack-infra/subunit2sql+branch:master+topic:176114,n,z | 14:31 |
| andreaf | hogepodge: some of the tests in http://git.openstack.org/cgit/openstack/defcore/tree/2015.04/2015.04.required.txt require run_ssh to be turned of, so they are usually skipped in the gate | 14:31 |
| hogepodge | andreaf: we've marked those as flagged | 14:32 |
| andreaf | hogepodge: as run_ssh is disabled there by default | 14:32 |
| *** SridharG has left #openstack-qa | 14:32 | |
| openstackgerrit | Merged openstack/tempest: Adjust registration of ami image in boto tests https://review.openstack.org/171222 | 14:33 |
| *** apevec has joined #openstack-qa | 14:33 | |
| hogepodge | andreaf: bluebox got some of them to pass by tweaking parameters, but we're hoping to have them fixed (or better understood) relatively soon. | 14:33 |
| andreaf | hogepodge: ok cool | 14:33 |
| *** achanda has joined #openstack-qa | 14:33 | |
| andreaf | hogepodge: thanks | 14:33 |
| hogepodge | andreaf: sure | 14:33 |
| sdague | mtreinish: so on the smoke tag trim, I was going to start with removing it from all negative and admin tests | 14:34 |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/157054 failed gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error | 14:35 |
| *** weshay has quit IRC | 14:35 | |
| *** Longgeek has quit IRC | 14:35 | |
| *** Longgeek has joined #openstack-qa | 14:36 | |
| afazekas | mtreinish, BTW: looks like it is still better to use reraise even with py3 :( : 1. just very new six supports the raise_from and even ipython does not displays the chaining properly (regular python3 does) | 14:36 |
| mtreinish | sdague: I'm fine with doing that, I think there might be a couple admin tests which could be useful in smoke, but we can drop them all and those which make sense in the future | 14:37 |
| *** tsekiyama has joined #openstack-qa | 14:38 | |
| *** weshay has joined #openstack-qa | 14:39 | |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Switch all uses of urllib and HTMLParser to import from six https://review.openstack.org/176788 | 14:39 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Use six.moves.builtins to mock open in unit tests https://review.openstack.org/177425 | 14:39 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Fix raise syntax in test.py for python3 compat https://review.openstack.org/176787 | 14:39 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Fix comprehension syntax error with python3 https://review.openstack.org/176814 | 14:39 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Switch all uses of iteritems to use six instead https://review.openstack.org/176815 | 14:39 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Fix tenant isolation and unit tests with py3 https://review.openstack.org/176934 | 14:39 |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Switch use of hashing strings to unicode in accounts https://review.openstack.org/176862 | 14:39 |
| mtreinish | afazekas: :( oh well, I guess my respun version of ^^^ won't work then | 14:39 |
| *** achanda has quit IRC | 14:41 | |
| afazekas | mtreinish, the reraise can work , I had issues with raise_from | 14:41 |
| *** dmellado has quit IRC | 14:42 | |
| mtreinish | ah, ok | 14:42 |
| sdague | mtreinish: grrrr my patch series failed on pypi download | 14:46 |
| sdague | http://logs.openstack.org/59/177759/1/check/check-grenade-dsvm-partial-ncpu/652fbe0//logs/grenade.sh.txt.gz#_2015-04-27_14_23_26_309 | 14:46 |
| sdague | will recheck as soon as it pops | 14:46 |
| openstackgerrit | Dan Smith proposed openstack-dev/grenade: Make nova migrate flavor data, per Kilo->Lemming requirements https://review.openstack.org/174567 | 14:48 |
| *** zz_jgrimm is now known as jgrimm | 14:48 | |
| *** hemnafk is now known as hemna | 14:49 | |
| *** aopincar has quit IRC | 14:49 | |
| *** ibiris is now known as ibiris_away | 14:53 | |
| *** salv-orl_ has quit IRC | 14:54 | |
| sdague | mtreinish: so... do you just recheck grind tempest changes in? because patch 4 of my stack is green, all the ones in between a just borked by various races, and the number of devstack jobs means that's kind of high likelihood | 14:55 |
| *** ibiris_away is now known as ibiris | 14:56 | |
| mtreinish | sdague: yeah if that's what you have to do | 14:57 |
| sdague | so pypi mirror caught patch 1 | 14:57 |
| sdague | boot volume on juno caught patch 2 | 14:57 |
| *** k4n0 has quit IRC | 14:57 | |
| sdague | tempest.scenario.test_network_advanced_server_ops.TestNetworkAdvancedServerOps.test_server_connectivity_stop_start on icehouse also died in patch 2 | 14:58 |
| sdague | patch 3 is boot_volume on partial-cpu in master | 14:58 |
| sdague | and patch 4 is currently green | 14:59 |
| *** yfried_ is now known as yfried|afk | 15:00 | |
| *** tsekiyama has quit IRC | 15:00 | |
| *** ChuckC_ has joined #openstack-qa | 15:01 | |
| *** dkranz has quit IRC | 15:04 | |
| *** gszasz has quit IRC | 15:04 | |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from admin tests https://review.openstack.org/177810 | 15:05 |
| hogepodge | mtreinish: We're talking about this https://review.openstack.org/#/c/172605/ | 15:06 |
| *** yfried|afk is now known as yfried_ | 15:07 | |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove smoke tag from admin tests https://review.openstack.org/177810 | 15:07 |
| *** dkranz has joined #openstack-qa | 15:07 | |
| mtreinish | hogepodge: what about it? :) I haven't had a chance to take another look at it yet | 15:09 |
| *** davideagnello has joined #openstack-qa | 15:09 | |
| *** aix has quit IRC | 15:09 | |
| hogepodge | they were just asking for reviews is all | 15:10 |
| mtreinish | ah, ok. I'll try to take another look today | 15:10 |
| mtreinish | mriedem: is there already a query/bug for: http://logs.openstack.org/57/162457/2/gate/gate-tempest-dsvm-full-ceph/22b038a//logs/screen-n-cpu.txt.gz?level=TRACE#_2015-04-27_14_54_15_909 | 15:12 |
| mriedem | hmm | 15:13 |
| mriedem | ceph job huh | 15:13 |
| mriedem | jbernard: ^? | 15:13 |
| mriedem | mtreinish: https://review.openstack.org/#/c/162457/ - are we sure that's not just a change that's breaking ceph? | 15:13 |
| *** davideagnello has quit IRC | 15:13 | |
| mriedem | mtreinish: i pulled it out of the gate | 15:14 |
| *** sabeen1 has joined #openstack-qa | 15:14 | |
| mtreinish | mriedem: oh, I didn't even look at the patch I just assumed it was unrelated because it was in the gate | 15:14 |
| mtreinish | I guess that teaches me | 15:15 |
| *** afazekas has quit IRC | 15:15 | |
| *** salv-orlando has joined #openstack-qa | 15:16 | |
| *** aopincar has joined #openstack-qa | 15:16 | |
| jbernard | mriedem: one sec | 15:17 |
| mriedem | let me check logstash | 15:18 |
| jbernard | mriedem: the warnings above look concerning | 15:18 |
| *** aopincar has quit IRC | 15:18 | |
| jbernard | mriedem: but ill take a closer look, this is failing everywhere? | 15:19 |
| jbernard | mriedem: ahh, just that patch | 15:19 |
| mriedem | jbernard: not sure yet | 15:19 |
| *** rossella_s has joined #openstack-qa | 15:20 | |
| jbernard | mriedem: ok, ill try it myself and see what i can come up with | 15:20 |
| *** rdekel has quit IRC | 15:20 | |
| *** Guest59146 is now known as pglass | 15:22 | |
| mtreinish | mriedem, jbernard: it's probably just that patch | 15:22 |
| *** yfried_ has quit IRC | 15:22 | |
| *** pglass is now known as Guest35930 | 15:22 | |
| *** marun has joined #openstack-qa | 15:22 | |
| *** Guest35930 has left #openstack-qa | 15:23 | |
| jbernard | mtreinish: did ceph originally pass on that patch? | 15:23 |
| *** dboik_ has joined #openstack-qa | 15:23 | |
| *** sabeen1 has quit IRC | 15:23 | |
| *** dkranz has quit IRC | 15:23 | |
| *** artom_ has joined #openstack-qa | 15:25 | |
| *** artom has quit IRC | 15:25 | |
| *** dboik has quit IRC | 15:26 | |
| *** sabeen1 has joined #openstack-qa | 15:26 | |
| *** sks has quit IRC | 15:28 | |
| *** paul_glass has joined #openstack-qa | 15:29 | |
| *** paul_glass is now known as pglass | 15:30 | |
| *** itzikb has quit IRC | 15:31 | |
| mtreinish | jbernard: I don't think it did | 15:31 |
| *** dboik_ has quit IRC | 15:36 | |
| *** dboik has joined #openstack-qa | 15:37 | |
| *** dkranz has joined #openstack-qa | 15:37 | |
| *** sputnik13 has quit IRC | 15:37 | |
| sdague | mtreinish: do you know what the allowed_address_pair extension really does in neutron? The info in the tempest files is very lacking | 15:37 |
| *** armax has joined #openstack-qa | 15:39 | |
| *** rdekel has joined #openstack-qa | 15:39 | |
| mtreinish | sdague: off the top of my head I have no idea | 15:39 |
| sdague | ok | 15:39 |
| mtreinish | mestery, armax: ^^^ ? | 15:39 |
| mtreinish | sdague: most of those neutron tests shouldn't be smoke. I'd be fine if you wanted to push a change to remove them all from the tag | 15:40 |
| mestery | sdague: Here's a good overview of what it does from arosen: http://blog.aaronorosen.com/implementing-high-availability-instances-with-neutron-using-vrrp/ | 15:40 |
| sdague | yeh, the patch is inbound, I'm writing the commit message now | 15:40 |
| mtreinish | oh, ok | 15:40 |
| openstackgerrit | Sean Dague proposed openstack/tempest: trim network smoke tests https://review.openstack.org/177829 | 15:41 |
| sdague | the commit message has the justification for those changes | 15:41 |
| *** Swami has joined #openstack-qa | 15:41 | |
| *** flashgordon is now known as jogo | 15:42 | |
| *** annashen has joined #openstack-qa | 15:42 | |
| *** annashen has quit IRC | 15:44 | |
| *** jroll has quit IRC | 15:44 | |
| *** tosky has quit IRC | 15:46 | |
| *** jroll has joined #openstack-qa | 15:46 | |
| *** rdekel has quit IRC | 15:48 | |
| *** yamamoto has quit IRC | 15:48 | |
| *** sabeen1 has left #openstack-qa | 15:48 | |
| *** sabeen1 has joined #openstack-qa | 15:49 | |
| *** yamamoto has joined #openstack-qa | 15:49 | |
| *** bkopilov has quit IRC | 15:52 | |
| *** jroll has quit IRC | 15:53 | |
| *** jroll has joined #openstack-qa | 15:53 | |
| *** pglass has quit IRC | 15:53 | |
| *** salv-orlando has quit IRC | 15:53 | |
| *** e0ne is now known as e0ne_ | 15:53 | |
| * armax scrolls back | 15:53 | |
| *** e0ne_ is now known as e0ne | 15:54 | |
| *** yog__ has quit IRC | 15:54 | |
| *** mlavalle has joined #openstack-qa | 15:55 | |
| armax | mtreinish: in a nutshell allowed address pairs gives the user the ability to extend security groups rules to a list of IP’s+MAC's | 15:56 |
| armax | mtreinish: is that what you were wondering? | 15:56 |
| *** Sukhdev has joined #openstack-qa | 15:56 | |
| *** bkopilov has joined #openstack-qa | 15:58 | |
| *** jordanP has quit IRC | 15:59 | |
| *** afazekas has joined #openstack-qa | 16:00 | |
| openstackgerrit | Sean Dague proposed openstack/tempest: remove qos tests from volume smoke tag https://review.openstack.org/177840 | 16:00 |
| openstackgerrit | Sean Dague proposed openstack/tempest: trim compute smoke tag https://review.openstack.org/177841 | 16:00 |
| *** david-ly_ is now known as david-lyle | 16:01 | |
| *** tsekiyama has joined #openstack-qa | 16:03 | |
| openstackrecheck | openstack/nova change: https://review.openstack.org/171299 failed gate-tempest-dsvm-neutron-full in the gate queue with an unrecognized error | 16:04 |
| *** sergsh has quit IRC | 16:05 | |
| *** jlanoux has quit IRC | 16:05 | |
| *** artom_ has quit IRC | 16:05 | |
| *** dkranz has quit IRC | 16:06 | |
| *** dmellado has joined #openstack-qa | 16:08 | |
| *** salv-orlando has joined #openstack-qa | 16:09 | |
| *** ChuckC_ has quit IRC | 16:09 | |
| *** mpaolino has quit IRC | 16:09 | |
| *** salv-orl_ has joined #openstack-qa | 16:10 | |
| *** SpamapS has quit IRC | 16:12 | |
| openstackgerrit | Sean Dague proposed openstack/tempest: adjust smoke definition to not include *all* scenario https://review.openstack.org/177852 | 16:13 |
| sdague | mtreinish: that last patch... would like to know if you are good with that part of the direction as well | 16:13 |
| *** safchain has quit IRC | 16:13 | |
| *** salv-orlando has quit IRC | 16:13 | |
| *** Yogi has quit IRC | 16:14 | |
| *** Sukhdev has quit IRC | 16:15 | |
| *** sabeen2 has joined #openstack-qa | 16:15 | |
| *** sabeen1 has quit IRC | 16:15 | |
| samueldmq | andreaf, ping - you around ? | 16:16 |
| kiran-r | Hi, I am getting this error while executing git review http://paste.openstack.org/show/208539/ | 16:17 |
| sdague | oh, wait... those are *our* mirrors | 16:19 |
| *** dkranz has joined #openstack-qa | 16:19 | |
| *** rbak_ has quit IRC | 16:21 | |
| *** amotoki_ has joined #openstack-qa | 16:21 | |
| *** amotoki has quit IRC | 16:21 | |
| *** amotoki_ is now known as amotoki | 16:21 | |
| *** armax has quit IRC | 16:22 | |
| *** rbak has joined #openstack-qa | 16:23 | |
| *** armax has joined #openstack-qa | 16:23 | |
| sdague | mtreinish: it looks like one of the issues with the icehouse jobs is running out of fixed ips, perhaps just tests leaking? | 16:24 |
| *** mpaolino has joined #openstack-qa | 16:27 | |
| mtreinish | sdague: hmm maybe, do you have a link? We should be pretty good about cleaning things up now because the accounts stuff would catch leaks pretty quickly | 16:29 |
| sdague | mtreinish: http://logs.openstack.org/61/177761/1/check/check-tempest-dsvm-full-icehouse/405bb8d//logs/screen-n-cpu.txt.gz?level=TRACE#_2015-04-27_15_48_37_023 is the hint for it | 16:29 |
| sdague | one of the odd tests in that run came back with "No Hosts Available" | 16:30 |
| *** dboik_ has joined #openstack-qa | 16:30 | |
| sdague | anyway, the patches pass all the tests, in aggregate. Getting them to pass all the tests all at once is just a matter of recheck at this point | 16:31 |
| *** sabeen1 has joined #openstack-qa | 16:32 | |
| *** pglass has joined #openstack-qa | 16:32 | |
| samueldmq | andreaf, in your tempest config, don't we need to turn on api_v3 = true ? besides auth_version=v3 | 16:33 |
| samueldmq | andreaf, http://logs.openstack.org/81/153681/40/experimental/check-tempest-dsvm-keystonev3-full/a7526e2/logs/tempest_conf.txt.gz | 16:33 |
| *** dboik has quit IRC | 16:34 | |
| openstackrecheck | openstack/ceilometer change: https://review.openstack.org/177675 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 16:35 |
| mtreinish | sdague: the thing is that fixed ips are a local quota not global, so it the test class would have to be that test class leaking ips | 16:35 |
| *** sabeen2 has quit IRC | 16:35 | |
| mtreinish | but that quota is set to -1 http://logs.openstack.org/61/177761/1/check/check-tempest-dsvm-full-icehouse/405bb8d/logs/screen-n-cpu.txt.gz#_2015-04-27_15_45_54_807 | 16:35 |
| mtreinish | so my guess is that it's that bug mriedem fixed on master a while ago just never got backported | 16:35 |
| *** afazekas has quit IRC | 16:35 | |
| *** shashankhegde has joined #openstack-qa | 16:36 | |
| *** afazekas has joined #openstack-qa | 16:36 | |
| dansmith | sdague: like, woohoo: https://review.openstack.org/#/c/174567/ | 16:38 |
| mriedem | mtreinish: that bug was never actually fixed | 16:39 |
| mriedem | the fixed_ips over quota one | 16:39 |
| mtreinish | oh, it disappeared for the most part so I assumed we had a fix | 16:39 |
| mriedem | mtreinish: i added some more debug logging when we hit the case though | 16:39 |
| mtreinish | heh, although not on icehouse :) | 16:40 |
| mriedem | no, would be kilo | 16:40 |
| * mtreinish wonders when icehouse actually dies | 16:40 | |
| mriedem | never ever | 16:40 |
| *** armax has quit IRC | 16:49 | |
| openstackgerrit | Sean Dague proposed openstack-infra/elastic-recheck: Add query for pypi mirrors disconnecting runs https://review.openstack.org/177871 | 16:50 |
| *** Sukhdev has joined #openstack-qa | 16:51 | |
| *** afazekas has quit IRC | 16:52 | |
| mriedem | mtreinish: were you already looking at this for juno? http://logs.openstack.org/periodic-stable/periodic-glance-python27-juno/9cd2c28/console.html#_2015-04-27_06_16_02_554 | 16:56 |
| mtreinish | mriedem: not yet | 16:58 |
| mtreinish | mriedem: although there were 3 releases on the ML today, so... | 16:58 |
| *** erlon has joined #openstack-qa | 16:58 | |
| mriedem | was actually an oslo release on 4/21 | 16:59 |
| mtreinish | oh, heh go figure | 16:59 |
| *** afazekas has joined #openstack-qa | 17:00 | |
| mriedem | fun | 17:01 |
| mriedem | http://logs.openstack.org/18/173118/2/check/gate-glance-python27/c0368f7/console.html.gz#_2015-04-22_13_05_25_379 | 17:01 |
| openstackgerrit | Sean Dague proposed openstack/tempest: adjust smoke definition to not include *all* scenario https://review.openstack.org/177852 | 17:01 |
| mriedem | pycadf 0.9.0 getting into stable/juno | 17:01 |
| mriedem | although capped on <0.7.0 | 17:01 |
| mriedem | https://github.com/openstack/requirements/blob/stable/juno/global-requirements.txt#L91 | 17:01 |
| *** Nic has joined #openstack-qa | 17:02 | |
| *** patrickeast has joined #openstack-qa | 17:02 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/177306 failed gate-requirements-integration-dsvm in the gate queue with an unrecognized error | 17:02 |
| sdague | mtreinish: so up through - https://review.openstack.org/#/c/177829 we have tests reporting and basically working | 17:03 |
| sdague | I screwed up the top patch, that just got respun, tests coming back on some of the others over the next hour. Going to get a walk in, then see where we stand on grinding the rest through. | 17:03 |
| sdague | though, realistically you should contemplate the smoke test changes a bit, but I think this is the right trimming down we want. A few more services also need to be trimmed, but this seems like a sane starting point. | 17:04 |
| *** davideagnello has joined #openstack-qa | 17:05 | |
| *** Nic has quit IRC | 17:06 | |
| *** suneelb has joined #openstack-qa | 17:07 | |
| *** yamahata has joined #openstack-qa | 17:07 | |
| *** Yogi has joined #openstack-qa | 17:07 | |
| sdague | it should also make the neutron grenade test run in < 1 hr | 17:09 |
| dhellmann | jogo: I think we're hitting https://bugs.launchpad.net/devstack/+bug/1403284 again in some stable/kilo jobs like http://logs.openstack.org/72/177772/4/check/gate-tempest-dsvm-largeops-src-oslo.messaging/9580f1e/logs/screen-n-cpu-1.txt.gz#_2015-04-27_15_48_34_141 I see your original patch to reduce the number of services passed. Should we do that again, or do you think there's another approach to take? | 17:09 |
| openstack | Launchpad bug 1403284 in devstack "mysql (1040, 'Too many connections')" [Undecided,In progress] - Assigned to Joe Gordon (jogo) | 17:09 |
| *** wuhg has quit IRC | 17:09 | |
| jogo | dhellmann: looking | 17:09 |
| jogo | dhellmann: yeah makes sense to me | 17:10 |
| jogo | dhellmann: I can push a patch up for kilo if you want | 17:11 |
| dhellmann | jogo: that would be awesome | 17:11 |
| dhellmann | jogo: it looks like the error is in another file, so the elastic recheck pattern isn't matching | 17:11 |
| dhellmann | I'll see about fixing that | 17:11 |
| *** anshul has joined #openstack-qa | 17:14 | |
| *** Yogi has quit IRC | 17:14 | |
| openstackgerrit | Martin Pavlásek proposed openstack/tempest: Added test of default domain in Keystone https://review.openstack.org/31782 | 17:15 |
| *** shashankhegde has quit IRC | 17:16 | |
| *** EmilienM is now known as EmilienM|afk | 17:17 | |
| *** Yogi has joined #openstack-qa | 17:18 | |
| *** afazekas has quit IRC | 17:18 | |
| jogo | https://review.openstack.org/#/c/177885/ | 17:18 |
| jogo | dhellmann: ^ that should do it | 17:18 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/elastic-recheck: Update query for bug 1403284 https://review.openstack.org/177886 | 17:19 |
| openstack | bug 1403284 in OpenStack-Gate "mysql (1040, 'Too many connections')" [Undecided,Fix committed] https://launchpad.net/bugs/1403284 - Assigned to Joe Gordon (jogo) | 17:19 |
| *** kiran-r has quit IRC | 17:19 | |
| dhellmann | jogo: great, thanks! | 17:19 |
| dhellmann | jogo: can you check my query update, too? ^^ | 17:19 |
| mriedem | dhellmann: looking | 17:19 |
| dhellmann | mriedem: thanks | 17:20 |
| jogo | mriedem: beat you! | 17:20 |
| jogo | dhellmann: looks good, thanks | 17:20 |
| mriedem | jogo: well, | 17:20 |
| *** achanda has joined #openstack-qa | 17:20 | |
| *** sputnik13 has joined #openstack-qa | 17:21 | |
| mriedem | i was going to ask, why are we even filtering on those specific logs in this case? | 17:21 |
| mriedem | couldn't we jsut filter on tags:"screen"? | 17:21 |
| *** e0ne has quit IRC | 17:21 | |
| jogo | that would work too I guess yeah /// | 17:21 |
| dhellmann | mriedem: I don't know? I wasn't sure if there was some performance reason for filtering | 17:23 |
| dhellmann | I could change it? | 17:23 |
| mriedem | not a huge deal | 17:23 |
| mriedem | just next time there is a mole to whack we'll have to update this again | 17:23 |
| dhellmann | mriedem: I'll update it while I'm thinking about it | 17:24 |
| *** mpaolino has quit IRC | 17:24 | |
| mriedem | k | 17:25 |
| openstackgerrit | Doug Hellmann proposed openstack-infra/elastic-recheck: Make the query for bug 1403284 more generic https://review.openstack.org/177888 | 17:26 |
| openstack | bug 1403284 in OpenStack-Gate "mysql (1040, 'Too many connections')" [Undecided,Fix committed] https://launchpad.net/bugs/1403284 - Assigned to Joe Gordon (jogo) | 17:26 |
| dhellmann | mriedem: jogo approved the other, so here's a second patch ^^ | 17:26 |
| *** salv-orlando has joined #openstack-qa | 17:26 | |
| mriedem | got it, thanks | 17:26 |
| *** salv-orl_ has quit IRC | 17:27 | |
| *** Yogi has quit IRC | 17:29 | |
| *** mpaolino has joined #openstack-qa | 17:29 | |
| *** marun has quit IRC | 17:30 | |
| *** yfried_ has joined #openstack-qa | 17:30 | |
| *** armax has joined #openstack-qa | 17:36 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Add query for pypi mirrors disconnecting runs https://review.openstack.org/177871 | 17:37 |
| *** marun has joined #openstack-qa | 17:37 | |
| *** Yogi has joined #openstack-qa | 17:38 | |
| *** Nic has joined #openstack-qa | 17:39 | |
| *** openstackrecheck has quit IRC | 17:41 | |
| *** openstackrecheck has joined #openstack-qa | 17:41 | |
| *** shashankhegde has joined #openstack-qa | 17:41 | |
| *** amotoki has quit IRC | 17:47 | |
| *** cdent has quit IRC | 17:51 | |
| *** harlowja_away is now known as harlowja_ | 17:52 | |
| *** mpaolino has quit IRC | 17:52 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Update query for bug 1403284 https://review.openstack.org/177886 | 17:56 |
| openstack | bug 1403284 in OpenStack-Gate "mysql (1040, 'Too many connections')" [Undecided,Fix committed] https://launchpad.net/bugs/1403284 - Assigned to Joe Gordon (jogo) | 17:56 |
| mriedem | seems we should just cap requirements on stable even down to the minor version and if people really really need to adjust those for a bug fix we should then make a change to do it so we test and maybe fail only that one change rather than all of stable | 17:58 |
| mriedem | when some library releases a minor version with bad transitive dep versions | 17:58 |
| *** jogo has quit IRC | 18:03 | |
| openstackgerrit | Merged openstack-infra/elastic-recheck: Make the query for bug 1403284 more generic https://review.openstack.org/177888 | 18:04 |
| openstack | bug 1403284 in OpenStack-Gate "mysql (1040, 'Too many connections')" [Undecided,Fix committed] https://launchpad.net/bugs/1403284 - Assigned to Joe Gordon (jogo) | 18:04 |
| *** kiran-r has joined #openstack-qa | 18:04 | |
| *** EmilienM|afk is now known as EmilienM | 18:06 | |
| openstackgerrit | Merged openstack/tempest: Switch all uses of urlparse to import from six https://review.openstack.org/176784 | 18:09 |
| *** artom has joined #openstack-qa | 18:09 | |
| *** davideagnello has quit IRC | 18:10 | |
| openstackrecheck | openstack/tempest change: https://review.openstack.org/176785 failed gate-tempest-dsvm-nova-v21-full in the gate queue with an unrecognized error | 18:11 |
| mtreinish | ugh, timeout fixture: http://logs.openstack.org/85/176785/2/gate/gate-tempest-dsvm-nova-v21-full/ed842aa//console.html#_2015-04-27_17_43_43_634 | 18:11 |
| mriedem | mtreinish: lots of errors in the glance logs | 18:12 |
| *** davideagnello has joined #openstack-qa | 18:12 | |
| mtreinish | oh, it's all swifts fault... | 18:13 |
| mriedem | i don't see any errors in the swift logs | 18:13 |
| *** yamahata has quit IRC | 18:14 | |
| mtreinish | er, yeah only one of the tracebacks was swiftclient | 18:14 |
| *** openstackrecheck has quit IRC | 18:14 | |
| *** openstackrecheck has joined #openstack-qa | 18:15 | |
| mtreinish | the rest were all 404's from the glance registry client | 18:15 |
| mtreinish | what does that even do | 18:15 |
| *** rbak_ has joined #openstack-qa | 18:15 | |
| *** rbak has quit IRC | 18:15 | |
| jlk | "all swifts fault" sounds like my day so far | 18:15 |
| mriedem | i'm looking to blame some library for breaking stable/juno | 18:16 |
| *** Yogi2 has joined #openstack-qa | 18:16 | |
| mtreinish | jlk: heh, well in this case it's actually all glances fault :) | 18:16 |
| mriedem | i need an easy way to dump a dependency tree on failing dependencies at the point of failure in a job using the venv | 18:16 |
| mtreinish | mriedem: it's not in the tox logs? | 18:17 |
| mriedem | e.g. i know cliff on stable/juno in a failing job is too new for the caps, but i don't know what's pulling in the latest cliff here | 18:17 |
| *** dboik_ has quit IRC | 18:17 | |
| mriedem | oooo maybe | 18:17 |
| mriedem | :) | 18:17 |
| *** Yogi has quit IRC | 18:17 | |
| *** anshul has quit IRC | 18:17 | |
| mriedem | neutronclient | 18:18 |
| mriedem | of course | 18:18 |
| *** dboik has joined #openstack-qa | 18:18 | |
| mriedem | neutronclient 2.3.11 | 18:18 |
| jlk | geez, 15 seconds of clock drift over a weekend. Makes swift tests not happy that are time dependent. | 18:19 |
| mtreinish | jlk: ugh, we still have timestamp dependent tests? I thought we killed all of those | 18:19 |
| jlk | tempurl seems to be time reloated | 18:19 |
| jlk | since the url is only valid for a certain amount of time | 18:19 |
| jlk | tempest.api.object_storage.test_object_expiry.ObjectExpiryTest.test_get_object_after_expiry_time | 18:20 |
| *** Sukhdev has quit IRC | 18:20 | |
| mtreinish | oh, the tempurl tests, yeah I guess they want to test it actually expired | 18:20 |
| *** gfidente has quit IRC | 18:20 | |
| jlk | if your test machine time is in the future compared to the swift machine time, you end up trying to do some weird future/past thing and get a traceback | 18:20 |
| mriedem | mtreinish: Collecting cliff>=1.7.0 (from python-neutronclient<2.4.0,>=2.3.6->-r /home/jenkins/workspace/gate-heat-python27/requirements.txt (line 29)) Downloading http://pypi.IAD.openstack.org/packages/py2/c/cliff/cliff-1.12.0-py2-none-any.whl | 18:21 |
| mtreinish | jlk: in the past we had really bad tests which did things like checked timestamps in resp objects from 2 consecutive reqs | 18:21 |
| mriedem | https://github.com/openstack/python-neutronclient/blob/2.3.11/requirements.txt#L6 | 18:21 |
| jlk | mtreinish: ah. | 18:21 |
| mtreinish | jlk: oh can you pastebin that traceback? I'm wondering if we can try to account for that in the test at all | 18:21 |
| mriedem | so neutronclient 2.3.11 requires cliff>=1.7.0, and it looks like it's just pulling in cliff 1.12 | 18:22 |
| mtreinish | like I'm sure it screws up if your tz is different | 18:22 |
| jlk | y'all wouldn't happen to know the best place to go to get help figuring out why my swift is throwing 401s for tempurl ? | 18:22 |
| mtreinish | if it's failing for you with ~15sec drift | 18:22 |
| openstackgerrit | Richard Winters proposed openstack/tempest: Adding multi-node test case for ML2 driver verification https://review.openstack.org/168880 | 18:22 |
| mriedem | and we have cliff capped in stable/juno g-r but not seeing anything capping cliff in these libraries | 18:22 |
| mtreinish | jlk: I think by default syslog is where things from swift go | 18:22 |
| mriedem | do we need to cap libraries on their stable/branches? | 18:22 |
| mriedem | seems like 'yes' | 18:22 |
| mtreinish | mriedem: yes, nothing will do the g-r sync automagically like devstack does on the unit test jobs | 18:23 |
| jlk | mtreinish: https://gist.github.com/j2sol/aa5d8b705c2f22ba9fea | 18:23 |
| jlk | mtreinish: yeah, swift logs aren't being particularly helpful here. | 18:23 |
| mriedem | hmm, | 18:24 |
| mriedem | https://github.com/openstack/python-neutronclient/blob/stable/kilo/requirements.txt#L6 | 18:24 |
| mriedem | so neutronclient on stable/kilo has capped requirements | 18:24 |
| mtreinish | jlk: hmm, unfortunately that's pretty much the norm... | 18:24 |
| mtreinish | jlk: oh, if its a 401 keystone might be helpful | 18:24 |
| mtreinish | unless it's local swift policy enforcement on some action | 18:24 |
| mriedem | so we need a neutronclient 2.3.12 created from stable/kilo? | 18:24 |
| jlk | I think it might be the local swift policy, but I'm trying to grok the logs | 18:25 |
| *** bkopilov has quit IRC | 18:25 | |
| mtreinish | mriedem: what's in neutronclient's requirements file? | 18:25 |
| mriedem | https://github.com/openstack/python-neutronclient/blob/stable/kilo/requirements.txt#L6 | 18:25 |
| mriedem | that's stable/kilo | 18:25 |
| mriedem | 2.3.11 release is uncapped | 18:25 |
| mriedem | and that's what's being pulled in on stable/juno jobs | 18:25 |
| jlk | So my client is getting Details: 401 Unauthorized: Temp URL invalid | 18:25 |
| *** bkopilov has joined #openstack-qa | 18:26 | |
| mriedem | latest cliff 1.12 requires newer stevedore: https://github.com/openstack/cliff/blob/1.12.0/requirements.txt#L10 | 18:26 |
| mriedem | which breaks the cap on stable/juno | 18:26 |
| mtreinish | mriedem: https://review.openstack.org/#/c/174340/1 | 18:26 |
| mtreinish | oh, I didn't even look at that | 18:27 |
| mtreinish | wrong branch | 18:27 |
| mriedem | seems i need to talk to dhellmann then | 18:27 |
| dhellmann | mriedem, mtreinish : talking this through with sdague it's not clear that we want to cap requirements in libraries | 18:27 |
| dhellmann | we may want to apply the caps to transitive dependencies within the apps, instead | 18:27 |
| mriedem | dhellmann: so e.g. neutronclient would have to cap stevedore b/c neutronclient requires cliff? | 18:28 |
| mriedem | or heat i should say | 18:28 |
| mriedem | heat -> neutronclient -> cliff -> stevedore | 18:29 |
| dhellmann | mriedem: no, the thing that uses neutronclient would -- right, the latter | 18:29 |
| mriedem | hrm | 18:29 |
| dhellmann | yeah, not great | 18:29 |
| mriedem | yeah :( | 18:29 |
| mriedem | so.... | 18:29 |
| dhellmann | I think what we *really* want is to have devstack build a single list of dependencies and install them all at once, so all of the constraints can be applied | 18:29 |
| *** kiran-r has quit IRC | 18:29 | |
| dhellmann | however, sdague pointed out that might make it harder to figure out what was causing a particular requirement failure | 18:30 |
| mriedem | would it be terrible to just have heat have neutronclient==$working_version | 18:30 |
| mriedem | ? | 18:30 |
| mriedem | that's basically how it works for us internally | 18:30 |
| dhellmann | yeah, we don't want pins, that makes it impossible to move things forward because you can't have 2 projects pinning to 2 versions of the same dependency | 18:30 |
| jlk | oooh, I see the 401 in the swift-proxy log, but no reason why. | 18:30 |
| jlk | grumble. | 18:30 |
| mriedem | once we release we rarely update dependent library versions on stable | 18:30 |
| dhellmann | mriedem: pinning also prevents us from releasing bug fixes | 18:31 |
| mtreinish | jlk: heh, useful isn't it :) | 18:31 |
| jlk | no, not at all. And this is with eventlet_debug = true and log-Level = debug | 18:31 |
| mtreinish | jlk: my only thought would be the user you're making the tempurl req with doesn't have the proper role | 18:31 |
| mtreinish | but I don't know | 18:31 |
| dhellmann | mtreinish, mriedem : see https://etherpad.openstack.org/p/requirements-future | 18:31 |
| jlk | yeah, seems like the temp url gets created fine, but accessing not as much? | 18:32 |
| *** suneelb has quit IRC | 18:33 | |
| *** suneelb1 has joined #openstack-qa | 18:34 | |
| *** tkammer has quit IRC | 18:34 | |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Add host counter to ansible inventory https://review.openstack.org/177920 | 18:34 |
| mtreinish | jlk: I've never really used tempurl myself. Google yielded: http://luisbg.blogalia.com/historias/74348 which seems straightforward enough | 18:34 |
| openstackgerrit | Clark Boylan proposed openstack-infra/devstack-gate: Add host counter to ansible inventory https://review.openstack.org/177920 | 18:35 |
| *** apevec has quit IRC | 18:36 | |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Rework get_network_from_name() function https://review.openstack.org/174473 | 18:43 |
| sdague | so, realistically stuff like cliff and stevedore are rarely our issues, they only really became issues because of conflicting requirements | 18:44 |
| sdague | mriedem: you are also building a product, so you're *sure* what versions you are going to ship | 18:45 |
| sdague | the upstream community code really shouldn't encode one product set of choices | 18:46 |
| *** e0ne has joined #openstack-qa | 18:46 | |
| mriedem | yeah i'm not suggesting upstream should pin to what we ship on stable | 18:46 |
| *** rbak_ has quit IRC | 18:47 | |
| *** Longgeek has quit IRC | 18:48 | |
| sdague | so, here is the thing, global-requirements works pretty well as a mechanism when it's both a sync, and an override | 18:49 |
| sdague | but for shipped pypi libraries, it's not an override | 18:49 |
| sdague | that's a big thing we lost | 18:49 |
| sdague | but we kept assuming the old enforcement policies would work fine once that was gone | 18:49 |
| sdague | but... I don't think they do | 18:50 |
| *** rbak has joined #openstack-qa | 18:50 | |
| *** suneelb has joined #openstack-qa | 18:51 | |
| *** suneelb1 has quit IRC | 18:51 | |
| *** Yogi2 has quit IRC | 18:51 | |
| *** Yogi has joined #openstack-qa | 18:52 | |
| mriedem | so if i wanted to unblock heat on stable/juno today i'd need to add the g-r cap for cliff, | 18:52 |
| mriedem | is that bad? | 18:52 |
| mriedem | would require a comment in the requirements.txt file saying 'transitive dep thingy, don't remove me' | 18:52 |
| mriedem | we have a similar issue in glance on stable/juno | 18:52 |
| *** Rockyg has joined #openstack-qa | 18:55 | |
| mtreinish | sdague: btw, did you see the respun version of?: https://review.openstack.org/#/c/176975 | 18:55 |
| *** marun_ has joined #openstack-qa | 18:55 | |
| sdague | mriedem: yeh, I think adding the transitive dependencies in the apps are the least bad approach here. | 18:56 |
| openstackrecheck | Console logs not available after 13:25s for gate-nova-python27 177332,4,5549213 | 18:56 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/176651 failed because of: gate-grenade-dsvm: https://bugs.launchpad.net/bugs/1449136 | 18:56 |
| openstack | Launchpad bug 1449136 in OpenStack-Gate "OpenStack pypi mirrors disconnecting connections" [Undecided,New] | 18:56 |
| sdague | mtreinish: +2 | 18:56 |
| mtreinish | sdague: thanks | 18:56 |
| sdague | mtreinish: so... tags :) | 18:57 |
| mriedem | sdague: ok, i'll start whacking some moles | 18:57 |
| sdague | before you run away to other things | 18:57 |
| sdague | mriedem: did you see my exploratory etherpad here? | 18:57 |
| sdague | mriedem: https://etherpad.openstack.org/p/requirements-future | 18:57 |
| mriedem | sdague: https://etherpad.openstack.org/p/requirements-future ? | 18:57 |
| sdague | yeh | 18:57 |
| mriedem | i skimmed | 18:57 |
| *** Sukhdev has joined #openstack-qa | 18:58 | |
| *** marun has quit IRC | 18:58 | |
| sdague | so, honestly, it's probably worth trying to think this whole thing through a bunch before another round of changes, because we do seem to keep whacking things in funny ways, just to have new issues pop up | 18:58 |
| mtreinish | sdague: I'm reviewing them now | 18:59 |
| *** suneelb has quit IRC | 18:59 | |
| *** suneelb has joined #openstack-qa | 18:59 | |
| sdague | mtreinish: ok | 18:59 |
| mtreinish | but I was thinking it might help to throw up a patch to the readme or something to explain what smoke is and it's scope/intent | 18:59 |
| mtreinish | assuming we don't have that already somehwere | 18:59 |
| sdague | sure, I'd agree, why don't you see if you think I'm on the right track with these, and we can update the policy after. Because based on my experience so far, this is going to just be recheck grind to land | 19:00 |
| sdague | fortunately, the grenade smoke based testing gets a bunch shorter at the end of it | 19:00 |
| sdague | so the likelihood of grenade rando fail on tempest is less likely | 19:01 |
| *** dkranz has quit IRC | 19:01 | |
| mriedem | mtreinish: if there is a readme at some point for what smoke tests are, i'd be interested since that's what we run against internally | 19:01 |
| mriedem | as mentioned last week | 19:01 |
| sdague | mriedem: well, I'm about to drop it considerably in size, if these patches land | 19:02 |
| mriedem | sdague: i saw | 19:02 |
| mtreinish | sdague: I've had a bunch of patches today and only had 2 bounces so far, you just must be unlucky today :) | 19:02 |
| mriedem | probably still bigger than refstack :) | 19:02 |
| mtreinish | mriedem: I think the defcore list is at like 150 or something | 19:02 |
| mriedem | yes it's around that | 19:02 |
| mtreinish | mriedem: I don't think sdague's changes make the set that small | 19:02 |
| mriedem | 140-150 | 19:02 |
| sdague | no, this is that small at the end | 19:03 |
| mriedem | basically i think we want/expect smoke to be CRUD ops like sdague said in the commit message and only on 'core' function, not bizarro optional extensions | 19:03 |
| mtreinish | oh it is, I haven't made it to the top of the branch yet | 19:03 |
| mtreinish | mriedem: no shelve while paused? | 19:03 |
| mriedem | well, there are other flags to disable those in your config anyway | 19:04 |
| mriedem | but e.g. we had to completely disable lbaas in our CI | 19:04 |
| mriedem | using the api_extensions flag or whatever | 19:04 |
| *** davideagnello has quit IRC | 19:04 | |
| mtreinish | mriedem: I need to get dougwig to remove those from tempest soon anyway | 19:04 |
| sdague | yeh, I dropped all neutron stuff that had an extension check | 19:04 |
| mtreinish | they were supposed to be out a while ago | 19:04 |
| mtreinish | because they barely ever work | 19:04 |
| *** sputnik13 has quit IRC | 19:04 | |
| sdague | yeh, so on nova network this ends up being 98 tests run, and on neutron it's 151 | 19:05 |
| sdague | for grenade, which only has some of the services enabled | 19:05 |
| *** sputnik13 has joined #openstack-qa | 19:06 | |
| *** davideagnello has joined #openstack-qa | 19:06 | |
| mriedem | we run with neutron so that's fine, right now smoke is around 300 in our runs after we strip some things out | 19:06 |
| mtreinish | mriedem: yeah for neutron smoke was all sorts of crazy before, because it basically was every neutron test | 19:06 |
| sdague | the neutron run is ~ 5 minutes, and nova network is ~ 3 minutes | 19:07 |
| mtreinish | (well all those added before the end of juno-3) | 19:07 |
| *** Yogi2 has joined #openstack-qa | 19:07 | |
| sdague | and, honestly, I kind of think some of the auto subclassing probably should be disabled, like marking a cinder test smoke should only run it on 1 api version, not both. | 19:08 |
| *** Yogi has quit IRC | 19:09 | |
| mtreinish | sdague: heh, that would make hogepodge happy. I had patches to clean it up and enforce when it could be used, because there are some places which make no sense | 19:09 |
| mtreinish | but if we want to drop it everywhere I'm fine with that too | 19:09 |
| sdague | yeh, I actually kind of think we should drop it everwhere. I'd be ok using test scenarios for that (that's actually where it makes some sense to use), however we'd need a bunch of documentation to explain to people what's going on, as it looks very magical | 19:11 |
| mtreinish | sdague: oh, that's an interesting idea. Test scenarios would probably be a good fit for those tests | 19:14 |
| mtreinish | (although that wouldn't make hogepodge happy anymore :) ) | 19:14 |
| sdague | heh | 19:14 |
| sdague | because they don't have uniq ids? | 19:15 |
| mtreinish | sdague: yeah | 19:15 |
| sdague | yeh.... that seems like a thing that's worth solving, we'd have to think about it though | 19:15 |
| * hogepodge shakes head at ids | 19:15 | |
| *** shashankhegde has quit IRC | 19:19 | |
| *** yamahata has joined #openstack-qa | 19:21 | |
| *** jogo has joined #openstack-qa | 19:23 | |
| openstackrecheck | openstack/requirements change: https://review.openstack.org/177193 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 19:27 |
| *** armax has quit IRC | 19:29 | |
| *** armax has joined #openstack-qa | 19:31 | |
| jlk | hahaha hahaha, whut | 19:33 |
| jlk | resize tests are failing, because guestfs can't be found so nova falls back to something else to check if the in-instance fileysystem can be expanded, and THAT tries to do a mount action without specifying a filesystem and mount gives it the finger. | 19:33 |
| mtreinish | jlk: lol, nice | 19:34 |
| *** dkranz has joined #openstack-qa | 19:35 | |
| openstackgerrit | Monty Taylor proposed openstack-infra/devstack-gate: Put input variables into ansible inventory https://review.openstack.org/177943 | 19:35 |
| openstackgerrit | Monty Taylor proposed openstack-infra/devstack-gate: Move all the ansible calls into playbooks https://review.openstack.org/177944 | 19:35 |
| dougwig | mriedem: what lbaas issues did you have? | 19:37 |
| mriedem | dougwig: it was awhile back, sometime in juno probably, i just don't think we were configuring it probably, i.e. we didn't have haproxy running | 19:37 |
| mriedem | there were probably some other issues, idk | 19:38 |
| mriedem | not really my full time job anymore | 19:38 |
| mriedem | *configuring it properly | 19:38 |
| *** tkammer has joined #openstack-qa | 19:39 | |
| dougwig | mriedem: well, short of an lbaas driver, i doubt you needed it. | 19:40 |
| *** marun_ is now known as marun | 19:40 | |
| dougwig | it's on my short list mtreinish. | 19:40 |
| mtreinish | dougwig: ok cool, just realized I hadn't bugged you about it lately :) | 19:41 |
| *** edleafe- is now known as edleafe | 19:44 | |
| *** ibiris is now known as ibiris_away | 19:45 | |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Rework get_network_from_name() function https://review.openstack.org/174473 | 19:51 |
| sdague | mtreinish: so, the 3 gate tag removal patches have jenkins +1 on them, so you want to +A those through? At least that will prevent one giant merge conflict issue. | 19:51 |
| mtreinish | sdague: sure, one sec I'll do that right now | 19:51 |
| sdague | dkranz / andreaf you folks want to look at the rest of that smoke trimming? | 19:52 |
| dkranz | sdague: ok. Is there something somewhere that describes what the intended end state is? | 19:53 |
| sdague | nothing separate from the commit messages | 19:53 |
| sdague | mostly, smoke tests should be a fast an small number of positive non admin tests | 19:54 |
| sdague | it's a series of purge patches that try to each explain why getting rid of what they get rid of is ok | 19:54 |
| sdague | it starts at - https://review.openstack.org/#/c/177745/ and goes from there | 19:55 |
| dkranz | sdague: I am fine with them except I wondered about the impact on the neutron icehouse test | 19:55 |
| andreaf | samueldmq: api_v3 is true by default | 19:55 |
| dkranz | sdague: but I will check and see if that is covered | 19:55 |
| dkranz | sdague: for smoke I mean | 19:55 |
| sdague | dkranz: yeh, so this will drop a lot of the tests there, but, honestly, I think it's time to reclaim our smoke tag | 19:55 |
| mtreinish | dkranz: it's coverage would decrease slightly, but to be honest I don't care about that too much | 19:55 |
| samueldmq | andreaf, nice! | 19:56 |
| mtreinish | the branch should have died already... | 19:56 |
| sdague | it leaves a number of representative tests that will ensure that some basic neutron functionality is running | 19:56 |
| samueldmq | andreaf, btw I just saw your review on that patch, I think just neutron is enough for us .. we just want to test it against devstack :) | 19:56 |
| dkranz | mtreinish: so I am fine with that change but think it should wait until we cut the kilo tag this week | 19:56 |
| samueldmq | andreaf, but let's see what other people think about | 19:56 |
| sdague | https://review.openstack.org/#/c/177829 is the trim of the network/api bit | 19:56 |
| mtreinish | dkranz: I'm fine with waiting until after we cut the tempest 4 tag | 19:57 |
| sdague | sure, that seems reasonable | 19:57 |
| dkranz | mtreinish: ok, I mean we should do this when we drop icehouse from the gate | 19:57 |
| sdague | it would be nice to get review regardless | 19:57 |
| dkranz | sdague: sure, will do | 19:57 |
| *** armax has quit IRC | 19:58 | |
| mtreinish | dkranz: oh, that's not the tempest 4 tag | 19:58 |
| sdague | dkranz: so this has a pretty big impact on grenade | 19:58 |
| andreaf | samueldmq: well at the moment you only configured nova network in there - you should replace it with neutron then | 19:58 |
| mtreinish | that's at least a month or 2 in the future | 19:58 |
| dkranz | mtreinish: ok | 19:58 |
| sdague | so basically if this isn't happening this week, I'm going to remove using tempest smoke in grenade and instead hand pick tests over there | 19:58 |
| mtreinish | but like I said that's not something I think we should care about | 19:58 |
| andreaf | samueldmq: but considering that the gate runs both n-net and neutron, if you want to migrate them to identity v3 eventually, then you should test both | 19:58 |
| dkranz | mtreinish: I just wnat to have a tag that can run icehouse as it is now, and kilo | 19:58 |
| sdague | because we get a lot of edge races that we shouldn't be tripping in grenade | 19:58 |
| mtreinish | dkranz: switch the job to just run network instead | 19:59 |
| mtreinish | but I don't feel the risk of regression on neutron icehouse is something that's important anymore | 19:59 |
| mtreinish | I mean you could barely run icehouse neutron | 19:59 |
| sdague | dkranz: is there a reason not to just run with tempest 3 for that? | 20:00 |
| sdague | because that's even more tempest than even existed when icehouse was released | 20:00 |
| dkranz | sdague: no, but I would rather use tempest 4 which will be in a few days | 20:00 |
| samueldmq | andreaf, just changing to the gate pipeline and gate-tempest-dsvm-keystonev3-full ? | 20:01 |
| mtreinish | I'm fine with doing that, but we should drop the icehouse jobs when we do that | 20:01 |
| mtreinish | I'll aslo mark in the tag notes we're dropping icehouse support for after tempest-4 | 20:01 |
| andreaf | samueldmq: so, check-tempest-dsvm-keystonev3-full runs with nova network, and check-tempest-dsvm-neutron-keystonev3-full runs with neutron | 20:02 |
| dkranz | mtreinish: I think that is fine. Basically we are saying the definition of "branchless tempest" is "we support current and two releases back" | 20:02 |
| andreaf | samueldmq: in your patch you only added the nova network job to devstack experimental pipeline | 20:02 |
| samueldmq | andreaf, ++ got it | 20:03 |
| sdague | mtreinish: so... it seems like the jobs would need to keep going, just on the tempest 4 release | 20:03 |
| mtreinish | heh, sure. Although that's a little different than what our original message was | 20:03 |
| mtreinish | sdague: I meant the tempest icehouse check jobs | 20:03 |
| *** marun has quit IRC | 20:03 | |
| mtreinish | if we're not running master tempest on icehouse commits anymore there is not reason to gate tempest changes on it | 20:03 |
| sdague | mtreinish: right but how will icehouse nova land a patch? | 20:03 |
| mtreinish | they run with tempest-4, I thought that's what you guys were proposing | 20:04 |
| sdague | yes | 20:04 |
| sdague | just making sure | 20:04 |
| dkranz | sdague, mtreinish : right, this is not really branchless tempest but the best we can reasonably do | 20:04 |
| mtreinish | it addresses my concerns with the dumb stable maint windows we've got now too | 20:04 |
| mtreinish | because I don't have to care about the old branches | 20:04 |
| dkranz | mtreinish: I just want our message about it to be consistent with what we are actually doing | 20:04 |
| sdague | so, honestly, I'm a little confused | 20:05 |
| sdague | because realistically branchless tempest never guarunteed the internal tag structure | 20:05 |
| sdague | so I don't feel like trimming smoke is a violation of that | 20:05 |
| sdague | and in the upstream gate all it means is less tests are run in a configuration that was never really working | 20:05 |
| samueldmq | andreaf, thanks | 20:06 |
| mtreinish | sdague: I don't care one way or the other. Like I said running less neutron tests on icehouse doesn't bother me | 20:06 |
| dkranz | sdague: it should not have. But we started using smoke instead of a neutron-busted tag | 20:06 |
| *** johnthetubaguy is now known as zz_johnthetubagu | 20:06 | |
| sdague | dkranz: right, and I'm saying, we did that as a favor to help them get to a full test run and not regress | 20:06 |
| sdague | which happened | 20:06 |
| dkranz | sdague: right | 20:06 |
| sdague | keeping the shoring up on icehouse is not really required | 20:06 |
| sdague | if someone wants to reconstruct that regex for upstream, go for it | 20:07 |
| dkranz | sdague: all I am requsting is we wait until we cut tempest 4 to remove them | 20:07 |
| sdague | dkranz: so that's fine, but I actually think we should run master on icehouse after that point still | 20:07 |
| dkranz | sdague: well, it can't hurt | 20:07 |
| sdague | it will just run light, and that's fine | 20:07 |
| sdague | because at this point icehouse patches should be getting a ton of manual review | 20:08 |
| sdague | for projects | 20:08 |
| jogo | mtreinish: want to review https://review.openstack.org/#/c/165233 | 20:08 |
| dkranz | sdague: I agree and am fine with letting the stable team for icehouse add tests if they want | 20:08 |
| mtreinish | pcrews: btw, if you're looking for example usage of subunit attachments, this might be helpful: http://git.openstack.org/cgit/openstack/os-testr/tree/os_testr/subunit_trace.py#n103 | 20:09 |
| dkranz | sdague: I just think we should spread what we are doing more widely (ml post) rather than just dumping it | 20:09 |
| mtreinish | jogo: sure I'll take a look in a bit | 20:09 |
| jogo | thanks | 20:09 |
| sdague | dkranz: yeh, I'm mostly fine with that. I'll send an email. This was something that we planned to just do at the sprint though. | 20:11 |
| *** Swami has quit IRC | 20:11 | |
| dstanek | has anyone noticed or reported issues using devstack master? i have been seeing failures on 14.04 today | 20:12 |
| sdague | dstanek: you need to be more specific | 20:12 |
| dstanek | i'll get a 500 from Keystone, but then i can't find anything in the logs to point to a error/tb | 20:12 |
| sdague | there must be something in the apache logs | 20:13 |
| sdague | if there is a 500 | 20:13 |
| dstanek | i can see the 500 in the access log, but nothing in the error log for that corresponding time | 20:13 |
| sdague | there are dedicated keystone logs | 20:13 |
| sdague | which I thought we all put into the screen sessions | 20:14 |
| dstanek | yep, nothing in their either - the ones in /opt/stack/logs | 20:14 |
| dstanek | this is why i'm confused | 20:14 |
| sdague | so, I don't know then. You're a keystone core, I'd be asking you :) | 20:14 |
| dstanek | it got to the point where keystone was running - screen had a shell, dstat and keystone panes, but nothing else | 20:15 |
| dstanek | sdague: :-) | 20:15 |
| dstanek | i'm going to start digging in on fedora if it's broken there - was hoping that there was already a known issue | 20:15 |
| sdague | ./clean.sh && ./stack.sh | tee stack.log and go through it with a fine toothed comb | 20:15 |
| sdague | yeh, I don't know, but I don't regularly run on fedora | 20:16 |
| mtreinish | dkranz: so I think andreaf might have just approved the start of the smoke tag trimming series... :) | 20:17 |
| dkranz | mtreinish: as I said in the network one, I think it is the only one that matters | 20:18 |
| dkranz | mtreinish: I am fine with the others since we already run a full icehouse job | 20:18 |
| andreaf | dkranz, mtreinish: upz, did I miss something? | 20:18 |
| dkranz | mtreinish: my only objection was that we re-used smoke to mean "neutron-icehouse-broken" | 20:18 |
| mtreinish | dkranz: but not with neutron | 20:18 |
| mtreinish | there are neutron negative tests, just saying | 20:19 |
| mtreinish | not that it really concerns me | 20:19 |
| dkranz | mtreinish: well, ok. That doesn't really concern me either | 20:19 |
| dkranz | mtreinish: our negative tests are lame anyway | 20:19 |
| mtreinish | andreaf: nothing, we were just discussing the time frame for the smoke tag stuff | 20:19 |
| andreaf | mtreinish, dkranz: ok | 20:20 |
| dkranz | mtreinish: and what happened to our policy of one "company" not just pushing stuff like this through? | 20:20 |
| *** pglass has quit IRC | 20:21 | |
| mtreinish | dkranz: it was never an official policy, just something sdague and I normally do to avoid the appearance of impropriety | 20:21 |
| mtreinish | clearly andreaf doesn't feel the same way :) | 20:21 |
| mtreinish | although sometimes it's hard for me to tell andreaf and I work for the same company :) | 20:22 |
| dkranz | mtreinish: I'll bet andreaf didn't think about it | 20:22 |
| dkranz | mtreinish: anyway, my bottom line is that I don't think we should be reducing coverage of gate jobs without any discussion or notice | 20:22 |
| andreaf | mtreinish, dkranz: yes I forgot about it - I'm sure you don't mind in this case? | 20:22 |
| dkranz | mtreinish: I was not thrilled when we did that by removing xml, but the win was huge | 20:23 |
| dkranz | mtreinish: I just don't see this as the same level of importance that it can't wait a small bit | 20:23 |
| mtreinish | andreaf: heh, sure it's np. It doesn't bug me one way or the other :) | 20:24 |
| mtreinish | dkranz: what about removing xml? | 20:24 |
| dkranz | mtreinish: we dropped testing of xml in the jobs running against (at the time) icehouse even though it was an active release | 20:24 |
| dkranz | mtreinish: that could allow a nova change to break xml and slip through the gate | 20:25 |
| mtreinish | dkranz: oh, but we brought that up on the ml and cross-project meeting and everyone said that it wasn't important at all | 20:25 |
| mtreinish | and I agree this isn't nearly as big a deal | 20:25 |
| *** shashankhegde has joined #openstack-qa | 20:26 | |
| dkranz | mtreinish: in this case grenade seems to be the driver. Otherwise I would have preferred to just let icehouse jobs die | 20:26 |
| dkranz | mtreinish: but I am ok with tagging 4 asap and then removing smoke from neutron | 20:27 |
| mtreinish | dkranz: heh, well I tried to kill them 2 months ago :) | 20:27 |
| dkranz | mtreinish: actually I would be ok to tag 4 now | 20:27 |
| dkranz | mtreinish: I didn't object to killing them, did I? | 20:27 |
| mtreinish | dkranz: hehe, fair enough :) | 20:28 |
| dkranz | mtreinish: so when can we tag? | 20:28 |
| mtreinish | dkranz: yeah I think we're probably in a decent place for tagging | 20:28 |
| dkranz | mtreinish: I mean there is always something we won't have | 20:28 |
| *** Rockyg has quit IRC | 20:29 | |
| mtreinish | dkranz: sure, it'll definitely be this week, I want to do a quick look over what's in progress before | 20:29 |
| mtreinish | I could see doing it tomorrow or Wed | 20:29 |
| dkranz | mtreinish: ok, thanks | 20:29 |
| *** yfried_ is now known as yfried|afk | 20:30 | |
| dansmith | jogo: sdague: clarkb: so, the partial-ncpu job should really be multinode, where we only dont-upgrade one of the computes | 20:34 |
| dansmith | I'm sure there are reasons why that's hard, but.. thoughts? | 20:34 |
| jogo | dansmith: yes!~ | 20:34 |
| jogo | dansmith: it was hard because we didn't have any multinode at all initially | 20:34 |
| dansmith | yes hard or yes agree? :) | 20:34 |
| dansmith | well, sure | 20:34 |
| jogo | yes agree | 20:34 |
| jogo | much easier now | 20:34 |
| *** beisner has quit IRC | 20:34 | |
| jogo | still a bit of work I think | 20:34 |
| dansmith | okay | 20:35 |
| sdague | grenade on multinode is going to be interesting | 20:36 |
| dansmith | presumably grenade needs to talk to the other node | 20:36 |
| dansmith | yeah | 20:36 |
| dansmith | ansible could help with that, right? | 20:36 |
| sdague | dansmith: if you want to take that on, cool, but it's going to be weird | 20:36 |
| dansmith | I saw clarkb did something there | 20:36 |
| mtreinish | jogo: well you'd have to teach grenade about multinode right? | 20:36 |
| sdague | right now all the multinode logic is actually above the tools | 20:36 |
| dansmith | sdague: well, I dunno that I do, but it needs doing I think | 20:36 |
| sdague | mtreinish: actually, you probably have to orchestrate at the d-g layer | 20:37 |
| *** mwagner_lap has quit IRC | 20:37 | |
| jogo | sdague: right, devstack-gate has the magic | 20:37 |
| jogo | but I don't think grenade has a multi node mode yet | 20:37 |
| dansmith | maybe clarkb sdague jogo and dansmith could have a pow-wow at summit about it? | 20:37 |
| *** beisner has joined #openstack-qa | 20:38 | |
| jogo | dansmith: ++ | 20:38 |
| sdague | sure | 20:38 |
| jogo | figure get multinode regular gating first | 20:38 |
| jogo | sdague: which we may not be far off from | 20:38 |
| dansmith | surely | 20:38 |
| *** Nic has quit IRC | 20:38 | |
| jogo | sdague: so multinode is running pretty well at the moment | 20:39 |
| jogo | in past 24 hours 61 success, 6 failures with at least 2 or 3 of those being either unrelated or valid failures | 20:39 |
| sdague | 10% failure rate is kind of brutal | 20:40 |
| jogo | sdague: let me dig into the specific failures a bit more | 20:40 |
| dansmith | it's 5% | 20:40 |
| *** pglass has joined #openstack-qa | 20:41 | |
| mtreinish | jogo: for comparison the overall gate tempest/grenade job failure rate for kilo is ~.77% | 20:41 |
| dansmith | either way, the past 24 hours is not a good metric, given it's monday | 20:42 |
| jogo | http://logs.openstack.org/14/166214/15/check/check-tempest-dsvm-multinode-full/d6c2efd/console.html#_2015-04-27_11_07_24_519 | 20:42 |
| jogo | not sure if that is a valid failure or not | 20:42 |
| jogo | dansmith: right, so going to retry tomorrow | 20:42 |
| dansmith | yep | 20:42 |
| *** miqui has quit IRC | 20:43 | |
| dansmith | jogo: that failure is from ansible, so might be infra related? | 20:43 |
| dansmith | jogo: and/or timing about knowing when the thing is up | 20:43 |
| dansmith | ? | 20:43 |
| jogo | dansmith: yeah asking in -infra | 20:44 |
| sdague | dansmith: the nodes should be up before any of that code runs | 20:44 |
| dansmith | 10 seconds before it gave up | 20:44 |
| dansmith | sdague: okay | 20:44 |
| sdague | it doesn't allocate nodes | 20:44 |
| dansmith | sdague: but still, this is just sshing into said nodes | 20:44 |
| sdague | sure | 20:44 |
| sdague | but that's part of the overall variability concerns I have with having a lot of different multinode configurations that pre-gate | 20:45 |
| dansmith | sdague: I'm saying hopefully that is just an artifact of not having the multinode setup stuff dialed in and not a crazy random nova race :) | 20:45 |
| dansmith | the former being hopefully easier to fix | 20:45 |
| sdague | I'm sure it's a nova race... at rax space | 20:45 |
| jogo | 3/6 look real | 20:45 |
| jogo | http://logs.openstack.org/05/176805/4/check/check-tempest-dsvm-multinode-full/305a5f8/console.html | 20:45 |
| sdague | so... I also think coupled with that kind of conversation needs to be how do we move some of the testing to post-gating, and not getting lost | 20:46 |
| dansmith | sdague: heh, I meant with nova in our deploys | 20:46 |
| sdague | otherwise we'll just grow a system that means no one lands any patches any more | 20:47 |
| jogo | sdague: well https://review.openstack.org/#/c/168486/ works | 20:47 |
| mtreinish | jogo: that doesn't solve the getting lost part | 20:47 |
| jogo | sdague: although I ahven't seen any neutron job s fail on the openstack-qa ML | 20:47 |
| jogo | mtreinish: ML | 20:47 |
| mtreinish | jogo: believe me that doesn't help | 20:47 |
| sdague | jogo: right, that doesn't solve the not getting lost part | 20:47 |
| dansmith | sdague: if we get to this point and a node is dead because it failed to deploy at RAX, is there any retry we can do, or do we just claim failure because the re-allocation can't happen? | 20:47 |
| mtreinish | we've had the qa periodic jobs do that for a long time and no one watches it | 20:48 |
| sdague | dansmith: some recovery exists | 20:48 |
| jogo | mtreinish: I wouldn't say it solves the getting lost but it helps a little | 20:48 |
| jogo | no one watching isn't a getting list issue its no one caring | 20:48 |
| jogo | but sure if it was easier to see maybe more would care | 20:48 |
| dansmith | sdague: what happens? do we fail the job in a way so it gets retried, or do we try to fix things in the middle? | 20:48 |
| sdague | things get restarted in the middle, but it's specific fails that get caught for that in zuul | 20:48 |
| sdague | you should chat with clarkb about it | 20:49 |
| dansmith | sdague: does that happen for single-instance jobs too? or not because nothing is running anyway? | 20:49 |
| sdague | so, I need to call it end of day, ask clarkb :) | 20:49 |
| sdague | I'll talk with folks tomorrow | 20:49 |
| dansmith | alright. | 20:50 |
| *** yfried|afk is now known as yfried_ | 20:50 | |
| jogo | YBso this looks like a pretty nasty failure http://logs.openstack.org/05/176805/4/check/check-tempest-dsvm-multinode-full/305a5f8/logs/testr_results.html.gz | 20:50 |
| dansmith | jogo: we have some cases that cascade failures, right | 20:51 |
| dansmith | ? | 20:51 |
| dansmith | like, if we get quotas really fscked or something | 20:51 |
| jogo | looks like the second node just hung | 20:51 |
| *** turul has joined #openstack-qa | 20:51 | |
| *** turul is now known as afazekas | 20:52 | |
| jogo | n-cpu stops at 16:45:45 | 20:53 |
| jogo | 176805 | 20:53 |
| jogo | http://logs.openstack.org/05/176805/4/check/check-tempest-dsvm-multinode-full/305a5f8/logs/subnode-1/screen-n-cpu.txt.gz#_2015-04-27_16_45_45_696 | 20:53 |
| mtreinish | jogo: is there any thing in the syslog or kernel log for the 2nd node? | 20:53 |
| *** shardy is now known as shardy_z | 20:53 | |
| openstackgerrit | Sean Dague proposed openstack/tempest: adjust smoke definition to not include *all* scenario https://review.openstack.org/177852 | 20:53 |
| jogo | mtreinish: I don' think so http://logs.openstack.org/05/176805/4/check/check-tempest-dsvm-multinode-full/305a5f8/logs/subnode-1/kern_log.txt.gz | 20:55 |
| *** yamahata has quit IRC | 20:55 | |
| *** yamahata has joined #openstack-qa | 20:55 | |
| *** Sukhdev has quit IRC | 20:58 | |
| *** yfried_ is now known as yfried|afk | 21:00 | |
| dkranz | mtreinish: how does https://review.openstack.org/#/c/176975/4 relate to lock_path in tempest.conf? | 21:00 |
| *** amotoki has joined #openstack-qa | 21:01 | |
| dkranz | mtreinish: doesn't your change mean lock_path is no longer required? | 21:01 |
| *** Sukhdev has joined #openstack-qa | 21:01 | |
| *** armax has joined #openstack-qa | 21:03 | |
| clarkb | what are people supposed to ask me? | 21:04 |
| dansmith | clarkb: I want to chat about multinode grenade and recovery when one node doesn't spawn | 21:04 |
| dansmith | but I have to run out for a bit now | 21:04 |
| clarkb | ok | 21:05 |
| *** marun has joined #openstack-qa | 21:05 | |
| dansmith | clarkb: not "out" mind you, else you'd have heard about it on the news, but... afk :) | 21:06 |
| clarkb | ha | 21:06 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/177186 failed gate-tempest-dsvm-large-ops, gate-devstack-dsvm-cells, gate-grenade-dsvm-partial-ncpu in the gate queue with an unrecognized error | 21:07 |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Add dstat to subnode in multinode mode https://review.openstack.org/177970 | 21:09 |
| *** Yogi2 has quit IRC | 21:10 | |
| *** Swami has joined #openstack-qa | 21:11 | |
| jlk | hogepodge: is it intended that to get blessed we have to support resizing guest filesystems /down/ ? Resize up I could see, but shrinking is a whole lot more complicated and seems pretty fragile. | 21:11 |
| *** rfolco has quit IRC | 21:12 | |
| hogepodge | jlk I think up is the minimum, although it's mainly whatever tempest tests for. | 21:12 |
| jlk | tempest appears to be testing resize down, by way of testing resize server revert | 21:13 |
| jlk | https://github.com/openstack/tempest/blob/master/tempest/api/compute/servers/test_server_actions.py#L236 | 21:14 |
| *** jogo has quit IRC | 21:15 | |
| mtreinish | dkranz: yeah, it'll mean you don't have to set it, unless you want to | 21:20 |
| *** dustins has quit IRC | 21:20 | |
| mtreinish | jlk: I thought a resize revert was somewhat special, but I dunno | 21:21 |
| mtreinish | I can definitely see an issue with a resize down | 21:21 |
| jlk | maybe my system should allow it, but I'm definitely having probs with resize down. | 21:22 |
| *** artom has quit IRC | 21:22 | |
| mtreinish | dansmith: if you're leaving your office I figured that would count as "out" | 21:22 |
| dkranz | mtreinish: ok, so we should change the text in config.py to say that. Do you want to re-spin, or I can do it? | 21:23 |
| mtreinish | dkranz: well the issue is we don't own the help for that option, since it's in oslo_concurrency | 21:24 |
| jlk | mtreinish: in limited testing, your patch to turn off network tenant isolation is aces. | 21:24 |
| mtreinish | dkranz: or did you just mean leave a comment? | 21:24 |
| mtreinish | jlk: awesome | 21:24 |
| jlk | I'll note in the review | 21:24 |
| *** tkammer has quit IRC | 21:24 | |
| dkranz | mtreinish: oh, I see. I meant change the help but I guess we cannot. Oh well. | 21:24 |
| dkranz | mtreinish: kind of a weird case. | 21:25 |
| mtreinish | dkranz: although I probably should update: http://docs.openstack.org/developer/tempest/configuration.html#lock-path | 21:25 |
| mtreinish | dkranz: well I think I've made my thoughts about config options from libs known :) | 21:25 |
| dkranz | mtreinish: :) | 21:25 |
| dkranz | mtreinish: ok, I'll +A and you can change the doc file (or just remove the reference to lock_path I suppose. | 21:26 |
| *** suneelb1 has joined #openstack-qa | 21:26 | |
| *** suneelb1 has quit IRC | 21:26 | |
| *** suneelb1 has joined #openstack-qa | 21:27 | |
| mtreinish | dkranz: yeah we can probably just drop that section | 21:27 |
| *** apevec has joined #openstack-qa | 21:27 | |
| *** suneelb has quit IRC | 21:28 | |
| *** Nic has joined #openstack-qa | 21:30 | |
| jlk | mtreinish: I may have spoken too soon. I'm seeing more failures than before, need to investigate | 21:33 |
| *** dkranz has quit IRC | 21:34 | |
| mtreinish | jlk: hmm, ok. Yeah I figured there was probably an edge case I wasn't accounting for | 21:34 |
| mtreinish | I think the underlying assumption for a bunch of tests with neutron enabled is that a new separate network will be there for each test | 21:35 |
| mtreinish | but, I guess with accounts.yaml there isn't a guarantee of that. So I dunno actually :) | 21:35 |
| dansmith | mtreinish: I leave my office and venture into the house quite often, thank you very much | 21:36 |
| jlk | I'll get some tracebacks here. Previous run ran off the scrollback because of resize failures :( | 21:36 |
| *** jgrimm is now known as zz_jgrimm | 21:38 | |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Drop lock_path section from config guide https://review.openstack.org/177978 | 21:38 |
| mtreinish | dkranz ^^^ | 21:38 |
| *** mriedem has quit IRC | 21:43 | |
| *** armax has quit IRC | 21:50 | |
| *** marun has quit IRC | 21:55 | |
| jlk | oooh | 21:57 |
| jlk | hahaha | 21:57 |
| jlk | mtreinish: so, tempest is trying to apply a role to the users, the role I'mt elling it to use for swift | 21:57 |
| jlk | only that role is a role users already have, so a 503 happens | 21:58 |
| jlk | and tempest gives up | 21:58 |
| mtreinish | jlk: hmm, tempest is supposed to handle the duplicate role assign and just log a warning | 21:59 |
| jlk | maye, maybe not. | 21:59 |
| jlk | I may be misattributing this 503 | 21:59 |
| mtreinish | 503 isn't the right error code for that | 21:59 |
| jlk | yeah I'm jumping to some conclusions I shouldn't. Disregard me, I just make noise | 21:59 |
| jlk | you were right, something is trying to create a network still | 22:00 |
| jlk | https://gist.github.com/j2sol/e3e30ff64d3b43491d0a | 22:00 |
| mtreinish | jlk: hmm that's inside the if block | 22:03 |
| mtreinish | d'oh I got the logic backwards | 22:03 |
| jlk | yeah I'm tracing | 22:03 |
| jlk | oooh | 22:03 |
| openstackrecheck | openstack/nova change: https://review.openstack.org/149371 failed gate-devstack-dsvm-cells in the gate queue with an unrecognized error | 22:03 |
| jlk | haha | 22:03 |
| jlk | yeah | 22:03 |
| *** bknudson has quit IRC | 22:03 | |
| *** yamamoto has quit IRC | 22:04 | |
| jlk | I was fooled by only some of the tests wanting an isolated network | 22:04 |
| *** e0ne has quit IRC | 22:05 | |
| openstackgerrit | Matthew Treinish proposed openstack/tempest: Add config option to disable network isolation https://review.openstack.org/177293 | 22:06 |
| mtreinish | jlk: ^^^ ok that should have the correct logic now | 22:06 |
| jlk | that's what I'm running with now, will check latest round of traces | 22:07 |
| *** apevec has quit IRC | 22:08 | |
| *** salv-orlando has quit IRC | 22:09 | |
| *** pglass has quit IRC | 22:09 | |
| *** suneelb1 has quit IRC | 22:15 | |
| *** suneelb1 has joined #openstack-qa | 22:16 | |
| *** suneelb2 has joined #openstack-qa | 22:17 | |
| *** suneelb1 has quit IRC | 22:17 | |
| *** cyeoh has quit IRC | 22:17 | |
| *** cyeoh has joined #openstack-qa | 22:20 | |
| *** suneelb has joined #openstack-qa | 22:20 | |
| *** marun has joined #openstack-qa | 22:23 | |
| *** suneelb2 has quit IRC | 22:25 | |
| *** mriedem has joined #openstack-qa | 22:25 | |
| *** amotoki has quit IRC | 22:27 | |
| *** salv-orlando has joined #openstack-qa | 22:29 | |
| *** mariza10 has joined #openstack-qa | 22:33 | |
| *** mriedem has quit IRC | 22:33 | |
| *** mriedem has joined #openstack-qa | 22:34 | |
| mtreinish | jogo, mriedem: so since Paris: http://paste.openstack.org/show/209007/ | 22:36 |
| mariza10 | can someone help with a tempest failure im getting ? i have a jenkins jobs that runs `tox -epy27` on tempest..i was a getting SUCCESS , until i made changes to etc/tempest.conf ... now the is failing for some weird error.. | 22:36 |
| *** dboik_ has joined #openstack-qa | 22:36 | |
| mariza10 | this is the complete stacktrace : http://fpaste.org/215993/01705281/ | 22:36 |
| *** suneelb has quit IRC | 22:36 | |
| *** suneelb has joined #openstack-qa | 22:37 | |
| mtreinish | mariza10: there is something in your config file which is causing an import error on a test file | 22:37 |
| *** jecarey has quit IRC | 22:37 | |
| mtreinish | mariza10: that test runs testr list-tests on tempest to make sure we don't break that with a bad test | 22:37 |
| mariza10 | mtreinish: you mean ".testr.conf" ?? | 22:37 |
| mtreinish | which runs an import of everything | 22:37 |
| mtreinish | mariza10: no you said you changed your tempest.conf | 22:37 |
| mariza10 | yes.. | 22:38 |
| mtreinish | something in there is invalid and causing an import error | 22:38 |
| mtreinish | run testr list-tests and it should show the failure | 22:38 |
| mtreinish | fwiw, on master we've fixed that test to print the traceback | 22:38 |
| *** mriedem has quit IRC | 22:38 | |
| mtreinish | so you don't need to run testr list-tests manually anymore | 22:38 |
| * jlk blinks | 22:39 | |
| jlk | so.... | 22:39 |
| *** dboik has quit IRC | 22:39 | |
| jlk | I've got these swift tests in my run input file | 22:39 |
| jlk | and tempest just seems to ignore them | 22:39 |
| jlk | no mention of them in the output | 22:39 |
| jlk | WAT. | 22:39 |
| *** mriedem has joined #openstack-qa | 22:39 | |
| mariza10 | mtreinish: you mean this test ? https://github.com/openstack/tempest/blob/master/tempest/tests/test_list_tests.py | 22:39 |
| mtreinish | jlk: that normally has something to do with testr selection, if you tell testr to not run the test (with a regex or something) it just drops it from the list | 22:40 |
| jlk | oh maybe I did something wrong to the code | 22:40 |
| jlk | I think I messed up the python files for the test trying to get logging to work | 22:40 |
| *** dboik_ has quit IRC | 22:41 | |
| mtreinish | jlk: oh yeah that could do it too :) | 22:41 |
| *** marun has quit IRC | 22:41 | |
| mtreinish | mariza10: yes that's the unit test which is failing for you | 22:41 |
| mtreinish | mariza10: oh, actually there is a potential bug in the logging there which is why you probably aren't seeing the traceback | 22:42 |
| mtreinish | but it doesn't change the cause of your unit test failure | 22:43 |
| *** jog0 has joined #openstack-qa | 22:45 | |
| jlk | welp. An entire day spent on it, no closer to getting swift temp_url to work, nor resize down to work. | 22:49 |
| jlk | time to go home and drink until I can't think about it any more. | 22:50 |
| mtreinish | jlk: heh, sounds like a good plan :) | 22:50 |
| *** jog0 is now known as jogo | 22:52 | |
| openstackgerrit | Joe Gordon proposed openstack-infra/devstack-gate: Add dstat to subnode in multinode mode https://review.openstack.org/177970 | 22:54 |
| *** shashankhegde has quit IRC | 22:54 | |
| *** mwagner_lap has joined #openstack-qa | 22:54 | |
| *** suneelb has quit IRC | 22:56 | |
| *** suneelb1 has joined #openstack-qa | 22:56 | |
| mariza10 | mtreinish: i removed etc/tempest and ran `testr list-tests` and this is the output ..http://fpaste.org/216009/30174844/ | 22:56 |
| mariza10 | etc/tempest.conf* ...before removal and after that, same result of testr list-tests | 22:56 |
| *** suneelb has joined #openstack-qa | 22:56 | |
| mtreinish | mariza10: update testrepository, python-subunit, and testtools | 22:57 |
| mtreinish | you're running an older version which is why that's not useful | 22:57 |
| mariza10 | ah ok.. | 22:57 |
| mariza10 | let me try.. | 22:57 |
| *** rbak has quit IRC | 23:00 | |
| *** suneelb1 has quit IRC | 23:00 | |
| *** Sukhdev has quit IRC | 23:01 | |
| *** mlavalle has quit IRC | 23:02 | |
| *** shashankhegde has joined #openstack-qa | 23:06 | |
| *** yamamoto has joined #openstack-qa | 23:07 | |
| *** armax has joined #openstack-qa | 23:07 | |
| mariza10 | mtreinish: thanks ..updating worked as it showed clearly what were missing deps (like paramiko, pyyaml etc ) ..but there is just one test now failing in 'testr list-tests' ... | 23:08 |
| mariza10 | mtreinish: http://fpaste.org/216011/43017603/ | 23:08 |
| mtreinish | mariza10: well sort of, you were relying on your tox env to provide that before | 23:09 |
| mariza10 | mtreinish: ohk..any idea how this can be fixed ?? ImportError: No module named tempest_lib.cli | 23:10 |
| mariza10 | im using tempest tag 3.. | 23:10 |
| mtreinish | mariza10: run source .tox/py27/bin/activate | 23:11 |
| mtreinish | and then do a testr list-tests again | 23:11 |
| *** hemna is now known as hemnafk | 23:11 | |
| mariza10 | mtreinish: awesome..it worked...:) thanks ...i was working outside venv ..gahhh | 23:12 |
| mariza10 | thanks for the help..:D | 23:12 |
| mtreinish | sure, np | 23:13 |
| openstackrecheck | openstack-dev/devstack change: https://review.openstack.org/160856 failed gate-grenade-dsvm in the gate queue with an unrecognized error | 23:15 |
| openstackgerrit | Waldemar Znoinski proposed openstack-dev/devstack: Set datapath to $OVS_DATAPATH_TYPE for bridges https://review.openstack.org/159885 | 23:16 |
| openstackgerrit | Waldemar Znoinski proposed openstack-dev/devstack: Set datapath to $OVS_DATAPATH_TYPE for bridges https://review.openstack.org/159885 | 23:20 |
| *** sabeen1 has quit IRC | 23:20 | |
| *** dimtruck is now known as zz_dimtruck | 23:20 | |
| *** yamamoto has quit IRC | 23:21 | |
| openstackgerrit | Waldemar Znoinski proposed openstack-dev/devstack: Set datapath to $OVS_DATAPATH_TYPE for bridges https://review.openstack.org/159885 | 23:28 |
| *** erlon has quit IRC | 23:41 | |
| *** tochi has joined #openstack-qa | 23:44 | |
| *** dims has quit IRC | 23:46 | |
| *** armax has quit IRC | 23:49 | |
| *** yamamoto has joined #openstack-qa | 23:56 | |
| *** armax has joined #openstack-qa | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!