*** sabeen1 has quit IRC | 00:03 | |
*** gszasz has quit IRC | 00:04 | |
*** wramthun has quit IRC | 00:09 | |
*** dmorita has joined #openstack-qa | 00:09 | |
*** wramthun has joined #openstack-qa | 00:09 | |
*** sweston has joined #openstack-qa | 00:11 | |
openstackrecheck | Console logs not available after 13:30s for check-tempest-dsvm-full 105307,6,1c0da03 | 00:12 |
---|---|---|
*** gfidente has quit IRC | 00:16 | |
*** dims__ has quit IRC | 00:20 | |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 100421,17,929a189 | 00:26 |
openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 98688,25,93120c0 | 00:39 |
*** dims__ has joined #openstack-qa | 00:44 | |
*** salv-orlando has quit IRC | 00:50 | |
openstackrecheck | Console logs not available after 13:24s for gate-heat-pep8 100424,17,70e57ad | 00:53 |
openstackgerrit | Ken'ichi Ohmichi proposed a change to openstack/tempest: Merge duplicate code for network quotas tests https://review.openstack.org/104737 | 00:55 |
openstackgerrit | A change was merged to openstack/tempest: Add Ironic show driver API test https://review.openstack.org/90035 | 01:01 |
openstackrecheck | Console logs not available after 13:26s for gate-heat-pep8 103441,11,a5f3ab6 | 01:06 |
*** mmaglana has quit IRC | 01:11 | |
*** ekhugen has joined #openstack-qa | 01:16 | |
*** ekhugen has quit IRC | 01:18 | |
*** obondarev has joined #openstack-qa | 01:19 | |
openstackrecheck | Console logs not available after 13:21s for check-grenade-dsvm 85453,17,de555f3 | 01:19 |
*** aysyd has quit IRC | 01:23 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 97985,23,bf53e20 | 01:33 |
openstackgerrit | A change was merged to openstack-infra/devstack-gate: Stop doing tricky things with mirrors https://review.openstack.org/103363 | 01:36 |
*** nosnos has joined #openstack-qa | 01:44 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 98687,25,6026a3e | 01:46 |
*** zhiyan_ is now known as zhiyan | 01:47 | |
*** arnaud has joined #openstack-qa | 01:47 | |
*** Sukhdev has quit IRC | 01:51 | |
*** zhiyan is now known as zhiyan_ | 01:56 | |
*** zhiyan_ is now known as zhiyan | 01:56 | |
openstackrecheck | Console logs not available after 13:21s for gate-heat-pep8 101782,13,fd66ae4 | 02:00 |
*** sweston_ has joined #openstack-qa | 02:03 | |
*** jamielennox|away is now known as jamielennox | 02:12 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 97983,23,6c7dcc7 | 02:13 |
*** jyuso has joined #openstack-qa | 02:16 | |
*** arnaud has quit IRC | 02:25 | |
openstackrecheck | Console logs not available after 13:21s for gate-heat-pep8 101397,14,80556f5 | 02:26 |
openstackrecheck | Console logs not available after 13:21s for gate-heat-pep8 101396,14,ca27b0d | 02:40 |
*** oomichi has joined #openstack-qa | 02:45 | |
*** SridharG has joined #openstack-qa | 02:47 | |
*** harlowja is now known as harlowja_away | 02:50 | |
openstackrecheck | Console logs not available after 13:21s for gate-keystone-docs 105597,1,fe59650 | 02:53 |
*** mmaglana has joined #openstack-qa | 02:58 | |
*** sweston has quit IRC | 02:58 | |
*** sweston_ has quit IRC | 02:59 | |
*** arnaud has joined #openstack-qa | 03:01 | |
*** shardy has quit IRC | 03:01 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 92443,6,686fe04 | 03:06 |
*** SridharG has quit IRC | 03:13 | |
*** adalbas has quit IRC | 03:17 | |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python26 105598,1,026ff37 | 03:20 |
*** nosnos has quit IRC | 03:20 | |
*** arnaud has quit IRC | 03:20 | |
*** nati_ueno has quit IRC | 03:22 | |
*** arnaud has joined #openstack-qa | 03:27 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-neutron 105599,1,bebc160 | 03:33 |
*** arnaud has quit IRC | 03:42 | |
*** arnaud has joined #openstack-qa | 03:43 | |
openstackrecheck | Console logs not available after 13:21s for gate-grenade-dsvm 100556,2,4657bbf | 03:46 |
*** dims__ has quit IRC | 03:55 | |
*** arnaud has quit IRC | 03:59 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 105065,1,f7cb970 | 04:00 |
*** marun has joined #openstack-qa | 04:03 | |
*** mmaglana has quit IRC | 04:03 | |
*** yamahata has joined #openstack-qa | 04:03 | |
*** arnaud has joined #openstack-qa | 04:05 | |
*** nosnos has joined #openstack-qa | 04:13 | |
openstackrecheck | Console logs not available after 14:17s for gate-grenade-dsvm 105538,1,5b25a3e | 04:14 |
*** sweston has joined #openstack-qa | 04:15 | |
*** sweston_ has joined #openstack-qa | 04:17 | |
*** sweston__ has joined #openstack-qa | 04:17 | |
jogo | mtreinish: you around? | 04:18 |
*** sweston has quit IRC | 04:18 | |
jogo | hm maybe not its late | 04:18 |
*** sweston has joined #openstack-qa | 04:18 | |
*** gmann has joined #openstack-qa | 04:22 | |
*** nosnos has quit IRC | 04:24 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 98683,25,280fe61 | 04:28 |
*** yfried__ has quit IRC | 04:30 | |
*** amitpp has joined #openstack-qa | 04:32 | |
*** amitpp has quit IRC | 04:34 | |
openstackgerrit | Ghanshyam Mann proposed a change to openstack/tempest: Adds test for Sec group default rule Nova V2 API https://review.openstack.org/87924 | 04:35 |
*** weshay has quit IRC | 04:38 | |
*** amitpp has joined #openstack-qa | 04:38 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-pep8 105613,1,50dc5a5 | 04:41 |
*** SridharG has joined #openstack-qa | 04:50 | |
*** dims__ has joined #openstack-qa | 04:51 | |
*** sweston has quit IRC | 04:54 | |
*** sweston_ has quit IRC | 04:54 | |
*** sweston__ has quit IRC | 04:54 | |
openstackrecheck | Console logs not available after 13:21s for gate-tempest-dsvm-postgres-full 102028,2,37be75d | 04:54 |
*** dims__ has quit IRC | 04:56 | |
*** dims__ has joined #openstack-qa | 04:57 | |
*** sweston has joined #openstack-qa | 04:57 | |
*** sweston_ has joined #openstack-qa | 04:58 | |
*** sweston__ has joined #openstack-qa | 04:58 | |
openstackgerrit | Joe Gordon proposed a change to openstack/tempest: Make os-quota-class-sets test not break quotas https://review.openstack.org/105651 | 05:01 |
*** dims__ has quit IRC | 05:02 | |
*** ildikov has quit IRC | 05:02 | |
jogo | sdague mtreinish: ^ | 05:04 |
jogo | I have a hunch my patch needs at least one more revision | 05:04 |
openstackrecheck | Console logs not available after 13:21s for gate-horizon-python26 94706,41,249bb54 | 05:08 |
*** Madan has joined #openstack-qa | 05:13 | |
*** morganfainberg_Z is now known as morganfainberg | 05:18 | |
*** chandankumar has joined #openstack-qa | 05:19 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 104049,6,602cc22 | 05:21 |
openstackrecheck | Console logs not available after 13:21s for gate-cinder-python26 104703,4,f6a6367 | 05:34 |
openstackrecheck | openstack/python-openstackclient change: https://review.openstack.org/104946 failed because of: gate-tempest-dsvm-full: https://bugs.launchpad.net/bugs/1298472 | 05:35 |
*** yfried__ has joined #openstack-qa | 05:36 | |
*** nmagnezi has joined #openstack-qa | 05:37 | |
*** nmagnezi has quit IRC | 05:38 | |
*** k4n0 has joined #openstack-qa | 05:38 | |
*** nmagnezi has joined #openstack-qa | 05:38 | |
*** ildikov has joined #openstack-qa | 05:46 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 105510,2,b275566 | 05:48 |
*** Anju has joined #openstack-qa | 05:49 | |
*** coolsvap|afk is now known as coolsvap | 05:51 | |
*** nmagnezi_ has joined #openstack-qa | 05:55 | |
*** nmagnezi has quit IRC | 05:56 | |
*** dims__ has joined #openstack-qa | 05:58 | |
*** afazekas is now known as __afazekas | 05:58 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 105541,3,873bcd3 | 06:01 |
*** dims__ has quit IRC | 06:03 | |
*** xianghui has joined #openstack-qa | 06:07 | |
*** mkoderer has joined #openstack-qa | 06:09 | |
*** ajayaa has joined #openstack-qa | 06:13 | |
openstackrecheck | Console logs not available after 13:21s for gate-nova-python26 105613,2,4bf1117 | 06:15 |
openstackgerrit | Amit Ugol proposed a change to openstack/tempest: Added Heat Software Config-Deploy API tests https://review.openstack.org/89790 | 06:23 |
*** morganfainberg is now known as morganfainberg_Z | 06:23 | |
*** tkammer has joined #openstack-qa | 06:24 | |
*** marun has quit IRC | 06:27 | |
openstackrecheck | Console logs not available after 13:22s for gate-ceilometer-python26 105625,1,de16aaa | 06:28 |
*** Longgeek has joined #openstack-qa | 06:36 | |
openstackrecheck | Console logs not available after 14:18s for check-tempest-dsvm-full 94130,8,c7a8698 | 06:42 |
openstackgerrit | afazekas proposed a change to openstack-infra/devstack-gate: Neutron default tenant network type vxlan https://review.openstack.org/105569 | 06:49 |
*** arnaud has quit IRC | 06:53 | |
openstackrecheck | Console logs not available after 13:21s for gate-trove-pep8 105632,2,1f96c1d | 06:56 |
*** dims__ has joined #openstack-qa | 06:58 | |
openstackgerrit | Sridhar Gaddam proposed a change to openstack/qa-specs: Spec: Add Provider Networking Scenario tests https://review.openstack.org/104098 | 06:59 |
*** dims__ has quit IRC | 07:03 | |
*** xianghui has quit IRC | 07:04 | |
*** luqas has joined #openstack-qa | 07:05 | |
*** luqas has quit IRC | 07:07 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-neutron-icehouse 81707,8,113e814 | 07:09 |
*** luqas has joined #openstack-qa | 07:10 | |
openstackgerrit | Sridhar Gaddam proposed a change to openstack/qa-specs: Spec: Add Provider Networking Scenario tests https://review.openstack.org/104098 | 07:15 |
*** jlibosva has joined #openstack-qa | 07:20 | |
openstackrecheck | Console logs not available after 13:22s for gate-keystonemiddleware-pep8 105634,1,13ca5ba | 07:23 |
openstackrecheck | openstack/python-saharaclient change: https://review.openstack.org/100933 failed gate-tempest-dsvm-full in the gate queue with an unrecognized error | 07:23 |
*** afazekas has joined #openstack-qa | 07:25 | |
openstackrecheck | Console logs not available after 13:24s for gate-python-glanceclient-python27 82126,28,d7bd03a | 07:37 |
*** gfidente has joined #openstack-qa | 07:39 | |
yfried__ | andreaf_: I think this one https://review.openstack.org/100143 could gain a lot from the refactored scenario tree. it shares a lot of functionality and setup with network_basic_ops specifically. | 07:46 |
*** shardy has joined #openstack-qa | 07:47 | |
openstackrecheck | Console logs not available after 13:23s for gate-grenade-dsvm 100776,3,1e303c8 | 07:50 |
*** masayukig has joined #openstack-qa | 07:54 | |
*** ildikov has quit IRC | 07:55 | |
*** safchain has joined #openstack-qa | 07:58 | |
*** jpich has joined #openstack-qa | 07:58 | |
*** ildikov has joined #openstack-qa | 07:58 | |
*** dims__ has joined #openstack-qa | 07:59 | |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-full 105635,3,8464ab6 | 08:04 |
*** pcrews has quit IRC | 08:04 | |
*** dims__ has quit IRC | 08:04 | |
*** xianghui has joined #openstack-qa | 08:06 | |
openstackrecheck | Console logs not available after 13:58s for gate-python-ceilometerclient-pypy 66776,13,f2ab021 | 08:18 |
*** zz_johnthetubagu is now known as johnthetubaguy | 08:26 | |
*** dims__ has joined #openstack-qa | 08:28 | |
openstackrecheck | Console logs not available after 13:24s for gate-python-heatclient-python27 105110,4,ec1a70d | 08:31 |
*** qba73 has joined #openstack-qa | 08:31 | |
*** jordanP has joined #openstack-qa | 08:32 | |
*** dims__ has quit IRC | 08:34 | |
*** Dafna_away is now known as Dafna | 08:34 | |
*** nmagnezi_ has quit IRC | 08:36 | |
*** gszasz has joined #openstack-qa | 08:37 | |
*** nmagnezi has joined #openstack-qa | 08:37 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-neutron-icehouse 81707,8,f6ed4c0 | 08:44 |
openstackgerrit | Sébastien Han proposed a change to openstack-dev/devstack: Implement Ceph backend for Glance / Cinder / Nova https://review.openstack.org/65113 | 08:44 |
openstackgerrit | Sébastien Han proposed a change to openstack-dev/devstack: Implement Ceph backend for Glance / Cinder / Nova https://review.openstack.org/65113 | 08:47 |
*** tosky has joined #openstack-qa | 08:59 | |
openstackgerrit | A change was merged to openstack-infra/devstack-gate: No more thinging in functions.sh https://review.openstack.org/105350 | 09:04 |
openstackrecheck | Console logs not available after 13:23s for gate-grenade-dsvm 101798,2,7b60132 | 09:10 |
*** jyuso has quit IRC | 09:22 | |
*** k4n0 has quit IRC | 09:24 | |
openstackrecheck | Console logs not available after 13:24s for gate-python-glanceclient-pep8 100440,4,6abbeb6 | 09:24 |
*** Longgeek has quit IRC | 09:27 | |
*** dims__ has joined #openstack-qa | 09:30 | |
*** ajayaa has quit IRC | 09:32 | |
openstackgerrit | Amit Ugol proposed a change to openstack/tempest: Added Heat Software Config-Deploy API tests https://review.openstack.org/89790 | 09:33 |
*** dims__ has quit IRC | 09:35 | |
*** k4n0 has joined #openstack-qa | 09:35 | |
*** sweston__ has quit IRC | 09:36 | |
*** sweston has quit IRC | 09:36 | |
*** sweston_ has quit IRC | 09:36 | |
openstackrecheck | Console logs not available after 13:22s for gate-nova-pep8 102064,10,85eca22 | 09:38 |
*** Anju has quit IRC | 09:40 | |
*** ajayaa has joined #openstack-qa | 09:45 | |
*** vrovachev has joined #openstack-qa | 09:46 | |
kashyap | afazekas, When you get a min, what do you think about this change -- https://review.openstack.org/#/c/105473/ I don't feel very strongly about this change (i.e. I'm fine with it despite my comment there) | 09:48 |
*** yamahata has quit IRC | 09:48 | |
*** SridharG has quit IRC | 09:51 | |
openstackrecheck | Console logs not available after 13:24s for gate-keystone-python26 105652,1,7dfbe29 | 09:52 |
openstackrecheck | openstack-infra/devstack-gate change: https://review.openstack.org/105350 failed gate-tempest-dsvm-postgres-full in the gate queue with an unrecognized error | 09:52 |
afazekas | kashyap: Which part is not done by devstack ? | 09:55 |
kashyap | afazekas, Configuring of EPEL repos | 09:56 |
kashyap | For getting python-pip | 09:56 |
*** ildikov has quit IRC | 09:57 | |
kashyap | Instead Matt suggests to download a script from here to avoid configuring systems with EPEL | 09:57 |
kashyap | https://bootstrap.pypa.io/get-pip.py | 09:57 |
*** Longgeek has joined #openstack-qa | 09:58 | |
*** Longgeek has quit IRC | 10:05 | |
openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 104026,5,380a8dc | 10:06 |
afazekas | keshyap: python-pip-1.3.1-4.el6.noarch.rpm in epel-6 pytthon-pip>=1.4.1 required | 10:10 |
afazekas | the `easy_install` would be another way.. | 10:12 |
*** ildikov has joined #openstack-qa | 10:16 | |
openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 101798,2,dd891fa | 10:19 |
*** MIDENN_ has quit IRC | 10:23 | |
*** dims__ has joined #openstack-qa | 10:31 | |
openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-full 104606,4,52ec3e5 | 10:33 |
*** tkammer has quit IRC | 10:34 | |
*** dims__ has quit IRC | 10:36 | |
*** NuclearW0rm has joined #openstack-qa | 10:38 | |
*** Shrews has quit IRC | 10:43 | |
NuclearW0rm | hi folks | 10:43 |
NuclearW0rm | how can print to stdout from tempest tests? or it's better to use logging? | 10:43 |
*** NuclearW0rm is now known as sergeysh | 10:44 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 103380,11,29a9423 | 10:46 |
*** Shrews has joined #openstack-qa | 10:47 | |
*** yamahata has joined #openstack-qa | 10:47 | |
*** xianghui has quit IRC | 10:49 | |
*** tkammer has joined #openstack-qa | 10:50 | |
*** weshay has joined #openstack-qa | 10:57 | |
openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 105069,3,c5e817c | 11:00 |
*** Anju has joined #openstack-qa | 11:00 | |
*** xianghui has joined #openstack-qa | 11:03 | |
*** gmann has quit IRC | 11:08 | |
*** dims__ has joined #openstack-qa | 11:10 | |
openstackrecheck | Console logs not available after 13:23s for gate-python-novaclient-pep8 105658,1,d5173ae | 11:13 |
openstackgerrit | Sean Dague proposed a change to openstack/tempest: Make os-quota-class-sets test not break quotas https://review.openstack.org/105651 | 11:14 |
*** tkammer has quit IRC | 11:15 | |
sdague | afazekas: can you look at that | 11:15 |
afazekas | kashyap: FYI the pip install is changed recently : https://review.openstack.org/#/c/103735/2 | 11:18 |
afazekas | sdague: -1 to 0 :) or :( ; good catch, I did not expected it.. | 11:21 |
sdague | yeh, jogo figured it out | 11:23 |
sdague | but his initial patch was a bit more intrusive, so I cleaned it up | 11:23 |
afazekas | sdague: Co-Authored-By: ? | 11:26 |
afazekas | sdague: Per the current commit message you are both the Author Committer and Co-Author | 11:27 |
openstackrecheck | Console logs not available after 13:27s for check-grenade-dsvm-icehouse 89790,14,7d388a4 | 11:27 |
kashyap | How do you frequently do "git pull" of /opt/stack repos? Or do you use 'RECLONE=yes' to do a fresh clone on every DevStack install? | 11:28 |
Anju | sdague, afazekas : Hii, I try to use devstack with oracle distribution, so came to know that devstack is not supported for oracle, although it is working for me now.so could we enable this?? | 11:29 |
kashyap | I use this crude script to do that - http://kashyapc.fedorapeople.org/virt/openstack/update-git.bash | 11:29 |
afazekas | sdague: can you fix the commit message ? | 11:31 |
openstackgerrit | Ajaya Agrawal proposed a change to openstack/tempest: Running without keystone v3 doesn't require uri_v3 https://review.openstack.org/105706 | 11:32 |
sdague | afazekas: I'm not the Author | 11:33 |
sdague | oh, I am the author | 11:34 |
sdague | wtf | 11:34 |
sdague | oh.... right I know why | 11:34 |
sdague | afazekas: yes, I will fix | 11:34 |
openstackgerrit | Sean Dague proposed a change to openstack/tempest: Make os-quota-class-sets test not break quotas https://review.openstack.org/105651 | 11:35 |
Anju | sdague: ^^^ | 11:38 |
*** Longgeek has joined #openstack-qa | 11:38 | |
openstackrecheck | Console logs not available after 13:30s for gate-nova-python26 94295,27,21f9191 | 11:41 |
*** tkammer has joined #openstack-qa | 11:44 | |
sdague | Anju: oracle's distro is basically just centos right? | 11:45 |
Anju | yes | 11:45 |
Anju | same packages. | 11:46 |
sdague | Anju: if it's only a flag to make is_fedora return true on oracle, I'm fine with it. However if it gets any more complicated than that, will have to discuss it | 11:46 |
Anju | I just change Two line in the script , that was working for me . | 11:46 |
Anju | yeah. I will raise a bug . whatever you think you can suggest their . | 11:47 |
sdague | Anju: a devstack patch is fine | 11:48 |
openstackgerrit | Cedric Brandily proposed a change to openstack/tempest: Add multi-provider extension positive test cases https://review.openstack.org/104526 | 11:48 |
openstackgerrit | Cedric Brandily proposed a change to openstack/tempest: Add partial specs (multi-)provider extension positive test cases https://review.openstack.org/105234 | 11:49 |
Anju | thankyou sean | 11:50 |
openstackrecheck | Console logs not available after 13:25s for check-devstack-dsvm-cells 105661,1,7e024c9 | 11:55 |
*** gszasz has quit IRC | 12:03 | |
*** dmorita has quit IRC | 12:03 | |
*** gszasz has joined #openstack-qa | 12:04 | |
*** Poornima has joined #openstack-qa | 12:04 | |
*** gszasz has quit IRC | 12:05 | |
*** gszasz has joined #openstack-qa | 12:05 | |
openstackgerrit | Cedric Brandily proposed a change to openstack/tempest: Add partial specs (multi-)provider extension positive test cases https://review.openstack.org/105234 | 12:05 |
*** nmagnezi has quit IRC | 12:12 | |
*** ekhugen has joined #openstack-qa | 12:16 | |
openstackrecheck | Console logs not available after 13:24s for gate-python-glanceclient-pypy 105468,2,6d4d53b | 12:17 |
*** aysyd has joined #openstack-qa | 12:23 | |
*** ArxCruz has joined #openstack-qa | 12:26 | |
*** nmagnezi has joined #openstack-qa | 12:28 | |
openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm 86115,10,184dbc4 | 12:31 |
*** cdent has joined #openstack-qa | 12:31 | |
*** dims__ has quit IRC | 12:32 | |
*** dims__ has joined #openstack-qa | 12:32 | |
*** mwagner_lap has joined #openstack-qa | 12:36 | |
*** cdent has quit IRC | 12:36 | |
yfried__ | I noticed that sometimes, testr workers might fail and decrease concurrency. is this a known issue? | 12:38 |
yfried__ | - WARNING: missing Worker 6! Race in testr accounting. | 12:38 |
*** adalbas has joined #openstack-qa | 12:38 | |
yfried__ | afazekas: http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiUmFjZSBpbiB0ZXN0ciBhY2NvdW50aW5nLlwiIiwiZmllbGRzIjpbXSwib2Zmc2V0IjowLCJ0aW1lZnJhbWUiOiJhbGwiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxNDA0OTA5NjA4MDA1LCJtb2RlIjoiIiwiYW5hbHl6ZV9maWVsZCI6IiJ9 | 12:41 |
*** cdent has joined #openstack-qa | 12:42 | |
*** mkoderer has quit IRC | 12:42 | |
openstackrecheck | Console logs not available after 13:23s for gate-tempest-dsvm-full 105628,1,cbd075f | 12:44 |
*** nmagnezi has quit IRC | 12:44 | |
*** nmagnezi has joined #openstack-qa | 12:45 | |
mtreinish | yfried__: that's just a counting error in testr, the workers are still there | 12:47 |
mtreinish | yfried__: https://bugs.launchpad.net/testrepository/+bug/1316858 | 12:48 |
yfried__ | mtreinish: I ran it with 2 workers and lost one, seems like all tests were executed by the remaining worker => serial run | 12:48 |
*** dkranz has joined #openstack-qa | 12:49 | |
mtreinish | yfried__: the concurrency didn't change it's still running with 2 workers | 12:50 |
mtreinish | it's just the worker # being reported is getting overwritten by testr incorrectly | 12:50 |
mtreinish | so it looks like it's serial | 12:50 |
mtreinish | look at the logs carefully you'll see 2 worker threads started | 12:51 |
yfried__ | tnx\ | 12:53 |
yfried__ | tnx | 12:53 |
yfried__ | mtreinish: tnx | 12:53 |
openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-neutron-heat-slow 104430,2,3c4c543 | 12:58 |
*** yfried__ has quit IRC | 12:58 | |
openstackgerrit | Sébastien Han proposed a change to openstack-dev/devstack: Implement Ceph backend for Glance / Cinder / Nova https://review.openstack.org/65113 | 12:59 |
vrovachev | mtreinish: hi, please see my answers in your spare time: https://review.openstack.org/#/c/71259/ | 13:00 |
*** sballe has joined #openstack-qa | 13:01 | |
*** asselin has quit IRC | 13:07 | |
*** asselin has joined #openstack-qa | 13:07 | |
*** Anju has quit IRC | 13:10 | |
*** oomichi has quit IRC | 13:10 | |
*** zul has quit IRC | 13:11 | |
openstackrecheck | Console logs not available after 13:22s for gate-tempest-dsvm-full 105361,2,1322906 | 13:11 |
*** ildikov has quit IRC | 13:14 | |
*** amotoki has quit IRC | 13:16 | |
*** nsaje is now known as nsaje-commute | 13:17 | |
*** mriedem has joined #openstack-qa | 13:18 | |
*** zul has joined #openstack-qa | 13:18 | |
sdague | mtreinish: so I posted a very long, different set of thoughts to eglynn-offic-afk's email, which I think is a different POV than you have. | 13:18 |
sdague | just read yours after mine went out | 13:19 |
mtreinish | sdague: ok, different opinions are good :) | 13:21 |
mtreinish | I'll read yours when I get it | 13:21 |
*** Madan has quit IRC | 13:23 | |
*** Longgeek has quit IRC | 13:25 | |
openstackrecheck | Console logs not available after 13:23s for gate-python-glanceclient-python26 105468,3,11c415a | 13:25 |
*** masayukig has quit IRC | 13:27 | |
sdague | mriedem: did you see that jogo figured out the fixed ip thing? :) | 13:28 |
sdague | mriedem: https://review.openstack.org/#/c/105651/ | 13:28 |
mriedem | sdague: nope, just got in | 13:28 |
mtreinish | sdague: oh, so it's mriedem fault then :) | 13:28 |
*** jgrimm has joined #openstack-qa | 13:28 | |
mriedem | yay | 13:28 |
*** Obulpathi has joined #openstack-qa | 13:29 | |
*** Obulpathi has quit IRC | 13:29 | |
*** Obulpathi has joined #openstack-qa | 13:29 | |
*** Obulpathi has quit IRC | 13:29 | |
*** rodrigods has joined #openstack-qa | 13:30 | |
*** rodrigods has joined #openstack-qa | 13:30 | |
mriedem | so does the lock not work, or unrelated? | 13:30 |
*** Obulpathi has joined #openstack-qa | 13:30 | |
mriedem | oh is devstack setting -1 for default quota (unlimited)? | 13:30 |
mtreinish | mriedem: it's not enough, because the quotas effect the server creates | 13:31 |
mtreinish | the lock only prevents us from overlapping with the other quota tests | 13:31 |
*** dustins has joined #openstack-qa | 13:31 | |
mtreinish | but that test really isn't safe for all of compute | 13:31 |
mtreinish | sdague: I think we have basically the same opinion on the branchless tempest side of things. | 13:32 |
mtreinish | I think the only real difference was I said the ceilo tests could have a place in tempest | 13:32 |
mriedem | mtreinish: sdague: ok i see so set it artificially high to avoid over quota as a result of racing with that test | 13:32 |
mriedem | i'm surprised that didn't hit in jenkins runs of the tempest change | 13:33 |
mtreinish | sdague: oh and then there was your rather large section about being burnt out on gate debug :) | 13:33 |
*** masayukig has joined #openstack-qa | 13:34 | |
*** rodrigods has quit IRC | 13:34 | |
*** Longgeek has joined #openstack-qa | 13:35 | |
mtreinish | mriedem: yeah it's just more heinsenbug races. We'll never see it when we're looking for it | 13:35 |
mriedem | i see that devstack sets default quotas to -1 if using the fake virt driver in nova, is that used in the gate? assuming only for the large ops test? | 13:36 |
mtreinish | yeah the fake driver should only be with the large ops tests | 13:36 |
mriedem | hmm, so how would we get a quota of -1 in a dsvm tempest full run? | 13:37 |
sdague | I thought the new default was -1 | 13:37 |
mriedem | in nova? news to me. | 13:37 |
mriedem | looking | 13:37 |
mtreinish | mriedem: don't we log the conf files? | 13:37 |
mriedem | mtreinish: yeah looking | 13:38 |
mriedem | sdague: mtreinish: http://logs.openstack.org/57/105257/4/check/check-tempest-dsvm-full/9bf9ad1/logs/screen-n-cpu.txt.gz#_2014-07-08_00_33_43_678 | 13:38 |
mriedem | quota_instances=10 by default there | 13:38 |
openstackrecheck | Console logs not available after 13:23s for gate-python-keystoneclient-python27 96763,4,95dd575 | 13:39 |
mriedem | so the -1 default quota should only be an issue in the large ops test | 13:39 |
*** tkammer has quit IRC | 13:39 | |
*** dims_ has joined #openstack-qa | 13:39 | |
*** markmcclain has joined #openstack-qa | 13:39 | |
*** pcrews has joined #openstack-qa | 13:40 | |
mriedem | confirming large ops http://logs.openstack.org/57/105257/4/check/gate-tempest-dsvm-large-ops/8b48553/logs/screen-n-cpu.txt.gz#_2014-07-08_00_37_48_536 | 13:40 |
mriedem | checking where that was failing mostly, if it was large ops then we're good | 13:41 |
*** eglynn-regus has joined #openstack-qa | 13:41 | |
*** yfried__ has joined #openstack-qa | 13:41 | |
*** davidlenwell_ has joined #openstack-qa | 13:41 | |
*** yfried__ has quit IRC | 13:41 | |
*** yfried__ has joined #openstack-qa | 13:42 | |
*** lifeless1 has joined #openstack-qa | 13:43 | |
*** SpamapS_ has joined #openstack-qa | 13:44 | |
mriedem | oh nevermind yeah default quota for fixed_ips is -1 | 13:44 |
mriedem | so this should fix it, nice work jogo! | 13:44 |
*** cnesa has joined #openstack-qa | 13:44 | |
mriedem | and vishy was on the case too | 13:45 |
*** dkranz has quit IRC | 13:45 | |
*** rodrigods has joined #openstack-qa | 13:45 | |
mriedem | maybe we should have made the quota 255? i think the fixed_ips table is loaded with a range of 256 | 13:46 |
*** MmikeM has joined #openstack-qa | 13:46 | |
*** mitz- has joined #openstack-qa | 13:47 | |
*** dansmith_ has joined #openstack-qa | 13:47 | |
*** mordred_ has joined #openstack-qa | 13:47 | |
*** nmagnezi has quit IRC | 13:48 | |
*** dims__ has quit IRC | 13:48 | |
*** Poornima has quit IRC | 13:48 | |
*** gema has quit IRC | 13:48 | |
*** coolsvap has quit IRC | 13:48 | |
*** Mmike has quit IRC | 13:48 | |
*** mitz has quit IRC | 13:48 | |
*** krtaylor has quit IRC | 13:48 | |
*** mordred has quit IRC | 13:48 | |
*** dansmith has quit IRC | 13:48 | |
*** mordred_ is now known as mordred | 13:48 | |
*** jogo has quit IRC | 13:48 | |
*** eezhova has quit IRC | 13:48 | |
*** davidlenwell has quit IRC | 13:48 | |
*** SpamapS has quit IRC | 13:48 | |
*** lifeless has quit IRC | 13:48 | |
*** anteaya has quit IRC | 13:48 | |
*** sdague has quit IRC | 13:48 | |
*** eglynn-offic-afk has quit IRC | 13:48 | |
*** dansmith_ is now known as dansmith | 13:48 | |
*** jogo has joined #openstack-qa | 13:49 | |
mtreinish | mriedem: what if there is more than one /24 though? | 13:49 |
*** Poornima has joined #openstack-qa | 13:49 | |
*** gema has joined #openstack-qa | 13:51 | |
*** cnesa2 has joined #openstack-qa | 13:51 | |
*** cnesa has quit IRC | 13:51 | |
*** andreaf has joined #openstack-qa | 13:51 | |
*** rodrigods has quit IRC | 13:52 | |
*** yamahata has quit IRC | 13:52 | |
*** yamahata has joined #openstack-qa | 13:53 | |
openstackrecheck | Console logs not available after 13:25s for gate-tempest-dsvm-full 105477,1,faa4423 | 13:53 |
*** cpallares has joined #openstack-qa | 13:53 | |
*** sdague has joined #openstack-qa | 13:53 | |
*** eezhova has joined #openstack-qa | 13:54 | |
*** anteaya has joined #openstack-qa | 13:55 | |
*** tkammer has joined #openstack-qa | 13:55 | |
afazekas | sdague: Can you be more specific what is the expected change ? https://review.openstack.org/#/c/98178/ | 13:55 |
*** nmagnezi has joined #openstack-qa | 13:56 | |
*** markmcclain has quit IRC | 13:56 | |
sdague | afazekas: ok, it wasn't clear from the commit message that it was a parity patch | 13:57 |
sdague | in the general case we shouldn't unconditionally install that I think, we should only do it when certain services exist | 13:58 |
*** coolsvap has joined #openstack-qa | 14:01 | |
*** cnesa2 has quit IRC | 14:01 | |
*** mordred has quit IRC | 14:02 | |
*** mordred has joined #openstack-qa | 14:02 | |
*** MmikeM is now known as Mmike | 14:03 | |
*** Mmike has quit IRC | 14:03 | |
*** Mmike has joined #openstack-qa | 14:03 | |
*** rbak has joined #openstack-qa | 14:04 | |
*** markmcclain has joined #openstack-qa | 14:06 | |
openstackrecheck | Console logs not available after 13:25s for check-grenade-dsvm 105665,4,eb89bb3 | 14:06 |
*** krtaylor has joined #openstack-qa | 14:09 | |
mtreinish | mriedem: on https://review.openstack.org/#/c/99984/ can you just respin it quickly to address oomichi's concern | 14:10 |
mtreinish | just what is fair game to decorate | 14:10 |
*** obondarev has quit IRC | 14:11 | |
mriedem | mtreinish: yeah, was going to get to that today | 14:11 |
*** amitpp has quit IRC | 14:12 | |
mtreinish | mriedem: cool, thanks | 14:12 |
mtreinish | today's a good day to do it, being the qa-specs review day :) | 14:12 |
mriedem | doing it now | 14:12 |
mriedem | sad part is i'm still having some weirdness in the code, but i guess that can be worked | 14:13 |
mtreinish | mriedem: yeah I'm sure we can figure out the code | 14:15 |
*** mdenny has joined #openstack-qa | 14:17 | |
openstackgerrit | Matt Riedemann proposed a change to openstack/qa-specs: Check minimum version for CLI tests https://review.openstack.org/99984 | 14:17 |
mriedem | let me know about that wording ^ | 14:18 |
mtreinish | mriedem: it looks fine to me | 14:19 |
openstackrecheck | Console logs not available after 13:25s for gate-tempest-dsvm-large-ops 103064,6,0b9dc5a | 14:20 |
openstackgerrit | Cedric Brandily proposed a change to openstack/tempest: Add partial specs (multi-)provider extension positive test cases https://review.openstack.org/105234 | 14:20 |
*** ajayaa has quit IRC | 14:21 | |
*** jaypipes has joined #openstack-qa | 14:27 | |
*** ChanServ sets mode: +o jaypipes | 14:27 | |
cpallares | sdague: ping | 14:29 |
*** Poornima has left #openstack-qa | 14:32 | |
openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds list users tests in v3 https://review.openstack.org/93704 | 14:32 |
kashyap | cpallares, You might want to ping with context. And, a friendly reminder :-) -- http://blogs.gnome.org/markmc/2014/02/20/naked-pings/ | 14:33 |
openstackrecheck | Console logs not available after 13:24s for gate-tempest-dsvm-neutron 105476,1,9f9e70e | 14:34 |
openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: add glance deadlock query https://review.openstack.org/105752 | 14:34 |
sdague | mriedem / mtreinish ^^^ | 14:36 |
mriedem | sdague: already got it | 14:36 |
*** mkoderer has joined #openstack-qa | 14:37 | |
mriedem | i'll push some e-r queries for uncategorized nova unit test gate fails shortly | 14:38 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: add glance deadlock query https://review.openstack.org/105752 | 14:39 |
sdague | mriedem: cool | 14:39 |
*** rodrigods has joined #openstack-qa | 14:39 | |
*** rodrigods has quit IRC | 14:39 | |
*** rodrigods has joined #openstack-qa | 14:39 | |
openstackgerrit | A change was merged to openstack/tempest: Make os-quota-class-sets test not break quotas https://review.openstack.org/105651 | 14:41 |
openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova unit test bug 1339739 https://review.openstack.org/105755 | 14:43 |
cpallares | kashyap: My bad, thanks for the link. | 14:44 |
openstackgerrit | nayna-patel proposed a change to openstack/tempest: Adds list users tests in v3 https://review.openstack.org/93704 | 14:45 |
*** rodrigods has quit IRC | 14:45 | |
*** gszasz has quit IRC | 14:45 | |
*** gszasz has joined #openstack-qa | 14:46 | |
*** morganfainberg_Z is now known as morganfainberg | 14:46 | |
openstackrecheck | Console logs not available after 13:24s for gate-cinder-docs 104445,1,516e377 | 14:47 |
*** obondarev has joined #openstack-qa | 14:48 | |
mtreinish | sdague: do you want to give mriedem's spec a review: https://review.openstack.org/99984 if it's all good then I'll push it through | 14:48 |
cpallares | sdague, kashyap: Hi! I'm an intern working on glance and I'm having an issue with tempest checks, they are failing on my patch for what appears to be reasons unrelated to my patch. I checked the logs a couple of times and I can't find the the specific failures related to the code I'm modifying https://review.openstack.org/#/c/98862/ | 14:48 |
mriedem | cpallares: did you ask for help in #openstack-glance first? | 14:50 |
mriedem | cpallares: this looks like the failure on setup http://logs.openstack.org/62/98862/8/check/check-tempest-dsvm-full/fda0968/logs/devstacklog.txt.gz#_2014-07-08_16_48_08_657 | 14:50 |
mriedem | cpallares: my guess is you're removing a shell command that devstack is using | 14:51 |
mriedem | so you'd have to cleanup devstack first | 14:51 |
cpallares | mriedem: Hi, yes they all seem to fail at setup. | 14:51 |
*** openstackrecheck has quit IRC | 14:51 | |
mriedem | cpallares: yeah, you're breaking devstack :) | 14:51 |
*** openstackrecheck has joined #openstack-qa | 14:51 | |
mriedem | is my bet at least | 14:51 |
kashyap | cpallares, No worries. You're in good hands of mriedem. I'm afraid I've to run out for dinner, it's a bit late here. | 14:51 |
cpallares | kashyap: Enjoy your dinner | 14:51 |
openstackgerrit | A change was merged to openstack/qa-specs: QA spec for bp:add-basic-tests-for-barbican https://review.openstack.org/99978 | 14:51 |
cpallares | mriedem: Thanks you, I'll check in with the devstack people then. | 14:52 |
mriedem | cpallares: the devstack people are here... | 14:52 |
mriedem | cpallares: just scan the code in devstack for whatever commands you're removing | 14:53 |
mriedem | http://git.openstack.org/cgit/openstack-dev/devstack/ | 14:53 |
cpallares | mriedem: Ah thanks! | 14:53 |
sdague | cpallares: have you read the logs yet? | 14:55 |
mriedem | jogo: is the uncategorized bugs page going back 2 weeks? looks like it must - shouldn't that be 10 days now? | 14:55 |
mriedem | reason i ask is there is a nova unit test fail blip that is over 7 days old and is already fixed | 14:55 |
sdague | cpallares: because it seems pretty clear where the failure - http://logs.openstack.org/62/98862/7/check/check-tempest-dsvm-full/79b87ac/logs/devstacklog.txt.gz#_2014-07-07_20_10_37_766 | 14:56 |
*** cdent has quit IRC | 14:56 | |
mtreinish | mriedem: didn't sdague just push the fix that switched it to 10 days from 14? | 14:57 |
sdague | mriedem: uncategorized uses ES to find 'all fails' | 14:57 |
mriedem | mtreinish: yeah, but the status pages are going back 2 weeks | 14:57 |
sdague | so it will always match the dataset in ES | 14:57 |
mriedem | ok | 14:58 |
*** cdent has joined #openstack-qa | 14:58 | |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova unit test bug 1339739 https://review.openstack.org/105755 | 14:58 |
mriedem | so i guess i can push a query to get this one nova unit test fail that is fixed off the uncategorized bugs page | 14:58 |
openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: Change of copyright to HP for keystone regions https://review.openstack.org/105759 | 14:59 |
*** Sukhdev has joined #openstack-qa | 15:00 | |
openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Categorize fixed fail for nova bug 1330431 https://review.openstack.org/105760 | 15:01 |
*** HenryG has joined #openstack-qa | 15:02 | |
*** weshay has quit IRC | 15:02 | |
*** afazekas has quit IRC | 15:04 | |
openstackgerrit | Ravikumar Venkatesan proposed a change to openstack/tempest: Change of copyright to HP for keystone regions https://review.openstack.org/105759 | 15:04 |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 105743,1,7910298 | 15:05 |
*** sabeen1 has joined #openstack-qa | 15:06 | |
*** openstackrecheck has quit IRC | 15:09 | |
*** openstackrecheck has joined #openstack-qa | 15:09 | |
*** zhiyan is now known as zhiyan_ | 15:10 | |
openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Categorize horizon gate rebase unit test fail https://review.openstack.org/105765 | 15:12 |
*** jlibosva has quit IRC | 15:13 | |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Categorize fixed fail for nova bug 1330431 https://review.openstack.org/105760 | 15:16 |
*** afazekas has joined #openstack-qa | 15:18 | |
*** SpamapS_ is now known as SpamapS | 15:21 | |
*** openstackrecheck has quit IRC | 15:21 | |
*** openstackrecheck has joined #openstack-qa | 15:21 | |
*** SpamapS has quit IRC | 15:21 | |
*** SpamapS has joined #openstack-qa | 15:21 | |
*** sballe has quit IRC | 15:21 | |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Categorize horizon gate rebase unit test fail https://review.openstack.org/105765 | 15:21 |
*** jaypipes is now known as jay-house-hunter | 15:22 | |
*** cdent_ has joined #openstack-qa | 15:22 | |
*** cdent_ has quit IRC | 15:23 | |
*** nmagnezi has quit IRC | 15:25 | |
*** cdent has quit IRC | 15:25 | |
*** cdent has joined #openstack-qa | 15:25 | |
*** openstackrecheck has quit IRC | 15:35 | |
*** openstackrecheck has joined #openstack-qa | 15:36 | |
*** Longgeek has quit IRC | 15:36 | |
*** masayukig has quit IRC | 15:37 | |
*** Longgeek has joined #openstack-qa | 15:37 | |
*** nmagnezi has joined #openstack-qa | 15:37 | |
openstackgerrit | Sean Dague proposed a change to openstack-infra/elastic-recheck: change to correct quoting, ER doesn't seem to pick this up https://review.openstack.org/105770 | 15:40 |
*** Longgeek has quit IRC | 15:40 | |
*** Longgeek has joined #openstack-qa | 15:40 | |
sdague | mriedem: I'm really confused why that isn't taking, but I think the quoting might be important | 15:40 |
*** Obulpathi has quit IRC | 15:40 | |
mriedem | sdague: i was wondering about that, but it did show up in kibana so figured it was ok | 15:41 |
sdague | yeh | 15:41 |
sdague | well I just fast approved the quote change | 15:41 |
sdague | if that's a thing, so be it | 15:41 |
*** masayukig has joined #openstack-qa | 15:42 | |
*** mlavalle has joined #openstack-qa | 15:46 | |
*** marun has joined #openstack-qa | 15:46 | |
openstackrecheck | Console logs not available after 13:23s for check-grenade-dsvm-partial-ncpu 104148,6,3c06948 | 15:49 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: change to correct quoting, ER doesn't seem to pick this up https://review.openstack.org/105770 | 15:53 |
*** afazekas has quit IRC | 15:58 | |
*** dkranz has joined #openstack-qa | 15:58 | |
*** safchain has quit IRC | 16:00 | |
*** sergeysh has quit IRC | 16:00 | |
*** k4n0 has quit IRC | 16:01 | |
openstackrecheck | Console logs not available after 13:22s for check-grenade-dsvm 96184,4,db6191c | 16:03 |
*** jay-house-hunter has quit IRC | 16:05 | |
*** afazekas has joined #openstack-qa | 16:06 | |
*** openstackrecheck has quit IRC | 16:06 | |
*** openstackrecheck has joined #openstack-qa | 16:06 | |
*** arnaud has joined #openstack-qa | 16:10 | |
openstackgerrit | Prashanth Hari proposed a change to openstack/tempest: Added function to get region from service contexts https://review.openstack.org/105777 | 16:11 |
*** vrovachev has quit IRC | 16:11 | |
openstackgerrit | Prashanth Hari proposed a change to openstack/tempest: Added function to get region from service contexts https://review.openstack.org/105777 | 16:12 |
openstackgerrit | Prashanth Hari proposed a change to openstack/tempest: Added function to get region from service contexts https://review.openstack.org/105782 | 16:13 |
*** chandankumar has quit IRC | 16:18 | |
*** coolsvap is now known as coolsvap|afk | 16:18 | |
*** nati_ueno has joined #openstack-qa | 16:19 | |
openstackrecheck | Console logs not available after 13:28s for gate-tempest-dsvm-postgres-full 100988,6,0b0b2ff | 16:21 |
*** SridharG has joined #openstack-qa | 16:21 | |
*** arnaud has quit IRC | 16:23 | |
*** jpich has quit IRC | 16:25 | |
openstackgerrit | Kyle Mestery proposed a change to openstack-dev/devstack: Make neutron the default in devstack https://review.openstack.org/105785 | 16:28 |
*** qba73 has quit IRC | 16:32 | |
openstackrecheck | Console logs not available after 13:29s for check-tempest-dsvm-postgres-full 105745,2,70ce7c6 | 16:34 |
openstackgerrit | Dean Troyer proposed a change to openstack-dev/devstack: Add pre-install phase for extras.d plugins https://review.openstack.org/105789 | 16:38 |
*** aimon has quit IRC | 16:38 | |
*** jgrimm has quit IRC | 16:42 | |
*** luqas has quit IRC | 16:46 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 104088,3,8738c85 | 16:48 |
openstackgerrit | Ramy Asselin proposed a change to openstack/qa-specs: Add qa spec for bp:stress-api-tracking https://review.openstack.org/97589 | 16:49 |
*** sweston has joined #openstack-qa | 16:52 | |
*** tkammer has quit IRC | 16:53 | |
*** sweston_ has joined #openstack-qa | 16:53 | |
*** sweston__ has joined #openstack-qa | 16:53 | |
mrodden | do we have any convention for test docstrings? | 16:54 |
mrodden | in tempest | 16:54 |
sdague | mrodden: not as of yet | 16:55 |
*** gszasz has quit IRC | 16:55 | |
mrodden | ok, was wondering after doing a bunch of reviews yesterday | 16:55 |
sdague | mtreinish: do we have a docs target to generate API docs? | 16:58 |
*** amitpp has joined #openstack-qa | 16:59 | |
*** Sukhdev has quit IRC | 17:00 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 105746,2,86a2e63 | 17:01 |
*** jordanP has quit IRC | 17:01 | |
jogo | sdague: thanks for fixing up my patch | 17:03 |
sdague | jogo: thanks for finding it | 17:03 |
EmilienM | Hello Tempest experts, maybe I could have a review on this patch, fixing a bug in Cinder tests: https://review.openstack.org/#/c/105434/ | 17:03 |
*** harlowja_away is now known as harlowja | 17:03 | |
EmilienM | sdague: and when you have some time, I have a few patches about Javelin. | 17:03 |
openstackgerrit | Steven Hardy proposed a change to openstack/tempest: orchestration add volume backup/restore scenario test https://review.openstack.org/90143 | 17:05 |
*** marun has quit IRC | 17:07 | |
*** mwagner_lap has quit IRC | 17:08 | |
*** marun has joined #openstack-qa | 17:09 | |
jogo | things look a bit funny today on http://jogo.github.io/gate/ | 17:10 |
jogo | partial-ncpu | 17:10 |
mriedem | jogo: i like sand castles | 17:12 |
clarkb | jogo: there is a new bug in partial-ncpu. I bugged dansmith about it last week but he wanted to weekend :) | 17:12 |
clarkb | jogo: I submitted a thing to nova bugs let me dig it up | 17:13 |
jogo | clarkb: thanks | 17:14 |
jogo | mriedem: yeah its a pretty awesome one | 17:14 |
openstackrecheck | Console logs not available after 13:21s for check-tempest-dsvm-postgres-full 105744,2,df22252 | 17:14 |
clarkb | arg by scrollback doesn't go back far enough /me goes to log archive | 17:15 |
mriedem | clarkb: you could just go to nova bugs in launchpad and click on 'reported by me' | 17:15 |
clarkb | mriedem: that didn't work | 17:15 |
clarkb | https://bugs.launchpad.net/nova/+bug/1337472 | 17:16 |
clarkb | because someone marked it invalid | 17:16 |
* clarkb reads | 17:16 | |
*** marun has quit IRC | 17:16 | |
mriedem | if bug 1326998 was for the same thing it should be duped, not invalidated | 17:16 |
clarkb | yes of course its using old code | 17:17 |
* clarkb tries not to yell | 17:17 | |
*** johnthetubaguy is now known as zz_johnthetubagu | 17:17 | |
mriedem | clarkb: so we just need to backport https://review.openstack.org/#/c/98667/ to stable/icehouse right? | 17:17 |
clarkb | jogo: any chance you can clear up 1337472 and point out that it is using old code as I explained in the bug | 17:17 |
clarkb | mriedem: that would be one fix yes | 17:17 |
clarkb | I am going to update the bug and mark it valid again | 17:18 |
mriedem | clarkb: you can probably mark grenade as invalid right? | 17:18 |
clarkb | mriedem: I think so | 17:18 |
mriedem | clarkb: ok, apparently not a clean cherry pick, i can do it locally | 17:18 |
clarkb | mriedem: jogo ok I updated the bug | 17:20 |
*** marun has joined #openstack-qa | 17:21 | |
jogo | clarkb: thanks. mriedem awesome | 17:21 |
jogo | do we have an e-r query for this bug? | 17:22 |
clarkb | no I had decided to join dansmith in the holidaying >_> | 17:22 |
*** afazekas has quit IRC | 17:23 | |
jogo | kk I'll put one together | 17:24 |
*** mmaglana has joined #openstack-qa | 17:24 | |
*** aimon has joined #openstack-qa | 17:24 | |
jogo | I wonder why we saw the sudden spike in this though | 17:24 |
clarkb | it could be something other than the bug above | 17:27 |
jogo | yeah, this may be a side effect | 17:27 |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 105743,2,2b0fb35 | 17:28 |
jogo | whoa logstash total jobs is way up | 17:28 |
sdague | jogo: yeh, we've not drained the logstash queue in days | 17:28 |
jogo | :/ | 17:28 |
mriedem | jogo: message:"UnboundLocalError: local variable 'domain' referenced before assignment" AND tags:"screen-n-cpu.txt" | 17:29 |
jogo | mriedem: yp thats what I have in my browser | 17:29 |
jogo | pushing a patch now | 17:29 |
mriedem | k | 17:29 |
clarkb | sdague: jogo: but we are getting closer to the problem there I hope. I now have the pipeline between logstash and elasticsearch decoupled even more so that I can see if elasticsearch or logstash is at fault. I am really hoping its logstash and then I can upgrade that pice | 17:29 |
sdague | clarkb: colio | 17:30 |
jogo | sweet | 17:30 |
sdague | clarkb: are you vacationing or not this week? | 17:31 |
clarkb | was vacation this week an option? | 17:32 |
openstackgerrit | Joe Gordon proposed a change to openstack-infra/elastic-recheck: Add query for bug 1326998 https://review.openstack.org/105806 | 17:32 |
jogo | mriedem: ^ | 17:32 |
clarkb | sdague: I am not vacationing :) though I will be watching the world cup semi final in a bit | 17:32 |
mriedem | jogo: got it | 17:33 |
sdague | clarkb: ok, I wanted to nudge on the os-loganalyze patches, because there is one more that I want to stick on there for date ranges so I can build a cli toolkit to snag all the fail data | 17:34 |
clarkb | sdague: ok. I have fallen behind on reviews again... | 17:34 |
clarkb | will push those to the top | 17:34 |
*** arnaud__ has joined #openstack-qa | 17:34 | |
jogo | mriedem: http://logs.openstack.org/79/94679/6/check/check-tempest-dsvm-postgres-full/d2225f5/console.html#_2014-07-09_17_27_46_844 | 17:35 |
jogo | have you seen that? | 17:35 |
jogo | boto fun | 17:35 |
dtroyer | sdague: when you need a break to think about something that isn't a race condition, the Cinder backend changes (https://review.openstack.org/100271) are working plus one more small change required to (https://review.openstack.org/105789, still in check queue) to clear the way for adding Ceph… | 17:35 |
mriedem | jogo: yes | 17:35 |
dkranz | clarkb: Is there a recheck bug number for failures like "Failed to fetch http://mirror.rackspace.com/ubuntu/dists/precise-security/multiverse/i18n/Translation-en_US Unable to connect to mirror.rackspace.com:http:" | 17:36 |
mriedem | jogo: https://bugs.launchpad.net/nova/+bug/1338841 | 17:36 |
mriedem | jogo: i couldn't find a good fingerprint to hit from tempest.txt | 17:36 |
clarkb | dkranz: yes there is an apt mirror bug | 17:36 |
*** mwagner_lap has joined #openstack-qa | 17:36 | |
mriedem | jogo: there was a boto release on 7/1 | 17:36 |
dkranz | clarkb: k, I'll look in infra | 17:36 |
clarkb | dkranz: https://bugs.launchpad.net/openstack-ci/+bug/1286818 thats it | 17:37 |
jogo | mriedem: i'll take a whack at a fingerprint | 17:37 |
mriedem | jogo: i do have this one: message:"Unexpected OperationalError raised: (OperationalError) asynchronous connection failed" AND tags:"screen-n-api.txt" | 17:37 |
dkranz | clarkb: thanks | 17:37 |
mriedem | jogo: i'll push that up for starters | 17:37 |
mriedem | jogo: looks like that only fails in postgres jobs | 17:38 |
*** Sukhdev has joined #openstack-qa | 17:38 | |
jogo | mriedem: that may make sense, http://jogo.github.io/gate/ postgres is spiking failure right now | 17:39 |
sdague | dtroyer: cool, will look in a couple | 17:39 |
jogo | although we don't have a lot of data at the moment so could be nothing | 17:39 |
sdague | jogo: postgres is also the metadata server | 17:39 |
*** afazekas has joined #openstack-qa | 17:39 | |
mriedem | ahhh | 17:39 |
sdague | honestly, the problem is almost always the metadata server | 17:39 |
sdague | there is so much network voodoo to make that work right | 17:39 |
sdague | magic network addresses are really silly :) | 17:40 |
mrodden | http://en.wikipedia.org/wiki/Link-local_address | 17:40 |
mrodden | they are actually really useful in a lot of cases | 17:40 |
*** atiwari has joined #openstack-qa | 17:41 | |
mrodden | but a link local might not be the best idea for the metadata service | 17:41 |
openstackrecheck | Console logs not available after 13:21s for gate-nova-docs 105748,1,55acb29 | 17:41 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for bug 1326998 https://review.openstack.org/105806 | 17:41 |
openstackgerrit | Matt Riedemann proposed a change to openstack-infra/elastic-recheck: Add query for nova ec2 concurrency bug 1338841 https://review.openstack.org/105811 | 17:42 |
mriedem | jogo: sdague: ^ that's at least part of it | 17:42 |
jogo | mriedem yeah that fingerprint looks about right | 17:42 |
jogo | but there should be stacktrace | 17:42 |
sdague | mriedem: looks good to me | 17:43 |
*** cpallares has quit IRC | 17:43 | |
mriedem | jogo: there isn't stack trace in the n-api log | 17:43 |
mriedem | the ec2 errors are a bit odd when they are in the log | 17:43 |
mriedem | nothing in the n-api-meta log either | 17:43 |
*** davidlenwell_ is now known as davidlelnwell | 17:44 | |
jogo | this may be a boto issue or something | 17:44 |
sdague | mriedem: you assume that n-api-meta has even vaguely sensible logging :) | 17:44 |
*** davidlelnwell is now known as davidlelnwell__ | 17:44 | |
*** davidlelnwell__ is now known as davidlelnwell | 17:44 | |
*** davidlelnwell is now known as davidlenwell | 17:44 | |
mriedem | sdague: well, no, i know better | 17:45 |
mriedem | jogo: if it started around 7/1 then it could be the new boto release | 17:45 |
mriedem | i'm running the query back 2 weeks | 17:45 |
openstackgerrit | Greg Lucas proposed a change to openstack-dev/devstack: Avoid uploading trove guest image twice https://review.openstack.org/105813 | 17:46 |
mriedem | doesn't seem to show up before 7/8 | 17:46 |
jogo | any changes to tempest tests boto tests around then? | 17:48 |
mriedem | jogo: nope | 17:49 |
jogo | :/ | 17:49 |
mriedem | that was the first thing i checked yesterday | 17:49 |
openstackgerrit | A change was merged to openstack-infra/elastic-recheck: Add query for nova ec2 concurrency bug 1338841 https://review.openstack.org/105811 | 17:49 |
*** tosky has quit IRC | 17:49 | |
*** tosky__ has joined #openstack-qa | 17:49 | |
*** openstackrecheck has quit IRC | 17:50 | |
*** openstackrecheck has joined #openstack-qa | 17:50 | |
jogo | mriedem: http://stackoverflow.com/questions/17514633/gevent-and-posgres-asynchronous-connection-failed | 17:51 |
mtreinish | sdague: api docs for tempest? | 17:51 |
mtreinish | we use to have the auto mod doc detection enabled, but a pbr upgrade broke that so we turned it off | 17:54 |
*** tosky__ is now known as tosky | 17:54 | |
sdague | ah | 17:54 |
jogo | mtreinish: oh i know how to fix taht | 17:54 |
jogo | mtreinish: https://review.openstack.org/#/c/105554/ | 17:54 |
*** Obulpathi has joined #openstack-qa | 17:54 | |
mtreinish | jogo: cool | 17:54 |
*** Obulpathi has quit IRC | 17:54 | |
mtreinish | jogo: want to push it to tempest also? | 17:55 |
* jogo yells at mordred | 17:55 | |
sdague | jogo: ok, let me add that to my patch | 17:55 |
jogo | mtreinish: sure | 17:55 |
sdague | jogo: actually, I'm building a tox target | 17:55 |
sdague | so I can add it here | 17:55 |
*** Obulpathi has joined #openstack-qa | 17:55 | |
jogo | sdague: thanks | 17:55 |
mtreinish | sdague: I thought I had a patch for docs target already | 17:55 |
mtreinish | oh it has a -1 from you: https://review.openstack.org/#/c/101075/ | 17:55 |
mtreinish | I missed that | 17:56 |
*** weshay has joined #openstack-qa | 17:56 | |
jogo | mriedem: this may be a real postgres bug | 17:56 |
jogo | psycopg/connection_int.c: msg = "asynchronous connection failed"; | 17:56 |
mriedem | jogo: just wondering what triggered it | 17:56 |
jogo | mriedem: yeah | 17:57 |
jogo | very odd | 17:57 |
mriedem | i'm not seeing much in nova commits related to this in the last 2 days | 17:57 |
mriedem | i added a requirement for posix_ipc but it was already in the vm's running the tests | 17:57 |
jogo | I am guessing a dependency thing | 17:57 |
mriedem | jogo: me too | 17:57 |
sdague | mtreinish: ok, you want me to just patch your patch? | 17:57 |
mtreinish | sdague: sure | 17:58 |
jogo | ii python-psycopg2 2.4.5-1build5 amd64 Python module for PostgreSQL | 17:58 |
mtreinish | I clearly forgot about it | 17:58 |
mriedem | jogo: hmm, check this: http://logs.openstack.org/79/94679/6/check/check-tempest-dsvm-postgres-full/d2225f5/logs/pip-freeze.txt.gz | 17:58 |
mriedem | jogo: boto==2.20.1 | 17:58 |
mriedem | latest boto is 2.30 | 17:58 |
openstackgerrit | A change was merged to openstack/qa-specs: Add qa spec for bp/cli-tests-coverage https://review.openstack.org/102433 | 17:58 |
jogo | mriedem: odd | 17:59 |
mordred | jogo: what did I do? | 17:59 |
*** ildikov has joined #openstack-qa | 17:59 | |
mriedem | jogo: psycopg2==2.4.5 is really old too | 17:59 |
jogo | mriedem: https://review.openstack.org/#/c/105554/ | 17:59 |
mriedem | wtf, why is this job running with such old dependencies? | 17:59 |
jogo | mriedem: I see those versions in dpkg -l | 18:00 |
mtreinish | mriedem, jogo: are they getting satisfied by site-packages somewhere? | 18:00 |
jogo | mtreinish: yeah | 18:00 |
mriedem | mtreinish: jogo: i was looking at pip-freeze | 18:00 |
jogo | mriedem: http://logs.openstack.org/79/94679/6/check/check-tempest-dsvm-postgres-full/d2225f5/logs/dpkg-l.txt.gz | 18:00 |
jogo | mordred: https://review.openstack.org/#/c/105554/ | 18:00 |
*** Longgeek has quit IRC | 18:01 | |
jogo | mordred: you broke autodocs | 18:01 |
mriedem | jogo: is there a bug for that? | 18:01 |
jogo | mriedem: for what? | 18:01 |
mriedem | i'm not familiar with the pbr docs building stuff | 18:01 |
jogo | oh I didn't file a bug | 18:01 |
jogo | figured it was by design | 18:02 |
mriedem | so it's really weird that we're running with boto 2.20 in these tests, this is nova's requirement: boto>=2.12.0,!=2.13.0 | 18:04 |
mriedem | boto 2.30 is in the pypi.openstack.org mirror | 18:04 |
mriedem | so why wouldn't we be running with boto 2.30 | 18:05 |
sdague | mriedem: is boto provided by the distro? | 18:05 |
jogo | sdague: yup | 18:05 |
jogo | dpkg -l lists it | 18:05 |
mriedem | oh ok | 18:05 |
sdague | mriedem: yep | 18:05 |
mriedem | hmm, well | 18:05 |
sdague | so we're now >= to distro level. We could always bump beyond that | 18:05 |
mriedem | jogo: sdague: should i try a nova patch that runs with latest boto / psycopg2? | 18:05 |
jogo | mriedem: so its not http://changelogs.ubuntu.com/changelogs/pool/main/p/psycopg2/psycopg2_2.4.5-1build5/changelog | 18:06 |
mriedem | or just psycopg2 i guess | 18:06 |
jogo | as that has been out for a while | 18:06 |
sdague | mriedem: I'd rather not compile psycopg2 from source | 18:06 |
mriedem | jogo: yeah since 2012 | 18:06 |
sdague | also, I'm more trusting of the distro for that | 18:06 |
jogo | mriedem: oh http://logs.openstack.org/79/94679/6/check/check-tempest-dsvm-postgres-full/d2225f5/logs/postgres.txt.gz | 18:06 |
jogo | mriedem: actually its from Sun, 23 Mar 2014 15:28:31 +0000 | 18:07 |
mriedem | ERROR: duplicate key value violates unique constraint "assignment_pkey" | 18:07 |
sdague | because instagram would break if that's borked | 18:07 |
sdague | mriedem: yeh, just looks like someone has crappy code | 18:07 |
mriedem | jogo: i was looking at this https://pypi.python.org/pypi/psycopg2/2.4.5 | 18:07 |
sdague | and that someone is.... keystone? | 18:07 |
mriedem | ah snap | 18:08 |
mriedem | yup has to be | 18:08 |
jogo | I like sorry, too many clients already | 18:08 |
sdague | also nova | 18:08 |
sdague | aggregates | 18:08 |
*** atiwari has quit IRC | 18:08 | |
sdague | actually, all those dupings are normal | 18:08 |
*** amitpp has quit IRC | 18:09 | |
sdague | data validation doesn't happen inside the projects | 18:09 |
sdague | they just send it down to the db, watch it explode, and catch that | 18:09 |
mriedem | ah | 18:09 |
openstackrecheck | Console logs not available after 13:24s for gate-horizon-python27 105794,1,b08ac55 | 18:09 |
sdague | UTC FATAL: sorry, too many clients already | 18:09 |
sdague | that I think is a real issue | 18:09 |
mriedem | i don't suppose we index that fil | 18:10 |
mriedem | *file | 18:10 |
*** amitpp has joined #openstack-qa | 18:10 | |
jogo | mriedem: I don't think so, we added these files pretty recently | 18:10 |
sdague | mriedem: nope | 18:11 |
*** xianghui has quit IRC | 18:11 | |
*** marun has quit IRC | 18:11 | |
jogo | but it looks like we should index them. | 18:11 |
sdague | jogo: maybe | 18:11 |
jogo | clarkb: thoughts on indexing http://logs.openstack.org/79/94679/6/check/check-tempest-dsvm-postgres-full/d2225f5/logs/postgres.txt.gz#_2014-07-09_17_05_35 | 18:11 |
sdague | that FATAL should expose up though | 18:11 |
*** amitpp has quit IRC | 18:11 | |
sdague | if it's not, that's a problem | 18:11 |
*** marun has joined #openstack-qa | 18:11 | |
mriedem | sdague: not seeing it in logstash yet for "sorry, too many clients already" | 18:12 |
jogo | I think that becomes Unexpected OperationalError raised: (OperationalError) asynchronous connection failed | 18:12 |
mriedem | yeah | 18:12 |
mriedem | probably | 18:12 |
clarkb | jogo: it looks fine for indexing but I would like to get idnexing under control before we add more variables | 18:12 |
sdague | yeh | 18:12 |
mriedem | so... https://review.openstack.org/#/c/103206/ | 18:12 |
jogo | time stampps match up | 18:12 |
sdague | so we really have the info one layer up | 18:12 |
mriedem | i hit the asynchronous issue there yesterday | 18:12 |
mriedem | now it's merged and it's showing up everywhere | 18:13 |
sdague | mriedem: oh... wait... you just added it to glance as well, right? | 18:13 |
mriedem | sdague: yeah | 18:13 |
mriedem | so nova and glance and cinder (and trove) are running nproc api workeres | 18:13 |
sdague | Bug 1339735 - Deadlock updating image properties in glance-registry daemon | 18:13 |
mriedem | ha | 18:13 |
mriedem | concurrency ftw | 18:13 |
sdague | right so importantly, none of these projects are safe to do that | 18:13 |
mriedem | sdague: to run concurrent api workers? | 18:14 |
sdague | mriedem: yep | 18:14 |
mriedem | why do they have options to enable it? | 18:14 |
sdague | because no one ever did it | 18:14 |
sdague | probably because they copied it from nova | 18:14 |
sdague | actually, this should probably get out on list | 18:14 |
sdague | because it's kind of a big deal | 18:15 |
jogo | sdague: all the bugs in the postgres logs? | 18:15 |
*** hemna_ is now known as hemna | 18:15 | |
sdague | jogo: looking | 18:15 |
sdague | jogo: nope, glance is mostly mysql | 18:15 |
jogo | sdague: I wanted to clarify what you think should be brought to the list | 18:16 |
jogo | mriedem: https://review.openstack.org/#/c/103206/ doesn't allign timestamp wise does it? | 18:16 |
sdague | that glance and cinder don't appear to be multi worker safe | 18:16 |
mriedem | jogo: yeah | 18:16 |
mriedem | jogo: i'm seeing 7/8 when the asynchronous call fail show up | 18:16 |
sdague | given their db access patterns | 18:16 |
mriedem | jogo: and i saw that in https://review.openstack.org/#/c/103206/ yesterday, see comment 4 in the bug | 18:16 |
mriedem | https://bugs.launchpad.net/nova/+bug/1338841/comments/4 | 18:17 |
*** atiwari has joined #openstack-qa | 18:17 | |
sdague | because this feels like a thing we gave neutron no end of shit for, but it turns out it was actually true across the board | 18:18 |
jogo | mriedem: ohh I see that went to the gate several times | 18:18 |
jogo | so the timestamps do match up | 18:18 |
mriedem | jogo: yup | 18:18 |
mriedem | jogo: sdague: shall i revert? | 18:18 |
jogo | is the issue just too many clients in postgres? | 18:18 |
sdague | mriedem: honestly, I'd like the ML thread to figure out what people think we should be doing | 18:18 |
jogo | and config can resolve? | 18:18 |
mriedem | jogo: configuring postgres you mean? | 18:19 |
mriedem | maybe | 18:19 |
mriedem | sdague: i'm good with that | 18:19 |
sdague | because reverting isn't the answer if it's still an option that won't work for people | 18:19 |
mriedem | sdague: because reverting the change doesn't fix the issue for the world given the options are there to set it in deployments | 18:19 |
mriedem | sdague: :) | 18:19 |
mriedem | jinx | 18:19 |
sdague | mriedem: :) | 18:19 |
* jogo is really happy he wrote https://review.openstack.org/#/c/94314/ | 18:20 | |
openstackgerrit | afazekas proposed a change to openstack-dev/devstack: Promote libffi-dev as a general dependency https://review.openstack.org/105831 | 18:20 |
afazekas | sdague: ^ | 18:20 |
sdague | jogo: oh, you want to write something else for me :) | 18:20 |
jogo | being I don't know anything about postgres, I am just making things up | 18:20 |
jogo | about the postgres config | 18:20 |
sdague | can we take db dumps at the end of the run? | 18:21 |
sdague | I realized that might be helpful in debug. | 18:21 |
jogo | hmmm yeah that could be pretty cool | 18:21 |
jogo | sdague: I'll add that to my TODO list | 18:21 |
sdague | dammit man, you are going to make me propose first, eh? | 18:22 |
sdague | mtreinish: so autodoc hits the config explode issue | 18:22 |
jogo | I find it funny how postgres is polite in its error message | 18:22 |
jogo | it says sorry | 18:23 |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 95696,6,2e5cbaa | 18:23 |
sdague | heh | 18:23 |
sdague | mtreinish: so I don't think we actually want autodoc | 18:24 |
sdague | because it's a ton of noise | 18:24 |
sdague | I want to figure out how to get just select classes | 18:24 |
*** dims_ has quit IRC | 18:24 | |
*** morganfainberg is now known as morganfainberg_Z | 18:25 | |
*** dims_ has joined #openstack-qa | 18:25 | |
jogo | mriedem: so for https://bugs.launchpad.net/nova/+bug/1338841 should we bring to ML or just look into finding the right postgres conf? | 18:26 |
mriedem | jogo: we can do both i think | 18:26 |
jogo | mriedem: cool, you want to draft the email? I am off to lunch | 18:26 |
mriedem | jogo: i'm trying to get this nova icehouse backport up quick for that grenade failure we talked about earlier | 18:26 |
jogo | mriedem: sweet | 18:26 |
mriedem | jogo: i can do that in a bit, coffee after this | 18:26 |
mordred | jogo: I'm sure I did it just to annoy you | 18:27 |
mriedem | but yeah i'll write it up since i wrote the thing that broke it | 18:27 |
mtreinish | sdague: oh you can do that manually | 18:27 |
mtreinish | I think you have to symlink the files to the doc source tree | 18:27 |
jogo | mordred: ^_^ | 18:27 |
mtreinish | and maybe include that in the index.rst to make it discoverable | 18:27 |
mtreinish | I'd have to check the sphinx docs to confirm though | 18:27 |
jogo | mtreinish: see https://review.openstack.org/#/c/105552/1/doc/source/devref/compute.rst | 18:28 |
jogo | you just need something like that | 18:28 |
sdague | jogo: oh, cool | 18:28 |
mtreinish | jogo: yeah exactly | 18:28 |
jogo | mriedem: and see http://git.openstack.org/cgit/openstack/nova/tree/doc/source/conf.py | 18:28 |
* jogo finds food | 18:28 | |
*** SridharG has quit IRC | 18:32 | |
*** lifeless1 is now known as lifeless | 18:33 | |
*** SridharG has joined #openstack-qa | 18:34 | |
openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-postgres-full 94599,12,6eebc6f | 18:36 |
*** mriedem has quit IRC | 18:45 | |
*** mriedem has joined #openstack-qa | 18:48 | |
openstackrecheck | Console logs not available after 13:21s for gate-python-novaclient-python26 104875,5,80219b4 | 18:49 |
mtreinish | sdague: was the intent of: https://review.openstack.org/#/c/103216/1 just to cover api extensions or to also include all feature flags? | 18:51 |
*** adalbas has quit IRC | 18:51 | |
*** cpallares has joined #openstack-qa | 18:52 | |
sdague | honestly, extensions or some other way in which it's end user discoverable | 18:54 |
sdague | I'm not really thrilled about non discoverable configs in there | 18:55 |
sdague | I think the rest of the times we've discussed feature flags they are things which are fine in devstack | 18:55 |
mtreinish | sdague: ok that's fine. Yeah I guess non-discoverable ones would work fine right in devstack if they're default false | 18:56 |
mtreinish | sdague: the only other discoverable one I can think of not covered by extensions is api versions | 18:57 |
sdague | yeh, api versions would be valid | 18:57 |
sdague | I think that would fit pretty reasonably in the framework I described | 18:58 |
mtreinish | ok, do you want to add api versions to the spec, or just assume it's implied? I'm fine either way | 19:00 |
mriedem | jogo: sdague: jgriffith: http://www.postgresql.org/docs/current/static/runtime-config-connection.html#GUC-MAX-CONNECTIONS | 19:01 |
mriedem | could try setting that, or synchronous_commit=off | 19:03 |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron-heat-slow 102178,1,5f15db4 | 19:03 |
*** dims_ has quit IRC | 19:03 | |
*** adalbas has joined #openstack-qa | 19:07 | |
jgriffith | mriedem: sdague so maybe as Duncan was mentioning we should do something more sane for devstack | 19:08 |
jgriffith | mriedem: sdague set a value based on num api's on the devstack node | 19:08 |
jgriffith | like num-cores / num-apis | 19:09 |
sdague | jgriffith: oh, you are thinking that we are exceeding that? | 19:09 |
sdague | any idea what the current value is? | 19:09 |
jgriffith | sdague: 100 is default right? | 19:09 |
sdague | it says that | 19:09 |
sdague | I find it hard to believe that we exceed that | 19:09 |
jgriffith | sdague: I don't see anything in startup to change | 19:10 |
mriedem | it says typically 100 but it's set in initdb | 19:10 |
mriedem | based on kernel settings | 19:10 |
jgriffith | sdague: well.. Duncan made an interesting point | 19:10 |
mriedem | "Determines the maximum number of concurrent connections to the database server. The default is typically 100 connections, but might be less if your kernel settings will not support it (as determined during initdb). This parameter can only be set at server start." | 19:10 |
jgriffith | sdague: 32 core box * cinder | 19:10 |
jgriffith | glancece nova etc | 19:10 |
sdague | sure | 19:10 |
jgriffith | although it should grab the VCPU count I guess | 19:10 |
sdague | though we know the gate concurency is 4 | 19:10 |
mriedem | sdague: but we're running with nproc and the vm's are running 8 VCPUs right? | 19:11 |
sdague | oh, right 8 | 19:11 |
sdague | well we have a process dump on one of those runs? | 19:11 |
mriedem | so nova (3) cinder (1) glance (2) trove (2) heat (1) neutron (1) ceilometer (1) workers * nproc | 19:12 |
jgriffith | mriedem: seems like we could hit 100 pretty quick | 19:12 |
sdague | yeh | 19:12 |
mriedem | http://logs.openstack.org/12/92512/9/gate/gate-tempest-dsvm-postgres-full/3664f6d/logs/screen-dstat.txt.gz | 19:13 |
sdague | so bumping that to 200 might be valid as a test | 19:13 |
mriedem | 15:02:36 | 19:13 |
sdague | mriedem: what are you pointing out there | 19:13 |
mriedem | looks like it's nova-api, swift and keystone when it blows up | 19:13 |
sdague | so that's just the most expensive process | 19:14 |
mriedem | sdague: that's dstat when postgres logs have the 'fatal too many client connections' thing | 19:14 |
sdague | which is not hugely interesting | 19:14 |
sdague | swift doesn't talk to the db | 19:14 |
sdague | so, I can buy the idea that we are hitting that. Want to propose increasing it to 200? | 19:15 |
sdague | I also wonder if there is a way to get the count over time | 19:15 |
mriedem | sdague: i can push something up to propose 200 sure | 19:15 |
sdague | mriedem: if it goes away after that, I think it's clear | 19:15 |
jgriffith | sdague: mriedem http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html | 19:15 |
mriedem | http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_max_connections | 19:16 |
mriedem | http://dev.mysql.com/doc/refman/5.5/en/too-many-connections.html | 19:16 |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-full 90143,17,53ebe74 | 19:16 |
mriedem | so mysql sets 151 | 19:16 |
mriedem | docs say it used to be 100 | 19:16 |
mriedem | but apparently that wasn't cutting the mustard | 19:17 |
sdague | yeh, so lets set pg to 200, if it goes away, so be it. | 19:17 |
mriedem | k, working on that now | 19:17 |
sdague | I kind of which jaypipes was on right now | 19:17 |
mriedem | i can still send something to the ML later to fish for PG experts | 19:17 |
sdague | because he might have some thoughts about what we should be doing here at a macro level | 19:17 |
sdague | mriedem: +1 | 19:17 |
mriedem | because fsync=off sounds interesting regardless | 19:17 |
mtreinish | mriedem: I really like the imagery of fishing PG experts :) | 19:18 |
*** arnaud__ has quit IRC | 19:19 | |
marun | dkranz: https://review.openstack.org/#/c/72585/ | 19:20 |
mtreinish | sdague: did you see my follow up question about your spec before the pg max connection discussion? | 19:20 |
sdague | mtreinish: nope | 19:20 |
sdague | oh, I'm going to say it's implied | 19:21 |
mtreinish | sdague: ok cool, then I'll push the spec through | 19:21 |
mriedem | 11*8 | 19:22 |
mtreinish | 88 | 19:23 |
mriedem | ha oops | 19:26 |
mriedem | would be nice if we archived postgresql.conf | 19:26 |
openstackgerrit | A change was merged to openstack/qa-specs: branchless tempest extensions spec https://review.openstack.org/103216 | 19:28 |
openstackrecheck | Console logs not available after 13:23s for gate-tempest-dsvm-postgres-full 105470,3,a45ada5 | 19:30 |
*** jlibosva has joined #openstack-qa | 19:32 | |
*** jlibosva has quit IRC | 19:37 | |
*** Sukhdev has quit IRC | 19:38 | |
*** amitpp has joined #openstack-qa | 19:39 | |
openstackgerrit | Matt Riedemann proposed a change to openstack-dev/devstack: Set postgresql max_connections=200 https://review.openstack.org/105854 | 19:42 |
mriedem | sdague: jogo: jgriffith: ^ | 19:42 |
jgriffith | mriedem: nice! | 19:42 |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 103229,9,049ffb3 | 19:43 |
sdague | mriedem: can you call the var MAX_DB_CONNECTIONS | 19:44 |
sdague | because it's global | 19:44 |
mriedem | sdague: sure | 19:44 |
sdague | I think after that, I'm good with a +2 | 19:44 |
openstackgerrit | Matt Riedemann proposed a change to openstack-dev/devstack: Set postgresql max_connections=200 https://review.openstack.org/105854 | 19:45 |
*** dims has joined #openstack-qa | 19:46 | |
*** SridharG has quit IRC | 19:49 | |
mriedem | sdague: is it weird that it's the ec2 tests hammering the postgresql failure? | 19:50 |
*** masayukig has quit IRC | 19:50 | |
mriedem | or maye that's n-api-meta related | 19:50 |
sdague | mriedem: honestly, I think it's probably n-api-meta | 19:50 |
sdague | and the fact that we're late in the run | 19:50 |
sdague | that adds another 8 connections, right? | 19:51 |
*** amitpp has quit IRC | 19:52 | |
*** mkoderer has quit IRC | 19:52 | |
mriedem | sdague: yup, n-api-meta has it's own workers | 19:53 |
openstackgerrit | Matt Riedemann proposed a change to openstack-dev/devstack: Set postgresql max_connections=200 https://review.openstack.org/105854 | 19:54 |
mriedem | ^ fixes the typo | 19:54 |
sdague | yeh, that all adds up then | 19:54 |
*** dims_ has joined #openstack-qa | 19:55 | |
*** masayukig has joined #openstack-qa | 19:55 | |
*** Sukhdev has joined #openstack-qa | 19:55 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 92976,3,de5a282 | 19:56 |
*** dims has quit IRC | 19:57 | |
*** morganfainberg_Z is now known as morganfainberg | 19:57 | |
openstackgerrit | Sébastien Han proposed a change to openstack-dev/devstack: Implement Ceph backend for Glance / Cinder / Nova https://review.openstack.org/65113 | 19:58 |
openstackgerrit | A change was merged to openstack/qa-specs: Add spec for bp:/fuzzy-test https://review.openstack.org/99585 | 19:59 |
*** arnaud has joined #openstack-qa | 20:02 | |
*** enikanorov__ has joined #openstack-qa | 20:05 | |
*** enikanorov has quit IRC | 20:06 | |
openstackrecheck | Console logs not available after 13:24s for gate-horizon-python26 105803,1,6fa1c1d | 20:10 |
*** afazekas has quit IRC | 20:10 | |
jogo | mriedem: nice | 20:11 |
*** krtaylor is now known as krtaylor_away | 20:14 | |
*** nati_uen_ has joined #openstack-qa | 20:18 | |
*** jamielennox is now known as jamielennox|away | 20:19 | |
*** nati_ueno has quit IRC | 20:20 | |
*** markmcclain has quit IRC | 20:23 | |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-postgres-full 105810,1,9c5d636 | 20:23 |
*** sweston_ has quit IRC | 20:27 | |
openstackrecheck | Console logs not available after 13:24s for gate-nova-python26 104001,2,6bce56f | 20:37 |
openstackgerrit | patrick-crews proposed a change to openstack-infra/elastic-recheck: Initial work to add query namespace to elastic-recheck. https://review.openstack.org/101958 | 20:43 |
openstackgerrit | patrick-crews proposed a change to openstack-infra/elastic-recheck: Initial work to add query namespace to elastic-recheck. https://review.openstack.org/101958 | 20:45 |
*** arnaud has quit IRC | 20:45 | |
*** nmagnezi has quit IRC | 20:47 | |
*** salv-orlando has joined #openstack-qa | 20:49 | |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-postgres-full 86257,8,9a026b6 | 20:50 |
*** sweston__ has quit IRC | 20:59 | |
openstackgerrit | Carl Baldwin proposed a change to openstack-dev/devstack: DEVSTACK PATCH FOR INTERIM DVR TESTING https://review.openstack.org/103180 | 20:59 |
openstackgerrit | Paul Michali proposed a change to openstack-dev/devstack: DEVSTACK PATCH FOR INTERIM DVR TESTING https://review.openstack.org/103180 | 21:04 |
openstackrecheck | Console logs not available after 13:35s for gate-nova-python26 105748,1,9cc6482 | 21:04 |
*** ekhugen has quit IRC | 21:04 | |
*** dustins has quit IRC | 21:07 | |
dkranz | mtreinish: what do you think of my last comment on https://review.openstack.org/#/c/94473/ (tempest.conf auto-gen) | 21:07 |
mtreinish | dkranz: heh, that was next on my list | 21:08 |
*** obondarev_ has joined #openstack-qa | 21:09 | |
dkranz | mtreinish: :) | 21:09 |
mtreinish | dkranz: but I agree, I think making the spec about the building blocks for the script instead of the script itself is a good call | 21:09 |
dkranz | mtreinish: great. So if you think my three components are controversisal you can comment, or add any others if you think of them. | 21:10 |
*** obondarev__ has joined #openstack-qa | 21:11 | |
*** jamielennox|away is now known as jamielennox | 21:11 | |
*** obondarev has quit IRC | 21:12 | |
*** obondarev_ has quit IRC | 21:12 | |
*** obondarev__ is now known as obondarev | 21:12 | |
*** mmaglana has quit IRC | 21:16 | |
*** mmaglana has joined #openstack-qa | 21:16 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-neutron 104148,6,887fc45 | 21:17 |
*** Obulpathi has quit IRC | 21:22 | |
*** markmcclain has joined #openstack-qa | 21:22 | |
openstackgerrit | melanie witt proposed a change to openstack-infra/elastic-recheck: Extend query for pip download timeout bug 1270710 https://review.openstack.org/105884 | 21:27 |
*** aysyd has quit IRC | 21:28 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 102585,5,edaa6af | 21:30 |
*** arnaud has joined #openstack-qa | 21:40 | |
*** sballe has joined #openstack-qa | 21:42 | |
*** jamielennox is now known as jamielennox|away | 21:43 | |
openstackrecheck | Console logs not available after 13:22s for check-tempest-dsvm-postgres-full 105813,1,27b715d | 21:44 |
*** Sukhdev has quit IRC | 21:55 | |
openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-full 105817,1,fa2ea5a | 21:57 |
mriedem | mtreinish: good with this now? https://review.openstack.org/#/c/99984/ | 22:00 |
*** gmann has joined #openstack-qa | 22:02 | |
mtreinish | mriedem: +A | 22:02 |
*** jay-house-hunter has joined #openstack-qa | 22:03 | |
mriedem | \o/ | 22:03 |
mriedem | suckers! | 22:03 |
openstackgerrit | A change was merged to openstack/qa-specs: Check minimum version for CLI tests https://review.openstack.org/99984 | 22:03 |
mtreinish | heh, I marked it as high prio so I'll bug you about it every week during the meeting :) | 22:04 |
*** Sukhdev has joined #openstack-qa | 22:04 | |
*** gmann has quit IRC | 22:04 | |
mriedem | nice, i just have to figure out the issue with the novaclient decorator thing, otherwise the code is basically ready to go | 22:04 |
*** gmann has joined #openstack-qa | 22:04 | |
mriedem | it seems to not be happy if the version you're filtering against is the exact version that's installed | 22:06 |
mriedem | if it's < the installed version like heatclient it's ok | 22:06 |
mriedem | i'll probably need to run a wip trace patch to debug what's coming back from the command | 22:06 |
*** mriedem has quit IRC | 22:09 | |
openstackrecheck | Console logs not available after 13:23s for check-tempest-dsvm-postgres-full 105815,1,4581e35 | 22:11 |
mtreinish | mriedem: hmm, is that possibly a versionutils bug? | 22:11 |
mtreinish | yeah I guess there really isn't a good way to tell unless we print out what we're getting from the command and how it differs from what's expected | 22:12 |
*** cdent_ has joined #openstack-qa | 22:20 | |
*** cdent has quit IRC | 22:23 | |
*** cdent_ is now known as cdent | 22:23 | |
openstackgerrit | Dean Troyer proposed a change to openstack-dev/devstack: Re-order top of stack.sh https://review.openstack.org/105899 | 22:24 |
openstackrecheck | Console logs not available after 13:24s for check-grenade-dsvm 105816,1,d0027fc | 22:24 |
*** weshay has quit IRC | 22:25 | |
*** sweston has quit IRC | 22:31 | |
*** weshay has joined #openstack-qa | 22:33 | |
*** weshay has quit IRC | 22:33 | |
*** sweston has joined #openstack-qa | 22:34 | |
*** boris-42 has quit IRC | 22:37 | |
*** jay-house-hunter has quit IRC | 22:38 | |
openstackrecheck | Console logs not available after 13:23s for gate-tempest-dsvm-neutron-large-ops 94679,6,debbb19 | 22:38 |
*** boris-42 has joined #openstack-qa | 22:39 | |
*** gmann has quit IRC | 22:51 | |
*** andreaf has quit IRC | 22:53 | |
openstackrecheck | Console logs not available after 13:29s for check-grenade-dsvm-partial-ncpu-icehouse 104290,6,469740b | 22:56 |
*** atiwari has quit IRC | 22:59 | |
*** adalbas has quit IRC | 23:00 | |
*** rbak has quit IRC | 23:03 | |
openstackgerrit | A change was merged to openstack/qa-specs: Add qa spec for bp/tempest-guest-tests https://review.openstack.org/95639 | 23:04 |
dkranz | mlavalle: https://bugs.launchpad.net/nova/+bug/1311500 | 23:04 |
morganfainberg | mtreinish, ping re https://review.openstack.org/#/c/104610/ have the +2 on matter and stable icehouse https://review.openstack.org/#/q/I13ce159cbe9739d4bf5d321fc4bd069245f32734,n,z we can move forward with the QA /tempest skip | 23:04 |
mtreinish | morganfainberg: do you have a link for the tempest skip? | 23:05 |
mtreinish | I'll give it my +2 | 23:05 |
morganfainberg | https://review.openstack.org/#/c/104610/ | 23:05 |
morganfainberg | it needs a recheck, transient failure it looks like | 23:05 |
openstackgerrit | Morgan Fainberg proposed a change to openstack/tempest: Add a skip for bug #1334368 https://review.openstack.org/104610 | 23:05 |
* morganfainberg rebased instead | 23:06 | |
morganfainberg | mtreinish, thanks! | 23:06 |
mtreinish | dkranz: ^^^ that should be an easy +2/+A | 23:07 |
mtreinish | dkranz: also on https://review.openstack.org/#/c/97589/ you had some concerns did asselin address them? | 23:07 |
morganfainberg | gotta run, no battery left and time to go get food | 23:07 |
asselin | I resolved some this morning from Masayuki Igawa | 23:08 |
*** dkranz has quit IRC | 23:08 | |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron 53750,10,3a38507 | 23:10 |
*** sweston has quit IRC | 23:11 | |
asselin | mtreinish,dkranz Masayuki Igawa had some questions unresolved about including an example for non-stress tests. I had a question of where to put the code: #. common/utils102 | 23:11 |
asselin | #. openstack/common | 23:11 |
mtreinish | asselin: Masayuki's nick is masayukig :) | 23:12 |
asselin | thanks | 23:13 |
mtreinish | asselin: either place is fine really. I personally have never really understood what the utils subdir was for | 23:13 |
mtreinish | it seems a bit redundant to have a common.utils.*utils.py | 23:14 |
mtreinish | but it's probably just a hold over | 23:14 |
*** HenryG has quit IRC | 23:16 | |
asselin | mtreinish, ok, I'll put it in common then. Perhaps a new subdir: decorators | 23:16 |
mtreinish | no I don't think that's necessary, you can just make it a module | 23:17 |
*** morganfainberg is now known as morganfainberg_Z | 23:17 | |
mtreinish | because the functionality isn't really about a decorator but more the tracking | 23:17 |
mtreinish | the decorator is just the convienent api you're proposing we use to enable it | 23:18 |
asselin | ok | 23:18 |
asselin | so common/tracking.py | 23:19 |
mtreinish | sure, that works | 23:20 |
*** Sukhdev has quit IRC | 23:21 | |
*** marun has quit IRC | 23:21 | |
*** markmcclain has quit IRC | 23:21 | |
*** obondarev has quit IRC | 23:22 | |
openstackrecheck | Console logs not available after 13:24s for check-tempest-dsvm-neutron-icehouse 105818,1,bf88450 | 23:23 |
*** krtaylor_away is now known as krtaylor | 23:24 | |
*** mlavalle has quit IRC | 23:25 | |
asselin | mtreinish, any concerns that there's still references to stress tests in that spec? I'm not sure about take some out, but I'm not attached to them. | 23:25 |
mtreinish | asselin: no I think concentrating on using it for stress tests is fine | 23:26 |
asselin | ok great | 23:27 |
mtreinish | asselin: my concerns were that this is probably generally useful and I didn't want to hard couple it to the stress tests | 23:27 |
openstackgerrit | Ramy Asselin proposed a change to openstack/qa-specs: Add qa spec for bp:stress-api-tracking https://review.openstack.org/97589 | 23:27 |
mtreinish | even if using elsewhere isn't explicitly part of the proposal | 23:27 |
asselin | mtreinish, yes I agree. it will be good to decouple it too up front. | 23:28 |
openstackrecheck | Console logs not available after 13:25s for check-tempest-dsvm-neutron-heat-slow 76002,19,c50b8e4 | 23:36 |
*** gmann has joined #openstack-qa | 23:37 | |
*** Sukhdev has joined #openstack-qa | 23:38 | |
*** sabeen1 has quit IRC | 23:41 | |
*** jamielennox|away is now known as jamielennox | 23:49 | |
openstackrecheck | Console logs not available after 13:24s for gate-tempest-dsvm-postgres-full 105671,1,e38bc0c | 23:50 |
*** marun has joined #openstack-qa | 23:51 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!