| *** tosky has quit IRC | 00:27 | |
| *** dave-mccowan has joined #openstack-sahara | 01:50 | |
| *** tellesnobrega_ has joined #openstack-sahara | 01:51 | |
| *** tellesnobrega has quit IRC | 01:53 | |
| openstackgerrit | gaobin proposed openstack/sahara master: fixed word error https://review.openstack.org/621054 | 01:55 |
|---|---|---|
| *** dave-mccowan has quit IRC | 02:46 | |
| *** dave-mccowan has joined #openstack-sahara | 02:54 | |
| *** dave-mccowan has quit IRC | 03:33 | |
| *** dave-mccowan has joined #openstack-sahara | 03:36 | |
| *** dave-mccowan has quit IRC | 04:30 | |
| *** pcaruana has quit IRC | 05:35 | |
| *** pcaruana has joined #openstack-sahara | 07:22 | |
| openstackgerrit | Filippo Inzaghi proposed openstack/python-saharaclient master: Add Python 3.6 classifier to setup.cfg https://review.openstack.org/621103 | 07:24 |
| *** rcernin has quit IRC | 08:06 | |
| *** tosky has joined #openstack-sahara | 08:42 | |
| *** pcaruana has quit IRC | 10:44 | |
| *** pcaruana has joined #openstack-sahara | 10:50 | |
| *** tellesnobrega_ has quit IRC | 11:34 | |
| *** pcaruana has quit IRC | 12:11 | |
| *** tellesnobrega_ has joined #openstack-sahara | 13:50 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: Fix the test template for storm 1.2 https://review.openstack.org/621166 | 14:29 |
| Gaasmann | 2018-11-30 15:45:31.664 8 ERROR sahara.service.ops [req-0119493b-e04f-408d-a977-e358ec210fe8 78e9b31dfce642fa9995a58d017458d1 a4b84a529e9e4eea8ff7bfbe51c48e32 - - -] [instance: none, cluster: 4e5cffe9-0f5e-4a5f-979e-106e687f95d3] Error during operating on cluster (reason: 'Operation with name 'Await starting Cloudera Manager'' timed out after 300 second(s) and following timeout was violated: | 15:46 |
| Gaasmann | Timeout for Cloudera Manager starting | 15:46 |
| Gaasmann | I'm not sur I understand, the instance is none, I don't see a command to start the Cloud Manager in the log or anything starting on the nodes | 15:47 |
| Gaasmann | Is it for a service something start to return or is it something else? | 15:48 |
| Gaasmann | s/Is it/Is it waiting/ | 15:48 |
| tosky | Gaasmann: the command executed is `service cloudera-scm-server start` | 16:09 |
| tosky | executed by sahara-engine connected through ssh on the manager node | 16:09 |
| Gaasmann | hum that one was executed and finished. I think my main disk ran out of space though, I'm spawning a cluster with bigger disk to test | 16:13 |
| tosky | oh, how much? | 16:15 |
| Gaasmann | 10G, testing with 40G | 16:16 |
| tosky | I usually assign 25G | 16:17 |
| tosky | to the manager node, I mean | 16:17 |
| Gaasmann | hum | 16:18 |
| Gaasmann | 2018-11-30 16:15:33.162 8 DEBUG sahara.utils.ssh_remote [req-8ddcd088-925c-4afa-b14c-5c0197154031 78e9b31dfce642fa9995a58d017458d1 a4b84a529e9e4eea8ff7bfbe51c48e32 - - -] [instance: a2455578-cd2d-4d30-8cf3-dc4dbafa2d4f, cluster: fd344395-085a-43b4-a3bf-e3473e831871] "Executing "service cloudera-scm-server start"" took 5.4 seconds to complete _log_command | 16:18 |
| Gaasmann | /var/lib/kolla/venv/local/lib/python2.7/site-packages/sahara/utils/ssh_remote.py:931 | 16:18 |
| Gaasmann | waiting for the end or timeout | 16:18 |
| Gaasmann | timeout as well | 16:22 |
| tosky | the condition is "being able to connect to the Cloudera Manager API port" | 16:25 |
| tosky | which is 7180 | 16:25 |
| *** pcaruana has joined #openstack-sahara | 16:26 | |
| tosky | this is the relevant function: http://git.openstack.org/cgit/openstack/sahara/tree/sahara/plugins/cdh/plugin_utils.py#n382 | 16:27 |
| Gaasmann | I can browse the website | 16:27 |
| openstackgerrit | Jesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara master: Enable overriding the service setup host python interpreter https://review.openstack.org/621222 | 16:29 |
| Gaasmann | tosky: The cluster is up and running. I wonder how the telnet is done. Specifically from where. Does it go through the netns ns or directly from the controller node, this kind of thing | 16:34 |
| tosky | from the the place where the code runs, so from the controller | 16:35 |
| tosky | but, unless you disabled it, sahara should open the required ports automatically | 16:35 |
| Gaasmann | sure, but for the ssh command, it uses the proxy-command with the ip netns exec. I wonder if the telnet does the same thing | 16:42 |
| Gaasmann | otherwise, it won´t work | 16:42 |
| tosky | oh! | 16:42 |
| tosky | good point | 16:42 |
| tosky | I suspect it does not | 16:43 |
| Gaasmann | erf | 16:43 |
| tosky | tellesnobrega_: ^^ | 16:43 |
| tosky | we have a problem | 16:43 |
| Gaasmann | my cluster is on a tenant network with use_namespace=True in sahara.conf :-) | 16:43 |
| tosky | maybe we can use the existing system for connection for that | 16:44 |
| tosky | instead of an ssh connection | 16:44 |
| *** tellesnobrega_ is now known as tellesnobrega | 16:50 | |
| tellesnobrega | tosky, I'm not sure I understand the issue here | 16:50 |
| tosky | when starting cloudera manager, this function is used: http://git.openstack.org/cgit/openstack/sahara/tree/sahara/plugins/cdh/plugin_utils.py#n382 | 16:51 |
| tosky | but then the cunction polls _check_cloudera_manager_started, so http://git.openstack.org/cgit/openstack/sahara/tree/sahara/plugins/cdh/plugin_utils.py#n374 | 16:51 |
| tosky | and that function uses telnetlib, which does not use the proxy settings | 16:52 |
| tellesnobrega | huum | 16:52 |
| tellesnobrega | we should change that to use ssh as well? | 16:53 |
| tosky | the problem is that we need just a ping on that port | 16:53 |
| tosky | not sure we can simulate it with paramiko | 16:53 |
| tosky | otherwise we need to find a way to reuse the proxy settings for a simple ping | 16:53 |
| tellesnobrega | I see | 16:54 |
| tosky | ok, first step first: a story | 16:54 |
| tosky | Gaasmann: can you please create a story, or do you want me to do it? | 16:55 |
| Gaasmann | I'll do it, you may need to review it as I feel I don't have all the right terms to describe the problem correctly. | 16:55 |
| tosky | sure; thanks! | 17:01 |
| *** pcaruana has quit IRC | 17:04 | |
| Gaasmann | https://storyboard.openstack.org/#!/story/2004476 | 17:09 |
| tellesnobrega | thanks Gaasmann | 17:10 |
| *** tellesnobrega_ has joined #openstack-sahara | 17:13 | |
| *** tellesnobrega has quit IRC | 17:15 | |
| *** openstackgerrit has quit IRC | 17:51 | |
| *** jeremyfreudberg has joined #openstack-sahara | 18:20 | |
| jeremyfreudberg | tellesnobrega_, tosky: i'm late to the party, but i've seen that cloudera issue too (and swept it under the rug) | 18:21 |
| jeremyfreudberg | it would explain why CDH never worked in my production clouds but always worked so perfectly in devstack | 18:21 |
| *** jeremyfreudberg has quit IRC | 18:23 | |
| tosky | oh, thanks | 18:27 |
| *** openstackgerrit has joined #openstack-sahara | 18:55 | |
| openstackgerrit | Merged openstack/sahara master: Add DEBIAN_FRONTEND=noninteractive in front of apt-get install commands https://review.openstack.org/620396 | 18:55 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara stable/rocky: Add DEBIAN_FRONTEND=noninteractive in front of apt-get install commands https://review.openstack.org/621265 | 18:59 |
| tosky | backport! | 19:00 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: New job for sahara-scenario based on python 3 https://review.openstack.org/608211 | 19:35 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: DNM Test print python version https://review.openstack.org/607145 | 19:35 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Add bugs url link to README https://review.openstack.org/598546 | 20:27 |
| *** tellesnobrega_ has quit IRC | 20:33 | |
| *** openstackgerrit has quit IRC | 20:50 | |
| Gaasmann | Is there a place telling what is the minimal base image to generate images? I tried to build mapr but it fails on Ubuntu Xenial as it tries to install openjdk-7-jre but this package does not exist (minimum openjdk-8) | 21:14 |
| tosky | mapr requires trusty | 21:18 |
| tosky | not our requirement, it's a requirement of MapR 5.2 | 21:19 |
| tosky | unfortunately we don't support yet MapR 6 | 21:19 |
| tosky | anyway, we recently restructured the documentation, so the User Guide documents the supported distributions for each plugin and version | 21:21 |
| tosky | here: https://docs.openstack.org/sahara/latest/user/index.html#plugins | 21:21 |
| tosky | for example, MapR is https://docs.openstack.org/sahara/latest/user/mapr-plugin.html#images | 21:22 |
| Gaasmann | erf, not sure why, I was looking at the Pike version of the doc :-) | 21:22 |
| Gaasmann | way clearer with the new version | 21:23 |
| tosky | I guess we should backport that patch | 21:23 |
| *** openstackgerrit has joined #openstack-sahara | 21:43 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Enable overriding the service setup host python interpreter https://review.openstack.org/621222 | 21:43 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-image-elements master: firstboot: make rc-local start after cloud-init https://review.openstack.org/621302 | 22:21 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara stable/rocky: doc: restructure the image building documentation https://review.openstack.org/621306 | 22:41 |
| *** tosky has quit IRC | 23:42 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!