| *** Bhujay has joined #openstack-sahara | 02:22 | |
| *** Bhujay has quit IRC | 02:57 | |
| *** prasen has joined #openstack-sahara | 02:59 | |
| *** prasen has quit IRC | 03:05 | |
| *** links has joined #openstack-sahara | 03:15 | |
| *** SergeyLukjanov has quit IRC | 04:08 | |
| *** chason has quit IRC | 04:09 | |
| *** SergeyLukjanov has joined #openstack-sahara | 04:13 | |
| *** chason has joined #openstack-sahara | 04:15 | |
| *** Bhujay has joined #openstack-sahara | 05:27 | |
| *** pcaruana has joined #openstack-sahara | 06:37 | |
| *** links has quit IRC | 07:13 | |
| *** Bhujay has quit IRC | 07:22 | |
| *** rcernin has quit IRC | 07:22 | |
| *** Bhujay has joined #openstack-sahara | 07:23 | |
| *** tesseract has joined #openstack-sahara | 07:42 | |
| *** tosky has joined #openstack-sahara | 07:55 | |
| *** links has joined #openstack-sahara | 08:13 | |
| openstackgerrit | bhujay kumar proposed openstack/sahara master: Sets correct permission for /etc/hosts https://review.openstack.org/586860 | 08:16 |
|---|---|---|
| *** maxbab has joined #openstack-sahara | 08:23 | |
| *** maxbab has left #openstack-sahara | 08:24 | |
| Bhujay | tosky , have put together the observations here https://storyboard.openstack.org/#!/project/935 , so that it can be explained better , i may be wrong , in case u r back , will be grateful if u can give some hints | 08:55 |
| tosky | Bhujay: hi, I commented on the review | 08:57 |
| Bhujay | tosky , thanks a lot , let me see | 08:57 |
| tosky | oh, a new story | 08:57 |
| Bhujay | yes tosky , i am badly stuck up on this and also on scaling cluster not happening with 500 internal server error http://10.174.112.51/api/v1/clusters/c1/config_groups | 09:00 |
| tosky | Bhujay: for ambari, you reported that there are no relevant logs on /var/lib/ambari-agent (the subpath is a bit different IIRC) - is it the case for all nodes? And in /var/lib/ambari-server? | 09:02 |
| tosky | and for 500 server error, the logs from sahara-api.log is definitely needed | 09:02 |
| Bhujay | tosky ,the api address is not sahara api , ambari client was trying make a call to ambari server on 8080 port | 09:04 |
| Bhujay | will sahara-api log be helpful ? | 09:05 |
| tosky | oh, sorry | 09:06 |
| tosky | I'm still catching up :) | 09:06 |
| Bhujay | tosky , giving the correct path for /var/lib... | 09:06 |
| tosky | so what's needed is the log again from /var/lib/ambari-server on the ambari node | 09:07 |
| Bhujay | tosky , you are very humble sir , it was my communication problem | 09:07 |
| tosky | no, no, it's definitely me | 09:07 |
| tosky | if the deployment times out during the installation phase, maybe you can setup a local mirror, or even better, a local transparent squid proxy | 09:08 |
| Bhujay | yeas , i am doing that from local mirrors | 09:09 |
| Bhujay | but even with that when the number of nodes are increasing ... its causing this problem | 09:10 |
| tosky | that's strange, because at that point it wouldn't be much a Sahara issue, but an Ambari issue | 09:11 |
| tosky | I mean, when Sahara starts the deployment, all the instances are up (so nova or ironic are no more involved) | 09:11 |
| Bhujay | I agree but .. | 09:11 |
| tosky | and it's just Sahara kindly poking Ambari to do its magic | 09:11 |
| tosky | so I'm a bit puzzled and curious | 09:12 |
| Bhujay | without this the whole purpose of deploying through sahara is defeated | 09:12 |
| Bhujay | u see from ambari console , i could restart the installation and make the service up within few minutes ...since actually the pkgs were all installed .... | 09:13 |
| tosky | there are two different problems, as you wrote in the story | 09:13 |
| tosky | on the sahara side, it should be possible to force a recheck of the status for a cluster in error state; this may require some changes in the internal code and in some assumptions | 09:14 |
| Bhujay | sorry for not being able to communicate again ..i wanted to two possible way of solving the problem , problem is one | 09:14 |
| tosky | or maybe there should be a different type of error state | 09:14 |
| Bhujay | tosky , yes thats what probably is needed | 09:14 |
| tosky | the other issue is that ambari is failing anyway to deploy, and you need to manually force the installation | 09:15 |
| tosky | by asking ambari to explicitly reinstall | 09:15 |
| Bhujay | yes | 09:15 |
| tosky | this is an ambari issue which may be related to some configuration missing, or something in ambari itself | 09:15 |
| tosky | if this is fixed, maybe the need for the other one can be postponed, it would be less pressing | 09:15 |
| Bhujay | agreed, one way or the other we need to see how sahara is successful | 09:18 |
| tosky | and sadly we need the ambari logs to understand what happened; if the package installation failed for a simple timeout when downloading, when installing, or somewhere else | 09:19 |
| tosky | maybe we can tune some timeout | 09:19 |
| Bhujay | agreed , i tired to find out that parameters , if u have any param in mind , let me try out them | 09:21 |
| Bhujay | and i will collect more logs by next two days | 09:21 |
| Bhujay | in the paste , the first portion is the log which is shown from the ambari ops windows and the second log is the output at the /var/lib/ambari...location | 09:22 |
| Bhujay | the problem is that i am not able to find out what failed , is it the installation or the command after the installation since the logs are getting overwriiten in the next retry it seems | 09:23 |
| tosky | uhm, when I check the logs I have the impression that they are simply appended | 09:33 |
| tosky | maybe they have been rotated and there are other log files? | 09:33 |
| Bhujay | hummm... let me check .. i need to login a prod env | 09:34 |
| Bhujay | tosky , yes there is another file by that name 38 .. getting the log | 09:48 |
| Bhujay | tosky , here it is http://paste.openstack.org/show/726927/ . i have to leave to catch a flight , sorry , will chk in afer 5 hrs , anything comes in your kindly share | 09:53 |
| tosky | sure | 09:56 |
| tosky | have a safe flight! | 09:56 |
| tosky | we can probably start by allowing a limited amount of retries for package installation (which seems to be False right now); even 2 or 3 should cover most of the problems | 09:57 |
| tosky | hopefully | 09:58 |
| tosky | maybe | 09:58 |
| Bhujay | tosky , i noticed that , from where can we set the retry any idea ? | 10:10 |
| openstackgerrit | Jonathan Rosser proposed openstack/openstack-ansible-os_sahara master: Use generic vars file for ubuntu https://review.openstack.org/586707 | 10:21 |
| *** Bhujay has quit IRC | 10:22 | |
| *** links has quit IRC | 13:09 | |
| *** links has joined #openstack-sahara | 13:26 | |
| *** prasen has joined #openstack-sahara | 14:13 | |
| *** links has quit IRC | 14:41 | |
| *** links has joined #openstack-sahara | 15:00 | |
| *** pcaruana has quit IRC | 15:10 | |
| *** links has quit IRC | 15:12 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Default MQ RPC/Notify credentials/vhosts to match https://review.openstack.org/587033 | 15:13 |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara-dashboard master: One missed hadoop_version->plugin_version https://review.openstack.org/587501 | 15:30 |
| *** links has joined #openstack-sahara | 15:49 | |
| *** links has quit IRC | 15:51 | |
| *** links has joined #openstack-sahara | 15:52 | |
| *** links has quit IRC | 15:55 | |
| *** links has joined #openstack-sahara | 15:55 | |
| *** links has quit IRC | 15:58 | |
| *** links has joined #openstack-sahara | 15:58 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/sahara master: Allow overriding of /etc/hosts entries https://review.openstack.org/572191 | 16:00 |
| *** links has quit IRC | 16:01 | |
| *** links has joined #openstack-sahara | 16:01 | |
| *** links has quit IRC | 16:10 | |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Use generic vars file for ubuntu https://review.openstack.org/586707 | 16:11 |
| *** links has joined #openstack-sahara | 16:21 | |
| *** links has quit IRC | 16:24 | |
| *** links has joined #openstack-sahara | 16:25 | |
| *** links has quit IRC | 16:27 | |
| *** links has joined #openstack-sahara | 16:28 | |
| *** links has quit IRC | 16:30 | |
| *** links has joined #openstack-sahara | 16:31 | |
| *** links has quit IRC | 16:33 | |
| *** links has joined #openstack-sahara | 16:33 | |
| *** links has quit IRC | 16:35 | |
| *** links has joined #openstack-sahara | 16:36 | |
| *** links has quit IRC | 16:38 | |
| *** links has joined #openstack-sahara | 16:39 | |
| *** links has quit IRC | 16:41 | |
| *** links has joined #openstack-sahara | 16:42 | |
| *** links has quit IRC | 16:44 | |
| *** links has joined #openstack-sahara | 16:45 | |
| *** links has quit IRC | 16:48 | |
| *** links has joined #openstack-sahara | 16:48 | |
| *** links has quit IRC | 16:51 | |
| *** links has joined #openstack-sahara | 16:51 | |
| *** links has quit IRC | 16:54 | |
| *** links has joined #openstack-sahara | 16:54 | |
| *** links has quit IRC | 16:56 | |
| *** links has joined #openstack-sahara | 16:57 | |
| *** links has quit IRC | 17:00 | |
| *** links has joined #openstack-sahara | 17:01 | |
| *** tesseract has quit IRC | 17:03 | |
| *** links has quit IRC | 17:03 | |
| *** links has joined #openstack-sahara | 17:04 | |
| *** links has quit IRC | 17:06 | |
| *** links has joined #openstack-sahara | 17:07 | |
| *** links has quit IRC | 17:09 | |
| *** links has joined #openstack-sahara | 17:10 | |
| *** links has quit IRC | 17:12 | |
| *** links has joined #openstack-sahara | 17:12 | |
| *** links has quit IRC | 17:15 | |
| *** links has joined #openstack-sahara | 17:15 | |
| *** links has quit IRC | 17:18 | |
| *** links has joined #openstack-sahara | 17:18 | |
| *** links has quit IRC | 17:21 | |
| *** links has joined #openstack-sahara | 17:21 | |
| *** links has quit IRC | 17:24 | |
| *** links has joined #openstack-sahara | 17:24 | |
| *** links has quit IRC | 17:27 | |
| *** links has joined #openstack-sahara | 17:27 | |
| tellesnobrega | tosky, small patch to fix apiv2 cluster creation | 17:28 |
| tellesnobrega | we missed on the reviews probably | 17:28 |
| tellesnobrega | I might send another one for the dashboard as well soon | 17:28 |
| *** tmckay has joined #openstack-sahara | 17:28 | |
| tosky | tellesnobrega: Jeremy already sent a patch for the dashboard (hadoop_plugin) | 17:29 |
| *** links has quit IRC | 17:30 | |
| tellesnobrega | that is the one I saw as well | 17:30 |
| *** links has joined #openstack-sahara | 17:30 | |
| *** links has quit IRC | 17:33 | |
| *** links has joined #openstack-sahara | 17:33 | |
| *** links has quit IRC | 17:36 | |
| *** links has joined #openstack-sahara | 17:37 | |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara master: Fixing cluster creation on APIv2 https://review.openstack.org/587561 | 17:39 |
| *** links has quit IRC | 17:39 | |
| *** links has joined #openstack-sahara | 17:40 | |
| *** links has quit IRC | 17:42 | |
| *** links has joined #openstack-sahara | 17:42 | |
| *** links has quit IRC | 17:45 | |
| *** links has joined #openstack-sahara | 17:46 | |
| *** links has quit IRC | 17:52 | |
| openstackgerrit | Merged openstack/sahara-dashboard master: One missed hadoop_version->plugin_version https://review.openstack.org/587501 | 18:02 |
| *** jeremyfreudberg has joined #openstack-sahara | 18:03 | |
| jeremyfreudberg | hey guys, just wanted to let you know that my TSP was approved for this ptg | 18:04 |
| tosky | good to hear | 18:09 |
| *** jeremyfreudberg has quit IRC | 18:29 | |
| *** Bhujay has joined #openstack-sahara | 18:38 | |
| *** Bhujay has quit IRC | 18:48 | |
| *** tmckay has quit IRC | 21:46 | |
| *** rcernin has joined #openstack-sahara | 22:26 | |
| *** tosky has quit IRC | 23:35 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!