| guimaluf | tellesnobrega, how could I run yarn within spark plugin image | 00:24 |
|---|---|---|
| guimaluf | ? | 00:24 |
| guimaluf | actually is not clear to me how, after creating a cluster, create jobs or data sources through sahara | 01:12 |
| *** dave-mccowan has joined #openstack-sahara | 01:56 | |
| *** guimaluf has quit IRC | 02:28 | |
| *** guimaluf has joined #openstack-sahara | 02:37 | |
| *** dave-mccowan has quit IRC | 03:08 | |
| *** Bhujay has joined #openstack-sahara | 05:01 | |
| *** Bhujay has quit IRC | 05:01 | |
| *** Bhujay has joined #openstack-sahara | 05:02 | |
| *** pcaruana has joined #openstack-sahara | 06:51 | |
| *** tosky has joined #openstack-sahara | 07:03 | |
| *** rcernin has quit IRC | 07:55 | |
| tellesnobrega | guimaluf, I'm not 100% sure, but I don't think our spark image comes with yarn | 10:55 |
| *** dave-mccowan has joined #openstack-sahara | 12:19 | |
| guimaluf | tellesnobrega, it comes with yarn, but isn't started... at least there is a yarn binary there... | 12:23 |
| tellesnobrega | hmm, what version of spark are you using? | 12:24 |
| tellesnobrega | 1.6.0 or 2.1.0? | 12:24 |
| guimaluf | tellesnobrega, 2.3.2 acctualy eheheh | 13:17 |
| guimaluf | I'm using 2.3.2 cause I felt no difference using 2.1.0 | 13:17 |
| guimaluf | and I can't run any example job :( | 13:19 |
| tellesnobrega | guimaluf, just reminding myself, you are running pike right? | 13:40 |
| guimaluf | tellesnobrega, exactly | 13:40 |
| guimaluf | I generated my imagem thorugh stable/pike branch | 13:40 |
| tellesnobrega | a spark 2.3 image? | 13:40 |
| Gaasmann | tellesnobrega: 2.7.5 wouldn't explain the oozie issue, I'll try to look at logs inside the server. I wonder if it's not some jar missing | 13:41 |
| tellesnobrega | Gaasmann, that is probably what is happening, we have to identify which one | 13:42 |
| guimaluf | tellesnobrega, yep 2.3.2, but I can run a 2.1.0 cluster since I generated this image also | 13:47 |
| guimaluf | but I'm struggling with jobs and testing... I can't tell for sure if I'm using my worker nodes or not | 13:47 |
| guimaluf | maybe I lack knowledge on spark tools :/ | 13:48 |
| *** tosky__ has joined #openstack-sahara | 13:51 | |
| *** tosky has quit IRC | 13:51 | |
| *** tosky__ is now known as tosky | 13:52 | |
| *** SergeyLukjanov has quit IRC | 14:04 | |
| *** Bhujay has quit IRC | 14:06 | |
| *** SergeyLukjanov has joined #openstack-sahara | 14:06 | |
| *** Bhujay has joined #openstack-sahara | 14:07 | |
| *** Bhujay has quit IRC | 14:16 | |
| Gaasmann | ok, I was able to make the oozie website working by deleting /opt/oozie/oozie-server/webapps/oozie/WEB-INF/lib/jsp-api-2.0.jar and restarting oozie | 14:50 |
| Gaasmann | as suggested here http://hadooptutorials.info/2017/09/18/part-7-install-oozie/ | 14:50 |
| Gaasmann | now I have an arror about an hadoop class missing when I try to exec an EDP job | 14:51 |
| Gaasmann | and I nearly able to exec the job by copying the hadoop jar in libext and run oozie-setup.sh prepare-war as mentionned in this article also | 15:00 |
| Gaasmann | the job ran but the status is KILLED in sahara and the result is nowhere, I have to check why | 15:00 |
| tosky | an error writing down the result on hdfs/swift? | 15:01 |
| Gaasmann | anyway, if it's a "correct" way to fix oozie problem, it sounds like an upstream problem but I'm very not a java/tomcat expert | 15:01 |
| Gaasmann | tosky: That's what I think, I'm looking at logs | 15:02 |
| Gaasmann | well in fact it didn't work. YARN is saying success but the container logs say it wasn't able to find the main class :-) | 15:10 |
| *** pcaruana has quit IRC | 15:33 | |
| *** pcaruana has joined #openstack-sahara | 18:24 | |
| *** pcaruana has quit IRC | 18:50 | |
| *** tosky has quit IRC | 22:24 | |
| *** rcernin has joined #openstack-sahara | 22:25 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!