| *** links has joined #openstack-sahara | 04:39 | |
| openstackgerrit | OpenStack Proposal Bot proposed openstack/openstack-ansible-os_sahara master: Updated from OpenStack Ansible Tests https://review.openstack.org/565127 | 05:16 |
|---|---|---|
| *** masber has joined #openstack-sahara | 05:41 | |
| *** masuberu has joined #openstack-sahara | 05:51 | |
| *** masber has quit IRC | 05:55 | |
| *** masuberu has quit IRC | 06:02 | |
| *** tosky has joined #openstack-sahara | 07:03 | |
| *** Guest1988 has joined #openstack-sahara | 07:11 | |
| *** Guest1988 has quit IRC | 07:17 | |
| *** hoonetorg has quit IRC | 07:41 | |
| *** gokhan has joined #openstack-sahara | 07:50 | |
| *** gokhan has quit IRC | 07:50 | |
| *** hoonetorg has joined #openstack-sahara | 07:55 | |
| *** gokhan has joined #openstack-sahara | 08:12 | |
| openstackgerrit | Merged openstack/sahara master: Replace Chinese punctuation with English punctuation https://review.openstack.org/565008 | 09:09 |
| openstackgerrit | Merged openstack/openstack-ansible-os_sahara master: Updated from OpenStack Ansible Tests https://review.openstack.org/565127 | 10:07 |
| *** dave-mccowan has joined #openstack-sahara | 11:27 | |
| *** dave-mccowan has quit IRC | 11:31 | |
| *** dave-mcc_ has joined #openstack-sahara | 11:32 | |
| *** tellesnobrega has joined #openstack-sahara | 11:48 | |
| openstackgerrit | Merged openstack/sahara master: Fix the openstack endpoint create failed https://review.openstack.org/564815 | 11:49 |
| *** tellesnobrega has quit IRC | 11:53 | |
| *** tellesnobrega has joined #openstack-sahara | 11:54 | |
| *** bhujay has joined #openstack-sahara | 12:04 | |
| bhujay | hello everybody , need a little help , I am trying to run the examplewordcount on a spark cluster. The job fails and in the stderr i see spark master is trying to connect keystone using ssl and java is unable to validate the cert hence the job fails. Here is the complete log http://paste.openstack.org/show/720130/ | 12:08 |
| bhujay | will be grateful if you can hint some solution | 12:09 |
| bhujay | tosky: any suggestion ^ | 12:09 |
| tosky | not really | 12:11 |
| tosky | I guess that you are using swift for data input/output? | 12:11 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara stable/queens: Replace Chinese punctuation with English punctuation https://review.openstack.org/565199 | 12:13 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara stable/queens: Fix the openstack endpoint create failed https://review.openstack.org/565200 | 12:13 |
| bhujay | tosky: thats right sir | 12:14 |
| *** links has quit IRC | 13:22 | |
| bhujay | tosky: i downloaded the keystone (haproxy) self signed certificate and imported into jre of spark master. The ssl error is gine now . May be we should keep an automated option in the sahara plugin . | 13:39 |
| tosky | uhm uhm | 13:40 |
| tosky | tellesnobrega: ^^ | 13:40 |
| tosky | bhujay: if you have an account, can you please report this as story on storyboard.openstack.org? | 13:40 |
| tosky | so we don't forget | 13:40 |
| tosky | describing the issue in detail and the solution | 13:41 |
| bhujay | tosky: sure will do so | 13:41 |
| tellesnobrega | bhujay, thanks for looking into that | 13:41 |
| bhujay | tosky: anytime glad to help , its amazing the way you guys have developed it | 13:43 |
| bhujay | tellesnobrega: ^ | 13:43 |
| tellesnobrega | let me know if we can be of any assistance | 13:44 |
| bhujay | tosky , tellesnobrega now the error seems to be on account of input path not exists for hdfs://sparkcluster-1-sparkm1-0:8020/spark-input-ds(my swift datasource name) . Is it due to designate ? i have not installed designate .is designate mandatory or sahara will map them through ip address ? | 13:47 |
| tosky | no, designate is not mandatory; iirc you should have the hostnames populated in /etc/hosts | 13:48 |
| bhujay | ok | 13:48 |
| bhujay | tellesnobrega: is launchpad acceptable for reporting this ? | 13:50 |
| tosky | bhujay: no :) | 13:51 |
| tosky | bhujay: we moved to storyboard | 13:51 |
| bhujay | ok | 13:51 |
| *** ukaynar has joined #openstack-sahara | 14:49 | |
| openstackgerrit | Merged openstack/sahara stable/queens: Replace Chinese punctuation with English punctuation https://review.openstack.org/565199 | 14:49 |
| openstackgerrit | Merged openstack/sahara stable/queens: Fix the openstack endpoint create failed https://review.openstack.org/565200 | 14:49 |
| bhujay | tosky, tellesnobrega when spark master is trying to contact keystone it gets 504 with error as httpclient.HttpMethodBase: Going to buffer response body of large or unknown size. Using getResponseBodyAsStream instead is recommended | 15:06 |
| bhujay | the complete log is here http://paste.openstack.org/show/720144/ | 15:06 |
| bhujay | i did a curl to keystone from the same shell and it goes through | 15:06 |
| tosky | uhm, is that the code of the swift adapter? Because otherwise we are in the domain of hadoop code | 15:08 |
| bhujay | looks like swift | 15:09 |
| tosky | long time ago (unfortunately in this galaxy and not in a one far away) the sahara project forked the swift adapter | 15:10 |
| tosky | in order to add keystone v3 support, if I'm not mistaken | 15:10 |
| tosky | long == 3 years ago | 15:10 |
| tosky | we have a long standing item in our TODO list about rechecking the status of the current swift adapter to see if we can switch back to the official one, maybe contribute if there is some difference | 15:11 |
| tosky | of course the job is open for everyone :) | 15:11 |
| bhujay | tosky: thats a good hint , I will check if at all it is within my capabilities to understand . Well I am using sahara ocata and spark image 1.3.1 from mirantis upstream . Do you think queen ocata will have solved this problem ? | 15:15 |
| tosky | bhujay: I don't think that there were updats to the swift/hadoop adapter between those two versions | 15:15 |
| bhujay | ok | 15:16 |
| tosky | if you have time and would like to contribute, you may try that, sure, and also try to replace the upstream jar for swift/hadoop | 15:16 |
| tosky | and see if it explodes or not :) | 15:16 |
| tosky | that would help us A LOT | 15:16 |
| bhujay | thats a great deal of info , i will check | 15:18 |
| tellesnobrega | bhujay, I'm not sure that it got fixed (we weren't aware of this issue) but I would try spark 1.6.0 instead of 1.3.1 if it is possible for you as well | 16:13 |
| tellesnobrega | 1.3.1 is out of support for quite some time now | 16:13 |
| tellesnobrega | we are at spark 2.1 at this point iirc | 16:13 |
| bhujay | tellesnobrega: I am novice in sahara and started with what i image i found in mirantis upstream , that was on 1.3.1 . Your suggestion seems very logical but for that do i need to build a 1.6.0 image to use with 1.6.0 plugin ? | 16:15 |
| tosky | bhujay: for any supported version of openstack you need to build the image; due to resource constraints, we are not providing pre-built images anymore | 16:18 |
| bhujay | i was looking at the dates of jar files you mentioned , it looks like they are quite updated and since documents saying that latest jar file will be included during image build , it seems that will solve the adapter problem as well | 16:20 |
| tosky | tellesnobrega: I know that you are busy, but between a test and another, this first patch should unlock the others (because it will force a rebase on top of this): https://review.openstack.org/#/c/562892/ | 16:20 |
| tosky | bhujay: the last jar file that we (sahara) provide, not the last jar file from hadoop | 16:21 |
| tellesnobrega | tosky, no problem at all | 16:22 |
| tellesnobrega | looking at it now | 16:23 |
| bhujay | ok got it | 16:23 |
| tosky | for some reason, all the mox patches include the same change in a common file, instead of rebasing them (well, less dependency on a single patch) | 16:23 |
| tellesnobrega | bhujay, if you need any help at all don't hesitate to ask here, we will try to respond to the best of our capabilities | 16:23 |
| *** ukaynar has quit IRC | 16:23 | |
| tosky | so merging the first one should force a change in the other patches, so that they will be basically close to be merged | 16:24 |
| *** ukaynar has joined #openstack-sahara | 16:24 | |
| *** ukaynar has quit IRC | 16:25 | |
| *** ukaynar has joined #openstack-sahara | 16:26 | |
| tellesnobrega | nice | 16:26 |
| bhujay | tosky , tellesnobrega you guys are very responsive , Many thanks for your support . Sahara is very important for my org and hence i jumped into it the only thing is there is a lot of java and other stuff in this project unlike other projects in openstack . Hope I will be able to manage with little support from you guys | 16:27 |
| tellesnobrega | bhujay, great to hear | 16:28 |
| tellesnobrega | can you tell us what org do you work for? we would appreciate feedback from you | 16:28 |
| openstackgerrit | Merged openstack/sahara-dashboard master: Mox removal for DataProcessingJobTests https://review.openstack.org/562892 | 16:30 |
| bhujay | Sure , www.itcportal.com and https://www.linkedin.com/in/bhujaykbhatta/ | 16:32 |
| bhujay | tellesnobrega, tosky ^ | 16:33 |
| tosky | oh | 16:34 |
| tellesnobrega | bhujay, nice | 16:35 |
| tellesnobrega | thanks | 16:35 |
| bhujay | thank you guys , I will take some time to work on the suggestions given by you and revert back | 16:37 |
| tellesnobrega | bhujay, anytime, feel free to ping me or tosky | 16:40 |
| tosky | we also have other cores around, not always connected, but they read the logs :) | 16:41 |
| tosky | (Jeremy, I know that you read this) | 16:41 |
| *** ukaynar has quit IRC | 16:42 | |
| *** bhujay has quit IRC | 16:42 | |
| tellesnobrega | true | 16:44 |
| *** ukaynar has joined #openstack-sahara | 16:57 | |
| *** ukaynar has quit IRC | 17:32 | |
| *** elmiko has quit IRC | 17:55 | |
| *** elmiko has joined #openstack-sahara | 17:56 | |
| *** ukaynar has joined #openstack-sahara | 18:12 | |
| *** dave-mcc_ has quit IRC | 18:21 | |
| *** dave-mccowan has joined #openstack-sahara | 18:29 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: Remove the (now obsolete) pip-missing-reqs tox target https://review.openstack.org/565277 | 18:35 |
| *** ukaynar has quit IRC | 18:56 | |
| *** ukaynar has joined #openstack-sahara | 19:04 | |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Drop mox3 from DataProcessingJobBinaryTests https://review.openstack.org/564290 | 19:18 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Drop mox3 from DataProcessingNodeGroupTests https://review.openstack.org/565047 | 19:18 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Remove mox3 from DataProcessingClusterTemplateTests https://review.openstack.org/563424 | 19:19 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Remove mox from DataProcessingClusterGuideTests https://review.openstack.org/563423 | 19:19 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Drop mox3 from DataProcessingPluginsTests https://review.openstack.org/563374 | 19:19 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Remove mox from DataProcessingClusterTests https://review.openstack.org/563419 | 19:20 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Remove mox3 from DataProcessingImageRegistryTests https://review.openstack.org/563380 | 19:20 |
| openstackgerrit | Chuck Short proposed openstack/sahara-dashboard master: Mox removal for DataProcessingDataSourceTests https://review.openstack.org/563370 | 19:20 |
| *** dave-mccowan has quit IRC | 20:02 | |
| *** ukaynar has quit IRC | 20:11 | |
| *** dave-mccowan has joined #openstack-sahara | 20:16 | |
| *** dave-mccowan has quit IRC | 20:55 | |
| *** ukaynar has joined #openstack-sahara | 20:58 | |
| *** ukaynar has quit IRC | 21:56 | |
| *** ukaynar has joined #openstack-sahara | 21:58 | |
| *** rcernin has joined #openstack-sahara | 22:13 | |
| *** abalutoiu_ has joined #openstack-sahara | 22:51 | |
| *** abalutoiu__ has quit IRC | 22:54 | |
| *** ukaynar has quit IRC | 23:02 | |
| *** ukaynar has joined #openstack-sahara | 23:11 | |
| *** ukaynar has quit IRC | 23:21 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!