| *** jamielennox is now known as jamielennox|away | 01:01 | |
| *** jamielennox|away is now known as jamielennox | 01:19 | |
| *** tuanluong has joined #openstack-sahara | 01:37 | |
| *** shuyingya has joined #openstack-sahara | 01:52 | |
| *** dave-mccowan has quit IRC | 02:09 | |
| *** links has joined #openstack-sahara | 03:09 | |
| *** shuyingya has quit IRC | 03:14 | |
| *** shuyingya has joined #openstack-sahara | 03:14 | |
| *** mpotdar has quit IRC | 03:38 | |
| *** mpotdar has joined #openstack-sahara | 03:43 | |
| *** shuyingy_ has joined #openstack-sahara | 03:51 | |
| *** shuyingya has quit IRC | 03:54 | |
| *** jamielennox is now known as jamielennox|away | 04:14 | |
| *** jamielennox|away is now known as jamielennox | 04:22 | |
| *** Poornima has joined #openstack-sahara | 05:04 | |
| *** pgadiya has joined #openstack-sahara | 05:10 | |
| *** rcernin has joined #openstack-sahara | 05:26 | |
| *** shuyingy_ has quit IRC | 05:30 | |
| *** shuyingya has joined #openstack-sahara | 05:30 | |
| *** shuyingy_ has joined #openstack-sahara | 05:39 | |
| *** pgadiya has quit IRC | 05:41 | |
| *** shuyingya has quit IRC | 05:42 | |
| *** nkrinner_afk is now known as nkrinner | 06:28 | |
| *** tuanluong has quit IRC | 06:50 | |
| *** zemuvier has joined #openstack-sahara | 07:19 | |
| *** pcaruana has joined #openstack-sahara | 07:20 | |
| *** tuanluong has joined #openstack-sahara | 07:31 | |
| *** tesseract has joined #openstack-sahara | 07:44 | |
| *** abehl has joined #openstack-sahara | 07:55 | |
| *** openstackgerrit has quit IRC | 08:33 | |
| *** shuyingy_ has quit IRC | 09:20 | |
| *** shuyingya has joined #openstack-sahara | 09:21 | |
| *** shuyingy_ has joined #openstack-sahara | 09:25 | |
| *** shuyingya has quit IRC | 09:28 | |
| *** vgridnev has joined #openstack-sahara | 09:39 | |
| *** tosky has joined #openstack-sahara | 09:45 | |
| *** openstackgerrit has joined #openstack-sahara | 10:05 | |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara-tests master: Add scripts for multinode job with spark https://review.openstack.org/446400 | 10:05 |
|---|---|---|
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara-tests master: Try to use m1.small flavor in multinode https://review.openstack.org/449564 | 10:07 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara-tests master: Try to use m1.small flavor in multinode https://review.openstack.org/449564 | 10:08 |
| openstackgerrit | Shu Yingya proposed openstack/sahara-dashboard master: Fix bugs of redirect url https://review.openstack.org/450156 | 10:36 |
| *** dave-mccowan has joined #openstack-sahara | 11:18 | |
| *** abalutoiu_ has joined #openstack-sahara | 11:51 | |
| *** abalutoiu has quit IRC | 11:54 | |
| *** tuanluong has quit IRC | 11:56 | |
| *** dave-mccowan has quit IRC | 12:12 | |
| *** iwonka has joined #openstack-sahara | 12:15 | |
| iwonka | hi everyone! | 12:17 |
| *** zemuvier has quit IRC | 12:17 | |
| iwonka | (i want to be an intern and i'm trying to do my first contribution) | 12:18 |
| iwonka | i've done these two changes: | 12:18 |
| iwonka | https://review.openstack.org/#/c/448291/5 | 12:18 |
| iwonka | https://review.openstack.org/#/c/448245/ | 12:18 |
| iwonka | and I wonder, what should be my next steps? | 12:18 |
| iwonka | I mean to correct them | 12:18 |
| tosky | tellesnobrega, vgridnev: you are in a better position than me to give some advice ^^ | 12:19 |
| tosky | (and please add the +2+w if needed to the reviews, they already have one +2) | 12:19 |
| tosky | oh, already done, didn't check the logs | 12:20 |
| openstackgerrit | Shu Yingya proposed openstack/sahara-dashboard master: Fix bugs of redirect url https://review.openstack.org/450156 | 12:21 |
| tellesnobrega | iwonka, hi, both patches are on the way to be merged, both were already approved, now you can continue with the application process | 12:21 |
| tellesnobrega | no action needed on this side anymore | 12:21 |
| iwonka | oh, okay | 12:21 |
| iwonka | so I just paste there links from review in my application? | 12:21 |
| iwonka | *these | 12:21 |
| tellesnobrega | iwonka, I'm not sure if you have to, let me check | 12:31 |
| iwonka | tellesnobrega: "Please describe your experience with the organization's product as a user and as a contributor (include the information, as well as a link or an attachment, for the required contribution you made to the project you are interested in here)" | 12:36 |
| *** shuyingy_ has quit IRC | 12:36 | |
| tellesnobrega | iwonka, so yes, paste those links there | 12:36 |
| iwonka | tellesnobrega: ok, thanks | 12:37 |
| *** links has quit IRC | 12:40 | |
| *** chlong has joined #openstack-sahara | 12:51 | |
| iwonka | one more question | 12:56 |
| iwonka | about my second patch (adding saharaclient) | 12:56 |
| tellesnobrega | iwonka, go ahead | 12:56 |
| iwonka | so, there are two approaches, one is to just install it, the other it to add it to requirements, which is not perfect, because it adds something unnecessary. so maybe it's better to add saharaclient only in sahara_dashboard is not enabled | 12:57 |
| iwonka | or should the requirements be the same always? | 12:57 |
| iwonka | i mean in function installsaharaclient (the bash one) | 12:58 |
| tellesnobrega | I think requirements should be the same always. It won't matter really, if both projects have the same requirement, it will only install once | 12:58 |
| iwonka | ok, so i'll just leave it as it is | 12:59 |
| openstackgerrit | Merged openstack/sahara master: install saharaclient from pypi if not from source https://review.openstack.org/448291 | 13:15 |
| openstackgerrit | Merged openstack/sahara master: Adds information about using bash to documentation. https://review.openstack.org/448245 | 13:15 |
| *** Poornima has quit IRC | 13:23 | |
| *** shuyingya has joined #openstack-sahara | 14:07 | |
| *** chlong has quit IRC | 14:09 | |
| *** dave-mccowan has joined #openstack-sahara | 14:16 | |
| *** abalutoiu__ has joined #openstack-sahara | 14:35 | |
| *** saphi has joined #openstack-sahara | 14:36 | |
| saphi | Hi everybody. I was installed sahara and launched cluster successful. But I can't run demo job (wordcount) with Input from Swift. After I launch job. The status is "Done with Error". My Lab Openstack Ocata Version, Spark Job and Spark cluster. Can you help me? | 14:38 |
| *** abalutoiu_ has quit IRC | 14:38 | |
| tosky | well, did you properly configure the swift datasource? Can the user that you specified access that swift bucket? Does the swift "URL" point to an empty "folder"? | 14:41 |
| saphi | Hi tosky! I followed this guide: https://github.com/openstack/sahara-tests/tree/master/sahara_tests/scenario/defaults/edp-examples/edp-spark. Datasource was added by args. | 14:45 |
| saphi | When I launch the cluster. I saw the log with swift credentials, The auth_url is pointed to keystone v2 not v3. I this is a problem. | 14:46 |
| tosky | is keystone v3 configured on your cluster? | 14:46 |
| tosky | was the URL something like swift://foo.sahara/bucket ? | 14:47 |
| saphi | Yes. Keystone v3 is configured. | 14:51 |
| tosky | and did you use the special URL for the swift resource as explained? | 14:52 |
| saphi | the input url conform swift://<container>/<object_name> | 14:52 |
| tosky | container.sahara | 14:52 |
| tosky | you need to add .sahara | 14:52 |
| saphi | let me try | 14:52 |
| tellesnobrega | tosky, the path being passed as args, I don't think the .sahara is necessary for spark | 15:00 |
| tosky | uhm | 15:01 |
| tosky | doesn't spark use hadoop-swift as well? | 15:01 |
| *** rcernin has quit IRC | 15:07 | |
| *** nkrinner is now known as nkrinner_afk | 15:08 | |
| tellesnobrega | tosky, I'm not sure, but I ran some stuff on it recently, and it worked without the .sahara, iirc | 15:11 |
| tellesnobrega | but it wasn't a data_source, it was a path on args when launching the job | 15:12 |
| *** Llllllllllll has joined #openstack-sahara | 15:13 | |
| saphi | Hi tellesnobrega, Did you run on Openstack ocata version? | 15:19 |
| *** Llllllllllll has quit IRC | 15:20 | |
| *** chlong has joined #openstack-sahara | 15:25 | |
| saphi | tosky, I added .sahara to URL but that is not solution. | 15:28 |
| saphi | I think the problem related swift info was passed to cluster: https://pastebin.com/mBGJPeMK | 15:29 |
| tosky | uhm uhm | 15:37 |
| tosky | I don't have an environment to reproduce properly right now, but how did you configure sahara? | 15:44 |
| saphi | tosky. My sahara.conf file: http://paste.ubuntu.com/24261509/ | 15:47 |
| tosky | saphi: identity_uri missing maybe; this is a (work-in-progress deployment, I don't have a cluster right now) of Ocata: http://paste.openstack.org/show/604337/ | 15:50 |
| saphi | I set. But nothing change | 16:04 |
| tosky | after restarting all, recreating the cluster, etc? | 16:05 |
| saphi | https://pastebin.com/Xf6dUCfB | 16:06 |
| saphi | I check stdout log of spark. It try to authenticate with keystone v2 | 16:06 |
| saphi | keystone v2 on ocata is not working anymore | 16:06 |
| *** vgridnev has quit IRC | 16:07 | |
| saphi | I'm creating new cluster | 16:08 |
| *** jeremyfreudberg has joined #openstack-sahara | 16:23 | |
| *** jeremyfreudberg has quit IRC | 16:23 | |
| *** abehl has quit IRC | 16:27 | |
| saphi | Nothing changes tosky | 16:31 |
| tosky | same values passed to swift for authentication? | 16:31 |
| saphi | I think you are running keystone v2. So Spark can connect to keystone to authenticate | 16:31 |
| saphi | You can check my log | 16:32 |
| *** shuyingya has quit IRC | 16:35 | |
| *** rcernin has joined #openstack-sahara | 16:37 | |
| tosky | so no keystone v2 at all? But it is still active on ocata (unless it was disabled in the deployment) | 16:38 |
| *** tesseract has quit IRC | 16:59 | |
| saphi | What version of Openstack did you try on? | 17:03 |
| tellesnobrega | saphi, I used newton, but we use v3 for keystone | 17:03 |
| saphi | Can you tell me how to launch a spark job example? I follow that guide (I posted it above) But It failed | 17:05 |
| tellesnobrega | saphi, sure | 17:05 |
| saphi | Yes, I'm sure | 17:06 |
| tellesnobrega | 1) Upload your spark binary file to a swift container | 17:06 |
| tellesnobrega | 2) Create job_binary on sahara giving swift path in the form swift://<container>.sahara/<path_to_file> | 17:07 |
| *** rcernin has quit IRC | 17:08 | |
| tellesnobrega | 3) Create job_template using only main_library and set type to spark | 17:08 |
| *** pcaruana has quit IRC | 17:12 | |
| tellesnobrega | 4) launch that on existing cluster passing its main class and on args add swift://<container>/<path_to_input_file> and swift://<container>/<path_to_output_file> | 17:13 |
| tellesnobrega | saphi, remember to delete output path if you want to run again, it fails if the output path exists | 17:13 |
| saphi | I have launched job same to you. But I upload binary to internal database instead of swift | 17:15 |
| saphi | Can you show me your sahara.conf file. Or you can review my config file here http://paste.ubuntu.com/24261509/ | 17:16 |
| *** chlong has quit IRC | 17:22 | |
| tellesnobrega | saphi, your file looks ok to me, maybe checking the identity_uri, but I'm not that would be an issue | 17:30 |
| tellesnobrega | I'm looking for the sahara.conf | 17:31 |
| tellesnobrega | I don't have direct access to it, but I'm asking for it | 17:31 |
| tellesnobrega | is your installation a devstack or full openstack deploy? | 17:31 |
| saphi | I installed openstack manually | 17:35 |
| tellesnobrega | ok | 17:35 |
| tellesnobrega | here is my sahara.conf | 17:35 |
| tellesnobrega | http://paste.openstack.org/show/HkPfdQQTNiPYJor6c927/ | 17:35 |
| tellesnobrega | but I don't see much difference | 17:36 |
| saphi | use_domain_for_proxy_users = True proxy_user_domain_name = sahara_proxy proxy_user_role_names = Member | 17:36 |
| saphi | I think these are difference | 17:37 |
| *** chlong has joined #openstack-sahara | 17:37 | |
| tosky | oh, right! | 17:39 |
| tosky | saphi: https://elmiko.github.io/2015/06/25/configuring-sahara-with-proxy-domains.html | 17:39 |
| tosky | (hi elmiko) | 17:39 |
| saphi | I will try it now | 17:41 |
| *** tosky has quit IRC | 17:45 | |
| *** dave-mccowan has quit IRC | 17:46 | |
| tellesnobrega | right, that might make sense | 17:50 |
| *** dave-mccowan has joined #openstack-sahara | 18:01 | |
| openstackgerrit | mimansa04 proposed openstack/sahara master: added timeout function in health check function https://review.openstack.org/449308 | 18:12 |
| saphi | hey tellesnobrega, I was created swift-proxy domain and configured sahara use that domain as you. But When launch job it can't create trust user with an error message: {"error": {"message": "Could not find role: 7159755453174dcc92d82ff7f62c6764.", "code": 404, "title": "Not Found"}} | 18:13 |
| saphi | I check the role id is exist | 18:14 |
| tellesnobrega | that is very weird, when I run devstack, I usually don't set proxy users for sahara, I'm not entirely sure how to proceed on this. what role is that? Member? admin? service? | 18:15 |
| saphi | the role name is 'user' | 18:16 |
| *** tosky has joined #openstack-sahara | 18:18 | |
| saphi | Thank tellesnobrega and tosky. But I'm in Viet Name. And now is 1h30 am. I have to sleep. I will try later. | 18:24 |
| *** saphi has quit IRC | 18:24 | |
| *** stanchan has joined #openstack-sahara | 18:32 | |
| *** stanchan has quit IRC | 18:59 | |
| *** stanchan has joined #openstack-sahara | 19:00 | |
| *** jamielennox is now known as jamielennox|away | 20:13 | |
| *** jamielennox|away is now known as jamielennox | 20:24 | |
| *** shuyingya has joined #openstack-sahara | 21:10 | |
| *** dave-mccowan has quit IRC | 21:27 | |
| *** chlong has quit IRC | 22:03 | |
| *** tosky has quit IRC | 23:09 | |
| openstackgerrit | Marianne Linhares Monteiro proposed openstack/sahara master: [APIv2] Convert update methods to use PATCH https://review.openstack.org/450484 | 23:11 |
| openstackgerrit | Marianne Linhares Monteiro proposed openstack/sahara master: [APIv2] Convert update methods to use PATCH https://review.openstack.org/450484 | 23:14 |
| *** shuyingya has quit IRC | 23:59 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!