| tmckay | I was using an older version | 00:01 |
|---|---|---|
| *** bradd1 has joined #openstack-sahara | 00:12 | |
| *** elmiko_ is now known as elmiko | 00:23 | |
| *** bradd1 has quit IRC | 00:28 | |
| tmckay | elmiko, you interested in bigpetstore instructions too? | 00:33 |
| elmiko | sure, why not :) | 00:33 |
| tmckay | okay, just running through it again to make sure it's up to date with the current git. I'll cc you on the updated instructions | 00:34 |
| elmiko | cool, thanks | 00:35 |
| tmckay | tomorrow maybe I can get to the bottom of the piglib. that would be great. With plenty of time to spare :) | 00:35 |
| *** witlessb has quit IRC | 00:48 | |
| tmckay | aignatov, okay, I sent you instructions. You should be able to build the bps jar yourself, and grab the oozie pig lib from a 2.3.0 node. I can compress them and send them to you otherwise. | 00:52 |
| *** tmckay is now known as _tmckay | 00:52 | |
| *** elmiko has quit IRC | 01:01 | |
| *** bradd1 has joined #openstack-sahara | 01:03 | |
| *** witlessb has joined #openstack-sahara | 01:56 | |
| *** witlessb has quit IRC | 02:00 | |
| *** ErikB2 has joined #openstack-sahara | 02:25 | |
| *** IlyaE has quit IRC | 02:34 | |
| *** IlyaE has joined #openstack-sahara | 02:43 | |
| *** akuznetsov has joined #openstack-sahara | 02:50 | |
| *** zhiyan_ is now known as zhiyan | 02:53 | |
| *** IlyaE has quit IRC | 03:05 | |
| *** ErikB2 has quit IRC | 03:24 | |
| *** Ch00k has joined #openstack-sahara | 03:33 | |
| *** bradd1 has quit IRC | 03:47 | |
| *** bradd1 has joined #openstack-sahara | 03:48 | |
| *** bradd1 has joined #openstack-sahara | 03:50 | |
| *** akuznetsov has quit IRC | 04:08 | |
| *** Ch00k has quit IRC | 04:20 | |
| *** akuznetsov has joined #openstack-sahara | 04:44 | |
| *** IlyaE has joined #openstack-sahara | 05:22 | |
| *** IlyaE has quit IRC | 05:31 | |
| *** Ch00k has joined #openstack-sahara | 06:20 | |
| *** IlyaE has joined #openstack-sahara | 06:32 | |
| *** IlyaE has quit IRC | 06:40 | |
| *** tnovacik has joined #openstack-sahara | 07:05 | |
| *** zhiyan is now known as zhiyan_ | 07:26 | |
| *** witlessb has joined #openstack-sahara | 07:29 | |
| *** skolekonov has joined #openstack-sahara | 07:33 | |
| *** Ch00k has quit IRC | 07:42 | |
| *** Ch00k has joined #openstack-sahara | 08:34 | |
| *** ylobankov has joined #openstack-sahara | 08:44 | |
| *** Ch00k has quit IRC | 08:54 | |
| *** Ch00k has joined #openstack-sahara | 08:59 | |
| *** Ch00k has quit IRC | 09:55 | |
| *** tosky has joined #openstack-sahara | 09:59 | |
| *** Ch00k has joined #openstack-sahara | 10:10 | |
| *** Ch00k has quit IRC | 10:36 | |
| *** Ch00k has joined #openstack-sahara | 11:34 | |
| *** crobertsrh has joined #openstack-sahara | 11:37 | |
| *** _mattf is now known as mattf | 11:39 | |
| *** ErikB1 has joined #openstack-sahara | 12:13 | |
| *** _tmckay is now known as tmckay | 12:54 | |
| openstackgerrit | Daniele Venzano proposed a change to openstack/sahara: Add Spark plugin to Sahara https://review.openstack.org/73989 | 13:30 |
| crobertsrh | anyone know off hand whether or not the keystone service catalog gets cached? | 13:41 |
| crobertsrh | ...and how to clear the cache manually? | 13:41 |
| crobertsrh | I'm thinking some caching must be going on. I setup my stack....ran horizon, got Serv Catalog exception....added sahara service type and endpoints, but still get SC exception | 13:42 |
| dmitryme | crobertsrh: where do you get it? | 13:54 |
| openstackgerrit | Maxence Dalmais proposed a change to openstack/sahara: Update Manifest.in to fix hdp resources path https://review.openstack.org/92589 | 13:54 |
| crobertsrh | Anytime I hit a sahara panel (which tries to get the sahara endpoint) | 13:55 |
| crobertsrh | I think I figured it out | 13:55 |
| crobertsrh | there is a config in keystone for cache expiration | 13:55 |
| dmitryme | crobertsrh: interesting, didn’t met the problem so far | 13:58 |
| crobertsrh | Hmm, maybe that didn't fix it | 13:58 |
| crobertsrh | I changed the expiry value and restarted keystone | 13:58 |
| crobertsrh | I'm still getting the service catalog exception | 13:58 |
| *** skolekonov has quit IRC | 13:59 | |
| dmitryme | run ‘keystone catalog’, do you get the right endpoint with the right name? | 13:59 |
| dmitryme | * with the right type - ‘data_processing’ | 14:01 |
| crobertsrh | Yes | 14:01 |
| dmitryme | I have no other ideas what might be missing | 14:05 |
| crobertsrh | I'm just restacking and hoping it will be resolved | 14:07 |
| crobertsrh | working fine now | 14:13 |
| *** shakamunyi has joined #openstack-sahara | 14:15 | |
| openstackgerrit | A change was merged to openstack/sahara: correcting the MANIFEST.in paths https://review.openstack.org/92441 | 14:21 |
| openstackgerrit | A change was merged to stackforge/puppet-sahara: Updating the puppet module to reference Sahara https://review.openstack.org/92111 | 14:21 |
| crobertsrh | tmckay: Looks like the tables not updating issue is actually a problem in the horizon code itself. I've talked with the horizon crew and filed a bug. https://bugs.launchpad.net/horizon/+bug/1317124 | 14:43 |
| tmckay | crobertsrh, cool, thanks! | 14:43 |
| *** Ch00k has quit IRC | 14:47 | |
| tmckay | aignatov, ping, did you see my email? Make sense? | 14:51 |
| tmckay | on the bigpetstore run | 14:51 |
| *** IlyaE has joined #openstack-sahara | 14:58 | |
| *** bradd1 has joined #openstack-sahara | 15:00 | |
| crobertsrh | tmckay: side note...I have also uploaded a fix for the bug. If you want, I can update the OS1 instance with the fix this afternoon. | 15:08 |
| *** shakamunyi has quit IRC | 15:20 | |
| openstackgerrit | Maxence Dalmais proposed a change to openstack/sahara: correcting the MANIFEST.in paths https://review.openstack.org/92608 | 15:22 |
| mattf | crobertsrh, service catalog is cached client side | 15:22 |
| crobertsrh | Of course it is...that explains why my earlier efforts were fruitless :) | 15:23 |
| mattf | or as part of the client's session | 15:23 |
| mattf | logout + in may do the trick | 15:23 |
| *** shakamunyi has joined #openstack-sahara | 15:23 | |
| crobertsrh | I eventually figured it out | 15:23 |
| mattf | ^^ hard fought knowledge | 15:24 |
| crobertsrh | And this time, I added the endpoint creation to my local.sh for my devstack. I shouldn't have to worry about it again. | 15:24 |
| tmckay | crobertsrh, yeah, an update would be fine | 15:50 |
| crobertsrh | Ok, I should be able to make that happen | 15:50 |
| aignatov | tmckay: yes, thanx for the detailed dscription for bigpetstore scenario | 16:08 |
| tmckay | aignatov, still poking at sharelib issues. Going to go ahead and see if I can run part 3 standalone first. I did find "mapreduce.job.classpath.files" buried in the conf.xml for a Java job, which references the sharelib. I think the key for Java which invokes Pig may be to figure out how to get the Pig lib listed in that value | 16:11 |
| tmckay | It's a comma separated list | 16:12 |
| tmckay | another solution in the future possibly is to implement <files> and <archives> tags, too. | 16:13 |
| *** IlyaE has quit IRC | 16:26 | |
| *** IlyaE has joined #openstack-sahara | 16:34 | |
| *** Ch00k has joined #openstack-sahara | 16:44 | |
| tmckay | aignatov, update, I think Pig will run, the case of "$input" and "$output" made it break. Now I just have to get the path right (I think we need my hdfs path patch) | 16:53 |
| tmckay | crobertsrh, ^^ any progress on UI fix for simple hdfs paths? | 16:54 |
| crobertsrh | I haven't had any chance to look yet. | 16:54 |
| crobertsrh | puppet madness mostly, then the horizon bug fix this morning | 16:54 |
| crobertsrh | Hopefully, I can get to it this afternoon | 16:54 |
| *** bradd1 has quit IRC | 16:55 | |
| tmckay | crobertsrh, you know what else we need? A UI link to pull up the Sahara logs for dev support | 17:01 |
| tmckay | well, maybe not. I should at least pipe the logs somewhere. | 17:02 |
| tmckay | hmm, I don't seem to be able to submit jobs to oozie anymore. Need a restart | 17:04 |
| crobertsrh | tmckay: mind if I restart horizon now? | 17:07 |
| tmckay | crobertsrh, no, that's fine. I was about to restart sahara | 17:09 |
| tmckay | I set the logfile to sahara.out | 17:09 |
| tmckay | crobertsrh, ok, it's logging to sahara.out and I turned on debug | 17:11 |
| crobertsrh | ah, ok :) | 17:11 |
| tmckay | Something happened with the connection to Oozie ... | 17:11 |
| tmckay | that would have been a good one to see | 17:11 |
| tmckay | seeing as Oozie is currently the cornerstone | 17:12 |
| crobertsrh | Ok, looks like horizon is up with my fix included | 17:12 |
| crobertsrh | Let me know if you continue to see non-updates. | 17:12 |
| tmckay | crobertsrh, thank you | 17:12 |
| tmckay | crobertsrh, hmm, had repeated failures to write to hdfs from sahara | 17:23 |
| tmckay | that's a new one. | 17:23 |
| *** ErikB1 has quit IRC | 17:24 | |
| crobertsrh | That is fresh and exciting | 17:25 |
| crobertsrh | Was it with an old cluster that had been tinkered with? | 17:25 |
| *** tosky has quit IRC | 17:29 | |
| *** IlyaE has quit IRC | 17:30 | |
| *** ErikB1 has joined #openstack-sahara | 17:33 | |
| *** tosky has joined #openstack-sahara | 17:37 | |
| tmckay | crobertsrh, it wasn't too old. I think I launched it end of last week | 17:38 |
| tmckay | It was working fine all day yesterday | 17:38 |
| crobertsrh | Strange. Any chance one of the processes died somehow? | 17:38 |
| *** bradd1 has joined #openstack-sahara | 17:48 | |
| tmckay | crobertsrh, not sure. I didn't poke at it too hard since I'm chasing other issues | 17:50 |
| tmckay | crobertsrh, btw, looks like updates are working | 17:50 |
| *** themistymay has quit IRC | 17:51 | |
| *** IlyaE has joined #openstack-sahara | 17:51 | |
| *** Ch00k has quit IRC | 17:53 | |
| crobertsrh | great! | 17:57 |
| *** bradd1 has quit IRC | 18:01 | |
| *** Ch00k has joined #openstack-sahara | 18:04 | |
| *** Ch00k has quit IRC | 18:07 | |
| tmckay | crobertsrh, heh, that hdfs error is apparently because there is something wrong with the binary internal. Download link from sahara fails too. | 18:16 |
| tmckay | I'm guess for some reason the data is coming back None. Perhaps upload failed in the first place? | 18:16 |
| tmckay | I may go back and instert pdb on that one later ... | 18:17 |
| *** tnovacik has quit IRC | 18:18 | |
| *** themistymay has joined #openstack-sahara | 18:20 | |
| crobertsrh | hmm, that's strange. Was it a fairly large file? | 18:20 |
| tmckay | no, 3k. I can't seem to upload any job binaries to internal on OS1 currently | 18:21 |
| tmckay | looks like it's failing during keystone auth of the request | 18:22 |
| tmckay | I'll try starting again | 18:23 |
| *** tosky has quit IRC | 18:23 | |
| *** bradd1 has joined #openstack-sahara | 18:29 | |
| crobertsrh | That's really weird. | 18:31 |
| crobertsrh | I'll give it a go as well. | 18:31 |
| crobertsrh | I can't even connect atm | 18:31 |
| tmckay | crobertsrh, I wonder if a token expired. | 18:33 |
| crobertsrh | Could be, but that's still strange. We would have seen it before, wouldn't we? | 18:34 |
| crobertsrh | I can't upload either | 18:34 |
| tmckay | UnicodeDecodeError: 'utf8' codec can't decode byte 0x85 in position 11: invalid start byte | 18:34 |
| tmckay | alright, I may just try a restart | 18:38 |
| tmckay | crobertsrh, so when we start this, we don't need to source an rc or anything, do we? | 18:44 |
| crobertsrh | no | 18:44 |
| tmckay | k, gonna try restsrt sahara/horizon both. | 18:45 |
| crobertsrh | go for it | 18:45 |
| crobertsrh | any chance binary upload works from cli? | 18:45 |
| tmckay | can't try the cli, because we still have that cert token problem | 18:46 |
| tmckay | We're not set up for it yet | 18:46 |
| crobertsrh | ah :) | 18:46 |
| crobertsrh | Is it the json parsing that is throwing that exception? | 18:47 |
| tmckay | yeah | 18:51 |
| tmckay | I wonder if a horizon update killed it | 18:52 |
| tmckay | JSONDecodeError: Expecting value: line 1 column 2 (char 1) | 18:52 |
| tmckay | is it because I have logging on? | 18:52 |
| tmckay | crobertsrh, yeah, it's below the "log_exchange.py" call | 18:52 |
| tmckay | because I had it output requests/response | 18:53 |
| tmckay | Let me turn that off. Heisenbug. | 18:53 |
| crobertsrh | Possible. I can revert to something less recent. | 18:54 |
| crobertsrh | I think there were a few other changes that came in with my change. | 18:55 |
| *** themistymay has quit IRC | 18:55 | |
| tmckay | that's it, crobertsrh. Logging request/response crashes Sahara request processing inside the middleware_auth_token stuff | 18:58 |
| tmckay | aignatov, fyi ^^. This needs a launchpad bug. | 18:58 |
| tmckay | sigh, finally. I got the bps_analytics to run as a standalone 3rd job. | 19:06 |
| tmckay | now the only question is what's up with the oozie share lib and Pig | 19:06 |
| tmckay | and having the relative path fix in would be nice. | 19:07 |
| *** Ch00k has joined #openstack-sahara | 19:24 | |
| *** IlyaE has quit IRC | 19:25 | |
| *** IlyaE has joined #openstack-sahara | 19:30 | |
| *** akuznets_ has joined #openstack-sahara | 19:47 | |
| *** akuznetsov has quit IRC | 19:48 | |
| *** akuznets_ has quit IRC | 19:53 | |
| *** IlyaE has quit IRC | 20:06 | |
| *** IlyaE has joined #openstack-sahara | 20:09 | |
| *** bradd1 has quit IRC | 20:12 | |
| *** shakamunyi has quit IRC | 20:27 | |
| *** bradd1 has joined #openstack-sahara | 20:32 | |
| *** IlyaE has quit IRC | 20:54 | |
| *** shakamunyi has joined #openstack-sahara | 20:55 | |
| *** Ch00k has quit IRC | 20:57 | |
| *** shakamunyi has quit IRC | 21:07 | |
| *** bradd1 has quit IRC | 21:14 | |
| *** crobertsrh is now known as _crobertsrh | 21:21 | |
| *** sballe has quit IRC | 21:24 | |
| *** sballe has joined #openstack-sahara | 21:25 | |
| *** bradd1 has joined #openstack-sahara | 21:38 | |
| *** shakamunyi has joined #openstack-sahara | 21:41 | |
| openstackgerrit | Robert Nettleton proposed a change to openstack/sahara-image-elements: Turns off ambari and hadoop2 services for initial VM boot in HDP images https://review.openstack.org/92715 | 22:16 |
| *** ErikB1 has quit IRC | 22:29 | |
| *** bradd1 has quit IRC | 22:34 | |
| *** bradd1 has joined #openstack-sahara | 22:42 | |
| *** shakamunyi has quit IRC | 22:51 | |
| *** boris-42 has quit IRC | 22:57 | |
| *** boris-42 has joined #openstack-sahara | 22:58 | |
| *** ErikB1 has joined #openstack-sahara | 23:15 | |
| *** bradd1 has quit IRC | 23:26 | |
| *** ErikB1 has quit IRC | 23:30 | |
| *** IlyaE has joined #openstack-sahara | 23:42 | |
| *** IlyaE has quit IRC | 23:56 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!