| *** witlessb has joined #openstack-sahara | 01:56 | |
| *** zhiyan_ is now known as zhiyan | 01:59 | |
| *** witlessb has quit IRC | 02:00 | |
| *** zhiyan is now known as zhiyan_ | 03:18 | |
| *** Ch00k has joined #openstack-sahara | 04:16 | |
| *** Ch00k has quit IRC | 04:46 | |
| *** mattf has quit IRC | 04:52 | |
| *** mattf has joined #openstack-sahara | 04:56 | |
| *** IlyaE has joined #openstack-sahara | 05:09 | |
| openstackgerrit | Talusani Mani Shanker proposed a change to openstack/sahara: Replaced the word components with component(s) https://review.openstack.org/91403 | 05:29 |
|---|---|---|
| *** IlyaE has quit IRC | 05:53 | |
| *** IlyaE has joined #openstack-sahara | 06:07 | |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/sahara: Imported Translations from Transifex https://review.openstack.org/91517 | 06:07 |
| *** IlyaE has quit IRC | 06:36 | |
| *** IlyaE has joined #openstack-sahara | 06:42 | |
| *** IlyaE has quit IRC | 06:51 | |
| openstackgerrit | A change was merged to openstack/python-saharaclient: Updated from global requirements https://review.openstack.org/91245 | 06:54 |
| *** witlessb has joined #openstack-sahara | 08:18 | |
| *** mattf has quit IRC | 10:38 | |
| *** mattf has joined #openstack-sahara | 10:38 | |
| *** Ch00k has joined #openstack-sahara | 10:41 | |
| *** Ch00k has quit IRC | 11:24 | |
| *** bogdando has quit IRC | 11:36 | |
| *** bogdando has joined #openstack-sahara | 11:39 | |
| *** ErikB1 has joined #openstack-sahara | 11:56 | |
| *** bogdando has quit IRC | 11:59 | |
| *** bogdando has joined #openstack-sahara | 11:59 | |
| tellesnobrega | hi, i was finally able to get machines working with sahara, but the cluster is in waiting state, how long normally it takes to start the cluster? | 12:30 |
| *** Ch00k has joined #openstack-sahara | 12:32 | |
| mattf | tellesnobrega, depending on size, give it 10-15 min for hdp plugin and 5 for vanilla | 12:35 |
| mattf | less if you're using a pre-populated image for hdp | 12:36 |
| tellesnobrega | mattf: im using ubuntu 13.03 vanilla image | 12:36 |
| mattf | how big is the cluster? | 12:36 |
| tellesnobrega | just two instances | 12:36 |
| tellesnobrega | the instances are up | 12:37 |
| tellesnobrega | i can connect to the via controller using fixed ips | 12:38 |
| tellesnobrega | *then | 12:38 |
| tellesnobrega | *them | 12:38 |
| mattf | does sahara see the management ips for the instances? (click on the cluster in horizon to see the instance list) | 12:39 |
| *** tmckay has left #openstack-sahara | 12:39 | |
| tellesnobrega | management ip is blank | 12:40 |
| tellesnobrega | does this management ip need to be a floating ip? or can it be a fixed_ip? | 12:41 |
| mattf | that's what sahara-api is waiting for | 12:41 |
| mattf | it can be either | 12:41 |
| mattf | argh, went to check something in my install and / is full again | 12:42 |
| * mattf goes and disables ceilometer again | 12:43 | |
| tellesnobrega | mattf: my setup is the following. We have a openstack cloud here. In this cloud i have one vm. In this machine i'm starting a devstack with sahara, so im using only private ips to make it work. I'm struggling with this for a couple days and haven't been able to run a job. I would like to run a wordcount just to see it working | 12:44 |
| mattf | centos or ubuntu vm? | 12:45 |
| mattf | is sahara configured w/ use_floating_ips = true or false? | 12:47 |
| mattf | is your cloud using neutron or nova-networking? | 12:48 |
| tellesnobrega | ubuntu | 12:51 |
| tellesnobrega | the cloud is using neutron | 12:51 |
| tellesnobrega | but in using nova-network im my devstack | 12:51 |
| mattf | and is your devstack install configured to use keystone/nova/glance from the cloud? | 12:51 |
| mattf | oh, hmm | 12:52 |
| tellesnobrega | and use_floating_ips is empty in sahara.conf | 12:52 |
| mattf | so you're trying to run devstack completely in the vm, hmmm | 12:52 |
| tellesnobrega | my devstack start all services | 12:52 |
| tellesnobrega | yes | 12:52 |
| mattf | that's basically nested vms | 12:52 |
| mattf | i prefer the setup where your vm just runs horizon w/ sahara app + sahara-api | 12:53 |
| mattf | and sahara-api is configured to use the resources of the surrounding cloud | 12:53 |
| tellesnobrega | hum | 12:54 |
| tellesnobrega | do you have a local.conf that i can use to set up like you said | 12:54 |
| mattf | credentials for a service account in the surrounding cloud is necessary, and is provided to sahara-api via the os_admin_user/pass config | 12:54 |
| *** _crobertsrh is now known as crobertsrh | 12:55 | |
| mattf | chad did a similar setup for us, and it worked like a charm | 12:55 |
| mattf | speak of the devil | 12:55 |
| mattf | i'd be wary of nesting openstack w/i openstack | 12:56 |
| mattf | i'd be -> i am | 12:56 |
| crobertsrh | Good morning. | 12:56 |
| tellesnobrega | good morning | 12:56 |
| openstackgerrit | A change was merged to openstack/sahara-image-elements: Updates package installs for HDP images https://review.openstack.org/90417 | 12:57 |
| mattf | tellesnobrega, maybe a shortcut tho, you need use_floating_ips=false, it defaults to true. so sahara-api is probably waiting for them to show up. | 12:58 |
| tellesnobrega | mattf: i will try that | 12:59 |
| *** elmiko has joined #openstack-sahara | 13:07 | |
| mattf | ugh, gerrit is uber slow for me today | 13:09 |
| *** Ch00k has quit IRC | 13:14 | |
| *** tmckay has joined #openstack-sahara | 13:47 | |
| tellesnobrega | mattf: it worked | 13:48 |
| mattf | yay | 13:48 |
| mattf | is it crazy slow? | 13:49 |
| tellesnobrega | haven't tried running anything yet | 13:50 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/python-saharaclient: Updated from global requirements https://review.openstack.org/91566 | 13:50 |
| openstackgerrit | OpenStack Proposal Bot proposed a change to openstack/sahara: Updated from global requirements https://review.openstack.org/91567 | 13:51 |
| *** shakamunyi has joined #openstack-sahara | 13:59 | |
| *** ErikB1 has quit IRC | 14:11 | |
| tellesnobrega | mattf: i want to run a wordcount job using my cluster, so i just need to upload the binary for the wordcount with map and reduce together? or do they need to be apart some how? | 14:32 |
| crobertsrh | tellesnobrega: I think it is possible for map and reduce to be separate. tmckay can say for sure. | 14:35 |
| tmckay | tellesnobregra, they don't *need* to be separate, but I think they could be. As long as all the jars are specified in the "lib" config, and you set the mapred.mapper.class and mapred.reducer.class configs correctly, it should work either way | 14:37 |
| tmckay | tellesnobrega, my sample Wordcount in sahara-extra has the mapper/reducer in the same .java file | 14:37 |
| tmckay | or "mapreduce.mapper.class", etc. Depending on which api version you are using. I forget which one is newer... | 14:38 |
| tmckay | tellesnobrega, have you seen the WordCount example in sahara-extra? There is a readme there that might help you. | 14:39 |
| tellesnobrega | tmckay: i will do that now | 14:39 |
| openstackgerrit | Trevor McKay proposed a change to openstack/python-saharaclient: Add cli cluster creation and teardown test https://review.openstack.org/89585 | 14:39 |
| *** ErikB1 has joined #openstack-sahara | 14:39 | |
| openstackgerrit | Trevor McKay proposed a change to openstack/python-saharaclient: Add EDP jobs to cli integration tests https://review.openstack.org/89692 | 14:45 |
| *** IlyaE has joined #openstack-sahara | 14:48 | |
| *** ErikB1 has quit IRC | 15:10 | |
| *** ErikB1 has joined #openstack-sahara | 15:15 | |
| *** IlyaE has quit IRC | 15:31 | |
| *** bob_nettleton has joined #openstack-sahara | 15:33 | |
| *** ErikB1 has quit IRC | 15:48 | |
| *** IlyaE has joined #openstack-sahara | 15:49 | |
| *** ErikB1 has joined #openstack-sahara | 15:56 | |
| *** IlyaE has quit IRC | 16:00 | |
| tellesnobrega | tmckay: i'm trying to enter one of the machines in my sahara cluster using this ssh -i sahara-keypair.pem ubuntu@10.0.0.3, but i'm getting access denied, i've changed the key permissions to 400 but im still getting the error | 16:05 |
| elmiko | tellesnobrega: i've had trouble with the .pem files generated by the ui. i usually create a keypair with ssh-keygen and then import it instead of creating a new keypair. | 16:11 |
| *** IlyaE has joined #openstack-sahara | 16:11 | |
| tellesnobrega | elmiko: hmm, thanks | 16:11 |
| tellesnobrega | im trying to compile edp-wordcount | 16:11 |
| elmiko | that definitely sounds like a tmckay related question | 16:13 |
| tmckay | tellesnobrega, yeah, I always use a public key file for a keypair | 16:14 |
| tmckay | haven't tried pem | 16:14 |
| tellesnobrega | tmckay: thanks | 16:14 |
| *** IlyaE has quit IRC | 16:15 | |
| bob_nettleton | elmiko, Hi, have you run into any problems with generating CentOS images lately? Yesterday, I pulled down the latest, and tried generating the HDP images without specifying a separate DIB repo, and this seems to be failing. I've also tried to generate one of the vanilla provider's CentOS images, which also appears to fail. I think there might be a new DIB bug here, but was curious to see if you've noticed this as well. | 16:20 |
| tellesnobrega | tmckay: i created a new .key file, copied the .key.pub content to sahara and started a new cluster, still got the same error, Permission denied (publickey) | 16:28 |
| crobertsrh | Permissions 0600? | 16:29 |
| tellesnobrega | 400 | 16:30 |
| crobertsrh | where did you get your .pem from? | 16:30 |
| tmckay | tellesnobrega, that is what I use to add my public key file to sahara: nova keypair-add keypair0 --pub-key ~/.ssh/id_rsa.pub | 16:31 |
| tellesnobrega | i will try that | 16:32 |
| tmckay | then of course you have to select keypair0 when you launch the cluster | 16:33 |
| tmckay | ssh -i ~/.ssh/id_rsa.pub ubuntu@1.2.3.4 is my login, using an ubuntu image | 16:33 |
| tellesnobrega | tmckay: ok, i will try that | 16:37 |
| *** ErikB1 has quit IRC | 16:45 | |
| elmiko | bob_nettleton: hey, i generated an image with -p hdp -v 2 just a few days ago, but i will try again | 16:47 |
| elmiko | bob_nettleton: i had to modify my dib repo though. i'll give it a try from a fresh pull. | 16:47 |
| tellesnobrega | tmckay: id_rsa.pub permissions should be 400? | 16:48 |
| elmiko | tellesnobrega: when you ssh, i use the private(non .pub) for the -i option | 16:49 |
| tmckay | I've got 600, but I don't see why 400 wouldn't work | 16:49 |
| tellesnobrega | hum | 16:49 |
| tmckay | elmiko, the difference may be the "--pub-key" that I use to set the keypair | 16:49 |
| elmiko | could be | 16:49 |
| tellesnobrega | elmiko: i tried ssh -i ~/.ssh/id_rsa.pub ubuntu@1.2.3.4 and got Permission denied (publickey), also tried with id_rsa and still the same thing | 16:51 |
| elmiko | tellesnobrega: here's my work flow. create a key with `ssh -t rsa -f cloud_key`. import a key from horizon, pasting the cloud_key.pub into the key section. start everything. then `ssh -i cloud_key <user>@<instance ip>. | 16:51 |
| tellesnobrega | elmiko: i've done that, i have no idea why its not working | 16:51 |
| tellesnobrega | i will give it another try | 16:51 |
| elmiko | did you try root user instead of ubuntu? | 16:52 |
| elmiko | i sometimes do that just to verify things | 16:52 |
| tellesnobrega | elmiko: i was able to connect using ec2-user | 16:55 |
| elmiko | nice | 16:56 |
| elmiko | is that a centos image? | 16:56 |
| *** ErikB1 has joined #openstack-sahara | 16:57 | |
| bob_nettleton | elmiko, thanks, I think some recent changes may have broken CentOS image generation. I can still workaround this with an older DIB repo, but just wanted to make sure I wasn't the only one seeing this. :) | 16:57 |
| elmiko | bob_nettleton: i'm doing a rebuild now, i'll let you know if about 5-10 minutes how it goes | 16:58 |
| elmiko | s/if/in | 16:58 |
| tellesnobrega | tmckay: do you compile the edp-example in one of the cluster machines? | 17:04 |
| tellesnobrega | or locally and upload it via sahara ui? | 17:04 |
| tellesnobrega | elmiko: no, im using ubuntu 13.04 | 17:09 |
| *** shakamunyi has quit IRC | 17:09 | |
| tellesnobrega | i have no idea why it asked for ec2-user | 17:09 |
| elmiko | tellesnobrega: cool, i was just curious because i've seen that user cropping up in the centos images too | 17:09 |
| tellesnobrega | in the cluster configuration i used ubuntu, and it works, the log shows that its connected | 17:09 |
| elmiko | tellesnobrega: i don't know either, in the past the cloud-user was created, i'm not sure where ec2-user comes from. | 17:09 |
| tellesnobrega | elmiko: yeah | 17:10 |
| elmiko | tellesnobrega: nice, glad to got it running | 17:10 |
| elmiko | s/to/you | 17:10 |
| tellesnobrega | elmiko: thanks | 17:10 |
| elmiko | bob_nettleton: i just ran `sudo diskimage-create -p hdp -v 2` on my fedora 20 machine with selinux in permissive mode against the trunk of sahara-image-elements and it created an image for me. no errors. | 17:11 |
| tmckay | tellesnobrega, I originally compiled the edp-jar on an ubuntu worker I launched from Sahara, then installed jdk 1.6 manually | 17:11 |
| tmckay | tellesnobrega, however | 17:11 |
| tmckay | I use that exact edp-java.jar in the edp-examples/edp-wordcount in the sahara integration tests, uploaded as a job binary. For both 1.2.1, and 2.3.0 (vanilla) | 17:12 |
| tmckay | tellesnobrega, so I would be surprised if you could not just use that edp-java.jar | 17:12 |
| tellesnobrega | tmckay: i didn't see the compiled one | 17:15 |
| tellesnobrega | my bad | 17:15 |
| tmckay | I think it's in the "libs" dir | 17:15 |
| bob_nettleton | elmiko, ok, thanks for trying that out. I'm using CentOS with selinux turned off, and the trunk fails for me. I'll have to debug this further. are you pointing to a separate diskimage-builder repo, or are you using the latest? | 17:17 |
| elmiko | bob_nettleton: i'm using the default dib repo, whatever sahara-image-elements is using. i could fire up my rhel6.5 box and give it a try there if you want another data point | 17:18 |
| bob_nettleton | elmiko, thanks for the info. don't worry about trying it on RHEL. I've been looking through the latest DIB commits, and I think some recent changes are causing this. Not sure why I'm seeing this on CentOS, but I'll dig into it at some point in the future. thanks again! | 17:22 |
| elmiko | bob_nettleton: no prob, glad to help | 17:22 |
| tmckay | crobertsrh, played more with ideas from yesterday, with modified sahara validation on hdfs path. Without "hdfs://" on the front of a path, hadoop will interpret "mypath" as "hdfs://namenode:port/user/currentuser/mypath" where in our case current user is "hadoop" (unless I assume you hack around). And ... | 17:34 |
| tmckay | "/mypath" will be interpreted as "hdfs://namenode:port/mypath" | 17:35 |
| tmckay | so, you can have a path relative to the user, or an absolute path from the tip of hdfs | 17:36 |
| tmckay | crobertsrh, if you do supply hdfs://, you also have to supply hostinfo | 17:36 |
| tmckay | I changed the sahara validation to check scheme/host only if scheme is not "" in the urlparse. All we need is relaxation in the UI :) Wondering if you can put the "hdfs" in the text box, prepopulated, instead of next to it. So someone can backspace over it. | 17:37 |
| tmckay | crobertsrh, note, this is all true for Hive ^^, I'm sure it's true for Pig/MapReduce/Java too but I'll verify | 17:38 |
| * tmckay suspects crobertsrh is eating lunch | 17:39 | |
| tellesnobrega | tmckay: to run, i just add the edp-java.jar as job binary, create a job mapreduce and select the edp-java.jar as lib? | 17:39 |
| tmckay | tellesnobrega, actually, that edp-java.jar is set up to run as a Java action | 17:39 |
| tellesnobrega | tmckay: hum | 17:40 |
| tmckay | so, you want the Java job type | 17:40 |
| elmiko | huh, looks like devstack comes with a default fedora image in the demo tenant | 17:40 |
| tmckay | tellesnobrega, it's because it has its own main() | 17:40 |
| tellesnobrega | tmckay: i see | 17:41 |
| tmckay | tellesnobrega, actually, it might run as a MapReduce job too, now that I think of it. I never tried :) | 17:41 |
| tmckay | The classes are there | 17:41 |
| tellesnobrega | tmckay: i tried and got killed state | 17:42 |
| tellesnobrega | dont really know what happened, im gonna look it up | 17:42 |
| tmckay | tellesnobrega, yes, you can get there through the oozie logs, or lately I've been having a lot of success with "/mnt/log/hadoop/hadoop/userlogs" on the worker where it ran | 17:43 |
| tellesnobrega | tmckay: thanks, i will take a look | 17:43 |
| tmckay | tellesnobrega, I am currently thinking about EDP usability, we'll talk about it at Summit, too, so any thoughts you have from your experiences are more than welcome | 17:46 |
| *** Ch00k has joined #openstack-sahara | 17:47 | |
| *** bob_nettleton has quit IRC | 17:48 | |
| tellesnobrega | tmckay: ok | 17:49 |
| tmckay | (crobertrsh, by the way, for those earlier tests I used the CLI to create the data source since the UI was in my way ;-)) | 17:51 |
| tellesnobrega | tmckay: i tried running the java action and got this output http://paste.openstack.org/show/78268/ | 17:54 |
| tmckay | hmm, that's a new one | 17:58 |
| tmckay | I've never seen that before | 17:58 |
| *** crobertsrh has left #openstack-sahara | 17:59 | |
| *** crobertsrh has joined #openstack-sahara | 17:59 | |
| tmckay | tellesnobrega, looks like somehow the upload to hdfs failed for the edp-java.jar, but beyond that I'm unsure | 18:01 |
| tmckay | crobertsrh, hey, did you see all that junk I posted earlier, about hdfs paths? | 18:02 |
| crobertsrh | Yeah, just saw it...back from lunch | 18:02 |
| tellesnobrega | tmckay: thanks | 18:02 |
| crobertsrh | Removing the forced hdfs:// is certainly workable | 18:03 |
| tmckay | tellesnobrega, if it's reproducible, we should make a bug and track it down | 18:03 |
| tellesnobrega | i will start a new configuration and run it again | 18:04 |
| tmckay | crobertrsh, verifying with Pig, just to make sure. Looks like the same behavior (as you'd expect) | 18:04 |
| tmckay | assuming hadoop 2.3.0 is the same, but I should also try there | 18:04 |
| crobertsrh | should be the same, but yes, worth a test | 18:05 |
| crobertsrh | No meeting today? | 18:05 |
| tmckay | crobertsrh, I don't think there is a corresponding change for swift. There is no fancy path handling for swift, what you see is what you get | 18:05 |
| * tmckay checks sahara list | 18:06 | |
| crobertsrh | It should be fairly simple to tweak the UI to handle that. Slightly more than what you'd expect, but very workable. | 18:06 |
| crobertsrh | I really should sub-filter the os-dev list emails :) | 18:06 |
| tellesnobrega | tmckay: also tried the pig job in the example and got killed | 18:07 |
| tmckay | aignatov, SergeyLukjanov, no meeting today, because of travels maybe? | 18:07 |
| elmiko | yea SergeyLukjanov sent an email | 18:08 |
| tmckay | tellesnobrega, that is sad. I just ran that, 1 minute ago. I wonder what's different | 18:08 |
| tellesnobrega | tmckay: what image are you using? | 18:09 |
| tmckay | savanna-0.3-vanilla-1.2.1-ubuntu-13.04.qcow2, older copy of what should be on the Sahara pages | 18:09 |
| tmckay | elmiko, where's that image page I pasted for you again? :) The one with the targets for wget | 18:12 |
| elmiko | let me see if i can dig it up | 18:12 |
| tmckay | I can't find, lol | 18:12 |
| tellesnobrega | im using http://sahara-files.mirantis.com/sahara-icehouse-vanilla-1.2.1-ubuntu-13.10.qcow2 | 18:13 |
| tellesnobrega | elmiko: i downloaded from here http://docs.openstack.org/developer/sahara/userdoc/vanilla_plugin.html | 18:13 |
| elmiko | that looks like the link tmckay shared wit hme | 18:13 |
| tmckay | tellesnobrega, yes, I've used that one too | 18:13 |
| tellesnobrega | tmckay: weird, i just did a swift list and got empty response, but in horizon shows it | 18:16 |
| elmiko | tellesnobrega: different tenants maybe? | 18:17 |
| tmckay | elmiko, didn't you have some trouble with v2/v3 auth tokens and urls? | 18:17 |
| tmckay | although, different tenants sounds more likely in this case | 18:17 |
| elmiko | if you sourced the keystone_admin_rc thingie, it will make the command line sutff use admin tenant, double check your tenant in horizon it might be demo | 18:18 |
| tellesnobrega | elmiko: found it | 18:18 |
| elmiko | tmckay: i did, but that was something else | 18:18 |
| elmiko | tellesnobrega: check the --help for those cli apps too, you can usually specify the tenant/user/passwd with different options | 18:19 |
| tellesnobrega | elmiko: yeah. that is what i did | 18:20 |
| tellesnobrega | tmckay: i'm trying again, just setting up a new cluster | 18:24 |
| tmckay | ok | 18:24 |
| *** ErikB1 has quit IRC | 18:44 | |
| *** ErikB has joined #openstack-sahara | 18:57 | |
| *** Ch00k has quit IRC | 19:05 | |
| tmckay | hmmm, so hive is installed on my 2.3.0 image, and it looks like I can probably start hiveserver by hand, but the vanilla 2.3.0 plugin has no trace of supporting hive through Sahara | 19:15 |
| tmckay | but 1.2.1 does | 19:16 |
| tmckay | I wonder why that is | 19:16 |
| elmiko | api differences maybe? | 19:18 |
| tmckay | maybe. I wonder if it's just been forgotten. I should check blueprints | 19:19 |
| tmckay | https://blueprints.launchpad.net/sahara/+spec/hive-vanilla2-support | 19:20 |
| tmckay | not much going on there | 19:20 |
| elmiko | lol, you ain't kidding | 19:20 |
| tmckay | sreshetnyak, ping, what is the status of hive on vanilla 2? Anything happening there? | 19:21 |
| tellesnobrega | tmckay: ran again, the job is still getting killed | 19:37 |
| tmckay | tellesnobrega, okay, this is 1.2.1? Does it actually say "KILLED" in the Oozie console? | 19:40 |
| tellesnobrega | tmckay: i can't access oozie console via UI, because i dont have a floating ip | 19:40 |
| tellesnobrega | but thats is the message that i get in horizon | 19:40 |
| tmckay | tellesnobrega, okay, there is a different way, probably. 1.2.1, right? | 19:42 |
| *** hqwhjasdjh has quit IRC | 19:45 | |
| tellesnobrega | yeas | 19:46 |
| tmckay | tellesnobrega, okay, we may be able to get some info out of the oozie cli, or from /mnt/log/hadoop/hadoop on the worker node. How many workers in your cluster? (one makes it easy :) ) | 19:49 |
| tmckay | /opt/oozie/bin/oozie jobs -oozie http://10.3.13.226:11000/oozie | 19:49 |
| *** zhiyan_ is now known as zhiyan | 19:49 | |
| tellesnobrega | i was trying four, i will make 1 worker and 1 master | 19:50 |
| tmckay | That will give you the top level oozie log. The URL is the master, oozie port is always 11000 I think | 19:50 |
| tmckay | well, we can probably still find it | 19:50 |
| tellesnobrega | i had stopped it earlier :( | 19:51 |
| tmckay | :) okay | 19:51 |
| tellesnobrega | i will start now and run, and i get back to you | 19:51 |
| tmckay | /opt/oozie/bin/oozie job -oozie http://10.3.13.226:11000/oozie -info 0000015-140429154218903-oozie-hado-W | 19:52 |
| tmckay | tellesnobrega, for instance, ^^ this *might* give you an error string | 19:53 |
| tmckay | sometimes the actual error is hidden from Oozie | 19:53 |
| *** Manishanker has joined #openstack-sahara | 19:54 | |
| *** schofield has joined #openstack-sahara | 19:56 | |
| *** schofield has quit IRC | 19:56 | |
| *** schofield has joined #openstack-sahara | 19:57 | |
| tmckay | crobertsrh, fyi, https://bugs.launchpad.net/sahara/+bug/1315126 | 19:58 |
| crobertsrh | ack | 19:58 |
| crobertsrh | probably won't get to it until after the merge is merged | 19:59 |
| elmiko | tmckay: you should just attach the patch as a file then no formatting flubs | 20:01 |
| tmckay | yeah, I thought of that after | 20:02 |
| elmiko | lol | 20:02 |
| tmckay | I'll just submit the CR | 20:02 |
| elmiko | Captain Hindsight | 20:02 |
| tmckay | I figure this is a usability tweak. If I'm using a long running cluster, and I have my stuff in hdfs, I don't want to have to paste in my host:port. And, for that matter, if I save my data sources in Sahara, and use them on other clusters in the future, they won't be re-usable if the host/port *are* pasted in there (assuming the original cluster is gone) | 20:03 |
| elmiko | makes sense | 20:04 |
| *** schofield has left #openstack-sahara | 20:04 | |
| tmckay | came up because I actually tried to use hive on 1.2.1, which doesn't support swift, and I started experimenting with hdfs | 20:04 |
| elmiko | good find | 20:05 |
| tmckay | elmiko, one thing I hate about launchpad is there is no "delete comment" button. There is a "hide" but I don't know what it does. Hide it from everyone, or just some people? | 20:06 |
| tmckay | no "edit" comment that I can find, either | 20:06 |
| elmiko | yea, i'm not sure what's up with that. i guess they don't want people to be able to adjust history. | 20:07 |
| SergeyLukjanov | folks, I'm preparing for traveling and traveling | 20:13 |
| SergeyLukjanov | will be completely available at monday in pst timezone | 20:13 |
| SergeyLukjanov | (I'll be in CA before summit) | 20:13 |
| elmiko | have a good trip :) | 20:14 |
| SergeyLukjanov | mattf, tmckay, elmiko, ErikB, ^^ | 20:14 |
| SergeyLukjanov | elmiko, thx :) | 20:14 |
| tmckay | SergeyLukjanov, okay, have fun! I'll be right *here* :) | 20:15 |
| openstackgerrit | Trevor McKay proposed a change to openstack/sahara: Allow HDFS data source paths without the hdfs:// scheme https://review.openstack.org/91664 | 20:21 |
| SergeyLukjanov | tmckay :) | 20:26 |
| mattf | SergeyLukjanov, safe and enjoyable travels! | 20:33 |
| SergeyLukjanov | mattf, thx! | 20:33 |
| SergeyLukjanov | mattf, it'll be just 3 flights including to moscow, amsterdam and san francisco ;) | 20:33 |
| Manishanker | SergeyLukjanov , Happy journey :) | 20:35 |
| tellesnobrega | tmckay: we can work now | 20:47 |
| tmckay | okay | 20:47 |
| tellesnobrega | tried again and got Killed again | 20:47 |
| tellesnobrega | /opt/oozie/bin/oozie jobs -oozie http://10.3.13.226:11000/oozie, this should be ran in the master node right? | 20:48 |
| tmckay | Yes. | 20:48 |
| tmckay | It might show us something | 20:49 |
| tmckay | tellesnobrega, actually, you can right it anywhere on the cluster | 20:49 |
| tmckay | run it | 20:49 |
| tellesnobrega | ok | 20:49 |
| tmckay | as long as you have the oozie server host ip right | 20:49 |
| tmckay | For the Java job, it might tell us something. For pig, I doubt it :) | 20:50 |
| tellesnobrega | Job ID App Name Status User Group Started Ended | 20:50 |
| tellesnobrega | ------------------------------------------------------------------------------------------------------------------------------------ | 20:51 |
| tellesnobrega | 0000000-140501202031760-oozie-hado-W job-wf KILLED hadoop - 2014-05-01 20:30 GMT 2014-05-01 20:31 GMT | 20:51 |
| tellesnobrega | ------------------------------------------------------------------------------------------------------------------------------------ | 20:51 |
| tellesnobrega | killed | 20:51 |
| tmckay | so try this one | 20:51 |
| tmckay | /opt/oozie/bin/oozie job -oozie http://10.3.13.226:11000/oozie -info 0000015-140429154218903-oozie-hado-W | 20:51 |
| tmckay | where you paste your id in at the end | 20:51 |
| *** akuznetsov has quit IRC | 20:54 | |
| *** akuznetsov has joined #openstack-sahara | 20:54 | |
| tellesnobrega | http://paste.openstack.org/show/78306/ | 20:56 |
| tmckay | k, not going to help. Next ... | 20:57 |
| tmckay | cd /mnt/log/hadoop/hadoop/userlogs/ | 20:57 |
| tmckay | sudo su hadoop | 20:57 |
| tmckay | cd job_whatever | 20:58 |
| tmckay | ls -l attempt*/ | 20:58 |
| tmckay | look for the one that has a non-zero stderr | 20:58 |
| tmckay | The stderr and stdout in that dir should have something helpful about what went wrong | 20:59 |
| tellesnobrega | tmckay: i just found out that the part where i should define input and output is missing | 21:00 |
| tellesnobrega | but just for this run | 21:00 |
| tellesnobrega | it was there before | 21:00 |
| tmckay | tellesnobrega, okay, try again. The above ^^ approach should help -- I've found a bunch of things this way when I can't get to the oozie web ui logs | 21:00 |
| tellesnobrega | ok | 21:01 |
| tellesnobrega | i will look it up tomorrow | 21:01 |
| tellesnobrega | i gotta go now | 21:01 |
| tmckay | ok, talk to you tomorrow! | 21:01 |
| tellesnobrega | sorry for bottering you too much today | 21:01 |
| tmckay | np, don't mind at all | 21:01 |
| tmckay | we need users that can run stuff :) | 21:02 |
| tellesnobrega | tmckay: i want to do a little more :P | 21:02 |
| tmckay | :) | 21:02 |
| tmckay | have to start somewhere | 21:02 |
| tellesnobrega | hopefully | 21:02 |
| tellesnobrega | yeah, of course | 21:02 |
| *** crobertsrh is now known as _crobertsrh | 21:02 | |
| tmckay | okay, I think I'm out too | 21:03 |
| elmiko | g'night tmckay | 21:04 |
| tmckay | see ya elmiko | 21:04 |
| *** tmckay has quit IRC | 21:05 | |
| *** Manishanker has quit IRC | 21:05 | |
| *** IlyaE has joined #openstack-sahara | 21:06 | |
| *** IlyaE has quit IRC | 21:21 | |
| *** IlyaE has joined #openstack-sahara | 21:21 | |
| *** zhiyan is now known as zhiyan_ | 21:24 | |
| *** witlessb has quit IRC | 21:30 | |
| *** IlyaE has quit IRC | 21:34 | |
| *** elmiko is now known as elmiko_ | 21:39 | |
| *** IlyaE has joined #openstack-sahara | 22:18 | |
| *** IlyaE has quit IRC | 22:42 | |
| *** akuznetsov has quit IRC | 22:55 | |
| *** elmiko_ has quit IRC | 23:19 | |
| *** IlyaE has joined #openstack-sahara | 23:30 | |
| *** IlyaE has quit IRC | 23:37 | |
| *** shakamunyi has joined #openstack-sahara | 23:42 | |
| *** IlyaE has joined #openstack-sahara | 23:49 | |
| *** IlyaE has quit IRC | 23:50 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!