Wednesday, 2014-01-08

« Tuesday, 2014-01-07 Index Thursday, 2014-01-09 »
*** sballe has quit IRC00:15
*** sballe has joined #savanna00:15
*** ruhe is now known as _ruhe00:21
*** SergeyLukjanov is now known as _SergeyLukjanov00:37
*** nosnos has joined #savanna00:53
*** jmaron has quit IRC01:24
*** jmaron has joined #savanna01:26
openstackgerritAndrew Lazarev proposed a change to openstack/savanna: Added cache for image_username  https://review.openstack.org/6540201:32
openstackgerritAndrew Lazarev proposed a change to openstack/savanna: Added 'oozie' service support to IDH plugin  https://review.openstack.org/6432301:32
openstackgerritAndrew Lazarev proposed a change to openstack/savanna: Added 'oozie' service support to IDH plugin  https://review.openstack.org/6432301:40
*** nosnos has quit IRC01:43
*** xuhanp has joined #savanna01:43
*** nosnos has joined #savanna01:44
*** IlyaE has quit IRC02:11
*** jmaron has quit IRC02:39
*** nosnos has quit IRC02:43
*** IlyaE has joined #savanna03:12
*** IlyaE has quit IRC04:05
*** IlyaE has joined #savanna04:12
*** IlyaE has quit IRC04:35
*** mattf is now known as _mattf04:40
*** IlyaE has joined #savanna04:50
*** IlyaE has quit IRC04:50
*** IlyaE has joined #savanna04:58
*** IlyaE has quit IRC04:59
*** IlyaE has joined #savanna05:00
*** IlyaE has quit IRC05:21
*** sballe_ has joined #savanna05:23
*** sballe has quit IRC05:26
*** IlyaE has joined #savanna05:48
*** IlyaE has quit IRC05:51
*** amrit has quit IRC06:18
*** IlyaE has joined #savanna06:21
*** IlyaE has quit IRC06:56
*** SergeyLukjanov has joined #savanna07:05
*** IlyaE has joined #savanna07:18
*** IlyaE has quit IRC07:21
*** nadya_ has joined #savanna08:10
*** nadya_ has quit IRC08:25
*** SergeyLukjanov is now known as _SergeyLukjanov08:44
*** _SergeyLukjanov has quit IRC08:44
*** SergeyLukjanov has joined #savanna08:59
*** nadya_ has joined #savanna09:03
*** nadya_ has quit IRC09:19
*** nadya_ has joined #savanna09:21
openstackgerritCyril Roelandt proposed a change to openstack/python-savannaclient: Python3: use six.moves.urllib.parse instead of urlparse  https://review.openstack.org/6543509:26
*** nadya_ has quit IRC09:33
*** nadya_ has joined #savanna10:01
*** xuhanp has quit IRC10:10
*** SergeyLukjanov has quit IRC10:17
*** nadya_ has quit IRC10:18
*** SergeyLukjanov has joined #savanna10:22
*** SergeyLukjanov has quit IRC10:23
*** nadya_ has joined #savanna10:28
*** nadya_ has quit IRC10:34
*** nadya_ has joined #savanna10:35
*** nadya_ has quit IRC10:45
*** SergeyLukjanov has joined #savanna10:58
*** _mattf is now known as mattf11:01
openstackgerritMina Park proposed a change to openstack/savanna-dashboard: Adding the ability to modify node group template  https://review.openstack.org/6194112:33
*** sballe_ has quit IRC12:45
*** sballe_ has joined #savanna12:45
*** _crobertsrh is now known as crobertsrh12:50
openstackgerritMatthew Farrellee proposed a change to openstack/python-savannaclient: Add initial image-* commands to CLI  https://review.openstack.org/6546412:58
*** sballe_ has quit IRC13:06
*** qwerty_nor has joined #savanna13:19
*** nadya_ has joined #savanna14:05
*** tmckay has quit IRC14:18
*** nadya_ has quit IRC14:34
*** tmckay has joined #savanna15:05
*** tmckay has quit IRC15:10
*** tmckay has joined #savanna15:19
openstackgerritCyril Roelandt proposed a change to openstack/python-savannaclient: Python 3: use six.iteritems() instead of iteritems()  https://review.openstack.org/6548215:31
*** akuznetsov has quit IRC15:44
*** sballe has joined #savanna15:51
*** sballe has quit IRC15:52
*** akuznetsov has joined #savanna15:56
*** IlyaE has joined #savanna16:18
*** nadya_ has joined #savanna16:20
*** SergeyLukjanov has quit IRC16:23
*** sballe has joined #savanna16:48
*** nadya_ has quit IRC16:51
*** nadya_ has joined #savanna16:53
*** jdev789 has joined #savanna16:56
*** jdev has joined #savanna16:56
*** SergeyLukjanov has joined #savanna17:00
*** jmaron has joined #savanna17:10
*** jdev789 has quit IRC17:12
*** mattf is now known as _mattf17:12
*** jdev has quit IRC17:13
*** sballe has quit IRC17:23
*** sballe has joined #savanna17:24
*** IlyaE has quit IRC17:27
*** jdev789 has joined #savanna17:28
*** jdev has joined #savanna17:28
*** nadya_ has quit IRC17:28
*** _mattf is now known as mattf17:30
*** nadya_ has joined #savanna17:34
*** IlyaE has joined #savanna17:41
*** IlyaE has quit IRC17:43
*** _ruhe is now known as ruhe17:53
*** DennyZhang has joined #savanna17:54
*** IlyaE has joined #savanna18:16
*** rnirmal has joined #savanna18:39
tmckaydoes anyone here happen to have links to good docs for accessing configuration parameters from oozie java action jobs, and which hadoop configs are typically applicable?  I've found some info on how to reference config values specified in the action itself via oozie.action.conf.xml but not how to access things from a default or site xml, for example19:05
tmckaytrying to get background for setting up config hings19:05
tmckayhints19:05
*** jmaron has quit IRC19:11
*** ruhe is now known as _ruhe19:21
*** nadya_ has quit IRC19:34
*** DennyZhang has quit IRC19:34
*** jmaron has joined #savanna19:36
tmckaycrobertsrh, so the conclusion that I'm arriving at is that there are no config hints for java actions (well, there might be mapred.job.queue.name)19:46
tmckayyou've got two cases I think ....19:46
tmckaysomebody is doing something simple, like generating a custom hdfs file somewhere as a precursor to a job, or calculating a fibonacci number on the cluster, in which case things are pretty standalone19:47
crobertsrhOk.  So I should prevent the UI from making such a call for the "Java" job type?19:47
tmckayor, someone is a crazy hadoop ninja doing all kinds of edgy junk in a custom launcher and they don't need any hints.  They can reference whatever they want from global configs, or pass everything as args, or specify configs for the action and them reference them in the oozie way.19:48
crobertsrhI have to look more closely, but I think the UI was using config hints to drive which fields (configs, params, args) show up on the page.  I should be able to short circuit that for java jobs.19:49
tmckaycrobertsrh, I think so, at this point.  Until someone comes up with config hints they think would be useful.  Java actions are a blank canvass, essentially, how can we hint anything?19:49
crobertsrhI agree.  I only thought of it because my config page bombed out for the java job type :)19:50
tmckaycrobertsrh, you still want configs and args for java actions.  Just, we don't give you a list of possibles19:50
crobertsrhI see that args are a [] for java rather than a {}, was that intentional?19:50
tmckaycrobertsrh, and there are the special things -- main-class and java-opts.  those are outside of configs/args/params19:50
tmckayyes19:50
crobertsrhOk, just making sure19:50
tmckayPig/hive seem to use named parameter passing of some sort, for java it's just like a cli19:51
crobertsrhI figured that was the case19:51
tmckaycrobertsrh, the one last thing I've stumbled on is the "capture output" setting, which lets you flow output from java/shell actions back into your workflow for decisions.  I think, though, in our canned case that does not apply, because we control the workflow.xml and don't allow multiple actions.  I think very soon we should allow fullblown custom oozie workflows to be uploaded, and then folks are *really* cut loose.19:52
crobertsrhYeah, that might be really neat19:53
crobertsrhand scary19:53
tmckayheh, I am being absorbed by the Ooze19:53
tmckaycrobertsrh, so, what would a blank response for java action look like?19:55
crobertsrhfor config hints?19:55
tmckaycrobertsrh, I think for completeness the config hints api should return a valid "nothing to see here" construct.  configs/args with empty payloads?19:55
tmckayyeah19:55
crobertsrhI'd like that.  Not sure what it would look like.  Maybe {}?19:55
*** jdev has quit IRC19:55
crobertsrhI'll have to see what a regular one looks like again19:56
tmckayokay, I'll play around.  The other types return parsed xml...19:56
tmckaysame here19:56
*** jdev789 has quit IRC19:56
tmckaylooks like it's just the dict {configs: {}, params: {}, args: {}} which in this case would have "args: []"19:57
tmckayI guess no params.  so just {configs: {}, args: []}19:58
tmckaycrobertsrh, if we come of up with useful standard configs then "configs" would actually have junk in it, so maybe you should test for that19:58
crobertsrhmight work19:59
tmckaythe {} vs [] for args is probably a pain for you ... ?  sorry :)20:00
tmckaycrobertsrh, oh, looks like "configs" is actually a list of dictionaries (name, value, description)20:07
crobertsrha bit, but should be ok20:08
tmckayso it would look like this (based on the REST api output)20:09
tmckay{20:09
tmckay    "job_config": {20:09
tmckay        "args": [],20:09
tmckay        "configs": []20:09
tmckay  }20:09
tmckay}20:09
crobertsrhthat should work20:13
*** jmaron has quit IRC20:14
*** _ruhe is now known as ruhe20:14
openstackgerritTrevor McKay proposed a change to openstack/savanna: Add Oozie java action workflows  https://review.openstack.org/6315620:28
*** IlyaE has quit IRC20:34
*** jmaron has joined #savanna20:38
*** IlyaE has joined #savanna20:47
*** jdev has joined #savanna20:54
*** jdev789 has joined #savanna20:54
*** jmaron has quit IRC20:55
*** jdev has quit IRC20:57
*** jdev789 has quit IRC20:57
*** jmaron has joined #savanna20:59
*** qwerty_nor has quit IRC21:32
*** tmckay has quit IRC21:33
*** SergeyLukjanov_ has joined #savanna21:34
*** SergeyLukjanov has quit IRC21:35
openstackgerritMatthew Farrellee proposed a change to openstack/python-savannaclient: Add initial image-* commands to CLI  https://review.openstack.org/6546421:37
openstackgerritMatthew Farrellee proposed a change to openstack/python-savannaclient: Add run_test.sh for running tests  https://review.openstack.org/6474121:37
*** tmckay has joined #savanna21:44
tmckayhmm, git won't let me pull or clone.  Anyone else having trouble?21:48
tmckay git clone git://github.com/openstack/savanna.git savanna321:48
tmckayCloning into 'savanna3'...21:48
tmckayfatal: Could not read from remote repository.21:48
tmckayPlease make sure you have the correct access rights21:48
tmckayand the repository exists.21:48
ruhetmckay: github is not kosher, we should use https://git.openstack.org/cgit/openstack/savanna21:49
ruheregarding your issues, it's probably a problem with your ssh keys21:49
ruheah21:50
ruhegithub is down21:50
tmckaythat's what I was suspecting.  ruhe, but yes I do have old configs, I should pull from openstack, thanks21:50
*** IlyaE has quit IRC21:57
*** IlyaE has joined #savanna22:03
*** SergeyLukjanov_ has quit IRC22:08
*** SergeyLukjanov has joined #savanna22:10
openstackgerritAndrew Lazarev proposed a change to openstack/savanna: Added 'oozie' service support to IDH plugin  https://review.openstack.org/6432322:16
openstackgerritA change was merged to openstack/python-savannaclient: Python 3: use six.iteritems() instead of iteritems()  https://review.openstack.org/6548222:17
*** crobertsrh is now known as _crobertsrh22:22
*** alazarev has joined #savanna22:26
*** alazarev has quit IRC22:38
*** IlyaE has quit IRC22:40
*** mattf is now known as _mattf22:40
*** _mattf is now known as mattf22:46
*** SergeyLukjanov_ has joined #savanna22:47
*** SergeyLukjanov_ has quit IRC22:47
*** SergeyLukjanov has quit IRC22:48
*** SergeyLukjanov_ has joined #savanna22:48
*** SergeyLukjanov_ has quit IRC22:48
tmckaythere is a tension between the joy of realizing your mistake and moving beyond it and the shame of having made the mistake in the first place22:58
*** rnirmal has quit IRC22:58
*** tmckay has left #savanna23:01
*** ruhe is now known as _ruhe23:41
« Tuesday, 2014-01-07 Index Thursday, 2014-01-09 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!