| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Minor exception text changes https://review.openstack.org/65827 | 00:03 |
|---|---|---|
| *** jmaron has joined #savanna | 00:05 | |
| *** _mattf is now known as mattf | 00:18 | |
| *** jdev789 has joined #savanna | 00:30 | |
| *** jdev has joined #savanna | 00:30 | |
| *** kbroughton has quit IRC | 00:36 | |
| *** alazarev has quit IRC | 00:36 | |
| *** jdev has quit IRC | 00:36 | |
| *** jdev789 has quit IRC | 00:36 | |
| *** ErikB has quit IRC | 00:39 | |
| *** sballe has quit IRC | 00:43 | |
| *** sballe has joined #savanna | 00:43 | |
| *** jmaron has quit IRC | 00:49 | |
| *** alazarev has joined #savanna | 00:53 | |
| *** rnirmal has quit IRC | 00:59 | |
| *** akuznetsov has quit IRC | 01:06 | |
| *** akuznetsov has joined #savanna | 01:08 | |
| *** jmaron has joined #savanna | 01:20 | |
| *** jmaron has quit IRC | 01:25 | |
| *** nosnos has joined #savanna | 01:33 | |
| *** xuhanp has joined #savanna | 01:34 | |
| *** jmaron has joined #savanna | 01:48 | |
| *** jdev789 has joined #savanna | 01:51 | |
| *** jdev has joined #savanna | 01:51 | |
| *** jmaron has quit IRC | 02:09 | |
| *** IlyaE has quit IRC | 02:14 | |
| *** IlyaE has joined #savanna | 02:19 | |
| *** IlyaE has quit IRC | 02:21 | |
| *** alazarev has quit IRC | 02:25 | |
| *** sballe has quit IRC | 02:37 | |
| *** sballe has joined #savanna | 02:38 | |
| *** jdev789 has quit IRC | 03:25 | |
| *** jdev has quit IRC | 03:25 | |
| *** amrit has joined #savanna | 03:53 | |
| *** nadya_ has joined #savanna | 05:19 | |
| *** nadya_ has quit IRC | 05:28 | |
| *** SergeyLukjanov has joined #savanna | 05:34 | |
| *** SergeyLukjanov has quit IRC | 05:41 | |
| *** alazarev has joined #savanna | 05:46 | |
| *** alazarev has quit IRC | 05:54 | |
| *** alazarev has joined #savanna | 06:30 | |
| *** alazarev has quit IRC | 06:30 | |
| *** IlyaE has joined #savanna | 07:15 | |
| *** aignatov has joined #savanna | 07:22 | |
| *** IlyaE has quit IRC | 08:06 | |
| openstackgerrit | A change was merged to openstack/savanna: Minor exception text changes https://review.openstack.org/65827 | 08:51 |
| *** IlyaE has joined #savanna | 09:18 | |
| *** xuhanp has quit IRC | 09:24 | |
| *** _ruhe is now known as ruhe | 09:33 | |
| *** nadya_ has joined #savanna | 09:41 | |
| *** nosnos has quit IRC | 09:49 | |
| *** nadya_ has quit IRC | 09:52 | |
| openstackgerrit | A change was merged to openstack/savanna: Add Oozie java action workflows https://review.openstack.org/63156 | 09:58 |
| openstackgerrit | Yaroslav Lobankov proposed a change to openstack/savanna: Some changes of integration tests https://review.openstack.org/65911 | 09:59 |
| *** nadya_ has joined #savanna | 10:03 | |
| *** nadya_ has quit IRC | 10:04 | |
| *** SergeyLukjanov has joined #savanna | 10:10 | |
| *** ruhe is now known as _ruhe | 10:26 | |
| *** dmitryme has joined #savanna | 10:39 | |
| *** bogdando has joined #savanna | 10:44 | |
| *** aignatov has quit IRC | 10:46 | |
| *** _ruhe is now known as ruhe | 11:00 | |
| *** ruhe is now known as ruhe_away | 11:04 | |
| *** ruhe_away is now known as ruhe | 11:04 | |
| *** aignatov has joined #savanna | 11:07 | |
| *** IlyaE has quit IRC | 11:18 | |
| *** igormarnat_ has joined #savanna | 11:19 | |
| *** qwerty_nor has joined #savanna | 11:24 | |
| *** aignatov has quit IRC | 11:33 | |
| *** ruhe is now known as _ruhe | 11:33 | |
| *** aignatov has joined #savanna | 11:36 | |
| *** _ruhe is now known as ruhe | 11:45 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/savanna: Remove unused node_group parameter in get_config_value https://review.openstack.org/65925 | 11:55 |
| *** aignatov has quit IRC | 11:56 | |
| *** aignatov has joined #savanna | 12:05 | |
| openstackgerrit | Matthew Farrellee proposed a change to openstack/python-savannaclient: Add initial node-group-template-* commands to CLI https://review.openstack.org/65929 | 12:21 |
| *** qwerty_nor has quit IRC | 12:41 | |
| *** aignatov has quit IRC | 12:50 | |
| *** dmitryme has quit IRC | 12:52 | |
| *** nadya_ has joined #savanna | 13:04 | |
| *** SergeyLukjanov is now known as _SergeyLukjanov | 13:05 | |
| *** SergeyLukjanov has joined #savanna | 13:08 | |
| *** nadya_ has quit IRC | 13:09 | |
| *** qwerty_nor has joined #savanna | 13:09 | |
| *** dmitryme has joined #savanna | 13:09 | |
| *** aignatov has joined #savanna | 13:10 | |
| *** jdev789 has joined #savanna | 14:13 | |
| *** jdev has joined #savanna | 14:13 | |
| *** kbroughton has joined #savanna | 14:14 | |
| *** igormarnat_ has quit IRC | 14:20 | |
| *** DinaBelova has joined #savanna | 14:30 | |
| *** akuznetsov has quit IRC | 14:59 | |
| *** akuznetsov has joined #savanna | 15:06 | |
| *** dmitryme has quit IRC | 15:12 | |
| _tmckay | aignatov, fyi, I think I'm going to put up a blueprint for streaming mapreduce today. There are two components, adding the <streaming> element for mapreduce jobs as a first pass, and then adding <files> and <archives> elements because without them <streaming> is limited to executables already on the system. Those last two will mean additional uploads from the job_manager. And I need _crobertsrh input on this | 15:16 |
| _tmckay | ^^ I say additional uploads because it looks like in Hadoop 1 files and archives must be hdfs paths, but in Hadoop 2 there is some indication that http:// paths are supported. I tried with a swift:// path yesterday but it failed -- if we want swift to work with distributed cache that might be additional hadoop/swift patching. | 15:18 |
| *** _tmckay is now known as tmckay | 15:18 | |
| *** kbroughton has quit IRC | 15:22 | |
| openstackgerrit | Yaroslav Lobankov proposed a change to openstack/savanna: Fixed HDP plugin to support Heat engine https://review.openstack.org/65981 | 15:24 |
| *** dmitryme has joined #savanna | 15:25 | |
| *** kbroughton has joined #savanna | 15:38 | |
| *** kbroughton has quit IRC | 15:43 | |
| *** kbroughton has joined #savanna | 15:44 | |
| aignatov | _tmckay: hi, great, sounds the good plan :) as for Hadoop 2, not sure if we have to think about it today | 15:53 |
| tmckay | ack. Looking more closely, I think <files> and <archives> is a separate blueprint. It's useful in general, in addition to making streaming more interesting. | 15:54 |
| *** igormarnat_ has joined #savanna | 16:06 | |
| *** jmaron has joined #savanna | 16:11 | |
| *** akuznetsov has quit IRC | 16:19 | |
| *** SergeyLukjanov has quit IRC | 16:30 | |
| *** igormarnat__ has joined #savanna | 16:30 | |
| *** igormarnat_ has quit IRC | 16:33 | |
| *** jdev has quit IRC | 16:42 | |
| *** jdev789 has quit IRC | 16:43 | |
| *** aignatov has quit IRC | 16:56 | |
| *** aignatov has joined #savanna | 16:58 | |
| *** ruhe is now known as _ruhe | 16:59 | |
| *** _ruhe is now known as ruhe | 17:07 | |
| *** aignatov_ has quit IRC | 17:08 | |
| *** aignatov_ has joined #savanna | 17:08 | |
| *** igormarnat__ has quit IRC | 17:10 | |
| *** igormarnat__ has joined #savanna | 17:10 | |
| *** rnirmal has joined #savanna | 17:11 | |
| *** igormarnat__ has quit IRC | 17:14 | |
| *** SergeyLukjanov has joined #savanna | 17:19 | |
| *** jmaron has quit IRC | 17:22 | |
| *** jmaron has joined #savanna | 17:23 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/savanna: Fix working savanna in secure network https://review.openstack.org/66027 | 17:43 |
| *** ruhe is now known as _ruhe | 17:43 | |
| *** IlyaE has joined #savanna | 17:46 | |
| *** aignatov has quit IRC | 17:49 | |
| *** dmitryme has quit IRC | 17:51 | |
| *** akuznetsov has joined #savanna | 18:17 | |
| *** jmaron has quit IRC | 18:26 | |
| *** mattf is now known as _mattf | 18:28 | |
| *** jmaron has joined #savanna | 18:28 | |
| *** IlyaE has quit IRC | 18:29 | |
| *** nadya_ has joined #savanna | 18:32 | |
| *** SergeyLukjanov has quit IRC | 18:35 | |
| *** _mattf is now known as mattf | 18:37 | |
| *** jmaron has quit IRC | 18:38 | |
| *** SergeyLukjanov has joined #savanna | 18:38 | |
| *** nadya_ has quit IRC | 18:45 | |
| openstackgerrit | Sergey Reshetnyak proposed a change to openstack/savanna: Extract common part of instances.py and instances_heat.py https://review.openstack.org/62842 | 18:47 |
| *** DinaBelova has quit IRC | 18:48 | |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added cache for image_username https://review.openstack.org/65402 | 18:48 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [IDH] Fixed work with cluster configs https://review.openstack.org/65818 | 19:03 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added 'oozie' service support to IDH plugin https://review.openstack.org/64323 | 19:03 |
| *** _ruhe is now known as ruhe | 19:18 | |
| *** dmitryme has joined #savanna | 19:20 | |
| *** IlyaE has joined #savanna | 19:34 | |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Fixed cluster template with no nodegroups creation https://review.openstack.org/65823 | 19:38 |
| *** IlyaE has quit IRC | 19:40 | |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [IDH] Fixed work with cluster configs https://review.openstack.org/65818 | 19:50 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added cache for image_username https://review.openstack.org/65402 | 19:50 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added 'oozie' service support to IDH plugin https://review.openstack.org/64323 | 19:50 |
| *** ruhe is now known as _ruhe | 19:51 | |
| *** dmitryme has left #savanna | 20:55 | |
| *** dmitryme has joined #savanna | 21:06 | |
| openstackgerrit | A change was merged to openstack/savanna: Adding IDH plugin basic implementation https://review.openstack.org/56105 | 21:07 |
| mattf | tmckay, am i correct in thinking that the reason to have .../data targets in https://blueprints.launchpad.net/savanna/+spec/v2-api is for savanna-api to proxy downloading content for a client? | 21:09 |
| mattf | i.e. a job-binary may be in swift or local db or somewhere else, and the client could get the details and go to the source to download the data, or just have savanna-api do it for them? | 21:10 |
| SergeyLukjanov | mattf, yup | 21:10 |
| mattf | i realize this is kinda weird having job-binaries-internal/../data too | 21:11 |
| SergeyLukjanov | mattf, http://docs.openstack.org/developer/savanna/userdoc/rest_api_v1.1_EDP.html#get-job-binary-internal-data | 21:11 |
| * mattf has to remember the docs.os.o - i keep using rtfm | 21:11 | |
| tmckay | mattf, yes, the /data target is to retrieve the raw file content for that binary | 21:12 |
| SergeyLukjanov | http://docs.openstack.org/developer/savanna/userdoc/rest_api_v1.1_EDP.html#get-job-binary-data | 21:12 |
| SergeyLukjanov | that's agood question about the need to have it in the job binary *internal* | 21:12 |
| tmckay | mattf, it's potentially for clients, the methods underlying it are used by savanna internally to do the same thing .. get the content and push to hdfs | 21:12 |
| tmckay | note, if there is no /data target in the api for job-binary-internal, then the only way to retrieve the data from a client is to make a job-binary that points to the internal and then use /data on the job-binary. Maybe that's okay, just noting. | 21:14 |
| mattf | imho, cleaner would be to POST /job-binary-internals w/ content, get an id back and then POST /job-binary-internals w/ a url of savanna-internal://<id> | 21:19 |
| mattf | so job-binary-internals is just a way to store files in savanna (since you don't have swift or some other data store, like manila) | 21:20 |
| mattf | tmckay, SergeyLukjanov, thoughts? | 21:20 |
| mattf | CORRECTION - imho, cleaner would be to POST /job-binary-internals w/ content, get an id back and then POST /job-binaries w/ a url of savanna-internal://<id> | 21:22 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [Vanilla] Added unit test on get_hadoop_ssh_keys method https://review.openstack.org/66067 | 21:24 |
| tmckay | mattf, I think the docs are wrong. I think what you say above is what happens, with the exception of using POST for job-binary-internal creation. The url for an internal should be using <id>, not a name | 21:27 |
| tmckay | mattf, for example, here is the url for a job-binary I have on my test cluster: savanna-db://b6e5d35c-dc43-4733-9d32-00375abecf1f | 21:28 |
| mattf | i'm flipping between docs and docs, possible the doc is wrong | 21:28 |
| mattf | lol | 21:28 |
| *** SergeyLukjanov has quit IRC | 21:29 | |
| mattf | friday evening -- flipping between code and docs | 21:29 |
| tmckay | I remember reading about the POST/PUT dilemna on internals, I came down ultimately on the side of PUT. Seemed to be more consistent with common usage but I'm not opposed to changing it. | 21:29 |
| tmckay | yeah, all the internals should be savanna-db://<id> | 21:30 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [Vanilla] Added unit test on get_hadoop_ssh_keys method https://review.openstack.org/66067 | 21:30 |
| tmckay | mattf, on your ealier comment, yes, job-binary-internals is just a way to store files in savanna | 21:31 |
| tmckay | the only purpose | 21:31 |
| mattf | ok, the /data should die then | 21:33 |
| mattf | the put .../<name> is err | 21:34 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added IDH plugin to savanna config https://review.openstack.org/66069 | 21:36 |
| *** IlyaE has joined #savanna | 21:58 | |
| openstackgerrit | A change was merged to openstack/savanna: Remove unused node_group parameter in get_config_value https://review.openstack.org/65925 | 22:13 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [IDH] Fixed work with cluster configs https://review.openstack.org/65818 | 22:26 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added cache for image_username https://review.openstack.org/65402 | 22:26 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: Added 'oozie' service support to IDH plugin https://review.openstack.org/64323 | 22:26 |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [IDH] Added config controlling hadoop-swift.jar URL https://review.openstack.org/65821 | 22:26 |
| *** kbroughton has quit IRC | 22:52 | |
| *** mattf is now known as _mattf | 22:54 | |
| *** dmitryme has quit IRC | 22:56 | |
| *** _mattf is now known as mattf | 23:02 | |
| *** qwerty_nor has quit IRC | 23:14 | |
| *** rnirmal has quit IRC | 23:22 | |
| openstackgerrit | Andrew Lazarev proposed a change to openstack/savanna: [Vanilla] Added unit test on get_hadoop_ssh_keys method https://review.openstack.org/66067 | 23:45 |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!