| *** dave-mccowan has joined #openstack-sahara | 00:15 | |
| *** dave-mccowan has quit IRC | 01:30 | |
| *** dave-mccowan has joined #openstack-sahara | 01:32 | |
| *** dave-mcc_ has joined #openstack-sahara | 01:48 | |
| *** dave-mccowan has quit IRC | 01:51 | |
| *** dave-mccowan has joined #openstack-sahara | 01:57 | |
| *** dave-mcc_ has quit IRC | 01:59 | |
| *** dave-mcc_ has joined #openstack-sahara | 02:18 | |
| *** dave-mccowan has quit IRC | 02:18 | |
| *** dave-mcc_ has quit IRC | 02:55 | |
| *** links has joined #openstack-sahara | 03:48 | |
| *** rcernin has joined #openstack-sahara | 07:04 | |
| *** pcaruana has joined #openstack-sahara | 07:08 | |
| *** tesseract has joined #openstack-sahara | 07:22 | |
| *** tosky has joined #openstack-sahara | 09:23 | |
| openstackgerrit | Merged openstack/sahara master: Add docs about template portability. https://review.openstack.org/500044 | 10:46 |
|---|---|---|
| tosky | warning: scenario tests are broken, probably due to the zuul migration (namely something in sahara-tests, tools/gate/scenario/post_test_hook.sh) | 10:47 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: Switch the remote user from jenkins to zuul https://review.openstack.org/508847 | 10:49 |
| tosky | let's see if this ^^ works or if it's something more complicated | 10:49 |
| *** dave-mccowan has joined #openstack-sahara | 11:07 | |
| tosky | soooo, https://review.openstack.org/508847 fixes the gates apparently | 12:09 |
| tosky | please merge it :) | 12:09 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara-tests master: Gate scripts: switch the user from jenkins to zuul https://review.openstack.org/508847 | 12:37 |
| *** links has quit IRC | 12:59 | |
| *** tosky_ has joined #openstack-sahara | 13:01 | |
| *** tosky has quit IRC | 13:01 | |
| *** tosky_ is now known as tosky | 13:01 | |
| *** lucasxu has joined #openstack-sahara | 13:34 | |
| *** tesseract has quit IRC | 14:46 | |
| *** tesseract has joined #openstack-sahara | 15:07 | |
| *** pcaruana has quit IRC | 15:27 | |
| *** dave-mccowan has quit IRC | 15:32 | |
| *** dave-mcc_ has joined #openstack-sahara | 15:32 | |
| *** rcernin has quit IRC | 15:41 | |
| *** ssmith has joined #openstack-sahara | 15:57 | |
| ssmith | Gooday, does anyone know if there's a Spark 2.2 image build anywhere? | 15:58 |
| tosky | no, I don't think we support it | 16:03 |
| tosky | we added Spark 2.1.0 in Pike | 16:03 |
| *** chlong has joined #openstack-sahara | 16:06 | |
| *** jeremyfreudberg has joined #openstack-sahara | 16:39 | |
| ssmith | OK 2.1 is acceptable. Any images available? | 16:39 |
| jeremyfreudberg | git clone https://github.com/openstack/sahara-image-elements && cd sahara-image-elements && sudo apt -y install kpartx qemu && tox -e sahara-image-create -p spark -s 2.1.0 | 16:41 |
| jeremyfreudberg | we do not have a place to host images for the latest release, unfortunately | 16:42 |
| tosky | so, the gates are stuck with some issue in the migration of the jobs (missing dependency); https://review.openstack.org/508934 should fix it, then https://review.openstack.org/508847, and then we should good to go | 16:45 |
| tosky | hopefully | 16:45 |
| tosky | and then we can start converting the jobs /o\ | 16:45 |
| *** dave-mcc_ has quit IRC | 16:46 | |
| ssmith | Will 2.1 work on Sahara Newton? | 16:51 |
| jeremyfreudberg | ssmith, it should work, luckily for the spark plugin the only differences in version is kept on the images, not in the provisioning part | 16:52 |
| jeremyfreudberg | you can actually register a 2.1 image in the 1.6 slot and it will work | 16:52 |
| jeremyfreudberg | to make it pretty just edit this line: https://github.com/openstack/sahara/blob/stable/newton/sahara/plugins/spark/plugin.py#L71 | 16:53 |
| *** chlong has quit IRC | 16:58 | |
| ssmith | Can I go return ['2.1.0', '1.6.0', '1.3.1']? | 17:07 |
| jeremyfreudberg | that was my thought | 17:07 |
| jeremyfreudberg | it should work | 17:07 |
| jeremyfreudberg | let me know if it doesn't, my boss wants me to do that in our (ocata) cloud too | 17:07 |
| ssmith | OK, yes. One of the DA guys is asking for 2.x | 17:08 |
| ssmith | One other question: Have you ever heard of Dask (https://github.com/dask/dask) | 17:08 |
| jeremyfreudberg | ssmith, no, I haven't heard of dask before, but that does seem quite useful/interesting | 17:09 |
| ssmith | Is Yarn or Mesos supported? | 17:12 |
| jeremyfreudberg | ssmith, vanilla plugin has yarn | 17:12 |
| ssmith | But not Spark plugin? | 17:13 |
| *** chlong has joined #openstack-sahara | 17:15 | |
| jeremyfreudberg | ssmith, so after a certain version we got Spark to run on Vanilla | 17:17 |
| jeremyfreudberg | the question is, does that spark installation properly interact with Yarn | 17:17 |
| ssmith | In "sahara/sahara/plugins/spark/resources/spark-env.sh.template" there's - MESOS_NATIVE_LIBRARY, to point to your libmesos.so if you use Mesos | 17:18 |
| ssmith | DA says that it will | 17:18 |
| jeremyfreudberg | hmm... let me poke around | 17:18 |
| ssmith | At which version did you get Spark to run on Vanilla? | 17:22 |
| jeremyfreudberg | https://review.openstack.org/#/q/topic:bp/spark-jobs-for-vanilla-hadoop i think ocata, but let me be sure | 17:24 |
| jeremyfreudberg | actually, that was the very end of newton :) | 17:24 |
| *** chlong has quit IRC | 17:30 | |
| *** chlong has joined #openstack-sahara | 17:30 | |
| *** tosky has quit IRC | 17:31 | |
| *** tesseract has quit IRC | 17:55 | |
| *** tosky has joined #openstack-sahara | 17:59 | |
| jeremyfreudberg | tosky, should we propose the sahara plugin deadline to the release schedule? we were all in agreement about that at Ptg, right? | 18:10 |
| *** jeremyfreudberg has quit IRC | 18:27 | |
| *** chlong has quit IRC | 18:45 | |
| *** chlong has joined #openstack-sahara | 18:46 | |
| tosky | jeremyfreudberg: I guess so, yes | 18:47 |
| *** rcernin has joined #openstack-sahara | 18:54 | |
| *** dave-mccowan has joined #openstack-sahara | 18:59 | |
| *** chlong_ has joined #openstack-sahara | 19:09 | |
| *** aolwas has joined #openstack-sahara | 20:03 | |
| *** cpusmith has joined #openstack-sahara | 20:09 | |
| *** ssmith has quit IRC | 20:09 | |
| *** cpusmith has quit IRC | 20:13 | |
| *** dave-mcc_ has joined #openstack-sahara | 20:13 | |
| *** dave-mccowan has quit IRC | 20:16 | |
| *** dave-mcc_ is now known as dave-mccowan | 20:17 | |
| *** dave-mccowan has quit IRC | 21:01 | |
| *** chlong_ has quit IRC | 21:20 | |
| *** chlong has quit IRC | 21:20 | |
| *** lucasxu has quit IRC | 21:47 | |
| *** rcernin has quit IRC | 22:02 | |
| *** tosky has quit IRC | 23:09 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!