| *** Bhujay has joined #openstack-sahara | 02:49 | |
| *** Bhujay has quit IRC | 03:07 | |
| openstackgerrit | Leopard Ma proposed openstack/sahara master: Add Blueprints notes link to README.rst https://review.openstack.org/605320 | 03:25 |
|---|---|---|
| *** dave-mccowan has quit IRC | 03:44 | |
| *** dave-mccowan has joined #openstack-sahara | 03:46 | |
| *** pcaruana has joined #openstack-sahara | 04:14 | |
| *** pcaruana has quit IRC | 04:38 | |
| openstackgerrit | Jeremy Freudberg proposed openstack/python-saharaclient master: [WIP] Refresh saharaclient doc https://review.openstack.org/605330 | 04:44 |
| *** Bhujay has joined #openstack-sahara | 05:26 | |
| *** Bhujay has quit IRC | 05:32 | |
| *** dave-mccowan has quit IRC | 05:39 | |
| *** pcaruana has joined #openstack-sahara | 05:43 | |
| *** Bhujay has joined #openstack-sahara | 05:49 | |
| *** rcernin has quit IRC | 07:02 | |
| *** rcernin has joined #openstack-sahara | 07:56 | |
| *** tosky has joined #openstack-sahara | 08:48 | |
| *** rcernin has quit IRC | 09:16 | |
| *** pcaruana has quit IRC | 11:15 | |
| *** guimaluf has quit IRC | 12:21 | |
| *** guimaluf has joined #openstack-sahara | 12:26 | |
| tellesnobrega | tosky, we could, but does it make sense to? | 13:18 |
| tosky | tellesnobrega: the idea is not to lose the tests for the validation feature in the sahara core | 13:21 |
| tosky | otherwise you would need specific unit tests for validation for each plugin, which may work too, depending on what the unit tests were checking | 13:22 |
| tellesnobrega | yeah, I'm not sure we want to have too much on the fake plugin thou. I will take a look | 13:22 |
| *** tellesnobrega has quit IRC | 13:56 | |
| *** Bhujay has quit IRC | 14:44 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: DNM test only - url curl in a failing image build job https://review.openstack.org/602507 | 14:46 |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: DNM test only - more debug for the error on image build https://review.openstack.org/602504 | 14:47 |
| *** pcaruana has joined #openstack-sahara | 15:01 | |
| openstackgerrit | Luigi Toscano proposed openstack/sahara master: DNM test only - more debug for the error on image build https://review.openstack.org/602504 | 15:06 |
| Gaasmann | 2018-09-26 15:10:00.527 19 ERROR sahara.service.ops [req-a8e39ef7-27ec-471a-89ad-5b443c397bb7 c2d70c68ef50422f84add4a7b107f8ae 43850a29dfee4b70b02edbfbc8d7c1eb - - -] [instance: none, cluster: 41fce076-fdbd-44d1-9c74-1211a56866d0] Error during operating on cluster (reason: 'Operation with name 'Await starting Cloudera Manager'' timed out after 300 second(s) and following timeout was violated: | 15:28 |
| Gaasmann | Timeout for Cloudera Manager starting | 15:28 |
| Gaasmann | Is there a way to increase this timeout? | 15:28 |
| tosky | Gaasmann: there should be an option in the cluster template configuration | 15:40 |
| *** Bhujay has joined #openstack-sahara | 15:46 | |
| guimaluf | I'm getting the same thing as Gaasmann, "WaitConditionTimeout: 0 of 1 received" on heat-engine. This probably is related to the excess of resourcers I'm using. What seems odd is that nova overprovision are very flexible, since I'm using a large value for overprovision. | 15:46 |
| tosky | uhm, that's a more generic error and it happens before | 15:48 |
| tosky | sahara asks heat "give me few machines with the following features", and here is guimaluf 's error; when heat is done, sahara starts to configure the systems | 15:49 |
| tosky | including starting services like cloudera manager, and here is Gaasmann's error | 15:50 |
| guimaluf | tosky, I'm investigatin and seems that my image is throwing disk error... maybe that's why the timeout happen | 15:51 |
| tosky | uh | 15:51 |
| guimaluf | tosky, where can I find details on sahara internals? | 15:52 |
| tosky | the documentation should give some general details | 15:52 |
| tosky | but which kind of disk errors? At which stage? | 15:52 |
| guimaluf | tosky, vm booting... | 15:53 |
| guimaluf | Serious errors were found while checking the disk drive for /. | 15:53 |
| guimaluf | keys:Press I to ignore, S to skip mounting, or M for manual recovery | 15:53 |
| tosky | guimaluf: can you boot those images with nova, outside sahara? | 15:53 |
| tosky | uhm, no | 15:53 |
| tosky | so the images are dead | 15:53 |
| *** dave-mccowan has joined #openstack-sahara | 15:53 | |
| tosky | which kind of image is that? | 15:53 |
| guimaluf | tosky, I figure out that this is happening when I boot outside sahara | 15:53 |
| guimaluf | tosky, the one I build using sahara-image-elements on stable/pike | 15:54 |
| tosky | centos7? Ubuntu? | 15:54 |
| guimaluf | ubuntu | 15:54 |
| tosky | how did you run sahara-image-create? (I may have already asked it) | 15:54 |
| guimaluf | the same happen with another image I generated | 15:54 |
| tosky | which plugin? | 15:54 |
| guimaluf | tosky, no, u didn't :) | 15:54 |
| tosky | I mean, the environment | 15:55 |
| guimaluf | -p spark -i ubuntu | 15:55 |
| tosky | ok, a spark image | 15:55 |
| guimaluf | yeah | 15:55 |
| guimaluf | My goal is to use spark | 15:55 |
| guimaluf | so I leave hadoop vanilla behind heheh | 15:55 |
| tosky | the image about how is: where did you run sahara-image-create? Do you have the full log? Did you run with tox -e ... from the stable/pike sources? | 15:56 |
| guimaluf | tox -e venv -- sahara-iamge-create -p spark -i ubuntu | 15:56 |
| guimaluf | tosky, I ran in my gentoo machine | 15:56 |
| *** dave-mccowan has quit IRC | 15:57 | |
| tosky | uh | 15:57 |
| tosky | can you please create a ubuntu (xenial) local VM and run sahara-image-create from it? | 15:57 |
| tosky | or an instance of ubuntu somewhere | 15:57 |
| tosky | in general, I don't reccomend to run diskimage-builder on production systems, but on isolated machines | 15:58 |
| guimaluf | tosky, I've seen this recommendation but ignored xD | 15:58 |
| tosky | and while it should not matter, I don't think we ever tried to run sahara-image-create from outside ubuntu, centos and sometimes fedora | 15:58 |
| *** tellesnobrega has joined #openstack-sahara | 16:00 | |
| guimaluf | tosky, I'll do this way | 16:00 |
| *** dave-mccowan has joined #openstack-sahara | 16:01 | |
| Gaasmann | tosky: thanks, I'll take a look | 16:05 |
| *** Bhujay has quit IRC | 16:29 | |
| Gaasmann | The key user-facing interface to this feature is the CLI script sahara-image-pack. This script will be installed with all other Sahara binaries. | 16:48 |
| Gaasmann | Where is sahara-image-pack? | 16:48 |
| Gaasmann | with python-saharaclient or sahara-image-elements ? | 16:49 |
| Gaasmann | I can't find it | 16:49 |
| *** dave-mccowan has quit IRC | 16:49 | |
| tellesnobrega | its within sahara | 16:49 |
| tellesnobrega | https://github.com/openstack/sahara/tree/master/sahara/cli/image_pack | 16:50 |
| Gaasmann | ho ok, does it need a working cluster or I can put that on a VM just to build images? | 16:51 |
| tosky | you can put it in a separate VM | 16:51 |
| tosky | or even if your local system | 16:51 |
| tosky | in RDO we package it in a separate package, which only depends on the common python modules of sahara, so it's a standalone command | 16:52 |
| Gaasmann | ok perfect, thanks | 16:52 |
| guimaluf | tosky, building the image through ubuntu works outside sahara, now I'll test inside :) | 17:10 |
| tosky | yay | 17:12 |
| tellesnobrega | goood | 17:14 |
| guimaluf | | spark-teste-02 | c7507a28-496f-4796-b49b-c245845193be | spark | 2.1.0 | Active | | 17:45 |
| guimaluf | uhhuuhuhuh | 17:45 |
| guimaluf | now I've to learn how to use spark cluster xD | 17:47 |
| tosky | :D | 18:07 |
| openstackgerrit | Merged openstack/sahara stable/rocky: Add template param for ambari pkg install timeout https://review.openstack.org/605226 | 18:07 |
| tosky | from time to time we have bugs too, but many many times is not sahara's fault :) | 18:09 |
| *** pcaruana has quit IRC | 18:24 | |
| Gaasmann | Can I use sahara-image-pack to build images for vanilla,spark and storm plugins? I only see cdh,ambari and mapr as available plugins | 18:26 |
| tosky | not yet | 18:29 |
| tosky | there is a WIP patch for spark | 18:30 |
| tosky | and the idea is to port the others too | 18:30 |
| tosky | patches welcome :) | 18:30 |
| Gaasmann | ok cool :-) | 18:31 |
| tosky | when we add the support for sahara-image-pack to all plugins, we can drop sahara-image-elements | 18:36 |
| guimaluf | tosky, is it possible to run spark with yarn? how could I generate that image, or run any job with yarn? | 18:50 |
| *** dave-mccowan has joined #openstack-sahara | 19:05 | |
| Gaasmann | sahara-image-pack uses kvm on qcow2 images only right? | 19:07 |
| tosky | Gaasmann: it should be able to use everything supported by libguestfs | 19:16 |
| tosky | guimaluf: uh, I think we discussed that during the PTG, I think it's not possible with that image, but you can do it with other Hadoop distributions which ships spark | 19:17 |
| * tosky ping tellesnobrega for double checking the sentence above | 19:17 | |
| guimaluf | tosky, checking the vanilla elements I realized it comes with spark... :) I think it will be enough | 19:18 |
| Gaasmann | tosky: I don't know how to pass the arguments then. It tries to spawn a VM but I'm already on a VM to build the images, so it complains about that and the raw format. | 19:24 |
| tosky | Gaasmann: which VM, and which error? | 19:25 |
| tosky | Gaasmann: the qcow2 format is the easy part (just get the qcow2 cloud image for the distribution, it's even easier), even if I was mostly sure that it would work with any supported format | 19:26 |
| tosky | uh, no | 19:26 |
| tosky | ok, we hardcode it to qcow2 | 19:26 |
| tosky | that's something that we could change | 19:26 |
| tosky | but for now you need a qcow2 image | 19:26 |
| Gaasmann | ok, and the process spawn a vm | 19:26 |
| Gaasmann | with -machine accel=kvm:tcg | 19:27 |
| tosky | the other error may be related to ubuntu, which does not have few kernel files readable by users | 19:27 |
| Gaasmann | I'm on a VM already, I don't have VT-x | 19:27 |
| tosky | which VM? | 19:27 |
| tosky | ubuntu? | 19:27 |
| Gaasmann | Yep I got the permission denied on vmlinuz also | 19:27 |
| tosky | sudo dpkg-statoverride --add --update root root 0644 /boot/vmlinuz-$(uname -r) | 19:27 |
| Gaasmann | Xenial yes | 19:27 |
| tosky | the ubuntu kernel maintainer decided something weird, hence: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/759725 | 19:28 |
| openstack | Launchpad bug 759725 in linux (Ubuntu) "The kernel is no longer readable by non-root users" [Medium,Won't fix] | 19:28 |
| *** rcernin has joined #openstack-sahara | 22:35 | |
| *** tosky has quit IRC | 22:37 | |
| *** rcernin has quit IRC | 22:53 | |
| *** rcernin has joined #openstack-sahara | 22:55 | |
| *** rcernin_ has joined #openstack-sahara | 23:41 | |
| *** rcernin has quit IRC | 23:43 | |
| *** openstackgerrit has quit IRC | 23:49 | |
Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!