| *** egafford has quit IRC | 00:00 | |
| *** dave-mccowan has quit IRC | 00:56 | |
| *** shuyingya has joined #openstack-sahara | 01:18 | |
| *** esikachev has joined #openstack-sahara | 01:46 | |
| *** esikachev has quit IRC | 01:50 | |
| *** masber has joined #openstack-sahara | 02:26 | |
| *** Poornima has joined #openstack-sahara | 02:52 | |
| *** sudipto has joined #openstack-sahara | 03:01 | |
| *** sudipto_ has joined #openstack-sahara | 03:01 | |
| *** links has joined #openstack-sahara | 03:10 | |
| *** links has quit IRC | 03:10 | |
| *** esikachev has joined #openstack-sahara | 03:46 | |
| *** esikachev has quit IRC | 03:51 | |
| *** tuanluong has joined #openstack-sahara | 04:06 | |
| *** sudipto has quit IRC | 04:59 | |
| *** sudipto_ has quit IRC | 04:59 | |
| *** sudipto has joined #openstack-sahara | 05:29 | |
| *** sudipto_ has joined #openstack-sahara | 05:29 | |
| *** esikachev has joined #openstack-sahara | 05:47 | |
| *** esikachev has quit IRC | 05:52 | |
| *** shuyingya has quit IRC | 06:27 | |
| *** shuyingya has joined #openstack-sahara | 06:29 | |
| *** masber has joined #openstack-sahara | 07:05 | |
| *** esikachev has joined #openstack-sahara | 07:06 | |
| *** esikachev has quit IRC | 07:11 | |
| *** zemuvier has joined #openstack-sahara | 07:11 | |
| masber | hi | 07:15 |
|---|---|---|
| masber | what is the ambari process when I am creating a node template? | 07:15 |
| masber | what is the functionality of the ambari process I mean? | 07:16 |
| *** exploreshaifali has joined #openstack-sahara | 07:17 | |
| *** shuyingya has quit IRC | 07:17 | |
| *** nkrinner_afk is now known as nkrinner | 07:19 | |
| *** shuyingya has joined #openstack-sahara | 07:20 | |
| *** pcaruana has joined #openstack-sahara | 07:21 | |
| *** esikachev has joined #openstack-sahara | 07:32 | |
| *** exploreshaifali has quit IRC | 07:34 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: Removed unused spark 1.3.1 job https://review.openstack.org/404067 | 07:41 |
| openstackgerrit | Alina Nesterova proposed openstack/sahara-tests: Add new negative cli tests for image and job binary https://review.openstack.org/397128 | 07:48 |
| masber | hi | 08:14 |
| masber | I am not sure why | 08:14 |
| masber | sahara is not creating clusters | 08:14 |
| masber | this is the error I can see --> http://pastebin.com/raw/DTVKJ9Gu | 08:14 |
| *** nkrinner has quit IRC | 08:28 | |
| ltosky[m] | check that you have enough resources to instantiate all the instances requested by the cluster template | 08:31 |
| ltosky[m] | Sahara uses heat to get the required resources (instances, volumes, etc) | 08:32 |
| *** nkrinner has joined #openstack-sahara | 08:34 | |
| *** nkrinner has quit IRC | 08:38 | |
| masber | ltosky[m], I am pretty sure I have enough resources, basically I have the whole server for myself and I am just running 1 vm now | 08:39 |
| masber | the cluster only has 1 nodes | 08:39 |
| masber | medium size | 08:39 |
| masber | would the quota be a problem here? | 08:40 |
| *** pgadiya has joined #openstack-sahara | 08:43 | |
| *** zemuvier has quit IRC | 08:49 | |
| *** rcernin has joined #openstack-sahara | 08:56 | |
| openstackgerrit | Shu Yingya proposed openstack/sahara: Fix error of CDH plugin scale up more than one node https://review.openstack.org/404091 | 08:59 |
| *** zemuvier has joined #openstack-sahara | 09:08 | |
| ltosky[m] | Quota could be a problem; check what nova logs say when the heat error happens | 09:09 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: [WIP] Added job for spark with python3 https://review.openstack.org/402588 | 09:27 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: [WIP] Added job for spark with python3 https://review.openstack.org/402588 | 09:32 |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-tests: testcommit for python 3 job https://review.openstack.org/402610 | 09:33 |
| *** mkoderer has joined #openstack-sahara | 09:37 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: [WIP] Added job for spark with python3 https://review.openstack.org/402588 | 09:40 |
| *** openstackgerrit has quit IRC | 09:48 | |
| *** openstackgerrit has joined #openstack-sahara | 09:49 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: [WIP] Added job for spark with python3 https://review.openstack.org/402588 | 09:59 |
| masber | ltosky[m], I think the problem was that I was assigning the public net as management network | 10:00 |
| masber | I have changed that and now it looks it goes further but still failing with same error | 10:00 |
| masber | the thing is that nova does not throw any error now, but heat is still throwing same error message | 10:01 |
| *** zemuvier has quit IRC | 10:12 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: [WIP] Added job for spark with python3 https://review.openstack.org/402588 | 10:20 |
| *** abehl has joined #openstack-sahara | 10:22 | |
| ltosky[m] | Then check the heat errors :) | 10:26 |
| ltosky[m] | Including openstack stack list | 10:26 |
| *** esikachev has quit IRC | 10:30 | |
| *** esikachev has joined #openstack-sahara | 10:31 | |
| *** esikachev has quit IRC | 10:35 | |
| *** tosky has joined #openstack-sahara | 10:48 | |
| *** sudipto_ has quit IRC | 11:03 | |
| *** sudipto has quit IRC | 11:03 | |
| tosky | masber: any luck so far? :) | 11:06 |
| * tosky backs in his IRC nick | 11:06 | |
| masber | tosky, mmmm I think nova is filtering out my host | 11:07 |
| masber | tosky, this is what I can see on nova logs --> http://pastebin.com/raw/PQk3t4ie | 11:08 |
| masber | no errors but the "Filtering removed all hosts for the request with instance ID" is what I think is causing the issue? | 11:08 |
| masber | anyway I put a message at #openstack but so far nobody has say anything | 11:09 |
| masber | it is not a quota limit nor disk space, I checked that already | 11:09 |
| tosky | but then " Instance spawned successfully." | 11:10 |
| tosky | are you running sahara, or did you try to spawn a normal instance | 11:10 |
| tosky | ? | 11:10 |
| masber | im running sahara | 11:11 |
| masber | im trying to launch a cluster | 11:11 |
| tosky | and did yoy try to spawn an instance of something different from cirros? An ubuntu or a centos image? | 11:16 |
| tosky | or more of them, but outside sahara | 11:16 |
| tosky | just to check where the issue is (pretty sure it's not sahara itself) | 11:16 |
| masber | tosky, I am using centos cloud image, I do can create an instance using horizon + nova | 11:20 |
| masber | havent try heat tho | 11:20 |
| tosky | and more instances? And yes, please try a simple heat templates too (which runs multiple instances) | 11:20 |
| *** esikachev has joined #openstack-sahara | 11:21 | |
| masber | tosky, ok, deplying heat will take time as I have no idea how to do it, will need to read about it unless you can give me straight instructions on how to do it | 11:22 |
| tosky | masber: heat is already deployed, you simply need to write a simple template and feed it to heat | 11:22 |
| tosky | masber: but first please try to run multiple instances | 11:23 |
| masber | ok | 11:23 |
| *** shuyingya has left #openstack-sahara | 11:24 | |
| *** shuyingya has joined #openstack-sahara | 11:24 | |
| masber | I was trying to create a cluster with 2 nodes (1 worker and 1 head) I am trying to deploy a cluster with only 1 node having both roles (head and worker) will let you know how it goes | 11:25 |
| shuyingya | masber here is a simple example for creating a heat stack, FYI | 11:28 |
| shuyingya | http://docs.openstack.org/developer/heat/getting_started/create_a_stack.html | 11:28 |
| masber | wow! thank you shuyingya | 11:29 |
| shuyingya | not at all ^^ | 11:29 |
| masber | ok, | 11:30 |
| masber | cluster with 1 node is created successful | 11:30 |
| masber | I will now try to create 2 different instances using same flavor | 11:30 |
| shuyingya | Hi tosky, still here? about patch https://review.openstack.org/#/c/373602 How about using pkg_resources to determine the version of saharaclient and then building the configs-file-name ? I want to find a proper way but I can't so far | 11:31 |
| *** esikachev has quit IRC | 11:31 | |
| *** esikachev has joined #openstack-sahara | 11:42 | |
| tosky | shuyingya: good question, the pkg_resources way does not work properly? | 11:43 |
| *** Poornima has quit IRC | 11:44 | |
| tosky | (in the worst case, you could try to remove the file which follows the new pattern, and ignore the other case - it's a limit case anyway, as the file should not be there) | 11:46 |
| shuyingya | it works properly, but I want to know is there any other more proper way? | 11:47 |
| *** esikachev has quit IRC | 11:47 | |
| shuyingya | aha, saying about the worst case, as I am concerned, using pkg_resources may be a good way to solve. | 11:49 |
| shuyingya | So I also need add a TODO ? | 11:50 |
| tosky | I mean, that code path is useful only to remove a file which should not be there normally because it should have been cleaned up at the end of the test | 11:51 |
| tosky | so even if we simply try to clean the new version of the file, that would be fine (and in 1 year it will be the only way) | 11:54 |
| tosky | pkg_resources, I'm not too much expert, is the 1.0.0 the proper version to compare? | 11:57 |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara: Adding name_check parameter for cluster_get_all https://review.openstack.org/404175 | 12:00 |
| tellesnobrega | vgridnev, can you take a look at ^ | 12:00 |
| shuyingya | Yes, I check it https://review.openstack.org/gitweb?p=openstack/python-saharaclient.git;a=shortlog;h=82cabbd3c9e3fe7b310a086a03a4c0730ccd148d | 12:00 |
| tosky | shuyingya: what exactly? | 12:02 |
| *** esikachev has joined #openstack-sahara | 12:03 | |
| shuyingya | There is a 1.0.0 tag include the patch with new config name | 12:05 |
| shuyingya | tosky | 12:05 |
| tosky | oh, we pushed to 1.0.0 | 12:06 |
| tosky | it's the tag, I see | 12:06 |
| tosky | ok | 12:06 |
| *** esikachev has quit IRC | 12:07 | |
| shuyingya | Hum, I still don't understand if we use saharaclient with old version but saharatest with new version name, the dirty env will also pass the test. Is it right? | 12:11 |
| vgridnev | tosky, we did 1.0.0 tag because it introduce incompatibility: old cli commands are removed | 12:11 |
| tosky | vgridnev: ah, right | 12:12 |
| vgridnev | tellesnobrega, yep, I will take a look on that | 12:12 |
| shuyingya | btw, I have reported a bug "https://bugs.launchpad.net/sahara-tests/+bug/1644747" Do you have idea? tosky | 12:12 |
| openstack | Launchpad bug 1644747 in OpenStack Data Processing ("Sahara") sahara-tests " tempest tests doesn't clean env after test" [Undecided,New] - Assigned to Shu Yingya (felixshu) | 12:12 |
| tellesnobrega | vgridnev, thanks | 12:12 |
| tosky | shuyingya: well, an addCleanup is missing somewhere | 12:12 |
| tosky | shuyingya: about the dirty env: if an old version of saharaclient is used, the name will be the old one | 12:13 |
| *** tuanluong has quit IRC | 12:14 | |
| tosky | anyway, I apologize, but what about trying addCleanup too in https://review.openstack.org/#/c/373602 for each post-test cleanup ? | 12:14 |
| shuyingya | tosky, I will do it later. :) | 12:17 |
| tosky | also, about https://review.openstack.org/#/c/367129/, I'm fine but we need a change for configure_tempest_for_sahara in devstack/plugin.sh in the stable/mitaka branch to set that value as false | 12:18 |
| masber | ok, so I can create the stack | 12:18 |
| masber | and 2 instances with same flavor | 12:18 |
| masber | but the cluster still fails | 12:19 |
| shuyingya | OK, thanks for your suggestion tosky. | 12:19 |
| openstackgerrit | Vitaly Gridnev proposed openstack/sahara: Updating list of plugins in config sample https://review.openstack.org/398879 | 12:22 |
| *** esikachev has joined #openstack-sahara | 12:23 | |
| *** exploreshaifali has joined #openstack-sahara | 12:24 | |
| *** esikachev has quit IRC | 12:28 | |
| *** exploreshaifali has quit IRC | 12:29 | |
| *** esikachev has joined #openstack-sahara | 12:38 | |
| *** shuyingya has quit IRC | 12:39 | |
| *** _elmiko is now known as elmiko | 12:41 | |
| *** esikachev has quit IRC | 12:55 | |
| *** masuberu has joined #openstack-sahara | 13:03 | |
| *** pgadiya has quit IRC | 13:14 | |
| openstackgerrit | Merged openstack/sahara-ci-config: Remove micro-osd.sh from config/devstack https://review.openstack.org/400145 | 13:22 |
| *** esikachev has joined #openstack-sahara | 13:23 | |
| *** abehl has quit IRC | 13:25 | |
| *** esikache1 has joined #openstack-sahara | 13:26 | |
| *** esikachev has quit IRC | 13:29 | |
| *** egafford has joined #openstack-sahara | 13:40 | |
| *** dave-mccowan has joined #openstack-sahara | 13:43 | |
| *** sudipto has joined #openstack-sahara | 13:45 | |
| *** sudipto_ has joined #openstack-sahara | 13:45 | |
| openstackgerrit | Telles Mota Vidal Nóbrega proposed openstack/sahara: Adding name_check parameter for cluster_get_all https://review.openstack.org/404175 | 14:18 |
| masber | ok, I realised I can creat clusters using sahara but only if there is no more than 1 volume for the whole cluster | 14:27 |
| masber | I will keep investigationg | 14:27 |
| raissa | masber: maybe with more than that you exceed the total available | 14:40 |
| raissa | (for volumes) | 14:40 |
| masber | I just got a clustr with 3 nodes and 2 volumes | 14:40 |
| masber | my quota is much bigger than that | 14:41 |
| masber | what is the ambari component? is it the one that deploys the hadoop packages on the cluster? | 14:42 |
| tosky | only for the HDP plugin | 14:43 |
| tosky | masber: how is your cinder configured? Maybe the total size available for volumes it's not enough | 14:43 |
| masber | tosky, yeah this is something I need to check | 14:43 |
| masber | but I have everything on a 9TB drive | 14:44 |
| masber | and it is empty | 14:44 |
| tosky | that's not important :) | 14:44 |
| tosky | I mean, when you install with RDO with packstack, for example, the default Cinder configuration uses LVM and the volume is 20G by default | 14:44 |
| masber | ?? | 14:44 |
| masber | I used pakstack | 14:45 |
| masber | let me see | 14:45 |
| tosky | Cinder does not create volumes out of the blue, using all the possible space, but it uses a specific backend | 14:45 |
| tosky | the default one in packstack may be too small | 14:45 |
| masber | how can I check that? | 14:46 |
| masber | is this what you mean? http://pastebin.com/raw/AXuBNpdb | 14:48 |
| masber | tosky, ok I think this is what you mean http://pastebin.com/raw/tmiwUdhL | 14:50 |
| masber | I can see a physical volume of 20Ggs | 14:50 |
| tosky | so reduce the size of the attached volumes | 14:51 |
| masber | tosky, I think you nailed it, I am reinstalling using 8TB for cinder | 14:54 |
| tosky | masber: oh, if it's a test instance, you need much less than that | 14:55 |
| masber | tosky, it is a test instance but we need to test haddop clusters! | 14:56 |
| tosky | it depends on which test clusters :D | 14:56 |
| tosky | but yeah | 14:56 |
| masber | tosky, so we need space because we want to deploy nodes with local storage | 14:56 |
| masber | tosky, the ambari component is the one that installs the hadoop/hbase/etc? | 15:06 |
| masber | or what is the purpose of having an ambari component on a hdp cluster? | 15:07 |
| tosky | no, it's an installer but only for HDP | 15:08 |
| tosky | Cloudera uses its own manager | 15:08 |
| tosky | same for MapR | 15:09 |
| masber | tosky, what do you mean by an installer? | 15:09 |
| tosky | https://ambari.apache.org/ manager | 15:10 |
| masber | tosky, I changed to CONFIG_CINDER_VOLUMES_SIZE=8000G | 15:13 |
| masber | and reinstalled packstack using the answer file, however the physical volume is still 20Gigs | 15:13 |
| tosky | rerun packstack? | 15:14 |
| tosky | maybe it does not update that | 15:14 |
| tosky | you need to do a bit of manual step to recreate that volume | 15:14 |
| masber | damn | 15:15 |
| *** zemuvier has joined #openstack-sahara | 15:15 | |
| masber | can I use pvresize?? | 15:16 |
| *** zemuvier has quit IRC | 15:21 | |
| tosky | I don't remember if it expands the backing file | 15:21 |
| tosky | I usually detach the VG, detach the PV, recreate the backing image with bigger size, recreate the PV, and the VG with the same name (cinder-volumes, iirc) | 15:22 |
| *** chlong has joined #openstack-sahara | 15:27 | |
| *** Poornima has joined #openstack-sahara | 15:33 | |
| *** esikache1 has quit IRC | 15:33 | |
| *** abehl has joined #openstack-sahara | 15:34 | |
| *** masuberu has quit IRC | 15:39 | |
| *** rcernin has quit IRC | 16:03 | |
| *** chlong has quit IRC | 16:07 | |
| *** Poornima has quit IRC | 16:19 | |
| *** chlong has joined #openstack-sahara | 16:20 | |
| *** abehl has quit IRC | 16:32 | |
| *** esikache1 has joined #openstack-sahara | 16:33 | |
| *** pcaruana has quit IRC | 16:36 | |
| *** esikache1 has quit IRC | 16:42 | |
| *** esikache1 has joined #openstack-sahara | 16:46 | |
| openstackgerrit | Evgeny Sikachev proposed openstack/sahara-ci-config: [WIP] Added job for spark with python3 https://review.openstack.org/402588 | 16:46 |
| *** esikache1 has quit IRC | 16:52 | |
| tosky | egafford: when you can, there is a bunch of backports (5) for the stable branches in the review queue | 17:05 |
| egafford | tosky: Cool; I just cleaned my other queues out and am going to hit reviews hard all afternoon. | 17:08 |
| tosky | #o/ | 17:08 |
| tosky | $!"%!"% keyboard layouts | 17:08 |
| egafford | What's that? | 17:08 |
| tosky | \o/ | 17:08 |
| egafford | Oh okay. | 17:08 |
| egafford | I thought maybe you had a shield in one hand and were shouting in triumph at your enemies. | 17:09 |
| tosky | (switching between UK keyboard, the native one on the laptop, and the US one, the external keyboard) | 17:09 |
| egafford | Which didn't seem very tosky, but one never knows. | 17:09 |
| tosky | (luckily compose keys allows me to not have to switch to the Italian for letters with accents) | 17:10 |
| *** chlong has quit IRC | 17:10 | |
| egafford | Woot. | 17:10 |
| tosky | I guess it's more complicated with people here with non-latin keyboards :) | 17:11 |
| egafford | One imagines, yes. :) | 17:12 |
| *** chlong has joined #openstack-sahara | 17:24 | |
| *** rodrigods is now known as rodrigods_ | 18:02 | |
| *** rodrigods_ is now known as rodrigods | 18:06 | |
| *** rodrigods has quit IRC | 18:07 | |
| *** rodrigods has joined #openstack-sahara | 18:07 | |
| *** mkoderer has quit IRC | 18:13 | |
| *** sudipto_ has quit IRC | 18:13 | |
| *** sudipto has quit IRC | 18:13 | |
| *** tosky has quit IRC | 18:26 | |
| *** esikache1 has joined #openstack-sahara | 19:19 | |
| *** masuberu has joined #openstack-sahara | 21:01 | |
| *** dave-mccowan has quit IRC | 21:06 | |
| *** esikache1 has quit IRC | 21:20 | |
| *** masuberu has quit IRC | 21:49 | |
| *** rodrigods is now known as rodrigods_ | 21:56 | |
| *** masuberu has joined #openstack-sahara | 22:00 | |
| *** masber has quit IRC | 22:03 | |
| *** masuberu has quit IRC | 22:05 | |
| *** masuberu has joined #openstack-sahara | 22:08 | |
| *** chlong has quit IRC | 22:12 | |
| *** esikache1 has joined #openstack-sahara | 22:17 | |
| *** esikache1 has quit IRC | 22:22 | |
| *** rcernin has joined #openstack-sahara | 22:31 | |
| *** rcernin has quit IRC | 23:01 | |
| *** rcernin has joined #openstack-sahara | 23:03 | |
| *** rcernin has quit IRC | 23:04 | |
| *** rcernin has joined #openstack-sahara | 23:04 | |
| *** rodrigods_ is now known as rodrigods | 23:17 | |
| *** esikache1 has joined #openstack-sahara | 23:18 | |
| *** esikache1 has quit IRC | 23:22 | |
| *** rcernin has quit IRC | 23:41 | |
Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!