Tuesday, 2014-06-03

« Monday, 2014-06-02 Index Wednesday, 2014-06-04 »
*** tosky has quit IRC00:09
*** themistymay has joined #openstack-sahara00:12
*** zhiyan_ is now known as zhiyan00:16
openstackgerritAndrew Lazarev proposed a change to openstack/sahara: Made Swift topology optional for data locality  https://review.openstack.org/9737600:21
*** IlyaE has joined #openstack-sahara00:22
openstackgerritAndrew Lazarev proposed a change to openstack/sahara: Made Swift topology optional for data locality  https://review.openstack.org/9737600:22
*** alazarev has quit IRC00:58
*** alazarev has joined #openstack-sahara01:16
*** IlyaE has quit IRC01:28
*** IlyaE has joined #openstack-sahara01:31
*** ViswaV_ has joined #openstack-sahara01:46
*** ViswaV has quit IRC01:50
*** alazarev has quit IRC02:00
*** IlyaE has quit IRC02:30
*** shakamunyi has quit IRC02:31
*** zhiyan is now known as zhiyan_02:50
*** zhiyan_ is now known as zhiyan03:03
*** alazarev has joined #openstack-sahara03:29
*** alazarev has quit IRC03:48
*** alazarev has joined #openstack-sahara03:59
*** alazarev has quit IRC04:48
*** zhiyan is now known as zhiyan_05:01
*** ViswaV_ has quit IRC05:12
*** ViswaV has joined #openstack-sahara05:13
*** alazarev has joined #openstack-sahara05:17
*** IlyaE has joined #openstack-sahara05:32
openstackgerritOpenStack Proposal Bot proposed a change to openstack/sahara: Imported Translations from Transifex  https://review.openstack.org/9700606:03
*** IlyaE has quit IRC06:36
*** alazarev has quit IRC06:48
*** IlyaE has joined #openstack-sahara07:00
*** skolekonov has joined #openstack-sahara07:12
*** IlyaE has quit IRC07:31
*** jpich has joined #openstack-sahara07:53
*** witlessb has joined #openstack-sahara07:58
*** witlessb has joined #openstack-sahara07:59
*** IvanBerezovskiy has joined #openstack-sahara08:37
*** Timotey has joined #openstack-sahara08:57
*** vrovachev has joined #openstack-sahara09:03
openstackgerritChristian Berendt proposed a change to openstack/python-saharaclient: removed unused methods from nova.utils  https://review.openstack.org/9593209:05
*** bogdando has joined #openstack-sahara09:26
*** IvanBerezovskiy has quit IRC10:00
*** themistymay has quit IRC10:05
*** tellesnobrega has quit IRC10:07
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Change IRC channel name to #openstack-sahara  https://review.openstack.org/9745010:12
openstackgerritDmitry Mescheryakov proposed a change to openstack/python-saharaclient: Fix network passing for cluster-template-create  https://review.openstack.org/9725510:12
*** tellesnobrega has joined #openstack-sahara10:13
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Updated architecture diagram in docs  https://review.openstack.org/9745110:13
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Create trusts for admin user with correct tenant name  https://review.openstack.org/9745310:15
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Fix running EDP job on transient cluster  https://review.openstack.org/9745410:16
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Added validate_edp method to Plugin SPI doc  https://review.openstack.org/9745610:19
*** tosky has joined #openstack-sahara10:20
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Change IRC channel name to #openstack-sahara  https://review.openstack.org/9745010:27
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Added validate_edp method to Plugin SPI doc  https://review.openstack.org/9745610:27
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Create trusts for admin user with correct tenant name  https://review.openstack.org/9745310:27
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Fix running EDP job on transient cluster  https://review.openstack.org/9745410:27
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Updated architecture diagram in docs  https://review.openstack.org/9745110:27
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Change IRC channel name to #openstack-sahara  https://review.openstack.org/9745010:27
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Fix hardcoded tenant name for job binaries  https://review.openstack.org/9745810:27
*** IvanBerezovskiy has joined #openstack-sahara10:36
*** IvanBerezovskiy has quit IRC10:39
*** themistymay has joined #openstack-sahara10:39
*** themistymay has quit IRC10:45
openstackgerritSergey Kolekonov proposed a change to stackforge/sahara-ci-config: add check for Sahara branch  https://review.openstack.org/9746611:12
*** IvanBerezovskiy has joined #openstack-sahara11:24
*** yanndegat has joined #openstack-sahara11:29
yanndegathi every on e11:29
yanndegati'm trying to launch a hdp 2.0.6 cluster11:30
yanndegatit stucks on a "waiting" state11:30
yanndegathow can i dig further to know in what precise state it is stucked ?11:30
*** miqui has quit IRC11:38
yanndegatFYI, i try to launch this cluster with the centos6 image provided for shara11:39
yanndegatsahara11:39
openstackgerritSergey Kolekonov proposed a change to stackforge/sahara-ci-config: add stable/icehouse branch support  https://review.openstack.org/9746611:40
*** IvanBerezovskiy has quit IRC11:40
dmitrymeyanndegat: run Sahara with debug logs, create a new cluster and see what is it tries to do11:40
yanndegatthe api ?11:41
yanndegatin the /etc/sahara/conf ?11:41
dmitrymeyou can just add a parameter to the cli line, like: sahara-api —debug —config-file …11:41
yanndegatyou mean "create a cluster via the CLI in debug mode" ?11:43
yanndegatbecause i try to create the cluster via horizon11:43
dmitrymeno, I mean: 1. restart sahara-api with the —debug parameter. 2. try recreating the cluster the way you like, horizon is fine11:44
yanndegatok11:49
*** themistymay has joined #openstack-sahara11:49
yanndegatalright11:54
yanndegatits a cluster with one master and 3 slaves11:54
yanndegatthe logs output the following debug messages periodically11:54
yanndegatite-packages/sahara/service/engine.py:9511:55
yanndegat2014-06-03 14:55:01.766 22119 DEBUG sahara.utils.ssh_remote [-] [BIGDATA-bigD-slave-002] Executing "ls .ssh/authorized_keys" _log_command /usr/lib/python2.6/site-packages/sahara/utils/ssh_remote.py:40711:55
yanndegat2014-06-03 14:55:08.057 22119 DEBUG sahara.openstack.common.periodic_task [-] Running periodic task SaharaPeriodicTasks.update_job_statuses run_periodic_tasks /usr/lib/python2.6/site-packages/sahara/openstack/common/periodic_task.py:17111:55
yanndegat2014-06-03 14:55:08.058 22119 DEBUG sahara.service.periodic [-] Updating job statuses update_job_statuses /usr/lib/python2.6/site-packages/sahara/service/periodic.py:6211:55
yanndegat2014-06-03 14:55:08.066 22119 DEBUG sahara.openstack.common.loopingcall [-] Dynamic looping call sleeping for 39.62 seconds _inner /usr/lib/python2.6/site-packages/sahara/openstack/common/loopingcall.py:13011:55
dmitrymeyanndegat: I think the problem is that Sahara can not access the VMs to configure them12:01
yanndegatyet i can access the vms12:02
yanndegatthrough the floatingips with the keypair provided12:02
yanndegathow could i debug it and find what maybe wrong ?12:03
yanndegatfrom my mlaptop i can ssh the cluster instances on their associated floating ips with the keypair12:03
yanndegatis sahara waiting for the VM to start some kind of services ?12:07
yanndegatbecause when i login the vms12:07
yanndegatthere are error during the boot12:07
dmitrymesorry, I am on the meeting, will reply as I get free12:08
yanndegatok12:08
*** IvanBerezovskiy has joined #openstack-sahara12:11
dmitrymeyanndegat: sahara just checks VMs availability by running ‘ls .ssh/authorized_keys’ via ssh12:17
yanndegatyes12:18
yanndegati have one VM that is never accessible12:18
dmitrymetry accessing VMs by their floating ips from the server where Sahara is installed12:18
yanndegati think that i must have one misconfigured compute node12:18
yanndegat3 on 4 vms are accessible12:18
dmitrymeis it a worker or master node?12:18
dmitrymeIf it is BIGDATA-bigD-slave-002, I assume there is at least one more VM of the very same kind and it is accessible, right?12:19
*** julienvey has joined #openstack-sahara12:20
yanndegatno the 3 other instances are accesible via ssh12:21
yanndegatone master and 2 slaves12:21
*** zhiyan_ is now known as zhiyan12:21
dmitrymethen yes, seems like that is the problem with your setup12:24
dmitrymeat least Sahara does the same things to start VMs if they belong to a single node group12:25
*** zhiyan is now known as zhiyan_12:31
*** _crobertsrh is now known as crobertsrh12:47
*** tmckay has joined #openstack-sahara12:48
*** themistymay has quit IRC12:56
*** skolekonov has quit IRC13:00
*** themistymay has joined #openstack-sahara13:03
*** miqui has joined #openstack-sahara13:04
*** openstackgerrit has quit IRC13:06
*** openstackgerrit has joined #openstack-sahara13:06
*** _elmiko is now known as elmiko13:10
*** liyuezho has joined #openstack-sahara13:24
*** liyuezho has quit IRC13:24
openstackgerritDaniele Venzano proposed a change to openstack/sahara: Documentation for the Spark plugin  https://review.openstack.org/9749813:34
openstackgerritSofiia Kostiuchenko proposed a change to openstack/sahara: [HDP] Integration tests for HDP 2.0.6  https://review.openstack.org/9573913:35
openstackgerritDmitry Mescheryakov proposed a change to openstack/sahara: Remove usage of remote from HDP Instance object  https://review.openstack.org/9750213:49
*** themistymay has quit IRC13:51
*** shakamunyi has joined #openstack-sahara13:51
*** IvanBerezovskiy has quit IRC13:53
openstackgerritSergey Lukjanov proposed a change to openstack/sahara: Rework keystone auth_token middleware configs  https://review.openstack.org/7468814:13
*** sreshetnyak has joined #openstack-sahara14:18
*** IvanBerezovskiy has joined #openstack-sahara14:23
*** alazarev has joined #openstack-sahara14:28
yanndegathi14:37
yanndegatnew error now14:37
yanndegat/dev/vdb is mounted; will not make a filesystem here!14:37
yanndegati'm trying to spawn a cluster with cindervolumes14:38
yanndegatit creates the volumes, attach them on /dev/vdc14:38
yanndegatbut i get an error in the sahara/api.log : /dev/vdb is mounted14:38
yanndegatwill not make a filesystem here!14:39
dmitrymeyanndegat: can you show the full stack trace? I didn’t see such error before14:40
yanndegathttp://paste.openstack.org/show/82610/14:42
dmitrymeyanndegat: seems our volume handling logic does not work properly in your setup :-(14:56
yanndegatwhat should i do to make it pass ?14:56
dmitrymegood question. Lets try to debug this14:56
dmitrymecan you comment the following line and recreate the cluster?14:57
dmitrymehttps://github.com/openstack/sahara/blob/master/sahara/service/direct_engine.py#L28814:57
dmitrymethe problem is that by default Sahara destroys cluster if provisioning fails14:57
dmitrymewe need to see what is going on live VM14:57
*** IlyaE has joined #openstack-sahara14:59
yanndegatrunning15:02
yanndegathere we are15:03
yanndegatwhat do you wnat me to do on the Vms.15:04
yanndegat?15:04
yanndegatha15:04
yanndegatit deleted the isntances15:04
yanndegati commented the 288 line15:04
dmitrymeplease show us output of two commands:15:04
dmitrymecat /proc/partitions15:05
dmitrymeand15:05
dmitrymemount15:05
yanndegatthe instances are deleted15:05
dmitrymehmm15:05
dmitrymehey, did you restart Sahara after you commented the code?15:06
yanndegatnope15:06
*** bradd1 has joined #openstack-sahara15:06
dmitrymeseems like we reproduced the issue. Say, the flavor you use, does it have ephemeral drive?15:07
yanndegatno15:08
yanndegatonly cinder volumes15:08
*** alazarev has quit IRC15:08
yanndegat10 Go Root disk15:08
yanndegat0 Ephemeral15:08
yanndegat1024Mo Swap15:08
dmitrymewe found the problem on flavor 20G root | 20G ephemeral | 0m swap15:10
yanndegatwhat could be a correct conf for the moment ?15:10
dmitrymeseems like we used for testing flavors which had only root drive, and no ephemeral or swap15:11
yanndegatthat's what i've configured15:11
dmitrymeif you want a workaround, make a flavor which have 0 ephemeral and 0 swap15:11
dmitrymethat should work15:11
yanndegatand in the nodegroup ?15:11
yanndegatwhat conf ?15:11
dmitrymewhat you mean “what conf in the nodegroup”?15:12
yanndegat[root@bigdatamec-bigd-slave-001 ~]# cat /proc/partitions15:12
yanndegatmajor minor  #blocks  name15:12
yanndegat 253        0   10485760 vda15:12
yanndegat 253        1   10474380 vda115:12
yanndegat 253       16    1048576 vdb15:12
yanndegat 253       32    2097152 vdc15:12
yanndegat[root@bigdatamec-bigd-slave-001 ~]# mount15:12
yanndegat/dev/vda1 on / type ext4 (rw)15:12
yanndegatproc on /proc type proc (rw)15:12
yanndegatsysfs on /sys type sysfs (rw)15:12
yanndegatdevpts on /dev/pts type devpts (rw,gid=5,mode=620)15:12
yanndegattmpfs on /dev/shm type tmpfs (rw,rootcontext="system_u:object_r:tmpfs_t:s0")15:12
yanndegatnone on /proc/sys/fs/binfmt_misc type binfmt_misc (rw)15:12
yanndegat"cinder voluem ? "15:13
yanndegatephemeral ?15:13
dmitrymeaha, got your question15:13
dmitrymeso, you have two options:15:13
dmitryme1. Use flavor which have 0 ephemeral and 0 swap. Cinder volumes should work with such flavor15:14
dmitryme2. Use any flavor, but only ephemeral drive will work there15:14
*** vrovachev has quit IRC15:22
yanndegattrying 115:23
*** skolekonov has joined #openstack-sahara15:27
dmitrymeBTW, I’ve made a fix for the issue with sahara CLI client you’ve bumped into yesterday: https://review.openstack.org/#/c/97255/15:28
yanndegatnice15:30
yanndegatthank you15:31
yanndegatanother error now15:31
yanndegat;)15:31
yanndegat2014-06-03 17:30:02.288 18823 TRACE sahara.service.api   File "/usr/lib/python2.6/site-packages/sahara/plugins/hdp/ambariplugin.py", line 57, in create_cluster15:31
yanndegat2014-06-03 17:30:02.288 18823 TRACE sahara.service.api     h.HadoopServer(host, cluster_spec.node_groups[host_role],15:31
yanndegat2014-06-03 17:30:02.288 18823 TRACE sahara.service.api KeyError: u'bigD-master'15:31
yanndegat2014-06-03 17:30:02.288 18823 TRACE sahara.service.api15:31
yanndegat2014-06-03 17:30:02.288 18823 ERROR sahara.service.api [-] Can't configure cluster 'BIGDATAMEC' (reason: u'bigD-master')15:31
yanndegat$15:31
dmitrymehmm, try using only lowercase letters and dashes in the name of cluster and node groups15:32
tmckaySergeyLukjanov, aignatov, is it the intention to make Hive work in the HDP plugin?  I assume yes. If so, I should add a blueprint for it (I don't know details but at least it's a starting point)15:43
*** IvanBerezovskiy has left #openstack-sahara15:49
yanndegatyeah15:53
yanndegatit works15:53
yanndegatin fact it deploys the cluster15:53
yanndegatconfigure hadoop15:53
yanndegatbut the slave is in error on hadoop15:53
*** alazarev has joined #openstack-sahara15:53
yanndegatwe'ill investigate why it is so15:53
yanndegatthanks a lot15:53
dmitrymeyanndegat: we had the same problem actually with HDP 2.015:55
dmitrymeHortonworks guys told us the problem is in the image15:55
yanndegatis it because15:56
yanndegatsome directories are missign ?15:56
yanndegat( ex: yarn ? )15:57
toskydmitryme: if I can jump in for an unrelated question; documentation says to create the endpoint with keystone create-endpoint [...] --internalurl "http://10.0.0.2:8386/v1.1/\$(tenant_id)s" <- why that instead of %(tenant_id)s as used by other projects?15:57
dmitrymewe didn’t dig it actually, they just gave another image which works15:57
tosky(asking you as you committed that change :)15:57
yanndegatok15:57
yanndegatand can i ask you a link to this image ?15:58
dmitrymeyanndegat: I don’t know why they didn’t updated the image in the docs, but I think it will be ok if we share it with you15:58
dmitrymegive us a moment, we’ll figure out how to share it15:59
yanndegatgreat15:59
dmitrymetosky: I don’t remember, I think I copy-pasted that from somewhere15:59
openstackgerritA change was merged to stackforge/sahara-ci-config: add cleanup after diskimage-integration tests  https://review.openstack.org/9721415:59
dmitrymetosky: I am pretty sure I copy-pasted it from our devstack scripts:16:00
dmitrymehttps://github.com/openstack-dev/devstack/blob/master/lib/sahara#L8316:00
toskydmitryme: thanks16:01
*** ViswaV_ has joined #openstack-sahara16:03
*** ViswaV has quit IRC16:06
dmitrymeyanndegat: we’ll try to send you a file via IRC16:11
dmitrymesreshetnyak will try, to be more specific16:11
dmitrymeyanndegat: do you see an offer in your client?16:14
dmitrymeIRC client16:14
openstackgerritAndrew Lazarev proposed a change to openstack/sahara: Migrated integration tests to testtools  https://review.openstack.org/9698116:23
openstackgerritAndrew Lazarev proposed a change to openstack/sahara: Migrated unit tests to testtools  https://review.openstack.org/9697916:23
*** IlyaE has quit IRC16:29
*** IlyaE has joined #openstack-sahara16:30
*** sreshetnyak has quit IRC16:33
*** sreshetnyak has joined #openstack-sahara16:33
*** ylobankov has quit IRC16:45
*** skolekonov has quit IRC16:50
*** julienvey has quit IRC17:01
*** jpich has quit IRC17:04
*** Timotey has quit IRC17:05
*** palar has joined #openstack-sahara17:06
*** bradd1 has quit IRC17:12
openstackgerritA change was merged to stackforge/sahara-ci-config: add stable/icehouse branch support  https://review.openstack.org/9746617:19
*** shakamunyi has quit IRC17:24
*** alazarev has quit IRC18:16
openstackgerritMichael McCune proposed a change to openstack/sahara: Adding disconnected mode fixes to hdp plugin  https://review.openstack.org/9446018:25
*** alazarev has joined #openstack-sahara18:31
*** IlyaE has quit IRC18:44
*** themistymay has joined #openstack-sahara19:04
*** IlyaE has joined #openstack-sahara19:10
*** IlyaE has quit IRC19:48
*** themistymay has quit IRC19:51
*** sballe has joined #openstack-sahara19:53
*** julienvey has joined #openstack-sahara19:59
*** palar_ has joined #openstack-sahara20:10
*** palar has quit IRC20:13
*** palar_ has quit IRC20:14
*** discretestates has joined #openstack-sahara20:27
*** miqui has quit IRC20:34
*** sballe has quit IRC20:35
*** miqui has joined #openstack-sahara20:36
*** miqui has quit IRC20:38
*** IlyaE has joined #openstack-sahara20:52
*** discretestates has quit IRC21:08
*** crobertsrh is now known as _crobertsrh21:15
*** IlyaE has quit IRC21:30
*** witlessb has quit IRC21:34
openstackgerritAndrew Lazarev proposed a change to openstack/sahara: Minor EDP refactoring  https://review.openstack.org/9763321:40
*** miqui has joined #openstack-sahara21:54
*** ViswaV_ has quit IRC22:05
*** IlyaE has joined #openstack-sahara22:07
openstackgerritAndrew Lazarev proposed a change to openstack/sahara: Changing job excecution status to 'FAILED' in case of exception  https://review.openstack.org/9763722:14
*** ViswaV has joined #openstack-sahara22:21
*** IlyaE has quit IRC22:23
*** IlyaE has joined #openstack-sahara22:24
*** tosky has quit IRC22:38
*** elmiko is now known as _elmiko22:39
*** tmckay has quit IRC22:48
*** julienvey has quit IRC22:59
openstackgerritAndrew Lazarev proposed a change to openstack/python-saharaclient: Don't set json content type for non-json data  https://review.openstack.org/9764223:01
*** ViswaV has quit IRC23:14
*** zhiyan_ is now known as zhiyan23:17
*** zhiyan is now known as zhiyan_23:18
*** ruhe has quit IRC23:20
*** ruhe2 has joined #openstack-sahara23:20
*** ruhe2 is now known as ruhe23:20
*** IlyaE has quit IRC23:34
*** ViswaV has joined #openstack-sahara23:36
*** IlyaE has joined #openstack-sahara23:38
*** IlyaE has quit IRC23:41
« Monday, 2014-06-02 Index Wednesday, 2014-06-04 »

Generated by irclog2html.py 2.14.0 by Marius Gedminas - find it at mg.pov.lt!