Thursday, 2018-03-15

« Wednesday, 2018-03-14 Index Friday, 2018-03-16 »
*** diablo_rojo has quit IRC00:28
openstackgerritOpenStack Proposal Bot proposed openstack/sahara-dashboard master: Updated from global requirements  https://review.openstack.org/55310900:31
*** diablo_rojo has joined #openstack-sahara00:31
*** abalutoiu has quit IRC00:39
*** diablo_rojo has quit IRC00:47
*** dave-mccowan has joined #openstack-sahara01:05
*** shuyingya has joined #openstack-sahara01:27
*** shuyingya has quit IRC01:27
*** shuyingya has joined #openstack-sahara01:27
*** dave-mccowan has quit IRC03:14
*** links has joined #openstack-sahara03:41
*** pgadiya has joined #openstack-sahara05:11
*** gguss7P3CU7 has joined #openstack-sahara05:54
gguss7P3CU7THIS IS A FREENODE BREAKING NEWS ALERT!! Hitechcg AND opal ARE GOING AT IT RIGHT NOW WITH A LOT OF FIGHTING AND ARGUING WOW YOU DON'T WANT TO MISS THIS!! TYPE /JOIN ## TO SEE THE ACTION...AGAIN TYPE /JOIN ## TO SEE THE ACTION!!05:54
gguss7P3CU7pgadiya links shuyingya rcernin rickflare tellesnobrega hoonetorg caowei SergeyLukjanov openstackgerrit DuncanT zhuli openstack openstackstatus rodrigods zigo wlmbasson mnaser samueldmq knikolla Jeffrey4l ltosky[m] afazekas ruhe dmitryme brad[] elmiko NikitaKonovalov aignatov chason vkmc mordred05:54
*** gguss7P3CU7 has quit IRC05:54
*** esikache1 has joined #openstack-sahara06:11
openstackgerritzhongshengping proposed openstack/puppet-sahara master: Switch acceptance test to use integration classes  https://review.openstack.org/55315706:27
*** esikache1 has quit IRC07:05
*** esikache1 has joined #openstack-sahara07:08
*** esikache1 has quit IRC07:29
openstackgerritOpenStack Proposal Bot proposed openstack/openstack-ansible-os_sahara master: Updated from global requirements  https://review.openstack.org/55222407:38
*** pcaruana has joined #openstack-sahara07:48
*** pcaruana has quit IRC07:50
*** pcaruana has joined #openstack-sahara07:50
*** pcaruana is now known as pcaruana|wfh|07:53
openstackgerritOpenStack Proposal Bot proposed openstack/sahara master: Updated from global requirements  https://review.openstack.org/55326808:08
openstackgerritOpenStack Proposal Bot proposed openstack/sahara-dashboard master: Updated from global requirements  https://review.openstack.org/55310908:08
*** tesseract has joined #openstack-sahara08:24
*** tosky has joined #openstack-sahara08:40
*** rcernin has quit IRC09:00
*** shuyingy_ has joined #openstack-sahara09:05
*** shuyingya has quit IRC09:08
openstackgerritMerged openstack/sahara-dashboard master: Updated from global requirements  https://review.openstack.org/55310909:08
openstackgerritLuigi Toscano proposed openstack/sahara-tests master: Update the doc generation (tox, remove warnings PTI)  https://review.openstack.org/55301109:58
*** caowei_ has joined #openstack-sahara11:19
*** mnaser_ has joined #openstack-sahara11:21
*** rodrigod` has joined #openstack-sahara11:23
*** pcaruana|wfh| has quit IRC11:27
*** caowei has quit IRC11:27
*** rodrigods has quit IRC11:27
*** mnaser has quit IRC11:27
*** caowei_ is now known as caowei11:27
*** mnaser_ is now known as mnaser11:27
*** SergeyLukjanov2 has joined #openstack-sahara11:28
*** pcaruana|wfh| has joined #openstack-sahara11:28
*** SergeyLukjanov has quit IRC11:28
*** mnaser has quit IRC11:28
*** mnaser has joined #openstack-sahara11:28
*** SergeyLukjanov2 is now known as SergeyLukjanov11:28
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara stable/newton: Add retries to apt cache update  https://review.openstack.org/55335511:33
*** pgadiya has quit IRC11:51
*** pcaruana|wfh| is now known as pcaruana12:14
tellesnobregashuyingy_, can you review https://review.openstack.org/#/c/550819/?12:42
toskyI suspect that the POST_FAILURE issue is not fixed yet12:45
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara-specs master: Adding Rocky folder to sahara-specs  https://review.openstack.org/55338212:53
openstackgerritMerged openstack/openstack-ansible-os_sahara stable/newton: Add retries to apt cache update  https://review.openstack.org/55335512:53
tellesnobregaok12:54
*** dave-mccowan has joined #openstack-sahara13:01
*** knylander has joined #openstack-sahara13:03
*** shuyingy_ has quit IRC13:03
*** shuyingya has joined #openstack-sahara13:03
*** shuyingya has quit IRC13:08
openstackgerritMerged openstack/sahara-specs master: Adding Rocky folder to sahara-specs  https://review.openstack.org/55338213:30
*** ukaynar has joined #openstack-sahara13:31
*** dave-mccowan has quit IRC13:34
openstackgerritMerged openstack/openstack-ansible-os_sahara master: Updated from global requirements  https://review.openstack.org/55222413:40
*** links has quit IRC13:42
*** shuyingya has joined #openstack-sahara13:46
*** jeremyfreudberg has joined #openstack-sahara13:52
*** esikache1 has joined #openstack-sahara14:02
*** shuyingy_ has joined #openstack-sahara14:07
openstackgerritMerged openstack/sahara-image-elements master: Preload soci-mysql and soci on RHEL7 images  https://review.openstack.org/55081914:09
*** shuyingya has quit IRC14:10
-openstackstatus- NOTICE: POST_FAILURE results on Tempest-based jobs since the most recent Zuul security fixes are being investigated; rechecking those won't help for now but we'll keep you posted once a solution is identified14:10
*** ChanServ changes topic to "POST_FAILURE results on Tempest-based jobs since the most recent Zuul security fixes are being investigated; rechecking those won't help for now but we'll keep you posted once a solution is identified"14:10
*** esikache1 has quit IRC14:39
openstackgerritMerged openstack/sahara-image-elements master: Add support to build 2.8.2 image  https://review.openstack.org/54218414:40
*** esikache1 has joined #openstack-sahara14:40
*** jeremyfreudberg has quit IRC14:41
tellesnobregashuyingy_, please propose a backport for your patch to queens14:41
shuyingy_ok14:42
*** esikache1 has quit IRC14:44
openstackgerritNguyen Hai proposed openstack/python-saharaclient master: Fix Local test fail with pypy  https://review.openstack.org/55341914:45
*** esikache1 has joined #openstack-sahara14:57
openstackgerritNguyen Hai proposed openstack/python-saharaclient master: Fix local test fails with pypy  https://review.openstack.org/55341914:59
*** esikache1 has quit IRC15:03
*** esikache1 has joined #openstack-sahara15:06
*** esikache1 has quit IRC15:11
*** esikache1 has joined #openstack-sahara15:25
*** esikache1 has quit IRC15:29
*** dave-mccowan has joined #openstack-sahara15:36
*** esikache1 has joined #openstack-sahara15:51
*** esikache1 has quit IRC15:56
*** tosky has quit IRC15:57
openstackgerritNguyen Hai proposed openstack/python-saharaclient master: Remove pypy from envlist in tox.ini  https://review.openstack.org/55341915:59
*** esikache1 has joined #openstack-sahara16:06
*** esikache1 has quit IRC16:10
*** esikache1 has joined #openstack-sahara16:13
*** esikache1 has quit IRC16:20
*** esikache1 has joined #openstack-sahara16:24
*** esikache1 has quit IRC16:28
*** masber has joined #openstack-sahara16:28
*** esikache1 has joined #openstack-sahara16:34
*** tosky has joined #openstack-sahara16:39
*** masber has quit IRC16:53
*** shuyingy_ has quit IRC16:57
*** knylander has quit IRC17:03
*** knylander has joined #openstack-sahara17:03
*** tosky has quit IRC17:09
*** knylander has quit IRC17:13
*** knylander has joined #openstack-sahara17:14
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara-specs master: Spec for splitting plugins code from Sahara core  https://review.openstack.org/55350417:39
openstackgerritJesse Pretorius (odyssey4me) proposed openstack/openstack-ansible-os_sahara master: tox.ini: Expose USER environment variable to execution environment  https://review.openstack.org/55355218:01
*** rtsisyk has joined #openstack-sahara18:16
rtsisykhi!18:18
ltosky[m]hi18:26
rtsisykI need help with Sahara :) I'm trying to use Sahara 6.0.3 with Ocata installed from RPM (RDO). I created HDP node templates for Hadoop master node, Hadoop worker node and cluster template with 1 x master and 1 x worker node. Heat sucessfully starts both instances and I can login to them ussing my ssh key, but Sahara cluster stalls in "Waiting" state. I checked Cluster -> MyCluster-> Cluster Events -> "Wait for instance accessibility" and r18:36
rtsisykCluster Status: https://cloud.mail.ru/public/5hD7/mYSUCM83B18:36
rtsisykinstances https://cloud.mail.ru/public/F9BB/a1iq2eTa918:37
rtsisykmaster && worker logs https://cloud.mail.ru/public/Fp8R/CrHQKjyMd https://cloud.mail.ru/public/2XdC/gVVgcD45v18:37
rtsisykheat log https://cloud.mail.ru/public/2XdC/gVVgcD45v, sahara log https://cloud.mail.ru/public/4nGD/gueyqQi3C18:38
rtsisykmy template: https://cloud.mail.ru/public/Bdx2/nejNPVkBm18:38
*** tesseract has quit IRC18:39
ltosky[m]I can't properly check the logs now (boarding a plane in few minutes) but ensure that you set the floating ip pool, or, if not, that the instances can be reached by the controller through the defined network18:40
ltosky[m]Others will probably read and answer later18:40
rtsisykI see that primary problem is that "Worker" node is not detected as started.18:40
ltosky[m]same suggestion :)18:41
rtsisykI have "No floating pool" in both templates18:41
rtsisyk"Don't assign floating IPs"18:42
rtsisykdo you mean that sahara-engine daemon should have access to my nodes via internal IP addresses?18:43
ltosky[m]the management ip is different (different class) for the master and the worker - is that expected?18:43
rtsisykI assigned floating IP to master node manually18:43
ltosky[m]yes18:43
ltosky[m]Or you need to use a node as proxy18:43
ltosky[m]either all nodes are reachable through the internal network, or at least one node should have a floating ip (not manually assigned) and should be set as proxy (there is an option in sahara.conf for that)18:45
rtsisykI used internal network for nodes. Internal network is not routable from sahara-engine host. It hidden somewhere inside neurton/vswitch18:45
tellesnobregartsisyk, basically you will have to setup a proxy node18:46
ltosky[m]you need to implement one of the solutions above (if it's not possible to  set floating ips for all nodes)18:46
tellesnobregaif you can't have them accessing with internal and only have floating ips on one node18:46
* ltosky[m] leaves that to tellesnobrega for now :)18:46
rtsisykltosky[m]: thanks for the valuable advice :)18:47
rtsisykI'll try to enable floating ips for all nodes18:47
rtsisykI wonder where logs of SSH agent can be found?18:48
tellesnobregartsisyk, all sahara logs are /var/logs/sahara18:49
tellesnobreganot sure if we have a specific one for ssh agent18:49
rtsisykI don't see any traces for SSH checks neigher in /var/log/sahara/sahara-engine.log nor /var/log/sahara/sahara-api.log18:50
rtsisykanyway, let's check the idea with floating IPs18:52
rtsisykI also see "Proxy gateway" check box in template18:53
tellesnobregathat one is to setup one instance that will link sahara to instances18:54
rtsisykyeah, I understand18:56
rtsisykBTW, what is "Autoconfiguration strategy: NEVEL_APPLY, ALWAYS_APPLY, ONLY_STACK_DEFAULTS_APPLY"?18:57
tellesnobregais that for templates?19:00
tellesnobregathis one? https://docs.openstack.org/sahara/ocata/userdoc/configs_recommendations.html19:01
rtsisykwow, it started to configure instances19:05
rtsisykfloating IPs resolved my first problem19:05
rtsisyknow I have another one19:05
rtsisykRemoteCommandException: Error during command execution: "rngd -r /dev/urandom -W 4096" Return code: 127 STDERR: bash: rngd: command not found Error ID: def8e430-ce39-468f-8db6-bb02b80f99d8 Error ID:19:06
ltosky[m]I wonder how it's not installed19:07
rtsisykI used http://cloud.centos.org/centos/7/images/CentOS-7-x86_64-GenericCloud.qcow2 image19:08
ltosky[m]there is some code for it19:08
rtsisyklet me check rngd on instance19:08
rtsisykoops, I forgot to enable my ssh keys for Sahara instances :(19:09
rtsisykone more iteration19:09
* ltosky[m] back to its boarding o/19:12
ltosky[m]You need a prebuild image - check sahara-image-elements19:12
ltosky[m]but I think that it's documented19:12
*** ukaynar has quit IRC19:12
ltosky[m]you need to build the image beforehand19:13
ltosky[m]o/19:13
rtsisykaccording to documentation HortonWorks plugin (a.k.a. Ambari) should use bare OS image19:16
rtsisykok, rngd actually is not installed19:18
rtsisykI need that same image plus `yum install -y rng-tools`19:18
*** ukaynar has joined #openstack-sahara19:22
rtsisykI also noticed that sahara-image-build uses /tmp/ directory for preitermediate image files. It can be problem when /tmp is tmpfs.19:22
*** esikache1 has quit IRC19:25
*** ChanServ changes topic to "OpenStack Sahara // IRC Meetings - http://eavesdrop.openstack.org/#OpenStack_Data_Processing_(Sahara)_Team_Meeting"19:26
-openstackstatus- NOTICE: The regression stemming from one of yesterday's Zuul security fixes has been rectified, and Devstack/Tempest jobs which encountered POST_FAILURE results over the past 24 hours can safely be rechecked now19:26
rtsisykis there any maillist related to sahara I can subscribe to?19:36
tellesnobrega rtsisyk we are at openstack-dev19:43
tellesnobregamost stuff we just talk there19:43
tellesnobregawith a tag [sahara]19:44
tellesnobregartsisyk, about the usage of bare os image, it does say it works, I'm not sure it does, best course of action is to get an image ready19:44
tellesnobregawe have some hosted images here http://sahara-files.mirantis.com/images/upstream/ocata/19:45
rtsisykI generated ambari image and now my cluster on "Set up Ambari management console" stage19:52
*** ukaynar has quit IRC19:53
rtsisykI see that it calls "service start ambari-console" on master node.19:53
*** dave-mccowan has quit IRC19:53
tellesnobregartsisyk, awesome19:54
*** esikache1 has joined #openstack-sahara19:54
rtsisykwow, "Completed Successfully"19:54
tellesnobrega\o/19:54
*** knylander has quit IRC19:58
rtsisykrngd consumes 100% CPU19:59
*** esikache1 has quit IRC19:59
tellesnobregawhat is the size of the instances you are using?19:59
rtsisyk2 VCPU of Skylake, 8GB RAM20:03
rtsisykit's a dev installation of openstack, I have limited resources20:04
rtsisykhttps://rhelblog.redhat.com/2015/03/09/red-hat-enterprise-linux-virtual-machines-access-to-random-numbers-made-easy/#more-73820:04
rtsisykprobably I need to play with virtio-rnd20:04
*** dave-mccowan has joined #openstack-sahara20:11
rtsisykWait registration of hosts: 'Operation' timed out after 600 second(s) Error ID: 17eb0af9-0959-49eb-9951-6d296d0500a920:12
*** esikache1 has joined #openstack-sahara20:12
rtsisykhttps://cloclo17.datacloudmail.ru/weblink/view/56YK/ekdz5o93E?etag=7360739E1DB9D1CB49656F2835380D0F3EA19984&key=0863aaf268c09777ed63981fc46424cc19f933ec20:13
rtsisykprobably the last step20:13
rtsisykI think this problem is caused by virtio-rnd.20:16
*** esikache1 has quit IRC20:17
*** esikache1 has joined #openstack-sahara20:37
rtsisykNetUtil.py:88 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)20:42
rtsisykI saw similar problem on my Debian. It probably caused by newer version of openssl + pycrypto.20:42
rtsisykWARNING 2018-03-15 20:31:46,984 NetUtil.py:116 - Server at https://test-hadoop-hdp-master-0.novalocal:8440 is not reachable, sleeping for 10 seconds...20:43
rtsisykINFO 2018-03-15 20:31:56,993 NetUtil.py:62 - Connecting to https://test-hadoop-hdp-master-0.novalocal:8440/ca20:43
rtsisykERROR 2018-03-15 20:31:57,110 NetUtil.py:88 - [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:579)20:43
rtsisykERROR 2018-03-15 20:31:57,110 NetUtil.py:89 - SSLError: Failed to connect. Please check openssl library versions.20:43
rtsisykRefer to: https://bugzilla.redhat.com/show_bug.cgi?id=1022468 for more details.20:43
openstackrtsisyk: Error: Error getting bugzilla.redhat.com bug #1022468: NotPermitted20:43
rtsisyk1022468 is not public20:43
rtsisykyeah, I can't get #1022468 too (20:45
*** esikache1 has quit IRC20:45
tellesnobregartsisyk, sorry I was out20:46
rtsisykSSL certificate is self-signed20:47
tellesnobregathe timeout is something that we've dealing a while with ambari, it takes a long time and if network is not too good it can fail20:47
tellesnobregaSSL issue is a whole other problem20:47
rtsisyktellesnobrega: it's SSL issue20:47
tellesnobregaI think ltosky[m] has a "solution" for that20:47
tellesnobregaI think you can disable the check20:47
rtsisykpycrypto doesn't like self-signed certificate20:48
tellesnobregathat was the solution for now while we work on a real solution20:48
rtsisykhow to disable this check?20:48
tellesnobregalet me check20:49
*** esikache1 has joined #openstack-sahara20:50
tellesnobregaso it is disabled for centos if you generate a new image20:50
tellesnobregawhat image are you using?20:50
tellesnobregartsisyk, https://review.openstack.org/#/c/544622/120:50
rtsisyk/usr/bin/sahara-image-create -i centos7 -p ambari20:51
rtsisykFeb 14, 2018, wow20:51
rtsisykI'll try to get sahara-image-create from git and re-generate image20:51
rtsisykthanks for help, I hope it's the last issue :)20:52
*** dave-mccowan has quit IRC20:53
tellesnobregartsisyk, hopefully yes20:53
tellesnobregaif you need more help and no one responds here just send out an email to ML openstack-dev and mark sahara20:54
*** esikache1 has quit IRC20:58
*** esikache1 has joined #openstack-sahara21:23
*** esikache1 has quit IRC21:28
*** dave-mccowan has joined #openstack-sahara21:39
*** esikache1 has joined #openstack-sahara21:58
*** rcernin has joined #openstack-sahara22:02
*** esikache1 has quit IRC22:02
*** masber has joined #openstack-sahara22:09
*** tosky has joined #openstack-sahara22:13
toskythe rngd thing is something that we should check, somehow: it's complicated because it depends on whether the instance is a VM (no rngd, it should be configured on the host) or baremetal (yes rngd)22:20
toskyI should probably create a story for that22:21
*** esikache1 has joined #openstack-sahara22:47
*** dave-mccowan has quit IRC22:48
*** esikache1 has quit IRC22:51
*** masber has quit IRC23:02
*** tellesnobrega has quit IRC23:11
*** esikache1 has joined #openstack-sahara23:14
*** esikache1 has quit IRC23:20
*** esikache1 has joined #openstack-sahara23:22
*** esikache1 has quit IRC23:26
*** esikache1 has joined #openstack-sahara23:29
*** esikache1 has quit IRC23:33
*** esikache1 has joined #openstack-sahara23:37
*** rcernin has quit IRC23:39
*** esikache1 has quit IRC23:42
*** tellesnobrega has joined #openstack-sahara23:44
*** esikache1 has joined #openstack-sahara23:44
*** esikache1 has quit IRC23:49
« Wednesday, 2018-03-14 Index Friday, 2018-03-16 »

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!