Friday, 2018-07-13

*** Bhujay has joined #openstack-sahara02:28
*** links has joined #openstack-sahara04:02
*** Bhujay has quit IRC06:28
*** Bhujay has joined #openstack-sahara06:29
*** Bhujay has quit IRC06:52
*** Bhujay has joined #openstack-sahara06:56
*** tesseract has joined #openstack-sahara07:12
*** masber has joined #openstack-sahara07:17
*** Bhujay has quit IRC07:29
*** Bhujay has joined #openstack-sahara07:29
*** Bhujay has quit IRC07:47
*** Bhujay has joined #openstack-sahara07:48
*** tosky has joined #openstack-sahara08:06
*** rcernin has quit IRC08:46
openstackgerritJean-Philippe Evrard proposed openstack/openstack-ansible-os_sahara master: [DNM] Test nova is behaving correctly  https://review.openstack.org/58247809:22
*** Bhujay has quit IRC09:50
*** Bhujay has joined #openstack-sahara09:51
openstackgerritLuigi Toscano proposed openstack/python-saharaclient master: Revert "Fix the cover tox target (switch to stestr)"  https://review.openstack.org/58251011:36
openstackgerritLuigi Toscano proposed openstack/python-saharaclient master: Fix the cover tox target (switch to stestr)  https://review.openstack.org/58251111:41
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/sahara master: Adding Ambari 2.6 to image pack  https://review.openstack.org/57902411:44
openstackgerritJeremy Freudberg proposed openstack/python-saharaclient master: Correct a missed job_execution->job  https://review.openstack.org/58251912:19
tellesnobregatosky, passed that stage on spark, now it is stuck checking if datanode is running12:31
toskyand is it really running?12:32
tellesnobregayes12:34
tellesnobregathe command returns connection refused12:34
tellesnobregahttp://paste.openstack.org/show/725820/12:35
toskya hostname issue maybe?12:36
tellesnobregaprobably not12:37
tellesnobrega/etc/hosts is configured correctly12:38
toskyso you can see both spark-spark-slave-0 and spark-spark-master-0, with and without the .novalocal suffix?12:42
toskyis the 8020 port on master open for all sources?12:43
toskyis the 8020 port open on netfilter/iptables?12:43
*** hoonetorg has quit IRC12:43
tellesnobregafor the first question12:44
tellesnobregayes, and I can ssh to it12:44
toskymore telnet or nc12:44
toskyoh, that's the first question, not about the port12:44
tellesnobrega8020 is open on security groups12:45
tellesnobregaIngressIPv4TCP80200.0.0.0/012:45
tellesnobregaIngressIPv6TCP8020::/012:46
toskyand locally on the master host?12:50
toskyfro mnetstat (or ss)12:50
toskyss -ntp12:51
tellesnobregahttp://paste.openstack.org/show/725823/12:51
tellesnobregass -ntp ^12:51
tellesnobregaiptables -L12:52
tellesnobregahttp://paste.openstack.org/show/725824/12:52
toskyI'm still learning ss (now that netstat is deprecated), so I think that the correct form to show listening ports should be12:54
toskyss -lutp12:55
toskyor something like that12:55
*** Bhujay has quit IRC12:55
*** Bhujay has joined #openstack-sahara12:55
tellesnobregahttp://paste.openstack.org/show/725825/12:56
tellesnobregadon't think it is open12:56
toskyuhm12:57
toskyor I still don't know ss properly12:57
tellesnobregass -l12:58
tellesnobregahttp://paste.openstack.org/show/725826/12:59
*** links has quit IRC12:59
*** Bhujay has quit IRC13:00
*** Bhujay has joined #openstack-sahara13:00
toskytellesnobrega: so I guess that if you run that same command hdfs dfsadmin etc on master directly, it would fail too?13:00
tellesnobregait does13:01
*** hoonetorg has joined #openstack-sahara13:01
tellesnobregaany ideas?13:03
toskyuh, check if the service did not start13:03
toskyrecheck the logs when it was supposed to start13:04
tellesnobregatosky, looks like you were right13:06
tellesnobregaSHUTDOWN_MSG: Shutting down NameNode at java.net.UnknownHostException: spark-spark-master-0: spark-spark-master-0: Name or service not known13:06
openstackgerritLuigi Toscano proposed openstack/openstack-ansible-os_sahara master: Add few more services to the inventory (nova, memcached)  https://review.openstack.org/58253813:19
openstackgerritLuigi Toscano proposed openstack/sahara master: api-ref: move to a v1.1 sub-folder  https://review.openstack.org/58254013:20
tellesnobregatosky, it is related to name.novalocal stuff13:57
tellesnobregabut even weirder is, if I stop the service and start it again, it works14:02
tellesnobregano need for other changes14:02
toskytellesnobrega: uhm14:07
toskyso maybe something is configured later?14:07
toskynot yet configured when the services is started first?14:08
toskyoh, the review which removes the legacy jobs for sahara-extra was merged14:08
toskyso we need this, or we won't be able to publish new changes in sahara-extra: https://review.openstack.org/#/c/532691/14:09
toskyit should work, but we will see only when it merges14:09
toskyas post job, it can't be tested before merging14:09
tellesnobregamerging now14:10
toskythanks14:10
toskywe have just one legacy job remaining, the integration test forsahara-dashboard; I have a WIP for that14:10
tellesnobreganice14:11
tellesnobregatried spark 1.6.0, config /etc/hosts looks the same14:13
openstackgerritMerged openstack/python-saharaclient master: Revert "Fix the cover tox target (switch to stestr)"  https://review.openstack.org/58251014:14
tellesnobregaso, when spark 1.6.0 starts the logs are named hadoop-hdfs-datanode-spark160-spark-slave160-0.novalocal.log14:18
tellesnobregawhile the 2.2 is named hadoop-hdfs-datanode-spark22-spark-slave22-0.log14:18
toskyand the content of /etc/hosts? Is it the same for both?14:19
tellesnobregawhen I stop and start again, 2.2 is named the same and it works14:19
tellesnobregayes14:19
tellesnobregafor some reason the first run on 2.2 is getting the wrong naming14:19
openstackgerritJeremy Freudberg proposed openstack/sahara-dashboard master: APIv2 in dashboard  https://review.openstack.org/58187814:20
tellesnobregatosky, maybe, just maybe, I need to stop the services after install14:24
tellesnobregathat might the problem14:24
toskyand why isn't it the case with 1.6?14:25
tellesnobregadifferent image gen process14:25
tellesnobregaI might have missed something14:25
toskyso maybe there is something else different14:25
toskyyep14:25
toskymaybe the service is disabled when installed with sie14:25
tellesnobregathat is what I'm thinking as well14:25
tellesnobregaI will try to run a stop_services script14:26
tellesnobregathere is turn-off services14:26
tellesnobregaI missed it14:26
*** jeremyfreudberg has joined #openstack-sahara14:29
jeremyfreudbergtellesnobrega, i ran out of time for decomission specific node14:29
tellesnobregajeremyfreudberg, dashboard part?14:29
jeremyfreudbergyes, dashboard14:29
tellesnobregano worries, I will give it a try14:30
jeremyfreudbergthanks14:30
tellesnobregadashboard is not my strong suit14:30
tellesnobregabut I will try14:30
jeremyfreudbergyeah, it can get rough quickly14:30
jeremyfreudberganyway -- my work is just about done. review it, merge it, i'll be around to respond to feedback but probably can't push any new patchsets14:32
tellesnobregasounds good14:32
tellesnobregaif any urgent stuff come up I can write a follow up patch14:32
jeremyfreudbergremember to do a saharaclient release when enough (all?) of the good stuff is merged, and  bump the sahara-dashboard requirements14:32
jeremyfreudbergyep14:33
jeremyfreudbergi will still be keeping an eye on things14:33
tellesnobregathanks14:33
jeremyfreudbergand have plenty of reminders and explanatinos for the confusing stuff14:33
tellesnobregasounds good14:34
jeremyfreudbergemail or review comments if you need anything14:37
jeremyfreudbergbye14:37
*** jeremyfreudberg has quit IRC14:37
openstackgerritMerged openstack/sahara master: Adding Storm 1.2.0 and 1.2.1  https://review.openstack.org/57781514:39
openstackgerritLuigi Toscano proposed openstack/openstack-ansible-os_sahara master: Add few more services to the inventory (nova, memcached)  https://review.openstack.org/58253814:42
toskyuh, thanks Jeremy14:42
openstackgerritMerged openstack/sahara-extra master: Add native Zuul v3 build and publishing jobs  https://review.openstack.org/53269114:44
*** tesseract has quit IRC15:21
*** Bhujay has quit IRC15:40
openstackgerritLuigi Toscano proposed openstack/openstack-ansible-os_sahara master: Add few more services to the inventory (nova, memcached)  https://review.openstack.org/58253816:08
openstackgerritJeremy Freudberg proposed openstack/python-saharaclient master: Clean S3 data source create/update  https://review.openstack.org/58264317:08
*** Bhujay has joined #openstack-sahara17:54
*** Bhujay has quit IRC17:59
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/python-saharaclient master: Adding boot from volume  https://review.openstack.org/57253619:34
openstackgerritTelles Mota Vidal Nóbrega proposed openstack/python-saharaclient master: Adding boot from volume to osc  https://review.openstack.org/58231919:40
openstackgerritLuigi Toscano proposed openstack/sahara-extra master: Fix the post-publishing job: correct src directory  https://review.openstack.org/58267619:53
toskyblindly hoping that it would work ^^19:54
openstackgerritMerged openstack/python-saharaclient master: Correct a missed job_execution->job  https://review.openstack.org/58251920:41
-openstackstatus- NOTICE: logs.openstack.org is offline, causing POST_FAILURE results from Zuul. Cause and resolution timeframe currently unknown.21:51
*** ChanServ changes topic to "logs.openstack.org is offline, causing POST_FAILURE results from Zuul. Cause and resolution timeframe currently unknown."21:51
*** ChanServ changes topic to "OpenStack Sahara // IRC Meetings - http://eavesdrop.openstack.org/#OpenStack_Data_Processing_(Sahara)_Team_Meeting"23:36
-openstackstatus- NOTICE: logs.openstack.org is back on-line. Changes with "POST_FAILURE" job results should be rechecked.23:36
*** tosky has quit IRC23:41

Generated by irclog2html.py 2.15.3 by Marius Gedminas - find it at mg.pov.lt!