フィルターのクリア

Info

この質問は閉じられています。 編集または回答するには再度開いてください。

How to solve "The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information."

1 回表示 (過去 30 日間)
Jingyu Ru
Jingyu Ru 2015 年 8 月 15 日
閉鎖済み: Jingyu Ru 2015 年 8 月 16 日
Hadoop version 1.2.1 Matlab version 2015a
Linux ubuntu 14.
At first, I try to run them at a cluster (one master and two slaves). I can run the example 'wordcount'in Hadoop. And it is success to read the data from the HDFS through the Matlab.
But when I try to run the example in Matlab 'Run mapreduce on a Hadoop Cluster',I failed.
It shows that the Map 0% and Reduce 0%.
Error using mapreduce (line 100)
The HADOOP job failed to complete. Check the HADOOP log files for job 1 for more information.
Error in run_mapreduce_on_a_hadoop (line 24)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,...
There is my Matlab codes
setenv('HADOOP_HOME','/usr/local/hadoop');
cluster = parallel.cluster.Hadoop;
cluster.HadoopProperties('mapred.job.tracker') = 'ubuntu:50031';
cluster.HadoopProperties('fs.default.name') = 'hdfs://ubuntu:8020';
outputFolder = '/home/rjy/logs/hadooplog';
mr = mapreducer(cluster);
ds = datastore('airlinesmall.csv','TreatAsMissing','NA',... 'SelectedVariableNames','ArrDelay','ReadSize',1000);
preview(ds)
meanDelay = mapreduce(ds,@meanArrivalDelayMapper,@meanArrivalDelayReducer,mr,... 'OutputFolder',outputFolder)
I don't know what it means in this logs as follows,
2015-08-14 22:41:40,981 INFO org.apache.hadoop.mapred.JobInProgress: Choosing a failed task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP)
'attempt_201508142238_0001_m_000000_3' to tip task_201508142238_0001_m_000000, for tracker
'tracker_slave1:localhost/127.0.0.1:41664'
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobInProgress: Choosing rack-local task
task_201508142238_0001_m_000000
2015-08-14 22:41:40,982 INFO org.apache.hadoop.mapred.JobTracker: Removing task 'attempt_201508142238_0001_m_000000_2'
2015-08-14 22:41:42,855 INFO org.apache.hadoop.mapred.TaskInProgress: Error from attempt_201508142238_0001_m_000000_3:
com.mathworks.toolbox.parallel.hadoop.MatlabStartupFailureException: parallel:mapreduce:HadoopTaskCouldNotFindMatlab
Caused by: java.lang.ClassNotFoundException: com.mathworks.toolbox.distcomp.util.Version
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
How to solve this problem? Thanks a lot.
I attach my Hadoop logs and Matlab logs.

回答 (0 件)

この質問は閉じられています。

製品

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by