hadoop 运行mapreduce,发现任务状态一直是Accepted

运行方法:

 ./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar terasort /home/suhuadong/teraga_data /home/suhuadong/terasort_data

[hadoop@su1 hadoop-2.6.0]$ ./bin/hadoop jar ./share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar teragen -Dmapred.map.tasks=2 100 /home/suhuadong/teraga_data
15/06/24 15:27:13 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
15/06/24 15:27:15 INFO client.RMProxy: Connecting to ResourceManager at su1/10.116.107.179:8032
15/06/24 15:27:15 INFO terasort.TeraSort: Generating 100 using 2
15/06/24 15:27:16 INFO mapreduce.JobSubmitter: number of splits:2
15/06/24 15:27:16 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
15/06/24 15:27:16 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1432603448423_0008
15/06/24 15:27:16 INFO impl.YarnClientImpl: Submitted application application_1432603448423_0008
15/06/24 15:27:16 INFO mapreduce.Job: The url to track the job: http://su1:8088/proxy/applicat ... 0008/
15/06/24 15:27:16 INFO mapreduce.Job: Running job: job_1432603448423_0008
777777777777777.png

 

VanquisherCsn - 一句话介绍

赞同来自: fish

http://docs.hortonworks.com/HD ... .html 可以参考这个。。

时间不再回头 - 一句话介绍

赞同来自:

到目前为止任务状态还是ACCEPTED

zp0824 - 好好学习,天天向上

赞同来自:

看过nm日志没?

时间不再回头 - 一句话介绍

赞同来自:

resourcemanager日志如下:  23:31:38 画云 2015/6/24 23:31:38 2015-06-24 15:27:15,488 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Allocated new applicationId: 8 2015-06-24 15:27:16,637 INFO org.apache.hadoop.yarn.server.resourcemanager.ClientRMService: Application with id 8 submitted by user hadoop 2015-06-24 15:27:16,637 INFO org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=hadoop   IP=10.116.107.179       OPERATION=Submit Application Request    TARGET=ClientRMService  RESULT=SUCCESS  APPID=application_1432603448423_0008 2015-06-24 15:27:16,637 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: Storing application with id application_1432603448423_0008 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1432603448423_0008 State change from NEW to NEW_SAVING 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore: Storing info for app: application_1432603448423_0008 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1432603448423_0008 State change from NEW_SAVING to SUBMITTED 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler: Accepted application application_1432603448423_0008 from user: hadoop, in queue: default, currently num of applications: 3 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl: application_1432603448423_0008 State change from SUBMITTED to ACCEPTED 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.ApplicationMasterService: Registering app attempt : appattempt_1432603448423_0008_000001 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1432603448423_0008_000001 State change from NEW to SUBMITTED 2015-06-24 15:27:16,638 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler: Added Application Attempt appattempt_1432603448423_0008_000001 to scheduler from user: hadoop 2015-06-24 15:27:16,639 INFO org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAttemptImpl: appattempt_1432603448423_0008_000001 State change from SUBMITTED to SCHEDULED ~

时间不再回头 - 一句话介绍

赞同来自:

888888888888888.png
是不是我这里面配置的不对,调度器搞不到资源去执行任务?

时间不再回头 - 一句话介绍

赞同来自:

9999999999999999999999.png
 

时间不再回头 - 一句话介绍

赞同来自:

fairscheduler.xml       hadoop-policy.xml           httpfs-site.xml          log4j.properties      mapred-site.xml.template    yarn-env.sh [hadoop@su1 hadoop]$ cat fairscheduler.xml  <?xml version="1.0"?> <allocations>   <queue name="infrastructure">     <minResources>512 mb, 1 vcores </minResources>     <maxResources>1536 mb, 3 vcores </maxResources>     <maxRunningApps>200</maxRunningApps>     <minSharePreemptionTimeout>300</minSharePreemptionTimeout>     <weight>1.0</weight>     <aclSubmitApps>root,hadoop</aclSubmitApps>   </queue>    <queue name="tool">       <minResources>512 mb, 1 vcores</minResources>       <maxResources>768 mb, 3 vcores</maxResources>    </queue>    <queue name="sentiment">       <minResources>512 mb, 1 vcores</minResources>       <maxResources>768 mb, 3 vcores</maxResources>    </queue> </allocations>

时间不再回头 - 一句话介绍

赞同来自:

我将yarn.nodemanager.resource.memory-mb调大,调到2200就可以了,之前的是1024 我不知道原因为啥?

zp0824 - 好好学习,天天向上

赞同来自:

这不是个正确的方法。 首先得搞清楚yarn.nodemanager.resource.memory-mb的意义,以及他与yarn.app.mapreduce.am.resource.mb、mapreduce.map.memory.mb、mapreduce.reduce.memory.mb、yarn.scheduler.minimum-allocation-mb、yarn.scheduler.increment-allocation-mb、yarn.scheduler.maximum-allocation-mb这几个参数的关系。

zp0824 - 好好学习,天天向上

赞同来自:

这几个资源相关的配置,请参见直播课http://www.chinahadoop.cn/cour ... /5811

小熊BN

赞同来自:

在那哪个文件改的啊?  

小熊BN

赞同来自:

修改 所有的 yarn-site.xml  加上下面的配置   <property> <name>yarn.scheduler.minimum-allocation-mb</name> <value>2048</value> <discription>单个任务可申请最少内存,默认1024MB</discription> </property> <property> <name>yarn.nodemanager.resource.memory-mb</name> <value>2048</value> <discription>nodemanager默认内存大小,默认为8192MB(value单位为MB)</discription> </property> <property> <name>yarn.nodemanager.resource.cpu-vcores</name> <value>1</value> <discription>nodemanager cpu内核数</discription> </property>

要回复问题请先登录注册