扫表 map阶段有GC overhead 会是什么原因

只是其中一个map
2016-07-11 18:41:52,081 INFO [IPC Server handler 21 on 37513] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttemp
t attempt_1467290505674_1892_m_000027_2 is : 0.0
2016-07-11 18:41:56,508 INFO [IPC Server handler 29 on 37513] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttemp
t attempt_1467290505674_1892_m_000027_2 is : 0.0
2016-07-11 18:42:16,396 FATAL [IPC Server handler 16 on 37513] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1467290
505674_1892_m_000027_2 - exited : GC overhead limit exceeded
2016-07-11 18:42:16,396 INFO [IPC Server handler 16 on 37513] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report fro
m attempt_1467290505674_1892_m_000027_2: Error: GC overhead limit exceeded
2016-07-11 18:42:16,396 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics 
report from attempt_1467290505674_1892_m_000027_2: Error: GC overhead limit exceeded
2016-07-11 18:42:16,396 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1467
290505674_1892_m_000027_2 TaskAttempt Transitioned from RUNNING to FAIL_CONTAINER_CLEANUP
2016-07-11 18:42:16,397 INFO [ContainerLauncher #26] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the
 event EventType: CONTAINER_REMOTE_CLEANUP for container container_1467290505674_1892_01_000224 taskAttempt attempt_1467290505674_1892
_m_000027_2
2016-07-11 18:42:16,397 INFO [ContainerLauncher #26] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attemp
t_1467290505674_1892_m_000027_2
2016-07-11 18:42:16,402 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1467
290505674_1892_m_000027_2 TaskAttempt Transitioned from FAIL_CONTAINER_CLEANUP to FAIL_TASK_CLEANUP
2016-07-11 18:42:16,404 WARN [CommitterEvent Processor #3] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Could not delet
e hdfs://hy41:8020/user/hbase/oldmobiledeviecenum/_temporary/1/_temporary/attempt_1467290505674_1892_m_000027_2
2016-07-11 18:42:16,404 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1467
290505674_1892_m_000027_2 TaskAttempt Transitioned from FAIL_TASK_CLEANUP to FAILED
2016-07-11 18:42:17,758 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics 
report from attempt_1467290505674_1892_m_000027_2: Container killed by the ApplicationMaster.

张晓慧

赞同来自:

fullgc了,内存不够了。你可以搜搜yarn设置进程启动和container内存限制的参数,一直读没有写的只能检查你的代码逻辑了

要回复问题请先登录注册