Hadoop mapreduce 容器以非零退出代码 1 退出答案

【问题标题】：Hadoop mapreduce container exited with a non-zero exit code 1Hadoop mapreduce 容器以非零退出代码 1 退出
【发布时间】：2014-07-08 10:40:53
【问题描述】：

我正在尝试运行一些 hadoop 程序来提取 Ubuntu 中一些摘要的关键字。当我使用 Hadoop 运行程序时，出现以下错误。

 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable



INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
 INFO input.FileInputFormat: Total input paths to process : 1
 INFO mapreduce.JobSubmitter: number of splits:1
 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1404812840999_0001
 INFO impl.YarnClientImpl: Submitted application application_1404812840999_0001
 INFO mapreduce.Job: The url to track the job: http://shiva-VirtualBox:8088/proxy/application_1404812840999_0001/
 INFO mapreduce.Job: Running job: job_1404812840999_0001
 INFO mapreduce.Job: Job job_1404812840999_0001 running in uber mode : false
 INFO mapreduce.Job:  map 0% reduce 0%
 INFO mapreduce.Job: Job job_1404812840999_0001 failed with state FAILED due to: Application application_1404812840999_0001 failed 2 times due to AM Container for appattempt_1404812840999_0001_000002 exited with  exitCode: 1 due to: Exception from container-launch: org.apache.hadoop.util.Shell$ExitCodeException: 
org.apache.hadoop.util.Shell$ExitCodeException: 
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
    at org.apache.hadoop.util.Shell.run(Shell.java:418)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
    at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
    at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:744)


Container exited with a non-zero exit code 1
.Failing this attempt.. Failing the application.
14/07/08 14:21:44 INFO mapreduce.Job: Counters: 0

这个错误的原因是什么？

请注意，我将我的 mapreduce 项目转换为 maven 项目，以便在我的代码中使用 lucene 库。

【问题讨论】：

你能在 yarn-site.xml 中检查这个属性值 yarn.application.classpath 吗？？
要了解真正的原因，您可以使用yarn log 实用程序检查完整的应用程序日志。您得到的错误很可能是因为后台出现 NoClassDefFoundError

标签： hadoop lucene mapreduce

【解决方案1】：

您的资源管理器真的在 /0.0.0.0:8032 上吗？它还会显示您没有使用 Toolrunner，因此请尝试重写您的 mapreduce Hadoop: Implementing the Tool interface for MapReduce driver。

希望对你有帮助

【讨论】：

【解决方案2】：

线程数增加，JVM内存和CPU被充分利用。请增加 JVM 大小并增加 Mapper 和 reducer 任务的内存限制。 conf.set("mapreduce.map.memory.mb", "4096"); conf.set("mapreduce.map.java.opts", "-Xmx3500m");

【讨论】：