Hadoop 映射但不减少答案

【问题标题】：Hadoop maps but not reduceHadoop 映射但不减少
【发布时间】：2015-01-26 08:29:49
【问题描述】：

当我在服务器上启动 MapReduce 作业时，该作业将 100% 减少 0%，然后挂起。

打开 Web 控制台显示所有地图作业都已完成，并且有 1 个“NEW”reduce，但 0 个“RUNNING”reduce。

作业的控制台输出为：

15/01/22 10:26:01 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
15/01/22 10:26:01 WARN mapreduce.JobSubmitter: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
15/01/22 10:26:02 INFO input.FileInputFormat: Total input paths to process : 1
15/01/22 10:26:02 INFO mapreduce.JobSubmitter: number of splits:1
15/01/22 10:26:02 INFO Configuration.deprecation: user.name is deprecated. Instead, use mapreduce.job.user.name
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.jar is deprecated. Instead, use mapreduce.job.jar
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class
15/01/22 10:26:02 INFO Configuration.deprecation: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class
15/01/22 10:26:02 INFO Configuration.deprecation: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.job.name is deprecated. Instead, use mapreduce.job.name
15/01/22 10:26:02 INFO Configuration.deprecation: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class
15/01/22 10:26:02 INFO Configuration.deprecation: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir
15/01/22 10:26:03 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1421950773318_0001
15/01/22 10:26:04 INFO impl.YarnClientImpl: Submitted application application_1421950773318_0001 to ResourceManager at /0.0.0.0:8032
15/01/22 10:26:04 INFO mapreduce.Job: The url to track the job: http://localhost:8088/proxy/application_1421950773318_0001/
15/01/22 10:26:04 INFO mapreduce.Job: Running job: job_1421950773318_0001
15/01/22 10:26:15 INFO mapreduce.Job: Job job_1421950773318_0001 running in uber mode : false
15/01/22 10:26:15 INFO mapreduce.Job:  map 0% reduce 0%
15/01/22 10:26:33 INFO mapreduce.Job:  map 100% reduce 0%

我在跑步：

64 位 CentOS 6.4 版
Hadoop 2.2.0-gphd-3.1.0.0

【问题讨论】：

这可能是您的代码或您的环境。首先，确保字数统计示例有效。 wiki.apache.org/hadoop/WordCount
您尝试运行自己的示例作业或代码？我看到一些警告和弃用消息。请发布您的代码，至少是主类。

标签： hadoop mapreduce

【解决方案1】：

我在 wiki.apache.org/hadoop/WordCount 上尝试了 wordcount 示例

该示例不起作用，但我能够通过为 reduce 操作分配更少的内存来解决我的问题。

【讨论】：