Hadoop 容器失败，甚至 100% 完成

Question

我已经设置了一个小型集群 Hadoop 2.7、Hbase 0.98 和 Nutch 2.3.1。我编写了一个自定义作业，首先简单地组合相同域的文档，然后首先从缓存中获取域的每个 URL（来自缓存，即列表），然后使用相应的键通过 datastore.get(url_key) 获取对象和然后更新分数后，通过context.write写入。

在处理完所有文档后，作业应该完成，但我观察到，如果由于超时和进度而失败，每次尝试都是 100% 完成显示。这是日志

attempt_1549963404554_0110_r_000001_1   100.00  FAILED  reduce > reduce node2:8042  logs    Thu Feb 21 20:50:43 +0500 2019  Fri Feb 22 02:11:44 +0500 2019  5hrs, 21mins, 0sec  AttemptID:attempt_1549963404554_0110_r_000001_1 Timed out after 1800 secs Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143
attempt_1549963404554_0110_r_000001_3   100.00  FAILED  reduce > reduce node1:8042  logs    Fri Feb 22 04:39:08 +0500 2019  Fri Feb 22 07:25:44 +0500 2019  2hrs, 46mins, 35sec AttemptID:attempt_1549963404554_0110_r_000001_3 Timed out after 1800 secs Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143
attempt_1549963404554_0110_r_000002_0   100.00  FAILED  reduce > reduce node3:8042  logs    Thu Feb 21 12:38:45 +0500 2019  Thu Feb 21 22:50:13 +0500 2019  10hrs, 11mins, 28sec    AttemptID:attempt_1549963404554_0110_r_000002_0 Timed out after 1800 secs Container killed by the ApplicationMaster. Container killed on request. Exit code is 143 Container exited with a non-zero exit code 143

它是什么意思，即当尝试 100.00% 完成时，它应该被标记为成功。不幸的是，对于我的情况，除了超时之外还有任何错误信息。如何调试这个问题？ 我的减速器有点发布到另一个问题 Apache Nutch 2.3.1 map-reduce timeout occurred while updating the score

Answer 1

我观察到，在上述 3 个日志中，执行所需的时间差异很大。请查看您正在执行的作业。