Hadoop集群处于安全模式（Namenode处于安全模式）我需要释放哪些资源才能删除安全模式？答案

【问题标题】：Hadoop cluster is in safe mode ( Namenode is in safe mode) What resources do I need to free in order to remove the safe mode?Hadoop集群处于安全模式（Namenode处于安全模式）我需要释放哪些资源才能删除安全模式？
【发布时间】：2018-10-13 11:48:22
【问题描述】：

我已经拍摄了集群的快照。以下是我的发现：

Safe mode is ON
Configured Capacity: 47430737653760 (43.14 TB)
Present Capacity: 20590420062208 (18.73 TB)
DFS Remaining: 19343468953600 (17.59 TB)
DFS Used: 1246951108608 (1.13 TB)
DFS Used%: 6.06%
Under replicated blocks: 2
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Live datanodes (5):

Name: 10.0.70.144:50010 (slave2)
Hostname: slave2
Decommission Status : Normal
Configured Capacity: 9486147530752 (8.63 TB)
DFS Used: 209829912576 (195.42 GB)
Non DFS Used: 4733044670464 (4.30 TB)
DFS Remaining: 4543272947712 (4.13 TB)
DFS Used%: 2.21%
DFS Remaining%: 47.89%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 13 16:57:21 IST 2018


Name: 10.0.70.143:50010 (slave1)
Hostname: slave1
Decommission Status : Normal
Configured Capacity: 9486147530752 (8.63 TB)
DFS Used: 206771748864 (192.57 GB)
Non DFS Used: 4070449033216 (3.70 TB)
DFS Remaining: 5208926748672 (4.74 TB)
DFS Used%: 2.18%
DFS Remaining%: 54.91%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 13 16:57:21 IST 2018


Name: 10.0.70.145:50010 (slave3)
Hostname: slave3
Decommission Status : Normal
Configured Capacity: 9486147530752 (8.63 TB)
DFS Used: 205542408192 (191.43 GB)
Non DFS Used: 5523446423552 (5.02 TB)
DFS Remaining: 3757158699008 (3.42 TB)
DFS Used%: 2.17%
DFS Remaining%: 39.61%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 13 16:57:21 IST 2018


Name: 10.0.70.147:50010 (slave5)
Hostname: slave5
Decommission Status : Normal
Configured Capacity: 9486147530752 (8.63 TB)
DFS Used: 209182961664 (194.82 GB)
Non DFS Used: 5516635717632 (5.02 TB)
DFS Remaining: 3760328851456 (3.42 TB)
DFS Used%: 2.21%
DFS Remaining%: 39.64%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 13 16:57:22 IST 2018


Name: 10.0.70.146:50010 (slave4)
Hostname: slave4
Decommission Status : Normal
Configured Capacity: 9486147530752 (8.63 TB)
DFS Used: 415624077312 (387.08 GB)
Non DFS Used: 6996741746688 (6.36 TB)
DFS Remaining: 2073781706752 (1.89 TB)
DFS Used%: 4.38%
DFS Remaining%: 21.86%
Configured Cache Capacity: 0 (0 B)
Cache Used: 0 (0 B)
Cache Remaining: 0 (0 B)
Cache Used%: 100.00%
Cache Remaining%: 0.00%
Xceivers: 1
Last contact: Sat Oct 13 16:57:21 IST 2018

我需要释放哪些资源才能使 NN 离开安全模式。我可以尝试 dfsadmin safemode leave 命令，但我需要确保下次运行 MR 作业不会失败。我正在尝试通过 MR 作业处理 400GB 的文本数据。我还有 3.2 TB 的数据需要处理。请协助我有效地处理数据。

【问题讨论】：

MR 作业通常不会有效地处理数据。你想做什么工作？当它们失败时你会遇到什么错误？因为存储显然不是原因，并且 YARN 作业通常不会导致名称节点进入安全模式......去看看它的日志
这是我在 hdfs 中对 raw_data 目录执行简单 ls 时的日志。 raw_data 目录有 180 万个文件：Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded
是的，Hadoop 不适合处理数百万个小文件。您应该将它们全部压缩并存档，或者您必须增加名称节点堆大小
但是，我认为这不是问题所在。主要问题是[hduser@secondary mapper]$ df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/centos-root 1.1T 1011G 19M 100% / centos-root 目录已 100% 满。我需要删除一些占用大部分空间的文件。

标签： hadoop mapreduce bigdata

【解决方案1】：

Namenode 存在磁盘空间问题。 /dev/mapper/centos-root 已满 100%。我从中删除了一些文件以创建一些空间。这是当前的快照：

[hduser@secondary ~]$ df 
Filesystem               1K-blocks      Used Available Use% Mounted on
/dev/mapper/centos-root 1116838084 557248752 502833928  53% /

创建空间后，我使用命令 hadoop dfsadmin -safemode leave 退出了安全模式并运行了成功完成的 MR 作业。

【讨论】：