under replicated blocks 
解决:
找出没有复制的block:
hdfs fsck / | grep 'Under replicated' | awk -F':' '{print $1}' >> /tmp/under_replicated_files 
然后循环修复:
for hdfsfile in `cat /tmp/under_replicated_files`; do echo "Fixing $hdfsfile :" ;  hadoop fs -setrep 3 $hdfsfile; done
输出如下:
Fixing /user/hdfs/.staging/job_1547173493660_0405/job.jar :
Replication 3 set: /user/hdfs/.staging/job_1547173493660_0405/job.jar
Fixing /user/hdfs/.staging/job_1547173493660_0405/job.split :
Replication 3 set: /user/hdfs/.staging/job_1547173493660_0405/job.split
Fixing /user/hdfs/.staging/job_1547173493660_0481/job.jar :
Replication 3 set: /user/hdfs/.staging/job_1547173493660_0481/job.jar
Fixing /user/hdfs/.staging/job_1547173493660_0481/job.split :
Replication 3 set: /user/hdfs/.staging/job_1547173493660_0481/job.split
Fixing /user/hdfs/.staging/job_1547173493660_0483/job.jar :
Replication 3 set: /user/hdfs/.staging/job_1547173493660_0483/job.jar
Fixing /user/hdfs/.staging/job_1547173493660_0483/job.split :
Replication 3 set: /user/hdfs/.staging/job_1547173493660_0483/job.split
Fixing /user/hdfs/.staging/job_1547197402450_0021/job.jar :
Replication 3 set: /user/hdfs/.staging/job_1547197402450_0021/job.jar
Fixing /user/hdfs/.staging/job_1547197402450_0021/job.split :
Replication 3 set: /user/hdfs/.staging/job_1547197402450_0021/job.split

 

相关文章:

  • 2021-05-28
  • 2021-12-12
  • 2022-12-23
  • 2021-07-26
  • 2022-12-23
  • 2021-09-09
  • 2021-08-05
  • 2021-12-08
猜你喜欢
  • 2022-12-23
  • 2022-12-23
  • 2021-07-09
  • 2021-07-25
  • 2021-12-19
  • 2022-12-23
相关资源
相似解决方案