Hadoop 分布式集群搭建

1 修改配置文件

1.1 hadoop-env.sh

export JAVA_HOME=/usr/apps/jdk1.8.0_181-amd64

1.2 core-site.xml

 <property>
        <name>fs.default.name</name>
        <value>hdfs://node1:8020</value>
 </property>

1.3 hdfs-site.xml

	<property>
        <name>dfs.replication</name>
        <value>2</value>
    </property>
    
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>/home/hadoop/hdp2.6-cdh5.7-data/namenodeData</value>
    </property>
    
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>/home/hadoop/hdp2.6-cdh5.7-data/datanodeData</value>
    </property>
    

1.4 yarn-site.xml

<property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
    
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>node1</value>
    </property>

1.5 mapred-site.xml

<property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
</property>

1.6 slaves

node2
node3
node4

2 分发安装包到 node2,node3,node4

[hadoop@node1 ~]$ scp -r /home/hadoop/apps/hadoop-2.6.0-cdh5.7.0/ node2:/home/hadoop/apps

3 格式化

[hadoop@node1 ~]$ hdfs namenode -format

4 启动集群

start-all.sh
浏览器访问 http://node1:50070