准备工作:
1,hadoop-2.7.1
2, hadoop-native-64-2.7.0
3, jdk-8U121-linux-x64.rpm
4.Centos-6.4_64
安装centos-mini
node-1 ip:192.168.18.140
node-2 ip: 192.168.18.141
node-3 ip: 192.168.18.142
设置node-1
1修改etc/sysconfig/network-scripts/ifcfg-eth0
修改内容
ONBOOT=yes BOOTPROTO=static
IPADDR=192.168.18.140
NETWORK=255.255.255.0
GATEWAY=192.168.18.255
DNS1=192.168.18.255 wq保存退出
重启网卡:service network restart
2.连接Xshell
3.配置yum命令参数
cd /etc/yum.repos.d/
mv CentOS-Base.repo CentOS-Base.bak
mv CentOS-Debuginfo.repo CentOS-Debuginfo.bak
mv CentOS-Vault.repo CentOS-Vault.bak
vi CentOS-Media.repo
挂载光盘:mount /dev/dvd /media/
更新yum源:yum clean all
安装yum软件:zip,vim openssh-server,openssh-clients
yum命令:yum install -y vim zip openssh-server openssh-clients
4.安装java
启动xftp
安装java:rpm -ivh jdk-8u121-linux-x64.rpm
测试java是否安装成功:java -version
5:搭建hadoop集群
1,上传hadoop.tar包,
2,解压:tar -zxvf hadoop.tar -C /usr/local/
3,cd /usr/local/hadoop-2.7.1/etc/hadoop配置核心文件
5.修改core-site.xml
修改:hadoop-env.sh
修改yarn-env.sh
复制mv mapred-site.xml.template mapred-site.xml
修改 mapred-site.xml
修改yarn-site,xml
修改slaves文件
6,关闭node-1,克隆node-1==>node-2,node-3
修改node-2
Vim /etc/sysconfig/network-scripts/idcfg-eth0
修改HWADDR,注释掉UUID开头的行,添加ip地址
修改机器名:vim /etc/sysconfig/network
重启node-2,重启网卡
(node-3步骤同上)
6.配置ssh免密码登录
使用ssh-******产生公钥与私钥对
Ssh-****** -t rsa 接着按三次enter键,生成密匙
用ssh-copy-id将公钥复制到远程机器上
ssh-copy-id -i /root/.ssh/id_rsa.pub node-2
ssh-copy-id -i /root/.ssh/id_rsa.pub node-3
测试登录
7:
配置时间同步
1,mount /dev/dvd /media挂载网盘
2,yum install -y ntp安装ntp
3,vim /etc/ntp.conf 配置ntp
4,永久关闭防火墙及永久打开ntp
5,service iptables stop & chkconfig iptables off
6,service ntpd start & chkconfig ntpd on
7,主节点配置如下
从节点
测试:ntpdate node-1
8:设置环境变量
Vim /etc/profile node-1,2,3
9:格式化集群
hdfs namenode -format
10:启动集群
start-dfs.sh
start-yarn.sh
mr-jobhistory-daemon.sh start historyserver