Hbase 配置多master节点

环境描述

正常运行一套hadoop-3.0.0集群，hbase配置，一个Hmaster，两个Hregionserver；

运行大数据计算时候，hbase的Hmaster容易因为内存资源问题自动退出；

需要配置多个Hmaster，多个Hregionserver；

功能需求

根据需求，部署hadoop-3.0.0基础功能架构，以六节点为安装环境，操作系统CentOS 7 x64；

openstack创建三台虚拟机，开始部署；

IP地址	主机名
10.10.204.31	master
10.10.204.32	slave1
10.10.204.33	slave2
10.10.204.34	datanode1
10.10.204.35	datanode2
10.10.204.36	datanode3

功能节点规划

master	slave1	slave2	datanode1	datanode2	datanode3
NameNode
			DataNode	DataNode	DataNode
JournalNode	JournalNode	JournalNode
zookeeper	zookeeper	zookeeper
kafka	kafka	kafka
			NodeManager	NodeManager	NodeManager
	ResourceManager	ResourceManager
HMaster	HRegionServer	HRegionServer	HRegionServer	HRegionServer	HRegionServer
Hive	Hive	Hive

配置记录

1.修改hbase配置文件；

hbase-site.xml

<name>hbase.cluster.distributed</name>

</property>

<name>hbase.rootdir</name>

<value>hdfs://vhcluster/hbase</value>

</property>

<name>hbase.zookeeper.quorum</name>

<value>master,slave1,slave2</value>

</property>

<name>hbase.zookeeper.property.clientPort</name>

</property>

<name>zookeeper.znode.parent</name>

<value>/hbase</value>

</property>

<name>hbase.master.info.port</name>

</property>

<name>hbase.regionserver.info.port</name>

</property>

<!--

<name>hbase.tmp.dir</name>

<value>/hbase/tmp</value>

</property>

-->

<name>hbase.unsafe.stream.capability.enforce</name>

<value>false</value>

</property>

<name>hbase.superuser</name>

<value>hbase</value>

</property>

<name>hbase.coprocessor.region.classes</name>

<value>org.apache.hadoop.hbase.security.access.AccessController</value>

</property>

<name>hbase.coprocessor.master.classes</name>

<value>org.apache.hadoop.hbase.security.access.AccessController</value>

</property>

<name>hbase.rpc.engine</name>

<value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value>

</property>

<name>hbase.security.authorization</name>

</property>

</configuration>

regionservers

slave1

slave2

datanode1

datanode2

datanode3

启动hbase服务

master

cd /opt/modules/hadoop-3.0.0/hbase-2.0.4

./bin/start-hbase.sh

slave1

cd /opt/modules/hadoop-3.0.0/hbase-2.0.4

./bin/hbase-daemon.sh start master

slave2

cd /opt/modules/hadoop-3.0.0/hbase-2.0.4

./bin/hbase-daemon.sh start master

#如何确定哪个Hmaster节点为主，直接通过查看日志方式即可；

[[email protected] logs]# tail hbase-root-master-slave1.log

2019-04-03 10:06:19,460 INFO [main] server.Server: jetty-9.3.19.v20170502

2019-04-03 10:06:19,500 INFO [main] handler.ContextHandler: Started [email protected]{/logs,file:///opt/modules/hadoop-3.0.0/hbase-2.0.4/logs/,AVAILABLE}

2019-04-03 10:06:19,500 INFO [main] handler.ContextHandler: Started [email protected]{/static,file:///opt/modules/hadoop-3.0.0/hbase-2.0.4/hbase-webapps/static/,AVAILABLE}

2019-04-03 10:06:19,611 INFO [main] handler.ContextHandler: Started [email protected]{/,file:///opt/modules/hadoop-3.0.0/hbase-2.0.4/hbase-webapps/master/,AVAILABLE}{file:/opt/modules/hadoop-3.0.0/hbase-2.0.4/hbase-webapps/master}

2019-04-03 10:06:19,617 INFO [main] server.AbstractConnector: Started [email protected]{HTTP/1.1,[http/1.1]}{0.0.0.0:16010}

2019-04-03 10:06:19,617 INFO [main] server.Server: Started @4392ms

2019-04-03 10:06:19,621 INFO [main] master.HMaster: hbase.rootdir=hdfs://vhcluster/hbase, hbase.cluster.distributed=true

2019-04-03 10:06:19,644 INFO [Thread-13] master.HMaster: Adding backup master ZNode /hbase/backup-masters/slave1,16000,1554257175910

2019-04-03 10:06:19,664 INFO [master/slave1:16000] regionserver.HRegionServer: ClusterId : ff77d704-a618-4703-a61b-7b70e70cda00

2019-04-03 10:06:19,715 INFO [Thread-13] master.ActiveMasterManager: Another master is the active master, master,16000,1554257116936; waiting to become the next active master

[[email protected] logs]# tail hbase-root-master-slave2.log

2019-04-03 10:10:02,535 INFO [main] server.Server: jetty-9.3.19.v20170502

2019-04-03 10:10:02,572 INFO [main] handler.ContextHandler: Started [email protected]{/logs,file:///opt/modules/hadoop-3.0.0/hbase-2.0.4/logs/,AVAILABLE}

2019-04-03 10:10:02,572 INFO [main] handler.ContextHandler: Started [email protected]{/static,file:///opt/modules/hadoop-3.0.0/hbase-2.0.4/hbase-webapps/static/,AVAILABLE}

2019-04-03 10:10:02,663 INFO [main] handler.ContextHandler: Started [email protected]{/,file:///opt/modules/hadoop-3.0.0/hbase-2.0.4/hbase-webapps/master/,AVAILABLE}{file:/opt/modules/hadoop-3.0.0/hbase-2.0.4/hbase-webapps/master}

2019-04-03 10:10:02,669 INFO [main] server.AbstractConnector: Started [email protected]{HTTP/1.1,[http/1.1]}{0.0.0.0:16010}

2019-04-03 10:10:02,669 INFO [main] server.Server: Started @3306ms

2019-04-03 10:10:02,672 INFO [main] master.HMaster: hbase.rootdir=hdfs://vhcluster/hbase, hbase.cluster.distributed=true

2019-04-03 10:10:02,694 INFO [Thread-13] master.HMaster: Adding backup master ZNode /hbase/backup-masters/slave2,16000,1554257400094

2019-04-03 10:10:02,722 INFO [master/slave2:16000] regionserver.HRegionServer: ClusterId : ff77d704-a618-4703-a61b-7b70e70cda00

2019-04-03 10:10:02,761 INFO [Thread-13] master.ActiveMasterManager: Another master is the active master, master,16000,1554257116936; waiting to become the next active master

查看服务状态

jps

[[email protected] ~]# jps

912 Kafka

20564 QuorumPeerMain

23047 NameNode

23592 DFSZKFailoverController

30712 RunJar

23369 JournalNode

11977 Jps

9549 HMaster

[[email protected] ~]# jps