最近开始接触Greenplum,线上也在使用了,感觉还不错,本次介绍一下集群的部署方法。那么Greenplum的架构如下:
(架构图来源网络)
简单来说GPDB是一个分布式数据库软件,其可以管理和处理分布在多个不同主机上的海量数据。对于GPDB来说,一个DB实例实际上是由多个独立的PostgreSQL实例组成的,它们分布在不同的物理主机上,协同工作,呈现给用户的是一个DB的效果。Master是GPDB系统的访问入口,其负责处理客户端的连接及SQL 命令、协调系统中的其他Instance(Segment)工作,Segment负责管理和处理用户数据。
环境准备:
操作系统:centos 6.3 64位
master 1台(架构图中的主节点),Standby 1台(架构图中的从节点),Segment 2台。共4台服务器。
ip 主机名 10.69.30.251 mdw 10.36.25.152 smdw 10.36.25.250 sdw1 221.5.32.152 sdw2
1. 修改/etc/hosts文件,添加下面内容(4台服务器相同的配置)
10.69.30.251 mdw 10.36.25.152 smdw 10.36.25.250 sdw1 10.37.237.152 sdw2
2. 服务器关闭selinux,防火墙4台服务器相互开放,测试环境可以直接先关闭防火墙。(4台服务器)
3. 操作系统参数设置
vim /etc/sysctl.conf (注:4台服务器)
kernel.shmmax = 500000000 kernel.shmmni = 4096 kernel.shmall = 4000000000 kernel.sem = 250 512000 100 2048 kernel.sysrq = 1 kernel.core_uses_pid = 1 kernel.msgmnb = 65536 kernel.msgmax = 65536 net.ipv4.tcp_syncookies = 1 net.ipv4.ip_forward = 0 net.ipv4.conf.default.accept_source_route = 0 net.ipv4.tcp_tw_recycle = 1 net.ipv4.tcp_max_syn_backlog = 4096 net.ipv4.conf.all.arp_filter = 1 net.ipv4.conf.default.arp_filter = 1 net.core.netdev_max_backlog = 10000 vm.overcommit_memory = 2 kernel.msgmni = 2048 net.ipv4.ip_local_port_range = 1025 65535
vim /etc/security/limits.conf (注:4台服务器)
* soft nofile 65536 * hard nofile 65536 * soft nproc 131072 * hard nproc 131072
我的操作系统是centos 6.3,所以还有一个文件要修改:那就是/etc/security/limits.d/90-nproc.conf (注:4台服务器都要修改)
修改后的内容如下:
* soft nproc 131072 * hard nproc 131072 * soft nofile 1048576 * hard nofile 1048576
磁盘预读参数及 deadline算法修改 (注:4台服务器)
blockdev --setra 65536 /dev/sda echo deadline > /sys/block/sda/queue/scheduler
软件下载地址:https://network.pivotal.io/products/pivotal-gpdb
在Master主机上安装GP二进制文件,也就是主机名是mdw的服务器。(注:在master上安装即可,后面通过批量的方法安装剩下的服务器)
我下载的文件是greenplum-db-4.3.7.3-build-2-RHEL5-x86_64.zip
文件解压之是一个可执行文件,greenplum-db-4.3.7.3-build-2-RHEL5-x86_64.bin,直接执行安装。
./greenplum-db-4.3.7.3-build-2-RHEL5-x86_64.bin
然后有些说明,一直回车,出现如下图按照提示安装即可完成。
在master上添加用户
useradd gpadmin
确保所有的服务器安装了ed软件,否则后面初始化集群就会报错。
yum install ed -y
准备文件,all_hosts ,seg_hosts放到/home/gpadmin。(用于批量安装软件以及后续集群的初始化)
[gpadmin@mdw ~]$ cat seg_hosts sdw1 sdw2 [gpadmin@mdw ~]$ cat all_hosts mdw smdw sdw1 sdw2 [gpadmin@mdw ~]$
建立节点服务器间的信任(我服务器已经打通了信任的,没有打通的会要求输入密码)
[root@mdw gpadmin]# gpssh-exkeys -f all_hosts [STEP 1 of 5] create local ID and authorize on local host ... /root/.ssh/id_rsa file exists ... key generation skipped [STEP 2 of 5] keyscan all hosts and update known_hosts file [STEP 3 of 5] authorize current user on remote hosts ... send to smdw ... send to sdw1 ... send to sdw2 [STEP 4 of 5] determine common authentication file content [STEP 5 of 5] copy authentication files to all remote hosts ... finished key exchange with smdw ... finished key exchange with sdw1 ... finished key exchange with sdw2 [INFO] completed successfully
批量安装软件(GP)
cd /home/gpadmin/
source /usr/local/greenplum-db/greenplum_path.sh gpseginstall -f all_hosts -u gpadmin -p gpadmin
最后出现如下提示代表批量安装成功。
20160827:13:58:47:020986 gpseginstall:greenplum_master:root-[INFO]:-SUCCESS -- Requested commands completed
检查是否安装完成
su - gpadmin source /usr/local/greenplum-db/greenplum_path.sh gpssh -f all_hosts -e ls -l $GPHOME
输出如下代表成功:
[ mdw] ls -l /usr/local/greenplum-db/. [ mdw] total 276 [ mdw] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 bin [ mdw] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 demo [ mdw] drwxr-xr-x 5 gpadmin gpadmin 4096 Feb 25 2016 docs [ mdw] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 etc [ mdw] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 ext [ mdw] -rw-r--r-- 1 gpadmin gpadmin 43025 Feb 25 2016 GPDB-LICENSE.txt [ mdw] -rw-rw-r-- 1 gpadmin gpadmin 735 Aug 27 12:30 greenplum_path.sh [ mdw] drwxr-xr-x 6 gpadmin gpadmin 4096 Feb 25 2016 include [ mdw] drwxr-xr-x 9 gpadmin gpadmin 4096 Feb 25 2016 lib [ mdw] -rw-r--r-- 1 gpadmin gpadmin 192912 Feb 25 2016 LICENSE.thirdparty [ mdw] drwxr-xr-x 2 gpadmin gpadmin 4096 Aug 27 15:16 sbin [ mdw] drwxr-xr-x 4 gpadmin gpadmin 4096 Feb 25 2016 share [sdw1] ls -l /usr/local/greenplum-db/. [sdw1] total 276 [sdw1] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 bin [sdw1] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 demo [sdw1] drwxr-xr-x 5 gpadmin gpadmin 4096 Feb 25 2016 docs [sdw1] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 etc [sdw1] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 ext [sdw1] -rw-r--r-- 1 gpadmin gpadmin 43025 Feb 25 2016 GPDB-LICENSE.txt [sdw1] -rw-rw-r-- 1 gpadmin gpadmin 735 Aug 27 12:30 greenplum_path.sh [sdw1] drwxr-xr-x 6 gpadmin gpadmin 4096 Feb 25 2016 include [sdw1] drwxr-xr-x 9 gpadmin gpadmin 4096 Feb 25 2016 lib [sdw1] -rw-r--r-- 1 gpadmin gpadmin 192912 Feb 25 2016 LICENSE.thirdparty [sdw1] drwxr-xr-x 2 gpadmin gpadmin 4096 Aug 27 15:16 sbin [sdw1] drwxr-xr-x 4 gpadmin gpadmin 4096 Feb 25 2016 share [smdw] ls -l /usr/local/greenplum-db/. [smdw] total 276 [smdw] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 bin [smdw] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 demo [smdw] drwxr-xr-x 5 gpadmin gpadmin 4096 Feb 25 2016 docs [smdw] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 etc [smdw] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 ext [smdw] -rw-r--r-- 1 gpadmin gpadmin 43025 Feb 25 2016 GPDB-LICENSE.txt [smdw] -rw-rw-r-- 1 gpadmin gpadmin 735 Aug 27 12:30 greenplum_path.sh [smdw] drwxr-xr-x 6 gpadmin gpadmin 4096 Feb 25 2016 include [smdw] drwxr-xr-x 9 gpadmin gpadmin 4096 Feb 25 2016 lib [smdw] -rw-r--r-- 1 gpadmin gpadmin 192912 Feb 25 2016 LICENSE.thirdparty [smdw] drwxr-xr-x 2 gpadmin gpadmin 4096 Aug 27 15:16 sbin [smdw] drwxr-xr-x 4 gpadmin gpadmin 4096 Feb 25 2016 share [sdw2] ls -l /usr/local/greenplum-db/. [sdw2] total 276 [sdw2] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 bin [sdw2] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 demo [sdw2] drwxr-xr-x 5 gpadmin gpadmin 4096 Feb 25 2016 docs [sdw2] drwxr-xr-x 2 gpadmin gpadmin 4096 Feb 25 2016 etc [sdw2] drwxr-xr-x 3 gpadmin gpadmin 4096 Feb 25 2016 ext [sdw2] -rw-r--r-- 1 gpadmin gpadmin 43025 Feb 25 2016 GPDB-LICENSE.txt [sdw2] -rw-rw-r-- 1 gpadmin gpadmin 735 Aug 27 12:30 greenplum_path.sh [sdw2] drwxr-xr-x 6 gpadmin gpadmin 4096 Feb 25 2016 include [sdw2] drwxr-xr-x 9 gpadmin gpadmin 4096 Feb 25 2016 lib [sdw2] -rw-r--r-- 1 gpadmin gpadmin 192912 Feb 25 2016 LICENSE.thirdparty [sdw2] drwxr-xr-x 2 gpadmin gpadmin 4096 Aug 27 15:16 sbin [sdw2] drwxr-xr-x 4 gpadmin gpadmin 4096 Feb 25 2016 share