基于二进制部署Cloudera Manager集群扩容实战案例（服役）
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
一.查看数据库名现有节点
[root@node101.yinzhengjie.org.cn ~]# mysql -uroot -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 18 Server version: 5.7.25-log MySQL Community Server (GPL) Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> mysql> use cdh Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> mysql> show tables; +--------------------------------+ | Tables_in_cdh | +--------------------------------+ | AUDITS | | CLIENT_CONFIGS | | CLIENT_CONFIGS_TO_HOSTS | | CLUSTERS | | CLUSTERS_AUD | | CLUSTER_ACTIVATED_RELEASES | | CLUSTER_ACTIVATED_RELEASES_AUD | | CLUSTER_MANAGED_RELEASES | | CLUSTER_UNDISTRIBUTED_RELEASES | | CM_PEERS | | CM_VERSION | | COMMANDS | | COMMANDS_DETAIL | | COMMAND_SCHEDULES | | CONFIGS | | CONFIGS_AUD | | CONFIG_CONTAINERS | | CREDENTIALS | | DIAGNOSTICS_EVENTS | | EXTERNAL_ACCOUNTS | | EXTERNAL_ACCOUNTS_AUD | | GLOBAL_SETTINGS | | HOSTS | | HOSTS_AUD | | HOST_TEMPLATES | | HOST_TEMPLATE_TO_ROLE_CONF_GRP | | METRICS | | PARCELS | | PARCEL_COMPONENTS | | PROCESSES | | PROCESSES_DETAIL | | PROCESS_ACTIVE_RELEASES | | RELEASES | | RELEASES_AUD | | REVISIONS | | ROLES | | ROLES_AUD | | ROLE_CONFIG_GROUPS | | ROLE_CONFIG_GROUPS_AUD | | ROLE_STALENESS_STATUS | | SCHEMA_VERSION | | SERVICES | | SERVICES_AUD | | SNAPSHOT_POLICIES | | USERS | | USER_ROLES | | USER_SETTINGS | +--------------------------------+ 47 rows in set (0.00 sec) mysql> mysql> SELECT * FROM HOSTS; +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ | HOST_ID | OPTIMISTIC_LOCK_VERSION | HOST_IDENTIFIER | NAME | IP_ADDRESS | RACK_ID | STATUS | CONFIG_CONTAINER_ID | MAINTENANCE_COUNT | DECOMMISSION_COUNT | CLUSTER_ID | NUM_CORES | TOTAL_PHYS_MEM_BYTES | PUBLIC_NAME | PUBLIC_IP_ADDRESS | CLOUD_PROVIDER | +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ | 1 | 10 | a047f9e1-50a6-4245-91f5-a02fb728e783 | node102.yinzhengjie.org.cn | 172.30.1.102 | /default | NA | 1 | 0 | 0 | 1 | 4 | 1907970048 | NULL | NULL | NULL | | 2 | 12 | 1f8e3d4c-5609-476c-81d6-93bea59b3e31 | node101.yinzhengjie.org.cn | 172.30.1.101 | /default | NA | 1 | 0 | 0 | 1 | 4 | 8182054912 | NULL | NULL | NULL | | 3 | 12 | c0ad9575-495e-4855-b4c8-a81d9e31c344 | node103.yinzhengjie.org.cn | 172.30.1.103 | /default | NA | 1 | 0 | 0 | 1 | 4 | 3954188288 | NULL | NULL | NULL | +---------+-------------------------+--------------------------------------+----------------------------+--------------+----------+--------+---------------------+-------------------+--------------------+------------+-----------+----------------------+-------------+-------------------+----------------+ 3 rows in set (0.00 sec) mysql>
二.在需要加入该节点的服务器上启动Cloudera Manager agent
1>.解压CM安装包
[root@node104.yinzhengjie.org.cn ~]# ll total 819236 -rw-r--r-- 1 root root 838894986 Jan 8 02:11 cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# mkdir /opt/cloudera-manager/ [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# tar -zxf cloudera-manager-centos7-cm5.15.1_x86_64.tar.gz -C /opt/cloudera-manager/ [root@node104.yinzhengjie.org.cn ~]#
2>.修改CM Agent的配置文件
[root@node104.yinzhengjie.org.cn ~]# grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini server_host=localhost [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# sed -i 's/server_host=localhost/server_host=node101.yinzhengjie.org.cn/g' /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini
[root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# grep server_host /opt/cloudera-manager/cm-5.15.1/etc/cloudera-scm-agent/config.ini server_host=node101.yinzhengjie.org.cn [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]#
3>.启动CM Agent服务
[root@node104.yinzhengjie.org.cn ~]# cd /opt/cloudera-manager/cm-5.15.1/etc/init.d/ [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ll total 32 -rwxr-xr-x 1 1106 4001 8871 Jul 31 2018 cloudera-scm-agent -rwxr-xr-x 1 1106 4001 8417 Jul 31 2018 cloudera-scm-server -rwxr-xr-x 1 1106 4001 4444 Jul 31 2018 cloudera-scm-server-db [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]# ./cloudera-scm-agent start Starting cloudera-scm-agent: [ OK ] [root@node104.yinzhengjie.org.cn /opt/cloudera-manager/cm-5.15.1/etc/init.d]#
4>.节点上创建clodera-scm用户
[root@node104.yinzhengjie.org.cn ~]# useradd --system --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# id cloudera-scm uid=978(cloudera-scm) gid=975(cloudera-scm) groups=975(cloudera-scm) [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]#
如果某个agent节点没有创建用户和组的操作，我们在安装服务时可能会出现如下提示：
5>.agent端创建parcels目录
[root@node104.yinzhengjie.org.cn ~]# mkdir -p /opt/cloudera/parcels [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# chown cloudera-scm:cloudera-scm /opt/cloudera/parcels [root@node104.yinzhengjie.org.cn ~]# [root@node104.yinzhengjie.org.cn ~]# ls -ld /opt/cloudera/parcels drwxr-xr-x 2 cloudera-scm cloudera-scm 6 Jun 28 04:49 /opt/cloudera/parcels [root@node104.yinzhengjie.org.cn ~]#
其实，我们不执行该步骤，也不会影响添加agent节点，但是我们在安装的时候可能会报错如下图所示，当然你可以选择忽略它，照样是可以添加成功的，不过建议你还是执行该步骤。
6>.同步时区及NTP服务
[root@node104.yinzhengjie.org.cn ~]# yum -y install ntp Loaded plugins: fastestmirror Determining fastest mirrors epel/x86_64/metalink | 8.0 kB 00:00:00 * base: mirrors.aliyun.com * epel: mirrors.yun-idc.com * extras: mirrors.aliyun.com * updates: mirrors.aliyun.com base | 3.6 kB 00:00:00 epel | 5.3 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/4): extras/7/x86_64/primary_db | 205 kB 00:00:00 (2/4): epel/x86_64/updateinfo | 977 kB 00:00:00 (3/4): updates/7/x86_64/primary_db | 6.4 MB 00:00:06 (4/4): epel/x86_64/primary_db | 6.8 MB 00:00:06 Resolving Dependencies --> Running transaction check ---> Package ntp.x86_64 0:4.2.6p5-28.el7.centos will be installed --> Processing Dependency: ntpdate = 4.2.6p5-28.el7.centos for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Processing Dependency: libopts.so.25()(64bit) for package: ntp-4.2.6p5-28.el7.centos.x86_64 --> Running transaction check ---> Package autogen-libopts.x86_64 0:5.18-5.el7 will be installed ---> Package ntpdate.x86_64 0:4.2.6p5-28.el7.centos will be installed --> Finished Dependency Resolution Dependencies Resolved ==================================================================================================================================================================================================== Package Arch Version Repository Size ==================================================================================================================================================================================================== Installing: ntp x86_64 4.2.6p5-28.el7.centos base 549 k Installing for dependencies: autogen-libopts x86_64 5.18-5.el7 base 66 k ntpdate x86_64 4.2.6p5-28.el7.centos base 86 k Transaction Summary ==================================================================================================================================================================================================== Install 1 Package (+2 Dependent packages) Total download size: 701 k Installed size: 1.6 M Downloading packages: ntpdate-4.2.6p5-28.el7.centos. FAILED http://mirrors.cloud.aliyuncs.com/centos/7/os/x86_64/Packages/ntpdate-4.2.6p5-28.el7.centos.x86_64.rpm: [Errno 14] curl#6 - "Could not resolve host: mirrors.cloud.aliyuncs.com; Unknown error" ETA Trying other mirror. (1/3): ntp-4.2.6p5-28.el7.centos.x86_64.rpm | 549 kB 00:00:00 (2/3): autogen-libopts-5.18-5.el7.x86_64.rpm | 66 kB 00:00:00 (3/3): ntpdate-4.2.6p5-28.el7.centos.x86_64.rpm | 86 kB 00:00:00 ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 2.3 MB/s | 701 kB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : autogen-libopts-5.18-5.el7.x86_64 1/3 Installing : ntpdate-4.2.6p5-28.el7.centos.x86_64 2/3 Installing : ntp-4.2.6p5-28.el7.centos.x86_64 3/3 Verifying : ntpdate-4.2.6p5-28.el7.centos.x86_64 1/3 Verifying : autogen-libopts-5.18-5.el7.x86_64 2/3 Verifying : ntp-4.2.6p5-28.el7.centos.x86_64 3/3 Installed: ntp.x86_64 0:4.2.6p5-28.el7.centos Dependency Installed: autogen-libopts.x86_64 0:5.18-5.el7 ntpdate.x86_64 0:4.2.6p5-28.el7.centos Complete! [root@node104.yinzhengjie.org.cn ~]#