Hadoop生态圈-CentOs7.5单机部署ClickHouse
作者：尹正杰
版权声明：原创作品，谢绝转载！否则将追究法律责任。
到了新的公司，认识了新的同事，生产环境也得你去适应新的集群环境，我新入职的公司的大数据开发同时并不看好hbase，而是对clickhouse青睐有加，听他们口头把ClickHouse吹的那么的出神入化，听的我是有种迫不及待想要对它一探究竟。
最新的一手资料请参考官网：https://clickhouse.yandex/
一.ClickHouse的独特功能
1>.真正的面向列的DBMS
因为有些系统可以单独存储不同列的值，但由于它们针对其他场景进行了优化，因此无法有效地处理分析查询。例如HBase，BigTable，Cassandra和HyperTable。在这些系统中，您将获得大约每秒十万行的吞吐量，但不会达到每秒数亿行。
值得注意的是，ClickHouse是一个数据库管理系统，而不是一个数据库。ClickHouse允许在运行时创建表和数据库，加载数据和运行查询，而无需重新配置和重新启动服务器。
2>.适用于在线查询
低延迟意味着可以在没有延迟的情况下处理查询，而无需在加载用户界面页面的同一时间提前准备答案。换句话说，在线。
3>.数据复制和数据完整性支持
使用异步多主机复制。写入任何可用副本后，数据将分发到后台的所有剩余副本。系统在不同的副本上维护相同的数据。在大多数故障后自动执行恢复，在复杂情况下 - 半自动执行。
4>.支持近似计算
ClickHouse提供了各种交易性能准确度的方法：
4.1>.聚合函数，用于近似计算不同值，中位数和分位数的数量。
4.2>.基于数据的一部分（样本）运行查询并获得近似结果。在这种情况下，从磁盘中检索的数据比例较少。
4.2>.为有限数量的随机密钥运行聚合，而不是为所有密钥运行聚合。在数据中密钥分发的某些条件下，这在使用较少资源的同时提供了相当准确的结果。
5>.SQL支持
ClickHouse支持基于SQL的声明性查询语言，在许多情况下与SQL标准相同。支持的查询包括GROUP BY，ORDER BY，FROM，IN和JOIN子句中的子查询以及标量子查询。不支持从属子查询和窗口函数。
以上说明并非我个人对ClickHouse的夸张说法，而是摘自官方文档的说明，可能官网对其软件有一定的夸张手法在里面，官网文档请参考：https://clickhouse.yandex/docs/en/
二.部署ClickHouse
推荐的安装方法参考：https://github.com/Altinity/clickhouse-rpm-install。文档写的头头是道，说下载一个安装shell脚本，结果你按照它的方法做会跑出一系列的坑，我在这里就不吐槽了，喜欢被虐的小伙伴可以去试试，我生产环境用的操作系统基本上都是CentOs的操作系统，我参考了网上多位网友的部署方法，找到了一种最简单有效的部署方法，我这里一句把坑给大家踩过了，只要我的按照步骤来部署就能成功！
1>.操作平台介绍
[root@yinzhengjie ~]# ip a | grep global | awk '{print $2}' | awk -F '/' '{print $1}' 10.1.2.105 [root@yinzhengjie ~]# [root@yinzhengjie ~]# hostname yinzhengjie [root@yinzhengjie ~]# [root@yinzhengjie ~]# cat /proc/cpuinfo| grep "processor"| wc -l 14 [root@yinzhengjie ~]# [root@yinzhengjie ~]# cat /proc/cpuinfo| grep "cpu cores"| uniq cpu cores : 7 [root@yinzhengjie ~]# [root@yinzhengjie ~]# cat /proc/cpuinfo| grep "physical id"| sort| uniq| wc -l 2 [root@yinzhengjie ~]# [root@yinzhengjie ~]# cat /proc/cpuinfo | grep name | cut -f2 -d: | uniq -c 14 Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz [root@yinzhengjie ~]# [root@yinzhengjie ~]# free -m total used free shared buff/cache available Mem: 11852 1406 8309 103 2136 9954 Swap: 6015 0 6015 [root@yinzhengjie ~]# [root@yinzhengjie ~]# free -g total used free shared buff/cache available Mem: 11 1 8 0 2 9 Swap: 5 0 5 [root@yinzhengjie ~]# [root@yinzhengjie ~]# [root@yinzhengjie ~]# uname -r 3.10.0-862.el7.x86_64 [root@yinzhengjie ~]# [root@yinzhengjie ~]# uname -m x86_64 [root@yinzhengjie ~]# [root@yinzhengjie ~]# cat /etc/redhat-release CentOS Linux release 7.5.1804 (Core) You have new mail in /var/spool/mail/root [root@yinzhengjie ~]#
2>.检查是否支持SSE4.2
[root@yinzhengjie ~]# grep -q sse4_2 /proc/cpuinfo && echo "SSE 4.2 supported" || echo "SSE 4.2 not supported" SSE 4.2 supported [root@yinzhengjie ~]#
3>.下载依赖包
我已经将上面的rpm包下载好了，以及需要的依赖包都下载好啦～大家可以直接下载来用即可。百度云链接:https://pan.baidu.com/s/16gPGxOnHKfRZj2ytOil7qQ 密码:rs4u
[root@yinzhengjie download]# ll total 42916 -rw-r--r-- 1 root root 43945614 Oct 10 20:10 yinzhengjie-clickhouse.tar.gz [root@yinzhengjie download]# [root@yinzhengjie download]# [root@yinzhengjie download]# tar zxf yinzhengjie-clickhouse.tar.gz -C ./ [root@yinzhengjie download]# [root@yinzhengjie download]# ll total 42920 drwxr-xr-x 2 root root 4096 Oct 10 20:03 clickhouse -rw-r--r-- 1 root root 43945614 Oct 10 20:10 yinzhengjie-clickhouse.tar.gz [root@yinzhengjie download]# [root@yinzhengjie download]# cd clickhouse/ [root@yinzhengjie clickhouse]# [root@yinzhengjie clickhouse]# ll total 43080 -rw-r--r-- 1 root root 3048 Oct 10 17:16 clickhouse-client-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 901472 Oct 10 17:16 clickhouse-compressor-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 9043256 Oct 10 17:16 clickhouse-debuginfo-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 33680608 Oct 10 17:18 clickhouse-server-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 8164 Oct 10 17:16 clickhouse-server-common-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 49036 Oct 10 18:35 libtool-ltdl-2.4.2-21.el7_2.x86_64.rpm -rw-r--r-- 1 root root 421792 Oct 10 18:20 unixODBC-2.3.1-11.el7.x86_64.rpm [root@yinzhengjie clickhouse]#
4>.安装ClickHouse的依赖及其rpm包
[root@yinzhengjie clickhouse]# ll total 43080 -rw-r--r-- 1 root root 3048 Oct 10 17:16 clickhouse-client-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 901472 Oct 10 17:16 clickhouse-compressor-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 9043256 Oct 10 17:16 clickhouse-debuginfo-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 33680608 Oct 10 17:18 clickhouse-server-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 8164 Oct 10 17:16 clickhouse-server-common-1.1.54236-4.el7.x86_64.rpm -rw-r--r-- 1 root root 49036 Oct 10 18:35 libtool-ltdl-2.4.2-21.el7_2.x86_64.rpm -rw-r--r-- 1 root root 421792 Oct 10 18:20 unixODBC-2.3.1-11.el7.x86_64.rpm [root@yinzhengjie clickhouse]# [root@yinzhengjie clickhouse]# You have new mail in /var/spool/mail/root [root@yinzhengjie clickhouse]# [root@yinzhengjie clickhouse]# yum -y localinstall *.rpm Loaded plugins: fastestmirror Examining clickhouse-client-1.1.54236-4.el7.x86_64.rpm: clickhouse-client-1.1.54236-4.el7.x86_64 Marking clickhouse-client-1.1.54236-4.el7.x86_64.rpm to be installed Examining clickhouse-compressor-1.1.54236-4.el7.x86_64.rpm: clickhouse-compressor-1.1.54236-4.el7.x86_64 Marking clickhouse-compressor-1.1.54236-4.el7.x86_64.rpm to be installed Examining clickhouse-debuginfo-1.1.54236-4.el7.x86_64.rpm: clickhouse-debuginfo-1.1.54236-4.el7.x86_64 Marking clickhouse-debuginfo-1.1.54236-4.el7.x86_64.rpm to be installed Examining clickhouse-server-1.1.54236-4.el7.x86_64.rpm: clickhouse-server-1.1.54236-4.el7.x86_64 Marking clickhouse-server-1.1.54236-4.el7.x86_64.rpm to be installed Examining clickhouse-server-common-1.1.54236-4.el7.x86_64.rpm: clickhouse-server-common-1.1.54236-4.el7.x86_64 Marking clickhouse-server-common-1.1.54236-4.el7.x86_64.rpm to be installed Examining libtool-ltdl-2.4.2-21.el7_2.x86_64.rpm: libtool-ltdl-2.4.2-21.el7_2.x86_64 Marking libtool-ltdl-2.4.2-21.el7_2.x86_64.rpm to be installed Examining unixODBC-2.3.1-11.el7.x86_64.rpm: unixODBC-2.3.1-11.el7.x86_64 Marking unixODBC-2.3.1-11.el7.x86_64.rpm to be installed Resolving Dependencies --> Running transaction check ---> Package clickhouse-client.x86_64 0:1.1.54236-4.el7 will be installed ---> Package clickhouse-compressor.x86_64 0:1.1.54236-4.el7 will be installed --> Processing Dependency: libicudata.so.50()(64bit) for package: clickhouse-compressor-1.1.54236-4.el7.x86_64 Determining fastest mirrors 10gen | 2.5 kB 00:00:00 base | 3.6 kB 00:00:00 centosplus | 3.4 kB 00:00:00 epel | 3.2 kB 00:00:00 extras | 3.4 kB 00:00:00 updates | 3.4 kB 00:00:00 (1/3): epel/x86_64/updateinfo | 948 kB 00:00:00 (2/3): epel/x86_64/primary | 3.6 MB 00:00:01 (3/3): updates/7/x86_64/primary_db | 6.0 MB 00:00:01 epel 12723/12723 --> Processing Dependency: libicui18n.so.50()(64bit) for package: clickhouse-compressor-1.1.54236-4.el7.x86_64 --> Processing Dependency: libicuuc.so.50()(64bit) for package: clickhouse-compressor-1.1.54236-4.el7.x86_64 ---> Package clickhouse-debuginfo.x86_64 0:1.1.54236-4.el7 will be installed ---> Package clickhouse-server.x86_64 0:1.1.54236-4.el7 will be installed ---> Package clickhouse-server-common.x86_64 0:1.1.54236-4.el7 will be installed ---> Package libtool-ltdl.x86_64 0:2.4.2-21.el7_2 will be installed ---> Package unixODBC.x86_64 0:2.3.1-11.el7 will be installed --> Running transaction check ---> Package libicu.x86_64 0:50.1.2-15.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved =================================================================================================================================================================================================================== Package Arch Version Repository Size =================================================================================================================================================================================================================== Installing: clickhouse-client x86_64 1.1.54236-4.el7 /clickhouse-client-1.1.54236-4.el7.x86_64 110 clickhouse-compressor x86_64 1.1.54236-4.el7 /clickhouse-compressor-1.1.54236-4.el7.x86_64 2.9 M clickhouse-debuginfo x86_64 1.1.54236-4.el7 /clickhouse-debuginfo-1.1.54236-4.el7.x86_64 48 M clickhouse-server x86_64 1.1.54236-4.el7 /clickhouse-server-1.1.54236-4.el7.x86_64 133 M clickhouse-server-common x86_64 1.1.54236-4.el7 /clickhouse-server-common-1.1.54236-4.el7.x86_64 16 k libtool-ltdl x86_64 2.4.2-21.el7_2 /libtool-ltdl-2.4.2-21.el7_2.x86_64 66 k unixODBC x86_64 2.3.1-11.el7 /unixODBC-2.3.1-11.el7.x86_64 1.2 M Installing for dependencies: libicu x86_64 50.1.2-15.el7 base 6.9 M Transaction Summary =================================================================================================================================================================================================================== Install 7 Packages (+1 Dependent package) Total size: 192 M Total download size: 6.9 M Installed size: 209 M Downloading packages: libicu-50.1.2-15.el7.x86_64.rpm | 6.9 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : libtool-ltdl-2.4.2-21.el7_2.x86_64 1/8 Installing : unixODBC-2.3.1-11.el7.x86_64 2/8 Installing : clickhouse-server-common-1.1.54236-4.el7.x86_64 3/8 Installing : libicu-50.1.2-15.el7.x86_64 4/8 Installing : clickhouse-server-1.1.54236-4.el7.x86_64 5/8 Installing : clickhouse-client-1.1.54236-4.el7.x86_64 6/8 Installing : clickhouse-compressor-1.1.54236-4.el7.x86_64 7/8 Installing : clickhouse-debuginfo-1.1.54236-4.el7.x86_64 8/8 Verifying : clickhouse-debuginfo-1.1.54236-4.el7.x86_64 1/8 Verifying : libtool-ltdl-2.4.2-21.el7_2.x86_64 2/8 Verifying : unixODBC-2.3.1-11.el7.x86_64 3/8 Verifying : libicu-50.1.2-15.el7.x86_64 4/8 Verifying : clickhouse-server-1.1.54236-4.el7.x86_64 5/8 Verifying : clickhouse-client-1.1.54236-4.el7.x86_64 6/8 Verifying : clickhouse-compressor-1.1.54236-4.el7.x86_64 7/8 Verifying : clickhouse-server-common-1.1.54236-4.el7.x86_64 8/8 Installed: clickhouse-client.x86_64 0:1.1.54236-4.el7 clickhouse-compressor.x86_64 0:1.1.54236-4.el7 clickhouse-debuginfo.x86_64 0:1.1.54236-4.el7 clickhouse-server.x86_64 0:1.1.54236-4.el7 clickhouse-server-common.x86_64 0:1.1.54236-4.el7 libtool-ltdl.x86_64 0:2.4.2-21.el7_2 unixODBC.x86_64 0:2.3.1-11.el7 Dependency Installed: libicu.x86_64 0:50.1.2-15.el7 Complete! [root@yinzhengjie clickhouse]#
三.配置clickhouse