使用多个 Docker 主机集群 Dockerized Elasticsearch答案

【问题标题】：Clustering Dockerized Elasticsearch with multiple Docker Host使用多个 Docker 主机集群 Dockerized Elasticsearch
【发布时间】：2019-12-13 18:38:32
【问题描述】：

尝试使用 docker compose 使其集群化。我有两个弹性搜索 docker 容器，它们部署在不同的 Docker 主机中。

docker version: 18.06.3-ce
elasticsearch : 6.5.2

docker-container-1 的 docker-compose.yml

 services:
   elasticsearch:
     restart: always
     hostname: elasticsearch
     image: docker-elk/elasticsearch:1.0.0
     build:
       context: elasticsearch
       dockerfile: Dockerfile
     environment:
       discovery.type: zen
     ports:
       - 9200:9200
       - 9300:9300
     env_file:
       - ./elasticsearch/elasticsearch.env
     volumes:
       - elasticsearch_data:/usr/share/elasticsearch/data

docker-compose.yml 用于 docker-container-2

 services:
   elasticsearch:
     restart: always
     hostname: elasticsearch
     image: docker-elk/elasticsearch:1.0.0
     build:
       context: elasticsearch
       dockerfile: Dockerfile
     environment:
       discovery.type: zen
     ports:
       - 9200:9200
       - 9300:9300
     env_file:
       - ./elasticsearch/elasticsearch.env
     volumes:
       - elasticsearch_data:/usr/share/elasticsearch/data

elasticsearch.yml 位于 Docker-Host 1 上的 elasticsearch-docker-container-1

 xpack.security.enabled: true
 cluster.name: es-cluster
 node.name: es1
 network.host: 0.0.0.0
 node.master: true
 node.data: true
 transport.tcp.port: 9300
 path.data: /usr/share/elasticsearch/data
 path.logs: /usr/share/elasticsearch/logs
 discovery.zen.minimum_master_nodes: 2
 gateway.recover_after_nodes: 1
 discovery.zen.ping.unicast.hosts: ["host1:9300", "host2:9300","host1:9200", "host2:9200"]
 network.publish_host: host1

elasticsearch.yml 位于 Docker-Host 2 上的 elasticsearch-docker-container-2

 xpack.security.enabled: true
 cluster.name: es-cluster
 node.name: es2
 network.host: 0.0.0.0
 node.master: true
 node.data: true
 transport.tcp.port: 9300
 path.data: /usr/share/elasticsearch/data
 path.logs: /usr/share/elasticsearch/logs
 discovery.zen.minimum_master_nodes: 2
 gateway.recover_after_nodes: 1
 discovery.zen.ping.unicast.hosts: ["host1:9300", "host2:9300","host1:9200", "host2:9200"]
 network.publish_host: host2

下面是 GET /_cluster/health?pretty 的结果，显示只有一个节点。

 {
   "cluster_name" : "dps_geocluster",
   "status" : "yellow",
   "timed_out" : false,
   "number_of_nodes" : 1,
   "number_of_data_nodes" : 1,
   "active_primary_shards" : 33,
   "active_shards" : 33,
   "relocating_shards" : 0,
   "initializing_shards" : 0,
   "unassigned_shards" : 30,
   "delayed_unassigned_shards" : 0,
   "number_of_pending_tasks" : 0,
   "number_of_in_flight_fetch" : 0,
   "task_max_waiting_in_queue_millis" : 0,
   "active_shards_percent_as_number" : 52.38095238095239
 }

根据下面的文档，至少需要三个 elasticsearch 节点。 https://www.elastic.co/guide/en/elasticsearch/reference/6.5/modules-node.html

每个 elasticsearch 容器应该在不同的 Docker 主机上？

【问题讨论】：

host1 host2 只是例子？如果不。是否可以使用这些 DNS 名称访问它们？
是的，可以通过使用诸如 host1 和 host2 之类的 DNS 名称从两个容器内部访问它们。

标签： docker elasticsearch docker-compose

【解决方案1】：

以下是错误的原因。用 sysctl 将 vm.max_map_count 的值增加到 262144 后，就可以正常工作了。

elasticsearch_1  | [1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

现在 number_of_nodes 是 2。

{
  "cluster_name" : "es-cluster",
  "status" : "yellow",
  "timed_out" : false,
  "number_of_nodes" : 2,
  "number_of_data_nodes" : 2,
  "active_primary_shards" : 35,
  "active_shards" : 37,
  "relocating_shards" : 0,
  "initializing_shards" : 2,
  "unassigned_shards" : 31,
  "delayed_unassigned_shards" : 0,
  "number_of_pending_tasks" : 0,
  "number_of_in_flight_fetch" : 0,
  "task_max_waiting_in_queue_millis" : 0,
  "active_shards_percent_as_number" : 52.85714285714286
}

【讨论】：