docker-compose.yml 用于弹性搜索和 kibana答案

【问题标题】：docker-compose.yml for elasticsearch and kibanadocker-compose.yml 用于弹性搜索和 kibana
【发布时间】：2016-11-02 06:02:37
【问题描述】：

我的目标是让来自 DockerHub 的 elasticsearch 和 kibana 镜像使用 Docker 在本地工作。

这可以解决问题并且完美运行......

docker network create mynetwork --driver=bridge

docker run -p 5601:5601 --name kibana -d --network mynetwork kibana 
docker run -p 9200:9200 -p 9300:9300 --name elasticsearch -d --network mynetwork elasticsearch

今天一只鸟在我耳边低语说我应该学习 docker-compose。所以我尝试在 docker-compose.yml 中完成上述所有操作。

这是我的尝试。

version: "2.0"
services:
  elasticsearch:
    image: elasticsearch:latest
    ports:
      - "9200:9200"
      - "9300:9300"
    networks:
      - docker_elk
  kibana:
    image: kibana:latest
    ports:
      - "5601:5601"
    networks:
      - docker_elk
networks:
  docker_elk:
    driver: bridge

不幸的是，这不起作用。我一直在琢磨为什么我在运行 docker-compse up 时总是得到如下所示的 ECONNREFUSED 错误。

$ docker-compose up
Starting training_elasticsearch_1
Recreating training_kibana_1
Attaching to training_elasticsearch_1, training_kibana_1
elasticsearch_1  | [2016-11-02 22:39:55,798][WARN ][bootstrap                ] unable to install syscall filter: seccomp unavailable: your kernel is buggy and you should upgrade
elasticsearch_1  | [2016-11-02 22:39:56,036][INFO ][node                     ] [Caliban] version[2.4.1], pid[1], build[c67dc32/2016-09-27T18:57:55Z]
elasticsearch_1  | [2016-11-02 22:39:56,036][INFO ][node                     ] [Caliban] initializing ...
elasticsearch_1  | [2016-11-02 22:39:56,713][INFO ][plugins                  ] [Caliban] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
elasticsearch_1  | [2016-11-02 22:39:56,749][INFO ][env                      ] [Caliban] using [1] data paths, mounts [[/usr/share/elasticsearch/data (/dev/vda2)]], net usable_space [54.8gb], net total_space [59gb], spins? [possibly], types [ext4]
elasticsearch_1  | [2016-11-02 22:39:56,749][INFO ][env                      ] [Caliban] heap size [990.7mb], compressed ordinary object pointers [true]
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:kibana@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:elasticsearch@1.0.0","info"],"pid":11,"state":"yellow","message":"Status changed from uninitialized to yellow - Waiting for Elasticsearch","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["error","elasticsearch"],"pid":11,"message":"Request error, retrying -- connect ECONNREFUSED 172.20.0.2:9200"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:kbn_vislib_vis_types@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["warning","elasticsearch"],"pid":11,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["warning","elasticsearch"],"pid":11,"message":"No living connections"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:elasticsearch@1.0.0","error"],"pid":11,"state":"red","message":"Status changed from yellow to red - Unable to connect to Elasticsearch at http://elasticsearch:9200.","prevState":"yellow","prevMsg":"Waiting for Elasticsearch"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:markdown_vis@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:metric_vis@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:spyModes@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:statusPage@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["status","plugin:table_vis@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from uninitialized to green - Ready","prevState":"uninitialized","prevMsg":"uninitialized"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:39:58Z","tags":["listening","info"],"pid":11,"message":"Server running at http://0.0.0.0:5601"}
elasticsearch_1  | [2016-11-02 22:39:58,515][INFO ][node                     ] [Caliban] initialized
elasticsearch_1  | [2016-11-02 22:39:58,515][INFO ][node                     ] [Caliban] starting ...
elasticsearch_1  | [2016-11-02 22:39:58,587][INFO ][transport                ] [Caliban] publish_address {172.20.0.2:9300}, bound_addresses {[::]:9300}
elasticsearch_1  | [2016-11-02 22:39:58,594][INFO ][discovery                ] [Caliban] elasticsearch/1Cf9qz7CSCqHBEEuwG7PQw
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:40:00Z","tags":["warning","elasticsearch"],"pid":11,"message":"Unable to revive connection: http://elasticsearch:9200/"}
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:40:00Z","tags":["warning","elasticsearch"],"pid":11,"message":"No living connections"}
elasticsearch_1  | [2016-11-02 22:40:01,650][INFO ][cluster.service          ] [Caliban] new_master {Caliban}{1Cf9qz7CSCqHBEEuwG7PQw}{172.20.0.2}{172.20.0.2:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
elasticsearch_1  | [2016-11-02 22:40:01,661][INFO ][http                     ] [Caliban] publish_address {172.20.0.2:9200}, bound_addresses {[::]:9200}
elasticsearch_1  | [2016-11-02 22:40:01,661][INFO ][node                     ] [Caliban] started
elasticsearch_1  | [2016-11-02 22:40:01,798][INFO ][gateway                  ] [Caliban] recovered [1] indices into cluster_state
elasticsearch_1  | [2016-11-02 22:40:02,149][INFO ][cluster.routing.allocation] [Caliban] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.kibana][0]] ...]).
kibana_1         | {"type":"log","@timestamp":"2016-11-02T22:40:03Z","tags":["status","plugin:elasticsearch@1.0.0","info"],"pid":11,"state":"green","message":"Status changed from red to green - Kibana index ready","prevState":"red","prevMsg":"Unable to connect to Elasticsearch at http://elasticsearch:9200."}
^CGracefully stopping... (press Ctrl+C again to force)
Stopping training_kibana_1 ... done
Stopping training_elasticsearch_1 ... done

有人可以帮我解释为什么吗？

谢谢

【问题讨论】：

您是否尝试过重新启动 Kibana？当 Kibana 尝试连接到 elasticsearch 时，可能没有完全启动？
嗨@R0MANARMY。我想你是对的。您能否检查我在日志中添加的额外信息以确认。您的评论促使我添加一些额外的日志信息来说明更多关于正在发生的事情。是否可以在 docker-compose 文件中添加启动依赖项或序列？你会怎么做。

标签： elasticsearch docker docker-compose kibana

【解决方案1】：

要为 kibana 添加对 elasticsearch 的硬依赖，您需要设置 depends_on 变量，如下所示。此外，要添加到@Phil McMillan 的答案，您可以在 kibana 中设置 elasticsearch_url 变量，没有使用Docker's inbuilt DNS mechanism 进行静态寻址。

version: '2.1'
services:
     elasticsearch:
       image: docker.elastic.co/elasticsearch/elasticsearch:5.4.3
       container_name: elasticsearch
       networks:
           docker-elk:

     kibana:
       image: docker.elastic.co/kibana/kibana:5.4.3
       container_name: kibana
       environment:
          - "ELASTICSEARCH_URL=http://elasticsearch:9200"
       networks:
          - docker-elk
       depends_on:
          - elasticsearch

networks:
  docker-elk:
    driver: bridge

请注意，环境变量 ELASTICSEARCH_URL=http://elasticsearch:9200 仅使用具有 Docker DNS 服务器能够解析的容器名称 (elasticsearch)。

【讨论】：

【解决方案2】：

您需要包含links。

version: "2.0"
services:
  elasticsearch:
    image: elasticsearch:latest
    ports:
      - "9200:9200"
      - "9300:9300"
    networks:
      - docker_elk
  kibana:
    image: kibana:latest
    ports:
      - "5601:5601"
    links:
      - elasticsearch
    networks:
      - docker_elk
networks:
  docker_elk:
    driver: bridge

更新

使用镜像elasticsearch:latest时，是Elasticsearch5.0，需要我们增加我们的Docker主机虚拟内存。

在运行 docker-compose 之前，请确保在您的 Docker 主机上运行此命令。

Linux：

su root
sysctl -w vm.max_map_count=262144

Windows (boot2docker)

docker-machine ssh default
sudo sysctl -w vm.max_map_count=262144

如果您不想更改 Docker 主机，只需在 elasticsearch:2 使用 Elasticsearch 2.x 映像

【讨论】：

嗨，我试过了。但我仍然得到同样的错误。我认为这可能与在 kibana 出现之前没有完全初始化弹性搜索有关。根据@R0MANARMY 的评论。我编辑了我的原始帖子以显示更多日志，我认为这些日志可以更好地说明问题。
嗨@Tuan。我认为在同一网络上运行的服务足以让他们说话。事实上，我认为网络是链接的继承者？但你是说你需要同时使用两者吗？
@Richie，最好的检查方法是进入 kibana 容器。从docker-compose -d up 开始。通过docker exec -it training_kibana_1 /bin/bash 访问kibana，然后curl http://elasticsearch:9200/。
@Richie，您是否在 Elasticsearch max virtual memory areas vm.max_map_count [65530] likely too low, increase to at least [262144] 中收到此错误？如果是，我们应该增加 Docker 主机中的 vm.max_map_count。我更新了我的答案。
@Richie，我重新查看了您的日志。是 ES 2.4.1！我相信你有 ES 和 Kibana 的旧图像。您可能需要在运行 docker-compose 之前重新拉取它们，例如 docker pull elasticsearch 和 docker pull kibana 。

【解决方案3】：

这对我有用 docker-compose.yml

    version: '3'
    services:
      elasticsearch:
        image: docker.elastic.co/elasticsearch/elasticsearch:7.6.2
        environment:
          - discovery.type=single-node
        ports:
          - 9200:9200
      kibana:
        image: docker.elastic.co/kibana/kibana:7.6.2
        ports:
          - 5601:5601

【讨论】：

【解决方案4】：

我有这个工作。不需要链接，它与在 kibana 之前开始的 elasticsearch 没有任何关系。问题是，在 compose 下运行时，新的桥接网络是用自己的一组 IP 定义的。 Kibana 需要通过此桥接网络与集群通信 - “localhost”不再可用于连接。

你需要做几件事：

您需要在 kibana.yml 或环境下设置几个值：compose 文件中的 kibana 部分）：

一个。 kibana.yml 中的 elasticsearch.url（或环境下的 ELASTICSEARCH_URL：compose 文件中的 kibana 部分）必须设置为集群的特定 IP 和端口 9200 - localhost 将不起作用，就像在 compose 之外运行时一样.

elasticsearch.url: "http://172.16.238.10:9200"

b.您还需要将 server.host (SERVER_HOST) 设置为 Kibana 容器的桥接 IP。

server.host: "172.16.238.12"

注意：您仍然可以通过 http://127.0.0.1:5601 访问 kibana UI，并且您仍然需要那些“端口”命令！

您需要在桥接网络下设置“ipam”配置并分配 elasticsearch 和 kibana 静态 ip，以便 kibana 可以通过上面的配置访问它。

这样就足够了：

elasticsearch:
  networks:
    esnet:
      ipv4_address: 172.16.238.10
kibana:
  networks:
    esnet:
      ipv4_address: 172.16.238.12
networks:
  esnet:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 172.16.238.0/24

不要忘记使用其中一种记录在案的方法来设置 Kibana 配置 - 需要设置 ELASTICSEARCH_URL！

我有一个 docker compose 文件，它创建了两个弹性搜索节点和一个 kibana 实例，它们都在同一个桥接网络上运行。这是可能的。

【讨论】：

【解决方案5】：

我根据this post使用docker-compose v3格式：

version: '3'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.2
    container_name: elasticsearch
    environment:
      - node.name=es-node
      - cluster.name=es-cluster
      - discovery.type=single-node
    ports:
      - 9200:9200
      - 9300:9300
    volumes:
      - local-es:/usr/share/elasticsearch/data
    networks:
      - es-net

  kibana:
    image: docker.elastic.co/kibana/kibana:7.10.2
    container_name: kibana
    environment:
      - "ELASTICSEARCH_URL=http://elasticsearch:9200"
    ports:
      - 5601:5601
    networks:
      - es-net
    depends_on:
      - elasticsearch
    restart: "unless-stopped"

networks:
  es-net:

volumes:
  local-es:

【讨论】：