如何使用docker在kafka中拥有多个集群？答案

【问题标题】：How to have multiple cluster in kafka by using docker?如何使用docker在kafka中拥有多个集群？
【发布时间】：2022-01-21 10:20:16
【问题描述】：

我正在尝试使用 docker 在 kafka 中建立具有三个节点的多个集群，我在 docker-compose 文件中遵循了这种方法，但这是错误的，我无法启动我在 docker-compose 文件中编写的那些 Kafka 请告诉我真正的 docker-compose 文件或使用 docker 在 Kafka 中建立多个集群（具有三个节点）的真正方法

version: '2'
services:
  zookeeper-1:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 1
      ZOOKEEPER_CLIENT_PORT: 22101
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS:      localhost:22000:23000;localhost:32000:33000;localhost:42000:43000
    network_mode: host
      extra_hosts:
        - "moby:127.0.0.1"

  zookeeper-2:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 2
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

  zookeeper-3:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_SERVER_ID: 3
      ZOOKEEPER_CLIENT_PORT: 42181
      ZOOKEEPER_TICK_TIME: 2000
      ZOOKEEPER_INIT_LIMIT: 5
      ZOOKEEPER_SYNC_LIMIT: 2
      ZOOKEEPER_SERVERS: localhost:22888:23888;localhost:32888:33888;localhost:42888:43888
    network_mode: host
    extra_hosts:
      - "moby:127.0.0.1"

kafka-1:
  image: confluentinc/cp-zookeeper:latest
  network_mode: host
  depends_on:
    - zookeeper-1
    - zookeeper-2
    - zookeeper-3
  environment:
  KAFKA_BROKER_ID: 1
  KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
  KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:19092
kafka-2:
   image: confluentinc/cp-zookeeper:latest
   network_mode: host
   depends_on:
     - zookeeper-1
     - zookeeper-2
     - zookeeper-3
   environment:
   KAFKA_BROKER_ID: 2
   KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
   KAFKA_ADVERTISED_LISTENER: PLAINTEXT://localhost:29092
 extra_hosts:
   - "moby:127.0.0.1"
kafka-3:
  image: confluentinc/cp-zookeeper:latest
  network_mode: host
  depends_on:
    - zookeeper-1
    - zookeeper-2
    - zookeeper-3
  environment:
    KAFKA_BROKER_ID: 3
    KAFKA_ZOOKEEPER_CONNECT: localhost:22181,localhost:32181,localhost:42181
    KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:39092
extra_hosts:
  - "moby:127.0.0.1"

【问题讨论】：

标签： docker apache-kafka docker-compose

【解决方案1】：

您的配置错误：每个 zk 节点都有一个进程（名为 QuorumPeerMain），它在默认端口 2181 上侦听客户端，并在端口 2888:3888 上相互通信。因此，当尝试定义 zookeeper quorum 时，将其定义为 zk1:[port],zk2:[port],...

 zookeeper-1:
      ZOOKEEPER_CLIENT_PORT: 2181
...
...
 zookeeper-2:
      ZOOKEEPER_CLIENT_PORT: 2181
...
...
 zookeeper-3:
      ZOOKEEPER_CLIENT_PORT: 2181

在 Kafka 属性中：

KAFKA_ZOOKEEPER_CONNECT: zookeeper-1:2181,zookeeper-2:2181,zookeeper-3:2181

点击此链接：Guide to Setting Up Apache Kafka Using Docker

【讨论】：

Zookeeper 属性很好。它们都被覆盖了，因为设置了network_mode: host

【解决方案2】：

我通过以下链接解决了我的问题：

https://github.com/conduktor/kafka-stack-docker-compose

我的问题是关于如何使用 docker-compose 在 Kafka 中拥有多个节点我使用了这个项目：Single Zookeeper / Multiple Kafka，来自那个存储库并运行了这个命令来使用它：

docker-compose -f zk-single-Kafka-single.yml up

然后，您可以使用此命令检查哪些容器已启动：

sudo docker ps

输出的名称是：

kafka-stack-docker-compose_kafka2_1

kafka-stack-docker-compose_kafka3_1

kafka-stack-docker-compose_kafka1_1

动物园1

他们的 ID 和 STATUS 以及 ...

【讨论】：