无法从另一个 docker 容器连接到 kafka docker 容器

Question

我正在尝试从另一个 docker 容器连接到 kafka docker 容器。但它没有连接。

有用于 kafka 消息传递的容器列表

网络 kafka-docker_default 有两个容器 kafka-docker_zookeeper_1 和 kafka-docker_kafka0_1

为了运行 kafka 和 zookeeper，我使用了 docker-compose 文件：

version: '2'
services:
  zookeeper:
    image: "confluentinc/cp-zookeeper:latest"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000

# This has three listeners you can experiment with. 
# BOB for internal traffic on the Docker network
# FRED for traffic from the Docker-host machine (`localhost`)
# ALICE for traffic from outside, reaching the Docker host on the DNS name `never-gonna-give-you-up`
# Use 
  kafka0:
    image: "confluentinc/cp-kafka"
    ports:
      - '9092:9092'
      - '29094:29094'
    depends_on:
      - zookeeper
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://kafka0:9092,LISTENER_ALICE://kafka0:29094
      KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka0:29092,LISTENER_FRED://localhost:9092,LISTENER_ALICE://never-gonna-give-you-up:29094
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_BOB:PLAINTEXT,LISTENER_FRED:PLAINTEXT,LISTENER_ALICE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_BOB
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

容器 dockercompose17138956372294708100_kafkatest.producer_1 和 dockercompose17138956372294708100_kafkatest.consumer_1 用于生产和消费消息。 虽然容器 dockercompose17138956372294708100_kafkatest.producer_1 正在尝试使用 bootstrap.servers 参数（localhost:9092 或 never-gonna-give-you-up:29094）连接到 kafka 集群，但它显示错误

%7|1569945335.015|BROKERFAIL|rdkafka#producer-1| [thrd:never-gonna-give-you-up:29094/bootstrap]: never-gonna-give-you-up:29094/bootstrap: failed: err: Local: Host resolution failure: (errno: Bad address)
%3|1569945335.015|FAIL|rdkafka#producer-1| [thrd:never-gonna-give-you-up:29094/bootstrap]: never-gonna-give-you-up:29094/bootstrap: Failed to resolve 'never-gonna-give-you-up:29094': Name or service not known (after 1656ms in state CONNECT)
%3|1569945335.015|ERROR|rdkafka#producer-1| [thrd:never-gonna-give-you-up:29094/bootstrap]: never-gonna-give-you-up:29094/bootstrap: Failed to resolve 'never-gonna-give-you-up:29094': Name or service not known (after 1656ms in state CONNECT)
%7|1569945335.015|STATE|rdkafka#producer-1| [thrd:never-gonna-give-you-up:29094/bootstrap]: never-gonna-give-you-up:29094/bootstrap: Broker changed state CONNECT -> DOWN
%3|1569945335.015|ERROR|rdkafka#producer-1| [thrd:never-gonna-give-you-up:29094/bootstrap]: 1/1 brokers are down

我该如何解决？

Answer 1

请试试这个配置：

version: '3'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    ports:
      - "32181:32181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 32181
      ZOOKEEPER_TICK_TIME: 2000

  kafka0:
    image: confluentinc/cp-kafka:latest
    ports:
     - "9090:9090"
    depends_on:
      - zookeeper
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181
      KAFKA_ADVERTISED_LISTENERS: LISTENER_INTERNAL://kafka0:29090,LISTENER_EXTERNAL://localhost:9090
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_INTERNAL:PLAINTEXT,LISTENER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_INTERNAL
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "false"

使用此配置，如果您从主机连接到代理的容器，则使用 IP localhost:9090。如果您从具有相同网络的其他容器连接，则使用kafka0:29090。

Answer 2

dockercompose17138956372294708100_kafkatest.producer_1 正在尝试使用bootstrap.servers 参数（localhost:9092 或never-gonna-give-you-up:29094）连接到 kafka 集群

来自另一个 Docker 容器，这些都不正确。

1. never-gonna-give-you-up 在 Docker 配置中不作为主机名/DNS 记录存在于任何地方。根据错误，您的外部 DNS 服务器也不知道那是什么

2. localhost 指的是 Kafka 客户端容器，而不是您的主机或代理容器。

根据您的撰写配置，您需要连接到 kafka0 主机/服务，该主机/服务在端口 29092 上为bootstrap.servers 通告

# BOB for internal traffic on the Docker network
KAFKA_ADVERTISED_LISTENERS: LISTENER_BOB://kafka0:29092

您还应该删除 LISTENER_ALICE，因为它没有提供任何有用的连接详细信息，除非您的主机的外部主机名实际上是 never-gonna-give-you-up

并且您需要将您的生产者和消费者作为您的撰写文件的一部分