kafka docker卡在配置上并且不能在mac上工作

Question

我正在尝试在 docker 上使用 kafka。但是在运行 docker compose 时，kafka 只是停留在配置模式，并不表示已经运行。我确实列出了 kafka 主题，但它不起作用或没有给出响应。这是我的 docker-compose.yml：

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    hostname: zookeeper
    ports:
      - 2181:2181
    environment:
      ZOO_MY_ID: 1
    networks:
      - kafka_net
  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
      - 9092:9092
    expose:
      - 9092
    depends_on:
      - zookeeper
    environment:
      KAFKA_ADVERTISED_HOST_NAME: localhost
      KAFKA_ADVERTISED_PORT: 9092
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:9092
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_BROKER_ID: 1
    restart: always
    networks:
      - kafka_net
networks:
  kafka_net:
    driver: "bridge"

这是 docker-compose 日志上的 zookeeper 响应：

2021-04-19 01:07:56,385 [myid:] - INFO [main:Environment@100] - Server environment:user.dir=/opt/zookeeper-3.4.13
2021-04-19 01:07:56,393 [myid:] - INFO [main:ZooKeeperServer@836] - tickTime set to 2000
2021-04-19 01:07:56,394 [myid:] - INFO [main:ZooKeeperServer@845] - minSessionTimeout set to -1
2021-04-19 01:07:56,394 [myid:] - INFO [main:ZooKeeperServer@854] - maxSessionTimeout set to -1
2021-04-19 01:07:56,418 [myid:] - INFO [main:ServerCnxnFactory@117] - Using org.apache.zookeeper.server.NIOServerCnxnFactory as server connection factory
2021-04-19 01:07:56,430 [myid:] - INFO [main:NIOServerCnxnFactory@89] - binding to port 0.0.0.0/0.0.0.0:2181

这是 docker-compose 日志上唯一的 kafka 响应：

kafka        | [Configuring] 'log.dirs' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'zookeeper.connect' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'listeners' in '/opt/kafka/config/server.properties'
kafka        | Excluding KAFKA_VERSION from broker config
kafka        | [Configuring] 'broker.id' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'listener.security.protocol.map' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'advertised.listeners' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'port' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'advertised.host.name' in '/opt/kafka/config/server.properties'
kafka        | Excluding KAFKA_HOME from broker config
kafka        | [Configuring] 'advertised.port' in '/opt/kafka/config/server.properties'
kafka        | [Configuring] 'inter.broker.listener.name' in '/opt/kafka/config/server.properties'

运行此命令时，它没有给出响应：

docker exec -it kafka /bin/sh
/ # cd /opt/kafka_2.13-2.7.0
/opt/kafka_2.13-2.7.0 # bin/kafka-topics.sh --list --zookeeper zookeeper:2181

我应该怎么做？我仍然试图让它运行。我已经尝试使用 kafka native（没有 docker）并且它运行完美，我可以创建一个主题

Answer 1

尝试使用 kafka 的另一个图像，例如 - bitnami。我在 Apple M1 Big Shur 上遇到了和你一样的问题，我使用了 bitnami，它工作正常！！这是我的 docker-compose.yml：

version: "2"

services:
    zookeeper:
        image: docker.io/bitnami/zookeeper:3
        ports:
            - "2181:2181"
        volumes:
            - "zookeeper_data:/bitnami"
        environment:
            - ALLOW_ANONYMOUS_LOGIN=yes
    kafka:
        image: docker.io/bitnami/kafka:2
        ports:
            - "9092:9092"
        volumes:
            - "kafka_data:/bitnami"
        environment:
            - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
            - ALLOW_PLAINTEXT_LISTENER=yes
            - KAFKA_LISTENERS=PLAINTEXT://:9092
            - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://127.0.0.1:9092
        depends_on:
            - zookeeper

volumes:
    zookeeper_data:
        driver: local
    kafka_data:
        driver: local

Answer 2

您似乎错过了docker-compose 日志中的一个重要部分。我在本地运行了您的 docker-compose 文件，并希望从日志中突出显示这部分。

kafka        | [2021-04-19 09:20:19,209] INFO Setting -D jdk.tls.rejectClientInitiatedRenegotiation=true to disable client-initiated TLS renegotiation (org.apache.zookeeper.common.X509Util)
kafka        | [2021-04-19 09:20:19,253] ERROR Exiting Kafka due to fatal exception (kafka.Kafka$)
kafka        | java.lang.IllegalArgumentException: requirement failed: Each listener must have a different port, listeners: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:9092
kafka        |  at kafka.utils.CoreUtils$.validate$1(CoreUtils.scala:265)
kafka        |  at kafka.utils.CoreUtils$.listenerListToEndPoints(CoreUtils.scala:276)
kafka        |  at kafka.server.KafkaConfig.$anonfun$listeners$1(KafkaConfig.scala:1680)
kafka        |  at kafka.server.KafkaConfig.listeners(KafkaConfig.scala:1679)
kafka        |  at kafka.server.KafkaConfig.validateValues(KafkaConfig.scala:1779)
kafka        |  at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1756)
kafka        |  at kafka.server.KafkaConfig.<init>(KafkaConfig.scala:1312)
kafka        |  at kafka.server.KafkaServerStartable$.fromProps(KafkaServerStartable.scala:34)
kafka        |  at kafka.Kafka$.main(Kafka.scala:68)
kafka        |  at kafka.Kafka.main(Kafka.scala)
kafka exited with code 1

因此，您使用相同的端口从 docker 网络内部和外部访问 Kafka。我更改了以下设置（INSIDE 端口），它对我有用。我能够创建和列出主题。我还能够从 kafka 容器中生成和使用简单的消息。

 KAFKA_LISTENERS: INSIDE://0.0.0.0:29092,OUTSIDE://0.0.0.0:9092
 KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:29092,OUTSIDE://localhost:9092

这是我在 kafka 容器中做的一些测试

docker exec -it kafka bash    

bash-4.4# cd /opt/kafka_2.13-2.7.0/
bash-4.4# bin/kafka-topics.sh --create --topic test-topic --partitions 16 --replication-factor 1 --zookeeper zookeeper:2181
Created topic test-topic.

bash-4.4# bin/kafka-topics.sh --list --zookeeper zookeeper:2181
test-topic

This 是一篇很棒的帖子，我想推荐给你。请让我知道这是否适合您。