无法使用 docker 设置 Kafka 和 Python

Question

我可以使用 docker 设置 Kafka，但是当我尝试使用 python 访问它时，我无法做到这一点。如果我在 kafka shell 中安装 python，我可以做到这一点，但是在 kafka shell 和 docker python 之外，我无法使用 kafka。

我的Producer.py 文件：

import time
import random
from kafka import KafkaProducer

# give broker IP from docker
producer = KafkaProducer(bootstrap_servers='kafka:9092')

# continuous loop
var = 1
while var == 1:

    # generate a random integer
    num = random.randint(0, 10)

    # message value and key must be raw bytes
    num_bytes = bytes(str(num), encoding='utf-8')

    # send to topic on broker
    producer.send('test', value=num_bytes, key=num_bytes)

    # wait 1 second
    time.sleep(1)

我的consumer.py 文件：

from kafka import KafkaConsumer

# continuous loop
var = 1
while var == 1:

    # initialize consumer to given topic and broker
    consumer = KafkaConsumer('test',
                            group_id='consumer-1',
                            bootstrap_servers='kafka:9092')

    # loop and print messages
    for msg in consumer:
        print (msg)

我的docker-compose 文件：

version: '3'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: "zoo1"
    ports:
      - "2181:2181"
  kafka:
    image: wurstmeister/kafka
    container_name: "kafka1"
    ports:
     - "9092:9092"
    expose:
     - "9093"
    depends_on:
     - zookeeper
    environment:
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9093,OUTSIDE://localhost:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9093,OUTSIDE://0.0.0.0:9092
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "test:1:1"
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true'
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

Answer 1

在你的 docker.compose 中试试这个（从实际配置中复制）

environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka:9093,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL