每当我在 docker 容器中获得 spark 工作时，都会断开 Bootstrap 代理 localhost:9092 (id: -1 rack: null)

Question

我得到的错误是：-

20/11/02 13:34:51 WARN NetworkClient: [Consumer clientId=consumer-spark-kafka-source-366ac503-c5a4-4338-869c-84786983aab3--188679505-driver-0-1, groupId=spark-kafka-source-366ac503-c5a4-4338-869c-84786983aab3--188679505-driver-0] Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected

我的 docker-compose.yml 文件

 version: "3"
services:
  zookeeper:
    image: wurstmeister/zookeeper
    container_name: zookeeper
    ports:
    - 2181:2181

  kafka:
    image: wurstmeister/kafka
    container_name: kafka
    ports:
    - 9092:9092
    
    environment:
      KAFKA_LISTENERS: PLAINTEXT://:9092
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_CREATE_TOPICS: "kafkatutorial:1:1"
      ALLOW_PLAINTEXT_LISTENER: "yes"
      SSL: localhost:9092
    expose:
     - 9092
      
  spark:
   build:
    dockerfile: DockerFileSpark
    context: .

   
   environment:
      - SPARK_MODE=master
      - SPARK_RPC_AUTHENTICATION_ENABLED=no
      - SPARK_RPC_ENCRYPTION_ENABLED=no
      - SPARK_LOCAL_STORAGE_ENCRYPTION_ENABLED=no
      - SPARK_SSL_ENABLED=no
   
   ports:
      - '8080:8080'
   links:
    - kafka
   
   depends_on:
    - zookeeper
    - kafka
    

Answer 1

1. SSL: localhost:9092 不是 kafka 容器的有效环境变量；您需要在两个侦听器变量中设置 SSL并将 ssl 证书添加到图像

2. 不清楚您的 Spark 代码在哪里运行，但在容器内，localhost 将永远无法连接到外部服务。在你的 yaml 中删除 links 并在代码中连接到 kafka:9092