无法从其他容器连接到 Kafka Docker 容器：连接被拒绝 - 1/1 代理已关闭

Question

我正在尝试将数据聚合器连接到 kafka-broker 目的是从 HTTP 端点收集数据，并定期将其保存到数据库中。两者都在 docker 中运行，使用以下 docker-compose.yml：

version: "3"

services:
  zk:
    image: confluentinc/cp-zookeeper:5.0.0
    container_name: zk
    ports:
      - 2181:2181
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    restart: on-failure:5
  kafka:
    image: confluentinc/cp-kafka:5.0.0
    container_name: kafka
    ports:
      - 9092:9092
      - 9094:9094
    environment:
      KAFKA_ZOOKEEPER_CONNECT: zk:2181
      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://localhost:9094
      KAFKA_LISTENERS: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:9094
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
      KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
      KAFKA_NUM_PARTITIONS: 4
    restart: on-failure:5
  tinygator-metric-aggregator:
    image: tinygator.aggregator:latest
    container_name: tg-aggregator
    environment:
      KAFKA_BOOTSTRAP_SERVER: localhost:9094
      KAFKA_TOPIC: test
      METRIC_ENDPOINT: "http://localhost:15672/api/queues/%2F/Test?sort=message_stats.publish_details.rate&sort_reverse=true&columns=name,message_stats.publish_details.rate,message_stats.deliver_get_details.rate"
      METRIC_METHOD: GET
      METRIC_INTERVAL: 5000
      METRIC_HEADERS: '{"authorization":"Basic Z3Vlc3Q6Z3Vlc3Q="}'
    restart: on-failure:5

Kafka 和 zk 似乎可以正常启动。但是，聚合器出现以下错误：

tg-aggregator                  | %3|1576348274.043|FAIL|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 0.0.0.0:9094/bootstrap: Connect to ipv4#0.0.0.0:9094 failed: Connection refused (after 4704ms in state CONNECT)
tg-aggregator                  | %3|1576348274.048|ERROR|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 0.0.0.0:9094/bootstrap: Connect to ipv4#0.0.0.0:9094 failed: Connection refused (after 4704ms in state CONNECT)
tg-aggregator                  | %3|1576348274.048|ERROR|rdkafka#producer-1| [thrd:0.0.0.0:9094/bootstrap]: 1/1 brokers are down
tg-aggregator                  | Unhandled exception. System.Net.Http.HttpRequestException: Cannot assign requested address
tg-aggregator                  |  ---> System.Net.Sockets.SocketException (99): Cannot assign requested address
tg-aggregator                  |    at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
tg-aggregator                  |    --- End of inner exception stack trace ---
tg-aggregator                  |    at System.Net.Http.ConnectHelper.ConnectAsync(String host, Int32 port, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
tg-aggregator                  |    at System.Net.Http.HttpClient.FinishSendAsyncBuffered(Task`1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts)
tg-aggregator                  |    at MetricAggregator.Program.Main(String[] args) in /app/Program.cs:line 83
tg-aggregator                  |    at MetricAggregator.Program.<Main>(String[] args)

现在，我和一位同事已经坐了好几个小时了（我们都是 Kafka 的超级新手）。在 MacOS 上编写聚合器（可以在 https://github.com/Azer0s/TinyGator 下找到）的人不会出现此问题。我在 Windows 10 上，我的同事在 Arch 上。我们尝试了很多不同的端口号，设置Kafka监听器和广告监听器的方法等等，总之，我们完全束手无策。

有人有想法吗？

更新：

在阅读了 Kafka 监听器之后，我现在修改了我的 docker-compose，更改了聚合器尝试连接的地址：

KAFKA_BOOTSTRAP_SERVER: kafka:9092

但错误仍然相同。

Answer 1

该错误来自metrics endpoint，它在发送Kafka 消息之前 被调用。

这表明您的指标端点 URL 不正确，因为 localhost 将是容器本身（它正在提取指标，本身没有指标或暴露 HTTP 接口）

这个例子看起来像是在拉RabbitMQ queue statistics

---

无论如何，kafka:9092 在您的上下文中是正确的

编写聚合器的人不会出现此问题

如果他提供了无效的指标端点，我相信它会的