【问题标题】:Not Able To Access Kafka Services Outside Docker Image无法访问 Docker Image 之外的 Kafka 服务
【发布时间】:2018-06-28 05:42:19
【问题描述】:

我在 centos 上创建了一个 kafka docker 镜像。在这里,我在同一个镜像上同时运行 Zookeeper 和 Kafka 服务器。

我看到服务在 docker 容器中启动并运行。我通过Kafka提供的kafka-console-producer.sh和kafka-console-consumer.sh脚本测试了kafka。所需的端口也暴露了。

PORTS
0.0.0.0:2182->2182/tcp, 22/tcp, 0.0.0.0:9093->9093/tcp

以下是在Kafka的server.properties做的配置:

listeners=PLAINTEXT://0.0.0.0:9093
zookeeper.connect=localhost:2182

我在 docker 容器中创建了一个主题。

我可以在运行 docker 映像的主机上使用 telnet 命令通过外部机器(在同一网络中)访问 Kafka 服务。

telnet 9093
Trying …
Connected to .
Escape character is ‘^]’.

telnet 2182
Trying …
Connected to .
Escape character is ‘^]’.

但是,将数据写入 Kafka 主题失败并出现 TimeoutExceptions:

2017-12-17 21:30:51 DEBUG NetworkClient:195 - [Producer clientId=KafkaExampleProducer] Using older server API v0 to send API_VERSIONS {} with correlation id 1 to node -1 
2017-12-17 21:30:51 DEBUG NetworkClient:189 - [Producer clientId=KafkaExampleProducer] Recorded API versions for node -1: (Produce(0): 0 to 2 [usable: 2], Fetch(1): 0 to 2 [usable: 2], ListOffsets(2): 0 [usable: 0], Metadata(3): 0 to 1 [usable: 1], LeaderAndIsr(4): 0 [usable: 0], StopReplica(5): 0 [usable: 0], UpdateMetadata(6): 0 to 2 [usable: 2], ControlledShutdown(7): 1 [usable: 1], OffsetCommit(8): 0 to 2 [usable: 2], OffsetFetch(9): 0 to 1 [usable: 1], FindCoordinator(10): 0 [usable: 0], JoinGroup(11): 0 [usable: 0], Heartbeat(12): 0 [usable: 0], LeaveGroup(13): 0 [usable: 0], SyncGroup(14): 0 [usable: 0], DescribeGroups(15): 0 [usable: 0], ListGroups(16): 0 [usable: 0], SaslHandshake(17): 0 [usable: 0], ApiVersions(18): 0 [usable: 0], CreateTopics(19): UNSUPPORTED, DeleteTopics(20): UNSUPPORTED, DeleteRecords(21): UNSUPPORTED, InitProducerId(22): UNSUPPORTED, OffsetForLeaderEpoch(23): UNSUPPORTED, AddPartitionsToTxn(24): UNSUPPORTED, AddOffsetsToTxn(25): UNSUPPORTED, EndTxn(26): UNSUPPORTED, WriteTxnMarkers(27): UNSUPPORTED, TxnOffsetCommit(28): UNSUPPORTED, DescribeAcls(29): UNSUPPORTED, CreateAcls(30): UNSUPPORTED, DeleteAcls(31): UNSUPPORTED, DescribeConfigs(32): UNSUPPORTED, AlterConfigs(33): UNSUPPORTED, AlterReplicaLogDirs(34): UNSUPPORTED, DescribeLogDirs(35): UNSUPPORTED, SaslAuthenticate(36): UNSUPPORTED, CreatePartitions(37): UNSUPPORTED)
2017-12-17 21:30:51 DEBUG NetworkClient:189 - [Producer clientId=KafkaExampleProducer] Sending metadata request (type=MetadataRequest, topics=sifs.email.in) to node <IP>:9093 (id: -1 rack: null) 
2017-12-17 21:30:51 DEBUG NetworkClient:195 - [Producer clientId=KafkaExampleProducer] Using older server API v1 to send METADATA {topics=[sifs.email.in]} with correlation id 2 to node -1 
2017-12-17 21:30:52 DEBUG Metadata:270 - Updated cluster metadata version 2 to Cluster(id = null, nodes = [0.0.0.0:9093 (id: 0 rack: null)], partitions = [Partition(topic = sifs.email.in, partition = 0, leader = 0, replicas = [0], isr = [0], offlineReplicas = [])]) 
2017-12-17 21:30:52 DEBUG NetworkClient:183 - [Producer clientId=KafkaExampleProducer] Initiating connection to node 0.0.0.0:9093 (id: 0 rack: null) org.apache.kafka.common.errors.TimeoutException: Expiring 50 record(s) for sifs.email.in-0: 55017 ms has passed since batch creation plus linger time org.apache.kafka.common.errors.TimeoutException: Expiring 50 record(s) for sifs.email.in-0: 55017 ms has passed since batch creation plus linger time org.apache.kafka.common.errors.TimeoutException: Expiring 50 record(s) for sifs.email.in-0: 55017 ms has passed since batch creation plus linger time org.apache.kafka.common.errors.TimeoutException: Expiring 50 record(s) for sifs.email.in-0: 55017 ms has passed since batch creation plus linger time org.apache.kafka.common.errors.TimeoutException: Expiring 50 record(s) for sifs.email.in-0: 55017 ms has passed since batch creation plus linger time org.apache.kafka.common.errors.TimeoutException: Expiring 50 record(s) for sifs.email.in-0: 55017 ms has passed since batch creation plus linger time 
2017-12-17 21:31:47 INFO  KafkaProducer:341 - [Producer clientId=KafkaExampleProducer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. 
2017-12-17 21:31:47 DEBUG Sender:177 - [Producer clientId=KafkaExampleProducer] Beginning shutdown of Kafka producer I/O thread, sending remaining records.

让我知道如何从外部机器将数据写入 Kafka 主题。

【问题讨论】:

    标签: docker apache-kafka


    【解决方案1】:

    这些步骤解决了我从 docker 容器外部连接的问题:

    1. 已部署 Docker 容器(带有 zookeeper 的 kafka 0.11):

    https://stackoverflow.com/a/51071716/2493852

    1. 从容器内外测试连接:

    https://stackoverflow.com/a/51071629/2493852

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2019-06-18
      • 1970-01-01
      • 2019-11-15
      • 2019-04-14
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多