Kafka-console-producer 返回主题超时错误答案

【问题标题】：Kafka-console-producer returns topic timeout errorKafka-console-producer 返回主题超时错误
【发布时间】：2020-08-09 06:53:13
【问题描述】：

我正在尝试使用 kafka-console-producer 生成消息。但最终出现以下错误。

$ /usr/bin/kafka-console-producer --broker-list confluent-kafka-0-service.ms-kafka-internal.svc:9092 --topic testTopic --producer.config ~/etc/kafka/client_security.properties

[2020-08-09 06:37:52,844] INFO Kafka 版本：5.4.2-ccs (org.apache.kafka.common.utils.AppInfoParser) [2020-08-09 06:37:52,845] INFO Kafka commitId: 2626d8cfb686c23e (org.apache.kafka.common.utils.AppInfoParser) [2020-08-09 06:37:52,845] INFO Kafka startTimeMs: 1596955072646 (org.apache.kafka.common.utils.AppInfoParser)

嗨欢迎来到 Confluent Kafka [2020-08-09 06:38:55,451] ERROR 向主题 testTopic 发送消息时出错，键：null，值：2 字节，错误：（org.apache.kafka.clients.producer.internals.ErrorLoggingCallback） org.apache.kafka.common.errors.TimeoutException：主题 testTopic 在 60000 毫秒后不存在于元数据中。

我已经运行了下面的消费者命令来查看消息。但随后也没有看到任何消息。

/usr/bin/kafka-console-consumer --bootstrap-server confluent-kafka-0-service.ms-kafka-internal.svc:9092 --topic testTopic --from-beginning --consumer.config ~/etc/bmw/kafka/client_security.properties

任何即时帮助将不胜感激。

谢谢，米娜

【问题讨论】：

标签： apache-kafka confluent-platform producer-consumer consumer producer

【解决方案1】：

请注意，您的生产者和消费者指向不同的路径：

--consumer.config ~/etc/bmw/kafka/client_security.properties

对

--producer.config ~/etc/kafka/client_security.propertie

也许你只是得到了错误的配置文件路径？

【讨论】：

嗨奥列格感谢您的回复。下面是我正在使用的命令。但随后以这个错误结束：在生产者命令之后：我没有名字！@confluent-kafka-0:/$ kafka-console-producer --broker-list confluent-kafka-0-service.ms-kafka-internal。 svc:9092 --topic testTopic --producer.config ~/etc/bmw/kafka/client_security.properties ERROR 向主题 testTopic 发送消息时出错，key:null，value:6 bytes with error:(org.apache.kafka. clients.producer.internals.ErrorLoggingCallback) org.apache.kafka.common.errors.TimeoutException: 60000 毫秒后元数据中不存在主题 testTopic。
嗨 Mina，你可以尝试 ping Kafka 代理吗？您还可以分享创建您正在使用的“testTopic”的命令吗？
嗨，我在 Openshift 上使用 Confluent Kafka，我无法选择 ping 或 ssh 到代理。在 Openshift 之外，我使用 oc 命令。下面是我用于创建主题的命令： kafka-topics --create --zookeeper localhost:2181 --replication-factor 3 --partitions 3 --topic testTopic
所以 Zookeeper 在您的机器上本地运行，但 Kafka 代理在 Openshift 上运行？
没有。 Zookeeper 也在同一个 openshift 命名空间中。

【解决方案2】：

This is the entire Content of Producer command.
I have no name!@confluent-kafka-0:/$ kafka-console-producer --broker-list confluent-kafka-0-service.ms-kafka-internal.svc:9092 --topic testTopic --producer.config ~/etc/bmw/kafka/client_security.properties
[2020-08-10 09:46:00,186] INFO Registered kafka:type=kafka.Log4jController MBean (kafka.utils.Log4jControllerRegistration$)
[2020-08-10 09:46:03,389] INFO ProducerConfig values:
        acks = 1
        batch.size = 16384
        bootstrap.servers = [confluent-kafka-0-service.ms-kafka-internal.svc:9092]
        buffer.memory = 33554432
        client.dns.lookup = default
        client.id = console-producer
        compression.type = none
        connections.max.idle.ms = 540000
        delivery.timeout.ms = 120000
        enable.idempotence = false
        interceptor.classes = []
        key.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
        linger.ms = 1000
        max.block.ms = 60000
        max.in.flight.requests.per.connection = 5
        max.request.size = 1048576
        metadata.max.age.ms = 300000
        metric.reporters = []
        metrics.num.samples = 2
        metrics.recording.level = INFO
        metrics.sample.window.ms = 30000
        partitioner.class = class org.apache.kafka.clients.producer.internals.DefaultPartitioner
        receive.buffer.bytes = 32768
        reconnect.backoff.max.ms = 1000
        reconnect.backoff.ms = 50
        request.timeout.ms = 1500
        retries = 3
        retry.backoff.ms = 100
        sasl.client.callback.handler.class = null
        sasl.jaas.config = null
        sasl.kerberos.kinit.cmd = /usr/bin/kinit
        sasl.kerberos.min.time.before.relogin = 60000
        sasl.kerberos.service.name = null
        sasl.kerberos.ticket.renew.jitter = 0.05
        sasl.kerberos.ticket.renew.window.factor = 0.8
        sasl.login.callback.handler.class = null
        sasl.login.class = null
        sasl.login.refresh.buffer.seconds = 300
        sasl.login.refresh.min.period.seconds = 60
        sasl.login.refresh.window.factor = 0.8
        sasl.login.refresh.window.jitter = 0.05
        sasl.mechanism = GSSAPI
        security.protocol = SSL
        security.providers = null
        send.buffer.bytes = 102400
        ssl.cipher.suites = null
        ssl.enabled.protocols = [TLSv1.2, TLSv1.1, TLSv1]
        ssl.endpoint.identification.algorithm = https
        ssl.key.password = [hidden]
        ssl.keymanager.algorithm = SunX509
        ssl.keystore.location = /etc/bmw/kafka/client_certs/kafka-client-internal-keystore.jks
        ssl.keystore.password = [hidden]
        ssl.keystore.type = JKS
        ssl.protocol = TLS
        ssl.provider = null
        ssl.secure.random.implementation = null
        ssl.trustmanager.algorithm = PKIX
        ssl.truststore.location = /etc/bmw/kafka/bmw_certs/bmw-truststore.jks
        ssl.truststore.password = [hidden]
        ssl.truststore.type = JKS
        transaction.timeout.ms = 60000
        transactional.id = null
        value.serializer = class org.apache.kafka.common.serialization.ByteArraySerializer
 (org.apache.kafka.clients.producer.ProducerConfig)
[2020-08-10 09:46:11,786] INFO Kafka version: 5.4.2-ccs (org.apache.kafka.common.utils.AppInfoParser)
[2020-08-10 09:46:11,786] INFO Kafka commitId: 2626d8cfb686c23e (org.apache.kafka.common.utils.AppInfoParser)
[2020-08-10 09:46:11,786] INFO Kafka startTimeMs: 1597052771592 (org.apache.kafka.common.utils.AppInfoParser)
>Om namo Narayanaya nama
Jai Sri ram
[2020-08-10 09:47:21,186] ERROR Error when sending message to topic testTopic with key: null, value: 23 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic testTopic not present in metadata after 60000 ms.
>^C[2020-08-10 09:47:25,485] INFO [Producer clientId=console-producer] Closing the Kafka producer with timeoutMillis = 9223372036854775807 ms. (org.apache.kafka.clients.producer.KafkaProducer)
org.apache.kafka.common.KafkaException: Producer closed while send in progress
        at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:888)
        at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:865)
        at kafka.tools.ConsoleProducer$.send(ConsoleProducer.scala:75)
        at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:57)
        at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.KafkaException: Requested metadata update after close
        at org.apache.kafka.clients.producer.internals.ProducerMetadata.awaitUpdate(ProducerMetadata.java:104)
        at org.apache.kafka.clients.producer.KafkaProducer.waitOnMetadata(KafkaProducer.java:1029)
        at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:885)`

Thanks,
Mina

【讨论】：