【问题标题】:Kafka is losing message when ack is oneKafka is losing message when ack is one
【发布时间】:2022-11-20 20:53:13
【问题描述】:

If we produce messages with ack=1 and a consumer is reading the messages, we are seeing that the consumer is losing messages(not processing all the messages). For example if I produce 50 messages, the consumer is only seeing 35 messages. I am not able to understand why it is happening.

【问题讨论】:

  • Need more info, such as partitions, consumer groups, retries on producer and consumer...
  • Hi Song, I think the cluster is doing a follower-fetch. I am trying to get more configuration details for the cluster and will confirm.
  • Checked this, client.rack is null so not sure why it is still behaving like this.

标签: apache-kafka confluent-platform


【解决方案1】:

Yes ack=1 can lost message

There is a scenario when the producer received ack, message was saved in leader, but just after the ACK the leader broker crashes and does not start and because other brokers (replicas) still don’t have the message while waiting a new leader is elected then the message is gone.

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 2015-05-01
    • 2022-12-01
    • 2013-10-31
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2022-12-27
    • 1970-01-01
    相关资源
    最近更新 更多