一段时间后，kafka 服务器在 Windows 环境中崩溃

Question

我目前正在尝试一些测试，其中包括 Kafka 发送和接收消息。在我的应用程序中，我有一个简单的 kafka 生产者，它定期生成消息，Kafka 接收器应用程序接收它。但是在运行此应用程序大约 36-37 小时后，Kafka 服务器因以下错误而崩溃。我也经历过this，但这似乎与我目前面临的问题没什么不同。我在本地的 Windows 机器上使用 Kafka 2.4.0。

java.nio.file.FileSystemException: C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000000000000.timeindex.cleaned -> C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000000000000.timeindex.swap: The process cannot access the file because it is being used by another process.

    at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
    at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
    at java.nio.file.Files.move(Unknown Source)
    at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:795)
    at kafka.log.AbstractIndex.renameTo(AbstractIndex.scala:209)
    at kafka.log.LogSegment.changeFileSuffixes(LogSegment.scala:497)
    at kafka.log.Log.$anonfun$replaceSegments$4(Log.scala:2286)
    at kafka.log.Log.$anonfun$replaceSegments$4$adapted(Log.scala:2286)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at kafka.log.Log.replaceSegments(Log.scala:2286)
    at kafka.log.Cleaner.cleanSegments(LogCleaner.scala:605)
    at kafka.log.Cleaner.$anonfun$doClean$6(LogCleaner.scala:530)
    at kafka.log.Cleaner.$anonfun$doClean$6$adapted(LogCleaner.scala:529)
    at scala.collection.immutable.List.foreach(List.scala:392)
    at kafka.log.Cleaner.doClean(LogCleaner.scala:529)
    at kafka.log.Cleaner.clean(LogCleaner.scala:503)
    at kafka.log.LogCleaner$CleanerThread.cleanLog(LogCleaner.scala:372)
    at kafka.log.LogCleaner$CleanerThread.cleanFilthiestLog(LogCleaner.scala:345)
    at kafka.log.LogCleaner$CleanerThread.tryCleanFilthiestLog(LogCleaner.scala:325)
    at kafka.log.LogCleaner$CleanerThread.doWork(LogCleaner.scala:314)
    at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:96)
    Suppressed: java.nio.file.FileSystemException: C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000000000000.timeindex.cleaned -> C:\kafka\temp\kafka-logs\__consumer_offsets-9\00000000000000000000.timeindex.swap: The process cannot access the file because it is being used by another process.

            at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
            at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
            at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
            at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
            at java.nio.file.Files.move(Unknown Source)
            at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:792)
            ... 17 more

仅供参考：我正在使用具有所有默认配置的 Kafka，并创建具有不同分区数的主题。

Answer 1

您需要正确停止 Kafka 和 zookeeper。如果仍然存在，那么您需要删除 C:\kafka\temp\kafka-logs\ 然后启动服务器。

您将丢失数据，并且偏移量将从 0 开始。