kafka replication-offset-checkpoint文件读取错误FileAlreadyExistsException

W 发表于: 2021-09-23   最后更新时间: 2021-09-23 18:49:11   3,482 游览

window部署单机kafka和zookeeper,运行一段时间后,自动挂掉,提示错误信息。

尝试修改配置的分区数,修改文件夹权限,重新启动能运行一段时间,但是都会崩掉。

kafka

错误信息如下:

[2021-09-23 15:37:12,490] ERROR [ReplicaManager broker=0] Error while writing to highwatermark file in directory C:\kafka\kafka\kafka_2.12-2.3.0-SNAPSHOT\.\klog (kafka.server.ReplicaManager)
org.apache.kafka.common.errors.KafkaStorageException: Error while writing to checkpoint file C:\kafka\kafka\kafka_2.12-2.3.0-SNAPSHOT\.\klog\replication-offset-checkpoint
Caused by: java.nio.file.FileAlreadyExistsException: C:\kafka\kafka\kafka_2.12-2.3.0-SNAPSHOT\.\klog\replication-offset-checkpoint.tmp -> C:\kafka\kafka\kafka_2.12-2.3.0-SNAPSHOT\.\klog\replication-offset-checkpoint
    at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
    at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
    at java.nio.file.Files.move(Unknown Source)
    at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:815)
    at kafka.server.checkpoints.CheckpointFile.liftedTree1$1(CheckpointFile.scala:72)
    at kafka.server.checkpoints.CheckpointFile.write(CheckpointFile.scala:50)
    at kafka.server.checkpoints.OffsetCheckpointFile.write(OffsetCheckpointFile.scala:59)
    at kafka.server.ReplicaManager.$anonfun$checkpointHighWatermarks$9(ReplicaManager.scala:1428)
    at kafka.server.ReplicaManager.$anonfun$checkpointHighWatermarks$9$adapted(ReplicaManager.scala:1428)
    at scala.Option.foreach(Option.scala:274)
    at kafka.server.ReplicaManager.$anonfun$checkpointHighWatermarks$7(ReplicaManager.scala:1428)
    at kafka.server.ReplicaManager.$anonfun$checkpointHighWatermarks$7$adapted(ReplicaManager.scala:1425)
    at scala.collection.TraversableLike$WithFilter.$anonfun$foreach$1(TraversableLike.scala:792)
    at scala.collection.immutable.Map$Map1.foreach(Map.scala:128)
    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:791)
    at kafka.server.ReplicaManager.checkpointHighWatermarks(ReplicaManager.scala:1425)
    at kafka.server.ReplicaManager.$anonfun$startHighWaterMarksCheckPointThread$1(ReplicaManager.scala:259)
    at kafka.utils.KafkaScheduler.$anonfun$schedule$2(KafkaScheduler.scala:114)
    at kafka.utils.CoreUtils$$anon$1.run(CoreUtils.scala:65)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown Source)
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Suppressed: java.nio.file.AccessDeniedException: C:\kafka\kafka\kafka_2.12-2.3.0-SNAPSHOT\.\klog\replication-offset-checkpoint.tmp -> C:\kafka\kafka\kafka_2.12-2.3.0-SNAPSHOT\.\klog\replication-offset-checkpoint
    at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
    at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
    at sun.nio.fs.WindowsFileCopy.move(Unknown Source)
    at sun.nio.fs.WindowsFileSystemProvider.move(Unknown Source)
    at java.nio.file.Files.move(Unknown Source)
    at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:812)
    ... 22 more
发表于 2021-09-23
W
  • 你是如何修改的配置的分区数,和修改文件夹权限的?kafka启动的权限是否匹配?半兽人 2年前 回复
    @半兽人 修改文件夹权限是:window系统添加everyone权限,完全控制 配置的分区数的话,创建topic 的时候创建了一个分区,然后配置文件中num.partitions=1 kafka启动的权限是在文件夹中打开cmd命令行,然后运行.\bin\windows\kafka-server-start.bat .\config\server.properties的W 回复
添加评论

刚才检索了一下,这个是windows系统的一个bug,需要升级:

我不太确定你的系统版本和kafka的版本,你可以参考一下相关链接:

https://community.microstrategy.com/s/article/Windows-Kafka-could-not-be-started-due-to-file-system-errors-in-MicroStrategy-2019-Update-2-and-up?language=en_US

https://www.ibm.com/support/pages/apar/JR60532

https://www.ibm.com/support/pages/node/878310

摘要:

如果你使用的是MicroStrategy 2019 Update 1或旧版本,请查看KB441447,并更新至MicroStrategy Update 2或新版本。这包含一个修复程序,并解决了大多数受影响系统的问题。

如果你使用的是MicroStrategy 2019 Update 2及以上版本,包括MicroStrategy 2020,并且仍然遇到这个问题,请联系MicroStrategy技术支援。

大哥,这个问题最终时怎么解决的?

你的答案

查看kafka相关的其他问题或提一个您自己的问题