kafka单个broker里多个topic的isc缺失

一方メ通行 发表于: 2022-04-26   最后更新时间: 2022-04-27 02:04:17   985 游览

缺失isc的broker-2错误日志

ERROR [ReplicaManager broker=2] Error processing fetch with max size 10485760 from replica [1] on partition AAAAAAAAAAAAAAAAAAAAAA:consumer_offsets-30: PartitionData(topicId=AAAAAAAAAAAAAAAAAAAAAA, fetchOffset=4577, logStartOffset=0, maxBytes=100000000, currentLeaderEpoch=Optional[1], lastFetchedEpoch=Optional[1]) (kafka.server.ReplicaManager) [data-plane-kafka-request-handler-15]
org.apache.kafka.common.errors.CorruptRecordException: Found record size 0 smaller than minimum record overhead (14) in file /var/lib/kafka/data-0/kafka-log2/
consumer_offsets-30/00000000000000000000.log.

其它broker的错误日志

ERROR [ReplicaFetcher replicaId=1, leaderId=2, fetcherId=7] Error for partition __consumer_offsets-30 at offset 4577 (kafka.server.ReplicaFetcherThread) [ReplicaFetcherThread-7-2]
org.apache.kafka.common.errors.CorruptRecordException: This message has failed its CRC checksum, exceeds the valid size, has a null key for a compacted topic, or is otherwise corrupt.

想问下这种问题如何解决,节点全部重启也无法恢复,出现问题的topic都在broker2节点上有缺失isc的情况。

发表于 2022-04-26
添加评论

你开启了压缩

--config cleanup.policy=compact

引起的,因为需要压缩的topic,消息的key不能为空。

你的答案

查看kafka相关的其他问题或提一个您自己的问题