Kafka Broker 日志警告无法消除 : Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch

1089 发表于: 2020-04-30   最后更新时间: 2020-04-30  

Broker日志提示大量警告

已查询该文章但未找到解决办法:https://www.orchome.com/1066

2020-04-30 16:50:37,010 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:16, offset:339490219592}, Current: {epoch:498, offset335305734105} for Partition: __consumer_offsets-47
2020-04-30 16:50:37,011 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:16, offset:339490219599}, Current: {epoch:498, offset335305734105} for Partition: __consumer_offsets-47
2020-04-30 16:50:37,011 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:14, offset:105463605}, Current: {epoch:336, offset104508994} for Partition: appDataToK3_test-12
2020-04-30 16:50:37,011 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:17, offset:92001836731}, Current: {epoch:479, offset86819324213} for Partition: __consumer_offsets-4
2020-04-30 16:50:37,012 WARN kafka.server.epoch.LeaderEpochFileCache: Received a PartitionLeaderEpoch assignment for an epoch < latestEpoch. This implies messages have arrived out of order. New: {epoch:16, offset:339490219606}, Current: {epoch:498, offset335305734105} for Partition: __consumer_offsets-47

描述

  1. kafka日常使用没有特殊操作

  2. 曾怀疑是follower 跟不上 leader 导致提示异常,查看topic ISR ,发现上面日志中 consumer_offsets-47 有报警告,但是下面查看topic 分区47的 consumer_offsets IRS是正常

     Topic: __consumer_offsets    Partition: 38    Leader: 153    Replicas: 153,154,150    Isr: 154,153
     Topic: __consumer_offsets    Partition: 39    Leader: 154    Replicas: 154,150,151    Isr: 154,151,150
     Topic: __consumer_offsets    Partition: 40    Leader: 150    Replicas: 150,152,153    Isr: 152,150,153
     Topic: __consumer_offsets    Partition: 41    Leader: 151    Replicas: 151,153,154    Isr: 151,154,153
     Topic: __consumer_offsets    Partition: 42    Leader: 152    Replicas: 152,154,150    Isr: 152,154,150
     Topic: __consumer_offsets    Partition: 43    Leader: 153    Replicas: 153,150,151    Isr: 151,153
     Topic: __consumer_offsets    Partition: 44    Leader: 154    Replicas: 154,151,152    Isr: 154,152,151
     Topic: __consumer_offsets    Partition: 45    Leader: 150    Replicas: 150,153,154    Isr: 154,150,153
     Topic: __consumer_offsets    Partition: 46    Leader: 151    Replicas: 151,154,150    Isr: 151,154,150
     Topic: __consumer_offsets    Partition: 47    Leader: 152    Replicas: 152,150,151    Isr: 152,150,151
     Topic: __consumer_offsets    Partition: 48    Leader: 153    Replicas: 153,151,152    Isr: 151,152,153
     Topic: __consumer_offsets    Partition: 49    Leader: 154    Replicas: 154,152,153    Isr: 154,152,153
    
  3. 下图是监控 JMX kafka.network:type=SocketServer,name=NetworkProcessorAvgIdlePercent 的值
    QQ截图20200430170630.png

问题:

  1. 什么问题导致该警告频繁打印,怎么能消除
  2. 如何解决 follower 跟不上leader问题,可以调整什么参数吗?


您需要解锁本帖隐藏内容请: 点击这里
本帖隐藏的内容




上一条: Windows环境consumer.poll(100)没反应
下一条: kafka C程序Producer堆积

  • num.replica.fetchers 这个可以调大试试

    这个一般是kafka一些版本的bug。
    1.1.0、2.0.0
    重新分配一下告警的分区,警告就会没有了,可参考。
    https://www.orchome.com/454#item-7

    • 好的,感谢回答,还有个问题,就是kafka正常使用的情况下
      2.如何解决 follower 跟不上leader问题,可以调整什么参数吗?

        • 节点满足这2个“同步”条件,以区分“活着”还是“故障”。leader跟踪“同步”节点。如果一个follower死掉,卡住,或落后,leader将从同步副本列表中移除它。落后是通过replica.lag.max.messages配置控制,卡住是通过replica.lag.time.max.ms配置控制的。