kafka扩容副本后,消费者无法消费

小夕夕 发表于: 2019-09-27   最后更新时间: 2019-09-27  

原始集群信息:

3个zk,2个kafka,自建topic为dsArchiveQueue,该topic设置2个分区,1个副本。

昨天对该topic扩容成了2个副本,且各项命令提示正常,停掉其中一台kafka后消费者无法正常消费。

今天对__consumer_offsets也进行了副本扩容,但停掉kafka后,消费者还是无法正常消费。

当前信息如下:

    Topic:dsArchiveQueue    PartitionCount:2    ReplicationFactor:2    Configs:
    Topic: dsArchiveQueue    Partition: 0    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: dsArchiveQueue    Partition: 1    Leader: 1    Replicas: 1,2    Isr: 2,1

    Topic:__consumer_offsets    PartitionCount:50    ReplicationFactor:2    Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
    Topic: __consumer_offsets    Partition: 0    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 1    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 2    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 3    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 4    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 5    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 6    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 7    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 8    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 9    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 10    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 11    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 12    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 13    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 14    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 15    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 16    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 17    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 18    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 19    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 20    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 21    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 22    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 23    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 24    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 25    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 26    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 27    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 28    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 29    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 30    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 31    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 32    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 33    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 34    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 35    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 36    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 37    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 38    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 39    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 40    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 41    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 42    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 43    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 44    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 45    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 46    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 47    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 48    Leader: 1    Replicas: 1,2    Isr: 2,1
    Topic: __consumer_offsets    Partition: 49    Leader: 1    Replicas: 1,2    Isr: 2,1


您需要解锁本帖隐藏内容请: 点击这里
本帖隐藏的内容




上一条: kafka集群配置,但是tps上不去,但是不知道是什么原因
下一条: kafka报错 :org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 704708608 larger than 524288)

  • 你的配置正确无误...