kafka 集群consumer 收不到消息

芣杺~汸蔠 发表于: 2019-09-20   最后更新时间: 2019-09-21  

搭建了 3个zookeeper 伪分布式集群,kafka 也是三个broker,现在查看 _consumer_offset 是这样的,消费者收不到消息:

 Topic:__consumer_offsets    PartitionCount:50    ReplicationFactor:1    Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
    Topic: __consumer_offsets    Partition: 0    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 1    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 2    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 3    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 4    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 5    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 6    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 7    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 8    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 9    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 10    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 11    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 12    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 13    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 14    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 15    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 16    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 17    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 18    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 19    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 20    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 21    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 22    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 23    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 24    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 25    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 26    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 27    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 28    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 29    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 30    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 31    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 32    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 33    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 34    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 35    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 36    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 37    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 38    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 39    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 40    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 41    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 42    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 43    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 44    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 45    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 46    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 47    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 48    Leader: 0    Replicas: 0    Isr: 0
    Topic: __consumer_offsets    Partition: 49    Leader: 0    Replicas: 0    Isr: 0


您需要解锁本帖隐藏内容请: 点击这里
本帖隐藏的内容




上一条: kafka V2.1.1 poll 方法过时
下一条: kafka offset.channel.backoff.ms 问题

  • 看这个是正常的,您可以先通过命令同时生产和消费,确认是否正常消费。
    可参考:https://www.orchome.com/454

    另外,如果还不行,得详细描述下你的操作步骤,不然我没办法帮你分析。

    • 但是我明明搭建的是3个kafka的集群,但isr副本为什么只有一个,replica也是一个,不是会有另外两个kafka吗
      现在我看只是broker=0的kafka服务起作用
      另外我关掉一台kafak服务后,kafka消费端一直报服务不能用

        • 1、3个kafka集群,但你需要在创建主题的时候,指定副本的数量,默认是1个。(你可以通过上面命令大全增加副本数)
          2、__consumer_offsets 这个主题一定要扩展,这个是存储offset的,不扩展,0宕机消费者就全不能用了。

            • 换个方式问吧!如果我想搭建kafka集群,相应的搭建 zookeeper 集群 来管理这个kafka 集群,则在配置过程中哪一项参数,保证了副本的耐用性,并且保证kafka集群的正常通信