问下,大佬,怎么知道自己的消费者已经连上kafka呢?

scattered and scattered 发表于: 2020-02-21   最后更新时间: 2021-03-30 14:30:36   4,332 游览

场景是这样的,客户给了我们消费主题,然后ip地址,我去获取了,但是却停在获取数据那了,有显示
一直在去拉取数据,可是没有数据拉取到,我也不敢问客户是不是没有生产数据,导致拉不到数据,怎么知道已经对接上他们的生产者了呢????求大佬帮忙

发表于 2020-02-21

你找台能访问他们kafka集群的机器,装一个kafka,用命令执行查看,地址连他们的kafka就可以了。

大佬,命令执行是执行消费者的命令吗?

‘您的意思是我安装kafka,然后需要运行kafka吗?还是只要启动消费者就可以呢

不需要运行,命令大全里,有查看消费者组的状态。
https://www.orchome.com/454

老大,不需要启动kafka吗?我今天试了一天不成功哎

大佬,我报错了

$ bin/kafka-consumer-groups.sh --bootstrap-server xxxxxxx:9092 --list
Error: Executing consumer group command failed due to org.apache.kafka.common.KafkaException: Failed to find brokers to send ListGroups
java.util.concurrent.ExecutionException: org.apache.kafka.common.KafkaException: Failed to find brokers to send ListGroups
        at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
        at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
        at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
        at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:262)
        at kafka.admin.ConsumerGroupCommand$ConsumerGroupService.listGroups(ConsumerGroupCommand.scala:132)
        at kafka.admin.ConsumerGroupCommand$.main(ConsumerGroupCommand.scala:58)
        at kafka.admin.ConsumerGroupCommand.main(ConsumerGroupCommand.scala)
Caused by: org.apache.kafka.common.KafkaException: Failed to find brokers to send ListGroups
        at org.apache.kafka.clients.admin.KafkaAdminClient$22.handleFailure(KafkaAdminClient.java:2610)
        at org.apache.kafka.clients.admin.KafkaAdminClient$Call.fail(KafkaAdminClient.java:614)
        at org.apache.kafka.clients.admin.KafkaAdminClient$TimeoutProcessor.handleTimeouts(KafkaAdminClient.java:730)
        at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.timeoutPendingCalls(KafkaAdminClient.java:798)
        at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1092)
        at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.

你的ip和端口正确吗

应该是对的,大佬,我用访问的那台服务器,启动消费者去链接报了这个,这个是???

[2020-02-25 10:56:37,138] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 2 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:37,421] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 9 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:37,543] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 12 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:37,660] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 13 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:37,777] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 14 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:37,900] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 15 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:38,017] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 16 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:38,162] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 17 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)
[2020-02-25 10:56:38,279] WARN [Consumer clientId=consumer-1, groupId=test-consumer-group] Error while fetching metadata with correlation id 19 : {loc-trade_p-088--from-beginning=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient)

topic不存在。

bin/kafka-topics.sh --list --bootstrap-server xxxxx:9092

这条命令能查出来吗?(ps:ip和端口改成客户提供的)。

大佬,还是报错了,

Exception in thread "main" joptsimple.UnrecognizedOptionException: bootstrap-server is not a recognized option
        at joptsimple.OptionException.unrecognizedOption(OptionException.java:108)
        at joptsimple.OptionParser.handleLongOptionToken(OptionParser.java:510)
        at joptsimple.OptionParserState$2.handleArgument(OptionParserState.java:56)
        at joptsimple.OptionParser.parse(OptionParser.java:396)
        at kafka.admin.TopicCommand$TopicCommandOptions.(TopicCommand.scala:361)
        at kafka.admin.TopicCommand$.main(TopicCommand.scala:44)
        at kafka.admin.TopicCommand.main(TopicCommand.scala)

你的kafka什么版本?看着是老版本那。
找个和客户的版本对应的?

大佬,已经解决了

我是直接问客户什么消费组名以及版本,然后改成一致的就可以了

怎么解决的 我也遇到了 求助

你的答案

查看kafka相关的其他问题或提一个您自己的问题