关于KAFKA11.0.1使用SASL_PLAINTEXT遇到的问题

+ 发表于: 2017-11-14   最后更新时间: 2017-11-14 15:49:20   16,336 游览

完成SASL之后,生产和消费的时候就报如下错误,请大神们帮忙,快疯了

消费者命令:./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
生产者命令:./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test

[2017-11-14 15:29:54,651] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:54,706] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:54,810] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:54,863] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:54,916] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:54,969] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,073] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,127] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,180] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,283] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,335] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,388] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,440] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,493] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,595] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,649] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,752] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,805] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,857] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:55,960] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:56,012] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)
[2017-11-14 15:29:56,064] WARN Bootstrap broker localhost:9092 (id: -1 rack: null) disconnected (org.apache.kafka.clients.NetworkClient)

server.properties配置如下:

port=9092
listeners=SASL_PLAINTEXT://localhost:9092
security.inter.broker.protocol=SASL_PLAINTEXT
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
super.users=User:admin

kafka_server_jaas.conf配置如下:

sasl_plaintext.KafkaServer {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin-pwd"
        user_admin="admin-pwd"
        user_bob="bob-pwd";
};

Client {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="admin"
        password="admin-pwd";
};

kafka_client_jaas.conf配置如下:

Client {
        org.apache.kafka.common.security.plain.PlainLoginModule required
        username="bob"
        password="bob-pwd";
};
发表于 2017-11-14
+
添加评论

https://www.orchome.com/500
先看下这个。你的执行命令没有带证书。

+ -> 半兽人 6年前

plaintext也要证书么,还是是这个export KAFKA_OPTS="-Djava.security.auth.login.config=/usr/kafka2/kafka_2.11-0.11.0.1/config/kafka_client_jaas.conf",这个我已经加了,也没用

半兽人 -> + 6年前

./bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning “后面的东西呢”

+ -> 半兽人 6年前

我的consumer.properties配置了security.protocol=SASL_PLAINTEXT group.id=test-consumer-group
现在又报"No serviceName defined in either JAAS or Kafka config"这个错O.O

半兽人 -> + 6年前

要指定的。

+ -> 半兽人 6年前

额,在哪指定,怎么指定,大神,我有点菜QAQ

半兽人 -> + 6年前

在config/producer.properties和config/consumer.properties增加配置

security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka

执行的时候带上

bin/kafka-console-producer.sh --broker-list 10.211.55.5:9093 --topic test --producer.config
config/producer.properties
+ -> 半兽人 6年前

哦哦,我懂了,弄出来了,谢谢大神

+ -> 半兽人 6年前

请问如何为一个consumer_group添加acl

+ -> 半兽人 6年前

我知道了,真是麻烦大神了QAQ

请问这个如何解决的

我也是这个问题,加上后缀之后,生产者可以写数据,消费者无法消费;请问下怎么可以排查下问题,怎么确认生产者是否写入成功,怎么确认是消费者的问题还是什么的问题?求大神指教

邱响 -> 邱响 6年前

消费者这样写有问题吗??读不出数据

[zk2 bin]$ egrep -v "^$|^#" ../config/consumer.properties
zookeeper.connect=ip1:12181,ip2:12181,ip3:12181
zookeeper.connection.timeout.ms=60000
group.id=test-consumer-group
security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
[zk2 bin]$ ./kafka-console-consumer.sh --bootstrap-server zk2:19092 --topic hello333   --consumer.config ../config/consumer.properties
[2018-01-04 15:14:43,956] WARN The configuration 'zookeeper.connect' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)
[2018-01-04 15:14:43,956] WARN The configuration 'zookeeper.connection.timeout.ms' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)
陈永潮 -> 邱响 5年前

首先要感谢“半兽人”解决了我的问题!
这位兄弟的问题是要把consummer.properties中的zookeeper.connect 和zookeeper.connection.timeout.ms 这两行删除掉

°fairyland -> 陈永潮 5年前

我也遇到了这个问题,生产和消费的时候报错,能加个联系方式请教请教嚒

陈永潮 -> °fairyland 5年前

企鹅号: 402279001

渐行渐远 -> 邱响 3年前

你好,我现在遇到了跟你一模一样的问题,能够生产数据,但是不能够消费。请问您是如何解决的呢

你的答案

查看kafka相关的其他问题或提一个您自己的问题