kafka报错 :org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 704708608 larger than 524288)

伴我同行 发表于: 2019-09-25   最后更新时间: 2019-09-26  

kafka报错 :org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 704708608 larger than 524288)

WARN Unexpected error from /172.17.0.136; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 704708608 larger than 524288)
        at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:95)
        at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:75)
        at org.apache.kafka.common.security.authenticator.SaslServerAuthenticator.authenticate(SaslServerAuthenticator.java:216)
        at org.apache.kafka.common.network.KafkaChannel.prepare(KafkaChannel.java:76)
        at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:376)
        at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
        at kafka.network.Processor.poll(SocketServer.scala:500)
        at kafka.network.Processor.run(SocketServer.scala:435)
        at java.lang.Thread.run(Thread.java:745)

我的kafka server.properties配置文件里面是有配置

socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600

但是好像配置并没有生效

而且我查看kafka server.log 发现内部有错误

[2019-09-25 19:13:56,879] FATAL  (kafka.Kafka$)
java.io.FileNotFoundException: ./config/server.properties (No such file or directory)
        at java.io.FileInputStream.open(Native Method)
        at java.io.FileInputStream.<init>(FileInputStream.java:146)
        at java.io.FileInputStream.<init>(FileInputStream.java:101)
        at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:428)
        at kafka.Kafka$.getPropsFromArgs(Kafka.scala:41)
        at kafka.Kafka$.main(Kafka.scala:57)
        at kafka.Kafka.main(Kafka.scala)

但是我不知道我哪里有获取这个文件,而且我的kafka正常启动了,所以想咨询一下大佬,有没有碰到我这个问题,我弄了一周了还没解决



您需要解锁本帖隐藏内容请: 点击这里
本帖隐藏的内容




上一条: kafka扩容副本后,消费者无法消费
下一条: zookeeper启动时报错空指针

  • 这是我启动的命令:

    cd /usr/kafka/kafka_2.11-0.11.0.1
    ./bin/kafka-server-start.sh  -daemon ./config/server.properties 1&gt;/dev/null 2&gt;&amp;1 &amp;
    
    • 我最困惑的还是这个错误Invalid receive (size = 704708608 larger than 524288) 明明我的server.properties里面有设置但是配置没生效感觉跟这个java.io.FileNotFoundException: ./config/server.properties (No such file or directory)错误有关系

        • 找不到目录或文件这种错,在启动时会起不来,而不是在运行中报的,第一次遇到,你只运行

          bin/kafka-server-start.sh config/server.properties
          

          看看是否可以启动成功。

            • 直接按照你的命令启动过不了多久又会出来
              [2019-09-26 14:08:51,191] INFO [Group Metadata Manager on Broker 191]: Finished loading offsets and group metadata from __consumer_offsets-21 in 0 milliseconds. (kafka.coordinator.group.GroupMetadataManager)
              [2019-09-26 14:09:46,469] FATAL (kafka.Kafka$)
              java.io.FileNotFoundException: ./config/server.properties (No such file or directory)
              at java.io.FileInputStream.open(Native Method)
              at java.io.FileInputStream.(FileInputStream.java:146)
              at java.io.FileInputStream.(FileInputStream.java:101)
              at org.apache.kafka.common.utils.Utils.loadProps(Utils.java:428)
              at kafka.Kafka$.getPropsFromArgs(Kafka.scala:41)
              at kafka.Kafka$.main(Kafka.scala:57)
              at kafka.Kafka.main(Kafka.scala)
              但是进程是在的
              [root@servername191 kafka_2.11-0.11.0.1]# jps
              19047 -- process information unavailable
              5039 Jps
              13565 core
              3430 Kafka
              13454 QuorumPeerMain
              14697 nimbus

                • 问题是我消费和生成的程序都可以正常运行,但是外网的就不行就会报:org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 704708608 larger than 524288) ,真的是没办法

                    你这么启动的kafka?命令贴一下。 是否是这样

                    bin/kafka-server-start.sh config/server.properties