kafka 报错Got error produce response with correlation id on topic-partition Interaction.MainPostLike_5-2, retrying. Error: NETWORK_EXCEPTION

SONG 发表于: 2019-12-18   最后更新时间: 2022-11-28 09:53:32   5,931 游览

使用kafka-connect时偶尔会报错
报错日志先是connect中两个ERROR

[ERROR] [pool-12-thread-1] connect.runtime.WorkerSourceTask - WorkerSourceTask{id=mgsource_etl_1_20191128-0} Failed to flush, timed out while waiting for producer to flush outstanding 53 messages
[ERROR] [pool-12-thread-1] connect.runtime.SourceTaskOffsetCommitter - WorkerSourceTask{id=mgsource_etl_1_20191128-0} Failed to commit offsets

然后是kafka中两条WARN

[WARN ] [kafka-producer-network-thread | producer-5] producer.internals.Sender - [Producer clientId=producer-5] Received invalid metadata error in produce request on partition Interaction.MainPostLike_6-1 due to org.apache.kafka.common.errors.NetworkException: The server disconnected before a response was received.. Going to request metadata update now
[WARN ] [kafka-producer-network-thread | producer-5] producer.internals.Sender - [Producer clientId=producer-5] Got error produce response with correlation id 1861556 on topic-partition Interaction.MainPostLike_5-2, retrying (9 attempts left). Error: NETWORK_EXCEPTION

集群没问题,各种压测也做了,很难复现,偶尔几天会有一两次报错。其实数据并没有丢,因为kafka和connect的重试机制,在报错后下次都flush出去了,找不出原因

添加评论
你的答案

查看kafka相关的其他问题或提一个您自己的问题