kafka新生产监控

    原创
半兽人 发表于: 2015-03-10   最后更新时间: 2020-01-12  


New producer monitoring


The following metrics are available on new producer instances.





















































































































































































































































Metric/Attribute name

Description

Mbean name

waiting-threads

The number of user threads blocked waiting for buffer memory to enqueue their records

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

buffer-total-bytes

The maximum amount of buffer memory the client can use (whether or not it is currently used).

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

buffer-available-bytes

The total amount of buffer memory that is not being used (either unallocated or in the free list).

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

bufferpool-wait-time

The fraction of time an appender waits for space allocation.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

batch-size-avg

The average number of bytes sent per partition per-request.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

batch-size-max

The max number of bytes sent per partition per-request.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

compression-rate-avg

The average compression rate of record batches.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-queue-time-avg

The average time in ms record batches spent in the record accumulator.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-queue-time-max

The maximum time in ms record batches spent in the record accumulator

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

request-latency-avg

The average request latency in ms

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

request-latency-max

The maximum request latency in ms

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-send-rate

The average number of records sent per second.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

records-per-request-avg

The average number of records per request.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-retry-rate

The average per-second number of retried record sends

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-error-rate

The average per-second number of record sends that resulted in errors

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-size-max

The maximum record size

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

record-size-avg

The average record size

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

requests-in-flight

The current number of in-flight requests awaiting a response.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

metadata-age

The age in seconds of the current producer metadata being used.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

connection-close-rate

Connections closed per second in the window.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

connection-creation-rate

New connections established per second in the window.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

network-io-rate

The average number of network operations (reads or writes) on all connections per second.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

outgoing-byte-rate

The average number of outgoing bytes sent per second to all servers.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

request-rate

The average number of requests sent per second.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

request-size-avg

The average size of all requests in the window.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

request-size-max

The maximum size of any request sent in the window.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

incoming-byte-rate

Bytes/second read off all sockets

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

response-rate

Responses received sent per second.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

select-rate

Number of times the I/O layer checked for new I/O to perform per second

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

io-wait-time-ns-avg

The average length of time the I/O thread spent waiting for a socket ready for reads or writes in nanoseconds.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

io-wait-ratio

The fraction of time the I/O thread spent waiting.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

io-time-ns-avg

The average length of time for I/O per select call in nanoseconds.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

io-ratio

The fraction of time the I/O thread spent doing I/O

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

connection-count

The current number of active connections.

kafka.producer:type=producer-metrics,client-id=([-.\w]+)

outgoing-byte-rate

The average number of outgoing bytes sent per second for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

request-rate

The average number of requests sent per second for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

request-size-avg

The average size of all requests in the window for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

request-size-max

The maximum size of any request sent in the window for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

incoming-byte-rate

The average number of responses received per second for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

request-latency-avg

The average request latency in ms for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

request-latency-max

The maximum request latency in ms for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

response-rate

Responses received sent per second for a node.

kafka.producer:type=producer-node-metrics,client-id=([-.\w]+),node-id=([0-9]+)

record-send-rate

The average number of records sent per second for a topic.

kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)

byte-rate

The average number of bytes sent per second for a topic.

kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)

compression-rate

The average compression rate of record batches for a topic.

kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)

record-retry-rate

The average per-second number of retried record sends for a topic

kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)

record-error-rate

The average per-second number of record sends that resulted in errors for a topic.

kafka.producer:type=producer-topic-metrics,client-id=([-.\w]+),topic=([-.\w]+)

We recommend monitor GC time and other stats and various server stats
such as CPU utilization, I/O service time, etc.
On the client side, we recommend monitor the message/byte rate (global
and per topic), request rate/size/time, and on the consumer side, max
lag in messages among all partitions and min fetch request rate. For a
consumer to keep up, max lag needs to be less than a threshold and min
fetch rate needs to be larger than 0.



您需要解锁本帖隐藏内容请: 点击这里
本帖隐藏的内容




上一条: kafka消息格式
下一条: kafka审计