O que Apache Kafka Consumer Lag Runbook faz?
Diagnoses Kafka consumer group lag using the Kafka AdminClient API and JMX metrics exposed via the Confluent Metrics API. Identifies slow consumers, topic partition hotspots, and broker rebalance storms that contribute to lag growth. Provides a step-by-step runbook to tune fetch.min.bytes, max.poll.records, and partition count.
Prerequisites
Java
Installation
Use the upstream install or setup path that matches your environment:
- docker run -p 9092:9092 apache/kafka:latest
Requirements and caveats from upstream:
- Using docker image:
- See docker/README.md for detailed information.
Basic usage or getting-started notes:
-
Build a JAR and run it
-
Run unit/integration tests
-
./gradlew test -Pkafka.test.run.flaky=true # runs tests that are marked as flaky
-
Source: https://github.com/apache/kafka
-
Extracted from upstream docs: https://raw.githubusercontent.com/apache/kafka/HEAD/README.md