druid

History

Tejaswini Bandlamudi bf10ff73a8 Fixes Kafka Supervisor Lag Report (#13380 ) Fixes inclusion of all stream partitions in all tasks. The PR (Adds Idle feature to `SeekableStreamSupervisor` for inactive stream) - https://github.com/apache/druid/pull/13144 updates the resulting lag calculation map in `KafkaSupervisor` to include all the latest partitions from the stream to set the idle state accordingly rather than the previous way of lag calculation only for the partitions actively being read from the stream. This led to an explosion of metrics in lag reports in cases where 1000s of tasks per supervisor are present. Changes: - Add a new method to generate lags for only those partitions a single task is actively reading from while updating the Supervisor reports.	2022-11-17 22:24:45 +05:30
..
src	Fixes Kafka Supervisor Lag Report (#13380 )	2022-11-17 22:24:45 +05:30
pom.xml	Dependency cleanup (#13194 )	2022-10-10 20:34:38 +08:00

Tejaswini Bandlamudi bf10ff73a8

Fixes Kafka Supervisor Lag Report (#13380 )

Fixes inclusion of all stream partitions in all tasks.

The PR (Adds Idle feature to `SeekableStreamSupervisor` for inactive stream) - https://github.com/apache/druid/pull/13144 updates the resulting lag calculation map in `KafkaSupervisor` to include all the latest partitions from the stream to set the idle state accordingly rather than the previous way of lag calculation only for the partitions actively being read from the stream. This led to an explosion of metrics in lag reports in cases where 1000s of tasks per supervisor are present.

Changes:
- Add a new method to generate lags for only those partitions a single task is actively reading from while updating the Supervisor reports.

2022-11-17 22:24:45 +05:30

src

Fixes Kafka Supervisor Lag Report (#13380 )

2022-11-17 22:24:45 +05:30

pom.xml

Dependency cleanup (#13194 )

2022-10-10 20:34:38 +08:00