mirror of https://github.com/apache/druid.git
58a3acc2c4
This commit adds a new class `InputStats` to track the total bytes processed by a task. The field `processedBytes` is published in task reports along with other row stats. Major changes: - Add class `InputStats` to track processed bytes - Add method `InputSourceReader.read(InputStats)` to read input rows while counting bytes. > Since we need to count the bytes, we could not just have a wrapper around `InputSourceReader` or `InputEntityReader` (the way `CountableInputSourceReader` does) because the `InputSourceReader` only deals with `InputRow`s and the byte information is already lost. - Classic batch: Use the new `InputSourceReader.read(inputStats)` in `AbstractBatchIndexTask` - Streaming: Increment `processedBytes` in `StreamChunkParser`. This does not use the new `InputSourceReader.read(inputStats)` method. - Extend `InputStats` with `RowIngestionMeters` so that bytes can be exposed in task reports Other changes: - Update tests to verify the value of `processedBytes` - Rename `MutableRowIngestionMeters` to `SimpleRowIngestionMeters` and remove duplicate class - Replace `CacheTestSegmentCacheManager` with `NoopSegmentCacheManager` - Refactor `KafkaIndexTaskTest` and `KinesisIndexTaskTest` |
||
|---|---|---|
| .. | ||
| aliyun-oss-extensions | ||
| ambari-metrics-emitter | ||
| cassandra-storage | ||
| cloudfiles-extensions | ||
| compressed-bigdecimal | ||
| distinctcount | ||
| dropwizard-emitter | ||
| gce-extensions | ||
| graphite-emitter | ||
| influx-extensions | ||
| influxdb-emitter | ||
| kafka-emitter | ||
| kubernetes-overlord-extensions | ||
| materialized-view-maintenance | ||
| materialized-view-selection | ||
| momentsketch | ||
| moving-average-query | ||
| opentelemetry-emitter | ||
| opentsdb-emitter | ||
| prometheus-emitter | ||
| redis-cache | ||
| sqlserver-metadata-storage | ||
| statsd-emitter | ||
| tdigestsketch | ||
| thrift-extensions | ||
| time-min-max | ||
| virtual-columns | ||
| README.md | ||
README.md
Community Extensions
Please contribute all community extensions in this directory and include a doc of how your extension can be used under docs/development/extensions-contrib/.
Please note that community extensions are maintained by their original contributors and are not packaged with the core Druid distribution. If you'd like to take on maintenance for a community extension, please post on dev@druid.apache.org to let us know!