mirror of https://github.com/apache/druid.git
59bca0951a
During ingestion, incremental segments are created in memory for the different time chunks and persisted to disk when certain thresholds are reached (max number of rows, max memory, incremental persist period etc). In the case where there are a lot of dimension and metrics (1000+) it was observed that the creation/serialization of incremental segment file format for persistence and persisting the file took a while and it was blocking ingestion of new data. This affected the real-time ingestion. This serialization and persistence can be parallelized across the different time chunks. This update aims to do that. The patch adds a simple configuration parameter to the ingestion tuning configuration to specify number of persistence threads. The default value is 1 if it not specified which makes it the same as it is today. |
||
|---|---|---|
| .. | ||
| aliyun-oss-extensions | ||
| ambari-metrics-emitter | ||
| cassandra-storage | ||
| cloudfiles-extensions | ||
| compressed-bigdecimal | ||
| ddsketch | ||
| distinctcount | ||
| dropwizard-emitter | ||
| druid-deltalake-extensions | ||
| druid-iceberg-extensions | ||
| gce-extensions | ||
| graphite-emitter | ||
| influx-extensions | ||
| influxdb-emitter | ||
| kafka-emitter | ||
| kubernetes-overlord-extensions | ||
| materialized-view-maintenance | ||
| materialized-view-selection | ||
| momentsketch | ||
| moving-average-query | ||
| opentelemetry-emitter | ||
| opentsdb-emitter | ||
| prometheus-emitter | ||
| redis-cache | ||
| spectator-histogram | ||
| sqlserver-metadata-storage | ||
| statsd-emitter | ||
| tdigestsketch | ||
| thrift-extensions | ||
| time-min-max | ||
| virtual-columns | ||
| README.md | ||
README.md
Community Extensions
Please contribute all community extensions in this directory and include a doc of how your extension can be used under docs/development/extensions-contrib/.
Please note that community extensions are maintained by their original contributors and are not packaged with the core Druid distribution. If you'd like to take on maintenance for a community extension, please post on dev@druid.apache.org to let us know!