mirror of https://github.com/apache/druid.git
227b57dd8e
* Compaction: Fetch segments one at a time on main task; skip when possible. Compact tasks include the ability to fetch existing segments and determine reasonable defaults for granularitySpec, dimensionsSpec, and metricsSpec. This is a useful feature that makes compact tasks work well even when the user running the compaction does not have a clear idea of what they want the compacted segments to be like. However, this comes at a cost: it takes time, and disk space, to do all of these fetches. This patch improves the situation in two ways: 1) When segments do need to be fetched, download them one at a time and delete them when we're done. This still takes time, but minimizes the required disk space. 2) Don't fetch segments on the main compact task when they aren't needed. If the user provides a full granularitySpec, dimensionsSpec, and metricsSpec, we can skip it. * Adjustments. * Changes from code review. * Fix logic for determining rollup. |
||
|---|---|---|
| .. | ||
| automatic-compaction.md | ||
| compaction.md | ||
| delete.md | ||
| index.md | ||
| schema-changes.md | ||
| update.md | ||