Apache
/
druid
mirror of https://github.com/apache/druid.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Fix CompactionTask doc (#5351) 

* Fix CompactionTask doc

* Update coordinator doc
This commit is contained in:
Jihoon Son 2018-02-05 22:38:34 -08:00 committed by Gian Merlino
parent 285dedd126
commit 0db696b7c9
2 changed files with 4 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
docs/content/configuration/coordinator.md
View File

@ -148,18 +148,6 @@ An example of compaction config is:
For the realtime dataSources, it's recommended to set `skipOffsetFromLatest` to some sufficiently large values to avoid frequent compact task failures. For the realtime dataSources, it's recommended to set `skipOffsetFromLatest` to some sufficiently large values to avoid frequent compact task failures.
## Compact Task TuningConfig
Compact task tuning config is a subset of the tuningConfig of IndexTask. See [TuningConfig](../ingestion/tasks.html#tuningconfig) for more details.
|Property|Required|
|--------|--------|
|`maxRowsInMemory`|no|
|`maxTotalRows`|no|
|`indexSpec`|no|
|`maxPendingPersists`|no|
|`publishTimeout`|no|
# Lookups Dynamic Config (EXPERIMENTAL) # Lookups Dynamic Config (EXPERIMENTAL)
These configuration options control the behavior of the Lookup dynamic configuration described in the [lookups page](../querying/lookups.html) These configuration options control the behavior of the Lookup dynamic configuration described in the [lookups page](../querying/lookups.html)

6
docs/content/ingestion/tasks.md
View File

@ -276,8 +276,10 @@ An example of compaction task is
} }
``` ```
This compaction task merges _all segments_ of the interval `2017-01-01/2018-01-01` into a _single segment_. This compaction task reads _all segments_ of the interval `2017-01-01/2018-01-01` and results in new segments.
To merge each day's worth of data into a separate segment, you can submit multiple `compact` tasks, one for each day. They will run in parallel. Note that intervals of the input segments are merged into a single interval of `2017-01-01/2018-01-01` no matter what the segmentGranularity was.
To controll the number of result segments, you can set `targetPartitionSize` or `numShards`. See [indexTuningConfig](#tuningconfig) for more details.
To merge each day's worth of data into separate segments, you can submit multiple `compact` tasks, one for each day. They will run in parallel.
A compaction task internally generates an `index` task spec for performing compaction work with some fixed parameters. A compaction task internally generates an `index` task spec for performing compaction work with some fixed parameters.
For example, its `firehose` is always the [ingestSegmentSpec](./firehose.html), and `dimensionsSpec` and `metricsSpec` For example, its `firehose` is always the [ingestSegmentSpec](./firehose.html), and `dimensionsSpec` and `metricsSpec`