druid/docs/content/ingestion
David Glasser a81b1b8c9c index_parallel: support !appendToExisting with no explicit intervals (#7046)
* index_parallel: support !appendToExisting with no explicit intervals

This enables ParallelIndexSupervisorTask to dynamically request locks at runtime
if it is run without explicit intervals in the granularity spec and with
appendToExisting set to false.  Previously, it behaved as if appendToExisting
was set to true, which was undocumented and inconsistent with IndexTask and
Hadoop indexing.

Also, when ParallelIndexSupervisorTask allocates segments in the explicit
interval case, fail if its locks on the interval have been revoked.

Also make a few other additions/clarifications to native ingestion docs.

Fixes #6989.

* Review feedback.

PR description on GitHub updated to match.

* Make native batch ingestion partitions start at 0

* Fix to previous commit

* Unit test. Verified to fail without the other commits on this branch.

* Another round of review

* Slightly scarier warning
2019-02-20 10:54:26 -08:00
..
batch-ingestion.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
command-line-hadoop-indexer.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
compaction.md Add support maxRowsPerSegment for auto compaction (#6780) 2019-01-10 09:50:14 -08:00
data-formats.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
delete-data.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
faq.md Add master/data/query server concepts to docs/packaging (#6916) 2019-01-30 19:41:07 -08:00
firehose.md Add Postgresql SqlFirehose (#6813) 2019-02-14 22:52:03 -08:00
flatten-json.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
hadoop-vs-native-batch.md Add doc for Hadoop-based ingestion vs Native batch ingestion (#7044) 2019-02-13 11:23:08 -08:00
hadoop.md Add doc for Hadoop-based ingestion vs Native batch ingestion (#7044) 2019-02-13 11:23:08 -08:00
index.md Fix and improve doc for partitioning of local index (#7064) 2019-02-13 11:20:52 -08:00
ingestion-spec.md index_parallel: support !appendToExisting with no explicit intervals (#7046) 2019-02-20 10:54:26 -08:00
locking-and-priority.md Add master/data/query server concepts to docs/packaging (#6916) 2019-01-30 19:41:07 -08:00
misc-tasks.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
native_tasks.md index_parallel: support !appendToExisting with no explicit intervals (#7046) 2019-02-20 10:54:26 -08:00
reports.md Add master/data/query server concepts to docs/packaging (#6916) 2019-01-30 19:41:07 -08:00
schema-changes.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
schema-design.md lol (#6985) 2019-02-01 14:21:13 -08:00
stream-ingestion.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
stream-pull.md Add master/data/query server concepts to docs/packaging (#6916) 2019-01-30 19:41:07 -08:00
stream-push.md Add master/data/query server concepts to docs/packaging (#6916) 2019-01-30 19:41:07 -08:00
tasks.md Add doc for Hadoop-based ingestion vs Native batch ingestion (#7044) 2019-02-13 11:23:08 -08:00
transform-spec.md Front Matter header needs to be on the first line for md to be rendered properly by jekyll (#6733) 2018-12-13 11:47:20 -08:00
update-existing-data.md Add master/data/query server concepts to docs/packaging (#6916) 2019-01-30 19:41:07 -08:00