druid/docs/content/ingestion/tasks.md

53 lines
2.0 KiB
Markdown
Raw Normal View History

---
layout: doc_page
---
2018-09-04 15:54:41 -04:00
# Tasks Overview
Tasks are run on middle managers and always operate on a single data source.
Tasks are submitted using POST requests to the Overlord. Please see [Overlord Task API](../operations/api-reference.html#overlord-tasks) for API details.
There are several different types of tasks.
2018-09-04 15:54:41 -04:00
## Segment Creation Tasks
2016-01-06 00:27:52 -05:00
### Hadoop Index Task
2018-09-04 15:54:41 -04:00
See [batch ingestion](../ingestion/hadoop.html).
2016-01-06 00:27:52 -05:00
### Native Index Tasks
Druid provides a native index task which doesn't need any dependencies on other systems.
See [native index tasks](./native_tasks.html) for more details.
2018-09-04 15:54:41 -04:00
### Kafka Indexing Tasks
2018-09-04 15:54:41 -04:00
Kafka Indexing tasks are automatically created by a Kafka Supervisor and are responsible for pulling data from Kafka streams. These tasks are not meant to be created/submitted directly by users. See [Kafka Indexing Service](../development/extensions-core/kafka-ingestion.html) for more details.
2018-09-04 15:54:41 -04:00
### Stream Push Tasks (Tranquility)
2018-09-04 15:54:41 -04:00
Tranquility Server automatically creates "realtime" tasks that receive events over HTTP using an [EventReceiverFirehose](../ingestion/firehose.html#eventreceiverfirehose). These tasks are not meant to be created/submitted directly by users. See [Tranquility Stream Push](../ingestion/stream-push.html) for more info.
2018-09-04 15:54:41 -04:00
## Compaction Tasks
2018-09-04 15:54:41 -04:00
Compaction tasks merge all segments of the given interval. Please see [Compaction](../ingestion/compaction.html) for details.
2016-01-06 00:27:52 -05:00
2018-09-04 15:54:41 -04:00
## Segment Merging Tasks
2018-09-04 15:54:41 -04:00
<div class="note info">
The documentation for the Append Task, Merge Task, and Same Interval Merge Task has been moved to <a href="../ingestion/misc-tasks.html">Miscellaneous Tasks</a>.
</div>
2018-09-04 15:54:41 -04:00
## Kill Task
2018-09-04 15:54:41 -04:00
Kill tasks delete all information about a segment and removes it from deep storage.
2018-09-04 15:54:41 -04:00
Please see [Deleting Data](../ingestion/delete-data.html) for details.
2018-09-04 15:54:41 -04:00
## Misc. Tasks
2018-09-04 15:54:41 -04:00
Please see [Miscellaneous Tasks](../ingestion/misc-tasks.html).
2018-09-04 15:54:41 -04:00
## Task Locking and Priority
2018-09-04 15:54:41 -04:00
Please see [Task Locking and Priority](../ingestion/locking-and-priority.html).