2018-12-13 14:47:20 -05:00
---
2019-08-21 00:48:59 -04:00
id: api-reference
title: "API reference"
2018-12-13 14:47:20 -05:00
---
2018-11-13 12:38:37 -05:00
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
2018-08-09 16:27:42 -04:00
This page documents all of the API endpoints for each Druid service type.
## Common
2019-02-28 21:10:39 -05:00
The following endpoints are supported by all processes.
2018-08-09 16:27:42 -04:00
2019-02-28 21:10:39 -05:00
### Process information
2018-08-09 16:27:42 -04:00
#### GET
* `/status`
2019-02-28 21:10:39 -05:00
Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the process.
2018-08-09 16:27:42 -04:00
* `/status/health`
An endpoint that always returns a boolean "true" value with a 200 OK response, useful for automated health checks.
* `/status/properties`
2019-02-28 21:10:39 -05:00
Returns the current configuration properties of the process.
2018-08-09 16:27:42 -04:00
2019-12-11 18:56:36 -05:00
* `/status/selfDiscovered/status`
2019-12-08 10:47:58 -05:00
Returns a JSON map of the form `{"selfDiscovered": true/false}` , indicating whether the node has received a confirmation
from the central node discovery mechanism (currently ZooKeeper) of the Druid cluster that the node has been added to the
cluster. It is recommended to not consider a Druid node "healthy" or "ready" in automated deployment/container
management systems until it returns `{"selfDiscovered": true}` from this endpoint. This is because a node may be
isolated from the rest of the cluster due to network issues and it doesn't make sense to consider nodes "healthy" in
this case. Also, when nodes such as Brokers use ZooKeeper segment discovery for building their view of the Druid cluster
(as opposed to HTTP segment discovery), they may be unusable until the ZooKeeper client is fully initialized and starts
to receive data from the ZooKeeper cluster. `{"selfDiscovered": true}` is a proxy event indicating that the ZooKeeper
client on the node has started to receive data from the ZooKeeper cluster and it's expected that all segments and other
nodes will be discovered by this node timely from this point.
* `/status/selfDiscovered`
2019-12-11 18:56:36 -05:00
Similar to `/status/selfDiscovered/status` , but returns 200 OK response with empty body if the node has discovered itself
2019-12-08 10:47:58 -05:00
and 503 SERVICE UNAVAILABLE if the node hasn't discovered itself yet. This endpoint might be useful because some
monitoring checks such as AWS load balancer health checks are not able to look at the response body.
2019-01-30 22:41:07 -05:00
## Master Server
2018-08-09 16:27:42 -04:00
2019-12-08 10:47:58 -05:00
This section documents the API endpoints for the processes that reside on Master servers (Coordinators and Overlords)
2020-12-17 16:37:43 -05:00
in the suggested [three-server configuration ](../design/processes.md#server-types ).
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
### Coordinator
#### Leadership
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/leader`
2019-01-30 22:41:07 -05:00
Returns the current leader Coordinator of the cluster.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/isLeader`
Returns a JSON object with field "leader", either true or false, indicating if this server is the current leader
2019-01-30 22:41:07 -05:00
Coordinator of the cluster. In addition, returns HTTP 200 if the server is the current leader and HTTP 404 if not.
2018-08-09 16:27:42 -04:00
This is suitable for use as a load balancer status check if you only want the active leader to be considered in-service
at the load balancer.
2019-02-14 09:38:09 -05:00
#### Segment Loading
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/loadstatus`
Returns the percentage of segments actually loaded in the cluster versus segments that should be loaded in the cluster.
* `/druid/coordinator/v1/loadstatus?simple`
2020-06-17 02:48:30 -04:00
Returns the number of segments left to load until segments that should be loaded in the cluster are available for queries. This does not include segment replication counts.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/loadstatus?full`
2020-06-17 02:48:30 -04:00
Returns the number of segments left to load in each tier until segments that should be loaded in the cluster are all available. This includes segment replication counts.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/loadqueue`
2019-02-28 21:10:39 -05:00
Returns the ids of segments to load and drop for each Historical process.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/loadqueue?simple`
2019-02-28 21:10:39 -05:00
Returns the number of segments to load and drop, as well as the total segment load and drop size in bytes for each Historical process.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/loadqueue?full`
2019-02-28 21:10:39 -05:00
Returns the serialized JSON of segments to load and drop for each Historical process.
2018-08-09 16:27:42 -04:00
2020-06-17 02:48:30 -04:00
#### Segment Loading by Datasource
Note that all _interval_ query parameters are ISO 8601 strings (e.g., 2016-06-27/2016-06-28).
Also note that these APIs only guarantees that the segments are available at the time of the call.
Segments can still become missing because of historical process failures or any other reasons afterward.
##### GET
* `/druid/coordinator/v1/datasources/{dataSourceName}/loadstatus?forceMetadataRefresh={boolean}&interval={myInterval}`
Returns the percentage of segments actually loaded in the cluster versus segments that should be loaded in the cluster for the given
datasource over the given interval (or last 2 weeks if interval is not given). `forceMetadataRefresh` is required to be set.
Setting `forceMetadataRefresh` to true will force the coordinator to poll latest segment metadata from the metadata store
(Note: `forceMetadataRefresh=true` refreshes Coordinator's metadata cache of all datasources. This can be a heavy operation in terms
of the load on the metadata store but can be necessary to make sure that we verify all the latest segments' load status)
Setting `forceMetadataRefresh` to false will use the metadata cached on the coordinator from the last force/periodic refresh.
If no used segments are found for the given inputs, this API returns `204 No Content`
* `/druid/coordinator/v1/datasources/{dataSourceName}/loadstatus?simple&forceMetadataRefresh={boolean}&interval={myInterval}`
Returns the number of segments left to load until segments that should be loaded in the cluster are available for the given datasource
over the given interval (or last 2 weeks if interval is not given). This does not include segment replication counts. `forceMetadataRefresh` is required to be set.
Setting `forceMetadataRefresh` to true will force the coordinator to poll latest segment metadata from the metadata store
(Note: `forceMetadataRefresh=true` refreshes Coordinator's metadata cache of all datasources. This can be a heavy operation in terms
of the load on the metadata store but can be necessary to make sure that we verify all the latest segments' load status)
Setting `forceMetadataRefresh` to false will use the metadata cached on the coordinator from the last force/periodic refresh.
If no used segments are found for the given inputs, this API returns `204 No Content`
* `/druid/coordinator/v1/datasources/{dataSourceName}/loadstatus?full&forceMetadataRefresh={boolean}&interval={myInterval}`
Returns the number of segments left to load in each tier until segments that should be loaded in the cluster are all available for the given datasource
over the given interval (or last 2 weeks if interval is not given). This includes segment replication counts. `forceMetadataRefresh` is required to be set.
Setting `forceMetadataRefresh` to true will force the coordinator to poll latest segment metadata from the metadata store
(Note: `forceMetadataRefresh=true` refreshes Coordinator's metadata cache of all datasources. This can be a heavy operation in terms
of the load on the metadata store but can be necessary to make sure that we verify all the latest segments' load status)
Setting `forceMetadataRefresh` to false will use the metadata cached on the coordinator from the last force/periodic refresh.
If no used segments are found for the given inputs, this API returns `204 No Content`
2019-01-30 22:41:07 -05:00
#### Metadata store information
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/metadata/datasources`
2019-07-17 10:18:48 -04:00
Returns a list of the names of data sources with at least one used segment in the cluster.
2018-08-09 16:27:42 -04:00
2019-07-17 10:18:48 -04:00
* `/druid/coordinator/v1/metadata/datasources?includeUnused`
2018-08-09 16:27:42 -04:00
2019-07-17 10:18:48 -04:00
Returns a list of the names of data sources, regardless of whether there are used segments belonging to those data
sources in the cluster or not.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/metadata/datasources?full`
2019-07-17 10:18:48 -04:00
Returns a list of all data sources with at least one used segment in the cluster. Returns all metadata about those data
sources as stored in the metadata store.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}`
Returns full metadata for a datasource as stored in the metadata store.
* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments`
Returns a list of all segments for a datasource as stored in the metadata store.
* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments?full`
Returns a list of all segments for a datasource with the full segment metadata as stored in the metadata store.
* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments/{segmentId}`
2020-01-27 14:24:29 -05:00
Returns full segment metadata for a specific segment as stored in the metadata store, if the segment is used. If the
segment is unused, or is unknown, a 404 response is returned.
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### POST
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments`
2019-02-19 13:52:05 -05:00
Returns a list of all segments, overlapping with any of given intervals, for a datasource as stored in the metadata store. Request body is array of string IS0 8601 intervals like [interval1, interval2,...] for example ["2012-01-01T00:00:00.000/2012-01-03T00:00:00.000", "2012-01-05T00:00:00.000/2012-01-07T00:00:00.000"]
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments?full`
2019-02-19 13:52:05 -05:00
Returns a list of all segments, overlapping with any of given intervals, for a datasource with the full segment metadata as stored in the metadata store. Request body is array of string ISO 8601 intervals like [interval1, interval2,...] for example ["2012-01-01T00:00:00.000/2012-01-03T00:00:00.000", "2012-01-05T00:00:00.000/2012-01-07T00:00:00.000"]
2018-08-09 16:27:42 -04:00
2019-08-21 00:48:59 -04:00
< a name = "coordinator-datasources" > < / a >
2019-01-30 22:41:07 -05:00
#### Datasources
2018-08-09 16:27:42 -04:00
2019-04-12 20:30:57 -04:00
Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/`
2019-02-19 13:52:05 -05:00
(e.g., 2016-06-27_2016-06-28).
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources`
Returns a list of datasource names found in the cluster.
* `/druid/coordinator/v1/datasources?simple`
2019-07-11 04:10:38 -04:00
Returns a list of JSON objects containing the name and properties of datasources found in the cluster. Properties include segment count, total segment byte size, replicated total segment byte size, minTime, and maxTime.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources?full`
Returns a list of datasource names found in the cluster with all metadata about those datasources.
* `/druid/coordinator/v1/datasources/{dataSourceName}`
2019-07-11 04:10:38 -04:00
Returns a JSON object containing the name and properties of a datasource. Properties include segment count, total segment byte size, replicated total segment byte size, minTime, and maxTime.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}?full`
Returns full metadata for a datasource .
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals`
Returns a set of segment intervals.
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals?simple`
Returns a map of an interval to a JSON object containing the total byte size of segments and number of segments for that interval.
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals?full`
Returns a map of an interval to a map of segment metadata to a set of server names that contain the segment for that interval.
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}`
2019-02-19 13:52:05 -05:00
Returns a set of segment ids for an interval.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?simple`
Returns a map of segment intervals contained within the specified interval to a JSON object containing the total byte size of segments and number of segments for an interval.
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?full`
Returns a map of segment intervals contained within the specified interval to a map of segment metadata to a set of server names that contain the segment for an interval.
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}/serverview`
Returns a map of segment intervals contained within the specified interval to information about the servers that contain the segment for an interval.
* `/druid/coordinator/v1/datasources/{dataSourceName}/segments`
Returns a list of all segments for a datasource in the cluster.
* `/druid/coordinator/v1/datasources/{dataSourceName}/segments?full`
Returns a list of all segments for a datasource in the cluster with the full segment metadata.
* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}`
Returns full segment metadata for a specific segment in the cluster.
* `/druid/coordinator/v1/datasources/{dataSourceName}/tiers`
Return the tiers that a datasource exists in.
2019-04-25 17:24:40 -04:00
#### Note for coordinator's POST and DELETE API's
2019-08-21 00:48:59 -04:00
The segments would be enabled when these API's are called, but then can be disabled again by the coordinator if any dropRule matches. Segments enabled by these API's might not be loaded by historical processes if no loadRule matches. If an indexing or kill task runs at the same time as these API's are invoked, the behavior is undefined. Some segments might be killed and others might be enabled. It's also possible that all segments might be disabled but at the same time, the indexing task is able to read data from those segments and succeed.
2019-04-25 17:24:40 -04:00
Caution : Avoid using indexing or kill tasks and these API's at the same time for the same datasource and time chunk. (It's fine if the time chunks or datasource don't overlap)
2019-01-30 22:41:07 -05:00
##### POST
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}`
2019-07-17 10:18:48 -04:00
Marks as used all segments belonging to a data source. Returns a JSON object of the form
`{"numChangedSegments": <number>}` with the number of segments in the database whose state has been changed (that is,
2019-08-21 00:48:59 -04:00
the segments were marked as used) as the result of this API call.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}`
2019-07-17 10:18:48 -04:00
Marks as used a segment of a data source. Returns a JSON object of the form `{"segmentStateChanged": <boolean>}` with
the boolean indicating if the state of the segment has been changed (that is, the segment was marked as used) as the
2019-08-21 00:48:59 -04:00
result of this API call.
2018-08-09 16:27:42 -04:00
2019-04-26 19:01:50 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}/markUsed`
2019-04-25 17:24:40 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}/markUnused`
2019-08-21 00:48:59 -04:00
Marks segments (un)used for a datasource by interval or set of segment Ids.
2019-04-26 19:01:50 -04:00
When marking used only segments that are not overshadowed will be updated.
The request payload contains the interval or set of segment Ids to be marked unused.
Either interval or segment ids should be provided, if both or none are provided in the payload, the API would throw an error (400 BAD REQUEST).
Interval specifies the start and end times as IS0 8601 strings. `interval=(start/end)` where start and end both are inclusive and only the segments completely contained within the specified interval will be disabled, partially overlapping segments will not be affected.
2019-04-25 17:24:40 -04:00
JSON Request Payload:
|Key|Description|Example|
|----------|-------------|---------|
|`interval`|The interval for which to mark segments unused|"2015-09-12T03:00:00.000Z/2015-09-12T05:00:00.000Z"|
|`segmentIds`|Set of segment Ids to be marked unused|["segmentId1", "segmentId2"]|
2019-08-21 00:48:59 -04:00
##### DELETE
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}`
2019-07-17 10:18:48 -04:00
Marks as unused all segments belonging to a data source. Returns a JSON object of the form
`{"numChangedSegments": <number>}` with the number of segments in the database whose state has been changed (that is,
2019-08-21 00:48:59 -04:00
the segments were marked as unused) as the result of this API call.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}`
2019-02-19 13:52:05 -05:00
* `@Deprecated. /druid/coordinator/v1/datasources/{dataSourceName}?kill=true&interval={myInterval}`
2018-08-09 16:27:42 -04:00
2019-08-21 00:48:59 -04:00
Runs a [Kill task ](../ingestion/tasks.md ) for a given interval and datasource.
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}`
2019-07-17 10:18:48 -04:00
Marks as unused a segment of a data source. Returns a JSON object of the form `{"segmentStateChanged": <boolean>}` with
the boolean indicating if the state of the segment has been changed (that is, the segment was marked as unused) as the
result of this API call.
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
#### Retention Rules
2018-08-09 16:27:42 -04:00
2019-04-12 20:30:57 -04:00
Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/`
2019-02-19 13:52:05 -05:00
(e.g., 2016-06-27_2016-06-28).
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/rules`
Returns all rules as JSON objects for all datasources in the cluster including the default datasource.
* `/druid/coordinator/v1/rules/{dataSourceName}`
Returns all rules for a specified datasource.
* `/druid/coordinator/v1/rules/{dataSourceName}?full`
Returns all rules for a specified datasource and includes default datasource.
* `/druid/coordinator/v1/rules/history?interval=<interval>`
2019-01-30 22:41:07 -05:00
Returns audit history of rules for all datasources. default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator runtime.properties
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/rules/history?count=<n>`
Returns last < n > entries of audit history of rules for all datasources.
* `/druid/coordinator/v1/rules/{dataSourceName}/history?interval=<interval>`
2019-01-30 22:41:07 -05:00
Returns audit history of rules for a specified datasource. default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in Coordinator runtime.properties
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/rules/{dataSourceName}/history?count=<n>`
Returns last < n > entries of audit history of rules for a specified datasource.
2019-08-21 00:48:59 -04:00
2019-01-30 22:41:07 -05:00
##### POST
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/rules/{dataSourceName}`
POST with a list of rules in JSON form to update rules.
Optional Header Parameters for auditing the config change can also be specified.
|Header Param Name| Description | Default |
|----------|-------------|---------|
|`X-Druid-Author`| author making the config change|""|
|`X-Druid-Comment`| comment describing the change being done|""|
2019-01-30 22:41:07 -05:00
#### Intervals
2018-08-09 16:27:42 -04:00
2019-04-12 20:30:57 -04:00
Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/`
2019-02-19 13:52:05 -05:00
(e.g., 2016-06-27_2016-06-28).
2018-08-09 16:27:42 -04:00
2019-02-19 13:52:05 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/intervals`
Returns all intervals for all datasources with total size and count.
* `/druid/coordinator/v1/intervals/{interval}`
Returns aggregated total size and count for all intervals that intersect given isointerval.
* `/druid/coordinator/v1/intervals/{interval}?simple`
Returns total size and count for each interval within given isointerval.
* `/druid/coordinator/v1/intervals/{interval}?full`
Returns total size and count for each datasource for each interval within given isointerval.
2020-04-15 17:27:33 -04:00
#### Compaction Status
##### GET
* `/druid/coordinator/v1/compaction/progress?dataSource={dataSource}`
Returns the total size of segments awaiting compaction for the given dataSource.
This is only valid for dataSource which has compaction enabled.
2020-10-14 09:42:22 -04:00
##### GET
* `/druid/coordinator/v1/compaction/status`
Returns the status and statistics from the latest auto compaction run of all dataSources which have/had auto compaction enabled.
The response payload includes a list of `latestStatus` objects. Each `latestStatus` represents the status for a dataSource (which has/had auto compaction enabled).
The `latestStatus` object has the following keys:
* `dataSource` : name of the datasource for this status information
* `scheduleStatus` : auto compaction scheduling status. Possible values are `NOT_ENABLED` and `RUNNING` . Returns `RUNNING ` if the dataSource has an active auto compaction config submitted otherwise, `NOT_ENABLED`
* `bytesAwaitingCompaction` : total bytes of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction)
* `bytesCompacted` : total bytes of this datasource that are already compacted with the spec set in the auto compaction config.
* `bytesSkipped` : total bytes of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.
* `segmentCountAwaitingCompaction` : total number of segments of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction)
* `segmentCountCompacted` : total number of segments of this datasource that are already compacted with the spec set in the auto compaction config.
* `segmentCountSkipped` : total number of segments of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.
* `intervalCountAwaitingCompaction` : total number of intervals of this datasource waiting to be compacted by the auto compaction (only consider intervals/segments that are eligible for auto compaction)
* `intervalCountCompacted` : total number of intervals of this datasource that are already compacted with the spec set in the auto compaction config.
* `intervalCountSkipped` : total number of intervals of this datasource that are skipped (not eligible for auto compaction) by the auto compaction.
##### GET
* `/druid/coordinator/v1/compaction/status?dataSource={dataSource}`
Similar to the API `/druid/coordinator/v1/compaction/status` above but filters response to only return information for the {dataSource} given.
Note that {dataSource} given must have/had auto compaction enabled.
2019-01-30 22:41:07 -05:00
#### Compaction Configuration
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
2018-11-13 12:46:13 -05:00
* `/druid/coordinator/v1/config/compaction`
2018-08-09 16:27:42 -04:00
Returns all compaction configs.
* `/druid/coordinator/v1/config/compaction/{dataSource}`
Returns a compaction config of a dataSource.
2019-01-30 22:41:07 -05:00
##### POST
2018-08-09 16:27:42 -04:00
2018-11-13 12:46:13 -05:00
* `/druid/coordinator/v1/config/compaction/taskslots?ratio={someRatio}&max={someMaxSlots}`
2018-08-09 16:27:42 -04:00
2018-11-13 12:46:13 -05:00
Update the capacity for compaction tasks. `ratio` and `max` are used to limit the max number of compaction tasks.
2019-09-17 15:47:30 -04:00
They mean the ratio of the total task slots to the compaction task slots and the maximum number of task slots for compaction tasks, respectively.
2018-11-13 12:46:13 -05:00
The actual max number of compaction tasks is `min(max, ratio * total task slots)` .
Note that `ratio` and `max` are optional and can be omitted. If they are omitted, default values (0.1 and unbounded)
2018-08-09 16:27:42 -04:00
will be set for them.
2018-11-13 12:46:13 -05:00
* `/druid/coordinator/v1/config/compaction`
2018-08-09 16:27:42 -04:00
2019-03-02 15:21:50 -05:00
Creates or updates the compaction config for a dataSource.
2020-12-17 16:37:43 -05:00
See [Compaction Configuration ](../configuration/index.md#compaction-dynamic-configuration ) for configuration details.
2018-08-09 16:27:42 -04:00
2019-02-19 13:52:05 -05:00
2019-01-30 22:41:07 -05:00
##### DELETE
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/config/compaction/{dataSource}`
Removes the compaction config for a dataSource.
2019-08-21 00:48:59 -04:00
#### Server information
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/coordinator/v1/servers`
Returns a list of servers URLs using the format `{hostname}:{port}` . Note that
2019-02-28 21:10:39 -05:00
processes that run with different types will appear multiple times with different
2018-08-09 16:27:42 -04:00
ports.
* `/druid/coordinator/v1/servers?simple`
Returns a list of server data objects in which each object has the following keys:
2019-02-19 13:52:05 -05:00
* `host` : host URL include (`{hostname}:{port}`)
2019-02-28 21:10:39 -05:00
* `type` : process type (`indexer-executor`, `historical` )
2019-02-19 13:52:05 -05:00
* `currSize` : storage size currently used
* `maxSize` : maximum storage size
* `priority`
* `tier`
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
### Overlord
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
#### Leadership
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
2019-08-21 00:48:59 -04:00
* `/druid/indexer/v1/leader`
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
Returns the current leader Overlord of the cluster. If you have multiple Overlords, just one is leading at any given time. The others are on standby.
2018-08-09 16:27:42 -04:00
* `/druid/indexer/v1/isLeader`
This returns a JSON object with field "leader", either true or false. In addition, this call returns HTTP 200 if the
server is the current leader and HTTP 404 if not. This is suitable for use as a load balancer status check if you
only want the active leader to be considered in-service at the load balancer.
2019-08-21 00:48:59 -04:00
#### Tasks
2018-08-09 16:27:42 -04:00
2019-04-12 20:30:57 -04:00
Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/`
2019-02-19 13:52:05 -05:00
(e.g., 2016-06-27_2016-06-28).
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/tasks`
Retrieve list of tasks. Accepts query string parameters `state` , `datasource` , `createdTimeInterval` , `max` , and `type` .
|Query Parameter |Description |
|---|---|
|`state`|filter list of tasks by task state, valid options are `running` , `complete` , `waiting` , and `pending` .|
| `datasource` | return tasks filtered by Druid datasource.|
2019-04-12 20:30:57 -04:00
| `createdTimeInterval` | return tasks created within the specified interval. |
2019-02-19 13:52:05 -05:00
| `max` | maximum number of `"complete"` tasks to return. Only applies when `state` is set to `"complete"` .|
2019-08-21 00:48:59 -04:00
| `type` | filter tasks by task type. See [task documentation ](../ingestion/tasks.md ) for more details.|
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/completeTasks`
Retrieve list of complete tasks. Equivalent to `/druid/indexer/v1/tasks?state=complete` .
* `/druid/indexer/v1/runningTasks`
Retrieve list of running tasks. Equivalent to `/druid/indexer/v1/tasks?state=running` .
* `/druid/indexer/v1/waitingTasks`
Retrieve list of waiting tasks. Equivalent to `/druid/indexer/v1/tasks?state=waiting` .
* `/druid/indexer/v1/pendingTasks`
Retrieve list of pending tasks. Equivalent to `/druid/indexer/v1/tasks?state=pending` .
* `/druid/indexer/v1/task/{taskId}`
Retrieve the 'payload' of a task.
2018-08-09 16:27:42 -04:00
* `/druid/indexer/v1/task/{taskId}/status`
Retrieve the status of a task.
* `/druid/indexer/v1/task/{taskId}/segments`
Retrieve information about the segments of a task.
2019-08-21 00:48:59 -04:00
> This API is deprecated and will be removed in future releases.
2018-09-26 19:28:02 -04:00
2018-09-24 20:45:05 -04:00
* `/druid/indexer/v1/task/{taskId}/reports`
2019-08-21 00:48:59 -04:00
Retrieve a [task completion report ](../ingestion/tasks.md#task-reports ) for a task. Only works for completed tasks.
2018-09-24 20:45:05 -04:00
2019-01-30 22:41:07 -05:00
##### POST
2018-08-09 16:27:42 -04:00
2019-08-21 00:48:59 -04:00
* `/druid/indexer/v1/task`
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
Endpoint for submitting tasks and supervisor specs to the Overlord. Returns the taskId of the submitted task.
2018-08-09 16:27:42 -04:00
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/task/{taskId}/shutdown`
2018-08-09 16:27:42 -04:00
Shuts down a task.
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/datasources/{dataSource}/shutdownAllTasks`
2018-08-17 12:57:09 -04:00
Shuts down all tasks for a dataSource.
2018-08-09 16:27:42 -04:00
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/taskStatus`
Retrieve list of task status objects for list of task id strings in request body.
##### DELETE
* `/druid/indexer/v1/pendingSegments/{dataSource}`
2019-04-12 20:30:57 -04:00
Manually clean up pending segments table in metadata storage for `datasource` . Returns a JSON object response with
`numDeleted` and count of rows deleted from the pending segments table. This API is used by the
2020-12-17 16:37:43 -05:00
`druid.coordinator.kill.pendingSegments.on` [coordinator setting ](../configuration/index.md#coordinator-operation )
2019-02-19 13:52:05 -05:00
which automates this operation to perform periodically.
2019-01-30 22:41:07 -05:00
#### Supervisors
2018-11-01 16:10:05 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-11-01 16:10:05 -04:00
* `/druid/indexer/v1/supervisor`
Returns a list of strings of the currently active supervisor ids.
* `/druid/indexer/v1/supervisor?full`
Returns a list of objects of the currently active supervisors.
|Field|Type|Description|
|---|---|---|
|`id`|String|supervisor unique identifier|
2020-12-17 16:37:43 -05:00
|`state`|String|basic state of the supervisor. Available states:`UNHEALTHY_SUPERVISOR`, `UNHEALTHY_TASKS` , `PENDING` , `RUNNING` , `SUSPENDED` , `STOPPING` . Check [Kafka Docs ](../development/extensions-core/kafka-ingestion.md#operations ) for details.|
|`detailedState`|String|supervisor specific state. (See documentation of specific supervisor for details), e.g. [Kafka ](../development/extensions-core/kafka-ingestion.md ) or [Kinesis ](../development/extensions-core/kinesis-ingestion.md ))|
2019-06-07 19:26:33 -04:00
|`healthy`|Boolean|true or false indicator of overall supervisor health|
2018-11-01 16:10:05 -04:00
|`spec`|SupervisorSpec|json specification of supervisor (See Supervisor Configuration for details)|
2019-06-07 19:26:33 -04:00
* `/druid/indexer/v1/supervisor?state=true`
Returns a list of objects of the currently active supervisors and their current state.
|Field|Type|Description|
|---|---|---|
|`id`|String|supervisor unique identifier|
2020-12-17 16:37:43 -05:00
|`state`|String|basic state of the supervisor. Available states: `UNHEALTHY_SUPERVISOR` , `UNHEALTHY_TASKS` , `PENDING` , `RUNNING` , `SUSPENDED` , `STOPPING` . Check [Kafka Docs ](../development/extensions-core/kafka-ingestion.md#operations ) for details.|
|`detailedState`|String|supervisor specific state. (See documentation of the specific supervisor for details, e.g. [Kafka ](../development/extensions-core/kafka-ingestion.md ) or [Kinesis ](../development/extensions-core/kinesis-ingestion.md ))|
2019-06-07 19:26:33 -04:00
|`healthy`|Boolean|true or false indicator of overall supervisor health|
2019-10-18 18:16:42 -04:00
|`suspended`|Boolean|true or false indicator of whether the supervisor is in suspended state|
2019-06-07 19:26:33 -04:00
2018-11-01 16:10:05 -04:00
* `/druid/indexer/v1/supervisor/<supervisorId>`
Returns the current spec for the supervisor with the provided ID.
* `/druid/indexer/v1/supervisor/<supervisorId>/status`
Returns the current status of the supervisor with the provided ID.
* `/druid/indexer/v1/supervisor/history`
Returns an audit history of specs for all supervisors (current and past).
* `/druid/indexer/v1/supervisor/<supervisorId>/history`
Returns an audit history of specs for the supervisor with the provided ID.
2019-01-30 22:41:07 -05:00
##### POST
2018-11-01 16:10:05 -04:00
* `/druid/indexer/v1/supervisor`
Create a new supervisor or update an existing one.
* `/druid/indexer/v1/supervisor/<supervisorId>/suspend`
Suspend the current running supervisor of the provided ID. Responds with updated SupervisorSpec.
* `/druid/indexer/v1/supervisor/suspendAll`
Suspend all supervisors at once.
* `/druid/indexer/v1/supervisor/<supervisorId>/resume`
Resume indexing tasks for a supervisor. Responds with updated SupervisorSpec.
* `/druid/indexer/v1/supervisor/resumeAll`
Resume all supervisors at once.
* `/druid/indexer/v1/supervisor/<supervisorId>/reset`
Reset the specified supervisor.
* `/druid/indexer/v1/supervisor/<supervisorId>/terminate`
Terminate a supervisor of the provided ID.
* `/druid/indexer/v1/supervisor/terminateAll`
Terminate all supervisors at once.
* `/druid/indexer/v1/supervisor/<supervisorId>/shutdown`
Shutdown a supervisor.
2019-08-21 00:48:59 -04:00
> This API is deprecated and will be removed in future releases.
> Please use the equivalent 'terminate' instead.
#### Dynamic configuration
2018-11-01 16:10:05 -04:00
2020-12-17 16:37:43 -05:00
See [Overlord Dynamic Configuration ](../configuration/index.md#overlord-dynamic-configuration ) for details.
2019-02-19 13:52:05 -05:00
2019-04-12 20:30:57 -04:00
Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/`
2019-02-19 13:52:05 -05:00
(e.g., 2016-06-27_2016-06-28).
##### GET
* `/druid/indexer/v1/worker`
2019-07-17 10:18:48 -04:00
Retrieves current overlord dynamic configuration.
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/worker/history?interval={interval}&counter={count}`
2019-04-12 20:30:57 -04:00
Retrieves history of changes to overlord dynamic configuration. Accepts `interval` and `count` query string parameters
2019-02-19 13:52:05 -05:00
to filter by interval and limit the number of results respectively.
2020-10-29 12:19:12 -04:00
* `/druid/indexer/v1/workers`
Retrieves a list of all the worker nodes in the cluster along with its metadata.
2019-02-19 13:52:05 -05:00
* `/druid/indexer/v1/scaling`
Retrieves overlord scaling events if auto-scaling runners are in use.
##### POST
* /druid/indexer/v1/worker
Update overlord dynamic worker configuration.
2019-01-30 22:41:07 -05:00
## Data Server
2019-04-12 20:30:57 -04:00
This section documents the API endpoints for the processes that reside on Data servers (MiddleManagers/Peons and Historicals)
2020-12-17 16:37:43 -05:00
in the suggested [three-server configuration ](../design/processes.md#server-types ).
2019-01-30 22:41:07 -05:00
### MiddleManager
2018-08-09 16:27:42 -04:00
2019-02-19 13:52:05 -05:00
##### GET
* `/druid/worker/v1/enabled`
2019-04-12 20:30:57 -04:00
Check whether a MiddleManager is in an enabled or disabled state. Returns JSON object keyed by the combined `druid.host`
2019-02-19 13:52:05 -05:00
and `druid.port` with the boolean state as the value.
```json
{"localhost:8091":true}
```
* `/druid/worker/v1/tasks`
2019-04-12 20:30:57 -04:00
Retrieve a list of active tasks being run on MiddleManager. Returns JSON list of taskid strings. Normal usage should
2019-02-19 13:52:05 -05:00
prefer to use the `/druid/indexer/v1/tasks` [Overlord API ](#overlord ) or one of it's task state specific variants instead.
```json
["index_wikiticker_2019-02-11T02:20:15.316Z"]
```
2019-04-12 20:30:57 -04:00
* `/druid/worker/v1/task/{taskid}/log`
2019-02-19 13:52:05 -05:00
Retrieve task log output stream by task id. Normal usage should prefer to use the `/druid/indexer/v1/task/{taskId}/log`
[Overlord API ](#overlord ) instead.
##### POST
* `/druid/worker/v1/disable`
2019-04-12 20:30:57 -04:00
'Disable' a MiddleManager, causing it to stop accepting new tasks but complete all existing tasks. Returns JSON object
2019-02-19 13:52:05 -05:00
keyed by the combined `druid.host` and `druid.port` :
```json
{"localhost:8091":"disabled"}
```
* `/druid/worker/v1/enable`
2019-04-12 20:30:57 -04:00
'Enable' a MiddleManager, allowing it to accept new tasks again if it was previously disabled. Returns JSON object
2019-02-19 13:52:05 -05:00
keyed by the combined `druid.host` and `druid.port` :
```json
{"localhost:8091":"enabled"}
```
* `/druid/worker/v1/task/{taskid}/shutdown`
2019-04-12 20:30:57 -04:00
Shutdown a running task by `taskid` . Normal usage should prefer to use the `/druid/indexer/v1/task/{taskId}/shutdown`
2019-02-19 13:52:05 -05:00
[Overlord API ](#overlord ) instead. Returns JSON:
```json
{"task":"index_kafka_wikiticker_f7011f8ffba384b_fpeclode"}
```
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
### Peon
2018-08-09 16:27:42 -04:00
2018-09-24 20:45:05 -04:00
#### GET
* `/druid/worker/v1/chat/{taskId}/rowStats`
2019-08-21 00:48:59 -04:00
Retrieve a live row stats report from a Peon. See [task reports ](../ingestion/tasks.md#task-reports ) for more details.
2018-09-24 20:45:05 -04:00
* `/druid/worker/v1/chat/{taskId}/unparseableEvents`
2019-08-21 00:48:59 -04:00
Retrieve an unparseable events report from a Peon. See [task reports ](../ingestion/tasks.md#task-reports ) for more details.
2018-09-24 20:45:05 -04:00
2019-01-30 22:41:07 -05:00
### Historical
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
#### Segment Loading
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
* `/druid/historical/v1/loadstatus`
Returns JSON of the form `{"cacheInitialized":<value>}` , where value is either `true` or `false` indicating if all
2019-02-28 21:10:39 -05:00
segments in the local cache have been loaded. This can be used to know when a Historical process is ready
2019-01-30 22:41:07 -05:00
to be queried after a restart.
* `/druid/historical/v1/readiness`
Similar to `/druid/historical/v1/loadstatus` , but instead of returning JSON with a flag, responses 200 OK if segments
in the local cache have been loaded, and 503 SERVICE UNAVAILABLE, if they haven't.
## Query Server
2020-12-17 16:37:43 -05:00
This section documents the API endpoints for the processes that reside on Query servers (Brokers) in the suggested [three-server configuration ](../design/processes.md#server-types ).
2019-01-30 22:41:07 -05:00
### Broker
#### Datasource Information
2019-04-12 20:30:57 -04:00
Note that all _interval_ URL parameters are ISO 8601 strings delimited by a `_` instead of a `/`
2019-02-19 13:52:05 -05:00
(e.g., 2016-06-27_2016-06-28).
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/v2/datasources`
Returns a list of queryable datasources.
* `/druid/v2/datasources/{dataSourceName}`
Returns the dimensions and metrics of the datasource. Optionally, you can provide request parameter "full" to get list of served intervals with dimensions and metrics being served for those intervals. You can also provide request param "interval" explicitly to refer to a particular interval.
2019-02-19 13:52:05 -05:00
If no interval is specified, a default interval spanning a configurable period before the current time will be used. The default duration of this interval is specified in ISO 8601 duration format via:
2018-08-09 16:27:42 -04:00
druid.query.segmentMetadata.defaultHistory
* `/druid/v2/datasources/{dataSourceName}/dimensions`
Returns the dimensions of the datasource.
2019-08-21 00:48:59 -04:00
> This API is deprecated and will be removed in future releases. Please use [SegmentMetadataQuery](../querying/segmentmetadataquery.md) instead
> which provides more comprehensive information and supports all dataSource types including streaming dataSources. It's also encouraged to use [INFORMATION_SCHEMA tables](../querying/sql.md#metadata-tables)
> if you're using SQL.
2018-09-24 20:56:38 -04:00
2018-08-09 16:27:42 -04:00
* `/druid/v2/datasources/{dataSourceName}/metrics`
Returns the metrics of the datasource.
2019-08-21 00:48:59 -04:00
> This API is deprecated and will be removed in future releases. Please use [SegmentMetadataQuery](../querying/segmentmetadataquery.md) instead
> which provides more comprehensive information and supports all dataSource types including streaming dataSources. It's also encouraged to use [INFORMATION_SCHEMA tables](../querying/sql.md#metadata-tables)
> if you're using SQL.
2018-09-24 20:56:38 -04:00
2019-02-19 13:52:05 -05:00
* `/druid/v2/datasources/{dataSourceName}/candidates?intervals={comma-separated-intervals}&numCandidates={numCandidates}`
2018-08-09 16:27:42 -04:00
Returns segment information lists including server locations for the given datasource and intervals. If "numCandidates" is not specified, it will return all servers for each interval.
2019-01-30 22:41:07 -05:00
#### Load Status
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### GET
2018-08-09 16:27:42 -04:00
* `/druid/broker/v1/loadstatus`
2019-12-10 20:26:13 -05:00
Returns a flag indicating if the Broker knows about all segments in the cluster. This can be used to know when a Broker process is ready to be queried after a restart.
* `/druid/broker/v1/readiness`
Similar to `/druid/broker/v1/loadstatus` , but instead of returning a JSON, responses 200 OK if its ready and otherwise 503 SERVICE UNAVAILABLE.
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
#### Queries
2018-08-09 16:27:42 -04:00
2019-01-30 22:41:07 -05:00
##### POST
2018-08-09 16:27:42 -04:00
* `/druid/v2/`
The endpoint for submitting queries. Accepts an option `?pretty` that pretty prints the results.
* `/druid/v2/candidates/`
Returns segment information lists including server locations for the given query..
2019-09-12 00:42:58 -04:00
### Router
#### GET
* `/druid/v2/datasources`
Returns a list of queryable datasources.
* `/druid/v2/datasources/{dataSourceName}`
Returns the dimensions and metrics of the datasource.
* `/druid/v2/datasources/{dataSourceName}/dimensions`
Returns the dimensions of the datasource.
* `/druid/v2/datasources/{dataSourceName}/metrics`
Returns the metrics of the datasource.
