diff --git a/docs/content/design/broker.md b/docs/content/design/broker.md index 7f20dfa0a48..1c3c3469857 100644 --- a/docs/content/design/broker.md +++ b/docs/content/design/broker.md @@ -31,45 +31,4 @@ historical nodes. Once the historical nodes return their results, the broker wil HTTP Endpoints -------------- -The broker node exposes several HTTP endpoints for interactions. - -### GET - -* `/status` - -Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the node. - -* `/druid/v2/datasources` - -Returns a list of queryable datasources. - -* `/druid/v2/datasources/{dataSourceName}` - -Returns the dimensions and metrics of the datasource. Optionally, you can provide request parameter "full" to get list of served intervals with dimensions and metrics being served for those intervals. You can also provide request param "interval" explicitly to refer to a particular interval. - -If no interval is specified, a default interval spanning a configurable period before the current time will be used. The duration of this interval is specified in ISO8601 format via: - -druid.query.segmentMetadata.defaultHistory - -* `/druid/v2/datasources/{dataSourceName}/dimensions` - -Returns the dimensions of the datasource. - -* `/druid/v2/datasources/{dataSourceName}/metrics` - -Returns the metrics of the datasource. - -* `/druid/v2/datasources/{dataSourceName}/candidates?intervals={comma-separated-intervals-in-ISO8601-format}&numCandidates={numCandidates}` - -Returns segment information lists including server locations for the given datasource and intervals. If "numCandidates" is not specified, it will return all servers for each interval. - -* `/druid/broker/v1/loadstatus` - -Returns a flag indicating if the broker knows about all segments in Zookeeper. This can be used to know when a broker node is ready to be queried after a restart. - - -### POST - -* `/druid/v2/candidates/` - -Returns segment information lists including server locations for the given query. +For a list of API endpoints supported by the Broker, please see the [API reference](../operations/api-reference.html#broker). diff --git a/docs/content/design/coordinator.md b/docs/content/design/coordinator.md index 9723fb2586d..462c616232e 100644 --- a/docs/content/design/coordinator.md +++ b/docs/content/design/coordinator.md @@ -82,297 +82,7 @@ If it finds such segments, it simply skips compacting them. HTTP Endpoints -------------- -The coordinator node exposes several HTTP endpoints for interactions. - -### GET - -* `/status` - -Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the node. - -#### Coordinator information - -* `/druid/coordinator/v1/leader` - -Returns the current leader coordinator of the cluster. - -* `/druid/coordinator/v1/isLeader` - -Returns a JSON object with field "leader", either true or false, indicating if this server is the current leader -coordinator of the cluster. In addition, returns HTTP 200 if the server is the current leader and HTTP 404 if not. -This is suitable for use as a load balancer status check if you only want the active leader to be considered in-service -at the load balancer. - -* `/druid/coordinator/v1/loadstatus` - -Returns the percentage of segments actually loaded in the cluster versus segments that should be loaded in the cluster. - - * `/druid/coordinator/v1/loadstatus?simple` - -Returns the number of segments left to load until segments that should be loaded in the cluster are available for queries. This does not include replication. - -* `/druid/coordinator/v1/loadstatus?full` - -Returns the number of segments left to load in each tier until segments that should be loaded in the cluster are all available. This includes replication. - -* `/druid/coordinator/v1/loadqueue` - -Returns the ids of segments to load and drop for each historical node. - -* `/druid/coordinator/v1/loadqueue?simple` - -Returns the number of segments to load and drop, as well as the total segment load and drop size in bytes for each historical node. - -* `/druid/coordinator/v1/loadqueue?full` - -Returns the serialized JSON of segments to load and drop for each historical node. - -#### Metadata store information - -* `/druid/coordinator/v1/metadata/datasources` - -Returns a list of the names of enabled datasources in the cluster. - -* `/druid/coordinator/v1/metadata/datasources?includeDisabled` - -Returns a list of the names of enabled and disabled datasources in the cluster. - -* `/druid/coordinator/v1/metadata/datasources?full` - -Returns a list of all enabled datasources with all metadata about those datasources as stored in the metadata store. - -* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}` - -Returns full metadata for a datasource as stored in the metadata store. - -* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments` - -Returns a list of all segments for a datasource as stored in the metadata store. - -* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments?full` - -Returns a list of all segments for a datasource with the full segment metadata as stored in the metadata store. - -* POST `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments` - -Returns a list of all segments, overlapping with any of given intervals, for a datasource as stored in the metadata store. Request body is array of string intervals like [interval1, interval2,...] for example ["2012-01-01T00:00:00.000/2012-01-03T00:00:00.000", "2012-01-05T00:00:00.000/2012-01-07T00:00:00.000"] - -* POST `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments?full` - -Returns a list of all segments, overlapping with any of given intervals, for a datasource with the full segment metadata as stored in the metadata store. Request body is array of string intervals like [interval1, interval2,...] for example ["2012-01-01T00:00:00.000/2012-01-03T00:00:00.000", "2012-01-05T00:00:00.000/2012-01-07T00:00:00.000"] - -* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments/{segmentId}` - -Returns full segment metadata for a specific segment as stored in the metadata store. - -#### Datasources information - -* `/druid/coordinator/v1/datasources` - -Returns a list of datasource names found in the cluster. - -* `/druid/coordinator/v1/datasources?simple` - -Returns a list of JSON objects containing the name and properties of datasources found in the cluster. Properties include segment count, total segment byte size, minTime, and maxTime. - -* `/druid/coordinator/v1/datasources?full` - -Returns a list of datasource names found in the cluster with all metadata about those datasources. - -* `/druid/coordinator/v1/datasources/{dataSourceName}` - -Returns a JSON object containing the name and properties of a datasource. Properties include segment count, total segment byte size, minTime, and maxTime. - -* `/druid/coordinator/v1/datasources/{dataSourceName}?full` - -Returns full metadata for a datasource . - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals` - -Returns a set of segment intervals. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals?simple` - -Returns a map of an interval to a JSON object containing the total byte size of segments and number of segments for that interval. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals?full` - -Returns a map of an interval to a map of segment metadata to a set of server names that contain the segment for that interval. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}` - -Returns a set of segment ids for an ISO8601 interval. Note that {interval} parameters are delimited by a `_` instead of a `/` (e.g., 2016-06-27_2016-06-28). - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?simple` - -Returns a map of segment intervals contained within the specified interval to a JSON object containing the total byte size of segments and number of segments for an interval. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?full` - -Returns a map of segment intervals contained within the specified interval to a map of segment metadata to a set of server names that contain the segment for an interval. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}/serverview` - -Returns a map of segment intervals contained within the specified interval to information about the servers that contain the segment for an interval. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/segments` - -Returns a list of all segments for a datasource in the cluster. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/segments?full` - -Returns a list of all segments for a datasource in the cluster with the full segment metadata. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}` - -Returns full segment metadata for a specific segment in the cluster. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/tiers` - -Return the tiers that a datasource exists in. - -#### Rules - -* `/druid/coordinator/v1/rules` - -Returns all rules as JSON objects for all datasources in the cluster including the default datasource. - -* `/druid/coordinator/v1/rules/{dataSourceName}` - -Returns all rules for a specified datasource. - - -* `/druid/coordinator/v1/rules/{dataSourceName}?full` - -Returns all rules for a specified datasource and includes default datasource. - -* `/druid/coordinator/v1/rules/history?interval=` - - Returns audit history of rules for all datasources. default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in coordinator runtime.properties - -* `/druid/coordinator/v1/rules/history?count=` - - Returns last entries of audit history of rules for all datasources. - -* `/druid/coordinator/v1/rules/{dataSourceName}/history?interval=` - - Returns audit history of rules for a specified datasource. default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in coordinator runtime.properties - -* `/druid/coordinator/v1/rules/{dataSourceName}/history?count=` - - Returns last entries of audit history of rules for a specified datasource. - -#### Intervals - -Note that {interval} parameters are delimited by a `_` instead of a `/` (e.g., 2016-06-27_2016-06-28). - -* `/druid/coordinator/v1/intervals` - -Returns all intervals for all datasources with total size and count. - -* `/druid/coordinator/v1/intervals/{interval}` - -Returns aggregated total size and count for all intervals that intersect given isointerval. - -* `/druid/coordinator/v1/intervals/{interval}?simple` - -Returns total size and count for each interval within given isointerval. - -* `/druid/coordinator/v1/intervals/{interval}?full` - -Returns total size and count for each datasource for each interval within given isointerval. - -#### Compaction Configs - -* `/druid/coordinator/v1/config/compaction/` - -Returns all compaction configs. - -* `/druid/coordinator/v1/config/compaction/{dataSource}` - -Returns a compaction config of a dataSource. - -#### Servers - -* `/druid/coordinator/v1/servers` - -Returns a list of servers URLs using the format `{hostname}:{port}`. Note that -nodes that run with different types will appear multiple times with different -ports. - -* `/druid/coordinator/v1/servers?simple` - -Returns a list of server data objects in which each object has the following keys: -- `host`: host URL include (`{hostname}:{port}`) -- `type`: node type (`indexer-executor`, `historical`) -- `currSize`: storage size currently used -- `maxSize`: maximum storage size -- `priority` -- `tier` - -### POST - -#### Datasources - -* `/druid/coordinator/v1/datasources/{dataSourceName}` - -Enables all segments of datasource which are not overshadowed by others. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}` - -Enables a segment. - -#### Rules - -* `/druid/coordinator/v1/rules/{dataSourceName}` - -POST with a list of rules in JSON form to update rules. - -Optional Header Parameters for auditing the config change can also be specified. - -|Header Param Name| Description | Default | -|----------|-------------|---------| -|`X-Druid-Author`| author making the config change|""| -|`X-Druid-Comment`| comment describing the change being done|""| - -#### Compaction Configs - -* `/druid/coordinator/v1/config/compaction?slotRatio={someRatio}&maxSlots={someMaxSlots}` - -Update the capacity for compaction tasks. `slotRatio` and `maxSlots` are used to limit the max number of compaction tasks. -They mean the ratio of the total task slots to the copmaction task slots and the maximum number of task slots for compaction tasks, respectively. -The actual max number of compaction tasks is `min(maxSlots, slotRatio * total task slots)`. -Note that `slotRatio` and `maxSlots` are optional and can be omitted. If they are omitted, default values (0.1 and unbounded) -will be set for them. - -* `/druid/coordinator/v1/config/compaction/{dataSource}` - -Creates or updates the compaction config for a dataSource. See [Compaction Configuration](../configuration/coordinator.html#compaction-configuration) for configuration details. - -### DELETE - -#### Datasources - -* `/druid/coordinator/v1/datasources/{dataSourceName}` - -Disables a datasource. - -* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}` -* `@Deprecated. /druid/coordinator/v1/datasources/{dataSourceName}?kill=true&interval={myISO8601Interval}` - -Runs a [Kill task](../ingestion/tasks.html) for a given interval and datasource. - -Note that {interval} parameters are delimited by a `_` instead of a `/` (e.g., 2016-06-27_2016-06-28). - -* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}` - -Disables a segment. - -#### Compaction Configs - -* `/druid/coordinator/v1/config/compaction/{dataSource}` - -Removes the compaction config for a dataSource. +For a list of API endpoints supported by the Coordinator, please see the [API reference](../operations/api-reference.html#coordinator). The Coordinator Console ------------------ diff --git a/docs/content/design/historical.md b/docs/content/design/historical.md index d6fd0f7ab9b..04f99dcbe7e 100644 --- a/docs/content/design/historical.md +++ b/docs/content/design/historical.md @@ -40,21 +40,5 @@ A historical can be configured to log and report metrics for every query it serv HTTP Endpoints -------------- -The historical node exposes several HTTP endpoints for interactions. +For a list of API endpoints supported by the Historical, please see the [API reference](../operations/api-reference.html#historical). -### GET - -* `/status` - -Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the node. - -* `/druid/historical/v1/loadstatus` - -Returns JSON of the form `{"cacheInitialized":}`, where value is either `true` or `false` indicating if all -segments in the local cache have been loaded. This can be used to know when a historical node is ready -to be queried after a restart. - -* `/druid/historical/v1/readiness` - -Similar to `/druid/historical/v1/loadstatus`, but instead of returning JSON with a flag, responses 200 OK if segments -in the local cache have been loaded, and 503 SERVICE UNAVAILABLE, if they haven't. diff --git a/docs/content/design/indexing-service.md b/docs/content/design/indexing-service.md index 43087b7e379..4435e69208f 100644 --- a/docs/content/design/indexing-service.md +++ b/docs/content/design/indexing-service.md @@ -33,52 +33,9 @@ In local mode overlord is also responsible for creating peons for executing task Local mode is typically used for simple workflows. In remote mode, the overlord and middle manager are run in separate processes and you can run each on a different server. This mode is recommended if you intend to use the indexing service as the single endpoint for all Druid indexing. -#### Leadership status +#### HTTP Endpoints -If you have multiple overlords, just one is leading at any given time. The others are on standby. To get the current -leader overlord of the cluster, call: - - -``` -http://:/druid/indexer/v1/leader -``` - -To see if a given server is the current leader overlord of the cluster, call: - -``` -http://:/druid/indexer/v1/isLeader -``` - -This returns a JSON object with field "leader", either true or false. In addition, this call returns HTTP 200 if the -server is the current leader and HTTP 404 if not. This is suitable for use as a load balancer status check if you -only want the active leader to be considered in-service at the load balancer. - -#### Submitting Tasks and Querying Task Status - -Tasks are submitted to the overlord node in the form of JSON objects. Tasks can be submitted via POST requests to: - -``` -http://:/druid/indexer/v1/task -``` -this will return the taskId of the submitted task. - -Tasks can be shut down via POST requests to: - -``` -http://:/druid/indexer/v1/task/{taskId}/shutdown -``` - -Task statuses can be retrieved via GET requests to: - -``` -http://:/druid/indexer/v1/task/{taskId}/status -``` - -Task segments can be retrieved via GET requests to: - -``` -http://:/druid/indexer/v1/task/{taskId}/segments -``` +For a list of API endpoints supported by the Overlord, please see the [API reference](../operations/api-reference.html#overlord). #### Overlord Console @@ -120,12 +77,3 @@ Tasks ----- See [Tasks](../ingestion/tasks.html). - -HTTP Endpoints --------------- - -### GET - -* `/status` - -Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the node. diff --git a/docs/content/design/middlemanager.md b/docs/content/design/middlemanager.md index 39d1be8fd88..cc1de96f4f1 100644 --- a/docs/content/design/middlemanager.md +++ b/docs/content/design/middlemanager.md @@ -20,8 +20,4 @@ io.druid.cli.Main server middleManager HTTP Endpoints -------------- -### GET - -* `/status` - -Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the node. +For a list of API endpoints supported by the MiddleManager, please see the [API reference](../operations/api-reference.html#middlemanager). diff --git a/docs/content/design/peons.md b/docs/content/design/peons.md index f4082ba3303..76bbc1f9fdb 100644 --- a/docs/content/design/peons.md +++ b/docs/content/design/peons.md @@ -21,3 +21,8 @@ io.druid.cli.Main internal peon The task file contains the task JSON object. The status file indicates where the task status will be output. + +HTTP Endpoints +-------------- + +For a list of API endpoints supported by the Peon, please see the [API reference](../operations/api-reference.html#peon). diff --git a/docs/content/operations/api-reference.md b/docs/content/operations/api-reference.md new file mode 100644 index 00000000000..d933c4f4052 --- /dev/null +++ b/docs/content/operations/api-reference.md @@ -0,0 +1,458 @@ +--- +layout: doc_page +--- + +# API Reference + +This page documents all of the API endpoints for each Druid service type. + +## Table of Contents + * [Common](#common) + * [Coordinator](#coordinator) + * [Overlord](#overlord) + * [MiddleManager](#middlemanager) + * [Peon](#peon) + * [Broker](#broker) + * [Historical](#historical) + +## Common + +The following endpoints are supported by all nodes. + +### Node information + +#### GET + +* `/status` + +Returns the Druid version, loaded extensions, memory used, total memory and other useful information about the node. + +* `/status/health` + +An endpoint that always returns a boolean "true" value with a 200 OK response, useful for automated health checks. + +* `/status/properties` + +Returns the current configuration properties of the node. + +## Coordinator + +### Leadership + +#### GET + +* `/druid/coordinator/v1/leader` + +Returns the current leader coordinator of the cluster. + +* `/druid/coordinator/v1/isLeader` + +Returns true if the coordinator receiving the request is the current leader. + +### Segment Loading + +#### GET + +Returns a JSON object with field "leader", either true or false, indicating if this server is the current leader +coordinator of the cluster. In addition, returns HTTP 200 if the server is the current leader and HTTP 404 if not. +This is suitable for use as a load balancer status check if you only want the active leader to be considered in-service +at the load balancer. + +* `/druid/coordinator/v1/loadstatus` + +Returns the percentage of segments actually loaded in the cluster versus segments that should be loaded in the cluster. + + * `/druid/coordinator/v1/loadstatus?simple` + +Returns the number of segments left to load until segments that should be loaded in the cluster are available for queries. This does not include replication. + +* `/druid/coordinator/v1/loadstatus?full` + +Returns the number of segments left to load in each tier until segments that should be loaded in the cluster are all available. This includes replication. + +* `/druid/coordinator/v1/loadqueue` + +Returns the ids of segments to load and drop for each historical node. + +* `/druid/coordinator/v1/loadqueue?simple` + +Returns the number of segments to load and drop, as well as the total segment load and drop size in bytes for each historical node. + +* `/druid/coordinator/v1/loadqueue?full` + +Returns the serialized JSON of segments to load and drop for each historical node. + +### Metadata store information + +#### GET + +* `/druid/coordinator/v1/metadata/datasources` + +Returns a list of the names of enabled datasources in the cluster. + +* `/druid/coordinator/v1/metadata/datasources?includeDisabled` + +Returns a list of the names of enabled and disabled datasources in the cluster. + +* `/druid/coordinator/v1/metadata/datasources?full` + +Returns a list of all enabled datasources with all metadata about those datasources as stored in the metadata store. + +* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}` + +Returns full metadata for a datasource as stored in the metadata store. + +* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments` + +Returns a list of all segments for a datasource as stored in the metadata store. + +* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments?full` + +Returns a list of all segments for a datasource with the full segment metadata as stored in the metadata store. + +* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments/{segmentId}` + +Returns full segment metadata for a specific segment as stored in the metadata store. + +#### POST + +* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments` + +Returns a list of all segments, overlapping with any of given intervals, for a datasource as stored in the metadata store. Request body is array of string intervals like [interval1, interval2,...] for example ["2012-01-01T00:00:00.000/2012-01-03T00:00:00.000", "2012-01-05T00:00:00.000/2012-01-07T00:00:00.000"] + +* `/druid/coordinator/v1/metadata/datasources/{dataSourceName}/segments?full` + +Returns a list of all segments, overlapping with any of given intervals, for a datasource with the full segment metadata as stored in the metadata store. Request body is array of string intervals like [interval1, interval2,...] for example ["2012-01-01T00:00:00.000/2012-01-03T00:00:00.000", "2012-01-05T00:00:00.000/2012-01-07T00:00:00.000"] + + +### Datasources + +#### GET + +* `/druid/coordinator/v1/datasources` + +Returns a list of datasource names found in the cluster. + +* `/druid/coordinator/v1/datasources?simple` + +Returns a list of JSON objects containing the name and properties of datasources found in the cluster. Properties include segment count, total segment byte size, minTime, and maxTime. + +* `/druid/coordinator/v1/datasources?full` + +Returns a list of datasource names found in the cluster with all metadata about those datasources. + +* `/druid/coordinator/v1/datasources/{dataSourceName}` + +Returns a JSON object containing the name and properties of a datasource. Properties include segment count, total segment byte size, minTime, and maxTime. + +* `/druid/coordinator/v1/datasources/{dataSourceName}?full` + +Returns full metadata for a datasource . + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals` + +Returns a set of segment intervals. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals?simple` + +Returns a map of an interval to a JSON object containing the total byte size of segments and number of segments for that interval. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals?full` + +Returns a map of an interval to a map of segment metadata to a set of server names that contain the segment for that interval. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}` + +Returns a set of segment ids for an ISO8601 interval. Note that {interval} parameters are delimited by a `_` instead of a `/` (e.g., 2016-06-27_2016-06-28). + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?simple` + +Returns a map of segment intervals contained within the specified interval to a JSON object containing the total byte size of segments and number of segments for an interval. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}?full` + +Returns a map of segment intervals contained within the specified interval to a map of segment metadata to a set of server names that contain the segment for an interval. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}/serverview` + +Returns a map of segment intervals contained within the specified interval to information about the servers that contain the segment for an interval. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/segments` + +Returns a list of all segments for a datasource in the cluster. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/segments?full` + +Returns a list of all segments for a datasource in the cluster with the full segment metadata. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}` + +Returns full segment metadata for a specific segment in the cluster. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/tiers` + +Return the tiers that a datasource exists in. + +#### POST + +* `/druid/coordinator/v1/datasources/{dataSourceName}` + +Enables all segments of datasource which are not overshadowed by others. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}` + +Enables a segment of a datasource. + +#### DELETE + +* `/druid/coordinator/v1/datasources/{dataSourceName}` + +Disables a datasource. + +* `/druid/coordinator/v1/datasources/{dataSourceName}/intervals/{interval}` +* `@Deprecated. /druid/coordinator/v1/datasources/{dataSourceName}?kill=true&interval={myISO8601Interval}` + +Runs a [Kill task](../ingestion/tasks.html) for a given interval and datasource. + +Note that {interval} parameters are delimited by a `_` instead of a `/` (e.g., 2016-06-27_2016-06-28). + +* `/druid/coordinator/v1/datasources/{dataSourceName}/segments/{segmentId}` + +Disables a segment. + +### Retention Rules + +#### GET + +* `/druid/coordinator/v1/rules` + +Returns all rules as JSON objects for all datasources in the cluster including the default datasource. + +* `/druid/coordinator/v1/rules/{dataSourceName}` + +Returns all rules for a specified datasource. + + +* `/druid/coordinator/v1/rules/{dataSourceName}?full` + +Returns all rules for a specified datasource and includes default datasource. + +* `/druid/coordinator/v1/rules/history?interval=` + + Returns audit history of rules for all datasources. default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in coordinator runtime.properties + +* `/druid/coordinator/v1/rules/history?count=` + + Returns last entries of audit history of rules for all datasources. + +* `/druid/coordinator/v1/rules/{dataSourceName}/history?interval=` + + Returns audit history of rules for a specified datasource. default value of interval can be specified by setting `druid.audit.manager.auditHistoryMillis` (1 week if not configured) in coordinator runtime.properties + +* `/druid/coordinator/v1/rules/{dataSourceName}/history?count=` + + Returns last entries of audit history of rules for a specified datasource. + +#### POST + +* `/druid/coordinator/v1/rules/{dataSourceName}` + +POST with a list of rules in JSON form to update rules. + +Optional Header Parameters for auditing the config change can also be specified. + +|Header Param Name| Description | Default | +|----------|-------------|---------| +|`X-Druid-Author`| author making the config change|""| +|`X-Druid-Comment`| comment describing the change being done|""| + +### Intervals + +#### GET + +Note that {interval} parameters are delimited by a `_` instead of a `/` (e.g., 2016-06-27_2016-06-28). + +* `/druid/coordinator/v1/intervals` + +Returns all intervals for all datasources with total size and count. + +* `/druid/coordinator/v1/intervals/{interval}` + +Returns aggregated total size and count for all intervals that intersect given isointerval. + +* `/druid/coordinator/v1/intervals/{interval}?simple` + +Returns total size and count for each interval within given isointerval. + +* `/druid/coordinator/v1/intervals/{interval}?full` + +Returns total size and count for each datasource for each interval within given isointerval. + +### Compaction Configuration + +#### GET + +* `/druid/coordinator/v1/config/compaction/` + +Returns all compaction configs. + +* `/druid/coordinator/v1/config/compaction/{dataSource}` + +Returns a compaction config of a dataSource. + +#### POST + +* `/druid/coordinator/v1/config/compaction?slotRatio={someRatio}&maxSlots={someMaxSlots}` + +Update the capacity for compaction tasks. `slotRatio` and `maxSlots` are used to limit the max number of compaction tasks. +They mean the ratio of the total task slots to the copmaction task slots and the maximum number of task slots for compaction tasks, respectively. +The actual max number of compaction tasks is `min(maxSlots, slotRatio * total task slots)`. +Note that `slotRatio` and `maxSlots` are optional and can be omitted. If they are omitted, default values (0.1 and unbounded) +will be set for them. + +* `/druid/coordinator/v1/config/compaction/{dataSource}` + +Creates or updates the compaction config for a dataSource. See [Compaction Configuration](../configuration/coordinator.html#compaction-configuration) for configuration details. + +#### DELETE + +* `/druid/coordinator/v1/config/compaction/{dataSource}` + +Removes the compaction config for a dataSource. + +### Server Information + +#### GET + +* `/druid/coordinator/v1/servers` + +Returns a list of servers URLs using the format `{hostname}:{port}`. Note that +nodes that run with different types will appear multiple times with different +ports. + +* `/druid/coordinator/v1/servers?simple` + +Returns a list of server data objects in which each object has the following keys: +- `host`: host URL include (`{hostname}:{port}`) +- `type`: node type (`indexer-executor`, `historical`) +- `currSize`: storage size currently used +- `maxSize`: maximum storage size +- `priority` +- `tier` + +## Overlord + +### Leadership + +#### GET + +* `/druid/indexer/v1/leader` + +Returns the current leader overlord of the cluster. If you have multiple overlords, just one is leading at any given time. The others are on standby. + +* `/druid/indexer/v1/isLeader` + +This returns a JSON object with field "leader", either true or false. In addition, this call returns HTTP 200 if the +server is the current leader and HTTP 404 if not. This is suitable for use as a load balancer status check if you +only want the active leader to be considered in-service at the load balancer. + +### Tasks + +#### GET + +* `/druid/indexer/v1/task/{taskId}/status` + +Retrieve the status of a task. + +* `/druid/indexer/v1/task/{taskId}/segments` + +Retrieve information about the segments of a task. + +#### POST + +* `/druid/indexer/v1/task` + +Endpoint for submitting tasks and supervisor specs to the overlord. Returns the taskId of the submitted task. + +* `druid/indexer/v1/task/{taskId}/shutdown` + +Shuts down a task. + + +## MiddleManager + +The MiddleManager does not have any API endpoints beyond the [common endpoints](#common). + +## Peon + +The Peon does not have any API endpoints beyond the [common endpoints](#common). + +## Broker + + +### Datasource Information + +#### GET + +* `/druid/v2/datasources` + +Returns a list of queryable datasources. + +* `/druid/v2/datasources/{dataSourceName}` + +Returns the dimensions and metrics of the datasource. Optionally, you can provide request parameter "full" to get list of served intervals with dimensions and metrics being served for those intervals. You can also provide request param "interval" explicitly to refer to a particular interval. + +If no interval is specified, a default interval spanning a configurable period before the current time will be used. The duration of this interval is specified in ISO8601 format via: + +druid.query.segmentMetadata.defaultHistory + +* `/druid/v2/datasources/{dataSourceName}/dimensions` + +Returns the dimensions of the datasource. + +* `/druid/v2/datasources/{dataSourceName}/metrics` + +Returns the metrics of the datasource. + +* `/druid/v2/datasources/{dataSourceName}/candidates?intervals={comma-separated-intervals-in-ISO8601-format}&numCandidates={numCandidates}` + +Returns segment information lists including server locations for the given datasource and intervals. If "numCandidates" is not specified, it will return all servers for each interval. + +### Load Status + +#### GET + +* `/druid/broker/v1/loadstatus` + +Returns a flag indicating if the broker knows about all segments in Zookeeper. This can be used to know when a broker node is ready to be queried after a restart. + + +### Queries + +#### POST + +* `/druid/v2/` + +The endpoint for submitting queries. Accepts an option `?pretty` that pretty prints the results. + +* `/druid/v2/candidates/` + +Returns segment information lists including server locations for the given query.. + + +## Historical + +### Segment Loading + +#### GET + +* `/druid/historical/v1/loadstatus` + +Returns JSON of the form `{"cacheInitialized":}`, where value is either `true` or `false` indicating if all +segments in the local cache have been loaded. This can be used to know when a historical node is ready +to be queried after a restart. + +* `/druid/historical/v1/readiness` + +Similar to `/druid/historical/v1/loadstatus`, but instead of returning JSON with a flag, responses 200 OK if segments +in the local cache have been loaded, and 503 SERVICE UNAVAILABLE, if they haven't. diff --git a/docs/content/toc.md b/docs/content/toc.md index a9d91c8b4fd..fcf905a4b6f 100644 --- a/docs/content/toc.md +++ b/docs/content/toc.md @@ -67,6 +67,13 @@ layout: toc * [ZooKeeper](/docs/VERSION/dependencies/zookeeper.html) ## Operations + * [API Reference](/docs/VERSION/operations/api-reference.html) + * [Coordinator](/docs/VERSION/operations/api-reference.html#coordinator) + * [Overlord](/docs/VERSION/operations/api-reference.html#overlord) + * [MiddleManager](/docs/VERSION/operations/api-reference.html#middlemanager) + * [Peon](/docs/VERSION/operations/api-reference.html#peon) + * [Broker](/docs/VERSION/operations/api-reference.html#broker) + * [Historical](/docs/VERSION/operations/api-reference.html#historical) * [Good Practices](/docs/VERSION/operations/recommendations.html) * [Including Extensions](/docs/VERSION/operations/including-extensions.html) * [Data Retention](/docs/VERSION/operations/rule-configuration.html)