From ee9cfc7e18e11704baa6c33856fdbb0f79a870d5 Mon Sep 17 00:00:00 2001 From: Nhi Pham <56242907+demo-kratia@users.noreply.github.com> Date: Thu, 27 Jul 2023 14:24:55 -0700 Subject: [PATCH] Tasks API documentation spacing update (#14633) --- docs/api-reference/tasks-api.md | 381 ++++++++++++++++++++------------ website/.spelling | 7 +- 2 files changed, 247 insertions(+), 141 deletions(-) diff --git a/docs/api-reference/tasks-api.md b/docs/api-reference/tasks-api.md index 1d745abfd29..4f7261a5b84 100644 --- a/docs/api-reference/tasks-api.md +++ b/docs/api-reference/tasks-api.md @@ -25,17 +25,18 @@ sidebar_label: Tasks This document describes the API endpoints for task retrieval, submission, and deletion for Apache Druid. Tasks are individual jobs performed by Druid to complete operations such as ingestion, querying, and compaction. -In this document, `http://:` is a placeholder for the server address of deployment and the service port. For example, on the quickstart configuration, replace `http://:` with `http://localhost:8888`. +In this topic, `http://ROUTER_IP:ROUTER_PORT` is a placeholder for the Router service address and port. For example, on the quickstart configuration, use `http://localhost:8888`. ## Task information and retrieval ### Get an array of tasks -#### URL -GET `/druid/indexer/v1/tasks` - Retrieves an array of all tasks in the Druid cluster. Each task object includes information on its ID, status, associated datasource, and other metadata. For definitions of the response properties, see the [Tasks table](../querying/sql-metadata-tables.md#tasks-table). +#### URL + +GET /druid/indexer/v1/tasks + #### Query parameters The endpoint supports a set of optional query parameters to filter results. @@ -53,14 +54,17 @@ The endpoint supports a set of optional query parameters to filter results. -
+ *Successfully retrieved list of tasks* + -
+ *Invalid `state` query parameter value* + -
+ *Invalid query parameter* + --- @@ -77,13 +81,16 @@ The following example shows how to retrieve a list of tasks filtered with the fo + ```shell -curl "http://:/druid/indexer/v1/tasks/?state=complete&datasource=wikipedia_api&createdTimeInterval=2015-09-12_2015-09-13&max=10&type=query_worker" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/tasks/?state=complete&datasource=wikipedia_api&createdTimeInterval=2015-09-12_2015-09-13&max=10&type=query_worker" ``` + + ```HTTP GET /druid/indexer/v1/tasks/?state=complete&datasource=wikipedia_api&createdTimeInterval=2015-09-12_2015-09-13&max=10&type=query_worker HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` @@ -156,11 +163,12 @@ Host: http://: ### Get an array of complete tasks -#### URL -GET `/druid/indexer/v1/completeTasks` - Retrieves an array of completed tasks in the Druid cluster. This is functionally equivalent to `/druid/indexer/v1/tasks?state=complete`. For definitions of the response properties, see the [Tasks table](../querying/sql-metadata-tables.md#tasks-table). +#### URL + +GET /druid/indexer/v1/completeTasks + #### Query parameters The endpoint supports a set of optional query parameters to filter results. @@ -177,11 +185,13 @@ The endpoint supports a set of optional query parameters to filter results. -
+ *Successfully retrieved list of complete tasks* + -
+ *Request sent to incorrect service* + --- @@ -191,14 +201,18 @@ The endpoint supports a set of optional query parameters to filter results. + ```shell -curl "http://:/druid/indexer/v1/completeTasks" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/completeTasks" ``` + + ```HTTP GET /druid/indexer/v1/completeTasks HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response @@ -251,11 +265,12 @@ Host: http://: ### Get an array of running tasks -#### URL -GET `/druid/indexer/v1/runningTasks` - Retrieves an array of running task objects in the Druid cluster. It is functionally equivalent to `/druid/indexer/v1/tasks?state=running`. For definitions of the response properties, see the [Tasks table](../querying/sql-metadata-tables.md#tasks-table). +#### URL + +GET /druid/indexer/v1/runningTasks + #### Query parameters The endpoint supports a set of optional query parameters to filter results. @@ -272,7 +287,7 @@ The endpoint supports a set of optional query parameters to filter results. -
+ *Successfully retrieved list of running tasks* @@ -284,14 +299,18 @@ The endpoint supports a set of optional query parameters to filter results. + ```shell -curl "http://:/druid/indexer/v1/runningTasks" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/runningTasks" ``` + + ```HTTP GET /druid/indexer/v1/runningTasks HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + @@ -327,11 +346,12 @@ Host: http://: ### Get an array of waiting tasks -#### URL -GET `/druid/indexer/v1/waitingTasks` - Retrieves an array of waiting tasks in the Druid cluster. It is functionally equivalent to `/druid/indexer/v1/tasks?state=waiting`. For definitions of the response properties, see the [Tasks table](../querying/sql-metadata-tables.md#tasks-table). +#### URL + +GET /druid/indexer/v1/waitingTasks + #### Query parameters The endpoint supports a set of optional query parameters to filter results. @@ -348,7 +368,7 @@ The endpoint supports a set of optional query parameters to filter results. -
+ *Successfully retrieved list of waiting tasks* @@ -360,14 +380,18 @@ The endpoint supports a set of optional query parameters to filter results. + ```shell -curl "http://:/druid/indexer/v1/waitingTasks" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/waitingTasks" ``` + + ```HTTP GET /druid/indexer/v1/waitingTasks HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response @@ -438,11 +462,11 @@ Host: http://: ### Get an array of pending tasks +Retrieves an array of pending tasks in the Druid cluster. It is functionally equivalent to `/druid/indexer/v1/tasks?state=pending`. For definitions of the response properties, see the [Tasks table](../querying/sql-metadata-tables.md#tasks-table). + #### URL -GET `/druid/indexer/v1/pendingTasks` - -Retrieves an array of pending tasks in the Druid cluster. It is functionally equivalent to `/druid/indexer/v1/tasks?state=pending`. For definitions of the response properties, see the [Tasks table](../querying/sql-metadata-tables.md#tasks-table). +GET /druid/indexer/v1/pendingTasks #### Query parameters @@ -460,8 +484,9 @@ The endpoint supports a set of optional query parameters to filter results. -
+ *Successfully retrieved list of pending tasks* + --- @@ -471,14 +496,18 @@ The endpoint supports a set of optional query parameters to filter results. + ```shell -curl "http://:/druid/indexer/v1/pendingTasks" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/pendingTasks" ``` + + ```HTTP GET /druid/indexer/v1/pendingTasks HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response @@ -531,19 +560,22 @@ Host: http://: ### Get task payload -#### URL -GET `/druid/indexer/v1/task/{taskId}` - Retrieves the payload of a task given the task ID. It returns a JSON object with the task ID and payload that includes task configuration details and relevant specifications associated with the execution of the task. +#### URL + +GET /druid/indexer/v1/task/:taskId + #### Responses + -
+ *Successfully retrieved payload of task* + -
+ *Cannot find task with ID* @@ -557,14 +589,18 @@ The following examples shows how to retrieve the task payload of a task with the + ```shell -curl "http://:/druid/indexer/v1/task/index_parallel_wikipedia_short_iajoonnd_2023-07-07T17:53:12.174Z" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task/index_parallel_wikipedia_short_iajoonnd_2023-07-07T17:53:12.174Z" ``` + + ```HTTP GET /druid/indexer/v1/task/index_parallel_wikipedia_short_iajoonnd_2023-07-07T17:53:12.174Z HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + @@ -727,20 +763,24 @@ Host: http://: ### Get task status -#### URL -GET `/druid/indexer/v1/task/{taskId}/status` - Retrieves the status of a task given the task ID. It returns a JSON object with the task's status code, runner status, task type, datasource, and other relevant metadata. +#### URL + +GET /druid/indexer/v1/task/:taskId/status + #### Responses + -
+ *Successfully retrieved task status* + -
+ *Cannot find task with ID* + --- @@ -752,18 +792,23 @@ The following examples shows how to retrieve the status of a task with the speci + ```shell -curl "http://:/druid/indexer/v1/task/query-223549f8-b993-4483-b028-1b0d54713cad/status" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task/query-223549f8-b993-4483-b028-1b0d54713cad/status" ``` + + ```HTTP GET /druid/indexer/v1/task/query-223549f8-b993-4483-b028-1b0d54713cad/status HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response +
Click to show sample response @@ -791,22 +836,24 @@ Host: http://: ### Get task segments -#### URL - -GET `/druid/indexer/v1/task/{taskId}/segments` - > This API is deprecated and will be removed in future releases. Retrieves information about segments generated by the task given the task ID. To hit this endpoint, make sure to enable the audit log config on the Overlord with `druid.indexer.auditLog.enabled = true`. In addition to enabling audit logs, configure a cleanup strategy to prevent overloading the metadata store with old audit logs which may cause performance issues. To enable automated cleanup of audit logs on the Coordinator, set `druid.coordinator.kill.audit.on`. You may also manually export the audit logs to external storage. For more information, see [Audit records](../operations/clean-metadata-store.md#audit-records). +#### URL + +GET /druid/indexer/v1/task/:taskId/segments + #### Responses + -
+ *Successfully retrieved task segments* + --- @@ -818,14 +865,18 @@ The following examples shows how to retrieve the task segment of the task with t + ```shell -curl "http://:/druid/indexer/v1/task/query-52a8aafe-7265-4427-89fe-dc51275cc470/reports" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task/query-52a8aafe-7265-4427-89fe-dc51275cc470/reports" ``` + + ```HTTP GET /druid/indexer/v1/task/query-52a8aafe-7265-4427-89fe-dc51275cc470/reports HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response @@ -834,15 +885,16 @@ A successful request returns a `200 OK` response and an array of the task segmen ### Get task log -#### URL - -GET `/druid/indexer/v1/task/{taskId}/log` - Retrieves the event log associated with a task. It returns a list of logged events during the lifecycle of the task. The endpoint is useful for providing information about the execution of the task, including any errors or warnings raised. Task logs are automatically retrieved from the Middle Manager/Indexer or in long-term storage. For reference, see [Task logs](../ingestion/tasks.md#task-logs). +#### URL + +GET /druid/indexer/v1/task/:taskId/log + #### Query parameters + * `offset` (optional) * Type: Int * Exclude the first passed in number of entries from the response. @@ -852,8 +904,9 @@ Task logs are automatically retrieved from the Middle Manager/Indexer or in long -
+ *Successfully retrieved task log* + --- @@ -865,14 +918,18 @@ The following examples shows how to retrieve the task log of a task with the spe + ```shell -curl "http://:/druid/indexer/v1/task/index_kafka_social_media_0e905aa31037879_nommnaeg/log" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task/index_kafka_social_media_0e905aa31037879_nommnaeg/log" ``` + + ```HTTP GET /druid/indexer/v1/task/index_kafka_social_media_0e905aa31037879_nommnaeg/log HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response @@ -946,19 +1003,20 @@ Host: http://: ### Get task completion report +Retrieves a [task completion report](../ingestion/tasks.md#task-reports) for a task. It returns a JSON object with information about the number of rows ingested, and any parse exceptions that Druid raised. + #### URL -GET `/druid/indexer/v1/task/{taskId}/reports` - -Retrieves a [task completion report](../ingestion/tasks.md#task-reports) for a task. It returns a JSON object with information about the number of rows ingested, and any parse exceptions that Druid raised. +GET /druid/indexer/v1/task/:taskId/reports #### Responses -
+ *Successfully retrieved task report* + --- @@ -970,17 +1028,22 @@ The following examples shows how to retrieve the completion report of a task wit + ```shell -curl "http://:/druid/indexer/v1/task/query-52a8aafe-7265-4427-89fe-dc51275cc470/reports" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task/query-52a8aafe-7265-4427-89fe-dc51275cc470/reports" ``` + + ```HTTP GET /druid/indexer/v1/task/query-52a8aafe-7265-4427-89fe-dc51275cc470/reports HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` + #### Sample response +
Click to show sample response @@ -1022,29 +1085,34 @@ Host: http://: ### Submit a task -#### URL - -POST `/druid/indexer/v1/task` - Submits a JSON-based ingestion spec or supervisor spec to the Overlord. It returns the task ID of the submitted task. For information on creating an ingestion spec, refer to the [ingestion spec reference](../ingestion/ingestion-spec.md). Note that for most batch ingestion use cases, you should use the [SQL-ingestion API](./sql-ingestion-api.md) instead of JSON-based batch ingestion. +#### URL + +POST /druid/indexer/v1/task + #### Responses + -
+ *Successfully submitted task* + -
+ *Missing information in query* + -
+ *Incorrect request body media type* + -
+ *Unexpected token or characters in request body* + --- @@ -1056,54 +1124,57 @@ The following request is an example of submitting a task to create a datasource + ```shell -curl "http://:/druid/indexer/v1/task" \ ---header "Content-Type: application/json" \ ---data "{ - \"type\" : \"index_parallel\", - \"spec\" : { - \"dataSchema\" : { - \"dataSource\" : \"wikipedia_auto\", - \"timestampSpec\": { - \"column\": \"time\", - \"format\": \"iso\" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task" \ +--header 'Content-Type: application/json' \ +--data '{ + "type" : "index_parallel", + "spec" : { + "dataSchema" : { + "dataSource" : "wikipedia_auto", + "timestampSpec": { + "column": "time", + "format": "iso" }, - \"dimensionsSpec\" : { - \"useSchemaDiscovery\": true + "dimensionsSpec" : { + "useSchemaDiscovery": true }, - \"metricsSpec\" : [], - \"granularitySpec\" : { - \"type\" : \"uniform\", - \"segmentGranularity\" : \"day\", - \"queryGranularity\" : \"none\", - \"intervals\" : [\"2015-09-12/2015-09-13\"], - \"rollup\" : false + "metricsSpec" : [], + "granularitySpec" : { + "type" : "uniform", + "segmentGranularity" : "day", + "queryGranularity" : "none", + "intervals" : ["2015-09-12/2015-09-13"], + "rollup" : false } }, - \"ioConfig\" : { - \"type\" : \"index_parallel\", - \"inputSource\" : { - \"type\" : \"local\", - \"baseDir\" : \"quickstart/tutorial/\", - \"filter\" : \"wikiticker-2015-09-12-sampled.json.gz\" + "ioConfig" : { + "type" : "index_parallel", + "inputSource" : { + "type" : "local", + "baseDir" : "quickstart/tutorial/", + "filter" : "wikiticker-2015-09-12-sampled.json.gz" }, - \"inputFormat\" : { - \"type\" : \"json\" + "inputFormat" : { + "type" : "json" }, - \"appendToExisting\" : false + "appendToExisting" : false }, - \"tuningConfig\" : { - \"type\" : \"index_parallel\", - \"maxRowsPerSegment\" : 5000000, - \"maxRowsInMemory\" : 25000 + "tuningConfig" : { + "type" : "index_parallel", + "maxRowsPerSegment" : 5000000, + "maxRowsInMemory" : 25000 } } -}" +}' + ``` + ```HTTP POST /druid/indexer/v1/task HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT Content-Type: application/json Content-Length: 952 @@ -1148,9 +1219,11 @@ Content-Length: 952 } } ``` + #### Sample response +
Click to show sample response @@ -1164,20 +1237,24 @@ Content-Length: 952 ### Shut down a task -#### URL -POST `/druid/indexer/v1/task/{taskId}/shutdown` - Shuts down a task if it not already complete. Returns a JSON object with the ID of the task that was shut down successfully. +#### URL + +POST /druid/indexer/v1/task/:taskId/shutdown + #### Responses + -
+ *Successfully shut down task* + -
+ *Cannot find task with ID or task is no longer running* + --- @@ -1189,18 +1266,22 @@ The following request shows how to shut down a task with the ID `query-52as 8aaf + ```shell -curl --request POST "http://:/druid/indexer/v1/task/query-52as 8aafe-7265-4427-89fe-dc51275cc470/shutdown" +curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/task/query-52as 8aafe-7265-4427-89fe-dc51275cc470/shutdown" ``` + + ```HTTP POST /druid/indexer/v1/task/query-52as 8aafe-7265-4427-89fe-dc51275cc470/shutdown HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` #### Sample response +
Click to show sample response @@ -1214,21 +1295,24 @@ Host: http://: ### Shut down all tasks for a datasource -#### URL -POST `/druid/indexer/v1/datasources/{dataSource}/shutdownAllTasks` - Shuts down all tasks for a specified datasource. If successful, it returns a JSON object with the name of the datasource whose tasks are shut down. +#### URL + +POST /druid/indexer/v1/datasources/:datasource/shutdownAllTasks + #### Responses -
+ *Successfully shut down tasks* + -
+ *Error or datasource does not have a running task* + --- @@ -1240,18 +1324,22 @@ The following request is an example of shutting down all tasks for datasource `w + ```shell -curl --request POST "http://:/druid/indexer/v1/datasources/wikipedia_auto/shutdownAllTasks" +curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/datasources/wikipedia_auto/shutdownAllTasks" ``` + + ```HTTP POST /druid/indexer/v1/datasources/wikipedia_auto/shutdownAllTasks HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` #### Sample response +
Click to show sample response @@ -1267,21 +1355,24 @@ Host: http://: ### Retrieve status objects for tasks -#### URL -POST `/druid/indexer/v1/taskStatus` - Retrieves list of task status objects for list of task ID strings in request body. It returns a set of JSON objects with the status, duration, location of each task, and any error messages. +#### URL + +POST /druid/indexer/v1/taskStatus + #### Responses -
+ *Successfully retrieved status objects* + -
+ *Missing request body or incorrect request body type* + --- @@ -1293,24 +1384,29 @@ The following request is an example of retrieving status objects for task ID `in + ```shell -curl "http://:/druid/indexer/v1/taskStatus" \ ---header "Content-Type: application/json" \ ---data "[\"index_parallel_wikipedia_auto_jndhkpbo_2023-06-26T17:23:05.308Z\", \"index_parallel_wikipedia_auto_jbgiianh_2023-06-26T23:17:56.769Z\"]" +curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/taskStatus" \ +--header 'Content-Type: application/json' \ +--data '["index_parallel_wikipedia_auto_jndhkpbo_2023-06-26T17:23:05.308Z","index_parallel_wikipedia_auto_jbgiianh_2023-06-26T23:17:56.769Z"]' ``` + + ```HTTP POST /druid/indexer/v1/taskStatus HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT Content-Type: application/json Content-Length: 134 ["index_parallel_wikipedia_auto_jndhkpbo_2023-06-26T17:23:05.308Z", "index_parallel_wikipedia_auto_jbgiianh_2023-06-26T23:17:56.769Z"] ``` + #### Sample response +
Click to show sample response @@ -1345,20 +1441,21 @@ Content-Length: 134 ### Clean up pending segments for a datasource -#### URL - -DELETE `/druid/indexer/v1/pendingSegments/{dataSource}` - Manually clean up pending segments table in metadata storage for `datasource`. It returns a JSON object response with `numDeleted` for the number of rows deleted from the pending segments table. This API is used by the `druid.coordinator.kill.pendingSegments.on` [Coordinator setting](../configuration/index.md#coordinator-operation) which automates this operation to perform periodically. +#### URL + +DELETE /druid/indexer/v1/pendingSegments/:datasource + #### Responses + -
+ *Successfully deleted pending segments* @@ -1372,18 +1469,22 @@ The following request is an example of cleaning up pending segments for the `wik + ```shell -curl --request DELETE "http://:/druid/indexer/v1/pendingSegments/wikipedia_api" +curl --request DELETE "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/pendingSegments/wikipedia_api" ``` + + ```HTTP DELETE /druid/indexer/v1/pendingSegments/wikipedia_api HTTP/1.1 -Host: http://: +Host: http://ROUTER_IP:ROUTER_PORT ``` #### Sample response +
Click to show sample response diff --git a/website/.spelling b/website/.spelling index 4540128c5ac..775c056dcf0 100644 --- a/website/.spelling +++ b/website/.spelling @@ -2290,10 +2290,15 @@ prometheus-emitter kubernetes-overlord-extensions UCS ISO646-US +completeTasks +runningTasks +waitingTasks +pendingTasks +shutdownAllTasks supervisorId suspendAll resumeAll terminateAll selfDiscovered -isLeader loadstatus +isLeader