From 0f157366878c1ecd2f72770fafeca12e130129ea Mon Sep 17 00:00:00 2001 From: Lisa Cawley Date: Wed, 20 Nov 2019 10:43:53 -0800 Subject: [PATCH] [DOCS] Reformat rollup API docs (#49397) --- docs/reference/rollup/apis/get-job.asciidoc | 6 +- docs/reference/rollup/apis/put-job.asciidoc | 2 +- .../rollup/apis/rollup-caps.asciidoc | 94 ++++++++------- .../rollup/apis/rollup-index-caps.asciidoc | 89 +++++++------- .../rollup/apis/rollup-search.asciidoc | 110 +++++++++++------- docs/reference/rollup/apis/start-job.asciidoc | 4 +- docs/reference/rollup/apis/stop-job.asciidoc | 4 +- 7 files changed, 176 insertions(+), 133 deletions(-) diff --git a/docs/reference/rollup/apis/get-job.asciidoc b/docs/reference/rollup/apis/get-job.asciidoc index a7de2115491..7a37aad061b 100644 --- a/docs/reference/rollup/apis/get-job.asciidoc +++ b/docs/reference/rollup/apis/get-job.asciidoc @@ -18,9 +18,9 @@ experimental[] [[rollup-get-job-prereqs]] ==== {api-prereq-title} -* You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster -privileges to use this API. For more information, see -<>. +* If the {es} {security-features} are enabled, you must have `monitor`, +`monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. +For more information, see <>. [[rollup-get-job-desc]] ==== {api-description-title} diff --git a/docs/reference/rollup/apis/put-job.asciidoc b/docs/reference/rollup/apis/put-job.asciidoc index 8916567e1fe..1816edc8038 100644 --- a/docs/reference/rollup/apis/put-job.asciidoc +++ b/docs/reference/rollup/apis/put-job.asciidoc @@ -32,7 +32,7 @@ Jobs are created in a `STOPPED` state. You can start them with the [[rollup-put-job-api-path-params]] ==== {api-path-parms-title} -`job_id`:: +``:: (Required, string) Identifier for the {rollup-job}. [[rollup-put-job-api-request-body]] diff --git a/docs/reference/rollup/apis/rollup-caps.asciidoc b/docs/reference/rollup/apis/rollup-caps.asciidoc index 8358ccfed26..4d4197a0250 100644 --- a/docs/reference/rollup/apis/rollup-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-caps.asciidoc @@ -1,17 +1,32 @@ [role="xpack"] [testenv="basic"] [[rollup-get-rollup-caps]] -=== Get rollup job capabilities API +=== Get {rollup-job} capabilities API ++++ Get rollup caps ++++ +Returns the capabilities of any {rollup-jobs} that have been configured for a +specific index or index pattern. + experimental[] -This API returns the capabilities of any rollup jobs that have been configured -for a specific index or index pattern. +[[rollup-get-rollup-caps-request]] +==== {api-request-title} -This API is useful because a rollup job is often configured to rollup only a +`GET _rollup/data/` + +[[rollup-get-rollup-caps-prereqs]] +==== {api-prereq-title} + +* If the {es} {security-features} are enabled, you must have `monitor`, +`monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. +For more information, see <>. + +[[rollup-get-rollup-caps-desc]] +==== {api-description-title} + +This API is useful because a {rollup-job} is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on that configuration. @@ -22,33 +37,20 @@ This API enables you to inspect an index and determine: 2. If yes to the first question, what fields were rolled up, what aggregations can be performed, and where does the data live? -==== Request +[[rollup-get-rollup-path-params]] +==== {api-path-parms-title} -`GET _rollup/data/{index}` - -//===== Description - -==== Path Parameters - -`index`:: - (string) Index, indices or index-pattern to return rollup capabilities for. `_all` may be used to fetch - rollup capabilities from all jobs +``:: + (string) Index, indices or index-pattern to return rollup capabilities for. + `_all` may be used to fetch rollup capabilities from all jobs. -==== Request Body +[[rollup-get-rollup-example]] +==== {api-examples-title} -There is no request body for the Get Rollup Caps API. - -==== Authorization - -You must have `monitor`, `monitor_rollup`, `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see -<>. - -==== Examples - -Imagine we have an index named `sensor-1` full of raw data. We know that the data will grow over time, so there -will be a `sensor-2`, `sensor-3`, etc. Let's create a Rollup job that targets the index pattern `sensor-*` to accommodate +Imagine we have an index named `sensor-1` full of raw data. We know that the +data will grow over time, so there will be a `sensor-2`, `sensor-3`, etc. Let's +create a {rollup-job} that targets the index pattern `sensor-*` to accommodate this future scaling: [source,console] @@ -83,7 +85,8 @@ PUT _rollup/job/sensor -------------------------------------------------- // TEST[setup:sensor_index] -We can then retrieve the rollup capabilities of that index pattern (`sensor-*`) via the following command: +We can then retrieve the rollup capabilities of that index pattern (`sensor-*`) +via the following command: [source,console] -------------------------------------------------- @@ -139,17 +142,21 @@ Which will yield the following response: } ---- -The response that is returned contains information that is similar to the original Rollup configuration, but formatted -differently. First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data, -the index pattern that the job was targeting. +The response that is returned contains information that is similar to the +original rollup configuration, but formatted differently. First, there are some +house-keeping details: the {rollup-job} ID, the index that holds the rolled data, +and the index pattern that the job was targeting. -Next it shows a list of fields that contain data eligible for rollup searches. Here we see four fields: `node`, `temperature`, -`timestamp` and `voltage`. Each of these fields list the aggregations that are possible. For example, you can use a min, max -or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`. +Next it shows a list of fields that contain data eligible for rollup searches. +Here we see four fields: `node`, `temperature`, `timestamp` and `voltage`. Each +of these fields list the aggregations that are possible. For example, you can +use a min, max or sum aggregation on the `temperature` field, but only a +`date_histogram` on `timestamp`. -Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index -or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various -configurations available. +Note that the `rollup_jobs` element is an array; there can be multiple, +independent jobs configured for a single index or index pattern. Each of these +jobs may have different configurations, so the API returns a list of all the +various configurations available. We could also retrieve the same information with a request to `_all`: @@ -159,7 +166,8 @@ GET _rollup/data/_all -------------------------------------------------- // TEST[continued] -But note that if we use the concrete index name (`sensor-1`), we'll retrieve no rollup capabilities: +But note that if we use the concrete index name (`sensor-1`), we'll retrieve no +rollup capabilities: [source,console] -------------------------------------------------- @@ -174,7 +182,9 @@ GET _rollup/data/sensor-1 } ---- -Why is this? The original rollup job was configured against a specific index pattern (`sensor-*`) not a concrete index -(`sensor-1`). So while the index belongs to the pattern, the rollup job is only valid across the entirety of the pattern -not just one of it's containing indices. So for that reason, the Rollup Capabilities API only returns information based -on the originally configured index name or pattern. +Why is this? The original {rollup-job} was configured against a specific index +pattern (`sensor-*`) not a concrete index (`sensor-1`). So while the index +belongs to the pattern, the {rollup-job} is only valid across the entirety of +the pattern not just one of it's containing indices. So for that reason, the +get rollup capabilities API only returns information based on the originally +configured index name or pattern. diff --git a/docs/reference/rollup/apis/rollup-index-caps.asciidoc b/docs/reference/rollup/apis/rollup-index-caps.asciidoc index 291d8fb4f19..9fb770d33c6 100644 --- a/docs/reference/rollup/apis/rollup-index-caps.asciidoc +++ b/docs/reference/rollup/apis/rollup-index-caps.asciidoc @@ -6,41 +6,48 @@ Get rollup index caps ++++ +Returns the rollup capabilities of all jobs inside of a rollup index (e.g. the +index where rollup data is stored). + experimental[] -This API returns the rollup capabilities of all jobs inside of a rollup index (e.g. the index where rollup data is stored). -A single rollup index may store the data for multiple rollup jobs, and may have a variety of capabilities depending on those jobs. +[[rollup-get-rollup-index-caps-request]] +==== {api-request-title} + +`GET /_rollup/data` + +[[rollup-get-rollup-index-caps-prereqs]] +==== {api-prereq-title} + +* If the {es} {security-features} are enabled, you must have the `read` index +privilege on the index that stores the rollup results. For more information, see +<>. + +[[rollup-get-rollup-index-caps-desc]] +==== {api-description-title} + +A single rollup index may store the data for multiple {rollup-jobs}, and may +have a variety of capabilities depending on those jobs. This API will allow you to determine: 1. What jobs are stored in an index (or indices specified via a pattern)? -2. What target indices were rolled up, what fields were used in those rollups and what aggregations can be performed on each job? +2. What target indices were rolled up, what fields were used in those rollups +and what aggregations can be performed on each job? -==== Request +[[rollup-get-rollup-index-caps-path-params]] +==== {api-path-parms-title} -`GET {index}/_rollup/data` +``:: + (Required, string) Index or index-pattern of concrete rollup indices to check + for capabilities. -//===== Description +[[rollup-get-rollup-index-caps-example]] +==== {api-examples-title} -==== Path Parameters - -`index`:: - (string) Index or index-pattern of concrete rollup indices to check for capabilities. - -==== Request Body - -There is no request body for the Get Jobs API. - -==== Authorization - -You must have the `read` index privilege on the index that stores the rollup results. -For more information, see -<>. - -==== Examples - -Imagine we have an index named `sensor-1` full of raw data. We know that the data will grow over time, so there -will be a `sensor-2`, `sensor-3`, etc. Let's create a Rollup job, which stores it's data in `sensor_rollup`: +Imagine we have an index named `sensor-1` full of raw data. We know that the +data will grow over time, so there will be a `sensor-2`, `sensor-3`, etc. +Let's create a {rollup-job} that stores its data in `sensor_rollup`: [source,console] -------------------------------------------------- @@ -74,8 +81,8 @@ PUT _rollup/job/sensor -------------------------------------------------- // TEST[setup:sensor_index] -If at a later date, we'd like to determine what jobs and capabilities were stored in the `sensor_rollup` index, we can use the Get Rollup -Index API: +If at a later date, we'd like to determine what jobs and capabilities were +stored in the `sensor_rollup` index, we can use the get rollup index API: [source,console] -------------------------------------------------- @@ -83,8 +90,8 @@ GET /sensor_rollup/_rollup/data -------------------------------------------------- // TEST[continued] -Note how we are requesting the concrete rollup index name (`sensor_rollup`) as the first part of the URL. -This will yield the following response: +Note how we are requesting the concrete rollup index name (`sensor_rollup`) as +the first part of the URL. This will yield the following response: [source,console-result] ---- @@ -133,20 +140,24 @@ This will yield the following response: ---- -The response that is returned contains information that is similar to the original Rollup configuration, but formatted -differently. First, there are some house-keeping details: the Rollup job's ID, the index that holds the rolled data, +The response that is returned contains information that is similar to the +original rollup configuration, but formatted differently. First, there are some +house-keeping details: the {rollup-job} ID, the index that holds the rolled data, the index pattern that the job was targeting. -Next it shows a list of fields that contain data eligible for rollup searches. Here we see four fields: `node`, `temperature`, -`timestamp` and `voltage`. Each of these fields list the aggregations that are possible. For example, you can use a min, max -or sum aggregation on the `temperature` field, but only a `date_histogram` on `timestamp`. +Next it shows a list of fields that contain data eligible for rollup searches. +Here we see four fields: `node`, `temperature`, `timestamp` and `voltage`. Each +of these fields list the aggregations that are possible. For example, you can +use a min, max, or sum aggregation on the `temperature` field, but only a +`date_histogram` on `timestamp`. -Note that the `rollup_jobs` element is an array; there can be multiple, independent jobs configured for a single index -or index pattern. Each of these jobs may have different configurations, so the API returns a list of all the various -configurations available. +Note that the `rollup_jobs` element is an array; there can be multiple, +independent jobs configured for a single index or index pattern. Each of these +jobs may have different configurations, so the API returns a list of all the +various configurations available. - -Like other APIs that interact with indices, you can specify index patterns instead of explicit indices: +Like other APIs that interact with indices, you can specify index patterns +instead of explicit indices: [source,console] -------------------------------------------------- diff --git a/docs/reference/rollup/apis/rollup-search.asciidoc b/docs/reference/rollup/apis/rollup-search.asciidoc index d5d43be0929..3bb4d1ac2fa 100644 --- a/docs/reference/rollup/apis/rollup-search.asciidoc +++ b/docs/reference/rollup/apis/rollup-search.asciidoc @@ -6,50 +6,66 @@ Rollup search ++++ +Enables searching rolled-up data using the standard query DSL. + experimental[] -The Rollup Search endpoint allows searching rolled-up data using the standard query DSL. The Rollup Search endpoint -is needed because, internally, rolled-up documents utilize a different document structure than the original data. The -Rollup Search endpoint rewrites standard query DSL into a format that matches the rollup documents, then takes the response -and rewrites it back to what a client would expect given the original query. +[[rollup-search-request]] +==== {api-request-title} -==== Request +`GET /_rollup_search` -`GET {index}/_rollup_search` +[[rollup-search-desc]] +==== {api-description-title} -//===== Description +The rollup search endpoint is needed because, internally, rolled-up documents +utilize a different document structure than the original data. The rollup search +endpoint rewrites standard query DSL into a format that matches the rollup +documents, then takes the response and rewrites it back to what a client would +expect given the original query. -==== Path Parameters +[[rollup-search-path-params]] +==== {api-path-parms-title} -`index`:: - (string) Index, indices or index-pattern to execute a rollup search against. This can include both rollup and non-rollup - indices. +``:: + (Required, string) Index, indices or index-pattern to execute a rollup search + against. This can include both rollup and non-rollup indices. Rules for the `index` parameter: -- At least one index/index-pattern must be specified. This can be either a rollup or non-rollup index. Omitting the index parameter, -or using `_all`, is not permitted +- At least one index/index-pattern must be specified. This can be either a +rollup or non-rollup index. Omitting the index parameter, or using `_all`, is +not permitted. - Multiple non-rollup indices may be specified -- Only one rollup index may be specified. If more than one are supplied an exception will be thrown -- Index patterns may be used, but if they match more than one rollup index an exception will be thrown. +- Only one rollup index may be specified. If more than one are supplied, an +exception occurs. +- Index patterns may be used, but if they match more than one rollup index an +exception occurs. -==== Request Body +[[rollup-search-request-body]] +==== {api-request-body-title} -The request body supports a subset of features from the regular Search API. It supports: +The request body supports a subset of features from the regular Search API. It +supports: -- `query` param for specifying an DSL query, subject to some limitations (see <> and <> +- `query` param for specifying an DSL query, subject to some limitations +(see <> and <> - `aggregations` param for specifying aggregations Functionality that is not available: -- `size`: because rollups work on pre-aggregated data, no search hits can be returned and so size must be set to zero or -omitted entirely. -- `highlighter`, `suggestors`, `post_filter`, `profile`, `explain` are similarly disallowed +- `size`: Because rollups work on pre-aggregated data, no search hits can be +returned and so size must be set to zero or omitted entirely. +- `highlighter`, `suggestors`, `post_filter`, `profile`, `explain`: These are +similarly disallowed. +[[rollup-search-example]] +==== {api-examples-title} -==== Historical-only search example +===== Historical-only search example -Imagine we have an index named `sensor-1` full of raw data, and we have created a rollup job with the following configuration: +Imagine we have an index named `sensor-1` full of raw data, and we have created +a {rollup-job} with the following configuration: [source,console] -------------------------------------------------- @@ -83,9 +99,10 @@ PUT _rollup/job/sensor -------------------------------------------------- // TEST[setup:sensor_index] -This rolls up the `sensor-*` pattern and stores the results in `sensor_rollup`. To search this rolled up data, we -need to use the `_rollup_search` endpoint. However, you'll notice that we can use regular query DSL to search the -rolled-up data: +This rolls up the `sensor-*` pattern and stores the results in `sensor_rollup`. +To search this rolled up data, we need to use the `_rollup_search` endpoint. +However, you'll notice that we can use regular query DSL to search the rolled-up +data: [source,console] -------------------------------------------------- @@ -104,8 +121,9 @@ GET /sensor_rollup/_rollup_search // TEST[setup:sensor_prefab_data] // TEST[s/_rollup_search/_rollup_search?filter_path=took,timed_out,terminated_early,_shards,hits,aggregations/] -The query is targeting the `sensor_rollup` data, since this contains the rollup data as configured in the job. A `max` -aggregation has been used on the `temperature` field, yielding the following response: +The query is targeting the `sensor_rollup` data, since this contains the rollup +data as configured in the job. A `max` aggregation has been used on the +`temperature` field, yielding the following response: [source,console-result] ---- @@ -132,12 +150,14 @@ aggregation has been used on the `temperature` field, yielding the following res // TESTRESPONSE[s/"took" : 102/"took" : $body.$_path/] // TESTRESPONSE[s/"_shards" : \.\.\. /"_shards" : $body.$_path/] -The response is exactly as you'd expect from a regular query + aggregation; it provides some metadata about the request -(`took`, `_shards`, etc), the search hits (which is always empty for rollup searches), and the aggregation response. +The response is exactly as you'd expect from a regular query + aggregation; it +provides some metadata about the request (`took`, `_shards`, etc), the search +hits (which is always empty for rollup searches), and the aggregation response. -Rollup searches are limited to functionality that was configured in the rollup job. For example, we are not able to calculate -the average temperature because `avg` was not one of the configured metrics for the `temperature` field. If we try -to execute that search: +Rollup searches are limited to functionality that was configured in the +{rollup-job}. For example, we are not able to calculate the average temperature +because `avg` was not one of the configured metrics for the `temperature` field. +If we try to execute that search: [source,console] -------------------------------------------------- @@ -176,11 +196,11 @@ GET sensor_rollup/_rollup_search ---- // TESTRESPONSE[s/"stack_trace": \.\.\./"stack_trace": $body.$_path/] -==== Searching both historical rollup and non-rollup data - -The Rollup Search API has the capability to search across both "live", non-rollup data as well as the aggregated rollup -data. This is done by simply adding the live indices to the URI: +===== Searching both historical rollup and non-rollup data +The rollup search API has the capability to search across both "live" +non-rollup data and the aggregated rollup data. This is done by simply adding +the live indices to the URI: [source,console] -------------------------------------------------- @@ -200,16 +220,18 @@ GET sensor-1,sensor_rollup/_rollup_search <1> // TEST[s/_rollup_search/_rollup_search?filter_path=took,timed_out,terminated_early,_shards,hits,aggregations/] <1> Note the URI now searches `sensor-1` and `sensor_rollup` at the same time -When the search is executed, the Rollup Search endpoint will do two things: +When the search is executed, the rollup search endpoint does two things: -1. The original request will be sent to the non-rollup index unaltered -2. A rewritten version of the original request will be sent to the rollup index. +1. The original request is sent to the non-rollup index unaltered. +2. A rewritten version of the original request is sent to the rollup index. -When the two responses are received, the endpoint will then rewrite the rollup response and merge the two together. -During the merging process, if there is any overlap in buckets between the two responses, the buckets from the non-rollup -index will be used. +When the two responses are received, the endpoint rewrites the rollup response +and merges the two together. During the merging process, if there is any overlap +in buckets between the two responses, the buckets from the non-rollup index are +used. -The response to the above query will look as expected, despite spanning rollup and non-rollup indices: +The response to the above query looks as expected, despite spanning rollup and +non-rollup indices: [source,console-result] ---- diff --git a/docs/reference/rollup/apis/start-job.asciidoc b/docs/reference/rollup/apis/start-job.asciidoc index d5b0a2a4076..28b09375dab 100644 --- a/docs/reference/rollup/apis/start-job.asciidoc +++ b/docs/reference/rollup/apis/start-job.asciidoc @@ -19,8 +19,8 @@ experimental[] [[rollup-start-job-prereqs]] ==== {api-prereq-title} -* You must have `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see +* If the {es} {security-features} are enabled, you must have `manage` or +`manage_rollup` cluster privileges to use this API. For more information, see <>. [[rollup-start-job-desc]] diff --git a/docs/reference/rollup/apis/stop-job.asciidoc b/docs/reference/rollup/apis/stop-job.asciidoc index 254935bf421..11b8ee497a8 100644 --- a/docs/reference/rollup/apis/stop-job.asciidoc +++ b/docs/reference/rollup/apis/stop-job.asciidoc @@ -19,8 +19,8 @@ experimental[] [[rollup-stop-job-prereqs]] ==== {api-prereq-title} -* You must have `manage` or `manage_rollup` cluster privileges to use this API. -For more information, see +* If the {es} {security-features} are enabled, you must have `manage` or +`manage_rollup` cluster privileges to use this API. For more information, see <>. [[rollup-stop-job-desc]]