diff --git a/docs/reference/aggregations/bucket/composite-aggregation.asciidoc b/docs/reference/aggregations/bucket/composite-aggregation.asciidoc index 337cdf8a196..c37e84799ab 100644 --- a/docs/reference/aggregations/bucket/composite-aggregation.asciidoc +++ b/docs/reference/aggregations/bucket/composite-aggregation.asciidoc @@ -1,5 +1,5 @@ [[search-aggregations-bucket-composite-aggregation]] -=== Composite Aggregation +=== Composite aggregation A multi-bucket aggregation that creates composite buckets from different sources. @@ -105,6 +105,7 @@ The name given to each sources must be unique. There are three different types of values source: +[[_terms]] ===== Terms The `terms` value source is equivalent to a simple `terms` aggregation. @@ -155,6 +156,7 @@ GET /_search } -------------------------------------------------- +[[_histogram]] ===== Histogram The `histogram` value source can be applied on numeric values to build fixed size @@ -208,8 +210,8 @@ GET /_search } -------------------------------------------------- - -===== Date Histogram +[[_date_histogram]] +===== Date histogram The `date_histogram` is similar to the `histogram` value source except that the interval is specified by date/time expression: diff --git a/docs/reference/redirects.asciidoc b/docs/reference/redirects.asciidoc index 85bd09ccd32..e6ee868fb80 100644 --- a/docs/reference/redirects.asciidoc +++ b/docs/reference/redirects.asciidoc @@ -878,17 +878,17 @@ See <>. [role="exclude",id="data-frame-transform-dest"] === Dest objects -See <>. +See <>. [role="exclude",id="data-frame-transform-source"] ==== Source objects -See <>. +See <>. [role="exclude",id="data-frame-transform-pivot"] ==== Pivot objects -See <>. +See <>. [role="exclude",id="configuring-monitoring"] === Configuring monitoring @@ -1026,3 +1026,10 @@ See <>. === Rollup job configuration See <>. + +[role="exclude",id="transform-resource"] +=== {transform-cap} resources + +This page was deleted. +See <>, <>, <>, +<>. diff --git a/docs/reference/rest-api/common-parms.asciidoc b/docs/reference/rest-api/common-parms.asciidoc index e5301f99715..0b2effa0ea4 100644 --- a/docs/reference/rest-api/common-parms.asciidoc +++ b/docs/reference/rest-api/common-parms.asciidoc @@ -57,6 +57,42 @@ This parameter also applies to <> that point to a missing or closed index. end::allow-no-indices[] +tag::allow-no-match-transforms1[] +Specifies what to do when the request: ++ +-- +* Contains wildcard expressions and there are no {transforms} that match. +* Contains the `_all` string or no identifiers and there are no matches. +* Contains wildcard expressions and there are only partial matches. + +The default value is `true`, which returns an empty `transforms` array when +there are no matches and the subset of results when there are partial matches. + +If this parameter is `false`, the request returns a `404` status code when there +are no matches or only partial matches. +-- +end::allow-no-match-transforms1[] + +tag::allow-no-match-transforms2[] +Specifies what to do when the request: ++ +-- +* Contains wildcard expressions and there are no {transforms} that match. +* Contains the `_all` string or no identifiers and there are no matches. +* Contains wildcard expressions and there are only partial matches. + +The default value is `true`, which returns a successful acknowledgement message +when there are no matches. When there are only partial matches, the API stops +the appropriate {transforms}. For example, if the request contains +`test-id1*,test-id2*` as the identifiers and there are no {transforms} +that match `test-id2*`, the API nonetheless stops the {transforms} +that match `test-id1*`. + +If this parameter is `false`, the request returns a `404` status code when there +are no matches or only partial matches. +-- +end::allow-no-match-transforms2[] + tag::analyzer[] `analyzer`:: (Optional, string) Analyzer to use for the query string. @@ -96,6 +132,18 @@ tag::default_operator[] Defaults to `OR`. end::default_operator[] +tag::dest[] +The destination for the {transform}. +end::dest[] + +tag::dest-index[] +The _destination index_ for the {transform}. +end::dest-index[] + +tag::dest-pipeline[] +The unique identifier for a <>. +end::dest-pipeline[] + tag::detailed[] `detailed`:: (Optional, boolean) @@ -202,11 +250,22 @@ https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html[HTTP accept header]. Valid values include JSON, YAML, etc. end::http-format[] +tag::frequency[] +The interval between checks for changes in the source indices when the +{transform} is running continuously. Also determines the retry interval in the +event of transient failures while the {transform} is searching or indexing. The +minimum value is `1s` and the maximum is `1h`. The default value is `1m`. +end::frequency[] + tag::from[] `from`:: (Optional, integer) Starting document offset. Defaults to `0`. end::from[] +tag::from-transforms[] +Skips the specified number of {transforms}. The default value is `0`. +end::from-transforms[] + tag::generation[] Generation number, such as `0`. {es} increments this generation number for each segment written. {es} then uses this number to derive the segment name. @@ -484,6 +543,12 @@ or use a value of `-1`. -- end::parent-task-id[] +tag::payloads[] +`payloads`:: +(Optional, boolean) If `true`, the response includes term payloads. +Defaults to `true`. +end::payloads[] + tag::pipeline[] `pipeline`:: (Optional, string) ID of the pipeline to use to preprocess incoming documents. @@ -495,11 +560,52 @@ tag::path-pipeline[] used to limit the request. end::path-pipeline[] -tag::payloads[] -`payloads`:: -(Optional, boolean) If `true`, the response includes term payloads. -Defaults to `true`. -end::payloads[] +tag::pivot[] +The method for transforming the data. These objects define the pivot function +`group by` fields and the aggregation to reduce the data. +end::pivot[] + +tag::pivot-aggs[] +Defines how to aggregate the grouped data. The following composite aggregations +are supported: ++ +-- +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> +* <> + +IMPORTANT: {transforms-cap} support a subset of the functionality in +composite aggregations. See <>. + +-- +end::pivot-aggs[] + +tag::pivot-group-by[] +Defines how to group the data. More than one grouping can be defined + per pivot. The following groupings are supported: ++ +-- +* <<_terms,Terms>> +* <<_histogram,Histogram>> +* <<_date_histogram,Date histogram>> +-- +end::pivot-group-by[] + +tag::pivot-max-page-search-size[] +Defines the initial page size to use for the composite aggregation for each +checkpoint. If circuit breaker exceptions occur, the page size is dynamically +adjusted to a lower value. The minimum value is `10` and the maximum is `10,000`. +The default value is `500`. +end::pivot-max-page-search-size[] tag::positions[] `positions`:: @@ -626,6 +732,11 @@ Configuration options for the target index. See <>. end::target-index-settings[] +tag::size-transforms[] +Specifies the maximum number of {transforms} to obtain. The default value is +`100`. +end::size-transforms[] + tag::slices[] `slices`:: (Optional, integer) The number of slices this task should be divided into. @@ -655,6 +766,21 @@ tag::source_includes[] field. end::source_includes[] +tag::source-transforms[] +The source of the data for the {transform}. +end::source-transforms[] + +tag::source-index-transforms[] +The _source indices_ for the {transform}. It can be a single index, an index +pattern (for example, `"myindex*"`), or an array of indices (for example, +`["index1", "index2"]`). +end::source-index-transforms[] + +tag::source-query-transforms[] +A query clause that retrieves a subset of data from the source index. See +<>. +end::source-query-transforms[] + tag::stats[] `stats`:: (Optional, string) Specific `tag` of the request for logging and statistical @@ -667,6 +793,24 @@ tag::stored_fields[] index rather than the document `_source`. Defaults to `false`. end::stored_fields[] +tag::sync[] +Defines the properties {transforms} require to run continuously. +end::sync[] + +tag::sync-time[] +Specifies that the {transform} uses a time field to synchronize the source and +destination indices. +end::sync-time[] + +tag::sync-time-field[] +The date field that is used to identify new documents in the source. +end::sync-time-field[] + +tag::sync-time-delay[] +The time delay between the current time and the latest input data time. The +default value is `60s`. +end::sync-time-delay[] + tag::target-index[] ``:: + @@ -727,6 +871,18 @@ Comma-separated list or wildcard expression of types used to limit the request. end::type[] +tag::transform-id[] +Identifier for the {transform}. This identifier can contain lowercase +alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start +and end with alphanumeric characters. +end::transform-id[] + +tag::transform-id-wildcard[] +Identifier for the {transform}. It can be a {transform} identifier or a wildcard +expression. If you do not specify one of these options, the API returns +information for all {transforms}. +end::transform-id-wildcard[] + tag::cat-v[] `v`:: (Optional, boolean) If `true`, the response includes column headings. diff --git a/docs/reference/rest-api/defs.asciidoc b/docs/reference/rest-api/defs.asciidoc index 5b8652fb442..265641db622 100644 --- a/docs/reference/rest-api/defs.asciidoc +++ b/docs/reference/rest-api/defs.asciidoc @@ -12,7 +12,6 @@ These resource definitions are used in APIs related to {ml-features} and * <> * <> * <> -* <> include::{es-repo-dir}/ml/anomaly-detection/apis/datafeedresource.asciidoc[] include::{es-repo-dir}/ml/df-analytics/apis/analysisobjects.asciidoc[] @@ -21,4 +20,3 @@ include::{es-repo-dir}/ml/anomaly-detection/apis/jobresource.asciidoc[] include::{es-repo-dir}/ml/anomaly-detection/apis/snapshotresource.asciidoc[] include::{xes-repo-dir}/rest-api/security/role-mapping-resources.asciidoc[] include::{es-repo-dir}/ml/anomaly-detection/apis/resultsresource.asciidoc[] -include::{es-repo-dir}/transform/apis/transformresource.asciidoc[] diff --git a/docs/reference/transform/apis/delete-transform.asciidoc b/docs/reference/transform/apis/delete-transform.asciidoc index d3d2c2b9b27..7257d2cbe4e 100644 --- a/docs/reference/transform/apis/delete-transform.asciidoc +++ b/docs/reference/transform/apis/delete-transform.asciidoc @@ -1,11 +1,11 @@ [role="xpack"] [testenv="basic"] [[delete-transform]] -=== Delete {transforms} API +=== Delete {transform} API [subs="attributes"] ++++ -Delete {transforms} +Delete {transform} ++++ Deletes an existing {transform}. @@ -31,7 +31,8 @@ these privileges. For more information, see <> and ==== {api-path-parms-title} ``:: - (Required, string) Identifier for the {transform}. +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] [[delete-transform-query-parms]] ==== {api-query-parms-title} diff --git a/docs/reference/transform/apis/get-transform-stats.asciidoc b/docs/reference/transform/apis/get-transform-stats.asciidoc index 1ac82e32a2a..2472b038856 100644 --- a/docs/reference/transform/apis/get-transform-stats.asciidoc +++ b/docs/reference/transform/apis/get-transform-stats.asciidoc @@ -50,35 +50,24 @@ specifying `*` as the ``, or by omitting the ==== {api-path-parms-title} ``:: - (Optional, string) Identifier for the {transform}. It can be a - {transform} identifier or a wildcard expression. If you do not - specify one of these options, the API returns information for all - {transforms}. +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id-wildcard] [[get-transform-stats-query-parms]] ==== {api-query-parms-title} `allow_no_match`:: - (Optional, boolean) Specifies what to do when the request: -+ --- -* Contains wildcard expressions and there are no {transforms} that match. -* Contains the `_all` string or no identifiers and there are no matches. -* Contains wildcard expressions and there are only partial matches. - -The default value is `true`, which returns an empty `transforms` array when -there are no matches and the subset of results when there are partial matches. -If this parameter is `false`, the request returns a `404` status code when there -are no matches or only partial matches. --- +(Optional, boolean) +include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-match-transforms1] `from`:: - (Optional, integer) Skips the specified number of {transforms}. The - default value is `0`. +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=from-transforms] `size`:: - (Optional, integer) Specifies the maximum number of {transforms} to obtain. The default value is `100`. +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=size-transforms] [[get-transform-stats-response]] ==== {api-response-body-title} diff --git a/docs/reference/transform/apis/get-transform.asciidoc b/docs/reference/transform/apis/get-transform.asciidoc index 0583b4b63af..30b552077c0 100644 --- a/docs/reference/transform/apis/get-transform.asciidoc +++ b/docs/reference/transform/apis/get-transform.asciidoc @@ -45,41 +45,38 @@ specifying `*` as the ``, or by omitting the ``. ==== {api-path-parms-title} ``:: - (Optional, string) Identifier for the {transform}. It can be a - {transform} identifier or a wildcard expression. If you do not - specify one of these options, the API returns information for all - {transforms}. +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id-wildcard] [[get-transform-query-parms]] ==== {api-query-parms-title} `allow_no_match`:: -(Optional, boolean) Specifies what to do when the request: -+ --- -* Contains wildcard expressions and there are no {transforms} that match. -* Contains the `_all` string or no identifiers and there are no matches. -* Contains wildcard expressions and there are only partial matches. - -The default value is `true`, which returns an empty `transforms` array when -there are no matches and the subset of results when there are partial matches. -If this parameter is `false`, the request returns a `404` status code when there -are no matches or only partial matches. --- +(Optional, boolean) +include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-match-transforms1] `from`:: - (Optional, integer) Skips the specified number of {transforms}. The - default value is `0`. +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=from-transforms] `size`:: - (Optional, integer) Specifies the maximum number of {transforms} to obtain. The default value is `100`. +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=size-transforms] [[get-transform-response]] ==== {api-response-body-title} -`transforms`:: - (array) An array of {transform} resources, which are sorted by the `id` value in - ascending order. See <>. +The API returns an array of {transform} resources, which are sorted by the `id` +value in ascending order. For the full list of properties, see +<>. + +`create_time`:: +(string) The time the {transform} was created. For example, `1576094542936`. +This property is informational; you cannot change its value. + +`version`:: +(string) The version of {es} that existed on the node when the {transform} was +created. [[get-transform-response-codes]] ==== {api-response-codes-title} @@ -149,7 +146,9 @@ The API returns the following results: } } }, - "description" : "Maximum priced ecommerce data by customer_id in Asia" + "description" : "Maximum priced ecommerce data by customer_id in Asia", + "version" : "7.5.0", + "create_time" : 1576094542936 } ] } diff --git a/docs/reference/transform/apis/preview-transform.asciidoc b/docs/reference/transform/apis/preview-transform.asciidoc index 29f5359f030..2e8e1647c17 100644 --- a/docs/reference/transform/apis/preview-transform.asciidoc +++ b/docs/reference/transform/apis/preview-transform.asciidoc @@ -1,11 +1,11 @@ [role="xpack"] [testenv="basic"] [[preview-transform]] -=== Preview {transforms} API +=== Preview {transform} API [subs="attributes"] ++++ -Preview {transforms} +Preview {transform} ++++ Previews a {transform}. @@ -37,24 +37,71 @@ on all the current data in the source index. [[preview-transform-request-body]] ==== {api-request-body-title} -`source`:: - (Required, object) The source configuration, which has the following - properties: - - `index`::: - (Required, string or array) The _source indices_ for the - {transform}. It can be a single index, an index pattern (for - example, `"myindex*"`), or an array of indices (for example, - `["index1", "index2"]`). - `query`::: - (Optional, object) A query clause that retrieves a subset of data from the - source index. See <>. +`description`:: +(Optional, string) Free text description of the {transform}. + +`dest`:: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest] + +`dest`.`index`::: +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] + +`dest`.`pipeline`::: +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] + +`frequency`:: +(Optional, <>) +include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] `pivot`:: - (Required, object) Defines the pivot function `group by` fields and the - aggregation to reduce the data. See <>. +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot] + +`pivot`.`aggregations` or `aggs`::: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-aggs] + +`pivot`.`group_by`::: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by] + +`pivot`.`max_page_search_size`::: +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-max-page-search-size] + +`source`:: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-transforms] + +`source`.`index`::: +(Required, string or array) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] + +`source`.`query`::: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-query-transforms] + +`sync`:: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync] +`sync`.`time`::: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time] + +`sync`.`time`.`delay`:::: +(Optional, <>) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-delay] + +`sync`.`time`.`field`:::: +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-field] + + [[preview-transform-response]] ==== {api-response-body-title} diff --git a/docs/reference/transform/apis/put-transform.asciidoc b/docs/reference/transform/apis/put-transform.asciidoc index d40b576ea4c..dfe11c96f1d 100644 --- a/docs/reference/transform/apis/put-transform.asciidoc +++ b/docs/reference/transform/apis/put-transform.asciidoc @@ -1,11 +1,11 @@ [role="xpack"] [testenv="basic"] [[put-transform]] -=== Create {transforms} API +=== Create {transform} API [subs="attributes"] ++++ -Create {transforms} +Create {transform} ++++ Instantiates a {transform}. @@ -63,9 +63,8 @@ IMPORTANT: You must use {kib} or this API to create a {transform}. ==== {api-path-parms-title} ``:: - (Required, string) Identifier for the {transform}. This identifier - can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and - underscores. It must start and end with alphanumeric characters. +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] [[put-transform-query-parms]] ==== {api-query-parms-title} @@ -82,48 +81,64 @@ IMPORTANT: You must use {kib} or this API to create a {transform}. (Optional, string) Free text description of the {transform}. `dest`:: - (Required, object) Required. The destination configuration, which has the - following properties: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest] - `index`::: - (Required, string) The _destination index_ for the {transform}. +`dest`.`index`::: +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] - `pipeline`::: - (Optional, string) The unique identifier for a <>. +`dest`.`pipeline`::: +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] `frequency`:: - (Optional, <>) The interval between checks for changes in the source - indices when the {transform} is running continuously. Also determines - the retry interval in the event of transient failures while the {transform} is - searching or indexing. The minimum value is `1s` and the maximum is `1h`. The - default value is `1m`. +(Optional, <>) +include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] `pivot`:: - (Required, object) Defines the pivot function `group by` fields and the aggregation to - reduce the data. See <>. +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot] + +`pivot`.`aggregations` or `aggs`::: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-aggs] + +`pivot`.`group_by`::: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-group-by] + +`pivot`.`max_page_search_size`::: +(Optional, integer) +include::{docdir}/rest-api/common-parms.asciidoc[tag=pivot-max-page-search-size] `source`:: - (Required, object) The source configuration, which has the following - properties: - - `index`::: - (Required, string or array) The _source indices_ for the - {transform}. It can be a single index, an index pattern (for - example, `"myindex*"`), or an array of indices (for example, - `["index1", "index2"]`). - - `query`::: - (Optional, object) A query clause that retrieves a subset of data from the - source index. See <>. +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-transforms] + +`source`.`index`::: +(Required, string or array) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] + +`source`.`query`::: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-query-transforms] `sync`:: - (Optional, object) Defines the properties required to run continuously. - `time`::: - (Required, object) Specifies that the {transform} uses a time - field to synchronize the source and destination indices. - `field`:::: - (Required, string) The date field that is used to identify new documents - in the source. +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync] + +`sync`.`time`::: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time] + +`sync`.`time`.`delay`:::: +(Optional, <>) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-delay] + +`sync`.`time`.`field`:::: +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-field] + -- TIP: In general, it’s a good idea to use a field that contains the @@ -132,9 +147,6 @@ you might need to set the `delay` such that it accounts for data transmission delays. -- - `delay`:::: - (Optional, <>) The time delay between the current time and the - latest input data time. The default value is `60s`. [[put-transform-example]] ==== {api-examples-title} diff --git a/docs/reference/transform/apis/start-transform.asciidoc b/docs/reference/transform/apis/start-transform.asciidoc index 6ce5ae7322a..6e8ce0fdd40 100644 --- a/docs/reference/transform/apis/start-transform.asciidoc +++ b/docs/reference/transform/apis/start-transform.asciidoc @@ -1,11 +1,11 @@ [role="xpack"] [testenv="basic"] [[start-transform]] -=== Start {transforms} API +=== Start {transform} API [subs="attributes"] ++++ -Start {transforms} +Start {transform} ++++ Starts one or more {transforms}. @@ -52,9 +52,8 @@ required privileges on the source and destination indices, the ==== {api-path-parms-title} ``:: - (Required, string) Identifier for the {transform}. This identifier - can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and - underscores. It must start and end with alphanumeric characters. +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] [[start-transform-example]] ==== {api-examples-title} diff --git a/docs/reference/transform/apis/stop-transform.asciidoc b/docs/reference/transform/apis/stop-transform.asciidoc index 03170d30f87..35251dd491a 100644 --- a/docs/reference/transform/apis/stop-transform.asciidoc +++ b/docs/reference/transform/apis/stop-transform.asciidoc @@ -45,32 +45,15 @@ All {transforms} can be stopped by using `_all` or `*` as the ==== {api-path-parms-title} ``:: - (Required, string) Identifier for the {transform}. This identifier - can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and - underscores. It must start and end with alphanumeric characters. - +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] [[stop-transform-query-parms]] ==== {api-query-parms-title} `allow_no_match`:: -(Optional, boolean) Specifies what to do when the request: -+ --- -* Contains wildcard expressions and there are no {transforms} that match. -* Contains the `_all` string or no identifiers and there are no matches. -* Contains wildcard expressions and there are only partial matches. - -The default value is `true`, which returns a successful acknowledgement message -when there are no matches. When there are only partial matches, the API stops -the appropriate {transforms}. For example, if the request contains -`test-id1*,test-id2*` as the identifiers and there are no {transforms} -that match `test-id2*`, the API nonetheless stops the {transforms} -that match `test-id1*`. - -If this parameter is `false`, the request returns a `404` status code when there -are no matches or only partial matches. --- +(Optional, boolean) +include::{docdir}/rest-api/common-parms.asciidoc[tag=allow-no-match-transforms2] `force`:: (Optional, boolean) Set to `true` to stop a failed {transform} or to @@ -92,7 +75,7 @@ are no matches or only partial matches. `wait_for_checkpoint`:: (Optional, boolean) If set to `true`, the transform will not completely stop - until the current checkpoint is completed. If set to `false`, the transform + until the current checkpoint is completed. If set to `false`, the {transform} stops as soon as possible. Defaults to `false`. [[stop-transform-response-codes]] diff --git a/docs/reference/transform/apis/transformresource.asciidoc b/docs/reference/transform/apis/transformresource.asciidoc deleted file mode 100644 index 190f827cd8e..00000000000 --- a/docs/reference/transform/apis/transformresource.asciidoc +++ /dev/null @@ -1,127 +0,0 @@ -[role="xpack"] -[testenv="basic"] -[[transform-resource]] -=== {transform-cap} resources - -{transform-cap} resources relate to the <>. - -For more information, see <>. - -[discrete] -[[transform-properties]] -==== {api-definitions-title} - -`description`:: - (string) A description of the {transform}. - -`dest`:: - (object) The destination for the {transform}. See - <>. - -`frequency`:: - (time units) The interval between checks for changes in the source indices - when the {transform} is running continuously. Also determines the - retry interval in the event of transient failures while the {transform} is - searching or indexing. The minimum value is `1s` and the maximum is `1h`. The - default value is `1m`. - -`id`:: - (string) A unique identifier for the {transform}. - -`pivot`:: - (object) The method for transforming the data. See - <>. - -`source`:: - (object) The source of the data for the {transform}. See - <>. - -[[transform-dest]] -==== Dest objects - -{transform-cap} resources contain `dest` objects. For example, when -you create a {transform}, you must define its destination. - -[discrete] -[[transform-dest-properties]] -===== {api-definitions-title} - -`index`:: - (string) The _destination index_ for the {transform}. - -`pipeline`:: - (string) The unique identifier for a <>. - -[[transform-source]] -==== Source objects - -{transform-cap} resources contain `source` objects. For example, when -you create a {transform}, you must define its source. - -[discrete] -[[transform-source-properties]] -===== {api-definitions-title} - -`index`:: - (string or array) The _source indices_ for the {transform}. It can - be a single index, an index pattern (for example, `"myindex*"`), or an array - of indices (for example, `["index1", "index2"]`). - -`query`:: - (object) A query clause that retrieves a subset of data from the source index. - See <>. - -[[transform-pivot]] -==== Pivot objects - -{transform-cap} resources contain `pivot` objects, which define the -pivot function `group by` fields and the aggregation to reduce the data. - -[discrete] -[[transform-pivot-properties]] -===== {api-definitions-title} - -`aggregations` or `aggs`:: - (object) Defines how to aggregate the grouped data. The following composite - aggregations are supported: -+ --- -* {ref}/search-aggregations-metrics-avg-aggregation.html[Average] -* {ref}/search-aggregations-metrics-weight-avg-aggregation.html[Weighted Average] -* {ref}/search-aggregations-metrics-cardinality-aggregation.html[Cardinality] -* {ref}/search-aggregations-metrics-geobounds-aggregation.html[Geo Bounds] -* {ref}/search-aggregations-metrics-geocentroid-aggregation.html[Geo Centroid] -* {ref}/search-aggregations-metrics-max-aggregation.html[Max] -* {ref}/search-aggregations-metrics-min-aggregation.html[Min] -* {ref}/search-aggregations-metrics-scripted-metric-aggregation.html[Scripted Metric] -* {ref}/search-aggregations-metrics-sum-aggregation.html[Sum] -* {ref}/search-aggregations-metrics-valuecount-aggregation.html[Value Count] -* {ref}/search-aggregations-pipeline-bucket-script-aggregation.html[Bucket Script] -* {ref}/search-aggregations-pipeline-bucket-selector-aggregation.html[Bucket Selector] - -IMPORTANT: {transforms-cap} support a subset of the functionality in -composite aggregations. See <>. - --- - -`group_by`:: - (object) Defines how to group the data. More than one grouping can be defined - per pivot. The following groupings are supported: -+ --- -* {ref}/search-aggregations-bucket-composite-aggregation.html#_terms[Terms] -* {ref}/search-aggregations-bucket-composite-aggregation.html#_histogram[Histogram] -* {ref}/search-aggregations-bucket-composite-aggregation.html#_date_histogram[Date Histogram] --- - -`max_page_search_size`:: - (integer) Defines the initial page size to use for the composite aggregation - for each checkpoint. If circuit breaker exceptions occur, the page size is - dynamically adjusted to a lower value. The minimum value is `10` and the - maximum is `10,000`. The default value is `500`. - -[[transform-example]] -==== {api-examples-title} - -See the -<>. diff --git a/docs/reference/transform/apis/update-transform.asciidoc b/docs/reference/transform/apis/update-transform.asciidoc index 3e98a6d0c47..c1d77d2a950 100644 --- a/docs/reference/transform/apis/update-transform.asciidoc +++ b/docs/reference/transform/apis/update-transform.asciidoc @@ -1,14 +1,14 @@ [role="xpack"] [testenv="basic"] [[update-transform]] -=== Update {transforms} API +=== Update {transform} API [subs="attributes"] ++++ -Update {transforms} +Update {transform} ++++ -Updates an existing {transform}. +Updates certain properties of a {transform}. beta[] @@ -30,29 +30,36 @@ privileges on the destination index. For more information, see [[update-transform-desc]] ==== {api-description-title} -This API updates an existing {transform}. All settings except description do not -take effect until after the {transform} starts the next checkpoint. This is -so there is consistency with the pivoted data in each checkpoint. +This API updates an existing {transform}. The list of properties that you can +update is a subset of the list that you can define when you create a {transform}. -IMPORTANT: When {es} {security-features} are enabled, your {transform} -remembers which roles the user who updated it had at the time of update and -runs with those privileges. +When the {transform} is updated, a series of validations occur to ensure its +success. You can use the `defer_validation` parameter to skip these checks. -IMPORTANT: You must use {kib} or this API to update a {transform}. - Do not update a {transform} directly via - `.transform-internal*` indices using the Elasticsearch index API. - If {es} {security-features} are enabled, do not give users any - privileges on `.transform-internal*` indices. If you used transforms - prior 7.5, also do not give users any privileges on - `.data-frame-internal*` indices. +All updated properties except description do not take effect until after the +{transform} starts the next checkpoint. This is so there is consistency with the +pivoted data in each checkpoint. + +[IMPORTANT] +==== + +* When {es} {security-features} are enabled, your {transform} remembers which +roles the user who updated it had at the time of update and runs with those +privileges. +* You must use {kib} or this API to update a {transform}. Do not update a +{transform} directly via `.transform-internal*` indices using the {es} index API. +If {es} {security-features} are enabled, do not give users any privileges on +`.transform-internal*` indices. If you used {transforms} prior 7.5, also do not +give users any privileges on `.data-frame-internal*` indices. + +==== [[update-transform-path-parms]] ==== {api-path-parms-title} ``:: - (Required, string) Identifier for the {transform}. This identifier - can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and - underscores. It must start and end with alphanumeric characters. +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=transform-id] [[update-transform-query-parms]] ==== {api-query-parms-title} @@ -69,44 +76,48 @@ IMPORTANT: You must use {kib} or this API to update a {transform}. (Optional, string) Free text description of the {transform}. `dest`:: - (Optional, object) The destination configuration, which has the - following properties: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest] - `index`::: - (Required, string) The _destination index_ for the {transform}. +`dest`.`index`::: +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-index] - `pipeline`::: - (Optional, string) The unique identifier for a <>. +`dest`.`pipeline`::: +(Optional, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=dest-pipeline] `frequency`:: - (Optional, <>) The interval between checks for changes - in the source indices when the {transform} is running continuously. - Also determines the retry interval in the event of transient failures while - the {transform} is searching or indexing. The minimum value is `1s` - and the maximum is `1h`. The default value is `1m`. +(Optional, <>) +include::{docdir}/rest-api/common-parms.asciidoc[tag=frequency] `source`:: - (Optional, object) The source configuration, which has the following - properties: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-transforms] - `index`::: - (Required, string or array) The _source indices_ for the - {transform}. It can be a single index, an index pattern (for - example, `"myindex*"`), or an array of indices (for example, - `["index1", "index2"]`). +`source`.`index`::: +(Required, string or array) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-index-transforms] - `query`::: - (Optional, object) A query clause that retrieves a subset of data from the - source index. See <>. +`source`.`query`::: +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=source-query-transforms] `sync`:: - (Optional, object) Defines the properties required to run continuously. - `time`::: - (Required, object) Specifies that the {transform} uses a time - field to synchronize the source and destination indices. - `field`:::: - (Required, string) The date field that is used to identify new documents - in the source. +(Optional, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync] + +`sync`.`time`::: +(Required, object) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time] + +`sync`.`time`.`delay`:::: +(Optional, <>) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-delay] + +`sync`.`time`.`field`:::: +(Required, string) +include::{docdir}/rest-api/common-parms.asciidoc[tag=sync-time-field] + -- TIP: In general, it’s a good idea to use a field that contains the @@ -115,9 +126,6 @@ you might need to set the `delay` such that it accounts for data transmission delays. -- - `delay`:::: - (Optional, <>) The time delay between the current - time and the latest input data time. The default value is `60s`. [[update-transform-example]] ==== {api-examples-title} @@ -196,9 +204,9 @@ When the {transform} is updated, you receive the updated configuration: "delay": "120s" } }, - "version": "7.4.0", + "version": "7.5.0", "create_time": 1518808660505 } ---- -// TESTRESPONSE[s/"version": "7.4.0"/"version": $body.version/] +// TESTRESPONSE[s/"version": "7.5.0"/"version": $body.version/] // TESTRESPONSE[s/"create_time": 1518808660505/"create_time": $body.create_time/]